Distributed X-ray photon correlation spectroscopy data reduction using Hadoop MapReduce

A distributed data processing scheme based on Hadoop for synchrotron radiation experiments

With the development of synchrotron radiation sources and high-frame-rate detectors, the amount of experimental data collected at synchrotron radiation beamlines has increased exponentially. As a result, data processing for synchrotron radiation experiments has entered the era of big data. It is becoming increasingly important for beamlines to have the capability to process large-scale data in parallel to keep up with the rapid growth of data. Currently, there is no set of data processing solutions based on the big data technology framework for beamlines. Apache Hadoop is a widely used distributed system architecture for solving the problem of massive data storage and computation. This paper presents a set of distributed data processing schemes for beamlines with experimental data using Hadoop. The Hadoop Distributed File System is utilized as the distributed file storage system, and Hadoop YARN serves as the resource scheduler for the distributed computing cluster. A distributed data processing pipeline that can carry out massively parallel computation is designed and developed using Hadoop Spark. The entire data processing platform adopts a distributed microservice architecture, which makes the system easy to expand, reduces module coupling and improves reliability.

Linking scientific instruments and computation: Patterns, technologies, and experiences

Powerful detectors at modern experimental facilities routinely collect data at multiple GB/s. Online analysis methods are needed to enable the collection of only interesting subsets of such massive data streams, such as by explicitly discarding some data elements or by directing instruments to relevant areas of experimental space. Thus, methods are required for configuring and running distributed computing pipelines-what we call flows-that link instruments, computers (e.g., for analysis, simulation, artificial intelligence [AI] model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. We review common patterns associated with such flows and describe methods for instantiating these patterns. We present experiences with the application of these methods to the processing of data from five different scientific instruments, each of which engages powerful computers for data inversion,model training, or other purposes. We also discuss implications of such methods for operators and users of scientific facilities.

pyXPCSviewer: an open-source interactive tool for X-ray photon correlation spectroscopy visualization and analysis

pyXPCSviewer, a Python-based graphical user interface that is deployed at beamline 8-ID-I of the Advanced Photon Source for interactive visualization of XPCS results, is introduced. pyXPCSviewer parses rich X-ray photon correlation spectroscopy (XPCS) results into independent PyQt widgets that are both interactive and easy to maintain. pyXPCSviewer is open-source and is open to customization by the XPCS community for ingestion of diversified data structures and inclusion of novel XPCS techniques, both of which are growing demands particularly with the dawn of near-diffraction-limited synchrotron sources and their dedicated XPCS beamlines.

Advancing Chemical Separations: Unraveling the Structure and Dynamics of Phase Splitting in Liquid-Liquid Extraction

Liquid-liquid extraction (LLE), the go-to process for a variety of chemical separations, is limited by spontaneous organic phase splitting upon sufficient solute loading, called third phase formation. In this study we explore the applicability of critical phenomena theory to gain insight into this deleterious phase behavior with the goal of improving separations efficiency and minimizing waste. A series of samples representative of rare earth purification were constructed to include each of one light and one heavy lanthanide (cerium and lutetium) paired with one of two common malonamide extractants (DMDOHEMA and DMDBTDMA). The resulting postextraction organic phases are chemically complex and often form rich hierarchical structures whose statics and dynamics near the critical point were probed herein with small-angle X-ray scattering and high-speed X-ray photon correlation spectroscopy. Despite their different extraction behaviors, all samples show remarkably similar critical behavior with exponents well described by classical critical point theory consistent with the 3D Ising model, where the critical behavior is characterized by fluctuations with a single diverging length scale. This unexpected result indicates a significant reduction in relevant chemical parameters at the critical point, indicating that the underlying behavior of phase transitions in LLE rely on far fewer variables than are generally assumed. The obtained scalar order parameter is attributed to the extractant fraction of the extractant/diluent mixture, revealing that other solution components and their respective concentrations simply shift the critical temperature but do not affect the nature of the critical fluctuations. These findings point to an opportunity to drastically simplify studies of liquid-liquid phase separation and phase diagram development in general while providing insights into LLE process improvement.

Spatially-resolved nanometer-scale measurement of cartilage extracellular matrix mobility

OBJECTIVE

Tissues have complex structures, comprised of solid and fluid phases. Improved understanding of interactions between joint fluid and extracellular matrix (ECM) is required in models of cartilage mechanics. X-ray photon correlation spectroscopy (XPCS) directly measures nanometer-scale dynamics and can provide insight into biofluid-biosolid interactions in cartilage. This study applies XPCS to evaluate dynamic interactions between intact cartilage and biofluids.

DESIGN

Cartilage biopsies were collected from bovine femoral condyles. During XPCS measurements, cartilage samples were exposed to different fluids: deionized water, PBS, synovial fluid, or sonicated synovial fluid. ECM-biofluid interactions were also assessed at different length scales and different depths from the cartilage surface.

RESULTS

Using XPCS, cartilage ECM mobility was detected at length scales from 50 to 207 nm. As length scale decreased, time scale for autocorrelation decay decreased, suggesting smaller ECM components are more mobile. ECM dynamics were slowed by dehydrating the sample, demonstrating XPCS assesses matrix mobility in hydrated environments. At all length scales, the matrix was more mobile in deionized water and slowest in synovial fluid. Using the 207 nm length scale assessment, ECM dynamics in synovial fluid were fastest at the cartilage surface and progressively slowed as depth into the sample increased, demonstrating XPCS can assess spatial distribution of ECM dynamics. Finally, ECM mobility increased for degraded synovial fluid.

CONCLUSIONS

This study demonstrates the potential of XPCS to provide unique insights into nanometer-scale cartilage ECM mobility in a spatially resolved manner and illustrates the importance of biosolid-biofluid interactions in dictating ECM dynamics.

Fast nanoparticle rotational and translational diffusion in synovial fluid and hyaluronic acid solutions

Nanoparticles are under investigation as diagnostic and therapeutic agents for joint diseases, such as osteoarthritis. However, there is incomplete understanding of nanoparticle diffusion in synovial fluid, the fluid inside the joint, which consists of a mixture of the polyelectrolyte hyaluronic acid, proteins, and other components. Here, we show that rotational and translational diffusion of polymer-coated nanoparticles in quiescent synovial fluid and in hyaluronic acid solutions is well described by the Stokes-Einstein relationship, albeit with an effective medium viscosity that is much smaller than the macroscopic low shear viscosity of the fluid. This effective medium viscosity is well described by an equation for the viscosity of dilute polymer chains, where the additional viscous dissipation arises because of the presence of the polymer segments. These results shed light on the diffusive behavior of polymer-coated inorganic nanoparticles in complex and crowded biological environments, such as in the joint.

Towards molecular movies with X-ray photon correlation spectroscopy

In this perspective article we highlight research opportunities and challenges in probing structural dynamics of molecular systems using X-ray Photon Correlation Spectroscopy (XPCS). The development of new X-ray sources, such as 4th generation storage rings and X-ray free-electron lasers (XFELs), provides promising new insights into molecular motion. Employing XPCS at these sources allows to capture a very broad range of timescales and lengthscales, spanning from femtoseconds to minutes and atomic scales to the mesoscale. Here, we discuss the scientific questions that can be addressed with these novel tools for two prominent examples: the dynamics of proteins in biomolecular condensates and the dynamics of supercooled water. Finally, we provide practical tips for designing and estimating feasibility of XPCS experiments as well as on detecting and mitigating radiation damage.

20 µs-resolved high-throughput X-ray photon correlation spectroscopy on a 500k pixel detector enabled by data-management workflow

The performance of the new 52 kHz frame rate Rigaku XSPA-500k detector was characterized on beamline 8-ID-I at the Advanced Photon Source at Argonne for X-ray photon correlation spectroscopy (XPCS) applications. Due to the large data flow produced by this detector (0.2 PB of data per 24 h of continuous operation), a workflow system was deployed that uses the Advanced Photon Source data-management (DM) system and high-performance software to rapidly reduce area-detector data to multi-tau and two-time correlation functions in near real time, providing human-in-the-loop feedback to experimenters. The utility and performance of the workflow system are demonstrated via its application to a variety of small-angle XPCS measurements acquired from different detectors in different XPCS measurement modalities. The XSPA-500k detector, the software and the DM workflow system allow for the efficient acquisition and reduction of up to ∼10⁹ area-detector data frames per day, facilitating the application of XPCS to measuring samples with weak scattering and fast dynamics.

Microscopic ergodicity breaking governs the emergence and evolution of elasticity in glass-forming nanoclay suspensions

We report a study combining x-ray photon correlation spectroscopy (XPCS) with in situ rheology to investigate the microscopic dynamics and mechanical properties of aqueous suspensions of the synthetic hectorite clay Laponite, which is composed of charged, nanometer-scale, disk-shaped particles. The suspensions, with particle concentrations ranging from 3.25 to 3.75 wt %, evolve over time from a fluid to a soft glass that displays aging behavior. The XPCS measurements characterize the localization of the particles during the formation and aging of the soft-glass state. The fraction of localized particles, f_{0}, increases rapidly during the early formation stage and grows more slowly during subsequent aging, while the characteristic localization length r_{loc} steadily decreases. Despite the strongly varying rates of aging at different concentrations, both f_{0} and r_{loc} scale with the elastic shear modulus G^{'} in a manner independent of concentration. During the later aging stage, the scaling between r_{loc} and G^{'} agrees quantitatively with a prediction of naive mode coupling theory. Breakdown of agreement with the theory during the early formation stage indicates the prevalence of dynamic heterogeneity, suggesting the soft solid forms through precursors of dynamically localized clusters.

Nanoscale Critical Phenomena in a Complex Fluid Studied by X-Ray Photon Correlation Spectroscopy

The advent of high-speed x-ray photon correlation spectroscopy now allows the study of critical phenomena in fluids to much smaller length scales and over a wider range of temperatures than is possible with dynamic light scattering. We present an x-ray photon correlation spectroscopy study of critical fluctuation dynamics in a complex fluid typical of those used in liquid-liquid extraction (LLE) of ions, dodecane-DMDBTDMA with extracted aqueous Ce(NO_{3})_{3}. We observe good agreement with both static and dynamic scaling without the need for significant noncritical background corrections. Critical exponents agree with 3D Ising values, and the fluctuation dynamics are described by simple exponential relaxation. The form of the dynamic master curve deviates somewhat from the Kawasaki result, with a more abrupt transition between the critical and noncritical asymptotic behavior. The concepts of critical phenomena thus provide a quantitative framework for understanding the structure and dynamics of LLE systems and a path forward to new LLE processes.