Analyzing Surgical Technique in Diverse Open Surgical Videos With Multitask Machine Learning

Objective

To overcome limitations of open surgery artificial intelligence (AI) models by curating the largest collection of annotated videos and to leverage this AI-ready data set to develop a generalizable multitask AI model capable of real-time understanding of clinically significant surgical behaviors in prospectively collected real-world surgical videos.

Design, Setting, and Participants

The study team programmatically queried open surgery procedures on YouTube and manually annotated selected videos to create the AI-ready data set used to train a multitask AI model for 2 proof-of-concept studies, one generating surgical signatures that define the patterns of a given procedure and the other identifying kinematics of hand motion that correlate with surgeon skill level and experience. The Annotated Videos of Open Surgery (AVOS) data set includes 1997 videos from 23 open-surgical procedure types uploaded to YouTube from 50 countries over the last 15 years. Prospectively recorded surgical videos were collected from a single tertiary care academic medical center. Deidentified videos were recorded of surgeons performing open surgical procedures and analyzed for correlation with surgical training.

Exposures

The multitask AI model was trained on the AI-ready video data set and then retrospectively applied to the prospectively collected video data set.

Main Outcomes and Measures

Analysis of open surgical videos in near real-time, performance on AI-ready and prospectively collected videos, and quantification of surgeon skill.

Results

Using the AI-ready data set, the study team developed a multitask AI model capable of real-time understanding of surgical behaviors-the building blocks of procedural flow and surgeon skill-across space and time. Through principal component analysis, a single compound skill feature was identified, composed of a linear combination of kinematic hand attributes. This feature was a significant discriminator between experienced surgeons and surgical trainees across 101 prospectively collected surgical videos of 14 operators. For each unit increase in the compound feature value, the odds of the operator being an experienced surgeon were 3.6 times higher (95% CI, 1.67-7.62; P = .001).

Conclusions and Relevance

In this observational study, the AVOS-trained model was applied to analyze prospectively collected open surgical videos and identify kinematic descriptors of surgical skill related to efficiency of hand motion. The ability to provide AI-deduced insights into surgical structure and skill is valuable in optimizing surgical skill acquisition and ultimately improving surgical care.

Controlling the SMSI Overlayer Structure in Pt/TiO2 Catalysts Prevents Particle Evaporation

Platinum nanoparticles (NPs) supported by titania exhibit a strong metal-support interaction (SMSI)[1] that can induce overlayer formation and encapsulation of the NP´s with a thin layer of support material. This encapsulation modifies the catalyst's properties, such as increasing its chemoselectivity[2] and stabilizing it against sintering.[3] Encapsulation is typically induced during high-temperature reductive activation and can be reversed through oxidative treatment.[1] However, recent findings indicate that the overlayer can be stable in oxygen.[4,5] Using in situ transmission electron microscopy, we investigated how the overlayer changes with varying conditions. We found that exposure oxygen below 400°C caused disorder and removal of the overlayer upon subsequent hydrogen treatment. In contrast, elevating the temperature to 900°C while maintaining the oxygen atmosphere preserved the overlayer, preventing platinum evaporation when exposed to oxygen. Our findings demonstrate how different treatments can influence the stability of nanoparticles with or without titania overlayers. expanding the concept of SMSI and enabling noble metal catalysts to operate in harsh environments without evaporation associated losses during burn-off cycling.

Templated encapsulation of platinum-based catalysts promotes high-temperature stability to 1,100 °C

Stable catalysts are essential to address energy and environmental challenges, especially for applications in harsh environments (for example, high temperature, oxidizing atmosphere and steam). In such conditions, supported metal catalysts deactivate due to sintering-a process where initially small nanoparticles grow into larger ones with reduced active surface area-but strategies to stabilize them can lead to decreased performance. Here we report stable catalysts prepared through the encapsulation of platinum nanoparticles inside an alumina framework, which was formed by depositing an alumina precursor within a separately prepared porous organic framework impregnated with platinum nanoparticles. These catalysts do not sinter at 800 °C in the presence of oxygen and steam, conditions in which conventional catalysts sinter to a large extent, while showing similar reaction rates. Extending this approach to Pd-Pt bimetallic catalysts led to the small particle size being maintained at temperatures as high as 1,100 °C in air and 10% steam. This strategy can be broadly applied to other metal and metal oxides for applications where sintering is a major cause of material deactivation.

Recycling of Solvent Allows for Multiple Rounds of Reproducible Nanoparticle Synthesis

Metal nanoparticles have superior properties for a variety of applications. In many cases, the improved performance of metal nanoparticles is tightly correlated with their size and atomic composition. To date, colloidal synthesis is the most commonly used technique to produce metal nanoparticles. However, colloidal synthesis is currently a laboratory scale technique that has not been applied at larger scales. One of the greatest challenges facing large-scale colloidal synthesis of metal nanoparticles is the large volume of long-chain hydrocarbon solvents and surfactants needed for the synthesis, which can dominate the cost of nanoparticle production. In this work, we demonstrate a protocol, based on solvent distillation, which enables the reuse of colloidal nanoparticle synthesis surfactants and solvents for over 10 rounds of successive syntheses and demonstrates that pure solvents and surfactants are not necessarily needed to produce uniform nanocrystals. We show that this protocol can be applied to the production of a wide variety of mono- and bimetallic nanoparticles with reproducible sizes and compositions, which leads to reproducible performance as heterogeneous catalysts. A techno-economic assessment demonstrates the potential of this technique to greatly reduce the solvent-related costs of colloidal metal nanoparticle synthesis, which could contribute to its wider application at commercial scale.

Monolayer Support Control and Precise Colloidal Nanocrystals Demonstrate Metal-Support Interactions in Heterogeneous Catalysts

Electronic and geometric interactions between active and support phases are critical in determining the activity of heterogeneous catalysts, but metal-support interactions are challenging to study. Here, it is demonstrated how the combination of the monolayer-controlled formation using atomic layer deposition (ALD) and colloidal nanocrystal synthesis methods leads to catalysts with sub-nanometer precision of active and support phases, thus allowing for the study of the metal-support interactions in detail. The use of this approach in developing a fundamental understanding of support effects in Pd-catalyzed methane combustion is demonstrated. Uniform Pd nanocrystals are deposited onto Al₂ O₃ /SiO₂ spherical supports prepared with control over morphology and Al₂ O₃ layer thicknesses ranging from sub-monolayer to a ≈4 nm thick uniform coating. Dramatic changes in catalytic activity depending on the coverage and structure of Al₂ O₃ situated at the Pd/Al₂ O₃ interface are observed, with even a single monolayer of alumina contributing an order of magnitude increase in reaction rate. By building the Pd/Al₂ O₃ interface up layer-by-layer and using uniform Pd nanocrystals, this work demonstrates the importance of controlled and tunable materials in determining metal-support interactions and catalyst activity.

A General Approach for Monolayer Adsorption of High Weight Loadings of Uniform Nanocrystals on Oxide Supports

Monodispersed metal and semiconductor nanocrystals have attracted great attention in fundamental and applied research due to their tunable size, morphology, and well-defined chemical composition. Utilizing these nanocrystals in a controllable way is highly desirable especially when using them as building blocks for the preparation of nanostructured materials. Their deposition onto oxide materials provide them with wide applicability in many areas, including catalysis. However, so far deposition methods are limited and do not provide control to achieve high particle loadings. This study demonstrates a general approach for the deposition of hydrophobic ligand-stabilized nanocrystals on hydrophilic oxide supports without ligand-exchange. Surface functionalization of the supports with primary amine groups either using an organosilane ((3-aminopropyl)trimethoxysilane) or bonding with aminoalcohols (3-amino-1,2-propanediol) were found to significantly improve the interaction between nanocrystals and supports achieving high loadings (>10 wt. %). The bonding method with aminoalcohols guarantees the opportunity to remove the binding molecules thus allowing clean metal/oxide materials to be obtained, which is of great importance in the preparation of supported nanocrystals for heterogeneous catalysis.

Size-controlled nanocrystals reveal spatial dependence and severity of nanoparticle coalescence and Ostwald ripening in sintering phenomena

A major aim in the synthesis of nanomaterials is the development of stable materials for high-temperature applications. Although the thermal coarsening of small and active nanocrystals into less active aggregates is universal in material deactivation, the atomic mechanisms governing nanocrystal growth remain elusive. By utilizing colloidally synthesized Pd/SiO2 powder nanocomposites with controlled nanocrystal sizes and spatial arrangements, we unravel the competing contributions of particle coalescence and atomic ripening processes in nanocrystal growth. Through the study of size-controlled nanocrystals, we can uniquely identify the presence of either nanocrystal dimers or smaller nanoclusters, which indicate the relative contributions of these two processes. By controlling and tracking the nanocrystal density, we demonstrate the spatial dependence of nanocrystal coalescence and the spatial independence of Ostwald (atomic) ripening. Overall, we prove that the most significant loss of the nanocrystal surface area is due to high-temperature atomic ripening. This observation is in quantitative agreement with changes in the nanocrystal density produced by simulations of atomic exchange. Using well-defined colloidal materials, we extend our analysis to explain the unusual high-temperature stability of Au/SiO2 materials up to 800 °C.

Nanoscale Spatial Distribution of Supported Nanoparticles Controls Activity and Stability in Powder Catalysts for CO Oxidation and Photocatalytic H2 Evolution

Supported metal nanoparticles are essential components of high-performing catalysts, and their structures are intensely researched. In comparison, nanoparticle spatial distribution in powder catalysts is conventionally not quantified, and the influence of this collective property on catalyst performance remains poorly investigated. Here, we demonstrate a general colloidal self-assembly method to control uniformity of nanoparticle spatial distribution on common industrial powder supports. We quantify distributions on the nanoscale using image statistics and show that the type of nanospatial distribution determines not only the stability, but also the activity of heterogeneous catalysts. Widely investigated systems (Au-TiO2 for CO oxidation thermocatalysis and Pd-TiO2 for H2 evolution photocatalysis) were used to showcase the universal importance of nanoparticle spatial organization. Spatially and temporally resolved microkinetic modeling revealed that nonuniformly distributed Au nanoparticles suffer from local depletion of surface oxygen, and therefore lower CO oxidation activity, as compared to uniformly distributed nanoparticles. Nanoparticle spatial distribution also determines the stability of Pd-TiO2 photocatalysts, because nonuniformly distributed nanoparticles sinter while uniformly distributed nanoparticles do not. This work introduces new tools to evaluate and understand catalyst collective (ensemble) properties in powder catalysts, which thereby pave the way to more active and stable heterogeneous catalysts.

Palladium oxidation leads to methane combustion activity: Effects of particle size and alloying with platinum

Pd- and Pt-based catalysts are highly studied materials due to their widespread use in emissions control catalysis. However, claims continue to vary regarding the active phase and oxidation state of the metals. Different conclusions have likely been reached due to the heterogeneous nature of such materials containing various metal nanoparticle sizes and compositions, which may each possess unique redox features. In this work, using uniform nanocrystal catalysts, we study the effect of particle size and alloying on redox properties of Pd-based catalysts and show their contribution to methane combustion activity using operando quick extended x-ray absorption fine structure measurements. Results demonstrate that for all studied Pd sizes (3 nm-16 nm), Pd oxidation directly precedes CH₄ combustion to CO₂, suggesting Pd oxidation as a prerequisite step to methane combustion, and an oxidation pretreatment shows equal or better catalysis than a reduction pretreatment. Results are then extended to uniform alloyed Pt_xPd_1-x nanoparticles, where oxidative pretreatments are shown to enhance low-temperature combustion. In these uniform alloys, we observe a composition-dependent effect with Pt-rich alloys showing the maximum difference between oxidative and reductive pretreatments. In Pt-rich alloys, we initially observe that the presence of Pt maintains Pd in a lower-activity reduced state. However, with time on stream, PdO eventually segregates under oxidizing combustion conditions, leading to a slowly increasing activity. Overall, across particle sizes and alloy compositions, we relate increased catalytic activity to Pd oxidation, thus shedding light on previous contrasting results related to the methane combustion activity of these catalysts.

Block-Co-polymer-Assisted Synthesis of All Inorganic Highly Porous Heterostructures with Highly Accessible Thermally Stable Functional Centers

Here, we propose a simple approach for the design of highly porous multicomponent heterostructures by infiltration of block-co-polymer templates with inorganic precursors in swelling solvents followed by gas-phase sequential infiltration synthesis and thermal annealing. This approach can prepare conformal coatings, free-standing membranes, and powders consisting of uniformly sized metal or metal oxide nanoparticles (NPs) well dispersed in a porous oxide matrix. We employed this new, versatile synthetic concept to synthesize catalytically active heterostructures of uniformly dispersed ∼4.3 nm PdO nanoparticles accessible through three-dimensional pore networks of the alumina support. Importantly, such materials reveal high resistance against sintering at 800 °C, even at relatively high loadings of NPs (∼10 wt %). At the same time, such heterostructures enable high mass transport due to highly interconnected nature of the pores. The surface of synthesized nanoparticles in the porous matrix is highly accessible, which enables their good catalytic performance in methane and carbon monoxide oxidation. In addition, we demonstrate that this approach can be utilized to synthesize heterostructures consisting of different types of NPs on a highly porous support. Our results show that swelling-based infiltration provides a promising route toward the robust and scalable synthesis of multicomponent structures.

Supported Catalyst Deactivation by Decomposition into Single Atoms Is Suppressed by Increasing Metal Loading

In the high-temperature environments needed to perform catalytic processes, supported precious metal catalysts severely lose their activity over time. Even brief exposure to high temperatures can lead to significant losses in activity, which forces manufacturers to use large amounts of noble metals to ensure effective catalyst function for a required lifetime. Generally, loss of catalytic activity is attributed to nanoparticle sintering, or processes by which larger particles grow at the expense of smaller ones. Here, by independently controlling particle size and particle loading using colloidal nanocrystals, we reveal the opposite process as a novel deactivation mechanism: nanoparticles rapidly lose activity by high-temperature nanoparticle decomposition into inactive single atoms. This deactivation route is remarkably fast, leading to severe loss of activity in as little as ten minutes. Importantly, this deactivation pathway is strongly dependent on particle density and concentration of support defect sites. A quantitative statistical model explains how for certain reactions, higher particle densities can lead to more stable catalysts.

Low-Temperature Restructuring of CeO2-Supported Ru Nanoparticles Determines Selectivity in CO2 Catalytic Reduction

CO₂ reduction to higher value products is a promising way to produce fuels and key chemical building blocks while reducing CO₂ emissions. The reaction at atmospheric pressure mainly yields CH₄ via methanation and CO via the reverse water-gas shift (RWGS) reaction. Describing catalyst features that control the selectivity of these two pathways is important to determine the formation of specific products. At the same time, identification of morphological changes occurring to catalysts under reaction conditions can be crucial to tune their catalytic performance. In this contribution we investigate the dependency of selectivity for CO₂ reduction on the size of Ru nanoparticles (NPs) and on support. We find that even at rather low temperatures (210 °C), oxidative pretreatment induces redispersion of Ru NPs supported on CeO₂ and leads to a complete switch in the performance of this material from a well-known selective methanation catalyst to an active and selective RWGS catalyst. By utilizing in situ X-ray absorption spectroscopy, we demonstrate that the low-temperature redispersion process occurs via decomposition of the metal oxide phase with size-dependent kinetics, producing stable single-site RuO _x/CeO₂ species strongly bound to the CeO₂ support that are remarkably selective for CO production. These results show that reaction selectivity can be heavily dependent on catalyst structure and that structural changes of the catalyst can occur even at low temperatures and can go unseen in materials with less defined structures.

Hyperplane-Approximation-Based Method for Many-Objective Optimization Problems with Redundant Objectives

For a many-objective optimization problem with redundant objectives, we propose two novel objective reduction algorithms for linearly and, nonlinearly degenerate Pareto fronts. They are called LHA and NLHA respectively. The main idea of the proposed algorithms is to use a hyperplane with non-negative sparse coefficients to roughly approximate the structure of the PF. This approach is quite different from the previous objective reduction algorithms that are based on correlation or dominance structure. Especially in NLHA, in order to reduce the approximation error, we transform a nonlinearly degenerate Pareto front into a nearly linearly degenerate Pareto front via a power transformation. In addition, an objective reduction framework integrating a magnitude adjustment mechanism and a performance metric σ* are also proposed here. Finally, to demonstrate the performance of the proposed algorithms, comparative experiments are done with two correlation-based algorithms, LPCA and NLMVUPCA, and with two dominance-structure-based algorithms, PCSEA and greedy δ- MOSS, on three benchmark problems: DTLZ5(I,M), MAOP(I,M), and WFG3(I,M). Experimental results show that the proposed algorithms are more effective.

Systematic Identification of Promoters for Methane Oxidation Catalysts Using Size- and Composition-Controlled Pd-Based Bimetallic Nanocrystals

Promoters enhance the performance of catalytic active phases by increasing rates, stability, and/or selectivity. The process of identifying promoters is in most cases empirical and relies on testing a broad range of catalysts prepared with the random deposition of active and promoter phases, typically with no fine control over their localization. This issue is particularly relevant in supported bimetallic systems, where two metals are codeposited onto high-surface area materials. We here report the use of colloidal bimetallic nanocrystals to produce catalysts where the active and promoter phases are colocalized to a fine extent. This strategy enables a systematic approach to study the promotional effects of several transition metals on palladium catalysts for methane oxidation. In order to achieve these goals, we demonstrate a single synthetic protocol to obtain uniform palladium-based bimetallic nanocrystals (PdM, M = V, Mn, Fe, Co, Ni, Zn, Sn, and potentially extendable to other metal combinations) with a wide variety of compositions and sizes based on high-temperature thermal decomposition of readily available precursors. Once the nanocrystals are supported onto oxide materials, thermal treatments in air cause segregation of the base metal oxide phase in close proximity to the Pd phase. We demonstrate that some metals (Fe, Co, and Sn) inhibit the sintering of the active Pd metal phase, while others (Ni and Zn) increase its intrinsic activity compared to a monometallic Pd catalyst. This procedure can be generalized to systematically investigate the promotional effects of metal and metal oxide phases for a variety of active metal-promoter combinations and catalytic reactions.

Synthesis of diverse β-quaternary ketones via palladium-catalyzed asymmetric conjugate addition of arylboronic acids to cyclic enones

The development and optimization of a palladium-catalyzed asymmetric conjugate addition of arylboronic acids to cyclic enone conjugate acceptors is described. These reactions employ air-stable and readily-available reagents in an operationally simple and robust transformation that yields β-quaternary ketones in high yields and enantioselectivities. Notably, the reaction itself is highly tolerant of atmospheric oxygen and moisture and therefore does not require the use of dry or deoxygenated solvents, specially purified reagents, or an inert atmosphere. The ring size and β-substituent of the enone are highly variable, and a wide variety of β-quaternary ketones can be synthesized. More recently, the use of NH4PF6 has further expanded the substrate scope to include heteroatom-containing arylboronic acids and β-acyl enone substrates.

Predicting conserved water-mediated and polar ligand interactions in proteins using a K-nearest-neighbors genetic algorithm

Water-mediated ligand interactions are essential to biological processes, from product displacement in thymidylate synthase to DNA recognition by Trp repressor, yet the structural chemistry influencing whether bound water is displaced or participates in ligand binding is not well characterized. Consolv, employing a hybrid k-nearest-neighbors classifier/genetic algorithm, predicts bound water molecules conserved between free and ligand-bound protein structures by examining the environment of each water molecule in the free structure. Four environmental features are used: the water molecule's crystallographic temperature factor, the number of hydrogen bonds between the water molecule and protein, and the density and hydrophilicity of neighboring protein atoms. After training on 13 non-homologous proteins, Consolv predicted the conservation of active-site water molecules upon ligand binding with 75% accuracy (Matthews coefficient Cm = 0.41) for seven new proteins. Mispredictions typically involved water molecules predicted to be conserved that were displaced by a polar ligand atom, indicating that Consolv correctly assesses polar binding sites; 90% accuracy (Cm = 0.78) was achieved for predicting conserved active-site water or polar ligand atom binding. Consolv thus provides an accurate means for optimizing ligand design by identifying sites favored to be occupied by either a mediating water molecule or a polar ligand atom, as well as water molecules likely to be displaced by the ligand. Accuracy for predicting first-shell water conservation between independently determined structures was 61% (Cm=0.23). The ability to predict water-mediated and polar interactions from the free protein structure indicates the surprising extent to which the conservation or displacement of active-site bound water is independent of the ligand, and shows that the protein micro-environment of each water molecule is the dominant influence.