1
|
Zhang H, Fichthorn KA. Structural classification of Ag and Cu nanocrystals with machine learning. NANOSCALE 2024; 16:17154-17164. [PMID: 39192812 DOI: 10.1039/d4nr02531h] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/29/2024]
Abstract
We use machine learning (ML) to classify the structures of mono-metallic Cu and Ag nanoparticles. Our datasets comprise a broad range of structures - both crystalline and amorphous - derived from parallel-tempering molecular dynamics simulations of nanoparticles in the 100-200 atom size range. We construct nanoparticle features using common neighbor analysis (CNA) signatures, and we utilize principal component analysis to reduce the dimensionality of the CNA feature set. To sort the nanoparticles into structural classes, we employed both K-means clustering and the Gaussian mixture model (GMM). We evaluated the performance of the clustering algorithms through the gap statistic and silhouette score, as well as by analysis of the CNA signatures. For Ag, we found five structural classes, with 14 detailed sub-classes, while for Cu, we found two broad classes (crystalline and amorphous), with the same five classes as for Ag, and 15 detailed sub-classes. Our results demonstrate that these ML methods are effective in identifying and categorizing nanoparticle structures to different levels of complexity, enabling us to classify nanoparticles into distinct and physically relevant structural classes with high accuracy. This capability is important for understanding nanoparticle properties and potential applications.
Collapse
Affiliation(s)
- Huaizhong Zhang
- Department of Chemical Engineering, The Pennsylvania State University, University Park, PA, 16802, USA
| | - Kristen A Fichthorn
- Department of Chemical Engineering and Department of Physics, The Pennsylvania State University, University Park, PA 16802, USA.
| |
Collapse
|
2
|
Wong J, Onizhuk M, Nagura J, Thind AS, Bindra JK, Wicker C, Grant GD, Zhang Y, Niklas J, Poluektov OG, Klie RF, Zhang J, Galli G, Heremans FJ, Awschalom DD, Alivisatos AP. Coherent Erbium Spin Defects in Colloidal Nanocrystal Hosts. ACS NANO 2024; 18:19110-19123. [PMID: 38980975 DOI: 10.1021/acsnano.4c04083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2024]
Abstract
We demonstrate nearly a microsecond of spin coherence in Er3+ ions doped in cerium dioxide nanocrystal hosts, despite a large gyromagnetic ratio and nanometric proximity of the spin defect to the nanocrystal surface. The long spin coherence is enabled by reducing the dopant density below the instantaneous diffusion limit in a nuclear spin-free host material, reaching the limit of a single erbium spin defect per nanocrystal. We observe a large Orbach energy in a highly symmetric cubic site, further protecting the coherence in a qubit that would otherwise rapidly decohere. Spatially correlated electron spectroscopy measurements reveal the presence of Ce3+ at the nanocrystal surface, which likely acts as extraneous paramagnetic spin noise. Even with these factors, defect-embedded nanocrystal hosts show tremendous promise for quantum sensing and quantum communication applications, with multiple avenues, including core-shell fabrication, redox tuning of oxygen vacancies, and organic surfactant modification, available to further enhance their spin coherence and functionality in the future.
Collapse
Affiliation(s)
- Joeson Wong
- James Franck Institute, University of Chicago, Chicago, Illinois 60637, United States
- Department of Chemistry, University of Chicago, Chicago, Illinois 60637, United States
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Mykyta Onizhuk
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Jonah Nagura
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Arashdeep Singh Thind
- Department of Physics, University of Illinois Chicago, Chicago, Illinois 60607, United States
| | - Jasleen K Bindra
- Chemical Sciences and Engineering Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Christina Wicker
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Gregory D Grant
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Yuxuan Zhang
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Jens Niklas
- Chemical Sciences and Engineering Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Oleg G Poluektov
- Chemical Sciences and Engineering Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Robert F Klie
- Department of Physics, University of Illinois Chicago, Chicago, Illinois 60607, United States
| | - Jiefei Zhang
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
- Center for Molecular Engineering, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Giulia Galli
- Department of Chemistry, University of Chicago, Chicago, Illinois 60637, United States
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
- Center for Molecular Engineering, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - F Joseph Heremans
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
- Center for Molecular Engineering, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - David D Awschalom
- Materials Science Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
- Center for Molecular Engineering, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - A Paul Alivisatos
- James Franck Institute, University of Chicago, Chicago, Illinois 60637, United States
- Department of Chemistry, University of Chicago, Chicago, Illinois 60637, United States
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| |
Collapse
|
3
|
Zhou L, Wen H, Kuschnerus IC, Chang SLY. Efficientand Robust Automated Segmentation of Nanoparticles and Aggregates from Transmission Electron Microscopy Images with Highly Complex Backgrounds. NANOMATERIALS (BASEL, SWITZERLAND) 2024; 14:1169. [PMID: 39057846 PMCID: PMC11279516 DOI: 10.3390/nano14141169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2024] [Revised: 06/26/2024] [Accepted: 07/02/2024] [Indexed: 07/28/2024]
Abstract
Morphologies of nanoparticles and aggregates play an important role in their properties for a range of applications. In particular, significant synthesis efforts have been directed toward controlling nanoparticle morphology and aggregation behavior in biomedical applications, as their size and shape have a significant impact on cellular uptake. Among several techniques for morphological characterization, transmission electron microscopy (TEM) can provide direct and accurate characterization of nanoparticle/aggregate morphology details. Nevertheless, manually analyzing a large number of TEM images is still a laborious process. Hence, there has been a surge of interest in employing machine learning methods to analyze nanoparticle size and shape. In order to achieve accurate nanoparticle analysis using machine learning methods, reliable and automated nanoparticle segmentation from TEM images is critical, especially when the nanoparticle image contrast is weak and the background is complex. These challenges are particularly pertinent in biomedical applications. In this work, we demonstrate an efficient, robust, and automated nanoparticle image segmentation method suitable for subsequent machine learning analysis. Our method is robust for noisy, low-electron-dose cryo-TEM images and for TEM cell images with complex, strong-contrast background features. Moreover, our method does not require any a priori training datasets, making it efficient and general. The ability to automatically, reliably, and efficiently segment nanoparticle/aggregate images is critical for advancing precise particle/aggregate control in biomedical applications.
Collapse
Affiliation(s)
- Lishi Zhou
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia; (L.Z.); (I.C.K.)
| | - Haotian Wen
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia; (L.Z.); (I.C.K.)
| | - Inga C. Kuschnerus
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia; (L.Z.); (I.C.K.)
- Electron Microscope Unit, Mark Wrainwright Analytical Centre, University of New South Wales, Sydney, NSW 2052, Australia
| | - Shery L. Y. Chang
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia; (L.Z.); (I.C.K.)
- Electron Microscope Unit, Mark Wrainwright Analytical Centre, University of New South Wales, Sydney, NSW 2052, Australia
| |
Collapse
|
4
|
Reyes-Vera E, Valencia-Arias A, García-Pineda V, Aurora-Vigo EF, Alvarez Vásquez H, Sánchez G. Machine Learning Applications in Optical Fiber Sensing: A Research Agenda. SENSORS (BASEL, SWITZERLAND) 2024; 24:2200. [PMID: 38610411 PMCID: PMC11014317 DOI: 10.3390/s24072200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Revised: 02/26/2024] [Accepted: 02/28/2024] [Indexed: 04/14/2024]
Abstract
The constant monitoring and control of various health, infrastructure, and natural factors have led to the design and development of technological devices in a wide range of fields. This has resulted in the creation of different types of sensors that can be used to monitor and control different environments, such as fire, water, temperature, and movement, among others. These sensors detect anomalies in the input data to the system, allowing alerts to be generated for early risk detection. The advancement of artificial intelligence has led to improved sensor systems and networks, resulting in devices with better performance and more precise results by incorporating various features. The aim of this work is to conduct a bibliometric analysis using the PRISMA 2020 set to identify research trends in the development of machine learning applications in fiber optic sensors. This methodology facilitates the analysis of a dataset comprised of documents obtained from Scopus and Web of Science databases. It enables the evaluation of both the quantity and quality of publications in the study area based on specific criteria, such as trends, key concepts, and advances in concepts over time. The study found that deep learning techniques and fiber Bragg gratings have been extensively researched in infrastructure, with a focus on using fiber optic sensors for structural health monitoring in future research. One of the main limitations is the lack of research on the use of novel materials, such as graphite, for designing fiber optic sensors. One of the main limitations is the lack of research on the use of novel materials, such as graphite, for designing fiber optic sensors. This presents an opportunity for future studies.
Collapse
Affiliation(s)
- Erick Reyes-Vera
- Departamento de Electrónica y Telecomunicaciones, Instituto Tecnológico Metropolitano, Medellín 050013, Colombia;
| | | | - Vanessa García-Pineda
- Departamento de Electrónica y Telecomunicaciones, Instituto Tecnológico Metropolitano, Medellín 050013, Colombia;
| | - Edward Florencio Aurora-Vigo
- Escuela Profesional de Ingeniería Agroindustrial y Comercio Exterior, Universidad Señor de Sipán, Chiclayo 14001, Peru;
| | - Halyn Alvarez Vásquez
- Facultad de Ingeniería, Arquitectura y Urbanismo, Universidad Señor de Sipán, Chiclayo 14001, Peru;
| | - Gustavo Sánchez
- Instituto de Investigación y Estudios de la Mujer, Universidad Ricardo Palma, Lima 15074, Peru;
| |
Collapse
|
5
|
Chee SW, Lunkenbein T, Schlögl R, Roldán Cuenya B. Operando Electron Microscopy of Catalysts: The Missing Cornerstone in Heterogeneous Catalysis Research? Chem Rev 2023; 123:13374-13418. [PMID: 37967448 PMCID: PMC10722467 DOI: 10.1021/acs.chemrev.3c00352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 10/14/2023] [Accepted: 10/20/2023] [Indexed: 11/17/2023]
Abstract
Heterogeneous catalysis in thermal gas-phase and electrochemical liquid-phase chemical conversion plays an important role in our modern energy landscape. However, many of the structural features that drive efficient chemical energy conversion are still unknown. These features are, in general, highly distinct on the local scale and lack translational symmetry, and thus, they are difficult to capture without the required spatial and temporal resolution. Correlating these structures to their function will, conversely, allow us to disentangle irrelevant and relevant features, explore the entanglement of different local structures, and provide us with the necessary understanding to tailor novel catalyst systems with improved productivity. This critical review provides a summary of the still immature field of operando electron microscopy for thermal gas-phase and electrochemical liquid-phase reactions. It focuses on the complexity of investigating catalytic reactions and catalysts, progress in the field, and analysis. The forthcoming advances are discussed in view of correlative techniques, artificial intelligence in analysis, and novel reactor designs.
Collapse
Affiliation(s)
- See Wee Chee
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Thomas Lunkenbein
- Department
of Inorganic Chemistry, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Robert Schlögl
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| | - Beatriz Roldán Cuenya
- Department
of Interface Science, Fritz-Haber Institute
of the Max-Planck Society, 14195 Berlin, Germany
| |
Collapse
|
6
|
Yan X, Yue T, Winkler DA, Yin Y, Zhu H, Jiang G, Yan B. Converting Nanotoxicity Data to Information Using Artificial Intelligence and Simulation. Chem Rev 2023. [PMID: 37262026 DOI: 10.1021/acs.chemrev.3c00070] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Decades of nanotoxicology research have generated extensive and diverse data sets. However, data is not equal to information. The question is how to extract critical information buried in vast data streams. Here we show that artificial intelligence (AI) and molecular simulation play key roles in transforming nanotoxicity data into critical information, i.e., constructing the quantitative nanostructure (physicochemical properties)-toxicity relationships, and elucidating the toxicity-related molecular mechanisms. For AI and molecular simulation to realize their full impacts in this mission, several obstacles must be overcome. These include the paucity of high-quality nanomaterials (NMs) and standardized nanotoxicity data, the lack of model-friendly databases, the scarcity of specific and universal nanodescriptors, and the inability to simulate NMs at realistic spatial and temporal scales. This review provides a comprehensive and representative, but not exhaustive, summary of the current capability gaps and tools required to fill these formidable gaps. Specifically, we discuss the applications of AI and molecular simulation, which can address the large-scale data challenge for nanotoxicology research. The need for model-friendly nanotoxicity databases, powerful nanodescriptors, new modeling approaches, molecular mechanism analysis, and design of the next-generation NMs are also critically discussed. Finally, we provide a perspective on future trends and challenges.
Collapse
Affiliation(s)
- Xiliang Yan
- Institute of Environmental Research at the Greater Bay Area, Key Laboratory for Water Quality and Conservation of the Pearl River Delta, Ministry of Education, Guangzhou University, Guangzhou 510006, China
| | - Tongtao Yue
- Key Laboratory of Marine Environment and Ecology, Ministry of Education, Institute of Coastal Environmental Pollution Control, Ocean University of China, Qingdao 266100, China
| | - David A Winkler
- Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052, Australia
- School of Pharmacy, University of Nottingham, Nottingham NG7 2QL, U.K
- Department of Biochemistry and Chemistry, La Trobe Institute for Molecular Science, La Trobe University, Melbourne, Victoria 3086, Australia
| | - Yongguang Yin
- State Key Laboratory of Environmental Chemistry and Ecotoxicology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, Beijing 100085, China
| | - Hao Zhu
- Department of Chemistry and Biochemistry, Rowan University, Glassboro, New Jersey 08028, United States
| | - Guibin Jiang
- State Key Laboratory of Environmental Chemistry and Ecotoxicology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, Beijing 100085, China
| | - Bing Yan
- Institute of Environmental Research at the Greater Bay Area, Key Laboratory for Water Quality and Conservation of the Pearl River Delta, Ministry of Education, Guangzhou University, Guangzhou 510006, China
| |
Collapse
|
7
|
Gumbiowski N, Loza K, Heggen M, Epple M. Automated analysis of transmission electron micrographs of metallic nanoparticles by machine learning. NANOSCALE ADVANCES 2023; 5:2318-2326. [PMID: 37056630 PMCID: PMC10089082 DOI: 10.1039/d2na00781a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
Metallic nanoparticles were analysed with respect to size and shape by a machine learning approach. This involved a separation of particles from the background (segmentation), a separation of overlapping particles, and the identification of individual particles. An algorithm to separate overlapping particles, based on ultimate erosion of convex shapes (UECS), was implemented. Finally, particle properties like size, circularity, equivalent diameter, and Feret diameter were computed for each particle of the whole particle population. Thus, particle size distributions can be easily created based on the various parameters. However, strongly overlapping particles are difficult and sometimes impossible to separate because of an a priori unknown shape of a particle that is partially lying in the shadow of another particle. The program is able to extract information from a sequence of images of the same sample, thereby increasing the number of analysed nanoparticles to several thousands. The machine learning approach is well-suited to identify particles at only limited particle-to-background contrast as is demonstrated for ultrasmall gold nanoparticles (2 nm).
Collapse
Affiliation(s)
- Nina Gumbiowski
- Inorganic Chemistry, Center for Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen 45117 Essen Germany
| | - Kateryna Loza
- Inorganic Chemistry, Center for Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen 45117 Essen Germany
| | - Marc Heggen
- Ernst-Ruska Centre for Microscopy and Spectroscopy with Electrons, Forschungszentrum Jülich GmbH 52428 Jülich Germany
| | - Matthias Epple
- Inorganic Chemistry, Center for Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen 45117 Essen Germany
| |
Collapse
|
8
|
Masson JF, Biggins JS, Ringe E. Machine learning for nanoplasmonics. NATURE NANOTECHNOLOGY 2023; 18:111-123. [PMID: 36702956 DOI: 10.1038/s41565-022-01284-0] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Accepted: 10/27/2022] [Indexed: 06/18/2023]
Abstract
Plasmonic nanomaterials have outstanding optoelectronic properties potentially enabling the next generation of catalysts, sensors, lasers and photothermal devices. Owing to optical and electron techniques, modern nanoplasmonics research generates large datasets characterizing features across length scales. Furthermore, optimizing syntheses leading to specific nanostructures requires time-consuming multiparametric approaches. These complex datasets and trial-and-error practices make nanoplasmonics research ripe for the application of machine learning (ML) and advanced data processing methods. ML algorithms capture relationships between synthesis, structure and performance in a way that far exceeds conventional simulation and theory approaches, enabling effective performance optimization. For example, neural networks can tailor the nanostructure morphology to target desired properties, identify synthetic conditions and extract quantitative information from complex data. Here we discuss the nascent field of ML for nanoplasmonics, describe the opportunities and limitations of ML in nanoplasmonic research, and conclude that ML is potentially transformative, especially if the community curates and shares its big data.
Collapse
Affiliation(s)
- Jean-Francois Masson
- Département de chimie, Quebec Center for Advanced Materials, Regroupement québécois sur les matériaux de pointe, and Centre interdisciplinaire de recherche sur le cerveau et l'apprentissage, Université de Montréal, Montréal, Quebec, Canada.
| | - John S Biggins
- Engineering Department, University of Cambridge, Cambridge, UK.
| | - Emilie Ringe
- Department of Material Science and Metallurgy, University of Cambridge, Cambridge, UK.
- Department of Earth Science, University of Cambridge, Cambridge, UK.
| |
Collapse
|
9
|
Joshi PB. Navigating with chemometrics and machine learning in chemistry. Artif Intell Rev 2023; 56:1-26. [PMID: 36714038 PMCID: PMC9870782 DOI: 10.1007/s10462-023-10391-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/09/2023] [Indexed: 01/25/2023]
Abstract
Chemometrics and machine learning are artificial intelligence-based methods stirring a transformative change in chemistry. Organic synthesis, drug discovery and analytical techniques are incorporating machine learning techniques at an accelerated pace. However, machine-assisted chemistry faces challenges while solving critical problems in chemistry due to complex relationships in data sets. Even with increasing publishing volumes on machine learning, its application in areas of chemistry is not a straightforward endeavour. A particular concern in applying machine learning in chemistry is data availability and reproducibility. The present review article discusses the various chemometric methods, expert systems, and machine learning techniques developed for solving problems of organic synthesis and drug discovery with selected examples. Further, a concise discussion on chemometrics and ML deployed in analytical techniques such as, spectroscopy, microscopy and chromatography are presented. Finally, the review reflects the challenges, opportunities and future perspectives on machine learning and automation in chemistry. The review concludes by pondering on some tough questions on applying machine learning and their possibility of navigation in the different terrains of chemistry.
Collapse
Affiliation(s)
- Payal B. Joshi
- Operations and Method Development, Shefali Research Laboratories, Ambernath (East), Thane, Maharashtra 421501 India
| |
Collapse
|
10
|
Burks GR, Yao L, Kalutantirige FC, Gray KJ, Bello E, Rajagopalan S, Bialik SB, Barrick JE, Alleyne M, Chen Q, Schroeder CM. Electron Tomography and Machine Learning for Understanding the Highly Ordered Structure of Leafhopper Brochosomes. Biomacromolecules 2023; 24:190-200. [PMID: 36516996 DOI: 10.1021/acs.biomac.2c01035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
Insects known as leafhoppers (Hemiptera: Cicadellidae) produce hierarchically structured nanoparticles known as brochosomes that are exuded and applied to the insect cuticle, thereby providing camouflage and anti-wetting properties to aid insect survival. Although the physical properties of brochosomes are thought to depend on the leafhopper species, the structure-function relationships governing brochosome behavior are not fully understood. Brochosomes have complex hierarchical structures and morphological heterogeneity across species, due to which a multimodal characterization approach is required to effectively elucidate their nanoscale structure and properties. In this work, we study the structural and mechanical properties of brochosomes using a combination of atomic force microscopy (AFM), electron microscopy (EM), electron tomography, and machine learning (ML)-based quantification of large and complex scanning electron microscopy (SEM) image data sets. This suite of techniques allows for the characterization of internal and external brochosome structures, and ML-based image analysis methods of large data sets reveal correlations in the structure across several leafhopper species. Our results show that brochosomes are relatively rigid hollow spheres with characteristic dimensions and morphologies that depend on leafhopper species. Nanomechanical mapping AFM is used to determine a characteristic compression modulus for brochosomes on the order of 1-3 GPa, which is consistent with crystalline proteins. Overall, this work provides an improved understanding of the structural and mechanical properties of leafhopper brochosomes using a new set of ML-based image classification tools that can be broadly applied to nanostructured biological materials.
Collapse
Affiliation(s)
- Gabriel R Burks
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Lehan Yao
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Falon C Kalutantirige
- Department of Chemistry, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Kyle J Gray
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Chemical and Biomolecular Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Elizabeth Bello
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Entomology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Shreyas Rajagopalan
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Entomology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Sarah B Bialik
- Department of Molecular Biosciences, The University of Texas at Austin, Austin, Texas 78712, United States
| | - Jeffrey E Barrick
- Department of Molecular Biosciences, The University of Texas at Austin, Austin, Texas 78712, United States
| | - Marianne Alleyne
- Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Entomology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Qian Chen
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Chemistry, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Chemical and Biomolecular Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| | - Charles M Schroeder
- Department of Materials Science and Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States.,Department of Chemical and Biomolecular Engineering, University of Illinois at Urbana-Champaign, Champaign, Illinois 61801, United States
| |
Collapse
|
11
|
Son J, Kim GH, Lee Y, Lee C, Cha S, Nam JM. Toward Quantitative Surface-Enhanced Raman Scattering with Plasmonic Nanoparticles: Multiscale View on Heterogeneities in Particle Morphology, Surface Modification, Interface, and Analytical Protocols. J Am Chem Soc 2022; 144:22337-22351. [PMID: 36473154 DOI: 10.1021/jacs.2c05950] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Surface-enhanced Raman scattering (SERS) provides significantly enhanced Raman scattering signals from molecules adsorbed on plasmonic nanostructures, as well as the molecules' vibrational fingerprints. Plasmonic nanoparticle systems are particularly powerful for SERS substrates as they provide a wide range of structural features and plasmonic couplings to boost the enhancement, often up to >108-1010. Nevertheless, nanoparticle-based SERS is not widely utilized as a means for reliable quantitative measurement of molecules largely due to limited controllability, uniformity, and scalability of plasmonic nanoparticles, poor molecular modification chemistry, and a lack of widely used analytical protocols for SERS. Furthermore, multiscale issues with plasmonic nanoparticle systems that range from atomic and molecular scales to assembled nanostructure scale are difficult to simultaneously control, analyze, and address. In this perspective, we introduce and discuss the design principles and key issues in preparing SERS nanoparticle substrates and the recent studies on the uniform and controllable synthesis and newly emerging machine learning-based analysis of plasmonic nanoparticle systems for quantitative SERS. Specifically, the multiscale point of view with plasmonic nanoparticle systems toward quantitative SERS is provided throughout this perspective. Furthermore, issues with correctly estimating and comparing SERS enhancement factors are discussed, and newly emerging statistical and artificial intelligence approaches for analyzing complex SERS systems are introduced and scrutinized to address challenges that cannot be fully resolved through synthetic improvements.
Collapse
Affiliation(s)
- Jiwoong Son
- Department of Chemistry, Seoul National University, Seoul 08826, South Korea
| | - Gyeong-Hwan Kim
- The Research Institute of Basic Sciences, Seoul National University, Seoul 08826, South Korea
| | - Yeonhee Lee
- Department of Chemistry, Seoul National University, Seoul 08826, South Korea
| | - Chungyeon Lee
- Department of Chemistry, Seoul National University, Seoul 08826, South Korea
| | - Seungsang Cha
- Department of Chemistry, Seoul National University, Seoul 08826, South Korea
| | - Jwa-Min Nam
- Department of Chemistry, Seoul National University, Seoul 08826, South Korea
| |
Collapse
|
12
|
Botifoll M, Pinto-Huguet I, Arbiol J. Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook. NANOSCALE HORIZONS 2022; 7:1427-1477. [PMID: 36239693 DOI: 10.1039/d2nh00377e] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.
Collapse
Affiliation(s)
- Marc Botifoll
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
| | - Ivan Pinto-Huguet
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
| | - Jordi Arbiol
- Catalan Institute of Nanoscience and Nanotechnology (ICN2), CSIC and BIST, Campus UAB, Bellaterra, 08193 Barcelona, Catalonia, Spain.
- ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Catalonia, Spain
| |
Collapse
|
13
|
Yao L, An H, Zhou S, Kim A, Luijten E, Chen Q. Seeking regularity from irregularity: unveiling the synthesis-nanomorphology relationships of heterogeneous nanomaterials using unsupervised machine learning. NANOSCALE 2022; 14:16479-16489. [PMID: 36285804 DOI: 10.1039/d2nr03712b] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Nanoscale morphology of functional materials determines their chemical and physical properties. However, despite increasing use of transmission electron microscopy (TEM) to directly image nanomorphology, it remains challenging to quantify the information embedded in TEM data sets, and to use nanomorphology to link synthesis and processing conditions to properties. We develop an automated, descriptor-free analysis workflow for TEM data that utilizes convolutional neural networks and unsupervised learning to quantify and classify nanomorphology, and thereby reveal synthesis-nanomorphology relationships in three different systems. While TEM records nanomorphology readily in two-dimensional (2D) images or three-dimensional (3D) tomograms, we advance the analysis of these images by identifying and applying a universal shape fingerprint function to characterize nanomorphology. After dimensionality reduction through principal component analysis, this function then serves as the input for morphology grouping through unsupervised learning. We demonstrate the wide applicability of our workflow to both 2D and 3D TEM data sets, and to both inorganic and organic nanomaterials, including tetrahedral gold nanoparticles mixed with irregularly shaped impurities, hybrid polymer-patched gold nanoprisms, and polyamide membranes with irregular and heterogeneous 3D crumple structures. In each of these systems, unsupervised nanomorphology grouping identifies both the diversity and the similarity of the nanomaterial across different synthesis conditions, revealing how synthetic parameters guide nanomorphology development. Our work opens possibilities for enhancing synthesis of nanomaterials through artificial intelligence and for understanding and controlling complex nanomorphology, both for 2D systems and in the far less explored case of 3D structures, such as those with embedded voids or hidden interfaces.
Collapse
Affiliation(s)
- Lehan Yao
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL 61801, USA.
| | - Hyosung An
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL 61801, USA.
- Department of Petrochemical Materials Engineering, Chonnam National University, Yeosu, 59631, Korea
| | - Shan Zhou
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL 61801, USA.
| | - Ahyoung Kim
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL 61801, USA.
| | - Erik Luijten
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL 60208, USA
- Department of Engineering Sciences and Applied Mathematics, Northwestern University, Evanston, IL 60208, USA
- Department of Chemistry, Northwestern University, Evanston, IL 60208, USA
- Department of Physics and Astronomy, Northwestern University, Evanston, IL 60208, USA
| | - Qian Chen
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL 61801, USA.
- Department of Chemistry, University of Illinois, Urbana, IL 61801, USA
- Materials Research Laboratory, University of Illinois, Urbana, IL 61801, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana, IL 61801, USA
| |
Collapse
|
14
|
Williamson EM, Ghrist AM, Karadaghi LR, Smock SR, Barim G, Brutchey RL. Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis. NANOSCALE 2022; 14:15327-15339. [PMID: 36214256 DOI: 10.1039/d2nr04292d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42 650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. A classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr3 nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.
Collapse
Affiliation(s)
- Emily M Williamson
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| | - Aaron M Ghrist
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| | - Lanja R Karadaghi
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| | - Sara R Smock
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| | - Gözde Barim
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| | - Richard L Brutchey
- Department of Chemistry, University of Southern California, Los Angeles, CA 90089, USA.
| |
Collapse
|
15
|
Sainju R, Chen WY, Schaefer S, Yang Q, Ding C, Li M, Zhu Y. DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time. Sci Rep 2022; 12:15705. [PMID: 36127375 PMCID: PMC9489724 DOI: 10.1038/s41598-022-19697-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 09/02/2022] [Indexed: 11/10/2022] Open
Abstract
In-situ irradiation transmission electron microscopy (TEM) offers unique insights into the millisecond-timescale post-cascade process, such as the lifetime and thermal stability of defect clusters, vital to the mechanistic understanding of irradiation damage in nuclear materials. Converting in-situ irradiation TEM video data into meaningful information on defect cluster dynamic properties (e.g., lifetime) has become the major technical bottleneck. Here, we present a solution called the DefectTrack, the first dedicated deep learning-based one-shot multi-object tracking (MOT) model capable of tracking cascade-induced defect clusters in in-situ TEM videos in real-time. DefectTrack has achieved a Multi-Object Tracking Accuracy (MOTA) of 66.43% and a Mostly Tracked (MT) of 67.81% on the test set, which are comparable to state-of-the-art MOT algorithms. We discuss the MOT framework, model selection, training, and evaluation strategies for in-situ TEM applications. Further, we compare the DefectTrack with four human experts in quantifying defect cluster lifetime distributions using statistical tests and discuss the relationship between the material science domain metrics and MOT metrics. Our statistical evaluations on the defect lifetime distribution suggest that the DefectTrack outperforms human experts in accuracy and speed.
Collapse
Affiliation(s)
- Rajat Sainju
- Department of Materials Science and Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Wei-Ying Chen
- Nuclear Science and Engineering Division, Argonne National Laboratory, Lemont, IL, 60439, USA
| | - Samuel Schaefer
- Department of Materials Science and Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Qian Yang
- Department of Computer Science and Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Caiwen Ding
- Department of Computer Science and Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Meimei Li
- Nuclear Science and Engineering Division, Argonne National Laboratory, Lemont, IL, 60439, USA
| | - Yuanyuan Zhu
- Department of Materials Science and Engineering, University of Connecticut, Storrs, CT, 06269, USA.
| |
Collapse
|
16
|
Ajay P, Nagaraj B, Kumar RA, Huang R, Ananthi P. Unsupervised Hyperspectral Microscopic Image Segmentation Using Deep Embedded Clustering Algorithm. SCANNING 2022; 2022:1200860. [PMID: 35800209 PMCID: PMC9192273 DOI: 10.1155/2022/1200860] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Accepted: 05/23/2022] [Indexed: 06/15/2023]
Abstract
Hyperspectral microscopy in biology and minerals, unsupervised deep learning neural network denoising SRS photos: hyperspectral resolution enhancement and denoising one hyperspectral picture is enough to teach unsupervised method. An intuitive chemical species map for a lithium ore sample is produced using k-means clustering. Many researchers are now interested in biosignals. Uncertainty limits the algorithms' capacity to evaluate these signals for further information. Even while AI systems can answer puzzles, they remain limited. Deep learning is used when machine learning is inefficient. Supervised learning needs a lot of data. Deep learning is vital in modern AI. Supervised learning requires a large labeled dataset. The selection of parameters prevents over- or underfitting. Unsupervised learning is used to overcome the challenges outlined above (performed by the clustering algorithm). To accomplish this, two processing processes were used: (1) utilizing nonlinear deep learning networks to turn data into a latent feature space (Z). The Kullback-Leibler divergence is used to test the objective function convergence. This article explores a novel research on hyperspectral microscopic picture using deep learning and effective unsupervised learning.
Collapse
Affiliation(s)
- P. Ajay
- Faculty of Information and Communication Engineering, Anna University, Chennai, India
| | - B. Nagaraj
- Department of ECE, Rathinam Technical Campus, India
| | - R. Arun Kumar
- Rathinam Technical Campus, Department of Electronics and Communication Engineering, India
| | | | - P. Ananthi
- Department of Artificial Intelligence and Data Science, Rathinam Technical Campus, India
| |
Collapse
|
17
|
Text-mined dataset of gold nanoparticle synthesis procedures, morphologies, and size entities. Sci Data 2022; 9:234. [PMID: 35618761 PMCID: PMC9135747 DOI: 10.1038/s41597-022-01321-6] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 04/08/2022] [Indexed: 12/13/2022] Open
Abstract
Gold nanoparticles are highly desired for a range of technological applications due to their tunable properties, which are dictated by the size and shape of the constituent particles. Many heuristic methods for controlling the morphological characteristics of gold nanoparticles are well known. However, the underlying mechanisms controlling their size and shape remain poorly understood, partly due to the immense range of possible combinations of synthesis parameters. Data-driven methods can offer insight to help guide understanding of these underlying mechanisms, so long as sufficient synthesis data are available. To facilitate data mining in this direction, we have constructed and made publicly available a dataset of codified gold nanoparticle synthesis protocols and outcomes extracted directly from the nanoparticle materials science literature using natural language processing and text-mining techniques. This dataset contains 5,154 data records, each representing a single gold nanoparticle synthesis article, filtered from a database of 4,973,165 publications. Each record contains codified synthesis protocols and extracted morphological information from a total of 7,608 experimental and 12,519 characterization paragraphs. Measurement(s) | gold nanoparticle morphology • gold nanoparticle size • gold nanoparticle synthesis data | Technology Type(s) | natural language processing |
Collapse
|
18
|
Zhang J, Yang X, Ji T, Wen C, Ye Z, Liu X, Liang L, Liu G, Xu X. Digestion and absorption properties of Lycium barbarum polysaccharides stabilized selenium nanoparticles. Food Chem 2022; 373:131637. [PMID: 34823931 DOI: 10.1016/j.foodchem.2021.131637] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2021] [Revised: 10/26/2021] [Accepted: 11/09/2021] [Indexed: 11/27/2022]
Abstract
In the present study, the digestion and absorption properties of Lycium barbarum polysaccharides stabilized selenium nanoparticles (LBP-SeNPs) were investigated. The results showed that selenium nanoparticles (SeNPs) exhibited a higher selenium release rate than LBP-SeNPs (p<0.05) after being digested in the stages of oral cavity, stomach and intestine. During the digestion process, the particle size of the LBP-SeNPs and SeNPs were both significantly increased, but the particle size of LBP-SeNPs was significantly smaller than that of SeNPs. The results of TEM further indicated that LBP-SeNPs can better maintain the morphology and properties of nanoparticles. Besides, the experiments of the intestinal sac model showed that LBP-SeNPs can better promote the absorption of selenium in various parts (duodenum, jejunum and ileum) of the intestine. Therefore, the LBP can help to improve the structural stability of SeNPs in the digestion process and improve the bioavailability of selenium.
Collapse
Affiliation(s)
- Jixian Zhang
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Xue Yang
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Tao Ji
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Chaoting Wen
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Zhiqiang Ye
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Xiaofang Liu
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Li Liang
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China
| | - Guoyan Liu
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China.
| | - Xin Xu
- College of Food Science and Engineering, Yangzhou University, Yangzhou 225127, China.
| |
Collapse
|
19
|
Chiriboga M, Green CM, Hastman DA, Mathur D, Wei Q, Díaz SA, Medintz IL, Veneziano R. Rapid DNA origami nanostructure detection and classification using the YOLOv5 deep convolutional neural network. Sci Rep 2022; 12:3871. [PMID: 35264624 PMCID: PMC8907326 DOI: 10.1038/s41598-022-07759-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2021] [Accepted: 02/24/2022] [Indexed: 01/05/2023] Open
Abstract
The intra-image identification of DNA structures is essential to rapid prototyping and quality control of self-assembled DNA origami scaffold systems. We postulate that the YOLO modern object detection platform commonly used for facial recognition can be applied to rapidly scour atomic force microscope (AFM) images for identifying correctly formed DNA nanostructures with high fidelity. To make this approach widely available, we use open-source software and provide a straightforward procedure for designing a tailored, intelligent identification platform which can easily be repurposed to fit arbitrary structural geometries beyond AFM images of DNA structures. Here, we describe methods to acquire and generate the necessary components to create this robust system. Beginning with DNA structure design, we detail AFM imaging, data point annotation, data augmentation, model training, and inference. To demonstrate the adaptability of this system, we assembled two distinct DNA origami architectures (triangles and breadboards) for detection in raw AFM images. Using the images acquired of each structure, we trained two separate single class object identification models unique to each architecture. By applying these models in sequence, we correctly identified 3470 structures from a total population of 3617 using images that sometimes included a third DNA origami structure as well as other impurities. Analysis was completed in under 20 s with results yielding an F1 score of 0.96 using our approach.
Collapse
Affiliation(s)
- Matthew Chiriboga
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, VA, 22030, USA
| | - Christopher M Green
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA
- National Research Council, Washington, DC, 20001, USA
| | - David A Hastman
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA
- Fischell Department of Bioengineering, A. James Clark School of Engineering, University of Maryland, College Park, MD, 20742, USA
| | - Divita Mathur
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA
- College of Science, George Mason University, Fairfax, VA, 22030, USA
| | - Qi Wei
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, VA, 22030, USA
| | - Sebastían A Díaz
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA
| | - Igor L Medintz
- Center for Bio/Molecular Science and Engineering Code 6900, U.S. Naval Research Laboratory, Washington, DC, 20375, USA.
| | - Remi Veneziano
- Department of Bioengineering, Volgenau School of Engineering, George Mason University, Fairfax, VA, 22030, USA.
| |
Collapse
|
20
|
An H, Smith JW, Ji B, Cotty S, Zhou S, Yao L, Kalutantirige FC, Chen W, Ou Z, Su X, Feng J, Chen Q. Mechanism and performance relevance of nanomorphogenesis in polyamide films revealed by quantitative 3D imaging and machine learning. SCIENCE ADVANCES 2022; 8:eabk1888. [PMID: 35196079 PMCID: PMC8865778 DOI: 10.1126/sciadv.abk1888] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2021] [Accepted: 12/28/2021] [Indexed: 06/14/2023]
Abstract
Biological morphogenesis has inspired many efficient strategies to diversify material structure and functionality using a fixed set of components. However, implementation of morphogenesis concepts to design soft nanomaterials is underexplored. Here, we study nanomorphogenesis in the form of the three-dimensional (3D) crumpling of polyamide membranes used for commercial molecular separation, through an unprecedented integration of electron tomography, reaction-diffusion theory, machine learning (ML), and liquid-phase atomic force microscopy. 3D tomograms show that the spatial arrangement of crumples scales with monomer concentrations in a form quantitatively consistent with a Turing instability. Membrane microenvironments quantified from the nanomorphologies of crumples are combined with the Spiegler-Kedem model to accurately predict methanol permeance. ML classifies vastly heterogeneous crumples into just four morphology groups, exhibiting distinct mechanical properties. Our work forges quantitative links between synthesis and performance in polymer thin films, which can be applicable to diverse soft nanomaterials.
Collapse
Affiliation(s)
- Hyosung An
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
- Materials Research Laboratory, University of Illinois, Urbana, IL, USA
| | - John W. Smith
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
| | - Bingqiang Ji
- Department of Mechanical Science and Engineering, University of Illinois, Urbana, IL, USA
| | - Stephen Cotty
- Department of Chemical and Biomolecular Engineering, University of Illinois, Urbana, IL, USA
| | - Shan Zhou
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
| | - Lehan Yao
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
| | | | - Wenxiang Chen
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
- Materials Research Laboratory, University of Illinois, Urbana, IL, USA
| | - Zihao Ou
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
| | - Xiao Su
- Department of Chemical and Biomolecular Engineering, University of Illinois, Urbana, IL, USA
| | - Jie Feng
- Materials Research Laboratory, University of Illinois, Urbana, IL, USA
- Department of Mechanical Science and Engineering, University of Illinois, Urbana, IL, USA
| | - Qian Chen
- Department of Materials Science and Engineering, University of Illinois, Urbana, IL, USA
- Materials Research Laboratory, University of Illinois, Urbana, IL, USA
- Department of Chemical and Biomolecular Engineering, University of Illinois, Urbana, IL, USA
- Department of Chemistry, University of Illinois, Urbana, IL, USA
- Beckman Institute for Advanced Science and Technology, University of Illinois, Urbana, IL, USA
| |
Collapse
|
21
|
Zhang W, Lopez H, Boselli L, Bigini P, Perez-Potti A, Xie Z, Castagnola V, Cai Q, Silveira CP, de Araujo JM, Talamini L, Panini N, Ristagno G, Violatto MB, Devineau S, Monopoli MP, Salmona M, Giannone VA, Lara S, Dawson KA, Yan Y. A Nanoscale Shape-Discovery Framework Supporting Systematic Investigations of Shape-Dependent Biological Effects and Immunomodulation. ACS NANO 2022; 16:1547-1559. [PMID: 34958549 PMCID: PMC8793145 DOI: 10.1021/acsnano.1c10074] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Accepted: 12/20/2021] [Indexed: 05/29/2023]
Abstract
Since it is now possible to make, in a controlled fashion, an almost unlimited variety of nanostructure shapes, it is of increasing interest to understand the forms of biological control that nanoscale shape allows. However, a priori rational investigation of such a vast universe of shapes appears to present intractable fundamental and practical challenges. This has limited the useful systematic investigation of their biological interactions and the development of innovative nanoscale shape-dependent therapies. Here, we introduce a concept of biologically relevant inductive nanoscale shape discovery and evaluation that is ideally suited to, and will ultimately become, a vehicle for machine learning discovery. Combining the reproducibility and tunability of microfluidic flow nanochemistry syntheses, quantitative computational shape analysis, and iterative feedback from biological responses in vitro and in vivo, we show that these challenges can be mastered, allowing shape biology to be explored within accepted scientific and biomedical research paradigms. Early applications identify significant forms of shape-induced biological and adjuvant-like immunological control.
Collapse
Affiliation(s)
- Wei Zhang
- Guangdong
Provincial Education Department Key Laboratory of Nano-Immunoregulation
Tumor Microenvironment, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou 510260, Guangdong P.R. China
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Hender Lopez
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
- School
of Physics and Optometric & Clinical Sciences, Technological University Dublin, Grangegorman D07XT95, Ireland
| | - Luca Boselli
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Paolo Bigini
- Istituto
di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
| | - André Perez-Potti
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Zengchun Xie
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Valentina Castagnola
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Qi Cai
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Camila P. Silveira
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Joao M. de Araujo
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
- Departamento
de Física Teórica e Experimental, Universidade Federal do Rio Grande do Norte, 59078970 Natal, RN, Brazil
| | - Laura Talamini
- Istituto
di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
| | - Nicolò Panini
- Istituto
di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
| | - Giuseppe Ristagno
- Department
of Pathophysiology and Transplantation, University of Milan, 20122 Milan, Italy
| | - Martina B. Violatto
- Istituto
di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
| | - Stéphanie Devineau
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Marco P. Monopoli
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Mario Salmona
- Istituto
di Ricerche Farmacologiche Mario Negri IRCCS, Via Mario Negri 2, 20156 Milan, Italy
| | - Valeria A. Giannone
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Sandra Lara
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Kenneth A. Dawson
- Guangdong
Provincial Education Department Key Laboratory of Nano-Immunoregulation
Tumor Microenvironment, The Second Affiliated Hospital, Guangzhou Medical University, Guangzhou 510260, Guangdong P.R. China
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
| | - Yan Yan
- Centre
for BioNano Interactions, School of Chemistry, University College Dublin, Belfield, Dublin 4, Ireland
- School of
Biomolecular and Biomedical Science, UCD Conway Institute of Biomolecular
and Biomedical Research, University College
Dublin, Belfield, Dublin 4, Ireland
| |
Collapse
|
22
|
Xu S, Deng X, Ji S, Chen L, Zhao T, Luo F, Qiu B, Lin Z, Guo L. An algorithm-assisted automated identification and enumeration system for sensitive hydrogen sulfide sensing under dark field microscopy. Analyst 2022; 147:1492-1498. [DOI: 10.1039/d2an00149g] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
A sensitive H2S sensing strategy has been developed based on the automated identification and enumeration algorithm.
Collapse
Affiliation(s)
- Shaohua Xu
- Jiangxi Engineering Research Centre for Translational Cancer Technology, Jiangxi University of Chinese Medicine, Nanchang, Jiangxi, 330004, China
- Jiaxing Key Laboratory of Molecular Recognition and Sensing; College of Biological, Chemical Sciences and Engineering, Jiaxing University, Jiaxing 314001, China
| | - Xiaoyu Deng
- Ministry of Education Key Laboratory of Modern Preparation of Traditional Chinese Medicine, Jiangxi University of Chinese Medicine, Nanchang, Jiangxi, 330004, China
| | - Shuyi Ji
- Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information, College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350116, China
| | - Lifen Chen
- Jiaxing Key Laboratory of Molecular Recognition and Sensing; College of Biological, Chemical Sciences and Engineering, Jiaxing University, Jiaxing 314001, China
| | - Tiesong Zhao
- Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information, College of Physics and Information Engineering, Fuzhou University, Fuzhou, Fujian 350116, China
| | - Fang Luo
- Ministry of Education Key Laboratory for Analytical Science of Food Safety and Biology, Fujian Provincial Key Laboratory of Analysis and Detection for Food Safety, College of Chemistry, Fuzhou University, Fuzhou, Fujian, 350116, China
| | - Bin Qiu
- Ministry of Education Key Laboratory for Analytical Science of Food Safety and Biology, Fujian Provincial Key Laboratory of Analysis and Detection for Food Safety, College of Chemistry, Fuzhou University, Fuzhou, Fujian, 350116, China
| | - Zhenyu Lin
- Ministry of Education Key Laboratory for Analytical Science of Food Safety and Biology, Fujian Provincial Key Laboratory of Analysis and Detection for Food Safety, College of Chemistry, Fuzhou University, Fuzhou, Fujian, 350116, China
| | - Longhua Guo
- Jiaxing Key Laboratory of Molecular Recognition and Sensing; College of Biological, Chemical Sciences and Engineering, Jiaxing University, Jiaxing 314001, China
- Ministry of Education Key Laboratory for Analytical Science of Food Safety and Biology, Fujian Provincial Key Laboratory of Analysis and Detection for Food Safety, College of Chemistry, Fuzhou University, Fuzhou, Fujian, 350116, China
| |
Collapse
|
23
|
Wen H, Xu X, Cheong S, Lo SC, Chen JH, Chang SLY, Dwyer C. Metrology of convex-shaped nanoparticles via soft classification machine learning of TEM images. NANOSCALE ADVANCES 2021; 3:6956-6964. [PMID: 36132371 PMCID: PMC9417281 DOI: 10.1039/d1na00524c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Accepted: 10/11/2021] [Indexed: 06/15/2023]
Abstract
The shape of nanoparticles is a key performance parameter for many applications, ranging from nanophotonics to nanomedicines. However, the unavoidable shape variations, which occur even in precision-controlled laboratory synthesis, can significantly impact on the interpretation and reproducibility of nanoparticle performance. Here we have developed an unsupervised, soft classification machine learning method to perform metrology of convex-shaped nanoparticles from transmission electron microscopy images. Unlike the existing methods, which are based on hard classification, soft classification provides significantly greater flexibility in being able to classify both distinct shapes, as well as non-distinct shapes where hard classification fails to provide meaningful results. We demonstrate the robustness of our method on a range of nanoparticle systems, from laboratory-scale to mass-produced synthesis. Our results establish that the method can provide quantitative, accurate, and meaningful metrology of nanoparticle ensembles, even for ensembles entailing a continuum of (possibly irregular) shapes. Such information is critical for achieving particle synthesis control, and, more importantly, for gaining deeper understanding of shape-dependent nanoscale phenomena. Lastly, we also present a method, which we coin the "binary DoG", which achieves significant progress on the challenging problem of identifying the shapes of aggregated nanoparticles.
Collapse
Affiliation(s)
- Haotian Wen
- School of Materials Science and Engineering, University of New South Wales Sydney NSW 2052 Australia
| | - Xiaoxue Xu
- School of Mathematical and Physical Sciences, University of Technology, Sydney Ultimo NSW 2007 Australia
| | - Soshan Cheong
- Electron Microscope Unit, Mark Wainwright Analytical Centre, University of New South Wales Sydney NSW 2052 Australia
| | - Shen-Chuan Lo
- Material and Chemical Research Laboratories, Industrial Technology Research Institute Hsinchu Taiwan
| | - Jung-Hsuan Chen
- Material and Chemical Research Laboratories, Industrial Technology Research Institute Hsinchu Taiwan
| | - Shery L Y Chang
- School of Materials Science and Engineering, University of New South Wales Sydney NSW 2052 Australia
- Electron Microscope Unit, Mark Wainwright Analytical Centre, University of New South Wales Sydney NSW 2052 Australia
| | - Christian Dwyer
- Electron Imaging and Spectroscopy Tools PO Box 506 Sans Souci NSW 2219 Australia
- Physics, School of Science, RMIT University Melbourne Victoria 3001 Australia
| |
Collapse
|
24
|
Wen H, Luna-Romera JM, Riquelme JC, Dwyer C, Chang SLY. Statistically Representative Metrology of Nanoparticles via Unsupervised Machine Learning of TEM Images. NANOMATERIALS 2021; 11:nano11102706. [PMID: 34685147 PMCID: PMC8539342 DOI: 10.3390/nano11102706] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 09/07/2021] [Accepted: 10/06/2021] [Indexed: 11/16/2022]
Abstract
The morphology of nanoparticles governs their properties for a range of important applications. Thus, the ability to statistically correlate this key particle performance parameter is paramount in achieving accurate control of nanoparticle properties. Among several effective techniques for morphological characterization of nanoparticles, transmission electron microscopy (TEM) can provide a direct, accurate characterization of the details of nanoparticle structures and morphology at atomic resolution. However, manually analyzing a large number of TEM images is laborious. In this work, we demonstrate an efficient, robust and highly automated unsupervised machine learning method for the metrology of nanoparticle systems based on TEM images. Our method not only can achieve statistically significant analysis, but it is also robust against variable image quality, imaging modalities, and particle dispersions. The ability to efficiently gain statistically significant particle metrology is critical in advancing precise particle synthesis and accurate property control.
Collapse
Affiliation(s)
- Haotian Wen
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia
- Correspondence: (H.W.); (S.L.Y.C.)
| | - José María Luna-Romera
- Software and Computing Systems, Universidad de Sevilla, 41004 Seville, Spain; (J.M.L.-R.); (J.C.R.)
| | - José C. Riquelme
- Software and Computing Systems, Universidad de Sevilla, 41004 Seville, Spain; (J.M.L.-R.); (J.C.R.)
| | - Christian Dwyer
- Electron Imaging and Spectroscopy Tools, Sydney, NSW 2219, Australia;
| | - Shery L. Y. Chang
- School of Materials Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia
- Mark Wainwright Analytical Centre, Electron Microscope Unit, University of New South Wales, Sydney, NSW 2052, Australia
- Correspondence: (H.W.); (S.L.Y.C.)
| |
Collapse
|
25
|
Singh M, Agarwal S, Tiwari RK, Chanda S, Singh K, Agarwal P, Kashyap A, Pancham P, Mall S, R. R, Sharma S. Neuroprotective Ability of Apocynin Loaded Nanoparticles (APO-NPs) as NADPH Oxidase (NOX)-Mediated ROS Modulator for Hydrogen Peroxide-Induced Oxidative Neuronal Injuries. Molecules 2021; 26:5011. [PMID: 34443598 PMCID: PMC8400077 DOI: 10.3390/molecules26165011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Revised: 07/10/2021] [Accepted: 07/13/2021] [Indexed: 12/12/2022] Open
Abstract
Apocynin (APO) is a known multi-enzymatic complexed compound, employed as a viable NADPH oxidase (NOX) inhibitor, extensively used in both traditional and modern-day therapeutic strategies to combat neuronal disorders. However, its therapeutic efficacy is limited by lower solubility and lesser bioavailability; thus, a suitable nanocarrier system to overcome such limitations is needed. The present study is designed to fabricate APO-loaded polymeric nanoparticles (APO-NPs) to enhance its therapeutic efficacy and sustainability in the biological system. The optimized APO NPs in the study exhibited 103.6 ± 6.8 nm and -13.7 ± 0.43 mV of particle size and zeta potential, respectively, along with further confirmation by TEM. In addition, the antioxidant (AO) abilities quantified by DPPH and nitric oxide scavenging assays exhibited comparatively higher AO potential of APO-NPs than APO alone. An in-vitro release profile displayed a linear diffusion pattern of zero order kinetics for APO from the NPs, followed by its cytotoxicity evaluation on the PC12 cell line, which revealed minimal toxicity with higher cell viability, even after treatment with a stress inducer (H2O2). The stability of APO-NPs after six months showed minimal AO decline in comparison to APO only, indicating that the designed nano-formulation enhanced therapeutic efficacy for modulating NOX-mediated ROS generation.
Collapse
Affiliation(s)
- Manisha Singh
- Centre for Emerging Diseases (CFED), Department of Biotechnology, Jaypee Institute of Information Technology, Sector-62, Noida 201309, Uttar Pradesh, India; (S.A.); (P.P.); (R.R.)
| | - Shriya Agarwal
- Centre for Emerging Diseases (CFED), Department of Biotechnology, Jaypee Institute of Information Technology, Sector-62, Noida 201309, Uttar Pradesh, India; (S.A.); (P.P.); (R.R.)
| | - Raj Kumar Tiwari
- Pharmacognosy and Phytochemistry, School of Health Sciences, Pharmaceutical Sciences, UPES, Dehradun 248007, Uttarakhand, India;
| | - Silpi Chanda
- Pharmacognosy and Phytochemistry, IEC School of Pharmacy, IEC University, Solan 174103, Himachal Pradesh, India;
| | - Kuldeep Singh
- Department of Chemistry, Maharishi Markandeshwar (Deemed to Be University), Mullana 133207, Haryana, India;
| | - Prakhar Agarwal
- Department of Biosciences and Bioengineering, Indian Institute of Technology, Bombay 400076, Maharashtra, India;
| | - Aishwarya Kashyap
- Department of Biotechnology, Vellore Institute of Technology, School of Bio Sciences & Technology (SBST), Vellore 632014, Tamil Nadu, India;
| | - Pranav Pancham
- Centre for Emerging Diseases (CFED), Department of Biotechnology, Jaypee Institute of Information Technology, Sector-62, Noida 201309, Uttar Pradesh, India; (S.A.); (P.P.); (R.R.)
| | - Shweta Mall
- Department of Animal Genetics and Breeding, Southern Regional Station of Indian Council of Agriculture Research—Research Institute, Bangalore 560030, Karnataka, India;
| | - Rachana R.
- Centre for Emerging Diseases (CFED), Department of Biotechnology, Jaypee Institute of Information Technology, Sector-62, Noida 201309, Uttar Pradesh, India; (S.A.); (P.P.); (R.R.)
| | - Shalini Sharma
- Sunder Deep Pharmacy College, NH-9, Delhi-Meerut Expressway, Ghaziabad 201015, Uttar Pradesh, India;
| |
Collapse
|
26
|
Mill L, Wolff D, Gerrits N, Philipp P, Kling L, Vollnhals F, Ignatenko A, Jaremenko C, Huang Y, De Castro O, Audinot JN, Nelissen I, Wirtz T, Maier A, Christiansen S. Synthetic Image Rendering Solves Annotation Problem in Deep Learning Nanoparticle Segmentation. SMALL METHODS 2021; 5:e2100223. [PMID: 34927995 DOI: 10.1002/smtd.202100223] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2021] [Revised: 04/17/2021] [Indexed: 05/14/2023]
Abstract
Nanoparticles occur in various environments as a consequence of man-made processes, which raises concerns about their impact on the environment and human health. To allow for proper risk assessment, a precise and statistically relevant analysis of particle characteristics (such as size, shape, and composition) is required that would greatly benefit from automated image analysis procedures. While deep learning shows impressive results in object detection tasks, its applicability is limited by the amount of representative, experimentally collected and manually annotated training data. Here, an elegant, flexible, and versatile method to bypass this costly and tedious data acquisition process is presented. It shows that using a rendering software allows to generate realistic, synthetic training data to train a state-of-the art deep neural network. Using this approach, a segmentation accuracy can be derived that is comparable to man-made annotations for toxicologically relevant metal-oxide nanoparticle ensembles which were chosen as examples. The presented study paves the way toward the use of deep learning for automated, high-throughput particle detection in a variety of imaging techniques such as in microscopies and spectroscopies, for a wide range of applications, including the detection of micro- and nanoplastic particles in water and tissue samples.
Collapse
Affiliation(s)
- Leonid Mill
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
| | - David Wolff
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Nele Gerrits
- Health Unit, Flemish Institute for Technological Research, Mol, 2400, Belgium
| | - Patrick Philipp
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Lasse Kling
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Florian Vollnhals
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Andrew Ignatenko
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Christian Jaremenko
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Yixing Huang
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Institut für Nanotechnologie und korrelative Mikroskopie, 91301, Forchheim, Germany
| | - Olivier De Castro
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Jean-Nicolas Audinot
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Inge Nelissen
- Health Unit, Flemish Institute for Technological Research, Mol, 2400, Belgium
| | - Tom Wirtz
- Advanced Instrumentation for Ion Nano-Analytics, Materials Research and Technology Department, Luxembourg Institute of Science and Technology, Belvaux, L-4422, Luxembourg
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
| | - Silke Christiansen
- Institute of Optics, Information and Photonics, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany
- Physics Department, Free University, 14195, Berlin, Germany
- Correlative Microscopy and Material Data Department, Fraunhofer Institute for Ceramic Technologies and Systems, 01277, Dresden, Germany
| |
Collapse
|