1
|
Park CF, Barzegar-Keshteli M, Korchagina K, Delrocq A, Susoy V, Jones CL, Samuel ADT, Rahi SJ. Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation. Nat Methods 2024; 21:142-149. [PMID: 38052988 DOI: 10.1038/s41592-023-02096-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Accepted: 10/20/2023] [Indexed: 12/07/2023]
Abstract
Reading out neuronal activity from three-dimensional (3D) functional imaging requires segmenting and tracking individual neurons. This is challenging in behaving animals if the brain moves and deforms. The traditional approach is to train a convolutional neural network with ground-truth (GT) annotations of images representing different brain postures. For 3D images, this is very labor intensive. We introduce 'targeted augmentation', a method to automatically synthesize artificial annotations from a few manual annotations. Our method ('Targettrack') learns the internal deformations of the brain to synthesize annotations for new postures by deforming GT annotations. This reduces the need for manual annotation and proofreading. A graphical user interface allows the application of the method end-to-end. We demonstrate Targettrack on recordings where neurons are labeled as key points or 3D volumes. Analyzing freely moving animals exposed to odor pulses, we uncover rich patterns in interneuron dynamics, including switching neuronal entrainment on and off.
Collapse
Affiliation(s)
- Core Francisco Park
- Department of Physics and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Mahsa Barzegar-Keshteli
- Laboratory of the Physics of Biological Systems, Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Kseniia Korchagina
- Laboratory of the Physics of Biological Systems, Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Ariane Delrocq
- Laboratory of the Physics of Biological Systems, Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Vladislav Susoy
- Department of Physics and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Corinne L Jones
- Swiss Data Science Center, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Aravinthan D T Samuel
- Department of Physics and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Sahand Jamal Rahi
- Laboratory of the Physics of Biological Systems, Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.
| |
Collapse
|
2
|
Pavarino EC, Yang E, Dhanyasi N, Wang MD, Bidel F, Lu X, Yang F, Francisco Park C, Bangalore Renuka M, Drescher B, Samuel ADT, Hochner B, Katz PS, Zhen M, Lichtman JW, Meirovitch Y. mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops. Front Neural Circuits 2023; 17:952921. [PMID: 37396399 PMCID: PMC10309043 DOI: 10.3389/fncir.2023.952921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Accepted: 04/17/2023] [Indexed: 07/04/2023] Open
Abstract
Connectomics is fundamental in propelling our understanding of the nervous system's organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from four different animals and five datasets, amounting to around 180 h of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of four pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/. With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.
Collapse
Affiliation(s)
- Elisa C. Pavarino
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Emma Yang
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Nagaraju Dhanyasi
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Mona D. Wang
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON, Canada
- Department of Molecular Genetics, University of Toronto, Toronto, ON, Canada
| | - Flavie Bidel
- Department of Neurobiology, Silberman Institute of Life Sciences, The Hebrew University, Jerusalem, Israel
| | - Xiaotang Lu
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Fuming Yang
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | | | - Mukesh Bangalore Renuka
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Brandon Drescher
- Department of Biology, University of Massachusetts Amherst, Amherst, MA, United States
| | | | - Binyamin Hochner
- Department of Neurobiology, Silberman Institute of Life Sciences, The Hebrew University, Jerusalem, Israel
| | - Paul S. Katz
- Department of Biology, University of Massachusetts Amherst, Amherst, MA, United States
| | - Mei Zhen
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON, Canada
| | - Jeff W. Lichtman
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| | - Yaron Meirovitch
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, United States
| |
Collapse
|
3
|
Pavarino EC, Yang E, Dhanyasi N, Wang M, Bidel F, Lu X, Yang F, Park CF, Renuka MB, Drescher B, Samuel AD, Hochner B, Katz PS, Zhen M, Lichtman JW, Meirovitch Y. mEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops. bioRxiv 2023:2023.04.17.537196. [PMID: 37131600 PMCID: PMC10153173 DOI: 10.1101/2023.04.17.537196] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Connectomics is fundamental in propelling our understanding of the nervous system’s organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from 4 different animals and 5 datasets, amounting to around 180 hours of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of 4 pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/ . With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.
Collapse
Affiliation(s)
| | - Emma Yang
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| | - Nagaraju Dhanyasi
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| | - Mona Wang
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, Ontario, Canada
- Department of Molecular Genetics, University of Toronto, Toronto, Ontario, Canada
| | - Flavie Bidel
- Department of Neurobiology, Silberman Institute of Life Sciences, The Hebrew University, Jerusalem, Israel
| | - Xiaotang Lu
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| | - Fuming Yang
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| | | | | | - Brandon Drescher
- Department Biology, University of Massachusetts Amherst, Amherst, MA, USA
| | | | - Binyamin Hochner
- Department of Neurobiology, Silberman Institute of Life Sciences, The Hebrew University, Jerusalem, Israel
| | - Paul S. Katz
- Department Biology, University of Massachusetts Amherst, Amherst, MA, USA
| | - Mei Zhen
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, Ontario, Canada
| | - Jeff W. Lichtman
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| | - Yaron Meirovitch
- Department of Cellular and Molecular Biology, Harvard University, Cambridge, MA, USA
| |
Collapse
|
4
|
Park CF, Allys E, Villaescusa-Navarro F, Finkbeiner D. Quantification of High-dimensional Non-Gaussianities and Its Implication to Fisher Analysis in Cosmology. Astrophys J 2023; 946:107. [PMID: 37681217 PMCID: PMC10482003 DOI: 10.3847/1538-4357/acbe3b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 02/04/2023] [Accepted: 02/21/2023] [Indexed: 09/09/2023]
Abstract
It is well known that the power spectrum is not able to fully characterize the statistical properties of non-Gaussian density fields. Recently, many different statistics have been proposed to extract information from non-Gaussian cosmological fields that perform better than the power spectrum. The Fisher matrix formalism is commonly used to quantify the accuracy with which a given statistic can constrain the value of the cosmological parameters. However, these calculations typically rely on the assumption that the sampling distribution of the considered statistic follows a multivariate Gaussian distribution. In this work, we follow Sellentin & Heavens and use two different statistical tests to identify non-Gaussianities in different statistics such as the power spectrum, bispectrum, marked power spectrum, and wavelet scattering transform (WST). We remove the non-Gaussian components of the different statistics and perform Fisher matrix calculations with the Gaussianized statistics using Quijote simulations. We show that constraints on the parameters can change by a factor of ∼2 in some cases. We show with simple examples how statistics that do not follow a multivariate Gaussian distribution can achieve artificially tight bounds on the cosmological parameters when using the Fisher matrix formalism. We think that the non-Gaussian tests used in this work represent a powerful tool to quantify the robustness of Fisher matrix calculations and their underlying assumptions. We release the code used to compute the power spectra, bispectra, and WST that can be run on both CPUs and GPUs.
Collapse
Affiliation(s)
| | - Erwan Allys
- Laboratoire de Physique de l'École Normale Supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France
| | - Francisco Villaescusa-Navarro
- Center for Computational Astrophysics, Flatiron Institute, 162 5th Avenue, New York, NY 10010, USA
- Department of Astrophysical Sciences, Princeton University, Peyton Hall, Princeton, NJ 08544, USA
| | | |
Collapse
|
5
|
Susoy V, Hung W, Witvliet D, Whitener JE, Wu M, Park CF, Graham BJ, Zhen M, Venkatachalam V, Samuel ADT. Natural sensory context drives diverse brain-wide activity during C. elegans mating. Cell 2021; 184:5122-5137.e17. [PMID: 34534446 PMCID: PMC8488019 DOI: 10.1016/j.cell.2021.08.024] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2020] [Revised: 05/18/2021] [Accepted: 08/18/2021] [Indexed: 10/20/2022]
Abstract
Natural goal-directed behaviors often involve complex sequences of many stimulus-triggered components. Understanding how brain circuits organize such behaviors requires mapping the interactions between an animal, its environment, and its nervous system. Here, we use brain-wide neuronal imaging to study the full performance of mating by the C. elegans male. We show that as mating unfolds in a sequence of component behaviors, the brain operates similarly between instances of each component but distinctly between different components. When the full sensory and behavioral context is taken into account, unique roles emerge for each neuron. Functional correlations between neurons are not fixed but change with behavioral dynamics. From individual neurons to circuits, our study shows how diverse brain-wide dynamics emerge from the integration of sensory perception and motor actions in their natural context.
Collapse
Affiliation(s)
- Vladislav Susoy
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA.
| | - Wesley Hung
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON M5G 1X5, Canada
| | - Daniel Witvliet
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Joshua E Whitener
- Department of Physics, Northeastern University, Boston, MA 02115, USA
| | - Min Wu
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON M5G 1X5, Canada
| | - Core Francisco Park
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Brett J Graham
- Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
| | - Mei Zhen
- Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital, Toronto, ON M5G 1X5, Canada
| | - Vivek Venkatachalam
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Department of Physics, Northeastern University, Boston, MA 02115, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA.
| | - Aravinthan D T Samuel
- Department of Physics, Harvard University, Cambridge, MA 02138, USA; Center for Brain Science, Harvard University, Cambridge, MA 02138, USA.
| |
Collapse
|
6
|
Colton HS, Park CF. ANOSMA OR "SQUEEZE-UPS". Science 1930; 72:579. [PMID: 17736106 DOI: 10.1126/science.72.1875.579] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|