1
|
Wiesner D, Suk J, Dummer S, Nečasová T, Ulman V, Svoboda D, Wolterink JM. Generative modeling of living cells with SO(3)-equivariant implicit neural representations. Med Image Anal 2024; 91:102991. [PMID: 37839341 DOI: 10.1016/j.media.2023.102991] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2023] [Revised: 08/20/2023] [Accepted: 10/02/2023] [Indexed: 10/17/2023]
Abstract
Data-driven cell tracking and segmentation methods in biomedical imaging require diverse and information-rich training data. In cases where the number of training samples is limited, synthetic computer-generated data sets can be used to improve these methods. This requires the synthesis of cell shapes as well as corresponding microscopy images using generative models. To synthesize realistic living cell shapes, the shape representation used by the generative model should be able to accurately represent fine details and changes in topology, which are common in cells. These requirements are not met by 3D voxel masks, which are restricted in resolution, and polygon meshes, which do not easily model processes like cell growth and mitosis. In this work, we propose to represent living cell shapes as level sets of signed distance functions (SDFs) which are estimated by neural networks. We optimize a fully-connected neural network to provide an implicit representation of the SDF value at any point in a 3D+time domain, conditioned on a learned latent code that is disentangled from the rotation of the cell shape. We demonstrate the effectiveness of this approach on cells that exhibit rapid deformations (Platynereis dumerilii), cells that grow and divide (C. elegans), and cells that have growing and branching filopodial protrusions (A549 human lung carcinoma cells). A quantitative evaluation using shape features and Dice similarity coefficients of real and synthetic cell shapes shows that our model can generate topologically plausible complex cell shapes in 3D+time with high similarity to real living cell shapes. Finally, we show how microscopy images of living cells that correspond to our generated cell shapes can be synthesized using an image-to-image model.
Collapse
Affiliation(s)
- David Wiesner
- Centre for Biomedical Image Analysis, Masaryk University, Brno, Czech Republic.
| | - Julian Suk
- Department of Applied Mathematics & Technical Medical Centre, University of Twente, Enschede, The Netherlands
| | - Sven Dummer
- Department of Applied Mathematics & Technical Medical Centre, University of Twente, Enschede, The Netherlands
| | - Tereza Nečasová
- Centre for Biomedical Image Analysis, Masaryk University, Brno, Czech Republic
| | - Vladimír Ulman
- IT4Innovations, VSB - Technical University of Ostrava, Ostrava, Czech Republic
| | - David Svoboda
- Centre for Biomedical Image Analysis, Masaryk University, Brno, Czech Republic
| | - Jelmer M Wolterink
- Department of Applied Mathematics & Technical Medical Centre, University of Twente, Enschede, The Netherlands.
| |
Collapse
|
2
|
Brémond-Martin C, Simon-Chane C, Clouchoux C, Histace A. Brain organoid data synthesis and evaluation. Front Neurosci 2023; 17:1220172. [PMID: 37650105 PMCID: PMC10465177 DOI: 10.3389/fnins.2023.1220172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Accepted: 07/24/2023] [Indexed: 09/01/2023] Open
Abstract
Introduction Datasets containing only few images are common in the biomedical field. This poses a global challenge for the development of robust deep-learning analysis tools, which require a large number of images. Generative Adversarial Networks (GANs) are an increasingly used solution to expand small datasets, specifically in the biomedical domain. However, the validation of synthetic images by metrics is still controversial and psychovisual evaluations are time consuming. Methods We augment a small brain organoid bright-field database of 40 images using several GAN optimizations. We compare these synthetic images to the original dataset using similitude metrcis and we perform an psychovisual evaluation of the 240 images generated. Eight biological experts labeled the full dataset (280 images) as syntetic or natural using a custom-built software. We calculate the error rate per loss optimization as well as the hesitation time. We then compare these results to those provided by the similarity metrics. We test the psychovalidated images in a training step of a segmentation task. Results and discussion Generated images are considered as natural as the original dataset, with no increase of the hesitation time by experts. Experts are particularly misled by perceptual and Wasserstein loss optimization. These optimizations render the most qualitative and similar images according to metrics to the original dataset. We do not observe a strong correlation but links between some metrics and psychovisual decision according to the kind of generation. Particular Blur metric combinations could maybe replace the psychovisual evaluation. Segmentation task which use the most psychovalidated images are the most accurate.
Collapse
Affiliation(s)
- Clara Brémond-Martin
- ETIS Laboratory UMR 8051 (CY Cergy Paris Université, ENSEA, CNRS), Cergy, France
- Witsee, Neoxia, Paris, France
| | - Camille Simon-Chane
- ETIS Laboratory UMR 8051 (CY Cergy Paris Université, ENSEA, CNRS), Cergy, France
| | | | - Aymeric Histace
- ETIS Laboratory UMR 8051 (CY Cergy Paris Université, ENSEA, CNRS), Cergy, France
| |
Collapse
|
3
|
Kozubek M. When Deep Learning Meets Cell Image Synthesis. Cytometry A 2019; 97:222-225. [PMID: 31889406 DOI: 10.1002/cyto.a.23957] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2019] [Accepted: 12/03/2019] [Indexed: 02/03/2023]
Affiliation(s)
- Michal Kozubek
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Czech Republic
| |
Collapse
|
4
|
Scalbert M, Couzinie-Devy F, Fezzani R. Generic Isolated Cell Image Generator. Cytometry A 2019; 95:1198-1206. [PMID: 31593370 PMCID: PMC6899488 DOI: 10.1002/cyto.a.23899] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2019] [Revised: 08/30/2019] [Accepted: 09/10/2019] [Indexed: 11/24/2022]
Abstract
Building automated cancer screening systems based on image analysis is currently a hot topic in computer vision and medical imaging community. One of the biggest challenges of such systems, especially those using state‐of‐the‐art deep learning techniques, is that they usually require a large amount of training data to be accurate. However, in the medical field, the confidentiality of the data and the need for medical expertise to label them significantly reduce the amount of training data available. A common practice to overcome this problem is to apply data set augmentation techniques to artificially increase the size of the training data set. Classical data set augmentation methods such as geometrical or color transformations are efficient but still produce a limited amount of new data. Hence, there has been interest in data set augmentation methods using generative models able to synthesize a wider variety of new data. VitaDX is actually developing an automated bladder cancer screening system based on the analysis of cell images contained in urinary cytology digital slides. Currently, the number of available labeled cell images is limited and therefore exploitation of the full potential of deep learning techniques is not possible. In an attempt to increase the number of labeled cell images, a new generic generator for 2D cell images has been developed and is described in this article. This framework combines previous works on cell image generation and a recent style transfer method referred to as doodle‐style transfer in this article. To the best of our knowledge, we are the first to use a doodle‐style transfer method for synthetic cell image generation. This framework is quite modular and could be applied to other cell image generation problems. A statistical evaluation has shown that features of real and synthetic cell images followed roughly the same distribution. Finally, the realism of the synthetic cell images has been assessed through a visual evaluation performed with the help of medical experts. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Marin Scalbert
- Department of Research & Development, VitaDX, Paris, France
| | | | - Riadh Fezzani
- Department of Research & Development, VitaDX, Paris, France
| |
Collapse
|
5
|
Glotsos D, Kostopoulos S, Ravazoula P, Cavouras D. Image quilting and wavelet fusion for creation of synthetic microscopy nuclei images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2018; 162:177-186. [PMID: 29903484 DOI: 10.1016/j.cmpb.2018.05.023] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2018] [Revised: 05/09/2018] [Accepted: 05/16/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND AND OBJECTIVE In this study a texture simulation methodology is proposed for composing synthetic tissue microscopy images that could serve as a quantitative gold standard for the evaluation of the reliability, accuracy and performance of segmentation algorithms in computer-aided diagnosis. METHODS A library of background and nuclei regions was generated using pre-segmented Haematoxylin and Eosin images of brain tumours. Background image samples were used as input to an image quilting algorithm that produced the synthetic background image. Randomly selected pre-segmented nuclei were randomly fused on the synthetic background using a wavelet-based fusion approach. To investigate whether the produced synthetic images are meaningful and similar to real world images, two different tests were performed, one qualitative by an experienced histopathologist and one quantitative using the normalized mutual information and the Kullback-Leibler tests. To illustrate the challenges that synthetic images may pose to object recognition algorithms, two segmentation methodologies were utilized for nuclei detection, one based on the Otsu thresholding and another based on the seeded region growing approach. RESULTS Results showed a satisfactory to good resemblance of the synthetic with the real world images according to both qualitative and quantitative tests. The segmentation accuracy was slightly higher for the seeded region growing algorithm (87.2%) than the Otsu's algorithm (86.3%). CONCLUSIONS Since we know the exact coordinates of the regions of interest within the synthesised images, these images could then serve as a 'gold standard' for evaluation of segmentation algorithms in computer-aided diagnosis in tissue microscopy.
Collapse
Affiliation(s)
- Dimitris Glotsos
- Medical Image and Signal Processing (medisp) Lab, Department of Biomedical Engineering, Technological Educational Institute of Athens, Ag. Spyridonos Street, Egaleo, 122 10 Athens, Greece.
| | - Spiros Kostopoulos
- Medical Image and Signal Processing (medisp) Lab, Department of Biomedical Engineering, Technological Educational Institute of Athens, Ag. Spyridonos Street, Egaleo, 122 10 Athens, Greece
| | | | - Dionisis Cavouras
- Medical Image and Signal Processing (medisp) Lab, Department of Biomedical Engineering, Technological Educational Institute of Athens, Ag. Spyridonos Street, Egaleo, 122 10 Athens, Greece
| |
Collapse
|
6
|
Chen W, Liao B, Li W, Dong X, Flavel M, Jois M, Li G, Xian B. Segmenting Microscopy Images of Multi-Well Plates Based on Image Contrast. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2017; 23:932-937. [PMID: 28712372 DOI: 10.1017/s1431927617012375] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Image segmentation is a key process in analyzing biological images. However, it is difficult to detect the differences between foreground and background when the image is unevenly illuminated. The unambiguous segmenting of multi-well plate microscopy images with various uneven illuminations is a challenging problem. Currently, no publicly available method adequately solves these various problems in bright-field multi-well plate images. Here, we propose a new method based on contrast values which removes the need for illumination correction. The presented method is effective enough to distinguish foreground and therefore a model organism (Caenorhabditis elegans) from an unevenly illuminated microscope image. In addition, the method also can solve a variety of problems caused by different uneven illumination scenarios. By applying this methodology across a wide range of multi-well plate microscopy images, we show that our approach can consistently analyze images with uneven illuminations with unparalleled accuracy and successfully solve various problems associated with uneven illumination. It can be used to process the microscopy images captured from multi-well plates and detect experimental subjects from an unevenly illuminated background.
Collapse
Affiliation(s)
- Weiyang Chen
- School of Information, Qilu University of Technology, Jinan 250353, China
| | - Bo Liao
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, Hunan 410082, China
| | - Weiwei Li
- School of Information, Qilu University of Technology, Jinan 250353, China
| | - Xiangjun Dong
- School of Information, Qilu University of Technology, Jinan 250353, China
| | - Matthew Flavel
- School of Life Sciences, La Trobe University, Bundoora, VIC 3083, Australia
| | - Markandeya Jois
- School of Life Sciences, La Trobe University, Bundoora, VIC 3083, Australia
| | - Guojun Li
- Beijing Center for Disease Prevention and Control/Beijing Center of Preventive Medicine Research, Beijing 100013, China
- School of Public Health, Capital Medical University, Beijing 100086, China
| | - Bo Xian
- Key Laboratory of Computational Biology, CAS Center for Excellence in Molecular Cell Science, Collaborative Innovation Center for Genetics and Developmental Biology, Chinese Academy of Sciences-Max Planck Partner Institute for Computational Biology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, 320 Yue Yang Road, Shanghai 200031, China
| |
Collapse
|
7
|
Circular shape constrained fuzzy clustering (CiscFC) for nucleus segmentation in Pap smear images. Comput Biol Med 2017; 85:13-23. [DOI: 10.1016/j.compbiomed.2017.04.008] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2016] [Revised: 03/31/2017] [Accepted: 04/12/2017] [Indexed: 01/24/2023]
|
8
|
Ulman V, Svoboda D, Nykter M, Kozubek M, Ruusuvuori P. Virtual cell imaging: A review on simulation methods employed in image cytometry. Cytometry A 2016; 89:1057-1072. [PMID: 27922735 DOI: 10.1002/cyto.a.23031] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2016] [Revised: 07/20/2016] [Accepted: 11/14/2016] [Indexed: 02/03/2023]
Abstract
The simulations of cells and microscope images thereof have been used to facilitate the development, selection, and validation of image analysis algorithms employed in cytometry as well as for modeling and understanding cell structure and dynamics beyond what is visible in the eyepiece. The simulation approaches vary from simple parametric models of specific cell components-especially shapes of cells and cell nuclei-to learning-based synthesis and multi-stage simulation models for complex scenes that simultaneously visualize multiple object types and incorporate various properties of the imaged objects and laws of image formation. This review covers advances in artificial digital cell generation at scales ranging from particles up to tissue synthesis and microscope image simulation methods, provides examples of the use of simulated images for various purposes ranging from subcellular object detection to cell tracking, and discusses how such simulators have been validated. Finally, the future possibilities and limitations of simulation-based validation are considered. © 2016 International Society for Advancement of Cytometry.
Collapse
Affiliation(s)
- Vladimír Ulman
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - David Svoboda
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Matti Nykter
- Institute of Biosciences and Medical Technology - BioMediTech, University of Tampere, Tampere, Finland
| | - Michal Kozubek
- Centre for Biomedical Image Analysis, Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Pekka Ruusuvuori
- Institute of Biosciences and Medical Technology - BioMediTech, University of Tampere, Tampere, Finland.,Pori Campus, Tampere University of Technology, Pori, Finland
| |
Collapse
|
9
|
Kovacheva VN, Snead D, Rajpoot NM. A model of the spatial tumour heterogeneity in colorectal adenocarcinoma tissue. BMC Bioinformatics 2016; 17:255. [PMID: 27342072 PMCID: PMC4919876 DOI: 10.1186/s12859-016-1126-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2016] [Accepted: 06/07/2016] [Indexed: 01/27/2023] Open
Abstract
Background There have been great advancements in the field of digital pathology. The surge in development of analytical methods for such data makes it crucial to develop benchmark synthetic datasets for objectively validating and comparing these methods. In addition, developing a spatial model of the tumour microenvironment can aid our understanding of the underpinning laws of tumour heterogeneity. Results We propose a model of the healthy and cancerous colonic crypt microenvironment. Our model is designed to generate synthetic histology image data with parameters that allow control over cancer grade, cellularity, cell overlap ratio, image resolution, and objective level. Conclusions To the best of our knowledge, ours is the first model to simulate histology image data at sub-cellular level for healthy and cancerous colon tissue, where the cells have different compartments and are organised to mimic the microenvironment of tissue in situ rather than dispersed cells in a cultured environment. Qualitative and quantitative validation has been performed on the model results demonstrating good similarity to the real data. The simulated data could be used to validate techniques such as image restoration, cell and crypt segmentation, and cancer grading. Electronic supplementary material The online version of this article (doi:10.1186/s12859-016-1126-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Violeta N Kovacheva
- Department of Systems Biology, University of Warwick, Coventry, CV4 7AL, UK.
| | - David Snead
- Department of HistopathologyUniversity Hospitals Coventry and Warwickshire, Coventry, CV2 2DX, UK
| | - Nasir M Rajpoot
- Department of Computer Science, University of Warwick, Coventry, CV4 7AL, UK.,Department of Computer Science and Engineering, Qatar University, Doha, Qatar
| |
Collapse
|
10
|
Kozubek M. Challenges and Benchmarks in Bioimage Analysis. ADVANCES IN ANATOMY, EMBRYOLOGY, AND CELL BIOLOGY 2016; 219:231-62. [PMID: 27207369 DOI: 10.1007/978-3-319-28549-8_9] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
Similar to the medical imaging community, the bioimaging community has recently realized the need to benchmark various image analysis methods to compare their performance and assess their suitability for specific applications. Challenges sponsored by prestigious conferences have proven to be an effective means of encouraging benchmarking and new algorithm development for a particular type of image data. Bioimage analysis challenges have recently complemented medical image analysis challenges, especially in the case of the International Symposium on Biomedical Imaging (ISBI). This review summarizes recent progress in this respect and describes the general process of designing a bioimage analysis benchmark or challenge, including the proper selection of datasets and evaluation metrics. It also presents examples of specific target applications and biological research tasks that have benefited from these challenges with respect to the performance of automatic image analysis methods that are crucial for the given task. Finally, available benchmarks and challenges in terms of common features, possible classification and implications drawn from the results are analysed.
Collapse
Affiliation(s)
- Michal Kozubek
- Faculty of Informatics, Centre for Biomedical Image Analysis, Masaryk University, Botanická 68a, Brno, 60200, Czech Republic.
| |
Collapse
|
11
|
Ulman V, Orémuš Z, Svoboda D. TRAgen: A Tool for Generation of Synthetic Time-Lapse Image Sequences of Living Cells. IMAGE ANALYSIS AND PROCESSING — ICIAP 2015 2015. [DOI: 10.1007/978-3-319-23231-7_56] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|