Classification of endomicroscopic images of the lung based on random subwindows and extra-trees

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

A novel flood risk management approach based on future climate and land use change scenarios

Climate change and increasing urbanization are two primary factors responsible for the increased risk of serious flooding around the world. The prediction and monitoring of the effects of land use/land cover (LULC) and climate change on flood risk are critical steps in the development of appropriate strategies to reduce potential damage. This study aimed to develop a new approach by combining machine learning (namely the XGBoost, CatBoost, LightGBM, and ExtraTree models) and hydraulic modeling to predict the effects of climate change and LULC change on land that is at risk of flooding. For the years 2005, 2020, 2035, and 2050, machine learning was used to model and predict flood susceptibility under different scenarios of LULC, while hydraulic modeling was used to model and predict flood depth and flood velocity, based on the RCP 8.5 climate change scenario. The two elements were used to build a flood risk assessment, integrating socioeconomic data such as LULC, population density, poverty rate, number of women, number of schools, and cultivated area. Flood risk was then computed, using the analytical hierarchy process, by combining flood hazard, exposure, and vulnerability. The results showed that the area at high and very high flood risk increased rapidly, as did the areas of high/very high exposure, and high/very high vulnerability. They also showed how flood risk had increased rapidly from 2005 to 2020 and would continue to do so in 2035 and 2050, due to the dynamics of climate change and LULC change, population growth, the number of women, and the number of schools - particularly in the flood zone. The results highlight the relationships between flood risk and environmental and socio-economic changes and suggest that flood risk management strategies should also be integrated in future analyses. The map built in this study shows past and future flood risk, providing insights into the spatial distribution of urban area in flood zones and can be used to facilitate the development of priority measures, flood mitigation being most important.

A Fused Multidimensional EEG Classification Method Based on an Extreme Tree Feature Selection

When a brain-computer interface (BCI) is designed, high classification accuracy is difficult to obtain for motor imagery (MI) electroencephalogram (EEG) signals in view of their relatively low signal-to-noise ratio. In this paper, a fused multidimensional classification method based on extreme tree feature selection (FMCM-ETFS) is proposed for discerning motor imagery EEG tasks. First, the EEG signal was filtered by a Butterworth filter for preprocessing. Second, C3, C4, and CZ channels were selected to extract time-frequency domain and spatial domain features using autoregressive (AR), common spatial pattern (CSP), and discrete wavelet transform (DWT). The extracted features were fused for a further feature elimination. Then, the features were selected using three feature selection methods: recursive feature elimination (RFE), principal component analysis method (PCA), and extreme trees (ET). The selected feature vectors were classified using support vector machines (SVM). Finally, a total of twelve subjects’ EEG data from Inner Mongolia University of Technology (IMUT data), the 2nd BCI competition in 2003, and the 4th BCI competition in 2008 were employed to show the effectiveness of this proposed FMCM-ETFS method. The results show that the classification accuracy using the multidimensional fused feature extraction (AR + CSP + DWT) is 3%–20% higher than those using the aforementioned three single feature extractions (AR, CSP, and DWT). Extreme trees (ET), which is a sort of tree-based model method, outperforms RFE and PCA by 1%–9% in term of classification accuracies, when these three methods were applied to the procedure of feature extraction, respectively.

Body Temperature Monitoring for Regular COVID-19 Prevention Based on Human Daily Activity Recognition

Existing wearable systems that use G-sensors to identify daily activities have been widely applied for medical, sports and military applications, while body temperature as an obvious physical characteristic that has rarely been considered in the system design and relative applications of HAR. In the context of the normalization of COVID-19, the prevention and control of the epidemic has become a top priority. Temperature monitoring plays an important role in the preliminary screening of the population for fever. Therefore, this paper proposes a wearable device embedded with inertial and temperature sensors that is used to apply human behavior recognition (HAR) to body surface temperature detection for body temperature monitoring and adjustment by evaluating recognition algorithms. The sensing system consists of an STM 32-based microcontroller, a 6-axis (accelerometer and gyroscope) sensor, and a temperature sensor to capture the original data from 10 individual participants under 4 different daily activity scenarios. Then, the collected raw data are pre-processed by signal standardization, data stacking and resampling. For HAR, several machine learning (ML) and deep learning (DL) algorithms are implemented to classify the activities. To compare the performance of different classifiers on the seven-dimensional dataset with temperature sensing signals, evaluation metrics and the algorithm running time are considered, and random forest (RF) is found to be the best-performing classifier with 88.78% recognition accuracy, which is higher than the case of the absence of temperature data (<78%). In addition, the experimental results show that participants' body surface temperature in dynamic activities was lower compared to sitting, which can be associated with the possible missing fever population due to temperature deviations in COVID-19 prevention. According to different individual activities, epidemic prevention workers are supposed to infer the corresponding standard normal body temperature of a patient by referring to the specific values of the mean expectation and variance in the normal distribution curve provided in this paper.

Solitary pulmonary nodule imaging approaches and the role of optical fibre-based technologies

Solitary pulmonary nodules (SPNs) are a clinical challenge, given there is no single clinical sign or radiological feature that definitively identifies a benign from a malignant SPN. The early detection of lung cancer has a huge impact on survival outcome. Consequently, there is great interest in the prompt diagnosis, and treatment of malignant SPNs. Current diagnostic pathways involve endobronchial/transthoracic tissue biopsies or radiological surveillance, which can be associated with suboptimal diagnostic yield, healthcare costs and patient anxiety. Cutting-edge technologies are needed to disrupt and improve, existing care pathways. Optical fibre-based techniques, which can be delivered via the working channel of a bronchoscope or via transthoracic needle, may deliver advanced diagnostic capabilities in patients with SPNs. Optical endomicroscopy, an autofluorescence-based imaging technique, demonstrates abnormal alveolar structure in SPNs in vivo Alternative optical fingerprinting approaches, such as time-resolved fluorescence spectroscopy and fluorescence-lifetime imaging microscopy, have shown promise in discriminating lung cancer from surrounding healthy tissue. Whilst fibre-based Raman spectroscopy has enabled real-time characterisation of SPNs in vivo Fibre-based technologies have the potential to enable in situ characterisation and real-time microscopic imaging of SPNs, which could aid immediate treatment decisions in patients with SPNs. This review discusses advances in current imaging modalities for evaluating SPNs, including computed tomography (CT) and positron emission tomography-CT. It explores the emergence of optical fibre-based technologies, and discusses their potential role in patients with SPNs and suspected lung cancer.

Image computing for fibre-bundle endomicroscopy: A review

Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.

Bayesian bacterial detection using irregularly sampled optical endomicroscopy images

Pneumonia is a major cause of morbidity and mortality of patients in intensive care. Rapid determination of the presence and gram status of the pathogenic bacteria in the distal lung may enable a more tailored treatment regime. Optical Endomicroscopy (OEM) is an emerging medical imaging platform with preclinical and clinical utility. Pulmonary OEM via multi-core fibre bundles has the potential to provide in vivo, in situ, fluorescent molecular signatures of the causes of infection and inflammation. This paper presents a Bayesian approach for bacterial detection in OEM images. The model considered assumes that the observed pixel fluorescence is a linear combination of the actual intensity value associated with tissues or background, corrupted by additive Gaussian noise and potentially by an additional sparse outlier term modelling anomalies (bacteria). The bacteria detection problem is formulated in a Bayesian framework and prior distributions are assigned to the unknown model parameters. A Markov chain Monte Carlo algorithm based on a partially collapsed Gibbs sampler is used to sample the posterior distribution of the unknown parameters. The proposed algorithm is first validated by simulations conducted using synthetic datasets for which good performance is obtained. Analysis is then conducted using two ex vivo lung datasets in which fluorescently labelled bacteria are present in the distal lung. A good correlation between bacteria counts identified by a trained clinician and those of the proposed method, which detects most of the manually annotated regions, is observed.

A murine model to study vasoreactivity and intravascular flow in lung isograft microvessels

Intravital microscopy of orthotopic lung tissue is technically demanding, especially for repeated investigations. Therefore, we have established a novel approach, which allows non-invasive repetitive in vivo microscopy of ectopic lung tissue in dorsal skinfold chambers. Syngeneic subpleural peripheral lung tissue and autologous endometrium (control) were transplanted onto the striated muscle within dorsal skinfold chambers of C57BL/6 mice. Grafts were analysed by intravital fluorescence microscopy over 14 days. Angiogenesis occurred in the grafts on day 3, as indicated by sinusoidal microvessels on the grafts’ edges with very slow blood flow, perifocal oedema, and haemorrhage. By day 10, lung transplants were completely revascularized, exhibited a dense network of microvessels with irregular diameters, chaotic angioarchitecture, and high blood flow. Compared to lung tissue, endometrial grafts contained a structured, glomerulus-like vessel architecture with lower blood flow. Despite missing ventilation, hypoxic vasoconstriction of the lung tissue arterioles occurred. In contrast, endometrium tissue arterioles dilated during hypoxia and constricted in hyperoxia. This demonstrates that ectopic lung grafts keep their ability for organ-specific hypoxic vasoconstriction. These findings indicate that our approach is suitable for repetitive in vivo pulmonary microcirculation analyses. The high blood flow and hypoxia-induced vasoconstriction in lung grafts suggest a physiological intrinsic vasoregulation independent of the recipient tissue.

Assessing the utility of autofluorescence-based pulmonary optical endomicroscopy to predict the malignant potential of solitary pulmonary nodules in humans

Solitary pulmonary nodules are common, often incidental findings on chest CT scans. The investigation of pulmonary nodules is time-consuming and often leads to protracted follow-up with ongoing radiological surveillance, however, clinical calculators that assess the risk of the nodule being malignant exist to help in the stratification of patients. Furthermore recent advances in interventional pulmonology include the ability to both navigate to nodules and also to perform autofluorescence endomicroscopy. In this study we assessed the efficacy of incorporating additional information from label-free fibre-based optical endomicrosopy of the nodule on assessing risk of malignancy. Using image analysis and machine learning approaches, we find that this information does not yield any gain in predictive performance in a cohort of patients. Further advances with pulmonary endomicroscopy will require the addition of molecular tracers to improve information from this procedure.

A Dataset for Breast Cancer Histopathological Image Classification

Today, medical image analysis papers require solid experiments to prove the usefulness of proposed methods. However, experiments are often performed on data selected by the researchers, which may come from different institutions, scanners, and populations. Different evaluation measures may be used, making it difficult to compare the methods. In this paper, we introduce a dataset of 7909 breast cancer histopathology images acquired on 82 patients, which is now publicly available from http://web.inf.ufpr.br/vri/breast-cancer-database. The dataset includes both benign and malignant images. The task associated with this dataset is the automated classification of these images in two classes, which would be a valuable computer-aided diagnosis tool for the clinician. In order to assess the difficulty of this task, we show some preliminary results obtained with state-of-the-art image classification systems. The accuracy ranges from 80% to 85%, showing room for improvement is left. By providing this dataset and a standardized evaluation protocol to the scientific community, we hope to gather researchers in both the medical and the machine learning field to advance toward this clinical application.