The application of improved densenet algorithm in accurate image recognition

Image recognition technology belongs to an important research field of artificial intelligence. In order to enhance the application value of image recognition technology in the field of computer vision and improve the technical dilemma of image recognition, the research improves the feature reuse method of dense convolutional network. Based on gradient quantization, traditional parallel algorithms have been improved. This improvement allows for independent parameter updates layer by layer, reducing communication time and data volume. The introduction of quantization error reduces the impact of gradient loss on model convergence. The test results show that the improvement strategy designed by the research improves the model parameter efficiency while ensuring the recognition effect. Narrowing the learning rate is conducive to refining the updating granularity of model parameters, and deepening the number of network layers can effectively improve the final recognition accuracy and convergence effect of the model. It is better than the existing state-of-the-art image recognition models, visual geometry group and EfficientNet. The parallel acceleration algorithm, which is improved by the gradient quantization, performs better than the traditional synchronous data parallel algorithm, and the improvement of the acceleration ratio is obvious. Compared with the traditional synchronous data parallel algorithm and stale synchronous parallel algorithm, the optimized parallel acceleration algorithm of the study ensures the image data training speed and solves the bottleneck problem of communication data. The model designed by the research improves the accuracy and training speed of image recognition technology and expands the use of image recognition technology in the field of computer vision.Please confirm the affiliation details of [1] is correct.The relevant detailed information in reference [1] has been confirmed to be correct.

IremulbNet: Rethinking the inverted residual architecture for image recognition

An increasing need of running Convolutional Neural Network (CNN) models on mobile devices encourages the studies on efficient and lightweight neural network model. In this paper, an Inverse Residual Multi-Branch Network named IremulbNet is proposed to solve the problem of insufficient classification accuracy in existing lightweight network models. The core module of this model is to reconstruct an inverse residual structure, in which a special feature fusion method, multi-branch feature extraction, and depthwise separable convolution techniques are used to improve the classification accuracy. The performance of model is tested using image databases. Experimental results show that for the fine-grained image dataset Imagenet-woof, IremulbNet achieved 10.9%, 12.2%, and 15.3% higher accuracy than that of MobileNet V3, ShuffleNet V2, and PeleeNet, respectively. Moreover, it can reduce inference time (GPU) about 42.09% and 75.56% compared to classic ResNet50 and DenseNet121.

Comprehensive watermelon disease recognition dataset

Plant diseases pose a significant obstacle to global agricultural productivity, impacting crop quality yield and causing substantial economic losses for farmers. Watermelon, a commonly cultivated succulent vine plant, is rich in hydration and essential nutrients. However, it is susceptible to various diseases due to unfavorable environmental conditions and external factors, leading to compromised quality and substantial financial setbacks. Swift identification and management of crop diseases are imperative to minimize losses, enhance yield, reduce costs, and bolster agricultural output. Conventional disease diagnosis methods are often labor-intensive, time-consuming, ineffective, and prone to subjectivity. As a result, there is a critical need to advance research into machine-based models for disease detection in watermelons. This paper presents a large dataset of watermelons that can be used to train a machine vision-based illness detection model. Images of healthy and diseased watermelons from the Mosaic Virus, Anthracnose, and Downy Mildew Disease are included in the dataset's five separate classifications. Images were painstakingly collected on June 25, 2023, in close cooperation with agricultural experts from the highly regarded Regional Horticulture Research Station in Lebukhali, Patuakhali.

Using deep learning-based artificial intelligence electronic images in improving middle school teachers' literacy

With the rapid development of societal information, electronic educational resources have become an indispensable component of modern education. In response to the increasingly formidable challenges faced by secondary school teachers, this study endeavors to analyze and explore the application of artificial intelligence (AI) methods to enhance their cognitive literacy. Initially, this discourse delves into the application of AI-generated electronic images in the training and instruction of middle school educators, subjecting it to thorough analysis. Emphasis is placed on elucidating the pivotal role played by AI electronic images in elevating the proficiency of middle school teachers. Subsequently, an integrated intelligent device serves as the foundation for establishing a model that applies intelligent classification and algorithms based on the Structure of the Observed Learning Outcome (SOLO). This model is designed to assess the cognitive literacy and teaching efficacy of middle school educators, and its performance is juxtaposed with classification algorithms such as support vector machine (SVM) and decision trees. The findings reveal that, following 600 iterations of the model, the SVM algorithm achieves a 77% accuracy rate in recognizing teacher literacy, whereas the SOLO algorithm attains 80%. Concurrently, the spatial complexities of the SVM-based and SOLO-based intelligent literacy improvement models are determined to be 45 and 22, respectively. Notably, it is discerned that, with escalating iterations, the SOLO algorithm exhibits higher accuracy and reduced spatial complexity in evaluating teachers' pedagogical literacy. Consequently, the utilization of AI methodologies proves highly efficacious in advancing electronic imaging technology and enhancing the efficacy of image recognition in educational instruction.

YOLOv8s-CGF: a lightweight model for wheat ear Fusarium head blight detection

Fusarium head blight (FHB) is a destructive disease that affects wheat production. Detecting FHB accurately and rapidly is crucial for improving wheat yield. Traditional models are difficult to apply to mobile devices due to large parameters, high computation, and resource requirements. Therefore, this article proposes a lightweight detection method based on an improved YOLOv8s to facilitate the rapid deployment of the model on mobile terminals and improve the detection efficiency of wheat FHB. The proposed method introduced a C-FasterNet module, which replaced the C2f module in the backbone network. It helps reduce the number of parameters and the computational volume of the model. Additionally, the Conv in the backbone network is replaced with GhostConv, further reducing parameters and computation without significantly affecting detection accuracy. Thirdly, the introduction of the Focal CIoU loss function reduces the impact of sample imbalance on the detection results and accelerates the model convergence. Lastly, the large target detection head was removed from the model for lightweight. The experimental results show that the size of the improved model (YOLOv8s-CGF) is only 11.7 M, which accounts for 52.0% of the original model (YOLOv8s). The number of parameters is only 5.7 × 106 M, equivalent to 51.4% of the original model. The computational volume is only 21.1 GFLOPs, representing 74.3% of the original model. Moreover, the mean average precision (mAP@0.5) of the model is 99.492%, which is 0.003% higher than the original model, and the mAP@0.5:0.95 is 0.269% higher than the original model. Compared to other YOLO models, the improved lightweight model not only achieved the highest detection precision but also significantly reduced the number of parameters and model size. This provides a valuable reference for FHB detection in wheat ears and deployment on mobile terminals in field environments.

One-Dimensional Covalent Organic Framework-Based Multilevel Memristors for Neuromorphic Computing

Memristors are essential components of neuromorphic systems that mimic the synaptic plasticity observed in biological neurons. In this study, a novel approach employing one-dimensional covalent organic framework (1D COF) films was explored to enhance the performance of memristors. The unique structural and electronic properties of two 1D COF films (COF-4,4'-methylenedianiline (MDA) and COF-4,4'-oxydianiline (ODA)) offer advantages for multilevel resistive switching, which is a key feature in neuromorphic computing applications. By further introducing a TiO2 layer on the COF-ODA film, a built-in electric field between the COF-TiO2 interfaces could be generated, demonstrating the feasibility of utilizing COFs as a platform for constructing memristors with tunable resistive states. The 1D nanochannels of these COF structures contributed to the efficient modulation of electrical conductance, enabling precise control over synaptic weights in neuromorphic circuits. This study also investigated the potential of these COF-based memristors to achieve energy-efficient and high-density memory devices.

Comprehensive image dataset for enhancing object detection in chemical experiments

The application of image recognition in chemical experiments has the potential to enhance experiment recording and risk management. However, the current scarcity of suitable benchmarking datasets restricts the applications of machine vision techniques in chemical experiments. This data article presents an image dataset featuring common chemical apparatuses and experimenter's hands. The images have been meticulously annotated, providing detailed information for precise object detection through deep learning methods. The images were captured from videos filmed in organic chemistry laboratories. This dataset comprises a total of 5078 images including diverse backgrounds and situations surrounding the objects. Detailed annotations are provided in accompanying text files. The dataset is organized into training, validation, and test subsets. Each subset is stored within independent folders for easy access and utilization.

A comprehensive dragon fruit image dataset for detecting the maturity and quality grading of dragon fruit

Dragon fruit, often referred to as pitaya, is a tropical fruit with various types, including both white-fleshed and red-fleshed varieties. Its distinctive appearance is complemented by a range of potential health advantages. These include its abundance of nutrients and antioxidants, which contribute to a robust immune system, aid in blood sugar regulation, and support the well-being of the heart, bones, and skin. Consequently, the global desire for dragon fruit is yielding substantial economic advantages for developing nations like Bangladesh, which in turn underscores the pressing need for an automated system to identify the optimal harvest time and differentiate between fresh and defective fruits to ensure quality. To accomplish this objective, this paper introduces an extensive collection of high-resolution dragon fruits because effective detection by machine learning models necessitates a substantial amount of data. The dataset was painstakingly gathered during a span of four months from three distinct locations in Bangladesh, with the valuable assistance of domain experts. Possible application of the dataset encompasses quality evaluation, robotic harvesting, and packaging systems, ultimately boosting the effectiveness of dragon fruit production procedures. The dataset has the potential to be a valuable resource for researchers interested in dragon fruit cultivation, offering a solid foundation for the application of computer vision and deep learning methods in the agricultural industry.

Machine learning constructs color features to accelerate development of long-term continuous water quality monitoring

Long-term continuous water quality monitoring (LTCM) is crucial to ensure the safety of water resources. However, lab-based pollutant detection via machine learning (ML) usually involves colorimetric materials or sensors, and it cannot be ignored that sensor limitations prevent their use for LTCM. To address this challenge, we propose a novel method that leverages image recognition to establish a relationship between pollutant concentration and color. By extracting efficient color variation features from raw pixel matrices using a combination of Kmeans clustering and RGB average features, the concentrations of pollutants that are difficult to distinguish by the naked eyes can be directly captured without the need for sensors and preprocessing. Four ML models (XGBoost, Linear, support vector regression (SVR), and Ridge) achieved up to a 95.9% increase in coefficient of determination (R2) compared to principal component analysis (PCA). In the prediction of the concentration of simulated pollutants such as Cu2+, Co2+, Rhodamine B, and the concentration of Cr(VI) in actual electroplating wastewater, natural resource water and drinking water, over 95% R2 was achieved. The method reported in our work can effectively capture subtle color changes that cannot be observed by the naked eyes without any preprocessing of water samples, providing a reliable method for LTCM.

An extensive real-world in field tomato image dataset involving maturity classification and recognition of fresh and defect tomatoes

Tomato, a fruiting plant species within the Solanaceae family, is a widely used ingredient in culinary dishes due to its sweet and acidic flavor profile, as well as its rich nutritional content. Recognized for its potential health benefits, including reducing the risk of coronary artery disease and specific types of cancer, tomatoes have become a staple in global cuisine. Traditional methods for tomato maturity assessment, harvesting, quality grading, and packaging are often labor-intensive and economically inefficient. This paper introduces an extensive dataset of high-resolution tomato images collected over an eight-month period from the demonstration fields of Sher-E-Bangla Agricultural University in Dhaka, Bangladesh, in collaboration with plant breeding experts of the same university. The dataset was meticulously curated to ensure precision and consistency, encompassing various stages of tomato maturity, including images of both fresh and defective tomatoes. This dataset is a valuable resource for researchers, stakeholders, and individuals interested in tomato production in Bangladesh, providing a robust foundation for leveraging computer vision and deep learning techniques in the agriculture sector. The dataset's potential applications extend to automating tasks such as robotic harvesting, quality assessment, and packaging systems, ultimately enhancing the efficiency of tomato production processes.

Dermatological disease prediction and diagnosis system using deep learning

The prevalence of skin illnesses is higher than that of other diseases. Fungal infection, bacteria, allergies, viruses, genetic factors, and environmental factors are among important causative factors that have continuously escalated the degree and incidence of skin diseases. Medical technology based on lasers and photonics has made it possible to identify skin illnesses considerably more rapidly and correctly. However, the cost of such a diagnosis is currently limited and prohibitively high and restricted to developed areas. The present paper develops a holistic, critical, and important skin disease prediction system that utilizes machine learning and deep learning algorithms to accurately identify up to 20 different skin diseases with a high F1 score and efficiency. Deep learning algorithms like Xception, Inception-v3, Resnet50, DenseNet121, and Inception-ResNet-v2 were employed to accurately classify diseases based on the images. The training and testing have been performed on an enlarged dataset, and classification was performed for 20 diseases. The algorithm developed was free from any inherent bias and treated all classes equally. The present model, which was trained using the Xception algorithm, is highly efficient and accurate for 20 different skin conditions, with a dataset of over 10,000 photos. The developed system was able to classify 20 different dermatological diseases with high accuracy and precision.

Colon Disease Classification Method Based on Deep Learning

Objective Colorectal cancer (CRC) is a common malignant tumor of the digestive system with a high incidence rate. It is prone to misdiagnosis or missed diagnosis in clinical practice. Therefore, researching computer-aided diagnostic methods for endoscopic colon disease image classification is of great importance. This study proposes a deep learning-based method for colon disease classification. It utilizes intestinal images or captures from an endoscope camera to achieve intelligent classification of gastrointestinal diseases, providing assistance to doctors in their decision-making process. Methods Firstly, the algorithm is used to preprocess the dataset by removing duplicates and applying enhancement techniques. Two different network architectures, namely A_Vit, MobileNet, are employed. The models are trained using the same parameters and dataset with the Adam optimizer. The training process generates loss curves, accuracy, and recall rates for each of the four network architectures. Results The results indicate that the training with A_Vit has shown better performance, achieving an accuracy rate of 95.76% and an impressive recall rate of 97.21%. Therefore, the model trained using the A_Vit network structure is ultimately selected as the preferred choice. Conclusion This method can improve the efficiency and accuracy of colon disease diagnosis.

Thermal imaging: The digital eye facilitates high-throughput phenotyping traits of plant growth and stress responses

Plant phenotyping is important for plants to cope with environmental changes and ensure plant health. Imaging techniques are perceived as the most critical and reliable tools for studying plant phenotypes. Thermal imaging has opened up new opportunities for nondestructive imaging of plant phenotyping. However, a comprehensive summary of thermal imaging in plant phenotyping is still lacking. Here we discuss the progress and future prospects of thermal imaging for assessing plant growth and stress responses. First, we classify thermal imaging into ground-based and aerial platforms based on their adaptability to different experimental environments (including laboratory, greenhouse, and field). It is convenient to collect phenotypic information of different dimensions. Second, in order to enhance the efficiency of thermal image processing, automatic algorithms based on deep learning are employed instead of traditional manual methods, greatly reducing the time cost of experiments. Considering its ease of implementation, handling and instant response, thermal imaging has been widely used in research on environmental stress, crop yield, and seed vigor. We have found that thermal imaging can detect thermal energy dissipation caused by living organisms (e.g., pests, viruses, bacteria, fungi, and oomycetes), enabling early disease diagnosis. It also recognizes changes leaf surface temperatures resulting from reduced transpiration rates caused by nutrient deficiency, drought, salinity, or freezing. Furthermore, thermal imaging predicts crop yield under different water states and forecasts the viability of dormant seeds after water absorption by monitoring temperature changes in the seeds. This work will assist biologists and agronomists in studying plant phenotypes and serve a guide for breeders to develop high-yielding, stress-tolerant, and superior crops.

Unlocking the Potential of Zebrafish Research with Artificial Intelligence: Advancements in Tracking, Processing, and Visualization

Zebrafish have become a widely accepted model organism for biomedical research due to their strong cortisol stress response, behavioral strain differences, and sensitivity to both drug treatments and predators. However, experimental zebrafish studies generate substantial data that must be analyzed through objective, accurate, and repeatable analysis methods. Recently, advancements in artificial intelligence (AI) have enabled automated tracking, image recognition, and data analysis, leading to more efficient and insightful investigations. In this review, we examine key AI applications in zebrafish research, including behavior analysis, genomics, and neuroscience. With the development of deep learning technology, AI algorithms have been used to precisely analyze and identify images of zebrafish, enabling automated testing and analysis. By applying AI algorithms in genomics research, researchers have elucidated the relationship between genes and biology, providing a better basis for the development of disease treatments and gene therapies. Additionally, the development of more effective neuroscience tools could help researchers better understand the complex neural networks in the zebrafish brain. In the future, further advancements in AI technology are expected to enable more extensive and in-depth medical research applications in zebrafish, improving our understanding of this important animal model. This review highlights the potential of AI technology in achieving the full potential of zebrafish research by enabling researchers to efficiently track, process, and visualize the outcomes of their experiments.

Developing tongue coating status assessment using image recognition with deep learning

PURPOSE

To build an image recognition network to evaluate tongue coating status.

METHODS

Two image recognition networks were built: one for tongue detection and another for tongue coating classification. Digital tongue photographs were used to develop both networks; images from 251 (178 women, 74.7±6.6 years) and 144 older adults (83 women, 73.8±7.3 years) who volunteered to participate were used for the tongue detection network and coating classification network, respectively. The learning objective of the tongue detection network is to extract a rectangular region that includes the tongue. You-Only-Look-Once (YOLO) v2 was used as the detection network, and transfer learning was performed using ResNet-50. The accuracy was evaluated by calculating the intersection over the union. For tongue coating classification, the rectangular area including the tongue was divided into a grid of 7×7. Five experienced panelists scored the tongue coating in each area using one of five grades, and the tongue coating index (TCI) was calculated. Transfer learning for tongue coating grades was performed using ResNet-18, and the TCI was calculated. Agreement between the panelists and network for the tongue coating grades in each area and TCI was evaluated using the kappa coefficient and intraclass correlation, respectively.

RESULTS

The tongue detection network recognized the tongue with a high intersection over union (0.885±0.081). The tongue coating classification network showed high agreement with tongue coating grades and TCI, with a kappa coefficient of 0.826 and an intraclass correlation coefficient of 0.807, respectively.

CONCLUSIONS

Image recognition enables simple and detailed assessment of tongue coating status.

A dataset for successful recognition of cucumber diseases

Plant disease is a common impediment to the productivity of the world's agricultural production, which adversely affects the quality and yield of crops and causes heavy economic losses to farmers. The cucumber is a frequently grown creeping vine plant that has few calories but is high in water and several vital vitamins and minerals. Due to the unfavorable ecological environment and non-biological circumstances, cucumber diseases will adversely harm the quality of cucumber and cause heavy financial loss. Early identification and protection of crop diseases are essential for disease management, crop yield enhancement, cost reduction, and boosting agricultural production. The traditional diagnosis of crop diseases is often time-consuming, laborious, ineffective, and subjective. To cope with this scenario, the development of a machine-based model which can detect cucumber diseases is a demand of time for increasing agricultural production. This article offers a major cucumber dataset to build an effective machine vision-based model which can detect more variety of cucumber diseases. The dataset includes eight different types of classes containing disease-affected and disease-free cucumber images (Anthracnose, Bacterial Wilt, Belly Rot, Downy Mildew, Pythium Fruit Rot, Gummy Stem Blight, Fresh leaves, and Fresh cucumber) which were collected from the 6th to 30th of May 2022 from real fields with the cooperation of an expert from an agricultural institution. The dataset is hosted by the Department of Computer Science and Engineering, Jahangirnagar University, and is freely accessible at https://data.mendeley.com/datasets/y6d3z6f8z9/1.

Effects of nutrition intervention on the rehabilitation level and quality of life of patients with diabetes foot: Image observation based on image recognition technology

Today, the application of CT scanning technology is still limited. Therefore, an image observation method based on image recognition technology is proposed in this paper to study the impact of nutritional intervention on the rehabilitation level and quality of life of diabetic foot patients. Firstly, in view of the noise problems existing in CT images, this paper adopts the denoising technology, in which Gaussian noise and salt and pepper noise denoising algorithms are mainly used. Noise interference in the image can be removed by denoising processing, and the clarity and quality of the image can be improved. Subsequently, in order to improve the recognition and classification accuracy of the image, data enhancement and data standardization techniques are adopted to normalize the pixel values of the image, so as to make different images comparable, so as to improve the stability and classification accuracy of the model. Finally, CNN model is used to study image classification. The effects of nutritional intervention on the rehabilitation level and quality of life of patients with diabetic foot were investigated by setting up a comparative experiment. The results showed that nutrition intervention nursing can effectively improve the rehabilitation level of diabetic foot patients, improve the quality of life, improve nursing satisfaction.

Tropical cyclone dataset for a high-resolution global nonhydrostatic atmospheric simulation

This dataset is a time series of tropical cyclones simulated using the high-resolution Nonhydrostatic Icosahedral Atmospheric Model (NICAM). By tracking tropical cyclones from 30 years of simulation data, 2,463 tracks that include the life stages of precursors (pre-TCs), tropical cyclones (TCs), and post-tropical cyclones (post-TCs), if any, were extracted. Each track data includes the time, latitude, longitude, maximum wind speed, minimum pressure, elapsed time since onset, and life-stage label of the tropical cyclone. The numbers of steps (6 h) for pre-TCs, TCs, and post-TCs were 45,288, 55,206, and 37,312, respectively. The dataset for each step also consists of atmospheric field data of multiple physical quantities, such as outgoing longwave radiation at the top-of-the-atmosphere, sea level pressure, sea surface temperature, specific humidity at 600 hPa, and zonal and meridional winds at 850 and 200 hPa over a 1000 km² area that includes a tropical cyclone at its center. This dataset can be used to develop machine-learning models for the detection, intensity prediction, and cyclogenesis prediction of tropical cyclones.

Facilitating COVID recognition from X-rays with computer vision models and transfer learning

Multimedia data plays an important role in medicine and healthcare since EHR (Electronic Health Records) entail complex images and videos for analyzing patient data. In this article, we hypothesize that transfer learning with computer vision can be adequately harnessed on such data, more specifically chest X-rays, to learn from a few images for assisting accurate, efficient recognition of COVID. While researchers have analyzed medical data (including COVID data) using computer vision models, the main contributions of our study entail the following. Firstly, we conduct transfer learning using a few images from publicly available big data on chest X-rays, suitably adapting computer vision models with data augmentation. Secondly, we aim to find the best fit models to solve this problem, adjusting the number of samples for training and validation to obtain the minimum number of samples with maximum accuracy. Thirdly, our results indicate that combining chest radiography with transfer learning has the potential to improve the accuracy and timeliness of radiological interpretations of COVID in a cost-effective manner. Finally, we outline applications of this work during COVID and its recovery phases with future issues for research and development. This research exemplifies the use of multimedia technology and machine learning in healthcare.

[Intelligent identification of livestock, a source of Schistosoma japonicum infection, based on deep learning of unmanned aerial vehicle images]

OBJECTIVE

To develop an intelligent recognition model based on deep learning algorithms of unmanned aerial vehicle (UAV) images, and to preliminarily explore the value of this model for remote identification, monitoring and management of cattle, a source of Schistosoma japonicum infection.

METHODS

Oncomelania hupensis snail-infested marshlands around the Poyang Lake area were selected as the study area. Image datasets of the study area were captured by aerial photography with UAV and subjected to augmentation. Cattle in the sample database were annotated with the annotation software VGG Image Annotator to create the morphological recognition labels for cattle. A model was created for intelligent recognition of livestock based on deep learning-based Mask R-convolutional neural network (CNN) algorithms. The performance of the model for cattle recognition was evaluated with accuracy, precision, recall, F1 score and mean precision.

RESULTS

A total of 200 original UAV images were obtained, and 410 images were yielded following data augmentation. A total of 2 860 training samples of cattle recognition were labeled. The created deep learning-based Mask R-CNN model converged following 200 iterations, with an accuracy of 88.01%, precision of 92.33%, recall of 94.06%, F1 score of 93.19%, and mean precision of 92.27%, and the model was effective to detect and segment the morphological features of cattle.

CONCLUSIONS

The deep learning-based Mask R-CNN model is highly accurate for recognition of cattle based on UAV images, which is feasible for remote intelligent recognition, monitoring, and management of the source of S. japonicum infection.

Deep learning-assisted automated sewage pipe defect detection for urban water environment management

A healthy sewage pipe system plays a significant role in urban water management by collecting and transporting wastewater and stormwater, which can be assessed by hydraulic model. However, sewage pipe defects have been observed frequently in recent years during regular pipe maintenance according to the captured interior videos of underground pipes by closed-circuit television (CCTV) robots. In this case, hydraulic model constructed based on a healthy pipe would produce large deviations with that in real hydraulic performance and even be out of work, which can result in unanticipated damages such as blockage collapse or stormwater overflows. Quick defect evaluation and defect quantification are the precondition to achieve risk assessment and model calibration of urban water management, but currently pipe defects assessment still largely relies on technicians to check the CCTV videos/images. An automated sewage pipe defect detection system is necessary to timely determine pipe issues and then rehabilitate or renew sewage pipes, while the rapid development of deep learning especially in recent five years provides a fantastic opportunity to construct automated pipe defect detection system by image recognition. Given the initial success of deep learning application in CCTV interpretation, the review (i) integrated the methodological framework of automated sewage pipe defect detection, including data acquisition, image pre-processing, feature extraction, model construction and evaluation metrics, (ii) discussed the state-of-the-art performance of deep learning in pipe defects classification, location, and severity rating evaluation (e.g., up to ~96 % of accuracy and 140 FPS of processing speed), and (iii) proposed risk assessment and model calibration in urban water management by considering pipe defects. This review introduces a novel practical application-oriented methodology including defect data acquisition by CCTV, model construction by deep learning, and model application, provides references for further improving accuracy and generalization ability of urban water management models in practical application.

Automated monitoring and detection of disease using a generic facial feature scoring system - A case study on FMD infected cows

Digital images are becoming more readily available and possibilities for image processing are developing rapidly. This opens the possibility to use digital images to monitor and detect diseases in animals. In this paper we present 1) a generic facial feature scoring system based on seven facial features, 2) manual scores of images of Holstein Frisian heifers during foot-and-mouth disease vaccine efficacy trials and 3) automatic disease scores of the same animals. The automatic scoring system was based on the manual version and trained on annotated images from the manual scoring system. For both systems we found an increase in disease scores three days post infection, followed by a recovery. This temporal pattern matched with observations made by animal caretakers. Importantly, the automatic system was able to discern animals that were protected by the vaccine, and did not develop blisters at the feet, and animals that were not protected. Finally, automatic scores could be used to detect healthy and sick animals with a sensitivity and specificity of 0.94 on the second and third days following infection in an experimental setting. This generic facial feature disease scoring system could be further developed and extended to lactating Holstein Frisian dairy cows, other breeds and other infectious diseases. The system could be applied during animal experiments or, after further development, in a farm setting.

Comparative approach to different convolutional neural network (CNN) architectures applied to human behavior detection

Medical diagnostics, product classification, surveillance and detection of inappropriate behavior are becoming increasingly sophisticated due to the development of methods based on image analysis using neural networks. Considering this, in this work, we evaluate state-of-the-art convolutional neural network architectures proposed in recent years to classify the driving behavior and distractions of drivers. Our main goal is to measure the performance of such architectures using only free resources (i.e., free graphic processing unit, open source) and to evaluate how much of this technological evolution is available to regular users.

Do you see what I see? Testing horses' ability to recognise real-life objects from 2D computer projections

The use of 2-dimensional representations (e.g. photographs or digital images) of real-life physical objects has been an important tool in studies of animal cognition. Horses are reported to recognise objects and individuals (conspecifics and humans) from printed photographs, but it is unclear whether image recognition is also true for digital images, e.g. computer projections. We expected that horses trained to discriminate between two real-life objects would show the same learnt response to digital images of these objects indicating that the images were perceived as objects, or representations of such. Riding-school horses (N = 27) learnt to touch one of two objects (target object counterbalanced between horses) to instantly receive a food reward. After discrimination learning (three consecutive sessions of 8/10 correct trials), horses were immediately tested with on-screen images of the objects over 10 image trials interspersed with five real object trials. At first image presentation, all but two horses spontaneously responded to the images with the learnt behaviour by contacting one of the two images, but the number of horses touching the correct image was not different from chance (14/27 horses, p > 0.05). Only one horse touched the correct image above chance level across 10 image trials (9/10 correct responses, p = 0.021). Our findings thus question whether horses recognise real-life objects from digital images. We discuss how methodological factors and individual differences (i.e. age, welfare state) might have influenced animals' response to the images, and the importance of validating the suitability of stimuli of this kind for cognitive studies in horses.

Classification of precancerous lesions based on fusion of multiple hierarchical features

PURPOSE

To investigate an identification method for precancerous gastric cancer based on the fusion of superficial features and deep features of gastroscopic images. The purpose of this study is to make most use of superficial features and deep features to provide clinicians with clinical decision support to assist the diagnosis of precancerous gastric diseases and reduce the workload of doctors.

METHODS

According to the nature of gastroscopic images, 75-dimensional shallow features were manually designed, including histogram features, texture features and high-order features of the image; then, based on the constructed convolutional neural networks such as ResNet and GoogLeNet, before the output layer. A fully connected layer is added as the deep feature of the image. In order to ensure consistent feature weights, the number of neurons in the fully connected layer is designed to be 75 dimensions. Therefore, the superficial and deep features of the image are concatenated, and a machine learning classifier is used to identify gastric polyps, there are three types of gastric precancerous diseases such as gastric polyps, gastric ulcers and gastric erosions.

RESULTS

A dataset with 420 images was collected for each disease, and divided into a training set and a test set with a ratio of 5:1, and then based on the dataset, three methods, such as traditional machine learning, deep learning, and feature fusion, were used respectively. For model training and testing of traditional machine learning and feature fusion, SVM, RF and BP neural network are used as the classification results of the classifier. For deep learning, the GoogLeNet, ResNet, and ResNeXt were implemented. The test results of the model on the test set show that the recognition accuracy of the proposed feature fusion method reaches (SVM: 85.18%; RF: 83.42%; BPNN: 85.18%), which is better than the traditional machine learning method (SVM: 80.17%; RF: 82.37%; BPNN: 84.12%) and the deep learning method (GoogLeNet: 82.54%; ResNet-18: 81.67%; ResNet-50: 81.67%; ResNeXt-50: 82.11%), which proves that this method has obvious advantages.

CONCLUSION

This study provides a new strategy for the identification of precancerous gastric cancer, improving the efficiency and accuracy of precancerous gastric cancer identification, and hopes to provide substantial practical help for the identification of gastric precancerous diseases.

Automatic generation of operation notes in endoscopic pituitary surgery videos using workflow recognition

Operation notes are a crucial component of patient care. However, writing them manually is prone to human error, particularly in high pressured clinical environments. Automatic generation of operation notes from video recordings can alleviate some of the administrative burdens, improve accuracy, and provide additional information. To achieve this for endoscopic pituitary surgery, 27-steps were identified via expert consensus. Then, for the 97-videos recorded for this study, a timestamp of each step was annotated by an expert surgeon. To automatically determine whether a step is present in a video, a three-stage architecture was created. Firstly, for each step, a convolution neural network was used for binary image classification on each frame of a video. Secondly, for each step, the binary frame classifications were passed to a discriminator for binary video classification. Thirdly, for each video, the binary video classifications were passed to an accumulator for multi-label step classification. The architecture was trained on 77-videos, and tested on 20-videos, where a 0.80 weighted-F1 score was achieved. The classifications were inputted into a clinically based predefined template, and further enriched with additional video analytics. This work therefore demonstrates automatic generation of operative notes from surgical videos is feasible, and can assist surgeons during documentation.

Recognition of necrotic regions in MRI images of chronic spinal cord injury based on superpixel

BACKGROUND AND OBJECTIVE

The cystic cavity and its surrounding dense glial scar formed in chronic spinal cord injury (SCI) hinder the regeneration of nerve axons. Accurate location of the necrotic regions formed by the scar and the cavity is conducive to eliminate the re-growth obstacles and promote SCI treatment. This work aims to realize the accurate and automatic location of necrotic regions in the chronic SCI magnetic resonance imaging (MRI).

METHODS

In this study, a method based on superpixel is proposed to identify the necrotic regions of spinal cord in chronic SCI MRI. Superpixels were obtained by a simple linear iterative clustering algorithm, and feature sets were constructed from intensity statistical features, gray level co-occurrence matrix features, Gabor texture features, local binary pattern features and superpixel areas. Subsequently, the recognition effects of support vector machine (SVM) and random forest (RF) classification model on necrotic regions were compared from accuracy (ACC), positive predictive value (PPV), sensitivity (SE), specificity (SP), Dice coefficient and algorithm running time.

RESULTS

The method is evaluated on T1- and T2-weighted MRI spinal cord images of 24 adult female Wistar rats. And an automatic recognition method for spinal cord necrosis regions was established based on the SVM classification model finally. The recognition results were 1.00±0.00 (ACC), 0.89±0.09 (PPV), 0.88±0.12 (SE), 1.00±0.00 (SP) and 0.88±0.07 (Dice), respectively.

CONCLUSIONS

The proposed method can accurately and noninvasively identify the necrotic regions in MRI, which is helpful for the pre-intervention assessment and post-intervention evaluation of chronic SCI research and treatments, and promoting the clinical transformation of chronic SCI research.

Weakly-supervised learning method for the recognition of potato leaf diseases

As a crucial food crop, potatoes are highly consumed worldwide, while they are also susceptible to being infected by diverse diseases. Early detection and diagnosis can prevent the epidemic of plant diseases and raise crop yields. To this end, this study proposed a weakly-supervised learning approach for the identification of potato plant diseases. The foundation network was applied with the lightweight MobileNet V2, and to enhance the learning ability for minute lesion features, we modified the existing MobileNet-V2 architecture using the fine-tuning approach conducted by transfer learning. Then, the atrous convolution along with the SPP module was embedded into the pre-trained networks, which was followed by a hybrid attention mechanism containing channel attention and spatial attention submodules to efficiently extract high-dimensional features of plant disease images. The proposed approach outperformed other compared methods and achieved a superior performance gain. It realized an average recall rate of 91.99% for recognizing potato disease types on the publicly accessible dataset. In practical field scenarios, the proposed approach separately attained an average accuracy and specificity of 97.33% and 98.39% on the locally collected image dataset. Experimental results present a competitive performance and demonstrate the validity and feasibility of the proposed approach.

Solar panel surface dirt detection and removal based on arduino color recognition

Color sensing is a technique for identifying physical changes in materials based on appearance assessment. Dirt deposition on solar panels can change their physical appearance and performance. Considering that dirt accumulation on solar panels needs monitoring to make efficient cleaning schedules, reduce unnecessary costs, and optimize solar panel output generation. Color sensing can achieve fast, accurate, and economical dirt detection, unlike the use of robotic cameras, mathematical formulae, and considering varying output current and voltage methods. Here, we introduce a method that detects and removes dirt on solar panels based on TCS3200 and Arduino Uno components. The approach targets (i.) Panel color measurement, calibration, threshold selection process, (ii.) comparison of color measurement values, and (iii.) align further calibration in response to discoloration of solar panels. This method aims to correct the dirt detection methods previously in use. Hence, a high-speed rolling brush arrangement is designed to improve the cleaning of the solar panel without using water. Further investigations of the panel's color may require some improvement in terms of increasing the sensitivity of the color sensor even with increased distance from the solar panel. Combining multiple color sensors may also be necessary.

Cov-Net: A computer-aided diagnosis method for recognizing COVID-19 from chest X-ray images via machine vision

In the context of global pandemic Coronavirus disease 2019 (COVID-19) that threatens life of all human beings, it is of vital importance to achieve early detection of COVID-19 among symptomatic patients. In this paper, a computer aided diagnosis (CAD) model Cov-Net is proposed for accurate recognition of COVID-19 from chest X-ray images via machine vision techniques, which mainly concentrates on powerful and robust feature learning ability. In particular, a modified residual network with asymmetric convolution and attention mechanism embedded is selected as the backbone of feature extractor, after which skip-connected dilated convolution with varying dilation rates is applied to achieve sufficient feature fusion among high-level semantic and low-level detailed information. Experimental results on two public COVID-19 radiography databases have demonstrated the practicality of proposed Cov-Net in accurate COVID-19 recognition with accuracy of 0.9966 and 0.9901, respectively. Furthermore, within same experimental conditions, proposed Cov-Net outperforms other six state-of-the-art computer vision algorithms, which validates the superiority and competitiveness of Cov-Net in building highly discriminative features from the perspective of methodology. Hence, it is deemed that proposed Cov-Net has a good generalization ability so that it can be applied to other CAD scenarios. Consequently, one can conclude that this work has both practical value in providing reliable reference to the radiologist and theoretical significance in developing methods to build robust features with strong presentation ability.

Computer vision for solid waste sorting: A critical review of academic research

Waste sorting is highly recommended for municipal solid waste (MSW) management. Increasingly, computer vision (CV), robotics, and other smart technologies are used for MSW sorting. Particularly, the field of CV-enabled waste sorting is experiencing an unprecedented explosion of academic research. However, little attention has been paid to understanding its evolvement path, status quo, and prospects and challenges ahead. To address the knowledge gap, this paper provides a critical review of academic research that focuses on CV-enabled MSW sorting. Prevalent CV algorithms, in particular their technical rationales and prediction performance, are introduced and compared. The distribution of academic research outputs is also examined from the aspects of waste sources, task objectives, application domains, and dataset accessibility. The review discovers a trend of shifting from traditional machine learning to deep learning algorithms. The robustness of CV for waste sorting is increasingly enhanced owing to the improved computation powers and algorithms. Academic studies were unevenly distributed in different sectors such as household, commerce and institution, and construction. Too often, researchers reported some preliminary studies using simplified environments and artificially collected data. Future research efforts are encouraged to consider the complexities of real-world scenarios and implement CV in industrial waste sorting practice. This paper also calls for open sharing of waste image datasets for interested researchers to train and evaluate their CV algorithms.

Adaptation capability of rainfall hotspots in water resilient cities using QGIS: a case study of Taichung City in Taiwan

In the context of extreme climate due to global climate transition, rainwater adaptation in resilient cities is a key issue for countries. The purpose of this study is to identify the rainfall hotspots in urban areas and investigate whether these hotspots have environmental conditions for rainfall adaptation. The study site is located in the Taichung area. This study collects rainfall data from rainfall stations at elevations below 500 m, employs QGIS (quantum GIS) to create an inverse distance weighted graphical distribution of rainfall to determine the hotspots where the maximum and minimum rainfalls occur, identifies the topography, green spaces, water areas, and buildings in the catchment, integrates the coverage area in the project, and estimates the amount of rainwater that could be directly absorbed by the land within the catchment. The results of this study show that, among the rainfall stations at an elevation below 100 m where most urban areas are located, the Taichung rainfall station is the area with the highest number of rainfall events from May to August. Without reliance on gully or river drainage, the natural infiltration of the land in the catchment could only adjust to 80 mm of heavy precipitation within 24 h of the rainfall warning level of the Central Weather Bureau.

Forensic bone age estimation of adolescent pelvis X-rays based on two-stage convolutional neural network

In the forensic estimation of bone age, the pelvis is important for identifying the bone age of teenagers. However, studies on this topic remain insufficient as a result of lower accuracy due to the overlapping of pelvic organs in X-ray images. Segmentation networks have been used to automate the location of key pelvic areas and minimize restrictions like doubling images of pelvic organs to increase the accuracy of estimation. This study conducted a retrospective analysis of 2164 pelvis X-ray images of Chinese Han teenagers ranging from 11 to 21 years old. Key areas of the pelvis were detected with a U-Net segmentation network, and the findings were combined with the original X-ray image for regional augmentation. Bone age estimation was conducted with the enhanced and not enhanced pelvis X-ray images by separately using three convolutional neural networks (CNNs). The root mean square errors (RMSE) of the Inception-V3, Inception-ResNet-V2, and VGG19 convolutional neural networks were 0.93 years, 1.12 years, and 1.14 years, respectively, and the mean absolute errors (MAE) of these networks were 0.67 years, 0.77 years, and 0.88 years, respectively. For comparison, a network without segmentation was employed to conduct the estimation, and it was found that the RMSE of the three CNNs above became 1.22 years, 1.25 years, and 1.63 years, respectively, and the MAE became 0.93 years, 0.96 years, and 1.23 years. Bland-Altman plots and attention maps were also generated to provide a visual comparison. The proposed segmentation network can be used to reduce the influence of restrictions like image overlapping of organs and can thus increase the accuracy of pelvic bone age estimation.

Combating data incompetence in pollen images detection and classification for pollinosis prevention

Automatic pollen images recognition is crucial for pollinosis symptoms prevention and treatment. The problem of pollen recognition can be efficiently solved using deep learning, however neural networks require tens of thousands of images to generalize. At the same time, the existing open pollen images datasets are very small. In this paper, we present a novel open pollen dataset annotated for both detection and classification tasks. Based on our dataset we study learning from a small data using different state-of-the-art approaches. For the detection task we propose to use our new Bayesian RetinaNet network, which models aleatoric uncertainty. We compare it with the baseline RetinaNet and demonstrate that our model allows for higher detection precision. For the classification task we compare the impact of pre-training on the synthetic images from generative adversarial networks (GANs) and metric-based few-shot learning. Namely, we pre-trained our small convolutional neural network and Siamese neural network classifiers on synthetic pollen images generated by two GANs: StyleGAN and Self-attention GAN. The best classifier is the convolutional neural network pre-trained on StyleGAN images. Our best models achieved 96.3% of mean average precision for the detection task and 97.7% of F1 measure for the classification task on 13 pollen plant species. We have implemented the best models in our pollen recognition web service, which is available for palynologists on request.

Waste image classification based on transfer learning and convolutional neural network

The rapid economic and social development has led to a rapid increase in the output of domestic waste. How to realize waste classification through intelligent methods has become a key factor for human beings to achieve sustainable development. Traditional waste classification technology has low efficiency and low accuracy. To improve the efficiency and accuracy of waste classification processing, this paper proposes a DenseNet169 waste image classification model based on transfer learning. Because of the disadvantages of the existing public waste dataset, such as uneven distribution of data, single background, obvious features, and small sample size of the waste image, the waste image dataset NWNU-TRASH is constructed. The dataset has the advantages of balanced distribution, high diversity, and rich background, which is more in line with real needs. 70% of the dataset is used as the training set and 30% as the test set. Based on the deep learning network DenseNet169 pre-trained model, we can form a DenseNet169 model suitable for this experimental dataset. The experimental results show that the accuracy of classification is over 82% in the DenseNet169 model after the transfer learning, which is better than other image classification algorithms.

Investigation of fiducial marker recognition possibility by water equivalent length in real-time tracking radiotherapy

Real-time tumor tracking radiotherapy (RTRT) systems typically use fiducial markers implanted near the tumor to track the target using X-ray fluoroscopy. Template pattern matching, used in tracking, is often used to automatically localize the fiducial markers. In radiotherapy of the liver, the thickness of the body that can recognize the fiducial markers must be clinically assessed. The purpose of this study was to quantify the recognition of fiducial markers according to body thickness in stereotactic body radiotherapy of the liver using clinical images obtained using SyncTraX FX4. The recognition scores of fiducial markers were examined in relation to water equivalent length (WEL), tube current, and each flat panel detector. The relationship between the contrast ratio of the fiducial marker and the background and the WEL was also investigated. The average recognition score was found to be less than 20 when the WEL was greater than 25 cm. The probability of successful tracking of image recognition was mostly smaller than 0.8 when the WEL was over 30 cm. The relationship between WEL and tube current did not significantly differ between 100 and 140 mA, but there was a significant difference (p < 0.05) for all other combinations. To ensure tracking of fiducial markers during SBRT, if the WEL representing body thickness is longer than 25 cm, the X-ray fluoroscopy arrangement should be determined based on the WEL.

Brief Report: Preferred Processing of Social Stimuli in Autism: A Perception Task

In this study we investigate whether persons with autism spectrum disorder (ASD) perceive social images differently than control participants (CON) in a graded perception task in which stimuli emerged from noise before dissipating into noise again. We presented either social stimuli (humans) or non-social stimuli (objects or animals). ASD were slower to recognize images during their emergence, but as fast as CON when indicating the dissipation of the image irrespective of its content. Social stimuli were recognized faster and remained discernable longer in both diagnostic groups. Thus, ASD participants show a largely intact preference for the processing of social images. An exploratory analysis of response subsets reveals subtle differences between groups that could be investigated in future studies.

A multi-class COVID-19 segmentation network with pyramid attention and edge loss in CT images

At the end of 2019, a novel coronavirus COVID-19 broke out. Due to its high contagiousness, more than 74 million people have been infected worldwide. Automatic segmentation of the COVID-19 lesion area in CT images is an effective auxiliary medical technology which can quantitatively diagnose and judge the severity of the disease. In this paper, a multi-class COVID-19 CT image segmentation network is proposed, which includes a pyramid attention module to extract multi-scale contextual attention information, and a residual convolution module to improve the discriminative ability of the network. A wavelet edge loss function is also proposed to extract edge features of the lesion area to improve the segmentation accuracy. For the experiment, a dataset of 4369 CT slices is constructed, including three symptoms: ground glass opacities, interstitial infiltrates, and lung consolidation. The dice similarity coefficients of three symptoms of the model achieve 0.7704, 0.7900, 0.8241 respectively. The performance of the proposed network on public dataset COVID-SemiSeg is also evaluated. The results demonstrate that this model outperforms other state-of-the-art methods and can be a powerful tool to assist in the diagnosis of positive infection cases, and promote the development of intelligent technology in the medical field.

Development of convolutional neural networks for recognition of tenogenic differentiation based on cellular morphology

BACKGROUND AND OBJECTIVE

The use of automated systems for image recognition is highly preferred for regenerative medicine applications to evaluate stem cell differentiation early in the culturing state with non-invasive methodologies instead of invasive counterparts. Bone marrow-derived mesenchymal stem cells (BMSCs) are able to differentiate into desired cell phenotypes, and thereby promise a proper cell source for tendon regeneration. The therapeutic success of stem cell therapy requires cellular characterization prior to the implantation of cells. The foremost problem is that traditional characterization techniques require cellular material which would be more useful for cell therapy, complex laboratory procedures, and human expertise. Convolutional neural networks (CNNs), a class of deep neural networks, have recently made great improvements in image-based classifications, recognition, and detection tasks. We, therefore, aim to develop a potential CNN model in order to recognize differentiated stem cells by learning features directly from image data of unlabelled cells.

METHODS

The differentiation of bone marrow mesenchymal stem cells (BMSCs) into tenocytes was induced with the treatment of bone morphogenetic protein-12 (BMP-12). Following the treatment and incubation step, the phase-contrast images of cells were obtained and immunofluorescence staining has been applied to characterize the differentiated state of BMSCs. CNN models were developed and trained with the phase-contrast cell images. The comparison of CNN models was performed with respect to prediction performance and training time. Moreover, we have evaluated the effect of image enhancement method, data augmentation, and fine-tuning training strategy to increase classification accuracy of CNN models. The best model was integrated into a mobile application.

RESULTS

All the CNN models can fit the biological data extracted from immunofluorescence characterization. CNN models enable the cell classification with satisfactory accuracies. The best result in terms of accuracy and training time is achieved by the model proposed based on Inception-ResNet V2 trained from scratch using image enhancement and data augmentation strategies (96.80%, 434.55 sec).

CONCLUSION

Our study reveals that the CNN models show good performance by identifying stem cell differentiation. Importantly this technique provides a faster and real-time tool in comparison to traditional methods enabling the adjustment of culture conditions during cultivation to improve the yield of therapeutic stem cells.

Detection of morphological changes caused by chemical stress in the cyanobacterium Planktothrix agardhii using convolutional neural networks

The presence of harmful algal bloom in many reservoirs around the world, alongside the lack of sanitation law/ordinance regarding cyanotoxin monitoring (particularly in developing countries), create a scenario in which the local population could potentially chronically consume cyanotoxin-contaminated waters. Therefore, it is crucial to develop low cost tools to detect possible systems failures and consequent toxin release inferred by morphological changes of cyanobacteria in the raw water. This paper aimed to look for the best combination of convolutional neural network (CNN), optimizer and image segmentation technique to differentiate P. agardhii trichomes before and after chemical stress caused by the addition of hydrogen peroxide. This method takes a step towards accurate monitoring of cyanobacteria in the field without the need for a mobile lab. After testing three different network architectures (AlexNet, 3ConvLayer and 2ConvLayer), four different optimizers (Adam, Adagrad, RMSProp and SDG) and five different image segmentations methods (Canny Edge Detection, Morphological Filter, HP filter, GrabCut and Watershed), the combination 2ConvLayer with Adam optimizer and GrabCut segmentation, provided the highest median accuracy (93.33%) for identifying H₂O₂-induced morphological changes in P. agardhii. Our results emphasize the fact that the trichome classification problem can be adequately tackled with a limited number of learned features due to the lack of complexity in micrographs from before and after chemical stress. To the authors' knowledge, this is the first time that CNNs were applied to detect morphological changes in cyanobacteria caused by chemical stress. Thus, it is a significant step forward in developing low cost tools based on image recognition, to shield water consumers, especially in the poorest regions, against cyanotoxin-contaminated water.

Towards using convolutional neural network to locate, identify and size defects in phased array ultrasonic testing

Machine learning algorithms are widely used in image recognition. In Phased Array Ultrasonic Testing (PAUT), images are typically formed through constructive and destructive superpositions of signals backscattered from flaws or geometric features. However, all PAUT data acquisition schemes require several emissions and the duration of the acquisition may be too slow in high-speed manufacturing. In this study, the Faster R-CNN was used to identify, locate and size flat bottom holes (FBH) and side-drilled holes (SDH) in an immersed test specimen using a single plane wave insonification. The training was performed on segmented and classified data generated using GPU-accelerated finite element simulations. SDH and FBH of different diameters, depths and lateral positions were included in the training set. The thickness of the test specimen was also variable. An ultrasonic phased array probe of 64 elements was simulated. All elements of the phased array probe were fired at the same time and the time traces from each element were recorded. The individual time traces were concatenated to form a matrix, which was then used in the training. This inspection scenario enables fast acquisition of data at the expense of poor lateral resolution in the resulting image. The trained neural network was initially tested using finite element simulations. Results were assessed in terms of the intersection of the union (IoU) between the ground truth geometry and the predicted geometry. With the simulated cases, the thickness of the test specimen was detected in all cases. When using a 40% IoU threshold, the detection rate of the FBH was 87% while only 20% for the SDH. The smallest detected FBH had a 0.56 wavelength depth and a lateral extent of 1.04 wavelength. Drawing a box using the -6dB drop method around the FBH always led to an IoU under 15%. On average, the lateral extent of the FBH using the -6dB method was three times larger than the diameter predicted by the proposed method. Then, the training was continued with a small augmented dataset of experiments (equivalent to 3% of the simulated dataset). In experiments, the results show that the test specimen was always correctly identified. When using a 40% IoU threshold the experimental detection rate of the FBH was 70%. The smallest detected defect in experiments had a depth of 2 wavelengths.

Image-based many-language programming language identification

Programming language identification (PLI) is a common need in automatic program comprehension as well as a prerequisite for deeper forms of code understanding. Image-based approaches to PLI have recently emerged and are appealing due to their applicability to code screenshots and programming video tutorials. However, they remain limited to the recognition of a small amount of programming languages (up to 10 languages in the literature). We show that it is possible to perform image-based PLI on a large number of programming languages (up to 149 in our experiments) with high (92%) precision and recall, using convolutional neural networks (CNNs) and transfer learning, starting from readily-available pretrained CNNs. Results were obtained on a large real-world dataset of 300,000 code snippets extracted from popular GitHub repositories. By scrambling specific character classes and comparing identification performances we also show that the characters that contribute the most to the visual recognizability of programming languages are symbols (e.g., punctuation, mathematical operators and parentheses), followed by alphabetic characters, with digits and indentation having a negligible impact.

Detecting anomalies in X-ray diffraction images using convolutional neural networks

Our understanding of life is based upon the interpretation of macromolecular structures and their dynamics. Almost 90% of currently known macromolecular models originated from electron density maps constructed using X-ray diffraction images. Even though diffraction images are critical for structure determination, due to their vast amounts and noisy, non-intuitive nature, their quality is rarely inspected. In this paper, we use recent advances in machine learning to automatically detect seven types of anomalies in X-ray diffraction images. For this purpose, we utilize a novel X-ray beam center detection algorithm, propose three different image representations, and compare the predictive performance of general-purpose classifiers and deep convolutional neural networks (CNNs). In benchmark tests on a set of 6,311 X-ray diffraction images, the proposed CNN achieved between 87% and 99% accuracy depending on the type of anomaly. Experimental results show that the proposed anomaly detection system can be considered suitable for early detection of sub-optimal data collection conditions and malfunctions at X-ray experimental stations.

Research on the process of small sample non-ferrous metal recognition and separation based on deep learning

Consumption of copper and aluminum has increased significantly in recent years; therefore, recycling these elements from the end-of-life vehicles (ELVs) will be of great economic value and social benefit. However, the separation of non-ferrous materials is difficult because of their different sources, various shapes and sizes, and complex surface conditions. In experimental study on the separation of these materials, few non-ferrous metal scraps can be used. To address these limitations, a traditional image recognition model and a small sample multi-target detection model (which can detect multiple targets simultaneously) based on deep learning and transfer learning were used to identify non-ferrous materials. The improved third version of You Only Look Once (YOLOv3) multi-target detection model using data augmentation, the loss function of focal loss, and a method of adjusting the threshold of Intersection over Union (IOU) between candidate bound and ground truth bound has superior target detection performance than methods. We obtained a 95.3% and 91.4% accuracy in identifying aluminum and copper scraps, respectively, and an operation speed of 18 FPS, meeting the real-time requirements of a sorting system. By using the improved YOLOv3 multi-target detection algorithm and equipment operation parameters selected, the accuracy and purity of the separation system exceeded 90%, meeting the needs of actual production.

Computer vision-based wood identification and its expansion and contribution potentials in wood science: A review

The remarkable developments in computer vision and machine learning have changed the methodologies of many scientific disciplines. They have also created a new research field in wood science called computer vision-based wood identification, which is making steady progress towards the goal of building automated wood identification systems to meet the needs of the wood industry and market. Nevertheless, computer vision-based wood identification is still only a small area in wood science and is still unfamiliar to many wood anatomists. To familiarize wood scientists with the artificial intelligence-assisted wood anatomy and engineering methods, we have reviewed the published mainstream studies that used or developed machine learning procedures. This review could help researchers understand computer vision and machine learning techniques for wood identification and choose appropriate techniques or strategies for their study objectives in wood science.

Prescreening and treatment of aortic dissection through an analysis of infinite-dimension data

Background

Aortic dissection (AD) is one of the most catastrophic aortic diseases associated with a high mortality rate. In contrast to the advances in most cardiovascular diseases, both the incidence and in-hospital mortality rate of AD have experienced deviant increases over the past 20 years, highlighting the need for fresh prospects on the prescreening and in-hospital treatment strategies.

Methods

Through two cross-sectional studies, we adopt image recognition techniques to identify pre-disease aortic morphology for prior diagnoses; assuming that AD has occurred, we employ functional data analysis to determine the optimal timing for BP and HR interventions to offer the highest possible survival rate.

Results

Compared with the healthy control group, the aortic centerline is significantly more slumped for the AD group. Further, controlling patients’ blood pressure and heart rate according to the likelihood of adverse events can offer the highest possible survival probability.

Conclusions

The degree of slumpness is introduced to depict aortic morphological changes comprehensively. The morphology-based prediction model is associated with an improvement in the predictive accuracy of the prescreening of AD. The dynamic model reveals that blood pressure and heart rate variations have a strong predictive power for adverse events, confirming this model’s ability to improve AD management.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13040-021-00249-8).

Cross Knowledge-based Generative Zero-Shot Learning approach with Taxonomy Regularization

Although zero-shot learning (ZSL) has an inferential capability of recognizing new classes that have never been seen before, it always faces two fundamental challenges of the cross modality and cross-domain challenges. In order to alleviate these problems, we develop a generative network-based ZSL approach equipped with the proposed Cross Knowledge Learning (CKL) scheme and Taxonomy Regularization (TR). In our approach, the semantic features are taken as inputs, and the output is the synthesized visual features generated from the corresponding semantic features. CKL enables more relevant semantic features to be trained for semantic-to-visual feature embedding in ZSL, while Taxonomy Regularization (TR) significantly improves the intersections with unseen images with more generalized visual features generated from generative network. Extensive experiments on several benchmark datasets (i.e., AwA1, AwA2, CUB, NAB and aPY) show that our approach is superior to these state-of-the-art methods in terms of ZSL image classification and retrieval.

New solutions for automated image recognition and identification: challenges to radiologic technology and forensic pathology

This paper outlines the history of biometrics for personal identification, the current status of the initial biological fingerprint techniques for digital chest radiography, and patient verification during medical imaging, such as computed tomography and magnetic resonance imaging. Automated image recognition and identification developed for clinical images without metadata could also be applied to the identification of victims in mass disasters or other unidentified individuals. The development of methods that are adaptive to a wide range of recent imaging modalities in the fields of radiologic technology, patient safety, forensic pathology, and forensic odontology is still in its early stages. However, its importance in practice will continue to increase in the future.

New fractional-order shifted Gegenbauer moments for image analysis and recognition

Orthogonal moments are used to represent digital images with minimum redundancy. Orthogonal moments with fractional-orders show better capabilities in digital image analysis than integer-order moments. In this work, the authors present new fractional-order shifted Gegenbauer polynomials. These new polynomials are used to define a novel set of orthogonal fractional-order shifted Gegenbauer moments (FrSGMs). The proposed method is applied in gray-scale image analysis and recognition. The invariances to rotation, scaling and translation (RST), are achieved using invariant fractional-order geometric moments. Experiments are conducted to evaluate the proposed FrSGMs and compare with the classical orthogonal integer-order Gegenbauer moments (GMs) and the existing orthogonal fractional-order moments. The new FrSGMs outperformed GMs and the existing orthogonal fractional-order moments in terms of image recognition and reconstruction, RST invariance, and robustness to noise.

Emotional sounds of crowds: spectrogram-based analysis using deep learning

Crowds express emotions as a collective individual, which is evident from the sounds that a crowd produces in particular events, e.g., collective booing, laughing or cheering in sports matches, movies, theaters, concerts, political demonstrations, and riots. A critical question concerning the innovative concept of crowd emotions is whether the emotional content of crowd sounds can be characterized by frequency-amplitude features, using analysis techniques similar to those applied on individual voices, where deep learning classification is applied to spectrogram images derived by sound transformations. In this work, we present a technique based on the generation of sound spectrograms from fragments of fixed length, extracted from original audio clips recorded in high-attendance events, where the crowd acts as a collective individual. Transfer learning techniques are used on a convolutional neural network, pre-trained on low-level features using the well-known ImageNet extensive dataset of visual knowledge. The original sound clips are filtered and normalized in amplitude for a correct spectrogram generation, on which we fine-tune the domain-specific features. Experiments held on the finally trained Convolutional Neural Network show promising performances of the proposed model to classify the emotions of the crowd.