Machine vision-based non-destructive dissolution prediction of meloxicam-containing tablets

Machine vision systems have emerged for quality assessment of solid dosage forms in the pharmaceutical industry. These can offer a versatile tool for continuous manufacturing while supporting the framework of process analytical technology, quality-by-design, and real-time release testing. The aim of this work is to develop a digital UV/VIS imaging-based system for predicting the in vitro dissolution of meloxicam-containing tablets. The alteration of the dissolution profiles of the samples required different levels of the critical process parameters, including compression force, particle size and content of the API. These process parameters were predicted non-destructively by multivariate analysis of UV/VIS images taken from the tablets. The dissolution profile prediction was also executed using solely the image data and applying artificial neural networks. The prediction error (RMSE) of the dissolution profile points was less than 5%. The alteration of the API content directly affected the maximum concentrations observed at the end of the dissolution tests. This parameter was predicted with a relative error of less than 10% by PLS models that are based on the color components of UV and VIS images. In conclusion, this paper presents a modern, non-destructive PAT solution for real-time testing of the dissolution of tablets.

UV-VIS imaging-based investigation of API concentration fluctuation caused by the sticking behaviour of pharmaceutical powder blends

Surface powder sticking in pharmaceutical mixing vessels poses a risk to the uniformity and quality of drug formulations. This study explores methods for evaluating the amount of pharmaceutical powder mixtures adhering to the metallic surfaces. Binary powder blends consisting of amlodipine and microcrystalline cellulose (MCC) were used to investigate the effect of the mixing order on the adherence to the vessel wall. Elevated API concentrations were measured on the wall and within the dislodged material from the surface, regardless of the mixing order of the components. UV imaging was used to determine the particle size and the distribution of the API on the metallic surface. The results were compared to chemical maps obtained by Raman chemical imaging. The combination of UV and VIS imaging enabled the rapid acquisition of chemical maps, covering a substantially large area representative of the analysed sample. UV imaging was also applied in tablet inspection to detect tablets that fail to meet the content uniformity criteria. The results present powder adherence as a possible source of poor content uniformity, highlighting the need for 100% inspection of pharmaceutical products to ensure product quality and safety.

Enhancing bighead carp cutting: Chilled storage insights and machine vision-based segmentation algorithm development

To enhance market demand and fish utilization, cutting processing is essential for fish. Bighead carp were cut into four primary cuts: head, dorsal, belly, and tail, collectively accounting for 77.03% of the fish's total weight. These cuts were refrigerated at 4 °C for 10 days, during which the muscle from each cut was analyzed. Pseudomonas.fragi proliferated most rapidly and was most abundant in eye muscle (EM), while Aeromonas.sobria showed similar growth patterns in tail muscle (TM). Notably, EM exhibited the highest rate of fat oxidation. TM experienced the most rapid protein degradation. Furthermore, to facilitate the cutting applied in mechanical processing, a machine vision-based algorithm was developed. This algorithm utilized color threshold and morphological parameters to segment image background and divide bighead carp region. Consequently, each cut of bighead carp had a different storage quality and the machine vision-based algorithm proved effective for processing bighead carp.

A new AI-based approach for automatic identification of tea leaf disease using deep neural network based on hybrid pooling

The degree of production efficiency and the quality of the commodities produced may both be directly impacted by the presence of illnesses in tea leaves. These days, this procedure may be automated with the use of artificial intelligence tools, and a number of approaches have been put out to satisfy these needs. Nonetheless, current research efforts have focused on improving diagnosis accuracy and expanding the variety of illnesses that might affect tea leaves. In this article, a new method is proposed for accurately diagnosing tea leaf diseases using artificial intelligence techniques. In the proposed method, the input images are preprocessed to remove redundant information. Then, a hybrid pooling-based Convolutional Neural Network (CNN) is employed to extract image features. In this method, the pooling layers of the CNN model are randomly adjusted based on either max pooling or average pooling functions. This strategy can enhance the efficiency of the CNN-based feature extraction model. In this method, the pooling layers of the CNN model are randomly adjusted based on either max pooling or average pooling functions. This strategy can enhance the efficiency of the CNN-based feature extraction model. After feature extraction, a weighted Random Forest (WRF) model is used for the detection of tea leaf diseases. The outputs of the decision tree models and their corresponding weights are used to identify tea leaf illnesses in this classification model, where each tree in the random forest is given a weight depending on how well it performs. The Cuckoo Search Optimization (CSO) method is used in the proposed classification model to give a weight to each tree. Tea Sickness Dataset (TSD) has been used as the basis for evaluating the suggested method's effectiveness. The findings show that the suggested approach has an average accuracy of 92.47% in identifying seven different forms of tea leaf illnesses. Additionally, the recall and accuracy metrics indicate results of 92.35 and 92.26, respectively, indicating improvements over earlier techniques.

Applying artificial intelligence to determination of legal age of majority from radiographic data

Forensic odontologists use biological patterns to estimate chronological age for the judicial system. The age of majority is a legally significant period with a limited set of reliable oral landmarks. Currently, experts rely on the questionable development of third molars to assess whether litigants can be prosecuted as legal adults. Identification of new and novel patterns may illuminate features more dependably indicative of chronological age, which have, until now, remained unseen. Unfortunately, biased perceptions and limited cognitive capacity compromise the ability of researchers to notice new patterns. The present study demonstrates how artificial intelligence can break through identification barriers and generate new estimation modalities. A convolutional neural network was trained with 4003 panoramic-radiographs to sort subjects into 'under-18' and 'over-18' age categories. The resultant architecture identified legal adults with a high predictive accuracy equally balanced between precision, specificity and recall. Moving forward, AI-based methods could improve courtroom efficiency, stand as automated assessment methods and contribute to our understanding of biological ageing.

A machine vision method for the evaluation of ship-to-ship collision risk

The development of ship technology and information technology has been driving the continuous improvement of ship intelligence, with safety being an inevitable requirement in the shipping industry. A machine vision-based ship collision warning method is proposed for high monitoring system cost and limited information acquisition in safety design of autonomous ship navigation. The method combines machine learning with image algorithms. Firstly, the backbone of YOLOv7 detector is replaced by EfficientFormerV2 network to achieve model lightweight while ensuring detection accuracy. Public datasets SeaShips, Flow and self-made ship pictures are combined, and the network is trained on this dataset. StrongSORT is used for target tracking. Secondly, a data fusion algorithm is introduced to determine the target point at the bow-bottom of the ship based on the time-varying attitude of the camera and the time-series features of the bounding boxes. Ship navigation trajectory estimation is performed using image algorithms. Finally, a collision evaluation model is established to calculate the collision risk index. Experimental results demonstrate that the improved YOLOv7 network maintains similar mAP.5 and Recall compared to the original model, while reducing the parameters by 31.2 % and GFLOPs by 58.4 %. The accuracy of target ship trajectory estimation is high, with MAE values below 1.5 % and RMSE values below 2 % in experiments. In ship collision warning experiments, the proposed method accurately identifies navigating vessels, estimates the trajectories, and provides timely warnings for imminent collision accidents. Compared to traditional ship collision warning methods, this paper offers a more intelligent and lightweight solution.

Image-based simultaneous particle size distribution and concentration measurement of powder blend components with deep learning and machine vision

This work presents a system, where deep learning was used on images captured with a digital camera to simultaneously determine the API concentration and the particle size distribution (PSD) of two components of a powder blend. The blend consisted of acetylsalicylic acid (ASA) and calcium hydrogen phosphate (CHP), and the predicted API concentration was found corresponding with the HPLC measurements. The PSDs determined with the method corresponded with those measured with laser diffraction particle size analysis. This novel method provides fast and simple measurements and could be suitable for detecting segregation in the powder. By examining the powders discharged from a batch blender, the API concentrations at the top and bottom of the container could be measured, yielding information about the adequacy of the blending and improving the quality control of the manufacturing process.

In-line particle size measurement during granule fluidization using convolutional neural network-aided process imaging

This paper presents a machine learning-based image analysis method to monitor the particle size distribution of fluidized granules. The key components of the direct imaging system are a rigid fiber-optic endoscope, a light source and a high-speed camera, which allow for real-time monitoring of the granules. The system was implemented into a custom-made 3D-printed device that could reproduce the particle movement characteristic in a fluidized-bed granulator. The suitability of the method was evaluated by determining the particle size distribution (PSD) of various granule mixtures within the 100-2000 μm size range. The convolutional neural network-based software was able to successfully detect the granules that were in focus despite the dense flow of the particles. The volumetric PSDs were compared with off-line reference measurements obtained by dynamic image analysis and laser diffraction. Similar trends were observed across the PSDs acquired with all three methods. The results of this study demonstrate the feasibility of performing real-time particle size analysis using machine vision as an in-line process analytical technology (PAT) tool.

Application of IoT technology based on neural networks in basketball training motion capture and injury prevention

Basketball players need to frequently engage in various physical movements during the game, which puts a certain burden on their bodies and can easily lead to various sports injuries. Therefore, it is crucial to prevent sports injuries in basketball teaching. This paper also studies basketball motion track capture. Basketball motion capture preserves the motion posture information of the target person in three-dimensional space. Because the motion capture system based on machine vision often encounters problems such as occlusion or self occlusion in the application scene, human motion capture is still a challenging problem in the current research field. This article designs a multi perspective human motion trajectory capture algorithm framework, which uses a two-dimensional human motion pose estimation algorithm based on deep learning to estimate the position distribution of human joint points on the two-dimensional image from each perspective. By combining the knowledge of camera poses from multiple perspectives, the three-dimensional spatial distribution of joint points is transformed, and the final evaluation result of the target human 3D pose is obtained. This article applies the research results of neural networks and IoT devices to basketball motion capture methods, further developing basketball motion capture systems.

A comprehensive survey of intestine histopathological image analysis using machine vision approaches

Colorectal Cancer (CRC) is currently one of the most common and deadly cancers. CRC is the third most common malignancy and the fourth leading cause of cancer death worldwide. It ranks as the second most frequent cause of cancer-related deaths in the United States and other developed countries. Histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of CRC. In order to improve the objectivity and diagnostic efficiency for image analysis of intestinal histopathology, Computer-aided Diagnosis (CAD) methods based on machine learning (ML) are widely applied in image analysis of intestinal histopathology. In this investigation, we conduct a comprehensive study on recent ML-based methods for image analysis of intestinal histopathology. First, we discuss commonly used datasets from basic research studies with knowledge of intestinal histopathology relevant to medicine. Second, we introduce traditional ML methods commonly used in intestinal histopathology, as well as deep learning (DL) methods. Then, we provide a comprehensive review of the recent developments in ML methods for segmentation, classification, detection, and recognition, among others, for histopathological images of the intestine. Finally, the existing methods have been studied, and the application prospects of these methods in this field are given.

Identifying the quality characteristics of pork floss structure based on deep learning framework

Pork floss is a traditional Chinese food with a long history. Nowadays, pork floss is known to consumers as a leisure food. It is made from pork through a unique process in which the muscle fibers become flaky or granular and tangled. In this study, a deep learning-based approach is proposed to detect the quality characteristics of pork floss structure. Describe that the experiments were conducted using widely recognized brands of pork floss available in the grocery market, omitting the use of abbreviations. A total of 8000 images of eight commercially available pork flosses were collected and processed using sharpening, image gray coloring, real-time shading correction, and binarization. After the machine learning model learned the features of the pork floss, the images were labeled using a manual mask. The coupling of residual enhancement mask and region-based convolutional neural network (CRE-MRCNN) based deep learning framework was used to segment the images. The results showed that CRE-MRCNN could be used to identify the knot features and pore features of different brands of pork floss to evaluate their quality. The combined results of the models based on the sensory tests and machine vision showed that the pork floss from TC was the best, followed by YJJ, DD and HQ. This also shows the potential of machine vision to help people recognize the quality characteristics of pork floss structure.

The use of RGB-tracking of color changes during indigo-reduction processes based on LabVIEW machine vision

The use of an RGB-tracking chart for monitoring the reduction of indigo (color changes) based on the LabVIEW machine vision is demonstrated for the first time. In contrast to a normal analytical chromatographic chart, the time scale is used on the X-axis, but the sum of "RGB-pixels" is used on the Y-axis, instead of "signal intensity". The RGB-tracking chart was obtained from an investigation of the process involved in the reduction of indigo, in which a PC camera was used as a detector and LabVIEW machine vision was simultaneously operated. As a result, when sodium dithionite (Na2S2O4) and yeast were used, respectively, during the indigo-reduction processes, two types of reduction processes were found; the optimized timing for dyeing can be easily determined from the RGB-tracking charts. Furthermore, based on the changes in HSV (hue, saturation, lightness), the use of sodium dithionite provides a higher number of hue and saturation when clothes & fabric were dyed. In contrast to this, a longer time was required for the yeast solution to reach the same high number for hue and saturation. After comparing several series of dyed fabrics, we found that the use of an RGB-tracking chart is indeed a reliable novel tool for measuring color changes that occur during the chemical reactions that are associated with this process.

Garbage detection and classification using a new deep learning-based machine vision system as a tool for sustainable waste recycling

Waste recycling is a critical issue for environment pollution management while garbage classification determines the recycling efficiency. In order to reduce labor costs and increase garbage classification capacity, a machine vision system is established based on the deep learning and transfer learning. In this new method, an improved MobileNetV2 deep learning model is proposed for garbage detection and classification, where the attention mechanism is introduced into the first and last convolution layers of the MobileNetV2 model to improve the recognition accuracy and the transfer learning uses a set of pre-trained weight parameters to extend the model generalization ability. In addition, the principal component analysis (PCA) is employed to reduce the dimension of the last fully connected layer to enable real-time operation of the developed model on an edge device. The experimental results demonstrate that the proposed method generates 90.7 % of the garbage classification accuracy on the "Huawei Cloud" datasets, the average inference time is 600 ms on the raspberry Pi 4B microprocessor, and the model volume compression is 30.1 % of the basic MobileNetV2 model. Furthermore, a garbage sorting porotype is designed and manufactured to evaluate the performance of the proposed MobileNetV2 model on the real-world garbage identification, which turns out that the average garbage classification accuracy is 89.26 %. Hence, the developed garbage sorting porotype can be used a effective tool for sustainable waste recycling.

Intraoperative Navigation and Robotics in Pediatric Spinal Deformity

Current technologies for image guidance navigation and robotic assistance with spinal surgery are improving rapidly with several systems commercially available. Newer machine vision technology has several potential advantages. Limited studies have shown similar outcomes to traditional navigation platforms with decreased intraoperative radiation and time required for registration. However, there are no active robotic arms that can be coupled with machine vision navigation. Further research is necessary to justify the cost, potential increased operative time, and workflow issues but the use of navigation and robotics will only continue to expand given the growing body of evidence supporting their use.

Fruit-CoV: An efficient vision-based framework for speedy detection and diagnosis of SARS-CoV-2 infections through recorded cough sounds

COVID-19 is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This deadly virus has spread worldwide, leading to a global pandemic since March 2020. A recent variant of SARS-CoV-2 named Delta is intractably contagious and responsible for more than four million deaths globally. Therefore, developing an efficient self-testing service for SARS-CoV-2 at home is vital. In this study, a two-stage vision-based framework, namely Fruit-CoV, is introduced for detecting SARS-CoV-2 infections through recorded cough sounds. Specifically, audio signals are converted into Log-Mel spectrograms, and the EfficientNet-V2 network is used to extract their visual features in the first stage. In the second stage, 14 convolutional layers extracted from the large-scale Pretrained Audio Neural Networks for audio pattern recognition (PANNs) and the Wavegram-Log-Mel-CNN are employed to aggregate feature representations of the Log-Mel spectrograms and the waveform. Finally, the combined features are used to train a binary classifier. In this study, a dataset provided by the AICovidVN 115M Challenge is employed for evaluation. It includes 7,371 recorded cough sounds collected throughout Vietnam, India, and Switzerland. Experimental results indicate that the proposed model achieves an Area Under the Receiver Operating Characteristic Curve (AUC) score of 92.8% and ranks first on the final leaderboard of the AICovidVN 115M Challenge. Our code is publicly available.

Towards an automated approach for smart sterility test examination

As new technologies emerge, deep learning applications are often integral parts of new products as features and often as differentiating benefits. This is especially notable in commercial consumer products in everyday applications, such as voice assistants or streaming content recommendation systems. Due to the power and applicability of these deep learning technologies significant efforts are being directed to the development and integration of appropriate models into science and engineering applications to supplant analogue systems that may be highly prone to human error. Here we present an innovative, low-cost approach to advance sterility assessment workflows that are required and regulated within drug release/manufacturing processes. The model system leverages off-the-shelf hardware as well as deep learning models to detect and classify different microbial contaminations in test containers. The paired hardware and software tools were evaluated in experiments using common model organisms (C. sporogenes, P. aeruginosa, S. aureus). With this approach we were able to detect all three test organisms across 40 experiments, furthermore we were capable of classifying the present organisms with an average classification accuracy of over 87%.

Cov-Net: A computer-aided diagnosis method for recognizing COVID-19 from chest X-ray images via machine vision

In the context of global pandemic Coronavirus disease 2019 (COVID-19) that threatens life of all human beings, it is of vital importance to achieve early detection of COVID-19 among symptomatic patients. In this paper, a computer aided diagnosis (CAD) model Cov-Net is proposed for accurate recognition of COVID-19 from chest X-ray images via machine vision techniques, which mainly concentrates on powerful and robust feature learning ability. In particular, a modified residual network with asymmetric convolution and attention mechanism embedded is selected as the backbone of feature extractor, after which skip-connected dilated convolution with varying dilation rates is applied to achieve sufficient feature fusion among high-level semantic and low-level detailed information. Experimental results on two public COVID-19 radiography databases have demonstrated the practicality of proposed Cov-Net in accurate COVID-19 recognition with accuracy of 0.9966 and 0.9901, respectively. Furthermore, within same experimental conditions, proposed Cov-Net outperforms other six state-of-the-art computer vision algorithms, which validates the superiority and competitiveness of Cov-Net in building highly discriminative features from the perspective of methodology. Hence, it is deemed that proposed Cov-Net has a good generalization ability so that it can be applied to other CAD scenarios. Consequently, one can conclude that this work has both practical value in providing reliable reference to the radiologist and theoretical significance in developing methods to build robust features with strong presentation ability.

Current advances in detection of abnormal egg: a review

Internal and external defects of eggs should be detected to prevent cross-contamination of intact eggs by abnormal eggs during storage. Emerging detection technologies for abnormal eggs were introduced as an alternative to human inspection. The advanced technologies could rapidly detect abnormal eggs. Abnormal egg detection technologies using acoustic response, machine vision, and spectroscopy have been commercialized in the poultry industry. Non-destructive egg quality assessment methods meanwhile could preserve the value of eggs and improve detection efficiency. In order to improve detection efficiency, it is essential to select a proper algorithm for classifying the types of abnormal eggs. This review deals with the performance of the detection technologies for various types of abnormal eggs in recently published resources. In addition, the discriminant methods and detection algorithms of abnormal eggs reported in the published literature were investigated. Although the majority of the studies were conducted on a laboratory scale, the developed detection technologies for internal and external defects in eggs were technically feasible to obtain the excellent detection accuracy. To apply the developed detection technologies to the poultry industry, it is necessary to achieve the detection rates required from the industry.

Pilot study of a new type of machine vision-assisted stereotactic neurosurgery for EVD placement

BACKGROUND

The usage of machine vision technologies for image-based analysis and inspection is increasing. With the advent of the ability to process high-dimension data instantly, the possibilities of machine vision multiply exponentially. Robots now use this technology to assist in surgery.

OBJECTIVE

The aim of this study is to explore the efficacy of Surgical Navigation Robot NaoTrac (Brain Navi Biotechnology Co., Ltd.), which utilizes machine vision-inspired technology for patient registration and stereotactic external ventricular drainage (EVD) by the robotic arm.

METHODS

Preoperative and postoperative computed tomography (CT) scans were acquired for each case. The surgeons planned the targets and trajectories with the preoperative CT images. The postoperative CT images were utilized in the accuracy measurements.

RESULTS

All 14 cases had cerebrospinal fluid drained through the catheter. The NaoTrac placed the catheter into the frontal horn in one attempt in 13 cases and was able to drain CSF in 12 cases. Not a single case had any bleeding or intraoperative complications. The average time spent on the patient registration was 142.8 s. The mean target deviation was 1.68 mm, and the mean angular deviation was 1.99°, all within the accepted tolerance for minimal tissue damage.

CONCLUSION

The results of this report demonstrate that machine vision-inspired patient registration is feasible and fast. NaoTrac has demonstrated its accuracy and safety in performing frameless catheter placement in 13 clinical cases. Other stereotactic neurosurgical operations such as stereotactic biopsy, depth electrode placement, deep brain stimulation electrode positioning, and neuroendoscopy may also be benefited from the assistance of NaoTrac.

Multistage adaptive control strategy based on image contour data for autonomous endoscope navigation

The physician burnout, poor ergonomics are hardly conducive to the sustainability and high quality of colonoscopy. In order to reduce doctors' workload and improve patients' experiences during colonoscopy, this paper proposes a multistage adaptive control approach based on image contour data to guide the autonomous navigation of endoscopes. First, a fast image preprocessing and contour extraction algorithms are designed. Second, different processing algorithms are developed according to the different contour information that can be clearly extracted to compute the endoscope control parameters. Third, when a clear contour cannot be extracted, a triple control method inspired by the turning of a novice car driver is devised to help the endoscope capture clear contours. The proposed multistage adaptive control approach is tested in an intestinal model over a variety of curved configurations and verified on the actual colonoscopy image. The results reveal the success of the strategy in both straight sections of this intestinal model and in tightly curved sections as small as 6 cm in radius of curvature. In the experiment, processing time for a single image is 20-25 ms and the accuracy of judging steering based on intestinal model pictures is 96.7%. Additionally, the average velocity reaches 3.04 cm/s in straight sections and 2.49 cm/s in curved sections respectively.

Inter-row navigation line detection for cotton with broken rows

BACKGROUND

The application of autopilot technology is conductive to achieving path planning navigation and liberating labor productivity. In addition, the self-driving vehicles can drive according to the growth state of crops to ensure the accuracy of spraying and pesticide effect. Navigation line detection is the core technology of self-driving technology, which plays a more important role in the development of Chinese intelligent agriculture. The general algorithms for seedling line extraction in the agricultural fields are for large seedling crops. At present, scholars focus more on how to reduce the impact of crop row adhesion on extraction of crop rows. However, for seedling crops, especially double-row sown seedling crops, the navigation lines cannot be extracted very effectively due to the lack of plants or the interference of rut marks caused by wheel pressure on seedlings. To solve these problems, this paper proposed an algorithm that combined edge detection and OTSU to determine the seedling column contours of two narrow rows for cotton crops sown in wide and narrow rows. Furthermore, the least squares were used to fit the navigation line where the gap between two narrow rows of cotton was located, which could be well adapted to missing seedlings and rutted print interference.

RESULTS

The algorithm was developed using images of cotton at the seedling stage. Apart from that, the accuracy of route detection was tested under different lighting conditions and in maize and soybean at the seedling stage. According to the research results, the accuracy of the line of sight for seedling cotton was 99.2%, with an average processing time of 6.63 ms per frame; the accuracy of the line of sight for seedling corn was 98.1%, with an average processing time of 6.97 ms per frame; the accuracy of the line of sight for seedling soybean was 98.4%, with an average processing time of 6.72 ms per frame. In addition, the standard deviation of lateral deviation is 2 cm, and the standard deviation of heading deviation is 0.57 deg.

CONCLUSION

The proposed rows detection algorithm could achieve state-of-the-art performance. Besides, this method could ensure the normal spraying speed by adapting to different shadow interference and the randomness of crop row growth. In terms of the applications, it could be used as a reference for the navigation line fitting of other growing crops in complex environments disturbed by shadow.

A review: The detection of cancer cells in histopathology based on machine vision

Machine vision is being employed in defect detection, size measurement, pattern recognition, image fusion, target tracking and 3D reconstruction. Traditional cancer detection methods are dominated by manual detection, which wastes time and manpower, and heavily relies on the pathologists' skill and work experience. Therefore, these manual detection approaches are not convenient for the inheritance of domain knowledge, and are not suitable for the rapid development of medical care in the future. The emergence of machine vision can iteratively update and learn the domain knowledge of cancer cell pathology detection to achieve automated, high-precision, and consistent detection. Consequently, this paper reviews the use of machine vision to detect cancer cells in histopathology images, as well as the benefits and drawbacks of various detection approaches. First, we review the application of image preprocessing and image segmentation in histopathology for the detection of cancer cells, and compare the benefits and drawbacks of different algorithms. Secondly, for the characteristics of histopathological cancer cell images, the research progress of shape, color and texture features and other methods is mainly reviewed. Furthermore, for the classification methods of histopathological cancer cell images, the benefits and drawbacks of traditional machine vision approaches and deep learning methods are compared and analyzed. Finally, the above research is discussed and forecasted, with the expected future development tendency serving as a guide for future research.

Real-time coating thickness measurement and defect recognition of film coated tablets with machine vision and deep learning

This paper presents a system, where images acquired with a digital camera are coupled with image analysis and deep learning to identify and categorize film coating defects and to measure the film coating thickness of tablets. There were 5 different classes of defective tablets, and the YOLOv5 algorithm was utilized to recognize defects, the accuracy of the classification was 98.2%. In order to characterize coating thickness, the diameter of the tablets in pixels was measured, which was used to measure the coating thickness of the tablets. The proposed system can be easily scaled up to match the production capability of continuous film coaters. With the developed technique, the complete screening of the produced tablets can be achieved in real-time resulting in the improvement of quality control.

Improved wood species identification based on multi-view imagery of the three anatomical planes

BACKGROUND

The identification of tropical African wood species based on microscopic imagery is a challenging problem due to the heterogeneous nature of the composition of wood combined with the vast number of candidate species. Image classification methods that rely on machine learning can facilitate this identification, provided that sufficient training material is available. Despite the fact that the three main anatomical sections contain information that is relevant for species identification, current methods only rely on transverse sections. Additionally, commonly used procedures for evaluating the performance of these methods neglect the fact that multiple images often originate from the same tree, leading to an overly optimistic estimate of the performance.

RESULTS

We introduce a new image dataset containing microscopic images of the three main anatomical sections of 77 Congolese wood species. A dedicated multi-view image classification method is developed and obtains an accuracy (computed using the naive but common approach) of 95%, outperforming the single-view methods by a large margin. An in-depth analysis shows that naive accuracy estimates can lead to a dramatic over-prediction, of up to 60%, of the accuracy.

CONCLUSIONS

Additional images from non-transverse sections can boost the performance of machine-learning-based wood species identification methods. Additionally, care should be taken when evaluating the performance of machine-learning-based wood species identification methods to avoid an overestimation of the performance.

Experimental data manipulations to assess performance of hyperspectral classification models of crop seeds and other objects

BACKGROUND

Optical sensing solutions are being developed and adopted to classify a wide range of biological objects, including crop seeds. Performance assessment of optical classification models remains both a priority and a challenge.

METHODS

As training data, we acquired hyperspectral imaging data from 3646 individual tomato seeds (germination yes/no) from two tomato varieties. We performed three experimental data manipulations: (1) Object assignment error: effect of individual object in the training data being assigned to the wrong class. (2) Spectral repeatability: effect of introducing known ranges (0-10%) of stochastic noise to individual reflectance values. (3) Size of training data set: effect of reducing numbers of observations in training data. Effects of each of these experimental data manipulations were characterized and quantified based on classifications with two functions [linear discriminant analysis (LDA) and support vector machine (SVM)].

RESULTS

For both classification functions, accuracy decreased linearly in response to introduction of object assignment error and to experimental reduction of spectral repeatability. We also demonstrated that experimental reduction of training data by 20% had negligible effect on classification accuracy. LDA and SVM classification algorithms were applied to independent validation seed samples. LDA-based classifications predicted seed germination with RMSE = 10.56 (variety 1) and 26.15 (variety 2), and SVM-based classifications predicted seed germination with RMSE = 10.44 (variety 1) and 12.58 (variety 2).

CONCLUSION

We believe this study represents the first, in which optical seed classification included both a thorough performance evaluation of two separate classification functions based on experimental data manipulations, and application of classification models to validation seed samples not included in training data. Proposed experimental data manipulations are discussed in broader contexts and general relevance, and they are suggested as methods for in-depth performance assessments of optical classification models.

A similarity-guided segmentation model for garbage detection under road scene

The development of computer vision technology provides a possible path for realizing intelligent control of road sweepers to reduce energy waste in urban street cleaning work. For garbage segmentation of seven categories under road scene, we introduce an efficient deep-learning-based method. Our model follows a lightweight structure with a feature pyramid attention (FPA) module employed in the decoder to enhance feature integration at multi-levels. Besides, a similarity guidance (SG) module is added to the decoder branches, which calculates the cosine distance between learned prototypes and feature maps to guide the segmentation results from a metric learning perspective. Our model has less than 3 M parameters and can run at over 65 FPS in an RTX 2070 GPU. Experimental results demonstrate that our method can yield competitive results in terms of speed and accuracy trade-off, with overall mean intersection-over-union (mIoU) reaching 0.87 and 0.67, respectively, on two garbage data sets we built. Besides, our model can perform acceptable category-balanced segmentation from less than 20 annotated images per category by introducing the SG module.

Research on moisture content detection method during green tea processing based on machine vision and near-infrared spectroscopy technology

Moisture content is an important indicator that affects green tea processing. In this study, taking Chuyeqi tea as the research object, a quantitative prediction model of the changes in moisture content during the processing of green tea was constructed based on machine vision and near-infrared spectroscopy technology. First, collect the spectrum and image information in the process of spreading, fixation, first-drying, carding, and second-drying. The competitive adaptive reweighted sampling (CARS) method is then used to extract the characteristic wavelengths in the spectrum, and the image's 9 color features and 6 texture features are combined to establish linear PLSR and nonlinear SVR prediction models by fusing the data information from the two sensors. The results show that, when compared to single data, the PLSR and SVR models based on low-level data fusion do not effectively improve the model's prediction accuracy, but rather produce poor prediction results. In contrast, the PLSR and SVR models established by middle-level data fusion have improved the prediction accuracy of moisture content in green tea processing. Among them, the established SVR model has the best effect. The correlation coefficient of the calibration set (Rc) and the root mean square error of calibration (RMSEC) are 0.9804 and 0.0425, respectively, the correlation coefficient of the prediction set (Rp) and the root mean square error of prediction (RMSEP) are 0.9777 and 0.0490 respectively, and the relative percent deviation is 4.5002. The results show that the middle data fusion based on machine vision and near-infrared spectroscopy technology can effectively predict the moisture content in the processing of green tea, which has important guiding significance for overcoming the low prediction accuracy of a single sensor.

Multiplex Sensing of Complex Mixtures by Machine Vision Analysis of TLC-SERS Images

Thin layer chromatography in tandem with surface-enhanced Raman scattering (TLC-SERS) has demonstrated tremendous potentials as a new analytical chemistry tool to detect a wide range of substances from real-world samples. However, it still faces significant challenges of multiplex sensing from complex mixtures due to the imperfect separation by TLC and the resulting interference of SERS detection. In this article, we propose a multiplex sensing method of complex mixtures by machine vision analysis of the scanning image of the TLC-SERS results. Briefly, various pure substances in solution and the complex mixture solution are separated by TLC followed by one-dimensional SERS scanning of the entire TLC plate, which generates TLC-SERS images of all target substances along the chromatography path. After that, a machine vision method is employed to extract the template images from the TLC-SERS images of pure substance solutions. Finally, we apply a feature point matching strategy based on the Winner-take-all principle, which matches the template image of each pure substance with the mixture image to confirm the existence and derive the position of each target substance in the TLC plate, respectively. Our experimental results based on the mixture solution of five different substances show that the proposed machine vision analysis is highly selective, sensitive and does not require artificial analysis of the SERS spectra. Therefore, we envision that the proposed machine vision analysis of the TLC-SERS imaging is an objective, accurate, and efficient method for multiplex sensing of trace level of target substances from complex mixtures.

Comparison of novel machine vision spinal image guidance system with existing 3D fluoroscopy-based navigation system: a randomized prospective study

BACKGROUND CONTEXT

The use of spinal image guidance systems (IGS) has increased patient safety, accuracy, operative efficiency, and reduced revision rates in pedicle screw placement procedures. Traditional intraoperative 3D fluoroscopy or CT imaging produces potentially harmful ionizing radiation and increases operative time to register the patient. An IGS, FLASH Navigation, uses machine vision through high resolution stereoscopic cameras and structured visible light to build a 3D topographical map of the patient's bony surface anatomy enabling navigation use without ionizing radiation.

PURPOSE

We aimed to compare FLASH navigation system to a widely used 3D fluoroscopic navigation (3D) platform by comparing radiation exposure and pedicle screw accuracy.

DESIGN

A randomized prospective comparative cohort study of consecutive patients undergoing open posterior lumbar instrumented fusion.

PATIENT SAMPLE

Adults diagnosed with spinal pathology requiring surgical treatment and planning for open posterior lumbar fusion with pedicle screws implanted into 1-4 vertebral levels.

OUTCOME MEASURES

Outcome measures included mean intraoperative fluoroscopy time and dose, mean CT dose length product (DLP) for preoperative and day 2 CT, pedicle screw accuracy by CT, estimated blood loss and revision surgery rate.

METHODS

Consecutive patients were randomized 1:1 to FLASH or 3D and underwent posterior lumbar instrumented fusion. Radiation doses were recorded from pre- and postoperative CT and intraoperative 3D fluoroscopy. 2 independent blinded radiologists reviewed pedicle screw accuracy on CT.

RESULTS

A total of 429 (n=210 FLASH, n=219 3D) pedicle screws were placed in 90 patients (n=45 FLASH, n=45 3D) over the 18-month study period. Mean age and indication for surgery were similar between both groups, with a non-significantly higher ratio of males in the 3D group. Mean intraoperative fluoroscopy time and doses were significantly reduced in FLASH compared to 3D (4.51±3.71s vs 79.6±23.0s, p<.001 and 80.9±68.1cGycm² vs 3704.1±3442.4 cGycm², p<.001, respectively). This represented a relative reduction of 94.3% in the total intraoperative radiation time and a 97.8% reduction in the total intraoperative radiation dose. Mean preoperative CT DLP and mean day 2 postoperative CT DLP were significantly reduced in FLASH compared to 3D (662.0±440.4mGy-cm vs 1008.9±616.3 mGy-cm, p<.001 and 577.9±294.3 mGy-cm vs 980.7±441.6 mGy-cm, p<.001, respectively). This represented relative reductions of 34.4% and 41.0% in the preoperative CT dose and postoperative total DLP, respectively. The FLASH group required an average of 1.2 registrations in each case with an average of 2447 (±961.3) data points registered with a mean registration time of 106s (±52.1). A rapid re-registration mechanism was utilized in 22% (n=10/45) of cases and took 22.7s (±11.3). Re-registration was used in 7% (n=3/45) in the 3D group. Pedicle screw accuracy was high in FLASH (98.1%) and 3D (97.3%) groups with no pedicle breach >2mm in either group (p<.001). EBL was not statistically different between the groups (p=.38). No neurovascular injuries occurred, and no patients required return to theatre for screw repositioning.

CONCLUSIONS

FLASH and 3D IGS demonstrate high accuracy for pedicle screw placement. FLASH showed significant reduction in intraoperative radiation time and dose with lower but non-significant blood loss. FLASH showed significant reduction in preoperative and postoperative radiation, but this may be associated to the lower number of males/females preponderance in this group. FLASH provides similar accuracy to contemporary IGS without requiring 3D-fluoroscopy or radiolucent operating tables. Reducing registration time and specialized equipment may reduce costs.

Wide-angle, monocular head tracking using passive markers

BACKGROUND

Camera images can encode large amounts of visual information of an animal and its environment, enabling high fidelity 3D reconstruction of the animal and its environment using computer vision methods. Most systems, both markerless (e.g. deep learning based) and marker-based, require multiple cameras to track features across multiple points of view to enable such 3D reconstruction. However, such systems can be expensive and are challenging to set up in small animal research apparatuses.

NEW METHODS

We present an open-source, marker-based system for tracking the head of a rodent for behavioral research that requires only a single camera with a potentially wide field of view. The system features a lightweight visual target and computer vision algorithms that together enable high-accuracy tracking of the six-degree-of-freedom position and orientation of the animal's head. The system, which only requires a single camera positioned above the behavioral arena, robustly reconstructs the pose over a wide range of head angles (360° in yaw, and approximately ± 120° in roll and pitch).

RESULTS

Experiments with live animals demonstrate that the system can reliably identify rat head position and orientation. Evaluations using a commercial optical tracker device show that the system achieves accuracy that rivals commercial multi-camera systems.

COMPARISON WITH EXISTING METHODS

Our solution significantly improves upon existing monocular marker-based tracking methods, both in accuracy and in allowable range of motion.

CONCLUSIONS

The proposed system enables the study of complex behaviors by providing robust, fine-scale measurements of rodent head motions in a wide range of orientations.

UV excited fluorescence image-based non-destructive method for early detection of strawberry (Fragaria × ananassa) spoilage

The soon spoiled strawberries need to be classified from healthy fruits in an early stage. In this research, a machine vision system is proposed for inspecting the quality of strawberries using ultraviolet (UV) light based on the excitation-emission matrix (EEM) results. Among the 100 fruits which were harvested and stored under 10 °C condition for 7 days, 7 fruits were confirmed to be spoiled by using a firmness meter. The EEM results show the fluorescence compound contributes to a whitish surface on the spoiled fruits. Based on the EEM results, UV fluorescence images from the bottom view of strawberries were used to classify the spoiled fruits and healthy fruits within 1 day after harvest. These results demonstrate the UV fluorescence imaging can be a fast, non-destructive, and low-cost method for inspecting the soon spoiled fruits. The proposed index related to the spoiling time can be a new indicator for qualifying strawberry.

An integrated system for automated measurement of airborne pollen based on electrostatic enrichment and image analysis with machine vision

Here we describe an automated and compact pollen detection system that integrates enrichment, in-situ detection and self-cleaning modules. The system can achieve continuous capture and enrichment of pollen grains in air samples by electrostatic adsorption. The captured pollen grains are imaged with a digital camera, and an automated image analysis based on machine vision is performed, which enables a quantification of the number of pollen particles as well as a preliminary classification into two types of pollen grains. In order to optimize and evaluate the system performance, we developed a testing approach that utilizes an airflow containing a precisely metered amount of pollen particles surrounded by a sheath flow to achieve the generation and lossless transmission of standard gas samples. We studied various factors affecting the pollen capture efficiency, including the applied voltage, air flow rate and humidity. Under optimized conditions, the system was successfully used in the measurement of airborne pollen particles within a wide range of concentrations, spanning 3 orders of magnitude.

Color analysis of horticultural produces using hue spectra fingerprinting

Color has great importance in agriculture due to its relationship with plant pigments and therefore, plant development and biochemical changes. Due to the trichromatic vision, instruments equipped with CCD or CMOS sensor represent color with the mixture of red, green and blue signals. These values are often transformed into HSL (hue, saturation, luminance) color space. Beyond average color of the visible surface area, histograms can represent color distribution. Interpretation of distribution can be challenging due to the information shared among histograms. Hue spectra fingerprinting offers color information suitable for analysis with common chemometric methods and easy to understand. Algorithm is presented with GNU Octave code.•Hue spectra is a histogram of hue angle over the captured scene but summarizes saturation instead of number of pixels. There are peaks of important colors, while others of low saturation disappear. Neutral backgrounds such as white, black or gray, are removed without the need of segmentation.•Color changes of fruits and vegetables are represented by displacement of color peaks. Since saturation is usually changing during ripening, storage and shelf life, peaks also change their shape by means of peak value and width.

Automated morphometrics on microscopy images of Atlantic cod larvae using Mask R-CNN and classical machine vision techniques

Measurements of morphometrical parameters on i.e., fish larvae are useful for assessing the quality and condition of the specimen in environmental research or optimal growth in the cultivation industry. Manually acquiring morphometrical parameters from microscopy images can be time consuming and tedious, this can be a limiting factor when acquiring samples for an experiment. Mask R-CNN, an instance segmentation neural network architecture, has been implemented for finding outlines on parts of interest on fish larvae (Atlantic cod, Gadus morhua). Using classical machine vision techniques on the outlines makes it is possible to acquire morphometrics such as area, diameter, length, and height. The combination of these techniques is providing accurate-, consistent-, and high-volume information on the morphometrics of small organisms, making it possible to sample more data for morphometric analysis.•Capabilities to automatically analyse a set of microscopy images in approximately 2-3 seconds per image, with results that have a high degree of accuracy when compared to morphometrics acquired manually by an expert.•Can be implemented on other species of ichthyoplankton or zooplankton and has successfully been tested on ballan wrasse, zebrafish, lumpsucker and calanoid copepods.

At the Pulse of Time: Machine Vision in Retinal Videos

Spontaneous venous pulsations (SVP) are a common finding in healthy people. The absence of SVP is associated with rapid progression in glaucoma and increased intracranial pressure. Traditionally, SVP has been documented qualitatively by clinicians during biomicroscopy. Nowadays numerous imaging devices recording the fundus exist. Hence, video data for objectification of SVP is readily available. Still, these clinical datasets are afflicted with various quality issues and artifacts. In this machine vision based study, we explore methods to overcome challenges in identifying SVP in fundus videos of varying quality and provide a detailed protocol thereof. Hereby, we aim to lower the burden of access of implementing machine vision in clinical video datasets and quantification of SVP.

Genetic dissection of seed appearance quality using recombinant inbred lines in soybean

Soybean seed appearance quality greatly affects the marketability. The objective of this study was to identify the quantitative trait loci (QTLs) that control the appearance quality of soybean seeds. A total of 256 recombinant inbred lines from Qi Huang No.34 × Ji Dou No.17 were utilized for QTL mapping. We innovatively applied a machine vision system to quantify the seed appearance of each line. As a result of QTL mapping, a total of 145 QTLs for the machine vision parameters were detected across three environments. We integrated QTLs mapped overlapped and obtained 16 QTL hotspots in total. Of these hotspots, hotspot-4-1 was suggested to be a major locus controlling seed size, and hotspot-15 was identified to affect the seed color and texture. The mapping for principal components of the seed appearance also supported it. This study comprehensively dug up the QTLs for seed appearance quality of soybean cultivars while providing an efficient method for phenotyping of seed appearance. These results would contribute to dissecting the genetic bases of seed appearance quality for the improvement of soybean.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11032-021-01262-9.

Cotton stubble detection based on wavelet decomposition and texture features

BACKGROUND

At present, the residual film pollution in cotton fields is crucial. The commonly used recycling method is the manual-driven recycling machine, which is heavy and time-consuming. The development of a visual navigation system for the recovery of residual film is conducive, in order to improve the work efficiency. The key technology in the visual navigation system is the cotton stubble detection. A successful cotton stubble detection can ensure the stability and reliability of the visual navigation system.

METHODS

Firstly, it extracts the three types of texture features of GLCM, GLRLM and LBP, from the three types of images of stubbles, residual films and broken leaves between rows. It then builds three classifiers: Random Forest, Back Propagation Neural Network and Support Vector Machine in order to classify the sample images. Finally, the possibility of improving the classification accuracy using the texture features extracted from the wavelet decomposition coefficients, is discussed.

RESULTS

The experiment proves that the GLCM texture feature of the original image has the best performance under the Back Propagation Neural Network classifier. As for the different wavelet bases, the vertical coefficient texture feature of coif3 wavelet decomposition, combined with the texture feature of the original image, is the feature having the best classification effect. Compared with the original image texture features, the classification accuracy is increased by 3.8%, the sensitivity is increased by 4.8%, and the specificity is increased by 1.2%.

CONCLUSIONS

The algorithm can complete the task of stubble detection in different locations, different periods and abnormal driving conditions, which shows that the wavelet coefficient texture feature combined with the original image texture feature is a useful fusion feature for detecting stubble and can provide a reference for different crop stubble detection.

Indirect monitoring of ultralow dose API content in continuous wet granulation and tableting by machine vision

This paper presents new machine vision-based methods for indirect real-time quantification of ultralow drug content during continuous twin-screw wet granulation and tableting. Granulation was performed with a solution containing carvedilol (CAR) as API in the ultralow dose range (0.05w/w% in the granule) and the addition of riboflavin (RI) as a coloured tracer. An in-line calibration in the range of 0.047-0.058 w/w% was prepared for the measurement of CAR concentration using colour analysis (CA) and particle size analysis (PSA), and the validation with HPLC resulted in respective relative errors of 2.62% and 2.30% showing great accuracy. To improve the technique, a second in-line calibration was conducted in a broader CAR concentration range of 0.039-0.063 w/w% utilizing only half the amount of RI (0.045 w/w%), while doubling the output of the granulation line to 2 kg/h, producing a relative error of 4.51% and 4.29%, respectively. Finally, it was shown that the CA technique can also be carried on to monitor the CAR content of tablets in the 42-62 μg dose range with a relative error of 5.20%. Machine vision was proven to be a potent indirect method for the in-line, determination and monitoring of ultralow API content during continuous manufacturing.

A feasibility research on the application of machine vision technology in appearance quality inspection of Xuesaitong dropping pills

Defect detection is a critical issue for the quality control of dropping pills, which is a special dosage form of traditional Chinese Medicine. Machine vision is a non-destructing testing technology and cost-effective with high accuracy that can be used to predict the detects of both interior and exterior of the sample by employing the camera. In this research, a machine vision system for inspecting quality of the Xuesaitong dropping pills (XDPs) that include non-spherical, abnormal sizes and colors was developed to evaluate the appearance quality of XDPs rapidly and accurately. Firstly, 270 images of XDPs containing qualified and three different types of defects were collected. Subsequently, the processing of the XDPs images were carried out. Finally, Three defecting categories classification models were developed and compared based on contour and color features. The experimental results showed that the Random Forest outperformed all the explored models and the classification accuracy for non-spherical, abnormal sizes and colors reached 98.52%, 100.00% and 100.00%, respectively. In summary, the method established in this research is scientific, reliable, fast and accurate, which has great application potential and can provide technical support for the automatic defect detection of dropping pills.

Detection of fraud in ginger powder using an automatic sorting system based on image processing technique and deep learning

Ginger is a well-known product in the food and pharmaceutical industries. Ginger is one of the spices which are adulterated for economic gain. The lack of marketability of grade 3 chickpeas (small and broken chickpeas) and their very low price have made them a good choice to be mixed with ginger in powder form and sold in the market. Demand for non-destructive methods of measuring food quality, such as machine vision and the growing need for food and spices, were the main motives to conduct this study. This study classified ginger powder images to detect fraud by improving convolutional neural networks (CNN) through a gated pooling function. The main approach to improving CNN is to use a pooling function that combines average pooling and max pooling. The Batch normalization (BN) technique is used in CNN to improve classification results. We show empirically that the combining operation used increases the accuracy of ginger powder classification compared to the baseline pooling method. For this purpose, 3360 image samples of ginger powder were prepared in 7 categories (pure ginger powder, chickpea powder, 10%, 20%, 30%, 40%, and 50% fraud in ginger powder). Moreover, MLP, Fuzzy, SVM, GBT, and EDT algorithms were used to compare the proposed CNN results with other classifiers. The results showed that using batch normalization based on gated pooling, the proposed CNN was able to grade the images of ginger powder with 99.70% accuracy compared to other classifiers. Therefore, it can be said that the CNN method and image processing technique effectively increase marketability, prevent ginger powder fraud, and promote traditional methods of ginger powder fraud detection.

An improved automated zebrafish larva high-throughput imaging system

As a typical multicellular model organism, the zebrafish has been increasingly used in biological research. Despite the efforts to develop automated zebrafish larva imaging systems, existing ones are still defective in terms of reliability and automation. This paper presents an improved zebrafish larva high-throughput imaging system, which makes improvements to the existing designs in the following aspects. Firstly, a single larva extraction strategy is developed to make larva loading more reliable. The aggregated larvae are identified, classified by their numbers and patterns, and separated by the aspiration pipette or water stream. Secondly, the dynamic model of larva motion in the capillary is established and an adaptive robust controller is designed for decelerating the fast-moving larva to ensure the survival rate. Thirdly, rotating the larva to the desired orientation is automated by developing an algorithm to estimate the larva's initial rotation angle. For validating the improved larva imaging system, a real-time heart rate monitoring experiment is conducted as an application example. Experimental results demonstrate that the goals of the improvements have been achieved. With these improvements, the improved zebrafish larva imaging system remarkably reduces human intervention and increases the efficiency and success/survival rates of larva imaging.

Performance of image guided navigation in laparoscopic liver surgery - A systematic review

BACKGROUND

Compared to open surgery, minimally invasive liver resection has improved short term outcomes. It is however technically more challenging. Navigated image guidance systems (IGS) are being developed to overcome these challenges. The aim of this systematic review is to provide an overview of their current capabilities and limitations.

METHODS

Medline, Embase and Cochrane databases were searched using free text terms and corresponding controlled vocabulary. Titles and abstracts of retrieved articles were screened for inclusion criteria. Due to the heterogeneity of the retrieved data it was not possible to conduct a meta-analysis. Therefore results are presented in tabulated and narrative format.

RESULTS

Out of 2015 articles, 17 pre-clinical and 33 clinical papers met inclusion criteria. Data from 24 articles that reported on accuracy indicates that in recent years navigation accuracy has been in the range of 8-15 mm. Due to discrepancies in evaluation methods it is difficult to compare accuracy metrics between different systems. Surgeon feedback suggests that current state of the art IGS may be useful as a supplementary navigation tool, especially in small liver lesions that are difficult to locate. They are however not able to reliably localise all relevant anatomical structures. Only one article investigated IGS impact on clinical outcomes.

CONCLUSIONS

Further improvements in navigation accuracy are needed to enable reliable visualisation of tumour margins with the precision required for oncological resections. To enhance comparability between different IGS it is crucial to find a consensus on the assessment of navigation accuracy as a minimum reporting standard.

Machine Vision for Real-Time Intraoperative Anatomic Guidance: A Proof-of-Concept Study in Endoscopic Pituitary Surgery

BACKGROUND

Current intraoperative orientation methods either rely on preoperative imaging, are resource-intensive to implement, or difficult to interpret. Real-time, reliable anatomic recognition would constitute another strong pillar on which neurosurgeons could rest for intraoperative orientation.

OBJECTIVE

To assess the feasibility of machine vision algorithms to identify anatomic structures using only the endoscopic camera without prior explicit anatomo-topographic knowledge in a proof-of-concept study.

METHODS

We developed and validated a deep learning algorithm to detect the nasal septum, the middle turbinate, and the inferior turbinate during endoscopic endonasal approaches based on endoscopy videos from 23 different patients. The model was trained in a weakly supervised manner on 18 and validated on 5 patients. Performance was compared against a baseline consisting of the average positions of the training ground truth labels using a semiquantitative 3-tiered system.

RESULTS

We used 367 images extracted from the videos of 18 patients for training, as well as 182 test images extracted from the videos of another 5 patients for testing the fully developed model. The prototype machine vision algorithm was able to identify the 3 endonasal structures qualitatively well. Compared to the baseline model based on location priors, the algorithm demonstrated slightly but statistically significantly (P < .001) improved annotation performance.

CONCLUSION

Automated recognition of anatomic structures in endoscopic videos by means of a machine vision model using only the endoscopic camera without prior explicit anatomo-topographic knowledge is feasible. This proof of concept encourages further development of fully automated software for real-time intraoperative anatomic guidance during surgery.

A microfluidic robot for rare cell sorting based on machine vision identification and multi-step sorting strategy

The identification, sorting and analysis of rare target single cells in human blood has always been a clinically meaningful medical challenge. Here, we developed a microfluidic robot platform for sorting specific rare cells from complex clinical blood samples based on machine vision-based image identification, liquid handling robot and droplet-based microfluidic techniques. The robot integrated a cell capture and droplet generation module, a laser-induced fluorescence imaging module, a target cell identification and data analysis module, and a system control module, which could automatically achieve the scanning imaging of cell array, cell identification, capturing, and droplet generation of rare target cells from blood samples containing large numbers of normal cells. Based on the robot platform, a novel "gold panning" multi-step sorting strategy was proposed to achieve the sorting of rare target cells in large-scale cell samples with high operation efficiency and high sorting purity (>90%). The robot platform and the multi-step sorting strategy were applied in the sorting of circulating endothelial progenitor cells (CEPCs) in human blood to demonstrate their feasibility and application potential in the sorting and analysis of rare specific cells. Approximately 1,000 CEPCs were automatically identified from 3,000,000 blood cells at a scanning speed of ca. 4,000 cells/s, and 20 25-nL droplets containing single CEPCs were generated.

Deep learning and machine vision for food processing: A survey

The quality and safety of food is an important issue to the whole society, since it is at the basis of human health, social development and stability. Ensuring food quality and safety is a complex process, and all stages of food processing must be considered, from cultivating, harvesting and storage to preparation and consumption. However, these processes are often labour-intensive. Nowadays, the development of machine vision can greatly assist researchers and industries in improving the efficiency of food processing. As a result, machine vision has been widely used in all aspects of food processing. At the same time, image processing is an important component of machine vision. Image processing can take advantage of machine learning and deep learning models to effectively identify the type and quality of food. Subsequently, follow-up design in the machine vision system can address tasks such as food grading, detecting locations of defective spots or foreign objects, and removing impurities. In this paper, we provide an overview on the traditional machine learning and deep learning methods, as well as the machine vision techniques that can be applied to the field of food processing. We present the current approaches and challenges, and the future trends.

A fully-annotated imagery dataset of sublittoral benthic species in Svalbard, Arctic

Underwater imagery is widely used for a variety of applications in marine biology and environmental sciences, such as classification and mapping of seabed habitats, marine environment monitoring and impact assessment, biogeographic reconstructions in the context of climate change, etc. This approach is relatively simple and cost-effective, allowing the rapid collection of large amounts of data. However, due to the laborious and time-consuming manual analysis procedure, only a small part of the information stored in the archives of underwater images is retrieved. Emerging novel deep learning methods open up the opportunity for more effective, accurate and rapid analysis of seabed images than ever before. We present annotated images of the bottom macrofauna obtained from underwater video recorded in Spitsbergen island's European Arctic waters, Svalbard Archipelago. Our videos were filmed in both the photic and aphotic zones of polar waters, often influenced by melting glaciers. We used artificial lighting and shot close to the seabed (<1 m) to preserve natural colours and avoid the distorting effect of muddy water. The underwater video footage was captured using a remotely operated vehicle (ROV) and a drop-down camera. The footage was converted to 2D mosaic images of the seabed. 2D mosaics were manually annotated by several experts using the Labelbox tool and co-annotations were refined using the SurveyJS platform. A set of carefully annotated underwater images associated with the original videos can be used by marine biologists as a biological atlas, as well as practitioners in the fields of machine vision, pattern recognition, and deep learning as training materials for the development of various tools for automatic analysis of underwater imagery.

Continuous blending monitored and feedback controlled by machine vision-based PAT tool

In a continuous powder blending process machine vision is utilized as a Process Analytical Technology (PAT) tool. While near-infrared (NIR) and Raman spectroscopy are reliable methods in this field, measurements become challenging when concentrations below 2 w/w% are quantified. However, an active pharmaceutical ingredient (API) with an intense color might be quantified in even lower quantities by images recorded with a digital camera. Riboflavin (RI) was used as a model API with orange color, its Limit of Detection was found to be 0.015 w/w% and the Limit of Quantification was 0.046 w/w% using a calibration based on the pixel value of images. A calibration for in-line measurement of RI concentration was prepared in the range of 0.2-0.45 w/w%, validation with UV/VIS spectrometry showed great accuracy with a relative error of 2.53 %. The developed method was then utilized for a residence time distribution (RTD) measurement in order to characterize the dynamics of the blending process. Lastly, the technique was applied in real-time feedback control of a continuous powder blending process. Machine vision based direct or indirect API concentration determination is a promising and fast method with a great potential for monitoring and control of continuous pharmaceutical processes.

Intelligent fish farm-the future of aquaculture

With the continuous expansion of aquaculture scale and density, contemporary aquaculture methods have been forced to overproduce resulting in the accelerated imbalance rate of water environment, the frequent occurrence of fish diseases, and the decline of aquatic product quality. Moreover, due to the fact that the average age profile of agricultural workers in many parts of the world are on the higher side, fishery production will face the dilemma of shortage of labor, and aquaculture methods are in urgent need of change. Modern information technology has gradually penetrated into various fields of agriculture, and the concept of intelligent fish farm has also begun to take shape. The intelligent fish farm tries to deal with the precise work of increasing oxygen, optimizing feeding, reducing disease incidences, and accurately harvesting through the idea of "replacing human with machine," so as to liberate the manpower completely and realize the green and sustainable aquaculture. This paper reviews the application of fishery intelligent equipment, IoT, edge computing, 5G, and artificial intelligence algorithms in modern aquaculture, and analyzes the existing problems and future development prospects. Meanwhile, based on different business requirements, the design frameworks for key functional modules in the construction of intelligent fish farm are proposed.

Automated cephalometric landmark detection with confidence regions using Bayesian convolutional neural networks

Background

Despite the integral role of cephalometric analysis in orthodontics, there have been limitations regarding the reliability, accuracy, etc. of cephalometric landmarks tracing. Attempts on developing automatic plotting systems have continuously been made but they are insufficient for clinical applications due to low reliability of specific landmarks. In this study, we aimed to develop a novel framework for locating cephalometric landmarks with confidence regions using Bayesian Convolutional Neural Networks (BCNN).

Methods

We have trained our model with the dataset from the ISBI 2015 grand challenge in dental X-ray image analysis. The overall algorithm consisted of a region of interest (ROI) extraction of landmarks and landmarks estimation considering uncertainty. Prediction data produced from the Bayesian model has been dealt with post-processing methods with respect to pixel probabilities and uncertainties.

Results

Our framework showed a mean landmark error (LE) of 1.53 ± 1.74 mm and achieved a successful detection rate (SDR) of 82.11, 92.28 and 95.95%, respectively, in the 2, 3, and 4 mm range. Especially, the most erroneous point in preceding studies, Gonion, reduced nearly halves of its error compared to the others. Additionally, our results demonstrated significantly higher performance in identifying anatomical abnormalities. By providing confidence regions (95%) that consider uncertainty, our framework can provide clinical convenience and contribute to making better decisions.

Conclusion

Our framework provides cephalometric landmarks and their confidence regions, which could be used as a computer-aided diagnosis tool and education.

A machine vision system for tracking population behavior of zooplankton in small-scale experiments: a case study on salmon lice (Lepeophtheirus salmonis Krøyer, 1838) copepodite population responses to different light stimuli

To achieve efficient and preventive measures against salmon lice (Lepeophtheirus salmonis Krøyer, 1838) infestation, a better understanding of behavioral patterns of the planktonic life stages is key. To investigate light responses in L. salmonis copepodites, a non-intrusive experimental system was designed to measure behavioral responses in a 12.5-l volume using machine vision technology and methodology. The experimental system successfully tracked the collective movement patterns of the sea lice population during exposure to different light stimuli emitted from alternating zones in the system. This system could further be used to study behavioral responses to different physical cues of various developmental stages of sea lice or other zooplankton.