Application of deep learning for image-based Chinese market food nutrients estimation

Recent advances on artificial intelligence-based approaches for food adulteration and fraud detection in the food industry: Challenges and opportunities

Food adulteration is the deceitful practice of misleading consumers about food to profit from it. The threat to public health and food quality or nutritional valuable make it a major issue. Food origin and adulteration should be considered to safeguard customers against fraud. It has been established that artificial intelligence is a cutting-edge technology in food science and engineering. In this study, it has been explained how AI detects food tampering. Applications of AI such as machine learning tools in food quality have been studied. This review covered several food quality detection web-based information sources. The methods used to detect food adulteration and food quality standards have been highlighted. Various comparisons between state-of-the-art techniques, datasets, and outcomes have been conducted. The outcomes of this investigation will assist researchers choose the best food quality method. It will help them identify of foods that have been explored by researchers and potential research avenues.

Application of machine vision in food computing: A review

With global intelligence advancing and the awareness of sustainable development growing, artificial intelligence technology is increasingly being applied to the food industry. This paper, grounded in practical application cases, reviews the current research status and prospects of machine vision-based image recognition technology in food computing. It explores the general workflow of image recognition, applications based on traditional machine learning and deep learning methods. The paper covers areas including food safety detection, dietary nutrition analysis, process monitoring, and enterprise management model optimization. The aim is to provide a solid theoretical foundation and technical guidance for the integration and cross-fertilization of the food industry with artificial intelligence technology.

Innovative food supply chain through spatial computing technologies: A review

The global food supply chain faces significant challenges related to inefficiencies, quality variability, and traceability issues, all of which contribute to food waste and consumer distrust. Spatial computing (SC) technologies, including augmented reality (AR), virtual reality (VR), and digital twins, offer promising solutions by enhancing precision agriculture, logistics, manufacturing, and retail operations. This review explores SC's potential across the entire food supply continuum, emphasizing improvements in resource management, supply chain transparency, and consumer engagement. Despite its promise, the widespread adoption of SC is limited by technical challenges and a lack of standardized protocols. The findings suggest that while SC has the potential to revolutionize the food supply chain by improving sustainability, efficiency, and safety, further interdisciplinary research and collaboration are essential to fully unlock its capabilities.

Nutritional composition analysis in food images: an innovative Swin Transformer approach

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Empowering Diabetics: Advancements in Smartphone-Based Food Classification, Volume Measurement, and Nutritional Estimation

Diabetes has emerged as a worldwide health crisis, affecting approximately 537 million adults. Maintaining blood glucose requires careful observation of diet, physical activity, and adherence to medications if necessary. Diet monitoring historically involves keeping food diaries; however, this process can be labor-intensive, and recollection of food items may introduce errors. Automated technologies such as food image recognition systems (FIRS) can make use of computer vision and mobile cameras to reduce the burden of keeping diaries and improve diet tracking. These tools provide various levels of diet analysis, and some offer further suggestions for improving the nutritional quality of meals. The current study is a systematic review of mobile computer vision-based approaches for food classification, volume estimation, and nutrient estimation. Relevant articles published over the last two decades are evaluated, and both future directions and issues related to FIRS are explored.

DeepTrayMeal: Automatic dietary assessment for Chinese tray meals based on deep learning

Tray meal is a popular way of eating in China, and tray-based automatic dietary assessment is important for public health. Relevant research is lacking because public tray meal datasets and suitable methods are unavailable. In this study, we established and published the first Chinese tray meal dataset, the ChinaLunchTray-99. We collected real-world 1185 tray meal images, covering 99 dish categories with corresponding manually annotated bounding box and category-level labels. We developed a new framework for automatic dietary assessment, which consists of dish image recognition, volume estimation and nutrition mapping. First, we demonstrated a tray meal detection model considering feature extraction, anchor scales, and loss function, resulting in a high mean Average Precision of 92.13%. Second, we proposed an automatic method to estimate volume via detection results and tray's information. Finally, nutrients were mapped from the estimated volume. Our research can promote applications of automatic dietary assessment for Chinese tray meals.

AI-based digital image dietary assessment methods compared to humans and ground truth: a systematic review

OBJECTIVE

Human error estimating food intake is a major source of bias in nutrition research. Artificial intelligence (AI) methods may reduce bias, but the overall accuracy of AI estimates is unknown. This study was a systematic review of peer-reviewed journal articles comparing fully automated AI-based (e.g. deep learning) methods of dietary assessment from digital images to human assessors and ground truth (e.g. doubly labelled water).

MATERIALS AND METHODS

Literature was searched through May 2023 in four electronic databases plus reference mining. Eligible articles reported AI estimated volume, energy, or nutrients. Independent investigators screened articles and extracted data. Potential sources of bias were documented in absence of an applicable risk of bias assessment tool.

RESULTS

Database and hand searches identified 14,059 unique publications; fifty-two papers (studies) published from 2010 to 2023 were retained. For food detection and classification, 79% of papers used a convolutional neural network. Common ground truth sources were calculation using nutrient tables (51%) and weighed food (27%). Included papers varied widely in food image databases and results reported, so meta-analytic synthesis could not be conducted. Relative errors were extracted or calculated from 69% of papers. Average overall relative errors (AI vs. ground truth) ranged from 0.10% to 38.3% for calories and 0.09% to 33% for volume, suggesting similar performance. Ranges of relative error were lower when images had single/simple foods.

CONCLUSIONS

Relative errors for volume and calorie estimations suggest that AI methods align with - and have the potential to exceed - accuracy of human estimations. However, variability in food image databases and results reported prevented meta-analytic synthesis. The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be adequate for training and testing and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.

A fine-grained recognition technique for identifying Chinese food images

As a crucial area of research in the field of computer vision, food recognition technology has become a core technology in many food-related fields, such as unmanned restaurants and food nutrition analysis, which are closely related to our healthy lives. Obtaining accurate classification results is the most important task in food recognition. Food classification is a fine-grained recognition process, which involves extracting features from a group of objects with similar appearances and accurately classifying them into different categories. In a such usage environment, the network is required to not only overview the overall image, but also capture the subtle details within it. In addition, since Chinese food images have unique texture features, the model needs to extract texture information from the image. However, existing CNN methods have not focused on and processed this information. To classify food as accurately as possible, this paper introduces the Laplace pyramid into the convolution layer and proposes a bilinear network that can perceive image texture features and multi-scale features (LMB-Net). The proposed model was evaluated on a public dataset, and the results demonstrate that LMB-Net achieves state-of-the-art classification performance.

Machine learning accurately predicts food exchange list and the exchangeable portion

Introduction

Food Exchange Lists (FELs) are a user-friendly tool developed to help individuals aid healthy eating habits and follow a specific diet plan. Given the rapidly increasing number of new products or access to new foods, one of the biggest challenges for FELs is being outdated. Supervised machine learning algorithms could be a tool that facilitates this process and allows for updated FELs-the present study aimed to generate an algorithm to predict food classification and calculate the equivalent portion.

Methods

Data mining techniques were used to generate the algorithm, which consists of processing and analyzing the information to find patterns, trends, or repetitive rules that explain the behavior of the data in a food database after performing this task. It was decided to approach the problem from a vector formulation (through 9 nutrient dimensions) that led to proposals for classifiers such as Spherical K-Means (SKM), and by developing this idea, it was possible to smooth the limits of the classifier with the help of a Multilayer Perceptron (MLP) which were compared with two other algorithms of machine learning, these being Random Forest and XGBoost.

Results

The algorithm proposed in this study could classify and calculate the equivalent portion of a single or a list of foods. The algorithm allows the categorization of more than one thousand foods with a confidence level of 97% at the first three places. Also, the algorithm indicates which foods exceed the limits established in sodium, sugar, and/or fat content and show their equivalents.

Discussion

Accurate and robust FELs could improve implementation and adherence to the recommended diet. Compared with manual categorization and calculation, machine learning approaches have several advantages. Machine learning reduces the time needed for manual food categorization and equivalent portion calculation of many food products. Since it is possible to access food composition databases of various populations, our algorithm could be adapted and applied in other databases, offering an even greater diversity of regional products and foods. In conclusion, machine learning is a promising method for automation in generating FELs. This study provides evidence of a large-scale, accurate real-time processing algorithm that can be useful for designing meal plans tailored to the foods consumed by the population. Our model allowed us not only to distinguish and classify foods within a group or subgroup but also to perform the calculation of an equivalent food. As a neural network, this model could be trained with other food bases and thus improve its predictive capacity. Although the performance of the SKM model was lower compared to other types of classifiers, our model allows selecting an equivalent food not from a group previously classified by machine learning but with a fully interpretable algorithm such as cosine similarity for comparing food.

Automatic Chinese Food recognition based on a stacking fusion model

With commercialization of deep learning models, daily precision dietary record based on images from smartphones becomes possible. This study took advantage of Deep-learning techniques on visual recognition tasks and proposed a big-data-driven Deep-learning model regressing from food images. We established the largest data set of Chinese dishes to date, named CNFOOD-241. It contained more than 190,000 images with 241 categories, covering Staple food, meat, vegetarian diet, mixed meat and vegetables, soups, dessert category. This study also compares the prediction results of three popular deep learning models on this dataset, ResNeXt101_32x32d achieving up to 82.05% for top-1 accuracy and 97.13% for top-5 accuracy. Besides, this paper uses a multi-model fusion method based on stacking in the field of food recognition for the first time. We built a meta-learner after the base model to integrate the three base models of different architectures to improve robustness. The accuracy achieves 82.88% for top-1 accuracy.Clinical Relevance-This study proves that the application of artificial intelligence technology in the identification of Chinese dishes is feasible, which can play a positive role in people who need to control their diet, such as diabetes and obesity.

Vision-based food nutrition estimation via RGB-D fusion network

With the development of deep learning technology, vision-based food nutrition estimation is gradually entering the public view for its advantage in accuracy and efficiency. In this paper, we designed one RGB-D fusion network, which integrated multimodal feature fusion (MMFF) and multi-scale fusion for visioin-based nutrition assessment. MMFF performed effective feature fusion by a balanced feature pyramid and convolutional block attention module. Multi-scale fusion fused different resolution features through feature pyramid network. Both enhanced feature representation to improve the performance of the model. Compared with state-of-the-art methods, the mean value of the percentage mean absolute error (PMAE) for our method reached 18.5%. The PMAE of calories and mass reached 15.0% and 10.8% via the RGB-D fusion network, improved by 3.8% and 8.1%, respectively. Furthermore, this study visualized the estimation results of four nutrients and verified the validity of the method. This research contributed to the development of automated food nutrient analysis (Code and models can be found at http://123.57.42.89/codes/RGB-DNet/nutrition.html).

Evaluation of vegetable sauerkraut quality during storage based on convolution neural network

Vegetable sauerkraut is a traditional fermented food. Due to oxidation reactions that occur during storage, the quality and flavor in different periods will change. In this study, the quality evaluation and flavor characteristics of 13 groups of vegetable sauerkraut samples with different storage time were analyzed by using physical and chemical parameters combined with electronic nose. Photographs of samples of various periods were collected, and a convolutional neural network (CNN) framework was established. The relationship between total phenol oxidative decomposition and flavor compounds was linearly negatively correlated. The vegetable sauerkraut during storage can be divided into three categories (full acceptance period, acceptance period and unacceptance period) by principal component analysis and Fisher discriminant analysis. The CNN parameters were fine-tuned based on the classification results, and its output results can reflect the quality changes and flavor characteristics of the samples, and have better fitting, prediction capabilities. After 50 epochs of the model, the accuracy of three sets of data namely training set, validation set and test set recorded 94%, 85% and 93%, respectively. In addition, the accuracy of CNN in identifying different quality sauerkraut was 95.30%. It is proved that the convolutional neural network has excellent performance in predicting the quality of Szechuan Sauerkraut with high reliability.

Neural network in food analytics

Neural network (i.e. deep learning, NN)-based data analysis techniques have been listed as a pivotal opportunity to protect the integrity and safety of the global food supply chain and forecast $11.2 billion in agriculture markets. As a general-purpose data analytic tool, NN has been applied in several areas of food science, such as food recognition, food supply chain security and omics analysis, and so on. Therefore, given the rapid emergence of NN applications in food safety, this review aims to provide a comprehensive overview of the NN application in food analysis for the first time, focusing on domain-specific applications in food analysis by introducing fundamental methodology, reviewing recent and notable progress, and discussing challenges and potential pitfalls. NN demonstrated that it has a bright future through effective collaboration between food specialist and the broader community in the food field, for example, superiority in food recognition, sensory evaluation, pattern recognition of spectroscopy and chromatography. However, major challenges impeded NN extension including void in the food scientist-friendly interface software package, incomprehensible model behavior, multi-source heterogeneous data, and so on. The breakthrough from other fields proved NN has the potential to offer a revolution in the immediate future.

Deep learning accurately predicts food categories and nutrients based on ingredient statements

Determining attributes such as classification, creating taxonomies and nutrients for foods can be a challenging and resource-intensive task, albeit important for a better understanding of foods. In this study, a novel dataset, 134 k BFPD, was collected from USDA Branded Food Products Database with modification and labeled with three food taxonomy and nutrient values and became an artificial intelligence (AI) dataset that covered the largest food types to date. Overall, the Multi-Layer Perceptron (MLP)-TF-SE method obtained the highest learning efficiency for food natural language processing tasks using AI, which achieved up to 99% accuracy for food classification and 0.98 R² for calcium estimation (0.93 ∼ 0.97 for calories, protein, sodium, total carbohydrate, total lipids, etc.). The deep learning approach has great potential to be embedded in other food classification and regression tasks and as an extension to other applications in the food and nutrient scope.