Applications of knowledge graphs for food science and industry

KG4NH: A Comprehensive Knowledge Graph for Question Answering in Dietary Nutrition and Human Health

It is commonly known that food nutrition is closely related to human health. The complex interactions between food nutrients and diseases, influenced by gut microbial metabolism, present challenges in systematizing and practically applying knowledge. To address this, we propose a method for extracting triples from a vast amount of literature, which is used to construct a comprehensive knowledge graph on nutrition and human health. Concurrently, we develop a query-based question answering system over our knowledge graph, proficiently addressing three types of questions. The results show that our proposed model outperforms other state-of-art methods, achieving a precision of 0.92, a recall of 0.81, and an F1 score of 0.86 in the nutrition and disease relation extraction task. Meanwhile, our question answering system achieves an accuracy of 0.68 and an F1 score of 0.61 on our benchmark dataset, showcasing competitiveness in practical scenarios. Furthermore, we design five independent experiments to assess the quality of the data structure in the knowledge graph, ensuring results characterized by high accuracy and interpretability. In conclusion, the construction of our knowledge graph shows significant promise in facilitating diet recommendations, enhancing patient care applications, and informing decision-making in clinical research.

IngredSAM: Open-World Food Ingredient Segmentation via a Single Image Prompt

Food semantic segmentation is of great significance in the field of computer vision and artificial intelligence, especially in the application of food image analysis. Due to the complexity and variety of food, it is difficult to effectively handle this task using supervised methods. Thus, we introduce IngredSAM, a novel approach for open-world food ingredient semantic segmentation, extending the capabilities of the Segment Anything Model (SAM). Utilizing visual foundation models (VFMs) and prompt engineering, IngredSAM leverages discriminative and matchable semantic features between a single clean image prompt of specific ingredients and open-world images to guide the generation of accurate segmentation masks in real-world scenarios. This method addresses the challenges of traditional supervised models in dealing with the diverse appearances and class imbalances of food ingredients. Our framework demonstrates significant advancements in the segmentation of food ingredients without any training process, achieving 2.85% and 6.01% better performance than previous state-of-the-art methods on both FoodSeg103 and UECFoodPix datasets. IngredSAM exemplifies a successful application of one-shot, open-world segmentation, paving the way for downstream applications such as enhancements in nutritional analysis and consumer dietary trend monitoring.

An Online Multimodal Food Data Exploration Platform for Specific Population Health: Development Study

BACKGROUND

Nutrient needs vary over the lifespan. Improving knowledge of both population groups and care providers can help with healthier food choices, thereby promoting population health and preventing diseases. Providing evidence-based food knowledge online is credible, low cost, and easily accessible.

OBJECTIVE

This study aimed to develop an online multimodal food data exploration platform for easy access to evidence-based diet- and nutrition-related data.

METHODS

We developed an online platform named Food Atlas in collaboration with a multidisciplinary expert group from the National Institute for Nutrition and Health and Peking Union Medical College Hospital in China. To demonstrate its feasibility for Chinese food for pregnant women, a user-friendly and high-quality multimodal food knowledge graph was constructed, and various interactions with graph-structured data were developed for easy access, including graph-based interactive visualizations, natural language retrieval, and image-text retrieval. Subsequently, we evaluated Food Atlas from both the system perspective and the user perspective.

RESULTS

The constructed multimodal food knowledge graph contained a total of 2011 entities, 10,410 triplets, and 23,497 images. Its schema consisted of 11 entity types and 26 types of semantic relations. Compared with 5 other online dietary platforms (Foodwake, Boohee, Xiachufang, Allrecipes, and Yummly), Food Atlas offers a distinct and comprehensive set of data content and system functions desired by target populations. Meanwhile, a total of 28 participants representing 4 different user groups were recruited to evaluate its usability: preparing for pregnancy (n=8), pregnant (n=12), clinicians (n=5), and dietitians (n=3). The mean System Usability Scale index of our platform was 82.5 (SD 9.94; range 40.0-82.5). This above-average usability score and the use cases indicated that Food Atlas is tailored to the needs of the target users. Furthermore, 96% (27/28) of the participants stated that the platform had high consistency, illustrating the necessity and effectiveness of health professionals participating in online, evidence-based resource development.

CONCLUSIONS

This study demonstrates the development of an online multimodal food data exploration platform and its ability to meet the rising demand for accessible, credible, and appropriate evidence-based online dietary resources. Further research and broader implementation of such platforms have the potential to popularize knowledge, thereby helping populations at different life stages make healthier food choices.

FoodAtlas: Automated knowledge extraction of food and chemicals from literature

Automated generation of knowledge graphs that accurately capture published information can help with knowledge organization and access, which have the potential to accelerate discovery and innovation. Here, we present an integrated pipeline to construct a large-scale knowledge graph using large language models in an active learning setting. We apply our pipeline to the association of raw food, ingredients, and chemicals, a domain that lacks such knowledge resources. By using an iterative active learning approach of 4120 manually curated premise-hypothesis pairs as training data for ten consecutive cycles, the entailment model extracted 230,848 food-chemical composition relationships from 155,260 scientific papers, with 106,082 (46.0 %) of them never been reported in any published database. To augment the knowledge incorporated in the knowledge graph, we further incorporated information from 5 external databases and ontology sources. We then applied a link prediction model to identify putative food-chemical relationships that were not part of the constructed knowledge graph. Validation of the 443 hypotheses generated by the link prediction model resulted in 355 new food-chemical relationships, while results show that the model score correlates well (R2 = 0.70) with the probability of a novel finding. This work demonstrates how automated learning from literature at scale can accelerate discovery and support practical applications through reproducible, evidence-based capture of latent interactions of diverse entities, such as food and chemicals.

Integration of Diffusion Transformer and Knowledge Graph for Efficient Cucumber Disease Detection in Agriculture

In this study, a deep learning method combining knowledge graph and diffusion Transformer has been proposed for cucumber disease detection. By incorporating the diffusion attention mechanism and diffusion loss function, the research aims to enhance the model's ability to recognize complex agricultural disease features and to address the issue of sample imbalance efficiently. Experimental results demonstrate that the proposed method outperforms existing deep learning models in cucumber disease detection tasks. Specifically, the method achieved a precision of 93%, a recall of 89%, an accuracy of 92%, and a mean average precision (mAP) of 91%, with a frame rate of 57 frames per second (FPS). Additionally, the study successfully implemented model lightweighting, enabling effective operation on mobile devices, which supports rapid on-site diagnosis of cucumber diseases. The research not only optimizes the performance of cucumber disease detection, but also opens new possibilities for the application of deep learning in the field of agricultural disease detection.

Developing a Personalized Meal Recommendation System for Chinese Older Adults: Observational Cohort Study

BACKGROUND

China's older population is facing serious health challenges, including malnutrition and multiple chronic conditions. There is a critical need for tailored food recommendation systems. Knowledge graph-based food recommendations offer considerable promise in delivering personalized nutritional support. However, the integration of disease-based nutritional principles and preference-related requirements needs to be optimized in current recommendation processes.

OBJECTIVE

This study aims to develop a knowledge graph-based personalized meal recommendation system for community-dwelling older adults and to conduct preliminary effectiveness testing.

METHODS

We developed ElCombo, a personalized meal recommendation system driven by user profiles and food knowledge graphs. User profiles were established from a survey of 96 community-dwelling older adults. Food knowledge graphs were supported by data from websites of Chinese cuisine recipes and eating history, consisting of 5 entity classes: dishes, ingredients, category of ingredients, nutrients, and diseases, along with their attributes and interrelations. A personalized meal recommendation algorithm was then developed to synthesize this information to generate packaged meals as outputs, considering disease-related nutritional constraints and personal dietary preferences. Furthermore, a validation study using a real-world data set collected from 96 community-dwelling older adults was conducted to assess ElCombo's effectiveness in modifying their dietary habits over a 1-month intervention, using simulated data for impact analysis.

RESULTS

Our recommendation system, ElCombo, was evaluated by comparing the dietary diversity and diet quality of its recommended meals with those of the autonomous choices of 96 eligible community-dwelling older adults. Participants were grouped based on whether they had a recorded eating history, with 34 (35%) having and 62 (65%) lacking such data. Simulation experiments based on retrospective data over a 30-day evaluation revealed that ElCombo's meal recommendations consistently had significantly higher diet quality and dietary diversity compared to the older adults' own selections (P<.001). In addition, case studies of 2 older adults, 1 with and 1 without prior eating records, showcased ElCombo's ability to fulfill complex nutritional requirements associated with multiple morbidities, personalized to each individual's health profile and dietary requirements.

CONCLUSIONS

ElCombo has shown enhanced potential for improving dietary quality and diversity among community-dwelling older adults in simulation tests. The evaluation metrics suggest that the food choices supported by the personalized meal recommendation system surpass autonomous selections. Future research will focus on validating and refining ElCombo's performance in real-world settings, emphasizing the robust management of complex health data. The system's scalability and adaptability pinpoint its potential for making a meaningful impact on the nutritional health of older adults.

Disambiguity and Alignment: An Effective Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval

As a prominent topic in food computing, cross-modal recipe retrieval has garnered substantial attention. However, the semantic alignment across food images and recipes cannot be further enhanced due to the lack of intra-modal alignment in existing solutions. Additionally, a critical issue named food image ambiguity is overlooked, which disrupts the convergence of models. To these ends, we propose a novel Multi-Modal Alignment Method for Cross-Modal Recipe Retrieval (MMACMR). To consider inter-modal and intra-modal alignment together, this method measures the ambiguous food image similarity under the guidance of their corresponding recipes. Additionally, we enhance recipe semantic representation learning by involving a cross-attention module between ingredients and instructions, which is effective in supporting food image similarity measurement. We conduct experiments on the challenging public dataset Recipe1M; as a result, our method outperforms several state-of-the-art methods in commonly used evaluation criteria.

Effectiveness of personalized meal recommendation in improving dietary behaviors of Chinese community-dwelling elders: study protocol for a cluster randomized controlled trial

BACKGROUND

Inappropriate eating behaviors, particularly a lack of food diversity and poor diet quality, have a significant impact on the prognosis of certain chronic conditions and exacerbate these conditions in the community-dwelling elderly population. Current dietary interventions for the elderly have not adequately considered the nutritional needs associated with multiple chronic conditions and personal dietary preferences of elderly individuals. A personalized recommendation system has been recognized as a promising approach to address this gap. However, its effectiveness as a component of an elderly-targeted dietary intervention in real-world settings remains unknown. Additionally, it is unclear whether this intervention approach will be user-friendly for the elderly. Therefore, this study aims to examine the effectiveness of a personalized meal recommendation system designed to improve dietary behavior in community-dwelling elders. The implementation process in terms of System usability and satisfaction will also be assessed.

METHODS

The trial has been designed as a 6-month, non-blinded, parallel two-arm trial. One hundred fifty community-dwelling elders who meet the eligibility criteria will be enrolled. Subjects will be allocated to either the intervention group, receiving personalized meal recommendations and access to corresponding food provided as one component of the intervention, as well as health education on elder nutrition topics, or the control group, which will receive nutritional health education lectures. Outcomes will be measured at three time points: baseline at 0 months, 3 months, and 6 months. The primary outcomes will include dietary diversity (DDS) and diet quality (CDGI-E) of enrolled community-dwelling elders, representing their dietary behavior improvement, along with dietary behavior adherence to recommended meals. Secondary outcomes will measure the perceived acceptability and usability of the personalized meal recommendation system for the intervention group. Exploratory outcomes will include changes in the nutritional status and anthropometric measurements of the community-dwelling elders.

DISCUSSION

This study aims to examine the effectiveness, acceptability, and usability of a personalized meal recommendation system as a data-driven dietary intervention to benefit community-dwelling elders. The successful implementation will inform the future development and integration of digital health strategies in daily nutrition support for the elderly.

TRIAL REGISTRATION

Chinese Clinical Trial Registry ChiCTR2300074912. Registered on August 20, 2023, https://www.chictr.org.cn/showproj.html?proj=127583.

Review of visual analytics methods for food safety risks

With the availability of big data for food safety, more and more advanced data analysis methods are being applied to risk analysis and prewarning (RAPW). Visual analytics, which has emerged in recent years, integrates human and machine intelligence into the data analysis process in a visually interactive manner, helping researchers gain insights into large-scale data and providing new solutions for RAPW. This review presents the developments in visual analytics for food safety RAPW in the past decade. Firstly, the data sources, data characteristics, and analysis tasks in the food safety field are summarized. Then, data analysis methods for four types of analysis tasks: association analysis, risk assessment, risk prediction, and fraud identification, are reviewed. After that, the visualization and interaction techniques are reviewed for four types of characteristic data: multidimensional, hierarchical, associative, and spatial-temporal data. Finally, opportunities and challenges in this area are proposed, such as the visual analysis of multimodal food safety data, the application of artificial intelligence techniques in the visual analysis pipeline, etc.

PO2/TransformON, an ontology for data integration on food, feed, bioproducts and biowaste engineering

We are witnessing an acceleration of the global drive to converge consumption and production patterns towards a more circular and sustainable approach to the food system. To address the challenge of reconnecting agriculture, environment, food and health, collections of large datasets must be exploited. However, building high-capacity data-sharing networks means unlocking the information silos that are caused by a multiplicity of local data dictionaries. To solve the data harmonization problem, we proposed an ontology on food, feed, bioproducts, and biowastes engineering for data integration in a circular bioeconomy and nexus-oriented approach. This ontology is based on a core model representing a generic process, the Process and Observation Ontology (PO2), which has been specialized to provide the vocabulary necessary to describe any biomass transformation process and to characterize the food, bioproducts, and wastes derived from these processes. Much of this vocabulary comes from transforming authoritative references such as the European food classification system (FoodEx2), the European Waste Catalogue, and other international nomenclatures into a semantic, world wide web consortium (W3C) format that provides system interoperability and software-driven intelligence. We showed the relevance of this new domain ontology PO2/TransformON through several concrete use cases in the fields of process engineering, bio-based composite making, food ecodesign, and relations with consumer's perception and preferences. Further works will aim to align with other ontologies to create an ontology network for bridging the gap between upstream and downstream processes in the food system.

GENA: A knowledge graph for nutrition and mental health

While a large number of knowledge graphs have previously been developed by automatically extracting and structuring knowledge from literature, there is currently no such knowledge graph that encodes relationships between food, biochemicals and mental illnesses, even though a large amount of knowledge about these relationships is available in the form of unstructured text in biomedical literature articles. To address this limitation, this article describes the development of GENA - (Graph of mEntal-health and Nutrition Association), a knowledge graph that represents relations between nutrition and mental health, extracted from biomedical abstracts. GENA is constructed from PubMed abstracts that contain keywords relating to chemicals, food, and health. A hybrid named entity recognition (NER) model is firstly applied to these abstracts to identify various entities of interest. Subsequently, a deep syntax-based relation extraction model is used to detect binary relations between the identified entities. Finally, the resulting relations are used to populate the GENA knowledge graph, whose relationships can be accessed in an intuitive and interpretable manner using the Neo4J Database Management System. To evaluate the reliability of GENA, two annotators manually assessed a subset of the extracted relations. The evaluation results show that our methods obtain high precision for the NER task and acceptable precision and relative recall for the relation extraction task. GENA consists of 43,367 relationships that encode information about nutrition and health, of which 94.04% are new relations that are not present in existing ontologies of food and diseases. GENA is constructed based on scientific principles, and has the potential to be used within further applications to contribute towards scientific research within the domain. It is a pioneering knowledge graph in nutrition and mental health, containing a diverse range of relationship types. All of our source code and results are publicly available at https://github.com/ddlinh/gena-db.

Healthcare knowledge graph construction: A systematic review of the state-of-the-art, open issues, and opportunities

The incorporation of data analytics in the healthcare industry has made significant progress, driven by the demand for efficient and effective big data analytics solutions. Knowledge graphs (KGs) have proven utility in this arena and are rooted in a number of healthcare applications to furnish better data representation and knowledge inference. However, in conjunction with a lack of a representative KG construction taxonomy, several existing approaches in this designated domain are inadequate and inferior. This paper is the first to provide a comprehensive taxonomy and a bird's eye view of healthcare KG construction. Additionally, a thorough examination of the current state-of-the-art techniques drawn from academic works relevant to various healthcare contexts is carried out. These techniques are critically evaluated in terms of methods used for knowledge extraction, types of the knowledge base and sources, and the incorporated evaluation protocols. Finally, several research findings and existing issues in the literature are reported and discussed, opening horizons for future research in this vibrant area.

Health-Aware Food Recommendation Based on Knowledge Graph and Multi-Task Learning

Current food recommender systems tend to prioritize either the user's dietary preferences or the healthiness of the food, without considering the importance of personalized health requirements. To address this issue, we propose a novel approach to healthy food recommendations that takes into account the user's personalized health requirements, in addition to their dietary preferences. Our work comprises three perspectives. Firstly, we propose a collaborative recipe knowledge graph (CRKG) with millions of triplets, containing user-recipe interactions, recipe-ingredient associations, and other food-related information. Secondly, we define a score-based method for evaluating the healthiness match between recipes and user preferences. Based on these two prior perspectives, we develop a novel health-aware food recommendation model (FKGM) using knowledge graph embedding and multi-task learning. FKGM employs a knowledge-aware attention graph convolutional neural network to capture the semantic associations between users and recipes on the collaborative knowledge graph and learns the user's requirements in both preference and health by fusing the losses of these two learning tasks. We conducted experiments to demonstrate that FKGM outperformed four competing baseline models in integrating users' dietary preferences and personalized health requirements in food recommendations and performed best on the health task.

Food safety in health: a model of extraction for food contaminants

Contaminants are the critical targets of food safety supervision and risk assessment. In existing research, food safety knowledge graphs are used to improve the efficiency of supervision since they supply the relationship between contaminants and foods. Entity relationship extraction is one of the crucial technologies of knowledge graph construction. However, this technology still faces the issue of single entity overlap. This means that a head entity in a text description may have multiple corresponding tail entities with different relationships. To address this issue, this work proposes a pipeline model with neural networks for multiple relations enhanced entity pairs extraction. The proposed model can predict the correct entity pairs in terms of specific relations by introducing the semantic interaction between relation identification and entity extraction. We conducted various experiments on our own dataset FC and on the open public available data set DuIE2.0. The results of experiments show our model reaches the state-of-the-art, and the case study indicates our model can correctly extract entity-relationship triplets to release the problem of single entity overlap.

Hot Spots and Trends in the Relationship between Cancer and Obesity: A Systematic Review and Knowledge Graph Analysis

Cancer is one of the most difficult medical problems in today's world. There are many factors that induce cancer in humans, and obesity has become an important factor in inducing cancer. This study systematically and quantitatively describes the development trend, current situation and research hotspot of the relationship between cancer and obesity by using document statistics and knowledge graph visualization technology. Through the visualization technology analysis of knowledge graph in this study, the research hotspot and knowledge base source of the relationship between cancer and obesity in the last 20 years have been ascertained. Obesity-related factors, such as immunity, insulin, adiponectin, adipocytokines, nonalcoholic fatty liver and inflammatory reaction, may affect the occurrence of obesity and increase the risk of cancer. Obesity-related cancers include respiratory cancer, colorectal cancer, hepatocellular cancer, prostate cancer, gastric cancer, etc. Our research provides direction and basis for future research in this field, as well as technical and knowledge basis support for experts and researchers in related medical fields.

Semantics of Dairy Fermented Foods: A Microbiologist’s Perspective

Food ontologies are acquiring a central role in human nutrition, providing a standardized terminology for a proper description of intervention and observational trials. In addition to bioactive molecules, several fermented foods, particularly dairy products, provide the host with live microorganisms, thus carrying potential “genetic/functional” nutrients. To date, a proper ontology to structure and formalize the concepts used to describe fermented foods is lacking. Here we describe a semantic representation of concepts revolving around what consuming fermented foods entails, both from a technological and health point of view, focusing actions on kefir and Parmigiano Reggiano, as representatives of fresh and ripened dairy products. We included concepts related to the connection of specific microbial taxa to the dairy fermentation process, demonstrating the potential of ontologies to formalize the various gene pathways involved in raw ingredient transformation, connect them to resulting metabolites, and finally to their consequences on the fermented product, including technological, health and sensory aspects. Our work marks an improvement in the ambition of creating a harmonized semantic model for integrating different aspects of modern nutritional science. Such a model, besides formalizing a multifaceted knowledge, will be pivotal for a rich annotation of data in public repositories, as a prerequisite to generalized meta-analysis.