Validation of Sensor-Based Food Intake Detection by Multicamera Video Observation in an Unconstrained Environment

Transforming Big Data into AI-ready data for nutrition and obesity research

OBJECTIVE

Big Data are increasingly used in obesity and nutrition research to gain new insights and derive personalized guidance; however, this data in raw form are often not usable. Substantial preprocessing, which requires machine learning (ML), human judgment, and specialized software, is required to transform Big Data into artificial intelligence (AI)- and ML-ready data. These preprocessing steps are the most complex part of the entire modeling pipeline. Understanding the complexity of these steps by the end user is critical for reducing misunderstanding, faulty interpretation, and erroneous downstream conclusions.

METHODS

We reviewed three popular obesity/nutrition Big Data sources: microbiome, metabolomics, and accelerometry. The preprocessing pipelines, specialized software, challenges, and how decisions impact final AI- and ML-ready products were detailed.

RESULTS

Opportunities for advances to improve quality control, speed of preprocessing, and intelligent end user consumption were presented.

CONCLUSIONS

Big Data have the exciting potential for identifying new modifiable factors that impact obesity research. However, to ensure accurate interpretation of conclusions arising from Big Data, the choices involved in preparing AI- and ML-ready data need to be transparent to investigators and clinicians relying on the conclusions.

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.

Vision-Based Methods for Food and Fluid Intake Monitoring: A Literature Review

Food and fluid intake monitoring are essential for reducing the risk of dehydration, malnutrition, and obesity. The existing research has been preponderantly focused on dietary monitoring, while fluid intake monitoring, on the other hand, is often neglected. Food and fluid intake monitoring can be based on wearable sensors, environmental sensors, smart containers, and the collaborative use of multiple sensors. Vision-based intake monitoring methods have been widely exploited with the development of visual devices and computer vision algorithms. Vision-based methods provide non-intrusive solutions for monitoring. They have shown promising performance in food/beverage recognition and segmentation, human intake action detection and classification, and food volume/fluid amount estimation. However, occlusion, privacy, computational efficiency, and practicality pose significant challenges. This paper reviews the existing work (253 articles) on vision-based intake (food and fluid) monitoring methods to assess the size and scope of the available literature and identify the current challenges and research gaps. This paper uses tables and graphs to depict the patterns of device selection, viewing angle, tasks, algorithms, experimental settings, and performance of the existing monitoring systems.

Dietary Behavior Assessments in Children-A Mixed-Method Research Exploring the Perspective of Pediatric Dieticians on Innovative Technologies

Background

Assessing dietary intake and eating behavior in children is challenging, owing to children's undeveloped food knowledge and perception of portion sizes. Additionally, caregivers cannot always provide complete surrogate information. Consequently, validated dietary behavior assessment methods for children are limited, but technological innovations offer opportunities for the development of new tools. One of the first steps in the developmental process of a newly developed pediatric dietary assessment tool includes an alignment of the needs and preferences of pediatric dieticians (PDs) as potential users.

Objectives

To explore opinions of Dutch PDs about traditional dietary behavior assessment methods for children and potential technological innovations to replace or support traditional methods.

Methods

Ten PDs participated in semistructured interviews (total of 7.5 h) based on 2 theoretical frameworks, and data saturation was reached after the seventh interview. Interview transcripts were inductively coded in an iterative process, and overarching themes and domains were identified. Interview data were then used as input for an extensive online survey completed by 31 PDs who were not involved in the initial interview rounds.

Results

PDs discussed their perspective on dietary behavior assessments in 4 domains: traditional methods, technological methods, future methods, and external influences on these methods. Generally, PDs felt that traditional methods supported them in reaching their desired goals. However, the time needed to obtain a comprehensive overview of dietary intake behavior and the reliability of conventional methods were mentioned as limitations. For future technologies, PDs mention the ease of use and engaging in children as opportunities.

Conclusions

PDs have a positive attitude toward the use of technology for dietary behavior assessments. Further development of assessment technologies should be tailored to the needs of children in different care situations and age categories to increase its usability among children, their caregivers, and dietician. Curr Dev Nutr 2023;xx:xx.

Capturing Eating Behavior from Video Analysis: A Systematic Review

Current methods to detect eating behavior events (i.e., bites, chews, and swallows) lack objective measurements, standard procedures, and automation. The video recordings of eating episodes provide a non-invasive and scalable source for automation. Here, we reviewed the current methods to automatically detect eating behavior events from video recordings. According to PRISMA guidelines, publications from 2010-2021 in PubMed, Scopus, ScienceDirect, and Google Scholar were screened through title and abstract, leading to the identification of 277 publications. We screened the full text of 52 publications and included 13 for analysis. We classified the methods in five distinct categories based on their similarities and analyzed their accuracy. Facial landmarks can count bites, chews, and food liking automatically (accuracy: 90%, 60%, 25%). Deep neural networks can detect bites and gesture intake (accuracy: 91%, 86%). The active appearance model can detect chewing (accuracy: 93%), and optical flow can count chews (accuracy: 88%). Video fluoroscopy can track swallows but is currently not suitable beyond clinical settings. The optimal method for automated counts of bites and chews is facial landmarks, although further improvements are required. Future methods should accurately predict bites, chews, and swallows using inexpensive hardware and limited computational capacity. Automatic eating behavior analysis will allow the study of eating behavior and real-time interventions to promote healthy eating behaviors.

Improved Wearable Devices for Dietary Assessment Using a New Camera System

An unhealthy diet is strongly linked to obesity and numerous chronic diseases. Currently, over two-thirds of American adults are overweight or obese. Although dietary assessment helps people improve nutrition and lifestyle, traditional methods for dietary assessment depend on self-report, which is inaccurate and often biased. In recent years, as electronics, information, and artificial intelligence (AI) technologies advanced rapidly, image-based objective dietary assessment using wearable electronic devices has become a powerful approach. However, research in this field has been focused on the developments of advanced algorithms to process image data. Few reports exist on the study of device hardware for the particular purpose of dietary assessment. In this work, we demonstrate that, with the current hardware design, there is a considerable risk of missing important dietary data owing to the common use of rectangular image screen and fixed camera orientation. We then present two designs of a new camera system to reduce data loss by generating circular images using rectangular image sensor chips. We also present a mechanical design that allows the camera orientation to be adjusted, adapting to differences among device wearers, such as gender, body height, and so on. Finally, we discuss the pros and cons of rectangular versus circular images with respect to information preservation and data processing using AI algorithms.

Thought on Food: A Systematic Review of Current Approaches and Challenges for Food Intake Detection

Nowadays, individuals have very stressful lifestyles, affecting their nutritional habits. In the early stages of life, teenagers begin to exhibit bad habits and inadequate nutrition. Likewise, other people with dementia, Alzheimer's disease, or other conditions may not take food or medicine regularly. Therefore, the ability to monitor could be beneficial for them and for the doctors that can analyze the patterns of eating habits and their correlation with overall health. Many sensors help accurately detect food intake episodes, including electrogastrography, cameras, microphones, and inertial sensors. Accurate detection may provide better control to enable healthy nutrition habits. This paper presents a systematic review of the use of technology for food intake detection, focusing on the different sensors and methodologies used. The search was performed with a Natural Language Processing (NLP) framework that helps screen irrelevant studies while following the PRISMA methodology. It automatically searched and filtered the research studies in different databases, including PubMed, Springer, ACM, IEEE Xplore, MDPI, and Elsevier. Then, the manual analysis selected 30 papers based on the results of the framework for further analysis, which support the interest in using sensors for food intake detection and nutrition assessment. The mainly used sensors are cameras, inertial, and acoustic sensors that handle the recognition of food intake episodes with artificial intelligence techniques. This research identifies the most used sensors and data processing methodologies to detect food intake.

Masticatory Behaviors and Gender Differences in People with Obesity as Measured via an Earphone-Style Light-Sensor-Based Mastication Meter

While people with obesity have been found to chew fewer times and for shorter durations, few studies have quantitatively evaluated mastication among this group. This study examined the relationship between the mastication characteristics of people with obesity and the factors correlated with obesity. To this end, 46 people with obesity and 41 healthy participants placed an earphone-style light sensor in the aperture of their outer ear. We also examined the partial correlation between this, their body composition, and various biochemical markers by gender. A two-way analysis of variance (ANOVA) regarding the masticatory index, gender, and the presence/absence of obesity for all three food items revealed the main effects in the gender difference and the presence/absence of obesity. Additionally, the number of times the salad was chewed showed an interaction between the gender and the presence/absence of obesity. In the BMI-corrected partial correlation analysis of the chewing index and the glucose/lipid metabolism index, the chewing time and the number of chews of all the food items negatively correlated with hemoglobin A1c(HbA1c), fasting plasma glucose (FPG), immunoreactive insulin (IRI), and homeostasis model assessment of insulin resistance (HOMA-R) in the female obese group. These findings might be used in weight-loss interventions for men with obesity and treatments that target the metabolic function among women with obesity.

Enhancing Nutrition Care Through Real-Time, Sensor-Based Capture of Eating Occasions: A Scoping Review

As food intake patterns become less structured, different methods of dietary assessment may be required to capture frequently omitted snacks, smaller meals, and the time of day when they are consumed. Incorporating sensors that passively and objectively detect eating behavior may assist in capturing these eating occasions into dietary assessment methods. The aim of this study was to identify and collate sensor-based technologies that are feasible for dietitians to use to assist with performing dietary assessments in real-world practice settings. A scoping review was conducted using the PRISMA extension for scoping reviews (PRISMA-ScR) framework. Studies were included if they were published between January 2016 and December 2021 and evaluated the performance of sensor-based devices for identifying and recording the time of food intake. Devices from included studies were further evaluated against a set of feasibility criteria to determine whether they could potentially be used to assist dietitians in conducting dietary assessments. The feasibility criteria were, in brief, consisting of an accuracy ≥80%; tested in settings where subjects were free to choose their own foods and activities; social acceptability and comfort; a long battery life; and a relatively rapid detection of an eating episode. Fifty-four studies describing 53 unique devices and 4 device combinations worn on the wrist (n = 18), head (n = 16), neck (n = 9), and other locations (n = 14) were included. Whilst none of the devices strictly met all feasibility criteria currently, continuous refinement and testing of device software and hardware are likely given the rapidly changing nature of this emerging field. The main reasons devices failed to meet the feasibility criteria were: an insufficient or lack of reporting on battery life (91%), the use of a limited number of foods and behaviors to evaluate device performance (63%), and the device being socially unacceptable or uncomfortable to wear for long durations (46%). Until sensor-based dietary assessment tools have been designed into more inconspicuous prototypes and are able to detect most food and beverage consumption throughout the day, their use will not be feasible for dietitians in practice settings.

Validation of an Automated Wearable Camera-Based Image-Assisted Recall Method and the 24-h Recall Method for Assessing Women's Time Allocation in a Nutritionally Vulnerable Population: The Case of Rural Uganda

Accurate data are essential for investigating relationships between maternal time-use patterns and nutritional outcomes. The 24 h recall (24HR) has traditionally been used to collect time-use data, however, automated wearable cameras (AWCs) with an image-assisted recall (IAR) may reduce recall bias. This study aimed to evaluate their concurrent criterion validity for assessing women’s time use in rural Eastern Ugandan. Women’s (n = 211) time allocations estimated via the AWC-IAR and 24HR methods were compared with direct observation (criterion method) using the Bland–Altman limits of agreement (LOA) method of analysis and Cronbach’s coefficient alpha (time allocation) or Cohen’s κ (concurrent activities). Systematic bias varied from 1 min (domestic chores) to 226 min (caregiving) for 24HR and 1 min (own production) to 109 min (socializing) for AWC-IAR. The LOAs were within 2 h for employment, own production, and self-care for 24HR and AWC-IAR but exceeded 11 h (24HR) and 9 h (AWC-IAR) for caregiving and socializing. The LOAs were within four concurrent activities for 24HR (−1.1 to 3.7) and AWC-IAR (−3.2 to 3.2). Cronbach’s alpha for time allocation ranged from 0.1728 (socializing) to 0.8056 (own production) for 24HR and 0.2270 (socializing) to 0.7938 (own production) for AWC-IAR. For assessing women’s time allocations at the population level, the 24HR and AWC-IAR methods are accurate and reliable for employment, own production, and domestic chores but poor for caregiving and socializing. The results of this study suggest the need to revisit previously published research investigating the associations between women’s time allocations and nutrition outcomes.

Assessing temporal eating pattern in free living humans through the myCircadianClock app

The quality and quantity of nutrition impact health. However, chrononutrition, the timing, and variation of food intake in relation to the daily sleep-wake cycle are also important contributors to health. This has necessitated an urgent need to measure, analyze, and optimize eating patterns to improve health and manage disease. While written food journals, questionnaires, and 24-hour dietary recalls are acceptable methods to assess the quantity and quality of energy consumption, they are insufficient to capture the timing and day-to-day variation of energy intake. Smartphone applications are novel methods for information-dense real-time food and beverage tracking. Despite the availability of thousands of commercial nutrient apps, they almost always ignore eating patterns, and the raw real-time data is not available to researchers for monitoring and intervening in eating patterns. Our lab developed a smartphone app called myCircadianClock (mCC) and associated software to enable long-term real-time logging that captures temporal components of eating patterns. The mCC app runs on iOS and android operating systems and can be used to track multiple cohorts in parallel studies. The logging burden is decreased by using a timestamped photo and annotation of the food/beverage being logged. Capturing temporal data of consumption in free-living individuals over weeks/months has provided new insights into diverse eating patterns in the real world. This review discusses (1) chrononutrition and the importance of understanding eating patterns, (2) the myCircadianClock app, (3) validation of the mCC app, (4) clinical trials to assess the timing of energy intake, and (5) strengths and limitations of the mCC app.

Impact of Masticatory Behaviors Measured With Wearable Device on Metabolic Syndrome: Cross-sectional Study

Background

It has been widely recognized that mastication behaviors are related to the health of the whole body and to lifestyle-related diseases. However, many studies were based on subjective questionnaires or were limited to small-scale research in the laboratory due to the lack of a device for measuring mastication behaviors during the daily meal objectively. Recently, a small wearable masticatory counter device, called bitescan (Sharp Co), for measuring masticatory behavior was developed. This wearable device is designed to assess objective masticatory behavior by being worn on the ear in daily life.

Objective

This study aimed to investigate the relation between mastication behaviors in the laboratory and in daily meals and to clarify the difference in mastication behaviors between those with metabolic syndrome (MetS) and those without (non-MetS) measured using a wearable device.

Methods

A total of 99 healthy volunteers (50 men and 49 women, mean age 36.4 [SD 11.7] years) participated in this study. The mastication behaviors (ie, number of chews and bites, number of chews per bite, and chewing rate) were measured using a wearable ear-hung device. Mastication behaviors while eating a rice ball (100 g) in the laboratory and during usual meals for an entire day were monitored, and the daily energy intake was calculated. Participants’ abdominal circumference, fasting glucose concentration, blood pressure, and serum lipids were also measured. Mastication behaviors in the laboratory and during meals for 1 entire day were compared. The participants were divided into 2 groups using the Japanese criteria for MetS (positive/negative for MetS or each MetS component), and mastication behaviors were compared.

Results

Mastication behaviors in the laboratory and during daily meals were significantly correlated (number of chews r=0.36; P<.001; number of bites r=0.49; P<.001; number of chews per bite r=0.33; P=.001; and chewing rate r=0.51; P<.001). Although a positive correlation was observed between the number of chews during the 1-day meals and energy intake (r=0.26, P=.009), the number of chews per calorie ingested was negatively correlated with energy intake (r=–0.32, P=.002). Of the 99 participants, 8 fit the criteria for MetS and 14 for pre-MetS. The number of chews and bites for a rice ball in the pre-MetS(+) group was significantly lower than the pre-MetS(–) group (P=.02 and P=.04, respectively). Additionally, scores for the positive abdominal circumference and hypertension subgroups were also less than the counterpart groups (P=.004 and P=.01 for chews, P=.006 and P=.02 for bites, respectively). The number of chews and bites for an entire day in the hypertension subgroup were significantly lower than in the other groups (P=.02 and P=.006). Furthermore, the positive abdominal circumference and hypertension subgroups showed lower numbers of chews per calorie ingested for 1-day meals (P=.03 and P=.02, respectively).

Conclusions

These results suggest a relationship between masticatory behaviors in the laboratory and those during daily meals and that masticatory behaviors are associated with MetS and MetS components.

Trial Registration

University Hospital Medical Information Network Clinical Trials Registry R000034453; https://tinyurl.com/mwzrhrua

Microstructural changes in human ingestive behavior after Roux-en-Y gastric bypass during liquid meals

BACKGROUND

Roux-en-Y gastric bypass (RYGB) decreases energy intake and is, therefore, an effective treatment of obesity. The behavioral bases of the decreased calorie intake remain to be elucidated. We applied the methodology of microstructural analysis of meal intake to establish the behavioral features of ingestion in an effort to discern the various controls of feeding as a function of RYGB.

METHODS

The ingestive microstructure of a standardized liquid meal in a cohort of 11 RYGB patients, in 10 patients with obesity, and in 10 healthy-weight adults was prospectively assessed from baseline to 1 year with a custom-designed drinkometer. Statistics were performed on log-transformed ratios of change from baseline so that each participant served as their own control, and proportional increases and decreases were numerically symmetrical. Data-driven (3 seconds) and additional burst pause criteria (1 and 5 seconds) were used.

RESULTS

At baseline, the mean meal size (909.2 versus 557.6 kCal), burst size (28.8 versus 17.6 mL), and meal duration (433 versus 381 seconds) differed between RYGB patients and healthy-weight controls, whereas suck volume (5.2 versus 4.6 mL) and number of bursts (19.7 versus 20.1) were comparable. At 1 year, the ingestive differences between the RYGB and healthy-weight groups disappeared due to significantly decreased burst size (P = 0.008) and meal duration (P = 0.034) after RYGB. The first-minute intake also decreased after RYGB (P = 0.022).

CONCLUSION

RYGB induced dynamic changes in ingestive behavior over the first postoperative year. While the eating pattern of controls remained stable, RYGB patients reduced their meal size by decreasing burst size and meal duration, suggesting that increased postingestive sensibility may mediate postbariatric ingestive behavior.

TRIAL REGISTRATION

NCT03747445; https://clinicaltrials.gov/ct2/show/NCT03747445.

FUNDING

This work was supported by the University of Zurich, the Swiss National Fund (32003B_182309), and the Olga Mayenfisch Foundation. Bálint File was supported by the Hungarian Brain Research Program Grant (grant no. 2017-1.2.1-NKP-2017-00002).

Short and Long-Term Innovations on Dietary Behavior Assessment and Coaching: Present Efforts and Vision of the Pride and Prejudice Consortium

Overweight, obesity and cardiometabolic diseases are major global health concerns. Lifestyle factors, including diet, have been acknowledged to play a key role in the solution of these health risks. However, as shown by numerous studies, and in clinical practice, it is extremely challenging to quantify dietary behaviors as well as influencing them via dietary interventions. As shown by the limited success of 'one-size-fits-all' nutritional campaigns catered to an entire population or subpopulation, the need for more personalized coaching approaches is evident. New technology-based innovations provide opportunities to further improve the accuracy of dietary assessment and develop approaches to coach individuals towards healthier dietary behaviors. Pride & Prejudice (P&P) is a unique multi-disciplinary consortium consisting of researchers in life, nutrition, ICT, design, behavioral and social sciences from all four Dutch Universities of Technology. P&P focuses on the development and integration of innovative technological techniques such as artificial intelligence (AI), machine learning, conversational agents, behavior change theory and personalized coaching to improve current practices and establish lasting dietary behavior change.

Occurrence of and Reasons for "Missing Events" in Mobile Dietary Assessments: Results From Three Event-Based Ecological Momentary Assessment Studies

Background

Establishing a methodology for assessing nutritional behavior comprehensively and accurately poses a great challenge. Mobile technologies such as mobile image-based food recording apps enable eating events to be assessed in the moment in real time, thereby reducing memory biases inherent in retrospective food records. However, users might find it challenging to take images of the food they consume at every eating event over an extended period, which might lead to incomplete records of eating events (missing events).

Objective

Analyzing data from 3 studies that used mobile image-based food recording apps and varied in their technical enrichment, this study aims to assess how often eating events (meals and snacks) were missed over a period of 8 days in a naturalistic setting by comparing the number of recorded events with the number of normative expected events, over time, and with recollections of missing events.

Methods

Participants in 3 event-based Ecological Momentary Assessment (EMA) studies using mobile image-based dietary assessments were asked to record all eating events (study 1, N=38, 1070 eating events; study 2, N=35, 934 eating events; study 3, N=110, 3469 eating events). Study 1 used a basic app; study 2 included 1 fixed reminder and the possibility to add meals after the actual eating events occurred instead of in the moment (addendum); and study 3 included 2 fixed reminders, an addendum feature, and the option to record skipped meals. The number of recalled missed events and their reasons were assessed by semistructured interviews after the EMA period (studies 1 and 2) and daily questionnaires (study 3).

Results

Overall, 183 participants reported 5473 eating events. Although the momentary adherence rate as indexed by a comparison with normative expected events was generally high across all 3 studies, a differential pattern of results emerged with a higher rate of logged meals in the more technically intensive study 3. Multilevel models for the logging trajectories of reported meals in all 3 studies showed a significant, albeit small, decline over time (b=−.11 to −.14, Ps<.001, pseudo-R²=0.04-0.06), mainly because of a drop in reported snacks between days 1 and 2. Intraclass coefficients indicated that 38% or less of the observed variance was because of individual differences. The most common reasons for missing events were competing activities and technical issues, whereas situational barriers were less important.

Conclusions

Three different indicators (normative, time stability, and recalled missing events) consistently indicated missing events. However, given the intensive nature of diet EMA protocols, the effect sizes were rather small and the logging trajectories over time were remarkably stable. Moreover, the individual’s actual state and context seemed to exert a greater influence on adherence rates than stable individual differences, which emphasizes the need for a more nuanced understanding of the factors that affect momentary adherence.

Validation of a life-logging wearable camera method and the 24-h diet recall method for assessing maternal and child dietary diversity

Accurate and timely data are essential for identifying populations at risk for undernutrition due to poor-quality diets, for implementing appropriate interventions and for evaluating change. Life-logging wearable cameras (LLWC) have been used to prospectively capture food/beverage consumed by adults in high-income countries. This study aimed to evaluate the concurrent criterion validity, for assessing maternal and child dietary diversity scores (DDS), of a LLWC-based image-assisted recall (IAR) and 24-h recall (24HR). Direct observation was the criterion method. Food/beverage consumption of rural Eastern Ugandan mothers and their 12-23-month-old child (n 211) was assessed, for the same day for each method, and the IAR and 24HR DDS were compared with the weighed food record DDS using the Bland-Altman limits of agreement (LOA) method of analysis and Cohen's κ. The relative bias was low for the 24HR (-0·1801 for mothers; -0·1358 for children) and the IAR (0·1227 for mothers; 0·1104 for children), but the LOA were wide (-1·6615 to 1·3012 and -1·6883 to 1·4167 for mothers and children via 24HR, respectively; -2·1322 to 1·8868 and -1·7130 to 1·4921 for mothers and children via IAR, respectively). Cohen's κ, for DDS via 24HR and IAR, was 0·68 and 0·59, respectively, for mothers, and 0·60 and 0·59, respectively, for children. Both the 24HR and IAR provide an accurate estimate of median dietary diversity, for mothers and their young child, but non-differential measurement error would attenuate associations between DDS and outcomes, thereby under-estimating the true associations between DDS - where estimated via 24HR or IAR - and outcomes measured.

Automatic Count of Bites and Chews From Videos of Eating Episodes

Methods for measuring of eating behavior (known as meal microstructure) often rely on manual annotation of bites, chews, and swallows on meal videos or wearable sensor signals. The manual annotation may be time consuming and erroneous, while wearable sensors may not capture every aspect of eating (e.g. chews only). The aim of this study is to develop a method to detect and count bites and chews automatically from meal videos. The method was developed on a dataset of 28 volunteers consuming unrestricted meals in the laboratory under video observation. First, the faces in the video (regions of interest, ROI) were detected using Faster R-CNN. Second, a pre-trained AlexNet was trained on the detected faces to classify images as a bite/no bite image. Third, the affine optical flow was applied in consecutively detected faces to find the rotational movement of the pixels in the ROIs. The number of chews in a meal video was counted by converting the 2-D images to a 1-D optical flow parameter and finding peaks. The developed bite and chew count algorithm was applied to 84 meal videos collected from 28 volunteers. A mean accuracy (±STD) of 85.4% (±6.3%) with respect to manual annotation was obtained for the number of bites and 88.9% (±7.4%) for the number of chews. The proposed method for an automatic bite and chew counting shows promising results that can be used as an alternative solution to manual annotation.

Challenges in the interpretation and therapeutic manipulation of human ingestive microstructure

This minireview focuses on the interpretative value of ingestive microstructure by summarizing observations from both rodent and human studies. Preliminary data on the therapeutic manipulation of distinct microstructural components of eating are also outlined. In rodents, the interpretative framework of ingestive microstructure mainly concentrates on deprivation state, palatability, satiation, and the role of learning from previous experiences. In humans, however, the control of eating is further influenced by genetic, psychosocial, cultural, and environmental factors, which add complexity and challenges to the interpretation of the microstructure of meal intake. Nevertheless, the presented findings stress the importance of microstructural analyses of ingestion, as a method to investigate specific behavioral variables that underlie the regulation of appetite control.

The development of food image detection and recognition model of Korean food for mobile dietary management

BACKGROUND/OBJECTIVES

The aim of this study was to develop Korean food image detection and recognition model for use in mobile devices for accurate estimation of dietary intake.

SUBJECTS/METHODS

We collected food images by taking pictures or by searching web images and built an image dataset for use in training a complex recognition model for Korean food. Augmentation techniques were performed in order to increase the dataset size. The dataset for training contained more than 92,000 images categorized into 23 groups of Korean food. All images were down-sampled to a fixed resolution of 150 × 150 and then randomly divided into training and testing groups at a ratio of 3:1, resulting in 69,000 training images and 23,000 test images. We used a Deep Convolutional Neural Network (DCNN) for the complex recognition model and compared the results with those of other networks: AlexNet, GoogLeNet, Very Deep Convolutional Neural Network, VGG and ResNet, for large-scale image recognition.

RESULTS

Our complex food recognition model, K-foodNet, had higher test accuracy (91.3%) and faster recognition time (0.4 ms) than those of the other networks.

CONCLUSION

The results showed that K-foodNet achieved better performance in detecting and recognizing Korean food compared to other state-of-the-art models.

Systematic Literature Review of Food-Intake Monitoring in an Aging Population

The dietary habits of people directly impact their health conditions. Especially in elder populations (in 2017, 6.7% of the world's population was over 65 years of age), these habits could lead to important-nutrient losses that could seriously affect their cognitive and functional state. Recently, a great research effort has been devoted to using different technologies and proposing different techniques for monitoring food-intake. Nevertheless, these techniques are usually generic but make use of the most innovative technologies and methodologies to obtain the best possible monitoring results. However, a large percentage of elderly people live in depopulated rural areas (in Spain, 28.1% of the elderly population lives in this type of area) with a fragile cultural and socioeconomic context. The use of these techniques in these environments is crucial to improving this group's quality of life (and even reducing their healthcare expenses). At the same time, it is especially challenging since they have very specific and strict requirements regarding the use and application of technology. In this Systematic Literature Review (SLR), we analyze the most important proposed technologies and techniques in order to identify whether they can be applied in this context and if they can be used to improve the quality of life of this fragile collective. In this SLR, we have analyzed 326 papers. From those, 29 proposals have been completely analyzed, taking into account the characteristics and requirements of this population.