Feasibility of smartphone colorimetry of the face as an anaemia screening tool for infants and young children in Ghana

Machine learning of blood haemoglobin and haematocrit levels via smartphone conjunctiva photography in Kenyan pregnant women: a clinical study protocol

INTRODUCTION

Anaemia during pregnancy is a widespread health burden globally, especially in low- and middle-income countries, posing a serious risk to both maternal and neonatal health. The primary challenge is that anaemia is frequently undetected or is detected too late, worsening pregnancy complications. The gold standard for diagnosing anaemia is a clinical laboratory blood haemoglobin (Hgb) or haematocrit (Hct) test involving a venous blood draw. However, this approach presents several challenges in resource-limited settings regarding accessibility and feasibility. Although non-invasive blood Hgb testing technologies are gaining attention, they remain limited in availability, affordability and practicality. This study aims to develop and validate a mobile health (mHealth) machine learning model to reliably predict blood Hgb and Hct levels in Black African pregnant women using smartphone photos of the conjunctiva.

METHODS AND ANALYSIS

This is a single-centre, cross-sectional and observational study, leveraging existing antenatal care services for pregnant women aged 15 to 49 years in Kenya. The study involves collecting smartphone photos of the conjunctiva alongside conventional blood Hgb tests. Relevant clinical data related to each participant's anaemia status will also be collected. The photo acquisition protocol will incorporate diverse scenarios to reflect real-world variability. A clinical training dataset will be used to refine a machine learning model designed to predict blood Hgb and Hct levels from smartphone images of the conjunctiva. Using a separate testing dataset, comprehensive analyses will assess its performance by comparing predicted blood Hgb and Hct levels with clinical laboratory and/or finger-prick readings.

ETHICS AND DISSEMINATION

This study is approved by the Moi University Institutional Research and Ethics Committee (Reference: IREC/585/2023 and Approval Number: 004514), Kenya's National Commission for Science, Technology, and Innovation (NACOSTI Reference: 491921) and Purdue University's Institutional Review Board (Protocol Number: IRB-2023-1235). Participants will include emancipated or mature minors. In Kenya, pregnant women aged 15 to 18 years are recognised as emancipated or mature minors, allowing them to provide informed consent independently. The study poses minimal risk to participants. Findings and results will be disseminated through submissions to peer-reviewed journals and presentations at the participating institutions, including Moi Teaching and Referral Hospital and Kenya's Ministry of Health. On completion of data collection and modelling, this study will demonstrate how machine learning-driven mHealth technologies can reduce reliance on clinical laboratories and complex equipment, offering accessible and scalable solutions for resource-limited and at-home settings.

Diagnosing anaemia via smartphone colorimetry of the eye in a population of pregnant women

Objective.Screening for disease using a smartphone camera is an emerging tool for conditions such as jaundice and anaemia, which are associated with a colour change (yellowing in jaundice; pallor in anaemia) of the external tissues. Based on this, we aimed to test a technique to non-invasively screen for anaemia in a population highly affected by anaemia: pregnant women in India. In this group, anaemia can have severe health consequences for both the mother and child.Approach.Over 3 years of data collection, in 486 pregnant women in India, we attempted to replicate a previously successful smartphone imaging technique to screen for anaemia. Using smartphone images of the eye and eyelid, we compared two techniques (white balancing and ambient subtraction) to control for variation in ambient lighting, and then extracted 'redness' features from images, which we used as features to predict anaemia via statistical modelling.Main results.We found that we were not able to predict anaemia with enough accuracy to be clinically useful, at 89.6% sensitivity and 26.1% specificity. We consider the hypothesis that this may be due to pigmentation on the sclera and palpebral conjunctiva. Visual judgement showed that pigmentation on the sclera, which may affect the measured colour, is more prevalent in pregnant women in India than in preschool aged children in Ghana (a population previously studied in this context). When participants with subjectively judged visible scleral pigmentation are removed, ability to screen for anaemia using the smartphone images slightly improves (93.1% sensitivity, 28.6% specificity).Significance.These findings provide evidence to reinforce that applying smartphone imaging techniques to understudied populations in the real world requires caution-a promising result in one group may not necessarily transfer to another demographic.

Machine/deep learning-assisted hemoglobin level prediction using palpebral conjunctival images

Palpebral conjunctival hue alteration is used in non-invasive screening for anaemia, whereas it is a qualitative measure. This study constructed machine/deep learning models for predicting haemoglobin values using 150 palpebral conjunctival images taken by a smartphone. The median haemoglobin value was 13.1 g/dL, including 10 patients with <11 g/dL. A segmentation model using U-net was successfully constructed. The segmented images were subjected to non-convolutional neural network (CNN)-based and CNN-based regression models for predicting haemoglobin values. The correlation coefficients between the actual and predicted haemoglobin values were 0.38 and 0.44 in the non-CNN-based and CNN-based models, respectively. The sensitivity and specificity for anaemia detection were 13% and 98% for the non-CNN-based model and 20% and 99% for the CNN-based model. The performance of the CNN-based model did not improve with a mask layer guiding the model's attention towards the conjunctival regions, however, slightly improved with correction by the aspect ratio and exposure time of input images. The gradient-weighted class activation mapping heatmap indicated that the lower half area of the conjunctiva was crucial for haemoglobin value prediction. In conclusion, the CNN-based model had better results than the non-CNN-based model. The prediction accuracy would improve by using more input data with anaemia.

Prediction of anemia in real-time using a smartphone camera processing conjunctival images

Anemia is defined as a low hemoglobin (Hb) concentration and is highly prevalent worldwide. We report on the performance of a smartphone application (app) that records images in RAW format of the palpebral conjunctivae and estimates Hb concentration by relying upon computation of the tissue surface high hue ratio. Images of bilateral conjunctivae were obtained prospectively from a convenience sample of 435 Emergency Department patients using a dedicated smartphone. A previous computer-based and validated derivation data set associating estimated conjunctival Hb (HBc) and the actual laboratory-determined Hb (HBl) was used in deriving Hb estimations using a self-contained mobile app. Accuracy of HBc was 75.4% (95% CI 71.3, 79.4%) for all categories of anemia, and Bland-Altman plot analysis showed a bias of 0.10 and limits of agreement (LOA) of (-4.73, 4.93 g/dL). Analysis of HBc estimation accuracy around different anemia thresholds showed that AUC was maximized at transfusion thresholds of 7 and 9 g/dL which showed AUC values of 0.92 and 0.90 respectively. We found that the app is sufficiently accurate for detecting severe anemia and shows promise as a population-sourced screening platform or as a non-invasive point-of-care anemia classifier.

Use of point-of-care haemoglobin tests to diagnose childhood anaemia in low- and middle-income countries: A systematic review

OBJECTIVES

Anaemia is a major cause of mortality and transfusion in children in low- and middle-income countries (LMICs); however, current diagnostics are slow, costly and frequently unavailable. Point-of-care haemoglobin tests (POC(Hb)Ts) could improve patient outcomes and use of resources by providing rapid and affordable results. We systematically reviewed the literature to investigate what, where and how POC(Hb)Ts are being used by health facilities in LMICs to diagnose childhood anaemia, and to explore challenges to their use.

METHODS

We searched a total of nine databases and trial registries up to 10 June 2022 using the concepts: anaemia, POC(Hb)T, LMIC and clinical setting. Adults ≥21 years and literature published >15 years ago were excluded. A single reviewer conducted screening, data extraction and quality assessment (of diagnostic studies) using QUADAS-2. Outcomes including POC(Hb)T used, location, setting, challenges and diagnostic accuracy were synthesised.

RESULTS

Of 626 records screened, 41 studies were included. Evidence is available on the use of 15 POC(Hb)Ts in hospitals (n = 28, 68%), health centres (n = 9, 22%) and clinics/units (n = 10, 24%) across 16 LMICs. HemoCue (HemoCue AB, Ängelholm, Sweden) was the most used test (n = 31, 76%). Key challenges reported were overestimation of haemoglobin concentration, clinically unacceptable limits of agreement, errors/difficulty in sampling, environmental factors, cost, inter-observer variability and supply of consumables. Five POC(Hb)Ts (33%) could not detect haemoglobin levels below 4.5 g/dL. Diagnostic accuracy varied, with sensitivity and specificity to detect anaemia ranging from 24.2% to 92.2% and 70% to 96.7%, respectively.

CONCLUSIONS

POC(Hb)Ts have been successfully utilised in health facilities in LMICs to diagnose childhood anaemia. However, limited evidence is available, and challenges exist that must be addressed before wider implementation. Further research is required to confirm accuracy, clinical benefits and cost-effectiveness.

Quantifying colors at micrometer scale by colorimetric microscopy (C-Microscopy) approach

The color is the primal property of the objects around us and is direct manifestation of light-matter interactions. The color information is used in many different fields of science, technology and industry to investigate material properties or for identification of concentrations of substances. Usually the color information is used as a global parameter in a macro scale. To quantitatively measure color information in micro scale one needs to use dedicated microscope spectrophotometers or specialized micro-reflectance setups. Here, the Colorimetric Microscopy (C-Microscopy) approach based on digital optical microscopy and a free software is presented. The C-Microscopy approach uses color calibrated image and colorimetric calculations to obtain physically meaningful quantities i.e., dominant wavelength and excitation purity maps at micro level scale. This allows for the discovery of the local color details of samples surfaces. Later, to fully characterize the optical properties, the hyperspectral reflectance data at micro scale (reflectance as a function of wavelength for a each point) are colorimetrically recovered. The C-Microscopy approach was successfully applied to various types of samples i.e., two metamorphic rocks unakite and lapis lazuli, which are mixtures of different minerals; and to the surface of gold 99.999 % pellet, which exhibits different types of surface features. The C-Microscopy approach could be used to quantify the local optical properties changes of various materials at microscale in an accessible way. The approach is freely available as a set of python jupyter notebooks.

Artificial intelligence-enabled non-invasive ubiquitous anemia screening: The HEMO-AI pilot study on pediatric population

Objective

Determine whether data collected from a smartphone camera can be used to detect anemia in a pediatric population.

Methods

HEMO-AI (Hemoglobin Easy Measurement by Optical Artificial Intelligence), a clinical study carried out from December 2020 to February 2023, recruited patients from the Pediatric Emergency Department, Pediatric Inpatient Department and Pediatric Hematology Unit of the Haemek Medical Center, Afula, Israel. A population-based sample of 823 patients aged 6 months to 18 years who had undergone a venous blood draw for a complete blood count since being admitted to the hospital were enrolled. Patients with total leukonychia, nailbed darkening or discoloration due to medication, nail clubbing, clinically indicated jaundice, subungual hematoma, nailbed lacerations, avulsion injuries, or nail polish applied on fingernails were not eligible for study recruitment. Video and images of the patients' hand placed in a collection chamber were collected using a smartphone camera.

Results

About 823 samples, 531 from a 12.2 megapixel camera and 256 from a 12.2 megapixel camera, were collected. About 26 samples were excluded by the study coordinator for irregularities. About 97% of fingernails and 68% of skin samples were successfully identified by a post-trained machine learning model. Separate models built to detect anemia using images taken from the Pixel 3 had an average precision of 0.64 and an average recall of 0.4, whereas models built using the Pixel 6 had an average precision of 0.8 and an average recall of 0.84. Further supplementation of training data with synthetic data boosted the precision of the latter to 0.84 and the average recall to 0.87.

Conclusions

This study lays the groundwork for the future evolution of non-invasive, pain-free, and accessible anemia screening tools tailored specifically for pediatric patients. It identifies important sample collection parameters and design, provides critical algorithms for the pre-processing of fingernail data, and reports an initial capability to detect anemia with 87% sensitivity and 84% specificity.

Trial Registration

Prospectively registered on www.clinicaltrials.gov (Identifier: NCT04573244) on 15 September 2020, prior to subject recruitment.

Assessment of bilirubin levels in patients with cirrhosis via forehead, sclera and lower eyelid smartphone images

One of the key biomarkers evaluating liver disease progression is an elevated bilirubin level. Here we apply smartphone imaging to non-invasive assessment of bilirubin in patients with cirrhosis. Image data was processed using two different approaches to remove variation introduced by ambient conditions and different imaging devices-a per-image calibration using a color chart in each image, and a two-step process using pairs of flash/ no-flash images to account for ambient light in combination with a one-time calibration. For the first time, results from the forehead, sclera (white of the eye) and lower eyelid were compared. The correlation coefficients between the total serum bilirubin and the predicted bilirubin via the forehead, sclera and lower eyelid were 0.79, 0.89 and 0.86 (all with p<0.001, n = 66), respectively. Given the simpler image capture for the sclera, the recommended imaging site for this patient cohort is the sclera.