Smartphone screening for neonatal jaundice via ambient-subtracted sclera chromaticity

Demystifying non-invasive approaches for screening jaundice in low resource settings: a review

All national and international pediatric guidelines universally prescribe meticulous bilirubin screening for neonates as a critical measure to mitigate the incidence of acute bilirubin encephalopathy (ABE) and Kernicterus. The prevailing gold standard for jaundice detection in neonates necessitates invasive blood collection, followed by subsequent biochemical testing. While the invasive procedure provides dependable bilirubin measurements and continues to be the sole gold standard diagnostic method for assessing bilirubin concentration. There exists a pressing need to innovate non-invasive screening tools that alleviate the sampling stress endured by newborns, mitigate iatrogenic anemia, and expedite the turnaround time for obtaining results. The exploration of non-invasive modalities for bilirubin measurements is gaining momentum, driven by the overarching goal of minimizing the number of pricks inflicted upon neonates, thereby rendering screening a swift, efficient, comfortable, and dependable process. This comprehensive review article delves extensively into the array of non-invasive approaches and digital solutions that have been proposed, implemented, and utilized for neonatal bilirubin screening, with a particular emphasis on their application in low-resource settings. Within this context, the review sheds light on the existing methodologies and their practical applications, with a specific focus on transcutaneous bilirubin meters. Moreover, it underscores the prevailing open challenges in this domain and outlines potential directions for future research endeavors. Notably, the review underscores the imperative need for robust educational programs targeted at both families and healthcare personnel to expedite the process of seeking timely care for neonatal jaundice. Additionally, it underscores the necessity for the development of enhanced screening and diagnostic tools that can offer greater accuracy in clinical practice.

Assessment of bilirubin levels in patients with cirrhosis via forehead, sclera and lower eyelid smartphone images

One of the key biomarkers evaluating liver disease progression is an elevated bilirubin level. Here we apply smartphone imaging to non-invasive assessment of bilirubin in patients with cirrhosis. Image data was processed using two different approaches to remove variation introduced by ambient conditions and different imaging devices-a per-image calibration using a color chart in each image, and a two-step process using pairs of flash/ no-flash images to account for ambient light in combination with a one-time calibration. For the first time, results from the forehead, sclera (white of the eye) and lower eyelid were compared. The correlation coefficients between the total serum bilirubin and the predicted bilirubin via the forehead, sclera and lower eyelid were 0.79, 0.89 and 0.86 (all with p<0.001, n = 66), respectively. Given the simpler image capture for the sclera, the recommended imaging site for this patient cohort is the sclera.

Performance of smartphone application to accurately quantify hyperbilirubinemia in neonates: a systematic review with meta-analysis

Neonatal jaundice is a common clinical condition that can progress to severe hyperbilirubinemia if identification and intervention are delayed. In this study, we aimed to analyze the current evidence on the accurate performance of smartphone applications to quantify bilirubin levels. PubMed, Embase, Emcare, MEDLINE, the Cochrane Library, and Google Scholar were searched from inception until July 2022. Grey literature was searched on "OpenGrey" and "MedNar" databases. We included prospective and retrospective cohort studies that recruited infants with a gestation of ≥ 35 weeks and reported paired total serum bilirubin (TSB) and smartphone app-based bilirubin (ABB) levels. We conducted the review using the guidelines of the Cochrane Collaboration Diagnostic Test Accuracy Working Group and reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-diagnostic test accuracy (PRISMA-DTA) statement. The data were pooled using the random effects model. The outcome of interest was agreement between ABB and TSB measurements, provided as correlation coefficient, mean difference, and standard deviation. Certainty of evidence (COE) was assessed based on GRADE guidelines. Fourteen studies were included in the meta-analysis. The number of infants in individual studies ranged between 35 and 530. The pooled correlation coefficient (r) between ABB and TSB was 0.77 (95% CI 0.69 to 0.83; p < 0.01). Reported sensitivities for predicting a TSB of 250 µmol/L in individual studies ranged between 75 and 100% and specificities ranged from 61 to 100%. Similarly, a sensitivity of 83 to 100% and a specificity of 19.5 to 76% were reported for predicting a TSB of 205 μmol/L. Overall COE was considered moderate.    Conclusion: Smartphone app-based bilirubin estimation showed a reasonable correlation to TSB levels. Well-designed studies are required to determine its utility as a screening tool for various TSB cut-off levels. What is Known: • Neonatal jaundice is a common clinical condition. Timely screening and intervention are necessary to prevent neurological morbidities • Transcutaneous bilirubinometer is a widely used non-invasive screening device but is mostly available in hospital settings and has cost limitations. Researchers have recently explored the utility of smartphone applications to estimate bilirubin levels in neonates. What is New: • This is the first systematic review and meta-analysis conducted to assess the performance of smartphone applications to detect neonatal hyperbilirubinemia. • Bilirubin estimates of newborn infants obtained through smartphone applications had a reasonable correlation with serum bilirubin levels.

The need for neonatal jaundice screening awareness in the Pakistani population: short communication

Neonatal jaundice is a common illness that affects around 80% of preterm and 50-60% of full-term newborn infants. It is one of the most common causes of neonatal death. Neonatal jaundice may be physiological or pathological. Physiologic jaundice is far more common than pathologic jaundice and accounts for most hyperbilirubinemia. Physiologic jaundice in neonates is due to greater hemoglobin breakdown compared to bilirubin clearance. While pathological jaundice occurs due to various infections, drug toxicity, inborn enzyme deficiencies, Rhesus fetal-maternal incompatibility, hypothyroidism, and congenital biliary duct obstruction diseases. In many parts of the world, midwives, and nurses perform spontaneous vaginal deliveries and they only rely on visual screening for neonatal jaundice. However, this is not reliable, especially for newborns having darker skin. Educating the mothers on screening for early detection of neonatal jaundice and seeking medical treatment in a country like Pakistan, which is considered a high-risk population, is crucial. Also, as most females give birth at home, hence, midwives' knowledge about neonatal jaundice also needs to be improved.

Identification and Quantification of Jaundice by Trans-Conjunctiva Optical Imaging Using a Human Brain-like Algorithm: A Cross-Sectional Study

Jaundice is caused by excess circulating bilirubin, known as hyperbilirubinemia. This symptom is sometimes caused by a critical hepatobiliary disorder, and is generally identified as yellowish sclera when bilirubin levels increase more than 3 mg/dL. It is difficult to identify jaundice accurately, especially via telemedicine. This study aimed to identify and quantify jaundice by trans-conjunctiva optical imaging. Patients with jaundice (total bilirubin ≥3 mg/dL) and normal control subjects (total bilirubin <3 mg/dL) were prospectively enrolled from June 2021 to July 2022. We took bilateral conjunctiva imaging with a built-in camera on a smartphone (1st generation iPhone SE) under normal white light conditions without any restrictions. We processed the images using an Algorithm Based on Human Brain (ABHB) (Zeta Bridge Corporation, Tokyo, Japan) and converted them into a hue degree of Hue Saturation Lightness (HSL) color space. A total of 26 patients with jaundice (9.57 ± 7.11 mg/dL) and 25 control subjects (0.77 ± 0.35 mg/dL) were enrolled in this study. The causes of jaundice among the 18 male and 8 female subjects (median age 61 yrs.) included hepatobiliary cancer (n = 10), chronic hepatitis or cirrhosis (n = 6), pancreatic cancer (n = 4), acute liver failure (n = 2), cholelithiasis or cholangitis (n = 2), acute pancreatitis (n = 1), and Gilbert's syndrome (n = 1). The maximum hue degree (MHD) optimal cutoff to identify jaundice was 40.8 (sensitivity 81% and specificity 80%), and the AUROC was 0.842. The MHD was moderately correlated to total serum bilirubin (TSB) levels (rS = 0.528, p < 0.001). TSB level (≥5 mg/dL) can be estimated by the formula 21.1603 - 0.7371 × 56.3-MHD2. In conclusion, the ABHB-based MHD of conjunctiva imaging identified jaundice using an ordinary smartphone without any specific attachments and deep learning. This novel technology could be a helpful diagnostic tool in telemedicine or self-medication.

Feasibility of smartphone colorimetry of the face as an anaemia screening tool for infants and young children in Ghana

BACKGROUND

Anaemia affects approximately a quarter of the global population. When anaemia occurs during childhood, it can increase susceptibility to infectious diseases and impair cognitive development. This research uses smartphone-based colorimetry to develop a non-invasive technique for screening for anaemia in a previously understudied population of infants and young children in Ghana.

METHODS

We propose a colorimetric algorithm for screening for anaemia which uses a novel combination of three regions of interest: the lower eyelid (palpebral conjunctiva), the sclera, and the mucosal membrane adjacent to the lower lip. These regions are chosen to have minimal skin pigmentation occluding the blood chromaticity. As part of the algorithm development, different methods were compared for (1) accounting for varying ambient lighting, and (2) choosing a chromaticity metric for each region of interest. In comparison to some prior work, no specialist hardware (such as a colour reference card) is required for image acquisition.

RESULTS

Sixty-two patients under 4 years of age were recruited as a convenience clinical sample in Korle Bu Teaching Hospital, Ghana. Forty-three of these had quality images for all regions of interest. Using a naïve Bayes classifier, this method was capable of screening for anaemia (<11.0g/dL haemoglobin concentration) vs healthy blood haemoglobin concentration (≥11.0g/dL) with a sensitivity of 92.9% (95% CI 66.1% to 99.8%), a specificity of 89.7% (72.7% to 97.8%) when acting on unseen data, using only an affordable smartphone and no additional hardware.

CONCLUSION

These results add to the body of evidence suggesting that smartphone colorimetry is likely to be a useful tool for making anaemia screening more widely available. However, there remains no consensus on the optimal method for image preprocessing or feature extraction, especially across diverse patient populations.

A novel smartphone scleral-image based tool for assessing jaundice in decompensated cirrhosis patients

BACKGROUND AND AIM

Serum bilirubin is an established marker of liver disease. Reliable tools for non-invasive assessment of jaundice in cirrhosis patients, at risk of clinical decompensation, are highly desirable. While smartphone-based imaging has been described in neonatal jaundice, it has not been investigated in advanced cirrhosis patients.

METHODS

We included 46 hospitalized patients with acute cirrhosis decompensation and jaundice. Scleral images using an Android smartphone were taken to derive "Scleral Color Values (SCV)," which were matched with same day serum bilirubin measurements. In 29 patients, repeat SCV and bilirubin measurements were performed over time. We analyzed the relationship of SCV and its dynamics with serum bilirubin, clinical scores, and patient outcomes.

RESULTS

Of 46 patients, 26 (57%) had alcoholic hepatitis as the decompensation precipitant. Seven patients died during admission; a further 12 following hospital discharge. SCV had an excellent linear correlation with serum bilirubin (rho = 0.90, P < 0.001); changes in SCV and serum bilirubin across different time points, were also closely associated (rho = 0.77, P < 0.001). SCV correlated significantly with CLIF Consortium Acute Decompensation score (rho = 0.38, P < 0.001) and grade of Acute-on-Chronic Liver Failure (rho = 0.42, P = 0.039). SCV was higher in patients who died, however, not significantly (86.1 [IQR 83.0-89.7] vs 82.3 [IQR 78.5-83.3], P = 0.22). The associations of SCV with clinical parameters mirrored those of serum bilirubin.

CONCLUSION

Smartphone-based assessment of jaundice shows excellent concordance with serum bilirubin and is associated with clinical parameters in acute cirrhosis decompensation. This approach offers promise for remote assessment of cirrhosis patients at-risk of decompensation, post hospital discharge.

Effects of a Smartphone-Based Out-of-Hospital Screening App for Neonatal Hyperbilirubinemia on Neonatal Readmission Rates and Maternal Anxiety: Randomized Controlled Trial

Background

Neonatal hyperbilirubinemia is one of the leading causes of neonatal readmission—especially severe hyperbilirubinemia and its complications—and it influences disease burden as well as neonatal and maternal health. Smartphones have been shown to have satisfactory accuracy in screening neonatal bilirubin levels, but the impact of this technology on neonatal health care service and maternal health outcomes is still unknown.

Objective

The aim of this study was to evaluate the impact of a smartphone-based out-of-hospital neonatal jaundice screening program on neonatal readmission rates for jaundice and related maternal anxiety.

Methods

This was a 2-arm, unblinded, randomized controlled trial with 30 days of intervention and follow-up periods. From August 2019 to August 2020, healthy mother-infant dyads were recruited on-site from 3 public hospitals in Hainan, China. Intervention group mothers used the smartphone app to routinely monitor neonatal jaundice at home under the web-based guidance of pediatricians. Control group participants received routine care. The primary study outcome was the neonatal readmission rate due to jaundice within 30 days of the first hospital discharge. The secondary outcome was the maternal anxiety score associated with neonatal jaundice. The data were collected through a self-assessed questionnaire. All participants were included in the analysis (intention-to-treat).

Results

In this study, 1424 mother-infant dyads were recruited, comprising 1424 mothers and 1424 newborns. The median age of the mothers was 29 (IQR 26-32) years, and there were 714 (50.1%) male neonates. These mother-infant dyads were randomly assigned to the intervention group and the control group, with 712 dyads in each group; only 1187 of these dyads completed the follow-up. We found that the adjusted 30-day neonatal readmission rate due to jaundice reduced by 10.5% (71/605, 11.7% vs 141/582, 24.2%; 95% CI 5%-15.9%; odds ratio 0.4, 95% CI 0.3-0.5; P<.001) and the relevant maternal anxiety mean score decreased by 3.6 (95% CI –4.4 to –2.8; β=–3.6, 95% CI –4.5 to –2.8; P<.001) in the intervention group compared to those in the routine care group.

Conclusions

Our study shows that the smartphone-based out-of-hospital screening method for neonatal hyperbilirubinemia decreased the neonatal readmission rate within 30 days from the first discharge and improved maternal mental health to some degree, thus demonstrating the usefulness of this screening app for follow-up in pediatric care.

Trial Registration

China Clinical Trial Registration Center, ChiCTR2100049567; http://www.chictr.org.cn/showproj.aspx?proj=64245

On AI Approaches for Promoting Maternal and Neonatal Health in Low Resource Settings: A Review

A significant challenge for hospitals and medical practitioners in low- and middle-income nations is the lack of sufficient health care facilities for timely medical diagnosis of chronic and deadly diseases. Particularly, maternal and neonatal morbidity due to various non-communicable and nutrition related diseases is a serious public health issue that leads to several deaths every year. These diseases affecting either mother or child can be hospital-acquired, contracted during pregnancy or delivery, postpartum and even during child growth and development. Many of these conditions are challenging to detect at their early stages, which puts the patient at risk of developing severe conditions over time. Therefore, there is a need for early screening, detection and diagnosis, which could reduce maternal and neonatal mortality. With the advent of Artificial Intelligence (AI), digital technologies have emerged as practical assistive tools in different healthcare sectors but are still in their nascent stages when applied to maternal and neonatal health. This review article presents an in-depth examination of digital solutions proposed for maternal and neonatal healthcare in low resource settings and discusses the open problems as well as future research directions.

Validating a Sclera-Based Smartphone Application for Screening Jaundiced Newborns in Ghana

OBJECTIVES

Reducing the burden of bilirubin-induced neurologic complications in low-resource countries requires reliable and accessible screening tools. We sought to optimize and validate a sclera-based smartphone application, Neonatal Scleral-Conjunctival Bilirubin (neoSCB), for screening neonatal jaundice.

METHODS

Using a cross-sectional design, consecutive eligible infants (aged 0-28 days, in the hospital, not critically ill) were enrolled in Ghana from March 2019 to April 2020. Jaundice screening was performed with neoSCB (Samsung Galaxy S8) to quantify SCB and JM-105 (Dräger) for transcutaneous bilirubin (TcB). Screening values were compared with total serum bilirubin (TSB) measured at the point of care.

RESULTS

Overall, 724 infants participated in the optimization and validation phases of the study. The analysis for validation included 336 infants with no previous treatment of jaundice. Single neoSCB image captures identified infants with TSB >14.62 mg/dL (250 μmol/L) with reasonably high sensitivity, specificity, and receiver operating characteristic area under the curve at 0.94 (95% confidence interval [CI], 0.91 to 0.97), 0.73 (95% CI, 0.68 to 0.78), and 0.90, respectively. These findings were comparable to the sensitivity and specificity of JM-105 (0.96 [95% CI, 0.90 to 0.99] and 0.81 [95% CI, 0.76 to 0.86], respectively). The TcB/TSB had a larger correlation coefficient (r = 0.93; P < .01) than SCB/TSB (r = 0.78; P < .01). Performance of both devices was lower in infants with previous phototherapy (n = 231).

CONCLUSIONS

The diagnostic performance of neoSCB was comparable to JM-105 and is a potential, affordable, contact-free screening tool for neonatal jaundice.

Current and emerging technologies for the timely screening and diagnosis of neonatal jaundice

Neonatal jaundice is one of the most common clinical conditions affecting newborns. For most newborns, jaundice is harmless, however, a proportion of newborns develops severe neonatal jaundice requiring therapeutic interventions, accentuating the need to have reliable and accurate screening tools for timely recognition across different health settings. The gold standard method in diagnosing jaundice involves a blood test and requires specialized hospital-based laboratory instruments. Despite technological advancements in point-of-care laboratory medicine, there is limited accessibility of the specialized devices and sample stability in geographically remote areas. Lack of suitable testing options leads to delays in timely diagnosis and treatment of clinically significant jaundice in developed and developing countries alike. There has been an ever-increasing need for a low-cost, simple to use screening technology to improve timely diagnosis and management of neonatal jaundice. Consequently, several point-of-care (POC) devices have been developed to address this concern. This paper aims to review the literature, focusing on emerging technologies in the screening and diagnosing of neonatal jaundice. We report on the challenges associated with the existing screening tools, followed by an overview of emerging sensors currently in pre-clinical development and the emerging POC devices in clinical trials to advance the screening of neonatal jaundice. The benefits offered by emerging POC devices include their ease of use, low cost, and the accessibility of rapid response test results. However, further clinical trials are required to overcome the current limitations of the emerging POC's before their implementation in clinical settings. Hence, the need for a simple to use, low-cost POC jaundice detection technology for newborns remains an unsolved challenge globally.

Neonatal Jaundice Diagnosis Using a Smartphone Camera Based on Eye, Skin, and Fused Features with Transfer Learning

Neonatal jaundice is a common condition worldwide. Failure of timely diagnosis and treatment can lead to death or brain injury. Current diagnostic approaches include a painful and time-consuming invasive blood test and non-invasive tests using costly transcutaneous bilirubinometers. Since periodic monitoring is crucial, multiple efforts have been made to develop non-invasive diagnostic tools using a smartphone camera. However, existing works rely either on skin or eye images using statistical or traditional machine learning methods. In this paper, we adopt a deep transfer learning approach based on eye, skin, and fused images. We also trained well-known traditional machine learning models, including multi-layer perceptron (MLP), support vector machine (SVM), decision tree (DT), and random forest (RF), and compared their performance with that of the transfer learning model. We collected our dataset using a smartphone camera. Moreover, unlike most of the existing contributions, we report accuracy, precision, recall, f-score, and area under the curve (AUC) for all the experiments and analyzed their significance statistically. Our results indicate that the transfer learning model performed the best with skin images, while traditional models achieved the best performance with eyes and fused features. Further, we found that the transfer learning model with skin features performed comparably to the MLP model with eye features.

Screening methods for neonatal hyperbilirubinemia: benefits, limitations, requirements, and novel developments

Severe neonatal hyperbilirubinemia (SNH) is a serious condition that occurs worldwide. Timely recognition with bilirubin determination is key in the management of SNH. Visual assessment of jaundice is unreliable. Fortunately, transcutaneous bilirubin measurement for screening newborn infants is routinely available in many hospitals and outpatient settings. Despite a few limitations, the use of transcutaneous devices facilitates early recognition and appropriate management of neonatal jaundice. Unfortunately, however, advanced and often costly screening modalities are not accessible to everyone, while there is an urgent need for inexpensive yet accurate instruments to assess total serum bilirubin (TSB). In the near future, novel icterometers, and in particular optical bilirubin estimates obtained with a smartphone camera and processed with a smartphone application (app), seem promising methods for screening for SNH. If proven reliable, these methods may empower outpatient health workers as well as parents at home to detect jaundice using a simple portable device. Successful implementation of ubiquitous bilirubin screening may contribute substantially to the reduction of the worldwide burden of SNH. The benefits of non-invasive bilirubin screening notwithstanding, any bilirubin determination obtained through non-invasive screening must be confirmed by a diagnostic method before treatment. IMPACT: Key message: Screening methods for neonatal hyperbilirubinemia facilitate early recognition and timely treatment of severe neonatal hyperbilirubinemia (SNH). Any bilirubin screening result obtained must be confirmed by a diagnostic method. What does this article add to the existing literature? Data on optical bilirubin estimation are summarized. Niche research strategies for prevention of SNH are presented. Impact: Transcutaneous screening for neonatal hyperbilirubinemia contributes to the prevention of SNH. A smartphone application with optical bilirubin estimation seems a promising low-cost screening method, especially in low-resource settings or at home.

Smartphone-based imaging systems for medical applications: a critical review

SIGNIFICANCE

Smartphones come with an enormous array of functionality and are being more widely utilized with specialized attachments in a range of healthcare applications. A review of key developments and uses, with an assessment of strengths/limitations in various clinical workflows, was completed.

AIM

Our review studies how smartphone-based imaging (SBI) systems are designed and tested for specialized applications in medicine and healthcare. An evaluation of current research studies is used to provide guidelines for improving the impact of these research advances.

APPROACH

First, the established and emerging smartphone capabilities that can be leveraged for biomedical imaging are detailed. Then, methods and materials for fabrication of optical, mechanical, and electrical interface components are summarized. Recent systems were categorized into four groups based on their intended application and clinical workflow: ex vivo diagnostic, in vivo diagnostic, monitoring, and treatment guidance. Lastly, strengths and limitations of current SBI systems within these various applications are discussed.

RESULTS

The native smartphone capabilities for biomedical imaging applications include cameras, touchscreens, networking, computation, 3D sensing, audio, and motion, in addition to commercial wearable peripheral devices. Through user-centered design of custom hardware and software interfaces, these capabilities have the potential to enable portable, easy-to-use, point-of-care biomedical imaging systems. However, due to barriers in programming of custom software and on-board image analysis pipelines, many research prototypes fail to achieve a prospective clinical evaluation as intended. Effective clinical use cases appear to be those in which handheld, noninvasive image guidance is needed and accommodated by the clinical workflow. Handheld systems for in vivo, multispectral, and quantitative fluorescence imaging are a promising development for diagnostic and treatment guidance applications.

CONCLUSIONS

A holistic assessment of SBI systems must include interpretation of their value for intended clinical settings and how their implementations enable better workflow. A set of six guidelines are proposed to evaluate appropriateness of smartphone utilization in terms of clinical context, completeness, compactness, connectivity, cost, and claims. Ongoing work should prioritize realistic clinical assessments with quantitative and qualitative comparison to non-smartphone systems to clearly demonstrate the value of smartphone-based systems. Improved hardware design to accommodate the rapidly changing smartphone ecosystem, creation of open-source image acquisition and analysis pipelines, and adoption of robust calibration techniques to address phone-to-phone variability are three high priority areas to move SBI research forward.

Artificial intelligence in precision medicine in hepatology

The advancement of investigation tools and electronic health records (EHR) enables a paradigm shift from guideline-specific therapy toward patient-specific precision medicine. The multiparametric and large detailed information necessitates novel analyses to explore the insight of diseases and to aid the diagnosis, monitoring, and outcome prediction. Artificial intelligence (AI), machine learning, and deep learning (DL) provide various models of supervised, or unsupervised algorithms, and sophisticated neural networks to generate predictive models more precisely than conventional ones. The data, application tasks, and algorithms are three key components in AI. Various data formats are available in daily clinical practice of hepatology, including radiological imaging, EHR, liver pathology, data from wearable devices, and multi-omics measurements. The images of abdominal ultrasonography, computed tomography, and magnetic resonance imaging can be used to predict liver fibrosis, cirrhosis, non-alcoholic fatty liver disease (NAFLD), and differentiation of benign tumors from hepatocellular carcinoma (HCC). Using EHR, the AI algorithms help predict the diagnosis and outcomes of liver cirrhosis, HCC, NAFLD, portal hypertension, varices, liver transplantation, and acute liver failure. AI helps to predict severity and patterns of fibrosis, steatosis, activity of NAFLD, and survival of HCC by using pathological data. Despite of these high potentials of AI application, data preparation, collection, quality, labeling, and sampling biases of data are major concerns. The selection, evaluation, and validation of algorithms, as well as real-world application of these AI models, are also challenging. Nevertheless, AI opens the new era of precision medicine in hepatology, which will change our future practice.

Accurate device-independent colorimetric measurements using smartphones

Smartphones provide an ideal platform for colorimetric measurements due to their low cost, portability and image quality. As with any imaging-based colorimetry system, ambient light and device variations introduce error which must be dealt with. We propose a novel processing method consisting of a one-time calibration stage to account for inter-phone variations, and an innovative use of ambient light subtraction with image pairs to account for variation in ambient light. Data collection is kept very simple, making it particularly useful for use in the field, since nothing additional is required in the images. Ambient subtraction is first demonstrated for a range of colors and phones (Samsung S8 and LG Nexus 5X), and the Subtracted Signal to Noise Ratio (SSNR) is defined as a metric for assessing whether an image pair is appropriate at the time of image capture. The experimentally determined SSNR threshold below which to suggest retaking the images is 3.4. The classification accuracy for results using the proposed calibration pipeline is then compared to the simplest image metadata-based alternative and is found to be greatly superior. Finally, a custom colorcard is shown to improve the accuracy of device-independent results for known smaller ranges of colors over a standard colorcard, making this a possible application-specific modification to the overall processing pipeline.