Automated Respiratory Rate Counter to Assess Children for Symptoms of Pneumonia: Protocol for Cross-Sectional Usability and Acceptability Studies in Ethiopia and Nepal

Health System Considerations for Community-Based Implementation of Automated Respiratory Counters to Identify Childhood Pneumonia in 5 Regions of Ethiopia: A Qualitative Study

BACKGROUND

In Ethiopia, childhood pneumonia is diagnosed in primary healthcare settings by measuring respiratory rate (RR) along with the presence of cough, chest indrawing, difficulty breathing, and fast breathing. Our aim was to identify health system-level lessons from implementing two automated RR counters, Children's Automated Respiration Monitor (ChARM) by Phillips® and Rad-G by Masimo®, to provide considerations for integrating such devices into child health programmes and health systems. This study was part of an initiative called the Acute Respiratory Infection Diagnostic Aids (ARIDA).

METHODS

Key informant interviews (KIIs) were conducted with 57 participants (health workers in communities and facilities, trainers of health workers, district management, and key decision-makers) in five regions of Ethiopia. Data were analyzed in ATLAS.ti using thematic content analysis and themes were categorized using the Tanahashi bottleneck analysis.

RESULTS

All participants recommended scaling up the ARIDA initiative nationally as part of Integrated Management of Newborn and Childhood Illness (IMNCI) in primary healthcare. Health workers perceived the devices as: time saving, acceptable by parents and children, and facilitating diagnosis and referrals. Health workers perceived an increased demand for services and reduced numbers of sick children not seeking care. Participants recommended increasing the number of devices distributed and health workers trained. Strengthening drug supply chains, improving oxygen gas availability, and strengthening referral networks would maximize perceived benefits. While training improved knowledge, more supportive supervision, integration with current guidelines and more guidance related to community engagement was recommended.

CONCLUSION

Automatic RR counters for the decentralized diagnosis of childhood pneumonia could have positive impact on improving the quality of diagnosis and management of pneumonia in children. However, the study has shown that a health system approach is required to ensure all steps along the pneumonia pathway are adequate, including drug and oxygen supply, community engagement, health worker training and support, and referral pathways.

Field Experiences with Handheld Diagnostic Devices to Triage Children under Five Presenting with Severe Febrile Illness in a District Hospital in DR Congo

As part of a field study (NCT04473768) in children presenting with severe febrile illness to Kisantu hospital (DR Congo), we retrospectively compiled user experiences (not performance) with handheld diagnostic devices assisting triage: tympanic thermometer, pulse oximeter (measuring heart rate, respiratory rate and oxygen saturation), hemoglobinometer and glucometer. Guidance documents for product selection were generic and scattered. Stock rupture, market withdrawal and unaffordable prices interfered with procurement. Challenges at implementation included environmental temperature, capillary blood sampling (antisepsis, order of multiple tests, filling microcuvettes and glucose strips), calibration (environmental temperature, cold chain) and liability-oriented communication with a manufacturer. Instructions for use were readable and contained symbol keys; two devices had printed French-language instructions. Shortcomings were poor integration of figures with text and distinct procedures for the oximeter and its sensor. Usability interview revealed appreciations for quick results, visibility of the display and memory function (three devices) but also problems of capillary blood sample transfer, cleaning, too long of a time-to-results (respiratory rate) and size, fitting and disposal of thermometer probes. Pictorial error messages were preferred over alphanumeric error codes but interpretation of symbols was poor. Alarm sounds of the oximeter caused unrest in children and caretakers perceived the device as associated with poor prognosis.

Performance of Automated Point-of-Care Respiratory Rate Counting versus Manual Counting in Children under Five Admitted with Severe Febrile Illness to Kisantu Hospital, DR Congo

To improve the early recognition of danger signs in children with severe febrile illness in low resource settings, WHO promotes automated respiratory rate (RR) counting, but its performance is unknown in this population. Therefore, we prospectively evaluated the field performance of automated point-of-care plethysmography-based RR counting in hospitalized children with severe febrile illness (<5 years) in DR Congo. A trained research nurse simultaneously counted the RR manually (comparative method) and automatically with the Masimo Rad G pulse oximeter. Valid paired RR measurements were obtained in 202 (83.1%) children, among whom 43.1% (87/202) had fast breathing according to WHO criteria based on manual counting. Automated counting frequently underestimated the RR (median difference of -1 breath/minute; p2.5-p97.5 limits of agreement: -34-6), particularly at higher RR. This resulted in a failure to detect fast breathing in 24.1% (21/87) of fast breathing children (positive percent agreement: 75.9%), which was not explained by clinical characteristics (p > 0.05). Children without fast breathing were mostly correctly classified (negative percent agreement: 98.3%). In conclusion, in the present setting the automated RR counter performed insufficiently to facilitate the early recognition of danger signs in children with severe febrile illness, given wide limits of agreement and a too low positive percent agreement.

Evaluating the Interrater Agreement and Acceptability of a New Reference Tool for Assessing Respiratory Rate in Children under Five with Cough and/or Difficulty Breathing

BACKGROUND

Manual assessment of respiratory rate (RR) in children is unreliable, but remains the main method to diagnose pneumonia in low-resource settings. While automated RR counters offer a potential solution, there is currently no gold standard to validate these diagnostic aids. A video-based reference tool is proposed that allows users to annotate breaths and distortions including movement periods, allowing the exclusion of distortions from the computation of RR measures similar to how new diagnostic aids account for distortions automatically. This study evaluated the interrater agreement and acceptability of the new reference tool.

METHODS

Annotations were based on previously recorded reference videos of children under five years old with cough and/or difficulty breathing (n = 50). Five randomly selected medical experts from a panel of ten annotated each video. RR measures (breaths per minute, bpm) were computed as the number of annotated certain breaths divided by the length of calm periods after removing annotated distorted periods.

RESULTS

Reviewers showed good interrater agreement on continuous RR {standard error of measurement (SEM) [4.8 (95%CI 4.4-5.3)]} and substantial agreement on classification of fast breathing (Fleiss kappa, κ 0.71). Agreement was lowest in the youngest age group [< 2 months: SEM 6.2 (5.4-7.4) bpm, κ 0.48; 2-11 months: 4.7 (4.0-5.8) bpm, κ 0.84; 12-59 months: 2.6 (2.2-3.1) bpm, κ 0.8]. Reviewers found the functionalities of the tool helpful in annotating breaths, but remained uncertain about the validity of their annotations.

CONCLUSIONS

Before the new tool can be considered a reference standard for RR assessments, interrater agreement in children younger than 2 months must be improved.

Usability and acceptability of a multimodal respiratory rate and pulse oximeter device in case management of children with symptoms of pneumonia: A cross-sectional study in Ethiopia

Aim

Pneumonia is the leading infectious cause of death among children under five globally. Many pneumonia deaths result from inappropriate treatment due to misdiagnosis of signs and symptoms. This study aims to identify whether health extension workers (HEWs) in Ethiopia, using an automated multimodal device (Masimo Rad‐G), adhere to required guidelines while assessing and classifying under five children with cough or difficulty breathing and to understand device acceptability.

Methods

A cross‐sectional study was conducted in three districts of Southern Nations, Nationalities, and Peoples' Region, Ethiopia. Between September and December 2018, 133 HEWs were directly observed using Rad‐G while conducting 599 sick child consultations. Usability was measured as adherence to the World Health Organization requirements to assess fast breathing and device manufacturer instructions for use. Acceptability was assessed using semi‐structured interviews with HEWs, first‐level health facility workers and caregivers.

Results

Adherence using the Rad‐G routinely for 2 months was 85.3% (95% CI 80.2, 89.3). Health workers and caregivers stated a preference for Rad‐G. Users highlighted a number of device design issues.

Conclusion

While demonstrating high levels of acceptability and usability, the device modifications to consider include better probe fit, improved user interface with exclusive age categories and simplified classification outcomes.

Usability and acceptability of an automated respiratory rate counter to assess childhood pneumonia in Nepal

AIM

Pneumonia is the leading cause of child death after the neonatal period, resulting from late care seeking and inappropriate treatment. Diagnosis involves counting respiratory rate (RR); however, RR counting remains challenging for health workers and miscounting, and misclassification of RR is common. We evaluated the usability of a new automated RR counter, the Philips Children's Respiratory Monitor (ChARM), to Female Community Health Volunteers (FCHVs), and its acceptability to FCHVs and caregivers in Nepal.

METHODS

A cross-sectional study was conducted in Jumla district, Nepal. About 133 FCHVs were observed between September and December 2018 when using ChARM during 517 sick child consultations, 264 after training and 253 after 2 months of routine use of ChARM. Acceptability of the ChARM was explored using semi-structured interviews.

RESULTS

FCHV adherence to guidelines after 2 months of using ChARM routinely was 52.8% (95% CI 46.6-58.9). The qualitative findings suggest that ChARM is acceptable to FCHVs and caregivers; however, capacity constraints such as older age and low literacy and impacted device usability were mentioned.

CONCLUSION

Further research on the performance, cost-effectiveness and implementation feasibility of this device is recommended, especially among low-literate CHWs.