Practical considerations for transitioning early childhood interventions to scale: lessons from the Saving Brains portfolio

Leveraging monitoring, evaluation, and learning to scale the Enabling Inclusion® program for children with disabilities in India and globally

Introduction

Children with disabilities in low- and middle-income countries face many challenges and lack adequate services, including access to rehabilitation professionals. To address this lack of access, Amar Seva Sangam Ayikudy (ASSA), a non-governmental organization (NGO) in India, designed a technology-leveraged rehabilitation program called Enabling Inclusion® (EI®), and implemented it in one state (Tamil Nadu, India) before scaling it. The model is supported by the EI® app, which enables organizations to screen, assess and monitor progress of children with disabilities via rehabilitation specialists and community rehabilitation workers, and to provide family-centered, goal-based interventions. An extensive monitoring, evaluation, and learning (MEL) framework is embedded into the program. This paper explores how this MEL system supported the scaling of the EI® model, reaching additional beneficiaries nationally and globally.

Methods

This paper describes ASSA's MEL framework and demonstrates its use for decision-making in the process of scaling. It also explores how collaborations with various government departments, NGOs, and private partners contributed to the scaling of the EI® model and technology.

Results

Scaling of the EI® program was achieved by (1) expansion of the program in rural Tamil Nadu (vertical scale-up) in partnership with the Tamil Nadu government and private partners, and (2) by licensing the EI® app and model to other NGOs in various states in India and globally (horizontal scale-up). Systematic examination of key program and performance indicators, as well as stakeholder feedback, informed decisions to modify the EI® app over time. This included further customizing to the needs of children and service providers, covering a greater range of age groups and contexts, and modifying service delivery models. Child functional independence, participation, and inclusion was further strengthened by mobilizing parent empowerment groups, community awareness programs, school advocacy, and entitlements from the government. Flexibility in the implementation model of the EI® app allowed for adaptation to local contexts and organizations, and facilitated its scale-up.

Conclusion

A dynamic, inclusive, and locally grounded MEL system, a flexible and collaborative approach, and an adaptive implementation model increased the accessibility of an early intervention and childhood rehabilitation program for children with disabilities and their families throughout the state of Tamil Nadu, across India, and internationally.

Sustaining and scaling a clinic-based approach to address health-related social needs

Objective

Scaling evidence-based interventions (EBIs) from pilot phase remains a pressing challenge in efforts to address health-related social needs (HRSN) and improve population health. This study describes an innovative approach to sustaining and further spreading DULCE (Developmental Understanding and Legal Collaboration for Everyone), a universal EBI that supports pediatric clinics to implement the American Academy of Pediatrics' Bright Futures™ guidelines for infants' well-child visits (WCVs) and introduces a new quality measure of families' HRSN resource use.

Methods

Between August 2018 and December 2019, seven teams in four communities in three states implemented DULCE: four teams that had been implementing DULCE since 2016 and three new teams. Teams received monthly data reports and individualized continuous quality improvement (CQI) coaching for six months, followed by lighter-touch support via quarterly group calls (peer-to-peer learning and coaching). Run charts were used to study outcome (percent of infants that received all WCVs on time) and process measures (percent of families screened for HRSN and connected to resources).

Results

Integrating three new sites was associated with an initial regression of outcome: 41% of infants received all WCVs on time, followed by improvement to 48%. Process performance was sustained or improved: among 989 participating families, 84% (831) received 1-month WCVs on time; 96% (946) were screened for seven HRSN, 54% (508) had HRSN, and 87% (444) used HRSN resources.

Conclusion

An innovative, lighter-touch CQI approach to a second phase of scale-up resulted in sustainment or improvements in most processes and outcomes. Outcomes-oriented CQI measures (family receipt of resources) are an important addition to more traditional process-oriented indicators.

A holistic approach to promoting early child development: a cluster randomised trial of a group-based, multicomponent intervention in rural Bangladesh

Introduction

In low- and middle-income countries, children experience multiple risks for delayed development. We evaluated a multicomponent, group-based early child development intervention including behavioural recommendations on responsive stimulation, nutrition, water, sanitation, hygiene, mental health and lead exposure prevention.

Methods

We conducted a 9-month, parallel, multiarm, cluster-randomised controlled trial in 31 rural villages in Kishoreganj District, Bangladesh. Villages were randomly allocated to: group sessions (‘group’); alternating groups and home visits (‘combined’); or a passive control arm. Sessions were delivered fortnightly by trained community members. The primary outcome was child stimulation (Family Care Indicators); the secondary outcome was child development (Ages and Stages Questionnaire Inventory, ASQi). Other outcomes included dietary diversity, latrine status, use of a child potty, handwashing infrastructure, caregiver mental health and knowledge of lead. Analyses were intention to treat. Data collectors were independent from implementers.

Results

In July–August 2017, 621 pregnant women and primary caregivers of children<15 months were enrolled (group n=160, combined n=160, control n=301). At endline, immediately following intervention completion (July–August 2018), 574 participants were assessed (group n=144, combined n=149, control n=281). Primary caregivers in both intervention arms participated in more play activities than control caregivers (age-adjusted means: group 4.22, 95% CI 3.97 to 4.47; combined 4.77, 4.60 to 4.96; control 3.24, 3.05 to 3.39), and provided a larger variety of play materials (age-adjusted means: group 3.63, 3.31 to 3.96; combined 3.81, 3.62 to 3.99; control 2.48, 2.34 to 2.59). Compared with the control arm, children in the group arm had higher total ASQi scores (adjusted mean difference in standardised scores: 0.39, 0.15 to 0.64), while in the combined arm scores were not significantly different from the control (0.25, –0.07 to 0.54).

Conclusion

Our findings suggest that group-based, multicomponent interventions can be effective at improving child development outcomes in rural Bangladesh, and that they have the potential to be delivered at scale.

Trial registration number

The trial is registered in ISRCTN (ISRCTN16001234).

Age-related patterns of early childhood development practices amongst rural families in Burkina Faso: findings from a nationwide survey of mothers of children aged 0-3 years

BACKGROUND

Around two thirds of children in Sub-Saharan Africa are at risk of not meeting their developmental potential. Scalable interventions, based on an understanding of local contexts, that promote nurturing care in children's early years are needed.

OBJECTIVES

To investigate age-related patterns of Early Childhood Development (ECD) practices amongst caretakers of children aged 0-3 years in rural households in Burkina Faso, in order to inform the design of a mass media campaign to be evaluated through a randomized controlled trial.

METHODS

A household survey using a structured questionnaire was used to collect data from 960 rural mothers of children aged 0-3 years in a regionally stratified random sample of 130 villages.

RESULTS

The mother was the main caretaker and engaged most in ECD-related activities at all ages (0-3 years). The father, grandmother and older children also engaged in ECD-related activities with older children (aged 1-3 years). Singing and playing occurred moderately frequently. Singing in the last three days: 36% at age 0-5 months increasing to 84% at age 3 years; playing in the last three days: 26% at age 0-5 months, increasing to 65% at age 3 years. Activities such as reading, counting, drawing, 'showing and naming' and 'chatting' were limited, particularly in the child's first year. Reasons for not engaging in these activities include lack of literacy, lack of books and toys or playthings and a belief that the child was too young.

CONCLUSION

Opportunities for learning, especially through verbal interactions, appeared to be limited during the developmentally crucial first three years, most notably in the first year of life. The challenge for ECD intervention development in Burkina Faso will be finding ways to promote more responsive interactions at an early age and finding ways of mobilizing other family members to become more engaged in stimulating activities in the child's early years.

Rapid-Cycle Evaluation in an Early Intervention Program for Children With Developmental Disabilities in South India: Optimizing Service Providers' Quality of Work-Life, Family Program Engagement, and School Enrollment

Background: This paper explores how implementation and refinement of an early intervention (EI) program for children with delayed development was informed by an iterative, intentional and structured process of measurement. Providing access to early intervention therapy for children in rural areas of India is challenging due to a lack of rehabilitation therapists and programs. Following a biopsychosocial framework and principles of community-based rehabilitation, a non-governmental organization, Amar Seva Sangam (ASSA), overcame those barriers by designing a digital technology supported EI program in rural Tamil Nadu, India. Program objectives included providing service access; supporting program engagement, child development and school enrollment; and positioning the intervention for scale-up. This paper contributes to a growing body of literature on how program design and implementation can be informed through a cyclical process of data collection, analysis, reflection, and adaptation.

Methods: Through several strands of data collection, the design and implementation of the EI program was adapted and improved. This included qualitative data from focus groups and interviews with caregivers and service providers, and a mobile application that collected and monitored longitudinal quantitative data, including program engagement rates, developmental progression, caregiver outcomes, and school enrollment status.

Results: Measurements throughout the program informed decision-making by identifying facilitators and barriers to service providers' quality of work-life, family program engagement, and school enrollment. Consultation with key stakeholders, including caregivers and service providers, and data driven decision making led to continual program changes that improved service provider quality of work-life, program engagement and school enrollment. These changes included addressing gender-related work challenges for service providers; forming caregiver support networks; introducing psychological counseling for caregivers; providing medical consultations and assistive devices; creating community awareness programs; improving access to therapy services; focusing on caregiver education, motivation and support; and advocacy for accessibility in schools.

Conclusion: The process of using evidence-informed and stakeholder driven adaptations to the early intervention program, led to improved service provider quality of work-life, greater program engagement, improved school enrollment and positioned the intervention for scale-up, providing lessons that may be beneficial in other contexts.

Measurement for Change: From Idea to Approach

Measurement for Change proposes an integration of monitoring, evaluation, and learning into decision-making systems that support sustainable transition of interventions to scale. It was developed using a cyclical, interactive 1-year dialogue between early childhood development (ECD) practitioners and academics from across the globe. Details are presented in Krapels et al. (1) as part of this special issue in Frontiers. In this paper, we trace the developments that inspired Measurement for Change and the novel ways in which the approach and the special issue was developed. The experience, and the reflections on this experience, are intended to inform those implementing initiatives that similarly seek to integrate practitioner- and academic experiences in support of sustainable transitions of interventions to scale.

The Aspirations of Measurement for Change

This Perspective presents the five key aspirations of an approach to data use, decision making and monitoring, evaluation, and learning (MEL) in Early Childhood Development (ECD) referred to as Measurement for Change. The core ideas of Measurement for Change gave rise to this series, and many of the papers submitted in this series speak to this approach, whether directly or indirectly. The five aspirations describe interconnected concepts that advocate for practitioners and researchers within ECD to build the capacity to use data in their decision making, by establishing a monitoring, evaluation, and learning system that strives to be: Dynamic; Inclusive; Informative; Interactive; and People-centered.

Human resources and curricula content for early child development implementation: multicountry mixed methods evaluation

OBJECTIVE

The WHO recommends responsive caregiving and early learning (RCEL) interventions to improve early child development (ECD), and to achieve the Sustainable Development Goals' vision of a world where all children thrive. Implementation of RCEL programmes in low and middle-income countries (LMIC) requires evidence to inform decisions about human resources and curricula content. We aimed to describe human resources and curricula content for implementation of RCEL projects across diverse LMICs, using data from the Grand Challenges Canada Saving Brains ECD portfolio.

SETTING

We evaluated 32 RCEL projects across 17 LMICs on four continents.

PARTICIPANTS

Overall, 2165 workers delivered ECD interventions to 25 909 families.

INTERVENTION

Projects were either stand-alone RCEL or RCEL combined with health and nutrition, and/or safety and security.

PRIMARY AND SECONDARY OUTCOMES

We undertook a mixed methods evaluation of RCEL projects within the Saving Brains portfolio. Quantitative data were collected through standardised reporting tools. Qualitative data were collected from ECD experts and stakeholders and analysed using thematic content analysis, informed by literature review.

RESULTS

Major themes regarding human resources included: worker characteristics, incentivisation, retention, training and supervision, and regarding curricula content: flexible adaptation of content and delivery, fidelity, and intervention duration and dosage. Lack of an agreed standard ECD package contributed to project heterogeneity. Incorporation of ECD into existing services may facilitate scale-up but overburdened workers plus potential reductions in service quality remain challenging. Supportive training and supervision, inducement, worker retention, dosage and delivery modality emerged as key implementation decisions.

CONCLUSIONS

This mixed methods evaluation of a multicountry ECD portfolio identified themes for consideration by policymakers and programme leaders relevant to RCEL implementation in diverse LMICs. Larger studies, which also examine impact, including high-quality process and costing evaluations with comparable data, are required to further inform decisions for implementation of RCEL projects at national and regional scales.

Scaling early child development: what are the barriers and enablers?

The Sustainable Development Goals, Global Strategy for Women's, Children's and Adolescents' Health (2016-2030) and Nurturing Care Framework all include targets to ensure children thrive However, many projects to support early childhood development (ECD) do not 'scale well' and leave large numbers of children unreached. This paper is the fifth in a series examining effective scaling of ECD programmes. This qualitative study explored experiences of scaling-up among purposively recruited implementers of ECD projects in low- and middle-income countries. Participants were sampled, by means of snowball sampling, from existing networks notably through Saving Brains®, Grand Challenges Canada®. Findings of a recent literature review on scaling-up frameworks, by the WHO, informed the development of a semistructured interview schedule. All interviews were conducted in English, via Skype, audio recorded and transcribed verbatim. Interviews were analysed using framework analysis. Framework analysis identified six major themes based on a standard programme cycle: planning and strategic choices, project design, human resources, financing and resource mobilisation, monitoring and evaluation, and leadership and partnerships. Key informants also identified an overarching theme regarding what scaling-up means. Stakeholders have not found existing literature and available frameworks helpful in guiding them to successful scale-up. Our research suggests that rather than proposing yet more theoretical guidelines or frameworks, it would be better to support stakeholders in developing organisational leadership capacity and partnership strategies to enable them to effectively apply a practical programme cycle or systematic process in their own contexts.

Contextual design choices and partnerships for scaling early child development programmes

Translating the Nurturing Care Framework and unprecedented global policy support for early child development (ECD) into action requires evidence-informed guidance about how to implement ECD programmes at national and regional scale. We completed a literature review and participatory mixed-method evaluation of projects in Saving Brains®, Grand Challenges Canada® funded ECD portfolio across 23 low- and middle-income countries (LMIC). Using an adapted programme cycle, findings from evaluation related to partnerships and leadership, situational analyses, and design for scaling ECD were considered. 39 projects (5 'Transition to Scale' and 34 'Seed') were evaluated. 63% were delivered through health and 84% focused on Responsive Caregiving and Early Learning (RCEL). Multilevel partnerships, leadership and targeted situational analysis were crucial to design and adaptation. A theory of change approach to consider pathways to impact was useful for design, but practical situational analysis tools and local data to guide these processes were lacking. Several RCEL programmes, implemented within government services, had positive impacts on ECD outcomes and created more enabling caregiving environments. Engagement of informal and private sectors provided an alternative approach for reaching children where government services were sparse. Cost-effectiveness was infrequently measured. At small-scale RCEL interventions can be successfully adapted and implemented across diverse settings through processes which are responsive to situational analysis within a partnership model. Accelerating progress will require longitudinal evaluation of ECD interventions at much larger scale, including programmes targeting children with disabilities and humanitarian settings with further exploration of cost-effectiveness, critical content and human resources.

Counting outcomes, coverage and quality for early child development programmes

Improved measurement in early child development (ECD) is a strategic focus of the WHO, UNICEF and World Bank Nurturing Care Framework. However, evidence-based approaches to monitoring and evaluation (M&E) of ECD projects in low-income and middle-income countries (LMIC) are lacking. The Grand Challenges Canada®-funded Saving Brains® ECD portfolio provides a unique opportunity to explore approaches to M&E of ECD programmes across diverse settings. Focused literature review and participatory mixed-method evaluation of the Saving Brains portfolio was undertaken using an adapted impact framework. Findings related to measurement of quality, coverage and outcomes for scaling ECD were considered. Thirty-nine ECD projects implemented in 23 LMIC were evaluated. Projects used a 'theory of change' based M&E approach to measure a range of inputs, outputs and outcomes. Over 29 projects measured cognitive, language, motor and socioemotional outcomes. 18 projects used developmental screening tools to measure outcomes, with a trade-off between feasibility and preferred practice. Environmental inputs such as the home environment were measured in 15 projects. Qualitative data reflected the importance of measurement of project quality and coverage, despite challenges measuring these constructs across contexts. Improved measurement of intervention quality and measurement of coverage, which requires definition of the numerator (ie, intervention) and denominator (ie, population in need/at risk), are needed for scaling ECD programmes. Innovation in outcome measurement, including intermediary outcome measures that are feasible and practical to measure in routine services, is also required, with disaggregation to better target interventions to those most in need and ensure that no child is left behind.