1
|
Gatiba P, Laury J, Steinhardt L, Hwang J, Thwing JI, Zulliger R, Emerson C, Gutman JR. Contextual Factors to Improve Implementation of Malaria Chemoprevention in Children: A Systematic Review. Am J Trop Med Hyg 2024; 110:69-78. [PMID: 38081055 DOI: 10.4269/ajtmh.23-0478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 09/15/2023] [Indexed: 01/05/2024] Open
Abstract
Malaria remains a leading cause of childhood morbidity and mortality in sub-Saharan Africa, particularly among children under 5 years of age. To help address this challenge, the WHO recommends chemoprevention for certain populations. For children and infants, the WHO recommends seasonal malaria chemoprevention (SMC), perennial malaria chemoprevention (PMC; formerly intermittent preventive treatment in infants [IPTi]), and, more recently, intermittent preventive treatment in school children (IPTsc). This review describes the contextual factors, including feasibility, acceptability, health equity, financial considerations, and values and preferences, that impact implementation of these strategies. A systematic search was conducted on July 5, 2022, and repeated April 13, 2023, to identify relevant literature. Two reviewers independently screened titles for eligibility, extracted data from eligible articles, and identified and summarized themes. Of 6,295 unique titles identified, 65 were included. The most frequently evaluated strategy was SMC (n = 40), followed by IPTi (n = 18) and then IPTsc (n = 6). Overall, these strategies were highly acceptable, although with IPTsc, there were community concerns with providing drugs to girls of reproductive age and the use of nonmedical staff for drug distribution. For SMC, door-to-door delivery resulted in higher coverage, improved caregiver acceptance, and reduced cost. Lower adherence was noted when caregivers were charged with giving doses 2 and 3 unsupervised. For SMC and IPTi, travel distances and inclement weather limited accessibility. Sensitization and caregiver education efforts, retention of high-quality drug distributors, and improved transportation were key to improving coverage. Additional research is needed to understand the role of community values and preferences in chemoprevention implementation.
Collapse
Affiliation(s)
- Peris Gatiba
- Public Health Institute, Oakland, California
- Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Jessica Laury
- Public Health Institute, Oakland, California
- Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Laura Steinhardt
- Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Jimee Hwang
- U.S. President's Malaria Initiative, Malaria Branch, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Julie I Thwing
- Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Rose Zulliger
- U.S. President's Malaria Initiative, United States Agency for International Development, Washington, District of Columbia
| | - Courtney Emerson
- U.S. President's Malaria Initiative, Malaria Branch, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Julie R Gutman
- Malaria Branch, Division of Parasitic Diseases and Malaria, Centers for Disease Control and Prevention, Atlanta, Georgia
| |
Collapse
|
2
|
Oyo-Ita A, Oduwole O, Arikpo D, Effa EE, Esu EB, Balakrishna Y, Chibuzor MT, Oringanje CM, Nwachukwu CE, Wiysonge CS, Meremikwu MM. Interventions for improving coverage of childhood immunisation in low- and middle-income countries. Cochrane Database Syst Rev 2023; 12:CD008145. [PMID: 38054505 PMCID: PMC10698843 DOI: 10.1002/14651858.cd008145.pub4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
BACKGROUND Immunisation plays a major role in reducing childhood morbidity and mortality. Getting children immunised against potentially fatal and debilitating vaccine-preventable diseases remains a challenge despite the availability of efficacious vaccines, particularly in low- and middle-income countries. With the introduction of new vaccines, this becomes increasingly difficult. There is therefore a current need to synthesise the available evidence on the strategies used to bridge this gap. This is a second update of the Cochrane Review first published in 2011 and updated in 2016, and it focuses on interventions for improving childhood immunisation coverage in low- and middle-income countries. OBJECTIVES To evaluate the effectiveness of intervention strategies to boost demand and supply of childhood vaccines, and sustain high childhood immunisation coverage in low- and middle-income countries. SEARCH METHODS We searched CENTRAL, MEDLINE, CINAHL, and Global Index Medicus (11 July 2022). We searched Embase, LILACS, and Sociological Abstracts (2 September 2014). We searched WHO ICTRP and ClinicalTrials.gov (11 July 2022). In addition, we screened reference lists of relevant systematic reviews for potentially eligible studies, and carried out a citation search for 14 of the included studies (19 February 2020). SELECTION CRITERIA Eligible studies were randomised controlled trials (RCTs), non-randomised RCTs (nRCTs), controlled before-after studies, and interrupted time series conducted in low- and middle-income countries involving children that were under five years of age, caregivers, and healthcare providers. DATA COLLECTION AND ANALYSIS We independently screened the search output, reviewed full texts of potentially eligible articles, assessed the risk of bias, and extracted data in duplicate, resolving discrepancies by consensus. We conducted random-effects meta-analyses and used GRADE to assess the certainty of the evidence. MAIN RESULTS Forty-one studies involving 100,747 participants are included in the review. Twenty studies were cluster-randomised and 15 studies were individually randomised controlled trials. Six studies were quasi-randomised. The studies were conducted in four upper-middle-income countries (China, Georgia, Mexico, Guatemala), 11 lower-middle-income countries (Côte d'Ivoire, Ghana, Honduras, India, Indonesia, Kenya, Nigeria, Nepal, Nicaragua, Pakistan, Zimbabwe), and three lower-income countries (Afghanistan, Mali, Rwanda). The interventions evaluated in the studies were health education (seven studies), patient reminders (13 studies), digital register (two studies), household incentives (three studies), regular immunisation outreach sessions (two studies), home visits (one study), supportive supervision (two studies), integration of immunisation services with intermittent preventive treatment of malaria (one study), payment for performance (two studies), engagement of community leaders (one study), training on interpersonal communication skills (one study), and logistic support to health facilities (one study). We judged nine of the included studies to have low risk of bias; the risk of bias in eight studies was unclear and 24 studies had high risk of bias. We found low-certainty evidence that health education (risk ratio (RR) 1.36, 95% confidence interval (CI) 1.15 to 1.62; 6 studies, 4375 participants) and home-based records (RR 1.36, 95% CI 1.06 to 1.75; 3 studies, 4019 participants) may improve coverage with DTP3/Penta 3 vaccine. Phone calls/short messages may have little or no effect on DTP3/Penta 3 vaccine uptake (RR 1.12, 95% CI 1.00 to 1.25; 6 studies, 3869 participants; low-certainty evidence); wearable reminders probably have little or no effect on DTP3/Penta 3 uptake (RR 1.02, 95% CI 0.97 to 1.07; 2 studies, 1567 participants; moderate-certainty evidence). Use of community leaders in combination with provider intervention probably increases the uptake of DTP3/Penta 3 vaccine (RR 1.37, 95% CI 1.11 to 1.69; 1 study, 2020 participants; moderate-certainty evidence). We are uncertain about the effect of immunisation outreach on DTP3/Penta 3 vaccine uptake in children under two years of age (RR 1.32, 95% CI 1.11 to 1.56; 1 study, 541 participants; very low-certainty evidence). We are also uncertain about the following interventions improving full vaccination of children under two years of age: training of health providers on interpersonal communication skills (RR 5.65, 95% CI 3.62 to 8.83; 1 study, 420 participants; very low-certainty evidence), and home visits (RR 1.29, 95% CI 1.15 to 1.45; 1 study, 419 participants; very low-certainty evidence). The same applies to the effect of training of health providers on interpersonal communication skills on the uptake of DTP3/Penta 3 by one year of age (very low-certainty evidence). The integration of immunisation with other services may, however, improve full vaccination (RR 1.29, 95% CI 1.16 to 1.44; 1 study, 1700 participants; low-certainty evidence). AUTHORS' CONCLUSIONS Health education, home-based records, a combination of involvement of community leaders with health provider intervention, and integration of immunisation services may improve vaccine uptake. The certainty of the evidence for the included interventions ranged from moderate to very low. Low certainty of the evidence implies that the true effect of the interventions might be markedly different from the estimated effect. Further, more rigorous RCTs are, therefore, required to generate high-certainty evidence to inform policy and practice.
Collapse
Affiliation(s)
- Angela Oyo-Ita
- Department of Community Health, University of Calabar Teaching Hospital, Calabar, Nigeria
| | - Olabisi Oduwole
- Department of Medical Laboratory Science, Achievers University, Owo, Nigeria
| | - Dachi Arikpo
- Cochrane Nigeria, Institute of Tropical Diseases Research and Prevention, University of Calabar Teaching Hospital, Calabar, Nigeria
| | - Emmanuel E Effa
- Internal Medicine, College of Medical Sciences, University of Calabar, Calabar, Nigeria
| | - Ekpereonne B Esu
- Department of Public Health, College of Medical Sciences, University of Calabar, Calabar, Nigeria
| | - Yusentha Balakrishna
- Biostatistics Unit, South African Medical Research Council, Durban, South Africa
| | - Moriam T Chibuzor
- Cochrane Nigeria, Institute of Tropical Diseases Research and Prevention, University of Calabar Teaching Hospital, Calabar, Nigeria
| | - Chioma M Oringanje
- GIDP Entomology and Insect Science, University of Tucson, Tucson, Arizona, USA
| | | | - Charles S Wiysonge
- Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
- Vaccine-Preventable Diseases Programme, World Health Organization Regional Office for Africa, Cité du Djoué, Brazzaville, Congo
| | - Martin M Meremikwu
- Department of Paediatrics, University of Calabar Teaching Hospital, Calabar, Nigeria
| |
Collapse
|
3
|
Runge M, Stahlfeld A, Ambrose M, Toh KB, Rahman S, Omoniwa OF, Bever CA, Oresanya O, Uhomoibhi P, Galatas B, Tibenderana JK, Gerardin J. Perennial malaria chemoprevention with and without malaria vaccination to reduce malaria burden in young children: a modelling analysis. Malar J 2023; 22:133. [PMID: 37095480 PMCID: PMC10124689 DOI: 10.1186/s12936-023-04564-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2023] [Accepted: 04/18/2023] [Indexed: 04/26/2023] Open
Abstract
BACKGROUND A recent WHO recommendation for perennial malaria chemoprevention (PMC) encourages countries to adapt dose timing and number to local conditions. However, knowledge gaps on the epidemiological impact of PMC and possible combination with the malaria vaccine RTS,S hinder informed policy decisions in countries where malaria burden in young children remains high. METHODS The EMOD malaria model was used to predict the impact of PMC with and without RTS,S on clinical and severe malaria cases in children under the age of two years (U2). PMC and RTS,S effect sizes were fit to trial data. PMC was simulated with three to seven doses (PMC-3-7) before the age of eighteen months and RTS,S with three doses, shown to be effective at nine months. Simulations were run for transmission intensities of one to 128 infectious bites per person per year, corresponding to incidences of < 1 to 5500 cases per 1000 population U2. Intervention coverage was either set to 80% or based on 2018 household survey data for Southern Nigeria as a sample use case. The protective efficacy (PE) for clinical and severe cases in children U2 was calculated in comparison to no PMC and no RTS,S. RESULTS The projected impact of PMC or RTS,S was greater at moderate to high transmission than at low or very high transmission. Across the simulated transmission levels, PE estimates of PMC-3 at 80% coverage ranged from 5.7 to 8.8% for clinical, and from 6.1 to 13.6% for severe malaria (PE of RTS,S 10-32% and 24.6-27.5% for clinical and severe malaria, respectively. In children U2, PMC with seven doses nearly averted as many cases as RTS,S, while the combination of both was more impactful than either intervention alone. When operational coverage, as seen in Southern Nigeria, increased to a hypothetical target of 80%, cases were reduced beyond the relative increase in coverage. CONCLUSIONS PMC can substantially reduce clinical and severe cases in the first two years of life in areas with high malaria burden and perennial transmission. A better understanding of the malaria risk profile by age in early childhood and on feasible coverage by age, is needed for selecting an appropriate PMC schedule in a given setting.
Collapse
Affiliation(s)
- Manuela Runge
- Department of Preventive Medicine, Institute for Global Health, Northwestern University, Chicago, IL USA
| | - Anne Stahlfeld
- Department of Preventive Medicine, Institute for Global Health, Northwestern University, Chicago, IL USA
| | - Monique Ambrose
- Institute for Disease Modeling, Bill and Melinda Gates Foundation, Seattle, USA
| | - Kok Ben Toh
- Department of Preventive Medicine, Institute for Global Health, Northwestern University, Chicago, IL USA
| | - Semiu Rahman
- Malaria Consortium Nigeria, 33 Pope John Paul Street, Off Gana Street, Maitama, Abuja-FCT Nigeria
| | - Omowunmi F. Omoniwa
- Malaria Consortium Nigeria, 33 Pope John Paul Street, Off Gana Street, Maitama, Abuja-FCT Nigeria
| | - Caitlin A. Bever
- Institute for Disease Modeling, Bill and Melinda Gates Foundation, Seattle, USA
| | - Olusola Oresanya
- Malaria Consortium Nigeria, 33 Pope John Paul Street, Off Gana Street, Maitama, Abuja-FCT Nigeria
| | - Perpetua Uhomoibhi
- National Malaria Elimination Programme, Federal Ministry of Health, Abuja, Nigeria
| | - Beatriz Galatas
- Global Malaria Programme, World Health Organization, Geneva, Switzerland
| | | | - Jaline Gerardin
- Department of Preventive Medicine, Institute for Global Health, Northwestern University, Chicago, IL USA
| |
Collapse
|
4
|
Cooper S, Schmidt BM, Sambala EZ, Swartz A, Colvin CJ, Leon N, Wiysonge CS. Factors that influence parents' and informal caregivers' views and practices regarding routine childhood vaccination: a qualitative evidence synthesis. Cochrane Database Syst Rev 2021; 10:CD013265. [PMID: 34706066 PMCID: PMC8550333 DOI: 10.1002/14651858.cd013265.pub2] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
BACKGROUND Childhood vaccination is one of the most effective ways to prevent serious illnesses and deaths in children. However, worldwide, many children do not receive all recommended vaccinations, for several potential reasons. Vaccines might be unavailable, or parents may experience difficulties in accessing vaccination services; for instance, because of poor quality health services, distance from a health facility, or lack of money. Some parents may not accept available vaccines and vaccination services. Our understanding of what influences parents' views and practices around childhood vaccination, and why some parents may not accept vaccines for their children, is still limited. This synthesis links to Cochrane Reviews of the effectiveness of interventions to improve coverage or uptake of childhood vaccination. OBJECTIVES - Explore parents' and informal caregivers' views and practices regarding routine childhood vaccination, and the factors influencing acceptance, hesitancy, or nonacceptance of routine childhood vaccination. - Develop a conceptual understanding of what and how different factors reduce parental acceptance of routine childhood vaccination. - Explore how the findings of this review can enhance our understanding of the related Cochrane Reviews of intervention effectiveness. SEARCH METHODS We searched MEDLINE, Embase, CINAHL, and three other databases for eligible studies from 1974 to June 2020. SELECTION CRITERIA We included studies that: utilised qualitative methods for data collection and analysis; focused on parents' or caregivers' views, practices, acceptance, hesitancy, or refusal of routine vaccination for children aged up to six years; and were from any setting globally where childhood vaccination is provided. DATA COLLECTION AND ANALYSIS We used a pre-specified sampling frame to sample from eligible studies, aiming to capture studies that were conceptually rich, relevant to the review's phenomenon of interest, from diverse geographical settings, and from a range of income-level settings. We extracted contextual and methodological data from each sampled study. We used a meta-ethnographic approach to analyse and synthesise the evidence. We assessed methodological limitations using a list of criteria used in previous Cochrane Reviews and originally based on the Critical Appraisal Skills Programme quality assessment tool for qualitative studies. We used the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in each finding. We integrated the findings of this review with those from relevant Cochrane Reviews of intervention effectiveness. We did this by mapping whether the underlying theories or components of trial interventions included in those reviews related to or targeted the overarching factors influencing parental views and practices regarding routine childhood vaccination identified by this review. MAIN RESULTS We included 145 studies in the review and sampled 27 of these for our analysis. Six studies were conducted in Africa, seven in the Americas, four in South-East Asia, nine in Europe, and one in the Western Pacific. Studies included urban and rural settings, and high-, middle-, and low-income settings. Many complex factors were found to influence parents' vaccination views and practices, which we divided into four themes. Firstly, parents' vaccination ideas and practices may be influenced by their broader ideas and practices surrounding health and illness generally, and specifically with regards to their children, and their perceptions of the role of vaccination within this context. Secondly, many parents' vaccination ideas and practices were influenced by the vaccination ideas and practices of the people they mix with socially. At the same time, shared vaccination ideas and practices helped some parents establish social relationships, which in turn strengthened their views and practices around vaccination. Thirdly, parents' vaccination ideas and practices may be influenced by wider political issues and concerns, and particularly their trust (or distrust) in those associated with vaccination programmes. Finally, parents' vaccination ideas and practices may be influenced by their access to and experiences of vaccination services and their frontline healthcare workers. We developed two concepts for understanding possible pathways to reduced acceptance of childhood vaccination. The first concept, 'neoliberal logic', suggests that many parents, particularly from high-income countries, understood health and healthcare decisions as matters of individual risk, choice, and responsibility. Some parents experienced this understanding as in conflict with vaccination programmes, which emphasise generalised risk and population health. This perceived conflict led some parents to be less accepting of vaccination for their children. The second concept, 'social exclusion', suggests that some parents, particularly from low- and middle-income countries, were less accepting of childhood vaccination due to their experiences of social exclusion. Social exclusion may damage trustful relationships between government and the public, generate feelings of isolation and resentment, and give rise to demotivation in the face of public services that are poor quality and difficult to access. These factors in turn led some parents who were socially excluded to distrust vaccination, to refuse vaccination as a form of resistance or a way to bring about change, or to avoid vaccination due to the time, costs, and distress it creates. Many of the overarching factors our review identified as influencing parents' vaccination views and practices were underrepresented in the interventions tested in the four related Cochrane Reviews of intervention effectiveness. AUTHORS' CONCLUSIONS Our review has revealed that parents' views and practices regarding childhood vaccination are complex and dynamic social processes that reflect multiple webs of influence, meaning, and logic. We have provided a theorised understanding of the social processes contributing to vaccination acceptance (or not), thereby complementing but also extending more individualistic models of vaccination acceptance. Successful development of interventions to promote acceptance and uptake of childhood vaccination will require an understanding of, and then tailoring to, the specific factors influencing vaccination views and practices of the group(s) in the target setting. The themes and concepts developed through our review could serve as a basis for gaining this understanding, and subsequent development of interventions that are potentially more aligned with the norms, expectations, and concerns of target users.
Collapse
Affiliation(s)
- Sara Cooper
- Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
- Division of Social and Behavioural Sciences, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa
| | - Bey-Marrié Schmidt
- School of Public Health, University of the Western Cape, Cape Town, South Africa
| | - Evanson Z Sambala
- Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
- School of Public Health, Kamuzu University of Health Sciences, Blantyre, Malawi
| | - Alison Swartz
- Division of Social and Behavioural Sciences, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa
| | - Christopher J Colvin
- Division of Social and Behavioural Sciences, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa
- Department of Public Health Sciences, University of Virginia, Charlottesville, Virginia, USA
| | - Natalie Leon
- Health Systems Research Unit, South African Medical Research Council, Cape Town, South Africa
- Department of Epidemiology, School of Public Health, Brown University, Providence, Rhode Island, USA
| | - Charles S Wiysonge
- Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa
- Department of Global Health, Stellenbosch University, Cape Town, South Africa
- Division of Epidemiology and Biostatistics, School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa
| |
Collapse
|
5
|
Ozawa S, Yemeke TT, Mitgang E, Wedlock PT, Higgins C, Chen HH, Pallas SW, Abimbola T, Wallace A, Bartsch SM, Lee BY. Systematic review of the costs for vaccinators to reach vaccination sites: Incremental costs of reaching hard-to-reach populations. Vaccine 2021; 39:4598-4610. [PMID: 34238610 PMCID: PMC10680154 DOI: 10.1016/j.vaccine.2021.05.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2021] [Revised: 04/07/2021] [Accepted: 05/06/2021] [Indexed: 11/24/2022]
Abstract
INTRODUCTION Economic evidence on how much it may cost for vaccinators to reach populations is important to plan vaccination programs. Moreover, knowing the incremental costs to reach populations that have traditionally been undervaccinated, especially those hard-to-reach who are facing supply-side barriers to vaccination, is essential to expanding immunization coverage to these populations. METHODS We conducted a systematic review to identify estimates of costs associated with getting vaccinators to all vaccination sites. We searched PubMed and the Immunization Delivery Cost Catalogue (IDCC) in 2019 for the following costs to vaccinators: (1) training costs; (2) labor costs, per diems, and incentives; (3) identification of vaccine beneficiary location; and (4) travel costs. We assessed if any of these costs were specific to populations that are hard-to-reach for vaccination, based on a framework for examining supply-side barriers to vaccination. RESULTS We found 19 studies describing average vaccinator training costs at $0.67/person vaccinated or targeted (SD $0.94) and $0.10/dose delivered (SD $0.07). The average cost for vaccinator labor and incentive costs across 29 studies was $2.15/dose (SD $2.08). We identified 13 studies describing intervention costs for a vaccinator to know the location of a beneficiary, with an average cost of $19.69/person (SD $26.65), and six studies describing vaccinator travel costs, with an average cost of $0.07/dose (SD $0.03). Only eight of these studies described hard-to-reach populations for vaccination; two studies examined incremental costs per dose to reach hard-to-reach populations, which were 1.3-2 times higher than the regular costs. The incremental cost to train vaccinators was $0.02/dose, and incremental labor costs for targeting hard-to-reach populations were $0.16-$1.17/dose. CONCLUSION Additional comparative costing studies are needed to understand the potential differential costs for vaccinators reaching the vaccination sites that serve hard-to-reach populations. This will help immunization program planners and decision-makers better allocate resources to extend vaccination programs.
Collapse
Affiliation(s)
- Sachiko Ozawa
- Division of Practice Advancement and Clinical Education, UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC, USA; Department of Maternal and Child Health, UNC Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC, USA.
| | - Tatenda T Yemeke
- Division of Practice Advancement and Clinical Education, UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC, USA
| | - Elizabeth Mitgang
- Public Health Informatics, Computational, and Operations Research (PHICOR), CUNY Graduate School of Public Health and Health Policy, New York City, NY, USA
| | - Patrick T Wedlock
- Public Health Informatics, Computational, and Operations Research (PHICOR), CUNY Graduate School of Public Health and Health Policy, New York City, NY, USA
| | - Colleen Higgins
- Division of Practice Advancement and Clinical Education, UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC, USA
| | - Hui-Han Chen
- Division of Practice Advancement and Clinical Education, UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC, USA
| | - Sarah W Pallas
- Global Immunization Division, U.S. Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA
| | - Taiwo Abimbola
- Global Immunization Division, U.S. Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA
| | - Aaron Wallace
- Global Immunization Division, U.S. Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA
| | - Sarah M Bartsch
- Public Health Informatics, Computational, and Operations Research (PHICOR), CUNY Graduate School of Public Health and Health Policy, New York City, NY, USA
| | - Bruce Y Lee
- Public Health Informatics, Computational, and Operations Research (PHICOR), CUNY Graduate School of Public Health and Health Policy, New York City, NY, USA
| |
Collapse
|
6
|
Audibert C, Tchouatieu AM. Perception of Malaria Chemoprevention Interventions in Infants and Children in Eight Sub-Saharan African Countries: An End User Perspective Study. Trop Med Infect Dis 2021; 6:tropicalmed6020075. [PMID: 34064620 PMCID: PMC8163176 DOI: 10.3390/tropicalmed6020075] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Revised: 05/03/2021] [Accepted: 05/06/2021] [Indexed: 11/16/2022] Open
Abstract
Preventive chemotherapy interventions have been identified as key tools for malaria prevention and control. Although a large number of publications have reported on the efficacy and safety profile of these interventions, little literature exists on end-user experience. The objective of this study was to provide insights on the perceptions and attitudes towards seasonal malaria chemoprevention (SMC) and intermittent preventive treatment of infants (IPTi) to identify drivers of and barriers to acceptance. A total of 179 in-depth qualitative interviews were conducted with community health workers (CHWs), health center managers, parents of children receiving chemoprevention, and national decision makers across eight countries in sub-Saharan Africa. The transcribed verbatim responses were coded and analyzed using a thematic approach. Findings indicate that, although SMC is largely accepted by end users, coverage remained below 100%. The main causes mentioned were children's absenteeism, children being sick, parents' reluctance, and lack of staff. Regarding IPTi, results from participants based in Sierra Leone showed that the intervention was generally accepted and perceived as efficacious. The main challenges were access to water, crushing the tablets, and high staff turnover. SMC and IPTi are perceived as valuable interventions. Our study identified the key elements that need to be considered to facilitate the expansion of these two interventions to different geographies or age groups.
Collapse
|
7
|
Omoniyi OS, Williams I. Realist Synthesis of the International Theory and Evidence on Strategies to Improve Childhood Vaccination in Low- and Middle-Income Countries: Developing Strategies for the Nigerian Healthcare System. Int J Health Policy Manag 2020; 9:274-285. [PMID: 32613799 PMCID: PMC7444436 DOI: 10.15171/ijhpm.2019.120] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2019] [Accepted: 11/12/2019] [Indexed: 11/09/2022] Open
Abstract
BACKGROUND Childhood vaccination coverage rates in low- and middle-income countries (LMICs) vary significantly, with some countries achieving higher rates than others. Several attempts have been made in Nigeria to achieve universal vaccination coverage but with limited success. This study aimed to analyse strategies used to improve childhood vaccine access and uptake in LMICs in order to inform strategy development for the Nigerian healthcare system. METHODS A realist synthesis approach was adopted in order to elucidate the contexts and mechanisms wherewith these strategies achieved their aim (or not). Nine databases were searched for relevant articles and 27 articles were included in the study. Programme theories were generated from the included articles, and data extraction was carried out paying particular attention to context, mechanism and outcomes configurations. RESULTS Interventions used in LMICs to improve vaccination coverage were categorised as follows: communication/ educational, reminder-type, incentives, social mobilisation, provider-directed strategies, health service integration and multi-pronged strategies. The strategies that appeared most likely to be effective in the health contexts of contemporary Nigeria include communication and educational interventions; employing informal change agents, and; monitoring and evaluation to strengthen communication. The programme theories for the use of reminders, social mobilisation, staff training and supportive supervision were observed in practice, and these strategies were generally successful within some contexts. By contrast, the use of monetary incentives in Nigeria is not supported by the evidence, although further research and evaluation is required. The integration of other interventions with routine immunisation (RI) to improve uptake was more effective when the perceived value of the other program was high. Adoption of multipronged interventions for hard to reach communities was beneficial. However, caution should be exercised because of varying levels of published evidence in respect of each intervention type and a relative lack of the rich description required to conduct a full realist analysis. CONCLUSION This paper adds to the evidence base on the adaption of strategies to improve vaccine access and uptake to the context of LMICs.
Collapse
Affiliation(s)
| | - Iestyn Williams
- Health Services Management Centre, University of Birmingham, Birmingham, UK
| |
Collapse
|
8
|
Optimal mode for delivery of seasonal malaria chemoprevention in Ouelessebougou, Mali: A cluster randomized trial. PLoS One 2018; 13:e0193296. [PMID: 29505578 PMCID: PMC5837084 DOI: 10.1371/journal.pone.0193296] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2017] [Accepted: 02/05/2018] [Indexed: 11/19/2022] Open
Abstract
Background Seasonal malaria chemoprevention (SMC), the administration of complete therapeutic courses of antimalarials to children aged 3–59 months during the malaria transmission season, is a new strategy recommended by the World Health Organization (WHO) for malaria control in Sahelian countries such as Mali with seasonal transmission. The strategy is a highly cost-effective approach to reduce malaria burden in these areas. Despite the substantial benefits of SMC on malaria infection and disease, the optimal approach to deliver SMC remains to be determined. While fixed-point delivery (FPD) and non-directly observed treatment (NDOT) by community health workers are logistically attractive, these need to be evaluated and compared to other modes of delivery for maximal coverage. Methods To determine the optimal mode fixed-point (FPD) vs door-to-door delivery (DDD); directly observed treatment (DOT) vs. non- directly observed treatment (NDOT)), 31 villages in four health sub-districts were randomized to receive three rounds of SMC with Sulfadoxine-pyrimethamine plus Amodiaquine (SP+AQ) at monthly intervals using one of the following methods: FPD+DOT; FPD+NDOT; DDD+DOT; DDD+NDOT. The primary endpoint was SMC coverage assessed by cross-sectional survey of 2,035 children at the end of intervention period. Results Coverage defined as the proportion of children who received all three days of SMC treatment during the three monthly rounds based information collected by interview (primary endpoint) was significantly higher in children who received SMC using DDD 74% (95% CI 69% - 80%) compared to FPD 60% (95% CI 50% - 70%); p = 0.009. It was similar in children who received SMC using DOT or NDOT 65%, (95% CI 55% - 76%) versus 68% (95% CI 57% - 79%); p = 0.72. Conclusions In summary, door-to-door delivery of SMC provides better coverage than FPD. Directly observed therapy, which requires more time and resources, did not improve coverage with SMC. Trial registration ClinicalTrials.gov NCT02646410
Collapse
|
9
|
Bright T, Felix L, Kuper H, Polack S. A systematic review of strategies to increase access to health services among children in low and middle income countries. BMC Health Serv Res 2017; 17:252. [PMID: 28381276 PMCID: PMC5382494 DOI: 10.1186/s12913-017-2180-9] [Citation(s) in RCA: 62] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2016] [Accepted: 03/21/2017] [Indexed: 12/02/2022] Open
Abstract
Background Universal Health Coverage is widely endorsed as the pivotal goal in global health, however substantial barriers to accessing health services for children in low and middle-income countries (LMIC) exist. Failure to access healthcare is an important contributor to child mortality in these settings. Barriers to access have been widely studied, however effective interventions to overcome barriers and increase access to services for children are less well documented. Methods We conducted a systematic review of effectiveness of interventions aimed at increasing access to health services for children aged 5 years and below in LMIC. Four databases (EMBASE, Global Health, MEDLINE, and PSYCINFO) were searched in January 2016. Studies were included if they evaluated interventions that aimed to increase: health care utilisation; immunisation uptake; and compliance with medication or referral. Randomised controlled trials and non-randomised controlled study designs were included in the review. A narrative approach was used to synthesise results. Results Fifty seven studies were included in the review. Approximately half of studies (49%) were conducted in sub-Saharan Africa. Most studies were randomised controlled trials (n = 44; 77%) with the remaining studies employing non-randomised designs. Very few studies were judged as high quality. Studies evaluated a diverse range of interventions and various outcomes. Supply side interventions included: delivery of services at or closer to home and service level improvements (eg. integration of services). Demand side interventions included: educational programmes, text messages, and financial or other incentives. Interventions that delivered services at or closer to home and text messages were in general associated with a significant improvement in relevant outcomes. A consistent pattern was not noted for the remaining studies. Conclusions This review fills a gap in the literature by providing evidence of the range and effectiveness of interventions that can be used to increase access for children aged ≤5 years in LMIC. It highlights some intervention areas that seem to show encouraging trends including text message reminders and delivery of services at or close to home. However, given the methodological limitations found in existing studies, the results of this review must be interpreted with caution. Systematic review registration PROSPERO CRD420160334200 Electronic supplementary material The online version of this article (doi:10.1186/s12913-017-2180-9) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Tess Bright
- International Centre for Evidence in Disability, London School of Hygiene & Tropical Medicine, London, UK.
| | - Lambert Felix
- International Centre for Evidence in Disability, London School of Hygiene & Tropical Medicine, London, UK
| | - Hannah Kuper
- International Centre for Evidence in Disability, London School of Hygiene & Tropical Medicine, London, UK
| | - Sarah Polack
- International Centre for Evidence in Disability, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
10
|
Oyo-Ita A, Wiysonge CS, Oringanje C, Nwachukwu CE, Oduwole O, Meremikwu MM. Interventions for improving coverage of childhood immunisation in low- and middle-income countries. Cochrane Database Syst Rev 2016; 7:CD008145. [PMID: 27394698 PMCID: PMC4981642 DOI: 10.1002/14651858.cd008145.pub3] [Citation(s) in RCA: 117] [Impact Index Per Article: 14.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
BACKGROUND Immunisation is a powerful public health strategy for improving child survival, not only by directly combating key diseases that kill children but also by providing a platform for other health services. However, each year millions of children worldwide, mostly from low- and middle-income countries (LMICs), do not receive the full series of vaccines on their national routine immunisation schedule. This is an update of the Cochrane review published in 2011 and focuses on interventions for improving childhood immunisation coverage in LMICs. OBJECTIVES To evaluate the effectiveness of intervention strategies to boost and sustain high childhood immunisation coverage in LMICs. SEARCH METHODS We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2016, Issue 4, part of The Cochrane Library. www.cochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 12 May 2016); MEDLINE In-Process and Other Non-Indexed Citations, MEDLINE Daily and MEDLINE 1946 to Present, OvidSP (searched 12 May 2016); CINAHL 1981 to present, EbscoHost (searched 12 May 2016); Embase 1980 to 2014 Week 34, OvidSP (searched 2 September 2014); LILACS, VHL (searched 2 September 2014); Sociological Abstracts 1952 - current, ProQuest (searched 2 September 2014). We did a citation search for all included studies in Science Citation Index and Social Sciences Citation Index, 1975 to present; Emerging Sources Citation Index 2015 to present, ISI Web of Science (searched 2 July 2016). We also searched the two Trials Registries: ICTRP and ClinicalTrials.gov (searched 5 July 2016) SELECTION CRITERIA: Eligible studies were randomised controlled trials (RCT), non-RCTs, controlled before-after studies, and interrupted time series conducted in LMICs involving children aged from birth to four years, caregivers, and healthcare providers. DATA COLLECTION AND ANALYSIS We independently screened the search output, reviewed full texts of potentially eligible articles, assessed risk of bias, and extracted data in duplicate; resolving discrepancies by consensus. We then conducted random-effects meta-analyses and used GRADE to assess the certainty of evidence. MAIN RESULTS Fourteen studies (10 cluster RCTs and four individual RCTs) met our inclusion criteria. These were conducted in Georgia (one study), Ghana (one study), Honduras (one study), India (two studies), Mali (one study), Mexico (one study), Nicaragua (one study), Nepal (one study), Pakistan (four studies), and Zimbabwe (one study). One study had an unclear risk of bias, and 13 had high risk of bias. The interventions evaluated in the studies included community-based health education (three studies), facility-based health education (three studies), household incentives (three studies), regular immunisation outreach sessions (one study), home visits (one study), supportive supervision (one study), information campaigns (one study), and integration of immunisation services with intermittent preventive treatment of malaria (one study).We found moderate-certainty evidence that health education at village meetings or at home probably improves coverage with three doses of diphtheria-tetanus-pertussis vaccines (DTP3: risk ratio (RR) 1.68, 95% confidence interval (CI) 1.09 to 2.59). We also found low-certainty evidence that facility-based health education plus redesigned vaccination reminder cards may improve DTP3 coverage (RR 1.50, 95% CI 1.21 to 1.87). Household monetary incentives may have little or no effect on full immunisation coverage (RR 1.05, 95% CI 0.90 to 1.23, low-certainty evidence). Regular immunisation outreach may improve full immunisation coverage (RR 3.09, 95% CI 1.69 to 5.67, low-certainty evidence) which may substantially improve if combined with household incentives (RR 6.66, 95% CI 3.93 to 11.28, low-certainty evidence). Home visits to identify non-vaccinated children and refer them to health clinics may improve uptake of three doses of oral polio vaccine (RR 1.22, 95% CI 1.07 to 1.39, low-certainty evidence). There was low-certainty evidence that integration of immunisation with other services may improve DTP3 coverage (RR 1.92, 95% CI 1.42 to 2.59). AUTHORS' CONCLUSIONS Providing parents and other community members with information on immunisation, health education at facilities in combination with redesigned immunisation reminder cards, regular immunisation outreach with and without household incentives, home visits, and integration of immunisation with other services may improve childhood immunisation coverage in LMIC. Most of the evidence was of low certainty, which implies a high likelihood that the true effect of the interventions will be substantially different. There is thus a need for further well-conducted RCTs to assess the effects of interventions for improving childhood immunisation coverage in LMICs.
Collapse
Affiliation(s)
- Angela Oyo-Ita
- Department of Community Health, University of Calabar Teaching HospitalCalabar, Nigeria
| | - Charles S Wiysonge
- Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch UniversityCape Town, South Africa
- Cochrane South Africa, South African Medical Research CouncilCape Town, South Africa
| | - Chioma Oringanje
- GIDP Entomology and Insect Science, University of TucsonTucson, USA
| | - Chukwuemeka E Nwachukwu
- GIDP Entomology and Insect Science, Excellence & Friends Management Consult (EFMC)Abuja, Nigeria
| | - Olabisi Oduwole
- Institute of Tropical Diseases Research and Prevention, University of Calabar Teaching Hospital (ITDR/P)Calabar, Nigeria
| | - Martin M Meremikwu
- Department of Paediatrics, University of Calabar Teaching HospitalCalabar, Nigeria
| |
Collapse
|
11
|
Affiliation(s)
- Christine Stabell Benn
- Research Centre for Vitamins and Vaccines, Bandim Health Project, Statens Serum Institut, Copenhagen DK-2300, Denmark.
| | | |
Collapse
|
12
|
Dicko A, Konare M, Traore D, Testa J, Salamon R, Doumbo O, Rogier C. The implementation of malaria intermittent preventive trialtreatment with sulphadoxine-pyrimethamine in infants reduced all-cause mortality in the district of Kolokani, Mali: results from a cluster randomized control. Malar J 2012; 11:73. [PMID: 22423611 PMCID: PMC3359262 DOI: 10.1186/1475-2875-11-73] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2011] [Accepted: 03/16/2012] [Indexed: 11/10/2022] Open
Abstract
Background Malaria intermittent preventive treatment of malaria in infant with sulphadoxine-pyrimethamine (IPTi-SP) reduced the incidence of malaria and anaemia by 30% and 20% respectively. The strategy is now a recommended policy for malaria control. However, there was no published study on the impact of the strategy on mortality. The present study assessed the impact of the implementation of IPTi-SP in health services in Mali on all-cause mortality. Methods The 22 health sub-districts of the district of Kolokani were randomized at a 1:1 ratio to either receive IPTi-SP or to serve as a control. The IPTi-SP was implemented for two years starting December 2006. Information on births and deaths through 31 March, 2009 was collected on all children who reached four months of age on 1 December, 2006, likely to be exposed to the intervention in 75 localities randomly selected in each zone. Results A total of 5,882 children (2,869 from the intervention zone and 3,013 from the nonintervention zone) who reached four months of age between 1 December, 2006 and 1 December, 2008 were surveyed between the age of four months to the age of 18 months from 1 December, 2006 to 31 March, 2009. In the cohort of four to 18 months of age, the mortality rate per 1,000 children was 2.53 in the intervention zone compared to 3.46 in the nonintervention zone, gender and season adjusted mortality rate ratio (MRR) = 0.73 (95% CI 0.55-0.97, p = 0.029). In the cohort of the four to 12 months of age, mortality rates per 1,000 children were 2.22 in the intervention zone and 3.13 in the non-intervention zone, MRR = 0.71 (95% CI 0.49-1.02, p = 0.064) adjusted for gender and season. Conclusion The implementation of the IPTi-SP resulted in a substantial reduction in all-cause mortality in children. The results of this study support the adoption and the implementation of IPTi-SP as malaria control strategy. Trial Registration ClinicalTrials.govNCT00766662
Collapse
Affiliation(s)
- Alassane Dicko
- Malaria Research and Training Center, Faculty of Medicine Pharmacy and Dentistry, University of Bamako, PO Box 1805, Bamako, Mali.
| | | | | | | | | | | | | |
Collapse
|
13
|
de Sousa A, Rabarijaona LP, Ndiaye JL, Sow D, Ndyiae M, Hassan J, Lambo N, Adovohekpe P, Guidetti F, Recht J, Affo A. Acceptability of coupling intermittent preventive treatment in infants with the expanded programme on immunization in three francophone countries in Africa. Trop Med Int Health 2011; 17:308-15. [PMID: 22128893 DOI: 10.1111/j.1365-3156.2011.02915.x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
OBJECTIVE Intermittent preventive treatment in infants (IPTi) is a malaria control strategy currently recommended by WHO for implementation at scale in Africa, consisting of administration of sulphadoxine-pyrimethamine (SP) coupled with routine immunizations offered to children under 1 year. In this study, we analysed IPTi acceptability by communities and health staff. METHODS Direct observation, in-depth interviews (IDIs) and focus group discussions (FGDs) were conducted in Benin, Madagascar and Senegal during IPTi pilot implementation. Villages were stratified by immunization coverage. Data were transcribed and analysed using NVivo7 software. RESULTS Communities' knowledge of malaria aetiology and diagnosis was good, although generally villagers did not seek treatment at health centres as their first choice. Perceptions and attitudes towards IPTi were very positive among communities and health workers. A misconception that SP was an antipyretic that prevents post-vaccinal fever contributed to IPTi's acceptability. No refusals or negative rumours related to IPTi coupling with immunizations were identified, and IPTi did not negatively influence attitudes towards other malaria control strategies. Healthcare decisions about children, normatively made by the father, are starting to shift to educated and financially independent mothers. DISCUSSION Intermittent preventive treatment in infants is well accepted by providers and communities, showing a synergic acceptability when coupled with routine immunizations. However, a misconception that SP alleviates fever should be addressed when scaling up implementation.
Collapse
|
14
|
Patouillard E, Conteh L, Webster J, Kweku M, Chandramohan D, Greenwood B. Coverage, adherence and costs of intermittent preventive treatment of malaria in children employing different delivery strategies in Jasikan, Ghana. PLoS One 2011; 6:e24871. [PMID: 22073137 PMCID: PMC3207811 DOI: 10.1371/journal.pone.0024871] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2011] [Accepted: 08/22/2011] [Indexed: 11/18/2022] Open
Abstract
BACKGROUND Intermittent preventive treatment of malaria in children (IPTc) involves the administration of a course of anti-malarial drugs at specified time intervals to children at risk of malaria regardless of whether or not they are known to be infected. IPTc provides a high level of protection against uncomplicated and severe malaria, with monthly sulphadoxine-pyrimethamine plus amodiaquine (SP&AQ) and sulphadoxine-pyrimethamine plus piperaquine being the most efficacious regimens. A key challenge is the identification of a cost-effective delivery strategy. METHODS A community randomized trial was undertaken in Jasikan district, Ghana to assess IPTc effectiveness and costs using SP&AQ delivered in three different ways. Twelve villages were randomly selected to receive IPTc from village health workers (VHWs) or facility-based nurses working at health centres' outpatient departments (OPD) or EPI outreach clinics. Children aged 3 to 59 months-old received one IPT course (three doses) in May, June, September and October. Effectiveness was measured in terms of children covered and adherent to a course and delivery costs were calculated in financial and economic terms using an ingredient approach from the provider perspective. RESULTS The economic cost per child receiving at least the first dose of all 4 courses was US$4.58 when IPTc was delivered by VHWs, US$4.93 by OPD nurses and US$ 5.65 by EPI nurses. The unit economic cost of receiving all 3 doses of all 4 courses was US$7.56 and US$8.51 when IPTc was delivered by VHWs or facility-based nurses respectively. The main cost driver for the VHW delivery was supervision, reflecting resources used for travelling to more remote communities rather than more intense supervision, and for OPD and EPI delivery, it was the opportunity cost of the time spent by nurses in dispensing IPTc. CONCLUSIONS VHWs achieve higher IPTc coverage and adherence at lower costs than facility-based nurses in Jasikan district, Ghana. TRIAL REGISTRATION ClinicalTrials.gov NCT00119132.
Collapse
Affiliation(s)
- Edith Patouillard
- Department of Global Health and Development, London School of Hygiene and Tropical Medicine, London, England.
| | | | | | | | | | | |
Collapse
|