Using a nonparametric multilevel latent Markov model to evaluate diagnostics for trachoma

Model-based two-way clustering of second-level units in ordinal multilevel latent Markov models

In this paper, an ordinal multilevel latent Markov model based on separate random effects is proposed. In detail, two distinct second-level discrete effects are considered in the model, one affecting the initial probability vector and the other affecting the transition probability matrix of the first-level ordinal latent Markov process. To model these separate effects, we consider a bi-dimensional mixture specification that allows to avoid unverifiable assumptions on the random effect distribution and to derive a two-way clustering of second-level units. Starting from a general model where the two random effects are dependent, we also obtain the independence model as a special case. The proposal is applied to data on the physical health status of a sample of elderly residents grouped into nursing homes. A simulation study assessing the performance of the proposal is also included.

Performance of molecular methods for the detection of Salmonella in human stool specimens

Background: The relationship between asymptomatic Salmonella exposure within the gastrointestinal tract and Salmonella bacteraemia is poorly understood, in part due to the low sensitivity of stool culture and the lack of validated molecular diagnostic tests for the detection of Salmonella in the stool. The study aimed to determine a reliable molecular diagnostic test for Salmonella in stool specimens.

Methods: We optimised an in-house monoplex real-time polymerase chain reaction (PCR) for the detection of Salmonellattr and InvA genes in stool by including a selenite broth pre-culture step for Salmonella before DNA extraction and validated their specificity against other local common pathogens. Then we assessed their performance against a well-validated multiplex PCR targeting the same ttr and InvA genes and against stool culture using clinical stool specimens collected from a cohort of 50 asymptomatic healthy Malawian children that were sampled at 1-month intervals over 12 months. We employed a latent Markov model to estimate the specificities and sensitivities of PCR methods.

Results: Ttr and InvA primers were both able to detect all the different Salmonella serovars tested and had superior limits of detection when DNA was extracted after selenite pre-culture. T tr sensitivity and specificity for monoplex-PCR were (99.53%, 95.46%) and for multiplex-PCR (90.30%, 99.30%) respectively. InvA specificity and specificity for using monoplex-PCR was (95.06%, 90.31%) and multiplex-PCRs (89.41%, 98.00%) respectively. Sensitivity and specificity for standard stool culture were 62.88% and 99.99%, respectively. Culture showed the highest PPV (99.73%), and monoplex- ttr had the highest NPV (99.67%).

Conclusion: Test methods demonstrated high concordance, although stool culture and monoplexed ttr primers had superior specificity and sensitivity, respectively. The use of selenite pre-enrichment step increased Salmonella detection rate. Taken together, molecular detection methods used here could be used to reveal the true extent of both asymptomatic and symptomatic Salmonella exposure events.

Implications of the COVID-19 pandemic in eliminating trachoma as a public health problem

Background

Progress towards elimination of trachoma as a public health problem has been substantial, but the coronavirus disease 2019 (COVID-19) pandemic has disrupted community-based control efforts.

Methods

We use a susceptible-infected model to estimate the impact of delayed distribution of azithromycin treatment on the prevalence of active trachoma.

Results

We identify three distinct scenarios for geographic districts depending on whether the basic reproduction number and the treatment-associated reproduction number are above or below a value of 1. We find that when the basic reproduction number is <1, no significant delays in disease control will be caused. However, when the basic reproduction number is >1, significant delays can occur. In most districts, 1 y of COVID-related delay can be mitigated by a single extra round of mass drug administration. However, supercritical districts require a new paradigm of infection control because the current strategies will not eliminate disease.

Conclusions

If the pandemic can motivate judicious, community-specific implementation of control strategies, global elimination of trachoma as a public health problem could be accelerated.

Insights from mathematical modelling and quantitative analysis on the proposed 2030 goals for trachoma

Trachoma is a neglected tropical disease and the leading infectious cause of blindness worldwide. The current World Health Organization goal for trachoma is elimination as a public health problem, defined as reaching a prevalence of trachomatous inflammation-follicular below 5% in children (1-9 years) and a prevalence of trachomatous trichiasis in adults below 0.2%. Current targets to achieve elimination were set to 2020 but are being extended to 2030. Mathematical and statistical models suggest that 2030 is a realistic timeline for elimination as a public health problem in most trachoma endemic areas. Although the goal can be achieved, it is important to develop appropriate monitoring tools for surveillance after having achieved the elimination target to check for the possibility of resurgence. For this purpose, a standardized serological approach or the use of multiple diagnostics in complement would likely be required.

Insights from mathematical modelling and quantitative analysis on the proposed 2030 goals for trachoma

Trachoma is a neglected tropical disease and the leading infectious cause of blindness worldwide. The current World Health Organization goal for trachoma is elimination as a public health problem, defined as reaching a prevalence of trachomatous inflammation-follicular below 5% in children (1-9 years) and a prevalence of trachomatous trichiasis in adults below 0.2%. Current targets to achieve elimination were set to 2020 but are being extended to 2030. Mathematical and statistical models suggest that 2030 is a realistic timeline for elimination as a public health problem in most trachoma endemic areas. Although the goal can be achieved, it is important to develop appropriate monitoring tools for surveillance after having achieved the elimination target to check for the possibility of resurgence. For this purpose, a standardized serological approach or the use of multiple diagnostics in complement would likely be required.

Trachoma Prevalence After Discontinuation of Mass Azithromycin Distribution

Background

As the World Health Organization seeks to eliminate trachoma by 2020, countries are beginning to control the transmission of trachomatous inflammation–follicular (TF) and discontinue mass drug administration (MDA) with oral azithromycin. We evaluated the effect of MDA discontinuation on TF_1–9 prevalence at the district level.

Methods

We extracted from the available data districts with an impact survey at the end of their program cycle that initiated discontinuation of MDA (TF_1–9 prevalence <5%), followed by a surveillance survey conducted to determine whether TF_1–9 prevalence remained below the 5% threshold, warranting discontinuation of MDA. Two independent analyses were performed, 1 regression based and 1 simulation based, that assessed the change in TF_1–9 from the impact survey to the surveillance survey.

Results

Of the 220 districts included, TF_1–9 prevalence increased to >5% from impact to surveillance survey in 9% of districts. Regression analysis indicated that impact survey TF_1–9 prevalence was a significant predictor of surveillance survey TF_1–9 prevalence. The proportion of simulations with >5% TF_1–9 prevalence in the surveillance survey was 2%, assuming the survey was conducted 4 years after MDA.

Conclusion

An increase in TF_1–9 prevalence may represent disease resurgence but could also be due to measurement error. Improved diagnostic tests are crucial to elimination of TF_1–9 as a public health problem.

Implications of the COVID-19 pandemic on eliminating trachoma as a public health problem

Background:

Progress towards elimination of trachoma as a public health problem has been substantial, but the COVID-19 pandemic has disrupted community-based control efforts.

Methods:

We use a susceptible-infected model to estimate the impact of delayed distribution of azithromycin treatment on the prevalence of active trachoma.

Results:

We identify three distinct scenarios for geographic districts depending on whether the basic reproduction number and the treatment-associated reproduction number are above or below a value of one. We find that when the basic reproduction number is below one, no significant delays in disease control will be caused. However, when the basic reproduction number is above one, significant delays can occur. In most districts a year of COVID-related delay can be mitigated by a single extra round of mass drug administration. However, supercritical districts require a new paradigm of infection control because the current strategies will not eliminate disease.

Conclusion:

If the pandemic can motivate judicious, community-specific implementation of control strategies, global elimination of trachoma as a public health problem could be accelerated.

Performance of molecular methods for the detection of Salmonella in human stool specimens

Background: The relationship between asymptomatic Salmonella exposure within the gastrointestinal tract and Salmonella bacteraemia is poorly understood, in part due to the low sensitivity of stool culture, and the lack of validated molecular diagnostic tests for the detection of Salmonella in stool. The study aimed to determine a reliable molecular diagnostic test for Salmonella in stool specimens. Methods: We optimized an in-house monoplex real time polymerase chain reaction (PCR) for the detection of Salmonella TTR and InvA genes in stool by including a selenite broth pre-culture step for Salmonella before DNA extraction, and validated their specificity against other local common pathogens. Then we assessed their performance  against a well-validated multiplex PCR targeting the same TTR and InvA genes, and against stool culture using clinical stool specimens collected from a cohort of 50 asymptomatic healthy Malawian children that were sampled at 1-month intervals over a period of 12 months. We employed a latent Markov model to estimate the specificities and sensitivities of PCR methods. Results: TTR and InvA primers were both able to detect all the different Salmonella serovars tested, and had superior limits of detection if DNA was extracted after selenite pre-culture. TTR sensitivity and specificity for monoplex-PCR were (99.53%, 95.46%) and for multiplex-PCR (90.30%, 99.30%) respectively. InvA specificity and specificity for using monoplex-PCR was (95.06%, 90.31%) and multiplex-PCRs (89.41%, 98.00%) respectively. Sensitivity and specificity for standard stool culture were 62.88% and 99.99% respectively. Culture showed the highest PPV (99.73%) and mono-TTR had the highest NPV (99.67%). Conclusion: Test methods demonstrated high concordance although stool culture and monoplexed TTR primers had superior specificity and sensitivity respectively. The use of selenite pre-enrichment step increased Salmonella detection rate. Taken together, molecular detection methods used here could be used to reveal the true extent of both asymptomatic and symptomatic Salmonella exposure events.

A multilevel latent Markov model for the evaluation of nursing homes' performance

The periodic evaluation of health care services is a primary concern for many institutions. We consider services provided by nursing homes with the aim of ranking a set of these structures with respect to their effect on resident health status. Since the overall health status is not directly observable, and given the longitudinal and multilevel structure of the available data, we rely on latent variable models and, in particular, on a multilevel latent Markov model where residents and nursing homes are the first and the second level units, respectively. The model includes individual covariates to account for resident characteristics. The impact of nursing home membership is modelled through a pair of random effects affecting the initial distribution and the transition probabilities between different levels of health status. Through the prediction of these random effects we obtain a ranking of the nursing homes. Furthermore, the proposed model accounts for nonignorable dropout due to resident death, which typically occurs in these contexts. The motivating dataset is gathered from the Long Term Care Facilities programme, a health care protocol implemented in Umbria (Italy). Our results show that differences in performance between nursing homes are statistically significant.

Mathematical Modelling of Trachoma Transmission, Control and Elimination

The World Health Organization has targeted the elimination of blinding trachoma by the year 2020. To this end, the Global Elimination of Blinding Trachoma (GET, 2020) alliance relies on a four-pronged approach, known as the SAFE strategy (S for trichiasis surgery; A for antibiotic treatment; F for facial cleanliness and E for environmental improvement). Well-constructed and parameterized mathematical models provide useful tools that can be used in policy making and forecasting in order to help to control trachoma and understand the feasibility of this large-scale elimination effort. As we approach this goal, the need to understand the transmission dynamics of infection within areas of different endemicities, to optimize available resources and to identify which strategies are the most cost-effective becomes more pressing. In this study, we conducted a review of the modelling literature for trachoma and identified 23 articles that included a mechanistic or statistical model of the transmission, dynamics and/or control of (ocular) Chlamydia trachomatis. Insights into the dynamics of trachoma transmission have been generated through both deterministic and stochastic models. A large body of the modelling work conducted to date has shown that, to varying degrees of effectiveness, antibiotic administration can reduce or interrupt trachoma transmission. However, very little analysis has been conducted to consider the effect of nonpharmaceutical interventions (and particularly the F and E components of the SAFE strategy) in helping to reduce transmission. Furthermore, very few of the models identified in the literature review included a structure that permitted tracking of the prevalence of active disease (in the absence of active infection) and the subsequent progression to disease sequelae (the morbidity associated with trachoma and ultimately the target of GET 2020 goals). This represents a critical gap in the current trachoma modelling literature, which makes it difficult to reliably link infection and disease. In addition, it hinders the application of modelling to assist the public health community in understanding whether trachoma programmes are on track to reach the GET goals by 2020. Another gap identified in this review was that of the 23 articles examined, only one considered the cost-effectiveness of the interventions implemented. We conclude that although good progress has been made towards the development of modelling frameworks for trachoma transmission, key components of disease sequelae representation and economic evaluation of interventions are currently missing from the available literature. We recommend that rapid advances in these areas should be urgently made to ensure that mathematical models for trachoma transmission can robustly guide elimination efforts and quantify progress towards GET 2020.

Markov Model Predicts Changes in STH Prevalence during Control Activities Even with a Reduced Amount of Baseline Information

Background

Estimating the reduction in levels of infection during implementation of soil-transmitted helminth (STH) control programmes is important to measure their performance and to plan interventions. Markov modelling techniques have been used with some success to predict changes in STH prevalence following treatment in Viet Nam. The model is stationary and to date, the prediction has been obtained by calculating the transition probabilities between the different classes of intensity following the first year of drug distribution and assuming that these remain constant in subsequent years. However, to run this model longitudinal parasitological data (including intensity of infection) are required for two consecutive years from at least 200 individuals. Since this amount of data is not often available from STH control programmes, the possible application of the model in control programme is limited. The present study aimed to address this issue by adapting the existing Markov model to allow its application when a more limited amount of data is available and to test the predictive capacities of these simplified models.

Method

We analysed data from field studies conducted with different combination of three parameters: (i) the frequency of drug administration; (ii) the drug distributed; and (iii) the target treatment population (entire population or school-aged children only). This analysis allowed us to define 10 sets of standard transition probabilities to be used to predict prevalence changes when only baseline data are available (simplified model 1). We also formulated three equations (one for each STH parasite) to calculate the predicted prevalence of the different classes of intensity from the total prevalence. These equations allowed us to design a simplified model (SM2) to obtain predictions when the classes of intensity at baseline were not known. To evaluate the performance of the simplified models, we collected data from the scientific literature on changes in STH prevalence during the implementation of 26 control programmes in 16 countries. Using the baseline data observed, we applied the simplified models and predicted the onward prevalence of STH infection at each time-point for which programme data were available. We then compared the output from the model with the observed data from the programme.

Results

The comparison between the model-predicted prevalence and the observed values demonstrated a good accuracy of the predictions. In 77% of cases the original model predicted a prevalence within five absolute percentage points from the observed figure, for the simplified model one in 69% of cases and for the simplified model two in 60% of cases. We consider that the STH Markov model described here could be an important tool for programme managers to monitor the progress of their control programmes and to select the appropriate intervention. We also developed, and made freely available online, a software tool to enable the use of the STH Markov model by personnel with limited knowledge of mathematical models.

Several million children are periodically dewormed to prevent the impairments to health and the economy caused by soil-transmitted helminths in endemic communities. It is important that managers of STH control programmes be able to anticipate the impact of the control measures on the prevalence of the diseases for two main purposes:

to select the most appropriate intervention; and

to be alerted in case of changes in prevalence that are less favourable than the expected values.

We therefore developed a model that predicts such changes, and also included modifications to enable its use in cases where a limited amount of data is available (SM1 and SM2). We retrospectively tested the performances of the model on 26 control programmes comparing the prevalence predicted by the model with those observed during programme implementation: in 77% of cases the prevalence predicted by the original model was within five percentage points of the observed values (69% in SM1 and 60% in SM2). We consider the performances of these models to be satisfactory. We designed a free online software to facilitate the use of these models by programme managers who may not be confident with modelling procedures.

Short-term forecasting of the prevalence of clinical trachoma: utility of including delayed recovery and tests for infection

Background

The World Health Organization aims to control blinding trachoma by 2020. Decisions on whether to start and stop mass treatments and when to declare that control has been achieved are currently based on clinical examination data generated in population-based surveys. Thresholds are based on the district-level prevalence of trachomatous inflammation–follicular (TF) in children aged 1–9 years. Forecasts of which districts may and may not meet TF control goals by the 2020 target date could affect resource allocation in the next few years.

Methods

We constructed a hidden Markov model fit to the prevalence of two clinical signs of trachoma and PCR data in 24 communities from the recent PRET-Niger trial. The prevalence of TF in children in each community at 36 months was forecast given data from earlier time points. Forecasts were scored by the likelihood of the observed results. We assessed whether use of TF with additional TI and PCR data rather than just the use of TF alone improves forecasts, and separately whether incorporating a delay in TF recovery is beneficial.

Results

Including TI and PCR data did not significantly improve forecasts of TF. Forecasts of TF prevalence at 36 months by the model with the delay in TF recovery were significantly better than forecasts by the model without the delay in TF recovery (p = 0.003). A zero-inflated truncated normal observation model was better than a truncated normal observation model, and better than a sensitivity-specificity observation model.

Conclusion

The results in this study suggest that future studies could consider using just TF data for forecasting, and should include a delay in TF recovery.

Trial registration

Clinicaltrials.gov NCT00792922

Electronic supplementary material

The online version of this article (doi:10.1186/s13071-015-1115-8) contains supplementary material, which is available to authorized users.

Short-term Forecasting of the Prevalence of Trachoma: Expert Opinion, Statistical Regression, versus Transmission Models

Background

Trachoma programs rely on guidelines made in large part using expert opinion of what will happen with and without intervention. Large community-randomized trials offer an opportunity to actually compare forecasting methods in a masked fashion.

Methods

The Program for the Rapid Elimination of Trachoma trials estimated longitudinal prevalence of ocular chlamydial infection from 24 communities treated annually with mass azithromycin. Given antibiotic coverage and biannual assessments from baseline through 30 months, forecasts of the prevalence of infection in each of the 24 communities at 36 months were made by three methods: the sum of 15 experts’ opinion, statistical regression of the square-root-transformed prevalence, and a stochastic hidden Markov model of infection transmission (Susceptible-Infectious-Susceptible, or SIS model). All forecasters were masked to the 36-month results and to the other forecasts. Forecasts of the 24 communities were scored by the likelihood of the observed results and compared using Wilcoxon’s signed-rank statistic.

Findings

Regression and SIS hidden Markov models had significantly better likelihood than community expert opinion (p = 0.004 and p = 0.01, respectively). All forecasts scored better when perturbed to decrease Fisher’s information. Each individual expert’s forecast was poorer than the sum of experts.

Interpretation

Regression and SIS models performed significantly better than expert opinion, although all forecasts were overly confident. Further model refinements may score better, although would need to be tested and compared in new masked studies. Construction of guidelines that rely on forecasting future prevalence could consider use of mathematical and statistical models.

Forecasts of infectious diseases are rarely made in a falsifiable manner. Trachoma trials offer an opportunity to actually compare forecasting methods in a masked fashion. The World Health Organization recommends at least three annual antibiotic mass drug administrations where the prevalence of trachoma is greater than 10% in children aged 1–9 years, with coverage at least at 80%. The Program for the Rapid Elimination of Trachoma trials estimated longitudinal prevalence of ocular chlamydial infection from 24 communities treated annually with mass azithromycin. Here, we compared forecasts of the prevalence of infection in each of the 24 communities at 36 months (given antibiotic coverage and biannual assessments from baseline through 30 months, and masked to the 36-month assessments) made by experts, statistical regression, and a transmission model. The transmission model was better than regression, with both far better than experts’ opinion. Construction of guidelines that rely on forecasting future prevalence could consider use of mathematical and statistical models.

The limits of medical interventions for the elimination of preventable blindness

Background: Health authorities are working toward the global elimination of trachoma by the year 2020 with actions focused on the World Health Organization SAFE strategy (surgery of trichiasis, antibiotics, face washing and environmental changes) with emphasis on hygienist approaches for education.

Objectives: The present survey was performed to assess the sustainability of the SAFE strategy 3 years after trachoma was eliminated from 6 villages.

Methods: In February 2013 a rapid trachoma assessment was conducted in 6 villages of Kolofata’s district, Extreme north Region, Cameroon, where trachoma was eliminated in 2010. A total of 300 children (1–10 years) from 6 villages were examined by trained staff.

Results: The prevalence of active trachoma (children aged > 1 and < 10 years) in 2013 was 15% and in at least 25% was observed absence of face washing and flies in their eyes and nose. Income level, quality of roads, hygiene, and illiteracy were similar in all the villages; they did not change between 2010 and 2013 and could not be analyzed as independent risk factors.

Discussion: The heterogeneity of methods described for clinical trials makes it inappropriate to conduct meta-analysis for the present and for other SAFE-related trials. The results obtained after implementation the SAFE strategy (recurrence) reveal that the causes (infectious agents and dirtiness) and effects (illness) were not connected by illiterate people living under conditions of extreme poverty. So far, antibiotics, surgery and hygiene education are insufficient for the sustainability of trachoma elimination and highlight that hypothetic-deductive processes seem not operational after implementing the awareness campaigns. Trachoma recurrence detected in 2013 in sedentary populations of Kolofata receiving efficacious treatments against Chlamydia sp. suggest that the elimination goals will be delayed if strategies are limited to medical actions. Restricting efforts to repeated pharmacological and surgical interventions for people infected with susceptible bacteria could be understood as the hidden side of a passive attitude toward basic education actions.

A latent Markov modelling approach to the evaluation of circulating cathodic antigen strips for schistosomiasis diagnosis pre- and post-praziquantel treatment in Uganda

Regular treatment with praziquantel (PZQ) is the strategy for human schistosomiasis control aiming to prevent morbidity in later life. With the recent resolution on schistosomiasis elimination by the 65th World Health Assembly, appropriate diagnostic tools to inform interventions are keys to their success. We present a discrete Markov chains modelling framework that deals with the longitudinal study design and the measurement error in the diagnostic methods under study. A longitudinal detailed dataset from Uganda, in which one or two doses of PZQ treatment were provided, was analyzed through Latent Markov Models (LMMs). The aim was to evaluate the diagnostic accuracy of Circulating Cathodic Antigen (CCA) and of double Kato-Katz (KK) faecal slides over three consecutive days for Schistosoma mansoni infection simultaneously by age group at baseline and at two follow-up times post treatment. Diagnostic test sensitivities and specificities and the true underlying infection prevalence over time as well as the probabilities of transitions between infected and uninfected states are provided. The estimated transition probability matrices provide parsimonious yet important insights into the re-infection and cure rates in the two age groups. We show that the CCA diagnostic performance remained constant after PZQ treatment and that this test was overall more sensitive but less specific than single-day double KK for the diagnosis of S. mansoni infection. The probability of clearing infection from baseline to 9 weeks was higher among those who received two PZQ doses compared to one PZQ dose for both age groups, with much higher re-infection rates among children compared to adolescents and adults. We recommend LMMs as a useful methodology for monitoring and evaluation and treatment decision research as well as CCA for mapping surveys of S. mansoni infection, although additional diagnostic tools should be incorporated in schistosomiasis elimination programs.

Schistosomiasis remains one of the most prevalent parasitic diseases in developing countries, with Schistosoma mansoni being the most widespread of the human-infecting schistosomes. For the routine surveillance of human S. mansoni infection more “field-applicable,” sensitive, and cost-effective diagnostics that replicate faecal samples over several consecutive days [the Kato-Katz (KK) method], are needed. We propose a statistical modelling framework in order to evaluate the diagnostic performance of the urine strip test for Circulating Cathodic Antigen (CCA) and single-day double KK measurements over three consecutive days for the diagnosis of S. mansoni infection in two different age groups from Uganda pre- and post- praziquantel (PZQ) treatment. We demonstrate that CCA is an appropriate tool for mapping surveys of S. mansoni infection. Our findings should allow for evaluation of the risk of potential misinterpretation with regards to diagnosis of S. mansoni infection through CCA or KK in this endemic setting pre- and post- PZQ treatment as the numbers and infection intensities are brought down, bridging existing important gaps in schistosomiasis diagnostics research. More generally, the proposed statistical analysis can reveal important biological insights from other diseases without gold standard diagnostic tools whenever longitudinal data are available.