Continuous monitoring tools for pediatric surgical outcomes: an example using biliary atresia

Am I out of control? The application of statistical process control charts to children's surgery

AIMS

To illustrate the construction of statistical control charts and show their potential application to analysis of outcomes in children's surgery.

PATIENTS AND METHODS

Two datasets recording outcomes following esophageal atresia repair and intestinal resection for Crohn's disease maintained by the author were used to construct four types of charts. The effects of altering the target signal, the alarm signal and the limits are illustrated. The dilemmas in choice of target rate are described. Simulated data illustrate the advantages over hypothesis testing.

RESULTS

The charts show the author's institutional leak rate for esophageal atresia repair may be within acceptable limits, but that this is dependent on the target set. The desirable target is contentious. The leak rate for anastomoses following intestinal resection for Crohn's disease leak is also within acceptable limits when compared to published experience, but may be deteriorating. The charts are able to detect deteriorating levels of performance well before hypothesis testing would suggest a systematic problem with outcomes.

CONCLUSIONS

Statistical process control charts can provide surgeons with early warning of systematic poor performance. They are robust to volume-outcome influences, since the outcome is tested sequentially after each procedure or patient. They have application in a specialty with low frequencies of operations such as children's surgery.

TYPE OF STUDY

Diagnostic test.

LEVEL OF EVIDENCE

Level II.

The world is waiting, use sequential analysis and get us the evidence-based treatment we need for COVID-19

In spite of the relatively high morbidity and mortality, there is no approved medication yet for COVID-19. There are more than 200 ongoing trials on different drugs or vaccines, but new medications may take until 2021 to develop. Defining the optimal number of patients to be included in a study is a considerable challenge in these interventional researches. Ethical considerations prompt researchers to minimize the number of patients included in a trial. This gains particular importance when the disease is rare or lethal which is particularly so in the case of COVID-19. It is of paramount importance to explore some of the available tools that could help accelerate the adoption of any or some of the many proposed modalities for the treatment of diseases. These tools should be effective, yet efficient, for rapid testing of such treatments. Sequential analysis has not been frequently used in many clinical trials where it should have been used. None of the authors in published literature, as far as we know, used sequential analysis techniques to test potential drugs for COVID-19. In addition to its usefulness when the results of new forms of treatment are quickly needed, other important benefit of sequential analysis includes the ability to reach a similar conclusion about the utility of a new drug without unduly exposing more patients to the side effect of the old drug, in particularly, for the treatment of a rare disease.

Sequential analysis in neonatal research-systematic review

As more new drugs are discovered, traditional designs come at their limits. Ten years after the adoption of the European Paediatric Regulation, we performed a systematic review on the US National Library of Medicine and Excerpta Medica database of sequential trials involving newborns. Out of 326 identified scientific reports, 21 trials were included. They enrolled 2832 patients, of whom 2099 were analyzed: the median number of neonates included per trial was 48 (IQR 22-87), median gestational age was 28.7 (IQR 27.9-30.9) weeks. Eighteen trials used sequential techniques to determine sample size, while 3 used continual reassessment methods for dose-finding. In 16 studies reporting sufficient data, the sequential design allowed to non-significantly reduce the number of enrolled neonates by a median of 24 (31%) patients (IQR - 4.75 to 136.5, p = 0.0674) with respect to a traditional trial. When the number of neonates finally included in the analysis was considered, the difference became significant: 35 (57%) patients (IQR 10 to 136.5, p = 0.0033).

CONCLUSION

Sequential trial designs have not been frequently used in Neonatology. They might potentially be able to reduce the number of patients in drug trials, although this is not always the case. What is known: • In evaluating rare diseases in fragile populations, traditional designs come at their limits. About 20% of pediatric trials are discontinued, mainly because of recruitment problems. What is new: • Sequential trials involving newborns were infrequently used and only a few (n = 21) are available for analysis. • The sequential design allowed to non-significantly reduce the number of enrolled neonates by a median of 24 (31%) patients (IQR - 4.75 to 136.5, p = 0.0674).

Outcome for biliary atresia patients treated at a low-volume centre

OBJECTIVES

The importance of case load for treatment of extrahepatic biliary atresia (BA) is debated. The aim of this study was to register results of BA treatment in a small volume centre.

METHODS

Retrospective chart review study of patients with BA treated from 2000 to 2017. The institutional review board approved the study.

RESULTS

Forty-five babies were identified of which 42 (93%) are alive. Forty-one patients had a Kasai portoenterostomy (PE), two had a hepaticojejunostomy and two had a primary liver transplantation. The age at PE/hepaticojejunostomy was median 63 (4-145 days). Seven surgeons performed the operations, and the median duration of the diagnostic work-up was 8 (3-24) days. Clearance of jaundice was achieved in 23/43 (53%) babies, and 3- and 5-year native liver survival was 47% and 40%, respectively. Clearance of jaundice post-PE/hepaticojejunostomy was a strong predictor of native liver survival (adjusted OR: 0.027; 95%; p = .009). Plasma level of conjugated bilirubin at time of referral was also a significant predictor of native liver survival (adjusted OR: 1.053; p = .017).

CONCLUSION

A small volume centre may achieve satisfactory results for BA patients. The study has, however, identified factors that may further improve results; earlier referral, optimizing diagnostic work-up and establishing one dedicated surgical team.

A Bayesian CUSUM plot: Diagnosing quality of treatment

OBJECTIVES

To present a CUSUM plot based on Bayesian diagnostic reasoning displaying evidence in favour of "healthy" rather than "sick" quality of treatment (QOT), and to demonstrate a technique using Kaplan-Meier survival curves permitting application to case series with ongoing follow-up.

METHODS

For a case series with known final outcomes: Consider each case a diagnostic test of good versus poor QOT (expected vs. increased failure rates), determine the likelihood ratio (LR) of the observed outcome, convert LR to weight taking log to base 2, and add up weights sequentially in a plot showing how many times odds in favour of good QOT have been doubled. For a series with observed survival times and an expected survival curve: Divide the curve into time intervals, determine "healthy" and specify "sick" risks of failure in each interval, construct a "sick" survival curve, determine the LR of survival or failure at the given observation times, convert to weights, and add up.

RESULTS

The Bayesian plot was applied retrospectively to 39 children with acute lymphoblastic leukaemia with completed follow-up, using Nordic collaborative results as reference, showing equal odds between good and poor QOT. In the ongoing treatment trial, with 22 of 37 children still at risk for event, QOT has been monitored with average survival curves as reference, odds so far favoring good QOT 2:1.

CONCLUSION

QOT in small patient series can be assessed with a Bayesian CUSUM plot, retrospectively when all treatment outcomes are known, but also in ongoing series with unfinished follow-up.

Learning curve evaluation using cumulative summation analysis-a clinical example of pediatric robot-assisted laparoscopic pyeloplasty

BACKGROUND

The cumulative summation (CUSUM) method for learning curve analysis remains under-utilized in the surgical literature in general, and is described in only a small number of publications within the field of pediatric surgery. This study introduces the CUSUM analysis technique and applies it to evaluate the learning curve for pediatric robot-assisted laparoscopic pyeloplasty (RP).

METHODS

Clinical data were prospectively recorded for consecutive pediatric RP cases performed by a single-surgeon. CUSUM charts and tests were generated for set-up time, docking time, console time, operating time, total operating room time, and postoperative complications. Conversions and avoidable operating room delay were separately evaluated with respect to case experience. Comparisons between case experience and time-based outcomes were assessed using the Student's t-test and ANOVA for bi-phasic and multi-phasic learning curves respectively. Comparison between case experience and complication frequency was assessed using the Kruskal-Wallis test.

RESULTS

A total of 90 RP cases were evaluated. The learning curve transitioned beyond the learning phase at cases 10, 15, 42, 57, and 58 for set-up time, docking time, console time, operating time, and total operating room time respectively. All comparisons of mean operating times between the learning phase and subsequent phases were statistically significant (P=<0.001-0.01). No significant difference was observed between case experience and frequency of post-operative complications (P=0.125), although the CUSUM chart demonstrated a directional change in slope for the last 12 cases in which there were high proportions of re-do cases and patients <6 months of age.

CONCLUSIONS

The CUSUM method has a valuable role for learning curve evaluation and outcome quality monitoring. In applying this statistical technique to the largest reported single surgeon series of pediatric RP, we demonstrate numerous distinctly shaped learning curves and well-defined learning phase transition points.

Are Standard Diagnostic Test Characteristics Sufficient for the Assessment of Continual Patient Monitoring?

Background. For diagnostic processes involving continual measurements from a single patient, conventional test characteristics, such as sensitivity and specificity, do not consider decision consistency, which might be a distinct, clinically relevant test characteristic. Objective. The authors investigated the performance of a decision-support classifier for the diagnosis of traumatic injury with blood loss, implemented with three different data-processing methods. For each method, they computed standard diagnostic test characteristics and novel metrics related to decision consistency and latency. Setting. Prehospital air ambulance transport. Patients. A total of 557 trauma patients. Design. Continually monitored vital-sign data from 279 patients (50%) were randomly selected for classifier development, and the remaining were used for testing. Three data-processing methods were evaluated over 16 min of patient monitoring: a 2-min moving window, time averaging, and postprocessing with the sequential probability ratio test (SPRT). Measurements. Sensitivity and specificity were computed. Consistency was quantified through cumulative counts of decision changes over time and the fraction of patients affected by false alarms. Latency was evaluated by the fraction of patients without a decision. Results. All 3 methods showed very similar final sensitivities and specificities. Yet, there were significant differences in terms of the fraction of patients affected by false alarms, decision changes through time, and latency. For instance, use of the SPRT led to a 75% reduction in the number of decision changes and a 36% reduction in the number of patients affected by false alarms, at the expense of 3% unresolved final decisions. Conclusion. The proposed metrics of decision consistency and decision latency provided additional information beyond what could be obtained from test sensitivity and specificity and are likely to be clinically relevant in some applications involving temporal decision making.

Advances in the understanding and treatment of biliary atresia

PURPOSE OF REVIEW

The purpose of this study is to review both the basic science and clinical advancements in the last 12 months that have furthered our understanding of biliary atresia.

RECENT FINDINGS

Early diagnosis and further understanding of the disease process may be the next major step in advancement. Stool color cards have been shown to be an accurate screening tool. Basic science developments have focused on defects in morphogenesis, immunologic dysregulation, and viral infection as the major theories of causes. There have been initial reports of minimally invasive approaches to hepatic portoenterostomy but there has been little comparative study. Postoperative corticosteroid therapy remains an area of debate without definitive data. Early postoperative testing of serum bilirubin levels and hepatobiliary scintigraphy are showing strong correlation with long-term outcomes. The comparison of regions with decentralization policies compared with those with central referral policies is providing a good forum to monitor real-time outcome data.

SUMMARY

Biliary atresia continues to represent a major challenge with many unanswered questions. The establishment of multicentered collaboration in both basic science and clinical research interests has been an important step in improving outcomes for this disease.

Biliary atresia: service delivery and outcomes

Biliary atresia is a complex disorder dependent on multidisciplinary management. A series of comprehensive national audits in the United Kingdom and France exposed a clear relationship between center volume and clinical outcomes. Different models were adopted in each country in an attempt to improve results. In the United Kingdom, the management of biliary atresia was centralized to three specialist units in 1999, whereas in France, a strategy of decentralized management with closer inter-unit cooperation was adopted in 1997. Both policy changes led to improved outcomes for infants with biliary atresia, but only centralization improved the overall results of Kasai portoenterostomy. Other countries have adopted alternative systems of audit based on voluntary registries, but the impact of these on clinical outcomes at a national level remains unknown. The utility of monitoring tools in assessing performance in biliary atresia, the importance of risk stratification, and the need for standardized definitions of outcome are highlighted.