Prognostic biomarkers to identify patients destined to develop severe Crohn's disease who may benefit from early biological therapy: protocol for a systematic review, meta-analysis and external validation

TOMAS-R: A template to identify and plan analysis for clinically important variation and multiplicity in diagnostic test accuracy systematic reviews

The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (DTA) provides guidance on important aspects of conducting a test accuracy systematic review. In this paper we present TOMAS-R (Template of Multiplicity and Analysis in Systematic Reviews), a structured template to use in conjunction with current Cochrane DTA guidance, to help identify complexities in the review question and to assist planning of data extraction and analysis when clinically important variation and multiplicity is present. Examples of clinically important variation and multiplicity could include differences in participants, index tests and test methods, target conditions and reference standards used to define them, study design and methodological quality. Our TOMAS-R template goes beyond the broad topic headings in current guidance that are sources of potential variation and multiplicity, by providing prompts for common sources of heterogeneity encountered from our experience of authoring over 100 reviews. We provide examples from two reviews to assist users. The TOMAS-R template adds value by supplementing available guidance for DTA reviews by providing a tool to facilitate discussions between methodologists, clinicians, statisticians and patient/public team members to identify the full breadth of review question complexities early in the process. The use of a structured set of prompting questions at the important stage of writing the protocol ensures clinical relevance as a main focus of the review, while allowing identification of key clinical components for data extraction and later analysis thereby facilitating a more efficient review process.

QUAPAS: An Adaptation of the QUADAS-2 Tool to Assess Prognostic Accuracy Studies

Whereas diagnostic tests help detect the cause of signs and symptoms, prognostic tests assist in evaluating the probable course of the disease and future outcome. Studies to evaluate prognostic tests are longitudinal, which introduces sources of bias different from those for diagnostic accuracy studies. At present, systematic reviews of prognostic tests often use the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to assess risk of bias and applicability of included studies because no equivalent instrument exists for prognostic accuracy studies. QUAPAS (Quality Assessment of Prognostic Accuracy Studies) is an adaptation of QUADAS-2 for prognostic accuracy studies. Questions likely to identify bias were evaluated in parallel and collated from QUIPS (Quality in Prognosis Studies) and PROBAST (Prediction Model Risk of Bias Assessment Tool) and paired to the corresponding question (or domain) in QUADAS-2. A steering group conducted and reviewed 3 rounds of modifications before arriving at the final set of domains and signaling questions. QUAPAS follows the same steps as QUADAS-2: Specify the review question, tailor the tool, draw a flow diagram, judge risk of bias, and identify applicability concerns. Risk of bias is judged across the following 5 domains: participants, index test, outcome, flow and timing, and analysis. Signaling questions assist the final judgment for each domain. Applicability concerns are assessed for the first 4 domains. The authors used QUAPAS in parallel with QUADAS-2 and QUIPS in a systematic review of prognostic accuracy studies. QUAPAS improved the assessment of the flow and timing domain and flagged a study at risk of bias in the new analysis domain. Judgment of risk of bias in the analysis domain was challenging because of sparse reporting of statistical methods.

Prognostic biomarkers to identify patients likely to develop severe Crohn's disease: a systematic review

BACKGROUND

Identification of biomarkers that predict severe Crohn's disease is an urgent unmet research need, but existing research is piecemeal and haphazard.

OBJECTIVE

To identify biomarkers that are potentially able to predict the development of subsequent severe Crohn's disease.

DESIGN

This was a prognostic systematic review with meta-analysis reserved for those potential predictors with sufficient existing research (defined as five or more primary studies).

DATA SOURCES

PubMed and EMBASE searched from inception to 1 January 2016, updated to 1 January 2018.

REVIEW METHODS

Eligible studies were studies that compared biomarkers in patients who did or did not subsequently develop severe Crohn's disease. We excluded biomarkers that had insufficient research evidence. A clinician and two statisticians independently extracted data relating to predictors, severe disease definitions, event numbers and outcomes, including odds/hazard ratios. We assessed risk of bias. We searched for associations with subsequent severe disease rather than precise estimates of strength. A random-effects meta-analysis was performed separately for odds ratios.

RESULTS

In total, 29,950 abstracts yielded just 71 individual studies, reporting 56 non-overlapping cohorts. Five clinical biomarkers (Montreal behaviour, age, disease duration, disease location and smoking), two serological biomarkers (anti-Saccharomyces cerevisiae antibodies and anti-flagellin antibodies) and one genetic biomarker (nucleotide-binding oligomerisation domain-containing protein 2) displayed statistically significant prognostic potential. Overall, the strongest association with subsequent severe disease was identified for Montreal B2 and B3 categories (odds ratio 4.09 and 6.25, respectively).

LIMITATIONS

Definitions of severe disease varied widely, and some studies confounded diagnosis and prognosis. Risk of bias was rated as 'high' in 92% of studies overall. Some biomarkers that are used regularly in daily practice, for example C-reactive protein, were studied too infrequently for meta-analysis.

CONCLUSIONS

Research for individual biomarkers to predict severe Crohn's disease is scant, heterogeneous and at a high risk of bias. Despite a large amount of potential research, we encountered relatively few biomarkers with data sufficient for meta-analysis, identifying only eight biomarkers with potential predictive capability.

FUTURE WORK

We will use existing data sets to develop and then validate a predictive model based on the potential predictors identified by this systematic review. Contingent on the outcome of that research, a prospective external validation may prove clinically desirable.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016029363.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 45. See the NIHR Journals Library website for further project information.

Few promising multivariable prognostic models exist for recovery of people with non-specific neck pain in musculoskeletal primary care: a systematic review

QUESTION

Which multivariable prognostic model(s) for recovery in people with neck pain can be used in primary care?

DESIGN

Systematic review of studies evaluating multivariable prognostic models.

PARTICIPANTS

People with non-specific neck pain presenting at primary care.

DETERMINANTS

Baseline characteristics of the participants.

OUTCOME MEASURES

Recovery measured as pain reduction, reduced disability, or perceived recovery at short-term and long-term follow-up.

RESULTS

Fifty-three publications were included, of which 46 were derivation studies, four were validation studies, and three concerned combined studies. The derivation studies presented 99 multivariate models, all of which were at high risk of bias. Three externally validated models generated usable models in low risk of bias studies. One predicted recovery in non-specific neck pain, while two concerned participants with whiplash-associated disorders (WAD). Discriminative ability of the non-specific neck pain model was area under the curve (AUC) 0.65 (95% CI 0.59 to 0.71). For the first WAD model, discriminative ability was AUC 0.85 (95% CI 0.79 to 0.91). For the second WAD model, specificity was 99% (95% CI 93 to 100) and sensitivity was 44% (95% CI 23 to 65) for prediction of non-recovery, and 86% (95% CI 73 to 94) and 55% (95% CI 41 to 69) for prediction of recovery, respectively. Initial Neck Disability Index scores and age were identified as consistent prognostic factors in these three models.

CONCLUSION

Three externally validated models were found to be usable and to have low risk of bias, of which two showed acceptable discriminative properties for predicting recovery in people with neck pain. These three models need further validation and evaluation of their clinical impact before their broad clinical use can be advocated.

REGISTRATION

PROSPERO CRD42016042204. [Wingbermühle RW, van Trijffel E, Nelissen PM, Koes B, Verhagen AP (2018) Few promising multivariable prognostic models exist for recovery of people with non-specific neck pain in musculoskeletal primary care: a systematic review. Journal of Physiotherapy 64: 16-23].