A Methodology for Determining Patients’ Eligibility for Clinical Trials

A knowledge base of clinical trial eligibility criteria

OBJECTIVE

We present the Clinical Trial Knowledge Base, a regularly updated knowledge base of discrete clinical trial eligibility criteria equipped with a web-based user interface for querying and aggregate analysis of common eligibility criteria.

MATERIALS AND METHODS

We used a natural language processing (NLP) tool named Criteria2Query (Yuan et al., 2019) to transform free text clinical trial eligibility criteria from ClinicalTrials.gov into discrete criteria concepts and attributes encoded using the widely adopted Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) and stored in a relational SQL database. A web application accessible via RESTful APIs was implemented to enable queries and visual aggregate analyses. We demonstrate CTKB's potential role in EHR phenotype knowledge engineering using ten validated phenotyping algorithms.

RESULTS

At the time of writing, CTKB contained 87,504 distinctive OMOP CDM standard concepts, including Condition (47.82%), Drug (23.01%), Procedure (13.73%), Measurement (24.70%) and Observation (5.28%), with 34.78% for inclusion criteria and 65.22% for exclusion criteria, extracted from 352,110 clinical trials. The average hit rate of criteria concepts in eMERGE phenotype algorithms is 77.56%.

CONCLUSION

CTKB is a novel comprehensive knowledge base of discrete eligibility criteria concepts with the potential to enable knowledge engineering for clinical trial cohort definition, clinical trial population representativeness assessment, electronical phenotyping, and data gap analyses for using electronic health records to support clinical trial recruitment.

Design aspects of a distributed clinical trials information system

Backgound Clinical trials informatics has evolved through the development of multiple applications addressing distinct parts of the clinical trials cycle. This setting creates difficulties in the transport and sharing of data among applications that serve a common functionality.

Purpose We present an alternative approach for the design of clinical trials information systems consisting of loosely coupled components standing on a comprehensive model of the global clinical trial process.

Methods We describe how such a structure is able to support the development and implementation of complex knowledge-driven modules, such as automated database query systems, reporting systems and intelligent data-analysis tools, and how currently available internet technologies may be used to support the independent development of applications and collaboration between researchers.

Results These principles were applied to the development of a fully functional system that has been in production for eight years in a diversity of pharmaceutical and academic drug trials. Marked time savings in database set-up and statistical reporting have been documented, as well as good reliability in the selection of appropriate statistical methods to clinical trial data analysis.

Conclusions In order to meet the expected functionality and to avoid the proliferation of databases and software applications, clinical trials information systems need to be based on a generic model of clinical trials and on a distributed architecture.

Computerized patient identification for the EMBRACA clinical trial using real-time data from the PRAEGNANT network for metastatic breast cancer patients

As breast cancer is a diverse disease, clinical trials are becoming increasingly diversified and are consequently being conducted in very small subgroups of patients, making study recruitment increasingly difficult. The aim of this study was to assess the use of data from a remote data entry system that serves a large national registry for metastatic breast cancer. The PRAEGNANT network is a real-time registry with an integrated biomaterials bank that was designed as a scientific study and as a means of identifying patients who are eligible for clinical trials, based on clinical and molecular information. Here, we report on the automated use of the clinical data documented to identify patients for a clinical trial (EMBRACA) for patients with metastatic breast cancer. The patients' charts were assessed by two independent physicians involved in the clinical trial and also by a computer program that tested patients for eligibility using a structured query language script. In all, 326 patients from two study sites in the PRAEGNANT network were included in the analysis. Using expert assessment, 120 of the 326 patients (37 %) appeared to be eligible for inclusion in the EMBRACA study; with the computer algorithm assessment, a total of 129 appeared to be eligible. The sensitivity of the computer algorithm was 0.87 and its specificity was 0.88. Using computer-based identification of patients for clinical trials appears feasible. With the instrument's high specificity, its application in a large cohort of patients appears to be feasible, and the workload for reassessing the patients is limited.

Employing computers for the recruitment into clinical trials: a comprehensive systematic review

BACKGROUND

Medical progress depends on the evaluation of new diagnostic and therapeutic interventions within clinical trials. Clinical trial recruitment support systems (CTRSS) aim to improve the recruitment process in terms of effectiveness and efficiency.

OBJECTIVE

The goals were to (1) create an overview of all CTRSS reported until the end of 2013, (2) find and describe similarities in design, (3) theorize on the reasons for different approaches, and (4) examine whether projects were able to illustrate the impact of CTRSS.

METHODS

We searched PubMed titles, abstracts, and keywords for terms related to CTRSS research. Query results were classified according to clinical context, workflow integration, knowledge and data sources, reasoning algorithm, and outcome.

RESULTS

A total of 101 papers on 79 different systems were found. Most lacked details in one or more categories. There were 3 different CTRSS that dominated: (1) systems for the retrospective identification of trial participants based on existing clinical data, typically through Structured Query Language (SQL) queries on relational databases, (2) systems that monitored the appearance of a key event of an existing health information technology component in which the occurrence of the event caused a comprehensive eligibility test for a patient or was directly communicated to the researcher, and (3) independent systems that required a user to enter patient data into an interface to trigger an eligibility assessment. Although the treating physician was required to act for the patient in older systems, it is now becoming increasingly popular to offer this possibility directly to the patient.

CONCLUSIONS

Many CTRSS are designed to fit the existing infrastructure of a clinical care provider or the particularities of a trial. We conclude that the success of a CTRSS depends more on its successful workflow integration than on sophisticated reasoning and data processing algorithms. Furthermore, some of the most recent literature suggest that an increase in recruited patients and improvements in recruitment efficiency can be expected, although the former will depend on the error rate of the recruitment process being replaced. Finally, to increase the quality of future CTRSS reports, we propose a checklist of items that should be included.

A review of approaches to identifying patient phenotype cohorts using electronic health records

Objective

To summarize literature describing approaches aimed at automatically identifying patients with a common phenotype.

Materials and methods

We performed a review of studies describing systems or reporting techniques developed for identifying cohorts of patients with specific phenotypes. Every full text article published in (1) Journal of American Medical Informatics Association, (2) Journal of Biomedical Informatics, (3) Proceedings of the Annual American Medical Informatics Association Symposium, and (4) Proceedings of Clinical Research Informatics Conference within the past 3 years was assessed for inclusion in the review. Only articles using automated techniques were included.

Results

Ninety-seven articles met our inclusion criteria. Forty-six used natural language processing (NLP)-based techniques, 24 described rule-based systems, 41 used statistical analyses, data mining, or machine learning techniques, while 22 described hybrid systems. Nine articles described the architecture of large-scale systems developed for determining cohort eligibility of patients.

Discussion

We observe that there is a rise in the number of studies associated with cohort identification using electronic medical records. Statistical analyses or machine learning, followed by NLP techniques, are gaining popularity over the years in comparison with rule-based systems.

Conclusions

There are a variety of approaches for classifying patients into a particular phenotype. Different techniques and data sources are used, and good performance is reported on datasets at respective institutions. However, no system makes comprehensive use of electronic medical records addressing all of their known weaknesses.

Identifying clinical/translational research cohorts: ascertainment via querying an integrated multi-source database

BACKGROUND

Ascertainment of potential subjects has been a longstanding problem in clinical research. Various methods have been proposed, including using data in electronic health records. However, these methods typically suffer from scaling effects-some methods work well for large cohorts; others work for small cohorts only.

OBJECTIVE

We propose a method that provides a simple identification of pre-research cohorts and relies on data available in most states in the USA: merged public health data sources.

MATERIALS AND METHODS

The Utah Population Database Limited query tool allows users to build complex queries that may span several types of health records, such as cancer registries, inpatient hospital discharges, and death certificates; in addition, these can be combined with family history information. The architectural approach incorporates several coding systems for medical information. It provides a front-end graphical user interface and enables researchers to build and run queries and view aggregate results. Multiple strategies have been incorporated to maintain confidentiality.

RESULTS

This tool was rapidly adopted; since its release, 241 users representing a wide range of disciplines from 17 institutions have signed the user agreement and used the query tool. Three examples are discussed: pregnancy complications co-occurring with cardiovascular disease; spondyloarthritis; and breast cancer.

DISCUSSION AND CONCLUSIONS

This query tool was designed to provide results as pre-research so that institutional review board approval would not be required. This architecture uses well-described technologies that should be within the reach of most institutions.

Dynamic categorization of clinical research eligibility criteria by hierarchical clustering

OBJECTIVE

To semi-automatically induce semantic categories of eligibility criteria from text and to automatically classify eligibility criteria based on their semantic similarity.

DESIGN

The UMLS semantic types and a set of previously developed semantic preference rules were utilized to create an unambiguous semantic feature representation to induce eligibility criteria categories through hierarchical clustering and to train supervised classifiers.

MEASUREMENTS

We induced 27 categories and measured the prevalence of the categories in 27,278 eligibility criteria from 1578 clinical trials and compared the classification performance (i.e., precision, recall, and F1-score) between the UMLS-based feature representation and the "bag of words" feature representation among five common classifiers in Weka, including J48, Bayesian Network, Naïve Bayesian, Nearest Neighbor, and instance-based learning classifier.

RESULTS

The UMLS semantic feature representation outperforms the "bag of words" feature representation in 89% of the criteria categories. Using the semantically induced categories, machine-learning classifiers required only 2000 instances to stabilize classification performance. The J48 classifier yielded the best F1-score and the Bayesian Network classifier achieved the best learning efficiency.

CONCLUSION

The UMLS is an effective knowledge source and can enable an efficient feature representation for semi-automated semantic category induction and automatic categorization for clinical research eligibility criteria and possibly other clinical text.

Comparing semi-automatic systems for recruitment of patients to clinical trials

OBJECTIVES

(i) To review contributions and limitations of decision support systems for automatic recruitment of patients to clinical trials (Clinical Trial Recruitment Support Systems, CTRSS). (ii) To characterize the important features of this domain, the main classes of approach that have been used, and their advantages and disadvantages. (iii) To assess the effectiveness and potential of such systems in improving trial recruitment rates.

DATA SOURCES

A systematic MESH keyword-based search of Pubmed, Embase, and Scholar Google for relevant CTRSS publications from January 1st 1998 to August 31st 2009 yielded 73 references, from which 33 relevant papers describing 28 distinct studies were chosen for review, based on their report of a novel decision support system for trial recruitment which reused already available patient data.

METHOD

The reviewed papers were classified using a modified version of an existing taxonomy for clinical decision support systems, using 10 axes relevant to the trial recruitment domain.

RESULTS

It proved possible and useful to characterize CTRSS on a relatively small number of dimensions and a number of clear trends emerge from the study. Only nine papers reported a useful evaluation of the effectiveness of the system in terms of trial pre-inclusion or enrolment rate. While all the systems reviewed re-use structured and coded patient data none attempts the more difficult task of using unstructured patient notes to pre-screen for trial inclusion. Few studies address acceptance of systems by clinicians, or integration into clinical workflow, and there is little evidence of use of interoperability standards.

CONCLUSIONS

System design, scope, and assessment methodology vary significantly between papers, making it difficult to establish the impact of different approaches on recruitment rate. It is clear, however, that the pre-screening phase of trial recruitment is the most effective part of the process to address with CTRSS, that clinical workflow integration and clinician acceptance are critical for this class of decision support, and that the current trends in this field are towards generalization and scalability.

The Brustkrebs-Studien.de website for breast cancer patients: User acceptance of a German internet portal offering information on the disease and treatment options, and a clinical trials matching service

Background

The internet portal http://www.brustkrebs-studien.de (BKS) was launched in 2000 by the German Society of Senology (DGS) and the Baden-Württemberg Institute for Women's Health (IFG) to provide expert-written information on breast cancer online and to encourage and facilitate the participation of breast cancer patients in clinical trials. We describe the development of BKS and its applications, and report on website statistics and user acceptance.

Methods

Existing registries, including ClinicalTrials.gov, were analysed before we designed BKS, which combines a trial registry, a knowledge portal, and an online second opinion service. An advisory board guided the process. Log files and patient enquiries for trial participation and second opinions were analysed. A two-week user satisfaction survey was conducted online.

Results

During 10/2005-06/2010, the portal attracted 702,655 visitors, generating 15,507,454 page views. By 06/2010, the website's active scientific community consisted of 189 investigators and physicians, and the registry covered 163 clinical trial protocols. In 2009, 143 patients requested trial enrolment and 119 sought second opinions or individual treatment advice from the expert panel. During the two-week survey in 2008, 5,702 BKS visitors submitted 507 evaluable questionnaires. Portal acceptance was high. Respondents trusted information correctness (80%), welcomed self-matching to clinical trials (79%) and planned to use the portal in the future (76%) and recommend it to others (81%).

Conclusions

BKS is an established and trusted breast cancer information platform offering up-to-date resources and protocols to the growing physician and patient community to encourage participation in clinical trials. Further studies are needed to assess potential increases in trial enrolment by eligibility matching services.

Formal representation of eligibility criteria: a literature review

Standards-based, computable knowledge representations for eligibility criteria are increasingly needed to provide computer-based decision support for automated research participant screening, clinical evidence application, and clinical research knowledge management. We surveyed the literature and identified five aspects of eligibility criteria knowledge representation that contribute to the various research and clinical applications: the intended use of computable eligibility criteria, the classification of eligibility criteria, the expression language for representing eligibility rules, the encoding of eligibility concepts, and the modeling of patient data. We consider three of these aspects (expression language, codification of eligibility concepts, and patient data modeling) to be essential constructs of a formal knowledge representation for eligibility criteria. The requirements for each of the three knowledge constructs vary for different use cases, which therefore should inform the development and choice of the constructs toward cost-effective knowledge representation efforts. We discuss the implications of our findings for standardization efforts toward knowledge representation for sharable and computable eligibility criteria.

An Internet-based cancer clinical trials matching resource

Background

Many patients are now accessing the Internet to obtain cancer clinical trials information. However, services offering clinical trials recruitment information have not been well defined.

Objectives

This study describes one of the first Web-based cancer clinical trials matching resources and the demographics of users who were successfully matched.

Methods

OncoLink is the Internet-based educational resource managed by the University of Pennsylvania Cancer Center (UPCC) and serves between 1 and 2 million pages per month to over 385000 unique IP addresses. OncoLink launched one of the first clinical trials matching resources on the Internet that allowed patients to enter demographic data through a secure connection and be matched to clinical trials. For patients with matches to potential trials, appointments were facilitated with the principal investigators.

Results

While we did not keep track of patients who could not be matched, 627 patients who submitted online applications between January 2002 and April 2003 were successfully matched for potential enrollment in clinical trials. The mean age of the patient population was 56 years (range 18–88 years). Males represented 60% of the patient population, and over 90% of users were Caucasian. Most of the applications were from patients with colorectal cancer (13%), lung cancer (14%), melanoma (10%), and non-Hodgkin's lymphoma (9%).

Conclusions

This report shows that a significant number of patients are willing to use the Internet for enrolling in clinical trials. Care must be taken to reach patients from a variety of socioeconomic and racial backgrounds. This Internet resource helps to facilitate a consultation with a cancer patient who is prescreened and motivated to enroll in clinical trials.

An ontology of randomized controlled trials for evidence-based practice: content specification and evaluation using the competency decomposition method

Randomized controlled trials (RCTs) are one of the least biased sources of clinical research evidence, and are therefore a critical resource for the practice of evidence-based medicine. With over 10,000 new RCTs indexed in Medline each year, knowledge systems are needed to help clinicians translate evidence into practice. Common ontologies for RCTs and other domains would facilitate the development of these knowledge systems. However, no standard method exists for developing domain ontologies. In this paper, we describe a new systematic approach to specifying and evaluating the conceptual content of ontologies. In this method, called competency decomposition, the target task for an ontology is hierarchically decomposed into subtasks and methods, and the ontology content is specified by identifying the domain information required to complete each of the subtasks. We illustrate the use of this competency decomposition approach for the content specification and evaluation of an RCT ontology for evidence-based practice.

Selection of patients for clinical trials: an interactive web-based system

The purpose of a clinical trial is to evaluate a new treatment procedure. When medical researchers conduct a trial, they recruit participants with appropriate health problems and medical histories. To select participants, they analyze medical records of the available patients, which has traditionally been a manual procedure. We describe an expert system that helps to select patients for clinical trials. If the available data are insufficient for choosing patients, the system suggests additional medical tests and finds an ordering of the tests that reduces their total cost. Experiments show that the system can increase the number of selected patients. We also present an interface that enables a medical researcher to add clinical trials and selection criteria without the help of a programmer. The addition of a new trial takes 10-20 min, and novice users learn the functionality of the interface in about an hour.

Web-based decision support for clinical trial eligibility determination in an international clinical trials network

Matching individuals to multisite cooperative clinical trials can be a complex and nonintuitive decision process that expends considerable time and may be prone to errors. We developed and tested a web-based decision support tool to aid investigators in matching patients to open clinical trials for children with rhabdomyosarcoma in the context of an international cooperative cancer clinical trials network. A decision tree for trial eligibility based on eight clinical variables representing major disease characteristics was translated into a web-based format. In a blinded fashion, we assessed the accuracy of the tool in assigning 100 randomly selected cases to the proper clinical trial. The web-based tool assigned patients to the proper clinical trial in all 100 randomly selected cases. The time needed to enter data and receive results using this tool is about 1 minute per patient entered. It is feasible to develop a web-based tool to help investigators in matching patients to clinical trials. When such decisions are complex and nonintuitive, such tools have the potential to improve the accuracy of clinical trial assignment and save time.

Using OncoDoc as a computer-based eligibility screening system to improve accrual onto breast cancer clinical trials

While clinical trials offer cancer patients the optimum treatment, historical accrual of such patients has not been very successful. OncoDoc is a decision support system designed to provide best therapeutic recommendations for breast cancer patients. Developed as a browsing tool of a knowledge base structured as a decision tree, OncoDoc allows physicians to control the contextual instantiation of patient characteristics to build the best formal equivalent of an actual patient. Used as a computer-based eligibility screening system, depending on whether instantiated patient parameters are matched against guideline knowledge or available clinical trial protocols, it provides either evidence-based therapeutic options or relevant patient-specific clinical trials. Implemented at the Gustave Roussy Institute and routinely used at the point of care during a 4-month period, it significantly improved physician compliance with guideline recommendations and enhanced physician awareness of open trials while increasing patient enrollment to clinical trials by 50%. But, when analyzing reasons of non-accrual of potentially eligible patients, it appeared that physicians' psychological reluctance to refer patients to clinical trials, measured during the experiment at 25%, may not be resolved by the simple dissemination of clinical trial information at the point of care.

Knowledge-based verification of clinical guidelines by detection of anomalies

As shown in numerous studies, a significant part of published clinical guidelines is tainted with different types of semantical errors that interfere with their practical application. The adaptation of generic guidelines, necessitated by circumstances such as resource limitations within the applying organization or unexpected events arising in the course of patient care, further promotes the introduction of defects. Still, most current approaches for the automation of clinical guidelines are lacking mechanisms, which check the overall correctness of their output. In the domain of software engineering in general and in the domain of knowledge-based systems (KBS) in particular, a common strategy to examine a system for potential defects consists in its verification. The focus of this work is to present an approach, which helps to ensure the semantical correctness of clinical guidelines in a three-step process. We use a particular guideline specification language called Asbru to demonstrate our verification mechanism. A scenario-based evaluation of our method is provided based on a guideline for the artificial ventilation of newborn infants. The described approach is kept sufficiently general in order to allow its application to several other guideline representation formats.

Design and implementation of a national clinical trials registry

The authors have developed a Web-based system that provides summary information about clinical trials being conducted throughout the United States. The first version of the system, publicly available in February 2000, contains more than 4,000 records representing primarily trials sponsored by the National Institutes of Health. The impetus for this system has come from the Food and Drug Administration (FDA) Modernization Act of 1997, which mandated a registry of both federally and privately funded clinical trials "of experimental treatments for serious or life-threatening diseases or conditions." The system design and implementation have been guided by several principles. First, all stages of system development were guided by the needs of the primary intended audience, patients and other members of the public. Second, broad agreement on a common set of data elements was obtained. Third, the system was designed in a modular and extensible way, and search methods that take extensive advantage of the National Library of Medicine's Unified Medical Language System (UMLS) were developed. Finally, since this will be a long-term effort involving many individuals and organizations, the project is being implemented in several phases.

Pilot study of a point-of-use decision support tool for cancer clinical trials eligibility

Many adults with cancer are not enrolled in clinical trials because caregivers do not have the time to match the patient's clinical findings with varying eligibility criteria associated with multiple trials for which the patient might be eligible. The authors developed a point-of-use portable decision support tool (DS-TRIEL) to automate this matching process. The support tool consists of a hand-held computer with a programmable relational database. A two-level hierarchic decision framework was used for the identification of eligible subjects for two open breast cancer clinical trials. The hand-held computer also provides protocol consent forms and schemas to further help the busy oncologist. This decision support tool and the decision framework on which it is based could be used for multiple trials and different cancer sites.

The guideline interchange format: a model for representing guidelines

OBJECTIVE

To allow exchange of clinical practice guidelines among institutions and computer-based applications.

DESIGN

The GuideLine Interchange Format (GLIF) specification consists of GLIF model and the GLIF syntax. The GLIF model is an object-oriented representation that consists of a set of classes for guideline entities, attributes for those classes, and data types for the attribute values. The GLIF syntax specifies the format of the test file that contains the encoding.

METHODS

Researchers from the InterMed Collaboratory at Columbia University, Harvard University (Brigham and Women's Hospital and Massachusetts General Hospital), and Stanford University analyzed four existing guideline systems to derive a set of requirements for guideline representation. The GLIF specification is a consensus representation developed through a brainstorming process. Four clinical guidelines were encoded in GLIF to assess its expressivity and to study the variability that occurs when two people from different sites encode the same guideline.

RESULTS

The encoders reported that GLIF was adequately expressive. A comparison of the encodings revealed substantial variability.

CONCLUSION

GLIF was sufficient to model the guidelines for the four conditions that were examined. GLIF needs improvement in standard representation of medical concepts, criterion logic, temporal information, and uncertainty.