An Internet-based cancer clinical trials matching resource

Analysis of Not Structurable Oncological Study Eligibility Criteria for Improved Patient-Trial Matching

Background  Higher enrolment rates of cancer patients into clinical trials are necessary to increase cancer survival. As a prerequisite, an improved semiautomated matching of patient characteristics with clinical trial eligibility criteria is needed. This is based on the computer interpretability, i.e., structurability of eligibility criteria texts. To increase structurability, the common content, phrasing, and structuring problems of oncological eligibility criteria need to be better understood.

Objectives  We aimed to identify oncological eligibility criteria that were not possible to be structured by our manual approach and categorize them by the underlying structuring problem. Our results shall contribute to improved criteria phrasing in the future as a prerequisite for increased structurability.

Methods  The inclusion and exclusion criteria of 159 oncological studies from the Clinical Trial Information System of the National Center for Tumor Diseases Heidelberg were manually structured and grouped into content-related subcategories. Criteria identified as not structurable were analyzed further and manually categorized by the underlying structuring problem.

Results  The structuring of criteria resulted in 4,742 smallest meaningful components (SMCs) distributed across seven main categories (Diagnosis, Therapy, Laboratory, Study, Findings, Demographics, and Lifestyle, Others). A proportion of 645 SMCs (13.60%) was not possible to be structured due to content- and structure-related issues. Of these, a subset of 415 SMCs (64.34%) was considered not remediable, as supplementary medical knowledge would have been needed or the linkage among the sentence components was too complex. The main category “Diagnosis and Study” contained these two subcategories to the largest parts and thus were the least structurable. In the inclusion criteria, reasons for lacking structurability varied, while missing supplementary medical knowledge was the largest factor within the exclusion criteria.

Conclusion  Our results suggest that further improvement of eligibility criterion phrasing only marginally contributes to increased structurability. Instead, physician-based confirmation of the matching results and the exclusion of factors harming the patient or biasing the study is needed.

OCTANE: Oncology Clinical Trial Annotation Engine

PURPOSE

Many targeted therapies are currently available only via clinical trials. Therefore, routine precision oncology using biomarker-based assignment to drug depends on matching patients to clinical trials. A comprehensive and up-to-date trial database is necessary for optimal patient-trial matching.

METHODS

We describe processes for establishing and maintaining a clinical trial database, focusing on genomically informed trials. Furthermore, we present OCTANE (Oncology Clinical Trial Annotation Engine), an informatics framework supporting these processes in a scalable fashion. To illustrate how the framework can be applied at an institution, we describe how we implemented an instance of OCTANE at a large cancer center. OCTANE consists of three modules. The data aggregation module automates retrieval, aggregation, and update of trial information. The annotation module establishes the database schema, implements data integration necessary for automation, and provides an annotation interface. The update module monitors trial change logs, identifies critical change events, and alerts the annotators when manual intervention may be needed.

RESULTS

Using OCTANE, we annotated 5,439 oncology clinical trials (4,438 genomically informed trials) that collectively were associated with 1,453 drugs, 779 genes, and 252 cancer types. To date, we have used the database to screen 4,220 patients for trial eligibility. We compared the update module with expert review, and the module achieved 98.5% accuracy, 0% false-negative rate, and 2.3% false-positive rate.

CONCLUSION

OCTANE is a general informatics framework that can be helpful for establishing and maintaining a comprehensive database necessary for automating patient-trial matching, which facilitates the successful delivery of personalized cancer care on a routine basis. Several OCTANE components are publically available and may be useful to other precision oncology programs.

A bilingual, Internet-based, targeted advertising campaign for prostate cancer clinical trials: Assessing the feasibility, acceptability, and efficacy of a novel recruitment strategy

BACKGROUND

To address limitations in recruitment and enrollment of diverse, low-literacy patients into prostate cancer clinical trials, we evaluated the feasibility, acceptability, and efficacy of an English and Spanish, Internet-based, multilevel recruitment intervention.

METHODS

Intervention components included (1) a low-literacy, bilingual, automated, Internet-based clinical trial matching tool; (2) a bilingual nurse who assisted individuals with questions and enrollment; and (3) a targeted, Internet-based advertising campaign. We evaluated (a) completion of matching tool, (b) expression of interest in a clinical trial, (c) number of patients who matched to clinical trials at a single institution, (d) discussion of risks and benefits of clinical trials (via follow-up interviews), and (e) effect of the advertising on completing the matching tool. Feasibility, acceptability, and preliminary estimates of efficacy were measured through user engagement with the matching tool and subsequent qualitative interviews with these same users.

RESULTS

During the 28-week study period, 523 users provided demographic information, 263 were identified with prostate cancer, 192 (73%) matched to at least one clinical trial, and 29 (15.1%) of those who matched provided contact information. During the study period, 17 prostate cancer clinical trials were available for matching. We completed follow-up interviews with 14 of the 29 men who provided contact information. Of the 14, 85.7% discussed the risks and benefits of clinical trials with their physician, and 35.7% enrolled in a clinical trial. The Internet-based advertising campaign resulted in an increased number of matching tool completions.

CONCLUSIONS

Our study demonstrates that an Internet-based clinical trial matching tool that is advertised using a targeted Internet-based campaign can provide an effective means to reach diverse, low-literacy patients. When implemented at scale and over a longer duration, such interventions may help increase trial participation among underrepresented populations.

Automatic data source identification for clinical trial eligibility criteria resolution

Clinical trial coordinators refer to both structured and unstructured sources of data when evaluating a subject for eligibility. While some eligibility criteria can be resolved using structured data, some require manual review of clinical notes. An important step in automating the trial screening process is to be able to identify the right data source for resolving each criterion. In this work, we discuss the creation of an eligibility criteria dataset for clinical trials for patients with two disparate diseases, annotated with the preferred data source for each criterion (i.e., structured or unstructured) by annotators with medical training. The dataset includes 50 heart-failure trials with a total of 766 eligibility criteria and 50 trials for chronic lymphocytic leukemia (CLL) with 677 criteria. Further, we developed machine learning models to predict the preferred data source: kernel methods outperform simpler learning models when used with a combination of lexical, syntactic, semantic, and surface features. Evaluation of these models indicates that the performance is consistent across data from both diagnoses, indicating generalizability of our method. Our findings are an important step towards ongoing efforts for automation of clinical trial screening.

Trial prospector: matching patients with cancer research studies using an automated and scalable approach

Cancer is responsible for approximately 7.6 million deaths per year worldwide. A 2012 survey in the United Kingdom found dramatic improvement in survival rates for childhood cancer because of increased participation in clinical trials. Unfortunately, overall patient participation in cancer clinical studies is low. A key logistical barrier to patient and physician participation is the time required for identification of appropriate clinical trials for individual patients. We introduce the Trial Prospector tool that supports end-to-end management of cancer clinical trial recruitment workflow with (a) structured entry of trial eligibility criteria, (b) automated extraction of patient data from multiple sources, (c) a scalable matching algorithm, and (d) interactive user interface (UI) for physicians with both matching results and a detailed explanation of causes for ineligibility of available trials. We report the results from deployment of Trial Prospector at the National Cancer Institute (NCI)-designated Case Comprehensive Cancer Center (Case CCC) with 1,367 clinical trial eligibility evaluations performed with 100% accuracy.

Employing computers for the recruitment into clinical trials: a comprehensive systematic review

BACKGROUND

Medical progress depends on the evaluation of new diagnostic and therapeutic interventions within clinical trials. Clinical trial recruitment support systems (CTRSS) aim to improve the recruitment process in terms of effectiveness and efficiency.

OBJECTIVE

The goals were to (1) create an overview of all CTRSS reported until the end of 2013, (2) find and describe similarities in design, (3) theorize on the reasons for different approaches, and (4) examine whether projects were able to illustrate the impact of CTRSS.

METHODS

We searched PubMed titles, abstracts, and keywords for terms related to CTRSS research. Query results were classified according to clinical context, workflow integration, knowledge and data sources, reasoning algorithm, and outcome.

RESULTS

A total of 101 papers on 79 different systems were found. Most lacked details in one or more categories. There were 3 different CTRSS that dominated: (1) systems for the retrospective identification of trial participants based on existing clinical data, typically through Structured Query Language (SQL) queries on relational databases, (2) systems that monitored the appearance of a key event of an existing health information technology component in which the occurrence of the event caused a comprehensive eligibility test for a patient or was directly communicated to the researcher, and (3) independent systems that required a user to enter patient data into an interface to trigger an eligibility assessment. Although the treating physician was required to act for the patient in older systems, it is now becoming increasingly popular to offer this possibility directly to the patient.

CONCLUSIONS

Many CTRSS are designed to fit the existing infrastructure of a clinical care provider or the particularities of a trial. We conclude that the success of a CTRSS depends more on its successful workflow integration than on sophisticated reasoning and data processing algorithms. Furthermore, some of the most recent literature suggest that an increase in recruited patients and improvements in recruitment efficiency can be expected, although the former will depend on the error rate of the recruitment process being replaced. Finally, to increase the quality of future CTRSS reports, we propose a checklist of items that should be included.

eTACTS: a method for dynamically filtering clinical trial search results

OBJECTIVE

Information overload is a significant problem facing online clinical trial searchers. We present eTACTS, a novel interactive retrieval framework using common eligibility tags to dynamically filter clinical trial search results.

MATERIALS AND METHODS

eTACTS mines frequent eligibility tags from free-text clinical trial eligibility criteria and uses these tags for trial indexing. After an initial search, eTACTS presents to the user a tag cloud representing the current results. When the user selects a tag, eTACTS retains only those trials containing that tag in their eligibility criteria and generates a new cloud based on tag frequency and co-occurrences in the remaining trials. The user can then select a new tag or unselect a previous tag. The process iterates until a manageable number of trials is returned. We evaluated eTACTS in terms of filtering efficiency, diversity of the search results, and user eligibility to the filtered trials using both qualitative and quantitative methods.

RESULTS

eTACTS (1) rapidly reduced search results from over a thousand trials to ten; (2) highlighted trials that are generally not top-ranked by conventional search engines; and (3) retrieved a greater number of suitable trials than existing search engines.

DISCUSSION

eTACTS enables intuitive clinical trial searches by indexing eligibility criteria with effective tags. User evaluation was limited to one case study and a small group of evaluators due to the long duration of the experiment. Although a larger-scale evaluation could be conducted, this feasibility study demonstrated significant advantages of eTACTS over existing clinical trial search engines.

CONCLUSION

A dynamic eligibility tag cloud can potentially enhance state-of-the-art clinical trial search engines by allowing intuitive and efficient filtering of the search result space.

Adoption, acceptability, and accuracy of an online clinical trial matching website for breast cancer

Background

Less than 5% of breast cancer patients participate in clinical trials. To increase patients’ awareness and access to trials, we created BreastCancerTrials.org, a clinical trial matching website. BreastCancerTrials.org matched patients to trials based on their self-reported breast cancer history. It also provided a messaging platform through which patients could self-refer themselves to participating research sites.

Objective

To assess adoption by research sites, acceptability to patients, and patients’ accuracy in providing information to BreastCancerTrials.org.

Methods

We approached 13 research sites in Northern California to list their trials on BreastCancerTrials.org. For adoption, we examined the willingness of contacted research sites to collaborate with BreastCancerTrials.org. For acceptability, we analyzed usage statistics of visitors who completed the BreastCancerTrials.org health history questionnaire in the first 14 months after launch and surveyed users who visited the website during its first year about their experience. For accuracy, we compared the self-reported health history of 20 patients against their medical records. The health history questionnaire was divided into four sections: About Me, personal information including date of birth and sex; My Health as of Today, current status including cancer stage, menopausal status, and sites with evidence of disease; My Cancer, diagnostic information such as hormone and human epidermal growth factor receptor 2 status; and My Treatment, an itemized record of past treatment including responses to therapy.

Results

A total of 12 sites contributed 55 trials. Regarding acceptability, 733 visitors registered on the website; 428 reported their health history; and 407 matched to at least one trial. Of 375 patients who were sent a survey, 75 responded (20%); 23 of the 75 (31%) contacted a research site, 12 of the 23 (52%) were eligible for a trial, and 5 of the 12 (42%) reported enrolling. As for accuracy, 20 clinic visitors reported 1456 health history items, 1324 of which matched their clinic record (90.93%).

Conclusions

BreastCancerTrials.org was adopted by research sites. Patients found it acceptable and were able to provide accurate information for trial matching. Based on our findings, we launched an upgraded version of BreastCancerTrials.org as a national service in October 2008.

Outcomes of a cancer clinical trial matching service

The American Cancer Society (ACS) and Coalition of Cancer Cooperative Groups (CCCG) provide a clinical trial (CT) information/matching/eligibility service (Clinical Trials Matching Service [CTMS]). Patients' demographic and clinical data, enrollment status, and self-reported barriers to CT participation were analyzed to assess enrollment rates and determinants of enrollment. During 3 years beginning October 1, 2007, the CTMS served 6,903 patients via the ACS call center. Among the 1,987 patients with follow-up information on enrollment, 219 (11.0%) enrolled in a CT; 48 of these 219 enrollees chose a CT they found via the CTMS. Patients were less likely to enroll if they had poor ECOG performance status (P = 0.032); were African American (P = 0.0003), were uninsured or had Medicaid coverage (P = 0.024), or had lower stage disease (P = 0.018). Enrollment varied by trial type/cancer site/system (P = .026). Several barriers significantly predicted nonenrollment. Broader availability of a CTMS might help improve patient participation in cancer clinical trials.

Dynamic categorization of clinical research eligibility criteria by hierarchical clustering

OBJECTIVE

To semi-automatically induce semantic categories of eligibility criteria from text and to automatically classify eligibility criteria based on their semantic similarity.

DESIGN

The UMLS semantic types and a set of previously developed semantic preference rules were utilized to create an unambiguous semantic feature representation to induce eligibility criteria categories through hierarchical clustering and to train supervised classifiers.

MEASUREMENTS

We induced 27 categories and measured the prevalence of the categories in 27,278 eligibility criteria from 1578 clinical trials and compared the classification performance (i.e., precision, recall, and F1-score) between the UMLS-based feature representation and the "bag of words" feature representation among five common classifiers in Weka, including J48, Bayesian Network, Naïve Bayesian, Nearest Neighbor, and instance-based learning classifier.

RESULTS

The UMLS semantic feature representation outperforms the "bag of words" feature representation in 89% of the criteria categories. Using the semantically induced categories, machine-learning classifiers required only 2000 instances to stabilize classification performance. The J48 classifier yielded the best F1-score and the Bayesian Network classifier achieved the best learning efficiency.

CONCLUSION

The UMLS is an effective knowledge source and can enable an efficient feature representation for semi-automated semantic category induction and automatic categorization for clinical research eligibility criteria and possibly other clinical text.

Consumer input into research: the Australian Cancer Trials website

BACKGROUND

The Australian Cancer Trials website (ACTO) was publicly launched in 2010 to help people search for cancer clinical trials recruiting in Australia, provide information about clinical trials and assist with doctor-patient communication about trials. We describe consumer involvement in the design and development of ACTO and report our preliminary patient evaluation of the website.

METHODS

Consumers, led by Cancer Voices NSW, provided the impetus to develop the website. Consumer representative groups were consulted by the research team during the design and development of ACTO which combines a search engine, trial details, general information about trial participation and question prompt lists. Website use was analysed. A patient evaluation questionnaire was completed at one hospital, one week after exposure to the website.

RESULTS

ACTO's main features and content reflect consumer input. In February 2011, it covered 1, 042 cancer trials. Since ACTO's public launch in November 2010, until the end of February 2011, the website has had 2, 549 new visits and generated 17, 833 page views. In a sub-study of 47 patient users, 89% found the website helpful for learning about clinical trials and all respondents thought patients should have access to ACTO.

CONCLUSIONS

The development of ACTO is an example of consumers working with doctors, researchers and policy makers to improve the information available to people whose lives are affected by cancer and to help them participate in their treatment decisions, including consideration of clinical trial enrolment. Consumer input has ensured that the website is informative, targets consumer priorities and is user-friendly. ACTO serves as a model for other health conditions.

Assessing the impact of user-centered research on a clinical trial eHealth tool via counterbalanced research design

OBJECTIVE

Informatics applications have the potential to improve participation in clinical trials, but their design must be based on user-centered research. This research used a fully counterbalanced experimental design to investigate the effect of changes made to the original version of a website, http://BreastCancerTrials.org/, and confirm that the revised version addressed and reinforced patients' needs and expectations.

DESIGN

Participants included women who had received a breast cancer diagnosis within the last 5 years (N=77). They were randomized into two groups: one group used and reviewed the original version first followed by the redesigned version, and the other group used and reviewed them in reverse order.

MEASUREMENTS

The study used both quantitative and qualitative measures. During use, participants' click paths and general reactions were observed. After use, participants were asked to answer survey items and open-ended questions to indicate their reactions and which version they preferred and met their needs and expectations better.

RESULTS

Overall, the revised version of the site was preferred and perceived to be clearer, easier to navigate, more trustworthy and credible, and more private and safe overall. However, users who viewed the original version last had similar attitudes toward both versions.

CONCLUSION

By applying research findings to the redesign of a website for clinical trial searching, it was possible to re-engineer the interface to better support patients' decisions to participate in clinical trials. The mechanisms of action in this case appeared to revolve around creating an environment that supported a sense of personal control and decisional autonomy.

The Brustkrebs-Studien.de website for breast cancer patients: User acceptance of a German internet portal offering information on the disease and treatment options, and a clinical trials matching service

Background

The internet portal http://www.brustkrebs-studien.de (BKS) was launched in 2000 by the German Society of Senology (DGS) and the Baden-Württemberg Institute for Women's Health (IFG) to provide expert-written information on breast cancer online and to encourage and facilitate the participation of breast cancer patients in clinical trials. We describe the development of BKS and its applications, and report on website statistics and user acceptance.

Methods

Existing registries, including ClinicalTrials.gov, were analysed before we designed BKS, which combines a trial registry, a knowledge portal, and an online second opinion service. An advisory board guided the process. Log files and patient enquiries for trial participation and second opinions were analysed. A two-week user satisfaction survey was conducted online.

Results

During 10/2005-06/2010, the portal attracted 702,655 visitors, generating 15,507,454 page views. By 06/2010, the website's active scientific community consisted of 189 investigators and physicians, and the registry covered 163 clinical trial protocols. In 2009, 143 patients requested trial enrolment and 119 sought second opinions or individual treatment advice from the expert panel. During the two-week survey in 2008, 5,702 BKS visitors submitted 507 evaluable questionnaires. Portal acceptance was high. Respondents trusted information correctness (80%), welcomed self-matching to clinical trials (79%) and planned to use the portal in the future (76%) and recommend it to others (81%).

Conclusions

BKS is an established and trusted breast cancer information platform offering up-to-date resources and protocols to the growing physician and patient community to encourage participation in clinical trials. Further studies are needed to assess potential increases in trial enrolment by eligibility matching services.

Patient internet use surrounding cancer clinical trials: clinician perceptions and responses

Clinician perceptions of patient internet use related to clinical trials are not well documented. This exploratory study surveyed how cancer care providers at one NCI-designated cancer center viewed patient internet use surrounding cancer trials, including whether it affected patient decision making regarding trial enrollment. The sample included 20 oncologists (59%) and 14 (41%) nurses (n=34). Most clinicians (n=26; 76%) perceived the internet as having an effect on whether or not patients decided to enroll in a cancer trial. Two thirds (n=17; 65%) felt that this effect was positive, including in terms of enhancing patient knowledge of, access to, and enrollment in trials. Clinicians were asked if they ever discussed with their patients the topic of going online to find out more about cancer trials. Over half (n=18; 58%) who responded (n=31) to this item said yes; the rest (n=13; 42%) said no. The majority (n=10; 77%) in the "no" category were among those who reported that the internet had an effect on patient decision making. These data provisionally suggest that clinicians may see the internet as having mostly a positive effect on patient decision making about cancer trials, but that their communication efforts with patients do not always logically follow from this perception. Provider-patient discussion about internet use may be an opportunity for clinicians to contribute to improved patient knowledge of and enrollment in cancer trials. More research is needed to confirm and explain the gap between clinician perception and communication regarding trial-related internet use by cancer patients.

Formal representation of eligibility criteria: a literature review

Standards-based, computable knowledge representations for eligibility criteria are increasingly needed to provide computer-based decision support for automated research participant screening, clinical evidence application, and clinical research knowledge management. We surveyed the literature and identified five aspects of eligibility criteria knowledge representation that contribute to the various research and clinical applications: the intended use of computable eligibility criteria, the classification of eligibility criteria, the expression language for representing eligibility rules, the encoding of eligibility concepts, and the modeling of patient data. We consider three of these aspects (expression language, codification of eligibility concepts, and patient data modeling) to be essential constructs of a formal knowledge representation for eligibility criteria. The requirements for each of the three knowledge constructs vary for different use cases, which therefore should inform the development and choice of the constructs toward cost-effective knowledge representation efforts. We discuss the implications of our findings for standardization efforts toward knowledge representation for sharable and computable eligibility criteria.

Using the Internet to search for cancer clinical trials: a comparative audit of clinical trial search tools

Advancing the clinical trial research process to improve cancer treatment necessitates helping people with cancer identify and enroll in studies, and researchers are using the power of the Internet to facilitate this process. This study used a content analysis of online cancer clinical trial search tools to understand what people with cancer might encounter. The content analysis revealed that clinical trial search tools were easy to identify using a popular search engine, but their functionality and content varied greatly. Most required that users be fairly knowledgeable about their medical condition and sophisticated in their web navigation skills. The ability to search by a specific health condition or type of cancer was the most common search strategy. The more complex tools required that users input detailed information about their personal medical history and have knowledge of specific clinical trial terminology. Search tools, however, only occasionally advised users to consult their doctors regarding clinical trial decision-making. This, along with the complexity of the tools suggests that online search tools may not adequately facilitate the clinical trial recruitment process. Findings from this analysis can be used as a framework from which to systematically examine actual consumer experience with online clinical trial search tools.

User-centered research on breast cancer patient needs and preferences of an Internet-based clinical trial matching system

Background

Internet-based clinical trial matching systems have the potential to streamline the search process for women with breast cancer seeking alternative treatments. A prototype system was developed to leverage the capabilities of a personal health record system for the purpose of identifying clinical trials.

Objective

This study examines how breast cancer patients perceive and interact with a preliminary version of an Internet-based clinical trial matching system, while taking into account the demands of diagnosis and treatment decision making.

Methods

Breast cancer patients participated in small group discussions and interacted with the prototype website in a two-phase qualitative research process. The first phase explored the experience of breast cancer patients (n = 8) with treatment decision making, initial responses to the idea of Internet-based clinical trial matching systems, and reactions to the prototype site. In the second phase, a different set of breast cancer patients (n = 7) reviewed revised website content and presentation and participated in a usability test in which they registered on the system and completed a personal health record to set up the matching process.

Results

Participants were initially skeptical of the prototype system because it emphasized registration, had a complicated registration process, and asked for complex medical information. Changing content and attending to usability guidelines improved the experience for women in the second phase of the research and enabled the identification of functionality and content issues, such as lack of clear information and directions on how to use the system.

Conclusions

This study showed that women felt favorably about the idea of using the Internet to search for clinical trials but that such a system needed to meet their expectations for credibility and privacy and be sensitive to their situation. Developers can meet these expectations by conforming to established usability guidelines and testing improvements with breast cancer patients. Future research is needed to verify these findings and to continue to improve systems of this nature.

Creating a framework for online cancer services research to facilitate timely and interdisciplinary applications

Researchers from a wide array of disciplines have conducted engaging and informative studies in recent years concerning the use of the Internet for cancer-related services. Typically, these publications provide key data related to utilization statistics, how online information can be used, what users want or expect from the Internet, outcomes or impacts, and quality and credibility of websites. These are important themes for understanding online cancer issues. However, this special issue of the Journal of Medical Internet Research seeks to recast these themes in a way that will facilitate pragmatic and applied means of employing data in prescriptive and interdisciplinary ways. This issue includes 14 papers that exemplify applications for the research framework recommended in this paper. This framework includes an expanded focus on the development and design of online cancer services, online consumer behavior/communication, behavior change, and living with cancer.