Reengineering Clinical Research with Informatics

Research data management in practice: Results from a cross-sectional survey of health and medical researchers from an academic institution in Australia

Background:

Building or acquiring research data management (RDM) capacity is a major challenge for health and medical researchers and academic institutes alike. Considering that RDM practices influence the integrity and longevity of data, targeting RDM services and support in recognition of needs is especially valuable in health and medical research.

Objective:

This project sought to examine the current RDM practices of health and medical researchers from an academic institution in Australia.

Method:

A cross-sectional survey was used to collect information from a convenience sample of 81 members of a research institute (68 academic staff and 13 postgraduate students). A survey was constructed to assess selected data management tasks associated with the earlier stages of the research data life cycle.

Results:

Our study indicates that RDM tasks associated with creating, processing and analysis of data vary greatly among researchers and are likely influenced by their level of research experience and RDM practices within their immediate teams.

Conclusion:

Evaluating the data management practices of health and medical researchers, contextualised by tasks associated with the research data life cycle, is an effective way of shaping RDM services and support in this group.

Implications:

This study recognises that institutional strategies targeted at tasks associated with the creation, processing and analysis of data will strengthen researcher capacity, instil good research practice and, over time, improve health informatics and research data quality.

Data management in clinical research: Synthesizing stakeholder perspectives

OBJECTIVE

This study assesses data management needs in clinical research from the perspectives of researchers, software analysts and developers.

MATERIALS AND METHODS

This is a mixed-methods study that employs sublanguage analysis in an innovative manner to link the assessments. We performed content analysis using sublanguage theory on transcribed interviews conducted with researchers at four universities. A business analyst independently extracted potential software features from the transcriptions, which were translated into the sublanguage. This common sublanguage was then used to create survey questions for researchers, analysts and developers about the desirability and difficulty of features. Results were synthesized using the common sublanguage to compare stakeholder perceptions with the original content analysis.

RESULTS

Individual researchers exhibited significant diversity of perspectives that did not correlate by role or site. Researchers had mixed feelings about their technologies, and sought improvements in integration, interoperability and interaction as well as engaging with study participants. Researchers and analysts agreed that data integration has higher desirability and mobile technology has lower desirability but disagreed on the desirability of data validation rules. Developers agreed that data integration and validation are the most difficult to implement.

DISCUSSION

Researchers perceive tasks related to study execution, analysis and quality control as highly strategic, in contrast with tactical tasks related to data manipulation. Researchers have only partial technologic support for analysis and quality control, and poor support for study execution.

CONCLUSION

Software for data integration and validation appears critical to support clinical research, but may be expensive to implement. Features to support study workflow, collaboration and engagement have been underappreciated, but may prove to be easy successes. Software developers should consider the strategic goals of researchers with regard to the overall coordination of research projects and teams, workflow connecting data collection with analysis and processes for improving data quality.

People, organizational, and leadership factors impacting informatics support for clinical and translational research

BACKGROUND

In recent years, there have been numerous initiatives undertaken to describe critical information needs related to the collection, management, analysis, and dissemination of data in support of biomedical research (J Investig Med 54:327-333, 2006); (J Am Med Inform Assoc 16:316-327, 2009); (Physiol Genomics 39:131-140, 2009); (J Am Med Inform Assoc 18:354-357, 2011). A common theme spanning such reports has been the importance of understanding and optimizing people, organizational, and leadership factors in order to achieve the promise of efficient and timely research (J Am Med Inform Assoc 15:283-289, 2008). With the emergence of clinical and translational science (CTS) as a national priority in the United States, and the corresponding growth in the scale and scope of CTS research programs, the acuity of such information needs continues to increase (JAMA 289:1278-1287, 2003); (N Engl J Med 353:1621-1623, 2005); (Sci Transl Med 3:90, 2011). At the same time, systematic evaluations of optimal people, organizational, and leadership factors that influence the provision of data, information, and knowledge management technologies and methods are notably lacking.

METHODS

In response to the preceding gap in knowledge, we have conducted both: 1) a structured survey of domain experts at Academic Health Centers (AHCs); and 2) a subsequent thematic analysis of public-domain documentation provided by those same organizations. The results of these approaches were then used to identify critical factors that may influence access to informatics expertise and resources relevant to the CTS domain.

RESULTS

A total of 31 domain experts, spanning the Biomedical Informatics (BMI), Computer Science (CS), Information Science (IS), and Information Technology (IT) disciplines participated in a structured surveyprocess. At a high level, respondents identified notable differences in theaccess to BMI, CS, and IT expertise and services depending on the establishment of a formal BMI academic unit and the perceived relationship between BMI, CS, IS, and IT leaders. Subsequent thematic analysis of the aforementioned public domain documents demonstrated a discordance between perceived and reported integration across and between BMI, CS, IS, and IT programs and leaders with relevance to the CTS domain.

CONCLUSION

Differences in people, organization, and leadership factors do influence the effectiveness of CTS programs, particularly with regard to the ability to access and leverage BMI, CS, IS, and IT expertise and resources. Based on this finding, we believe that the development of a better understanding of how optimal BMI, CS, IS, and IT organizational structures and leadership models are designed and implemented is critical to both the advancement of CTS and ultimately, to improvements in the quality, safety, and effectiveness of healthcare.

Chapter 1: Biomedical knowledge integration

The modern biomedical research and healthcare delivery domains have seen an unparalleled increase in the rate of innovation and novel technologies over the past several decades. Catalyzed by paradigm-shifting public and private programs focusing upon the formation and delivery of genomic and personalized medicine, the need for high-throughput and integrative approaches to the collection, management, and analysis of heterogeneous data sets has become imperative. This need is particularly pressing in the translational bioinformatics domain, where many fundamental research questions require the integration of large scale, multi-dimensional clinical phenotype and bio-molecular data sets. Modern biomedical informatics theory and practice has demonstrated the distinct benefits associated with the use of knowledge-based systems in such contexts. A knowledge-based system can be defined as an intelligent agent that employs a computationally tractable knowledge base or repository in order to reason upon data in a targeted domain and reproduce expert performance relative to such reasoning operations. The ultimate goal of the design and use of such agents is to increase the reproducibility, scalability, and accessibility of complex reasoning tasks. Examples of the application of knowledge-based systems in biomedicine span a broad spectrum, from the execution of clinical decision support, to epidemiologic surveillance of public data sets for the purposes of detecting emerging infectious diseases, to the discovery of novel hypotheses in large-scale research data sets. In this chapter, we will review the basic theoretical frameworks that define core knowledge types and reasoning operations with particular emphasis on the applicability of such conceptual models within the biomedical domain, and then go on to introduce a number of prototypical data integration requirements and patterns relevant to the conduct of translational bioinformatics that can be addressed via the design and use of knowledge-based systems.

Computational challenges and human factors influencing the design and use of clinical research participant eligibility pre-screening tools

Background

Clinical trials are the primary mechanism for advancing clinical care and evidenced-based practice, yet challenges with the recruitment of participants for such trials are widely recognized as a major barrier to these types of studies. Data warehouses (DW) store large amounts of heterogenous clinical data that can be used to enhance recruitment practices, but multiple challenges exist when using a data warehouse for such activities, due to the manner of collection, management, integration, analysis, and dissemination of the data. A critical step in leveraging the DW for recruitment purposes is being able to match trial eligibility criteria to discrete and semi-structured data types in the data warehouse, though trial eligibility criteria tend to be written without concern for their computability. We present the multi-modal evaluation of a web-based tool that can be used for pre-screening patients for clinical trial eligibility and assess the ability of this tool to be practically used for clinical research pre-screening and recruitment.

Methods

The study used a validation study, usability testing, and a heuristic evaluation to evaluate and characterize the operational characteristics of the software as well as human factors affecting its use.

Results

Clinical trials from the Division of Cardiology and the Department of Family Medicine were used for this multi-modal evaluation, which included a validation study, usability study, and a heuristic evaluation. From the results of the validation study, the software demonstrated a positive predictive value (PPV) of 54.12% and 0.7%, respectively, and a negative predictive value (NPV) of 73.3% and 87.5%, respectively, for two types of clinical trials. Heuristic principles concerning error prevention and documentation were characterized as the major usability issues during the heuristic evaluation.

Conclusions

This software is intended to provide an initial list of eligible patients to a clinical study coordinators, which provides a starting point for further eligibility screening by the coordinator. Because this software has a high “rule in” ability, meaning that it is able to remove patients who are not eligible for the study, the use of an automated tool built to leverage an existing enterprise DW can be beneficial to determining eligibility and facilitating clinical trial recruitment through pre-screening. While the results of this study are promising, further refinement and study of this and related approaches to automated eligibility screening, including comparison to other approaches and stakeholder perceptions, are needed and future studies are planned to address these needs.

The TOKEn project: knowledge synthesis for in silico science

OBJECTIVE

The conduct of investigational studies that involve large-scale data sets presents significant challenges related to the discovery and testing of novel hypotheses capable of supporting in silico discovery science. The use of what are known as Conceptual Knowledge Discovery in Databases (CKDD) methods provides a potential means of scaling hypothesis discovery and testing approaches for large data sets. Such methods enable the high-throughput generation and evaluation of knowledge-anchored relationships between complexes of variables found in targeted data sets.

METHODS

The authors have conducted a multipart model formulation and validation process, focusing on the development of a methodological and technical approach to using CKDD to support hypothesis discovery for in silico science. The model the authors have developed is known as the Translational Ontology-anchored Knowledge Discovery Engine (TOKEn). This model utilizes a specific CKDD approach known as Constructive Induction to identify and prioritize potential hypotheses related to the meaningful semantic relationships between variables found in large-scale and heterogeneous biomedical data sets.

RESULTS

The authors have verified and validated TOKEn in the context of a translational research data repository maintained by the NCI-funded Chronic Lymphocytic Leukemia Research Consortium. Such studies have shown that TOKEn is: (1) computationally tractable; and (2) able to generate valid and potentially useful hypotheses concerning relationships between phenotypic and biomolecular variables in that data collection.

CONCLUSIONS

The TOKEn model represents a potentially useful and systematic approach to knowledge synthesis for in silico discovery science in the context of large-scale and multidimensional research data sets.

Enabling distributed electronic research data collection for a rural Appalachian tobacco cessation study

Tobacco use is increasingly prevalent among vulnerable populations, such as people living in rural Appalachian communities. Owing to limited access to a reliable internet service in such settings, there is no widespread adoption of electronic data capture tools for conducting community-based research. By integrating the REDCap data collection application with a custom synchronization tool, the authors have enabled a workflow in which field research staff located throughout the Ohio Appalachian region can electronically collect and share research data. In addition to allowing the study data to be exchanged in near-real-time among the geographically distributed study staff and centralized study coordinator, the system architecture also ensures that the data are stored securely on encrypted laptops in the field and centrally behind the Ohio State University Medical Center enterprise firewall. The authors believe that this approach can be easily applied to other analogous study designs and settings.

TRIAD: The Translational Research Informatics and Data Management Grid

OBJECTIVE

Multi-disciplinary and multi-site biomedical research programs frequently require infrastructures capable of enabling the collection, management, analysis, and dissemination of heterogeneous, multi-dimensional, and distributed data and knowledge collections spanning organizational boundaries. We report on the design and initial deployment of an extensible biomedical informatics platform that is intended to address such requirements.

METHODS

A common approach to distributed data, information, and knowledge management needs in the healthcare and life science settings is the deployment and use of a service-oriented architecture (SOA). Such SOA technologies provide for strongly-typed, semantically annotated, and stateful data and analytical services that can be combined into data and knowledge integration and analysis "pipelines." Using this overall design pattern, we have implemented and evaluated an extensible SOA platform for clinical and translational science applications known as the Translational Research Informatics and Data-management grid (TRIAD). TRIAD is a derivative and extension of the caGrid middleware and has an emphasis on supporting agile "working interoperability" between data, information, and knowledge resources.

RESULTS

Based upon initial verification and validation studies conducted in the context of a collection of driving clinical and translational research problems, we have been able to demonstrate that TRIAD achieves agile "working interoperability" between distributed data and knowledge sources.

CONCLUSION

Informed by our initial verification and validation studies, we believe TRIAD provides an example instance of a lightweight and readily adoptable approach to the use of SOA technologies in the clinical and translational research setting. Furthermore, our initial use cases illustrate the importance and efficacy of enabling "working interoperability" in heterogeneous biomedical environments.

Translational informatics: enabling high-throughput research paradigms

A common thread throughout the clinical and translational research domains is the need to collect, manage, integrate, analyze, and disseminate large-scale, heterogeneous biomedical data sets. However, well-established and broadly adopted theoretical and practical frameworks and models intended to address such needs are conspicuously absent in the published literature or other reputable knowledge sources. Instead, the development and execution of multidisciplinary, clinical, or translational studies are significantly limited by the propagation of "silos" of both data and expertise. Motivated by this fundamental challenge, we report upon the current state and evolution of biomedical informatics as it pertains to the conduct of high-throughput clinical and translational research and will present both a conceptual and practical framework for the design and execution of informatics-enabled studies. The objective of presenting such findings and constructs is to provide the clinical and translational research community with a common frame of reference for discussing and expanding upon such models and methodologies.

Automation in an addiction treatment research clinic: computerised contingency management, ecological momentary assessment and a protocol workflow system

INTRODUCTION AND AIMS

A challenge in treatment research is the necessity of adhering to protocol and regulatory strictures while maintaining flexibility to meet patients' treatment needs and to accommodate variations among protocols. Another challenge is the acquisition of large amounts of data in an occasionally hectic environment, along with the provision of seamless methods for exporting, mining and querying the data.

DESIGN AND METHODS

We have automated several major functions of our outpatient treatment research clinic for studies in drug abuse and dependence. Here we describe three such specialised applications: the Automated Contingency Management (ACM) system for the delivery of behavioural interventions, the transactional electronic diary (TED) system for the management of behavioural assessments and the Protocol Workflow System (PWS) for computerised workflow automation and guidance of each participant's daily clinic activities. These modules are integrated into our larger information system to enable data sharing in real time among authorised staff.

RESULTS

ACM and the TED have each permitted us to conduct research that was not previously possible. In addition, the time to data analysis at the end of each study is substantially shorter. With the implementation of the PWS, we have been able to manage a research clinic with an 80 patient capacity, having an annual average of 18,000 patient visits and 7300 urine collections with a research staff of five. Finally, automated data management has considerably enhanced our ability to monitor and summarise participant safety data for research oversight.

DISCUSSION AND CONCLUSIONS

When developed in consultation with end users, automation in treatment research clinics can enable more efficient operations, better communication among staff and expansions in research methods.

Clinical research informatics: challenges, opportunities and definition for an emerging domain

OBJECTIVES

Clinical Research Informatics, an emerging sub-domain of Biomedical Informatics, is currently not well defined. A formal description of CRI including major challenges and opportunities is needed to direct progress in the field.

DESIGN

Given the early stage of CRI knowledge and activity, we engaged in a series of qualitative studies with key stakeholders and opinion leaders to determine the range of challenges and opportunities facing CRI. These phases employed complimentary methods to triangulate upon our findings.

MEASUREMENTS

Study phases included: 1) a group interview with key stakeholders, 2) an email follow-up survey with a larger group of self-identified CRI professionals, and 3) validation of our results via electronic peer-debriefing and member-checking with a group of CRI-related opinion leaders. Data were collected, transcribed, and organized for formal, independent content analyses by experienced qualitative investigators, followed by an iterative process to identify emergent categorizations and thematic descriptions of the data.

RESULTS

We identified a range of challenges and opportunities facing the CRI domain. These included 13 distinct themes spanning academic, practical, and organizational aspects of CRI. These findings also informed the development of a formal definition of CRI and supported further representations that illustrate areas of emphasis critical to advancing the domain.

CONCLUSIONS

CRI has emerged as a distinct discipline that faces multiple challenges and opportunities. The findings presented summarize those challenges and opportunities and provide a framework that should help inform next steps to advance this important new discipline.

An internet service supporting quality assessment of inpatient glycemic control

BACKGROUND

Several studies have linked the maintenance of normoglycemia in acutely ill inpatients with improved clinical outcomes. We previously proposed a few standard definitions for monitoring inpatient glycemic control, or "glucometrics." In clinical practice, limited data management resources for developing and refining measurement protocols can slow quality improvement efforts. With regard to glucometrics, there are few baseline data regarding the quality of hospital glycemic management. Furthermore, there are no reliable methods for hospitals to gauge the progress of their quality improvement efforts.

METHODS

We built a novel Web application that calculates glucometrics on anonymized blood glucose data files uploaded by registered users. This Web site also collects many key characteristics of the users and institutions utilizing the service. This application will allow us to pool data from several institutions to calculate aggregate glucometrics, providing baseline data for quality improvement efforts and ongoing metrics for institutions to gauge their progress.

RESULTS

The application, accessible at http://metrics.med.yale.edu, has already drawn visitors from several countries. A number of users have registered formally, and some have begun to upload institutional glucose data. The application delivers detailed glucometrics reports to registered users, complete with visual displays. Quality improvement staff from large health systems have been the predominant users.

CONCLUSIONS

We have created an open access Web application to facilitate quality monitoring and improvement efforts-as well as clinical research-regarding inpatient glycemic management. If employed widely, this application could help establish national performance standards for glycemic control.

People and organizational issues in research systems implementation

Knowledge about people and organizational issues pertinent to implementation and maintenance of clinical systems has grown steadily over the past fifteen years. Less is known about implementation of systems used for clinical and biomedical research. In conjunction with current National Institutes of Health Roadmap efforts that promote translational research, these issues should now be identified and addressed. During the 2007 American College of Medical Informatics Symposium, members discussed behavioral aspects of translational informatics. This article summarizes that discussion, which covered organizational issues, implications of how knowledge about clinical systems implementation can inform research systems implementation, and those issues unique to each kind of system.

Translational science: past, present, and future

The concept of translational science is at least 15 years old. However, in its most recent incarnation, it represents the identification of a funding category designed to encourage academic participation in a critical stage of the drug discovery and product development process. It is hoped that this will make the process both shorter and more efficient. In this review, the author first considers the historical development of the pharmaceutical R&D process. The place of translational science in the process, the scientific techniques involved, and aspects of the business environment necessary for its success are then considered. Translational science does not displace preclinical development. Both concepts are relevant to the paramount importance of successfully and expeditiously bridging the gap between preclinical science and clinical testing, “from bench to bedside.” Translational science is particularly likely to stimulate biomarker research in the universities and related business community and will probably give a modest boost to early clinical testing and commercialization of discoveries within the academic setting. Whether there will be a consequent improvement in the quality and efficiency of the overall process remains to be seen.