Alternatives to randomized controlled trials in telemedicine

[Strategic framework to Support the evaluation of complex and innovative digital health projects]

Digital technologies play a central role in strategies to improve access, quality and efficiency of health care and services. However, many digital health projects have failed to become sustainable and spread across health organizations and systems. This situation is partly due to the fact that these projects are often developed and evaluated by reducing the issues linked mainly to the technological dimension. Such tradition has paid little attention to the fact that technology is introduced into pluralistic and complex sociotechnical systems such as health organizations and systems. The aim of this article is to propose practical and theorical, non-prescriptive, elements of reflection that can serve as a basis for evaluating complex and innovative digital health projects. This reflection builds on the lessons learned from the application of a strategic framework for evaluating three major complex and innovative digital health projects in Quebec over the last 15 years.

Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study

BACKGROUND

Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase.

OBJECTIVE

The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase.

METHODS

Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an "eHealth evaluation cycle" and subsequently compose the online "eHealth methodology guide."

RESULTS

The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an "eHealth evaluation cycle" was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the "eHealth methodology guide" was composed by assigning the 75 evaluation approaches to the specific study phases of the "eHealth evaluation cycle."

CONCLUSIONS

Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online "eHealth methodology guide." By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the "eHealth evaluation cycle," the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions.

Online Guide for Electronic Health Evaluation Approaches: Systematic Scoping Review and Concept Mapping Study

BACKGROUND

Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase.

OBJECTIVE

The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase.

METHODS

Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an "eHealth evaluation cycle" and subsequently compose the online "eHealth methodology guide."

RESULTS

The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an "eHealth evaluation cycle" was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the "eHealth methodology guide" was composed by assigning the 75 evaluation approaches to the specific study phases of the "eHealth evaluation cycle."

CONCLUSIONS

Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online "eHealth methodology guide." By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the "eHealth evaluation cycle," the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions.

Reaching People With Disabilities in Underserved Areas Through Digital Interventions: Systematic Review

Background

People with disabilities need rehabilitation interventions to improve their physical functioning, mental status, and quality of life. Many rehabilitation interventions can be delivered electronically ("digitally") via telehealth systems. For people with disabilities in underserved areas, electronically delivered rehabilitation interventions may be the only feasible service available for them.

Objective

The objective of this study was to evaluate the current status of digital interventions for people with disabilities in remote and underserved areas.

Methods

A systematic review was conducted on this topic. Keyword searches in multiple databases (PubMed, CINAHL, and Inspec) were performed to collect articles published in this field. The obtained articles were selected based on our selection criteria. Of the 198 identified articles, 16 duplicates were removed. After a review of the titles and abstracts of the remaining articles, 165 were determined to be irrelevant to this study and were therefore removed. The full texts of the remaining 17 articles were reviewed, and 6 of these articles were removed as being irrelevant to this study. The 11 articles remaining were discussed and summarized by 2 reviewers.

Results

These 11 studies cover a few types of disabilities, such as developmental disabilities and mobility impairments as well as several types of disability-causing disorders such as stroke, multiple sclerosis, traumatic brain injury, and facio-scapulo-humeral muscular dystrophy. Most of these studies were small-scale case studies and relatively larger-scale cohort studies; the project evaluation methods were mainly pre-post comparison, questionnaires, and interviews. A few studies also performed objective assessment of functional improvement. The intervention technology was mainly videoconferencing. Moreover, 10 of these studies were for people with disabilities in rural areas and 1 was for people in urban communities.

Conclusions

A small number of small-scale studies have been conducted on digital interventions for people with disabilities in underserved areas. Although the results reported in these studies were mostly positive, they are not sufficient to prove the effectiveness of telehealth-based digital intervention in improving the situation among people with disabilities because of the small sample sizes and lack of randomized controlled trials.

Future directions in the design, development, and investigation of technology as a service delivery vehicle

Treatment outcome research with children and adolescents has progressed to such an extent that numerous handbooks have been devoted to reviewing and summarizing the evidence base. Ensuring that consumers of these advancements in state-of-the-field interventions have the opportunity to access, engage in, and benefit from this evidence base, however, has been wrought with challenge. As such, much discussion exists about innovative strategies for overcoming the gap between research and practice; yet no other potential solution that has received more attention in both the popular and academic press than technology. The promise of technology is not surprising given the fast-paced evolution in development and, in turn, a seemingly endless range of possibilities for novel service delivery platforms. Yet this is precisely the most formidable challenge threatening to upset the very promise of this potential solution: The rate of emerging technologies is far outpacing the field's capacity to demonstrate the conceptual or empirical benefits of such an approach. Accordingly, this article aims to provide a series of recommendations that better situate empirical enquiry at the core of a collaborative development, testing, and deployment process that must define this line of work if the promise of mental health technologies is going to be a reality for front-line clinicians and the clients they serve.

Telemedicine service use: a new metric

Background

Policy makers and funding agencies require relevant information on current practices of the use of telemedicine infrastructure and services. Several metrics have been used to describe average use of telemedicine services. None are adequate.

Objective

To identify and assess a new metric–consultations per site per week (C/S/W).

Methods

To determine existing usage, all papers and abstracts published between January 2005 and December 2009 in the Journal of Telemedicine and Telecare and Telemedicine Journal and eHealth were reviewed. Pilot studies, research projects, services reporting less than one year’s data and teleradiology services were excluded.

Results

In total, 210 reports of telemedicine services were identified, 77 of which provided sufficient data to calculate C/S/W. Average use was low, 1.8±3.5 (median 0.7) C/S/W, with 61% of services reporting less than 1 C/S/W and 71% reporting 2 or fewer C/S/W. Studies reporting on data from 2006 to 2009 showed less use (average 1.5±2.3; median 0.7 C/S/W) than earlier reports from 1996 to 2005 (1.7±2.5; median 0.7 C/S/W).

Conclusions

The use of this new metric, C/S/W, is proposed as a standard measure of telemedicine service use. The generally low results opens debate about how well current clinical services are used.

The effect of ICU telemedicine on mortality and length of stay

We conducted a retrospective, observational study of patient outcomes in two intensive care units in the same hospital. The surgical ICU (SICU) implemented telemedicine and electronic medical records, while the medical ICU (MICU) did not. Medical charts were reviewed for a one-year period before telemedicine and a one-year period afterwards. In the SICU, records were obtained for 246 patients before and 1499 patients after implementation; in the MICU, records were obtained for 220 patients and 285 patients in the same periods. The outcomes of interest were ICU length of stay and mortality, and hospital length of stay and mortality. Outcome variables were severity-adjusted using APACHE scoring. A bootstrap method, with 1000 replicates, was used to assess stability of the findings. The adjusted ICU length of stay, ICU mortality, and hospital mortality for the SICU patients all decreased significantly after the implementation of telemedicine. There was no change in adjusted outcome variables in the MICU patients. Implementation of telemedicine and electronic records in the surgical ICU was associated with a profound reduction in severity-adjusted ICU length of stay, ICU mortality, and hospital mortality. However, it is not possible to conclude definitively that the observed associations seen in the SICU were due to the intervention.

A pilot test of a tailored mobile and web-based diabetes messaging system for adolescents

We conducted a pilot trial of a new mobile and web-based intervention to improve diabetes adherence. The text messaging system was designed to motivate and remind adolescents about diabetes self-care tasks. Text messages were tailored according to individually-reported barriers to diabetes self-care. A total of 23 adolescents with type 1 diabetes used the system for a period of three months. On average, they received 10 text messages per week (range 8-12). A matched historical control group from the same clinic was used for comparison. After three months, system users rated the content, usability and experiences with the system, which were very favourable. Comparison of the intervention and control groups indicated a significant interaction between group and time. Both groups had similar HbA(1c) levels at baseline. After three months, the mean HbA(1c) level in the intervention group was unchanged (8.8%), but the mean level in the control group was significantly higher (9.9%), P = 0.006. The results demonstrate the feasibility of the messaging system, user acceptance and a promising effect on glycaemic control. Integrating this type of messaging system with online educational programming could prove to be beneficial.

Wheelchair Seating Assessment and Intervention: A Comparison Between Telerehabilitation and Face-to-Face Service

This study compared outcomes of wheelchair seating and positioning interventions provided by telerehabilitation (n=10) and face-to-face (n=20; 10 in each of two comparison groups, one urban and one rural). Comparison clients were matched to the telerehabilitation clients in age, diagnosis, and type of seating components received. Clients and referring therapists rated their satisfaction and identified if seating intervention goals were met. Clients recorded travel expenses incurred or saved, and all therapists recorded time spent providing service. Wait times and completion times were tracked. Clients seen by telerehabilitation had similar satisfaction ratings and were as likely to have their goals met as clients seen face-to-face; telerehabilitation clients saved travel costs. Rural referring therapists who used telerehabilitation spent more time in preparation and follow-up than the other groups. Clients assessed by telerehabilitation had shorter wait times for assessment than rural face-to-face clients, but their interventions took as long to complete.

[Difficulties of incorporating telemedicine in health organizations: analytical perspectives]

Socioeconomic transformations at the end of the twentieth century and the beginning of the twentyfirst century pose major challenges to health systems, particularly to western societies with high technological systems and an aging population, in a political context of health expense reduction. The incorporation of information and communication technologies to health care systems, which has been called <<e-Health>>, has created enormous expectations in this context. Telemedicine has been one of pioneer experiences. Despite the early beginnings of telemedicine and the efforts invested, more widespread use of this technology remains difficult and controversial. Most projects last just the feasibility phase and are then forgotten. The traditional model of medical technologies assessment explains this phenomenon, based on the difficulty of obtaining the empirical evidence needed to support widespread adoption of telemedicine, as a consequence of the problems of conducting traditional studies of clinical efficacy and cost-effectiveness. In the last few years, a different analytical approach has emerged. This perspective indicates that the consolidation (or otherwise) of telemedicine projects will depend on the results of interaction between technology and the context where it is applied and not only on clinical results. Better and deeper empirical knowledge of these interaction processes is needed to increase the spread of telemedicine.