Patients' use and views of real-time feedback technology in general practice

Use of Daily Web-Based, Real-Time Feedback to Improve Patient and Family Experience

Real-time feedback is a growing trend in patient- and family experience (PFE) work as it allows for immediate service recovery, though it typically requires a significant investment of time and financial resources. We describe a partnership with our "edutainment" system to administer an automated daily experience question (the "Daily Pulse Measure [DPM]") that allowed targeted just-in-time responses to low scores with minimal administrative cost. Through a series of Plan-Do-Study-Act cycles guided by family feedback, the question was created and modified, and the use of the question spread to all hospital units. The response rate was 23%, similar to our Hospital Consumer Assessment of Healthcare Providers and Systems survey response rate of 24% during the study period. Though the DPM did not have a consistent impact on the results of the 2 PFE survey questions we evaluated, units with improved PFE scores after the DPM roll-out tended to have more robust service recovery than those with low scores.

Efficacy and Safety outcomes of a novel model to assess new medical retina referrals in a high-volume medical retina virtual clinic

BACKGROUND

Ophthalmology outpatient attendances have significantly increased recently with rising pressure from backlogs arising from the pandemic. Medical retina digital surveillance clinics for stable follow-up appointments are well established. We present a model for assessing new referrals and evaluating clinical outcomes and long-term sustainability in a complex high-volume medical retina service.

METHODS

Suitable routine new patient referrals were identified from electronic referrals and referred to this new pathway. Structured history, visual acuities, and intraocular pressures were recorded, and widefield colour fundus and optical coherence tomography imaging were performed at a imaging hub for asynchronous consultant-led review.

RESULTS

1458 patients were invited to attend over four months, with a 13.2% did-not-attend (DNA) rate. Common diagnoses included stable diabetic retinopathy (19.9%), early age-related macular degeneration (6.7%), central serous retinopathy (8.8%), and retinal vein occlusion (6.3%). 7 patients (0.05%) required urgent same-day review. 61 (5.0%) required urgent face-to-face (F2F) assessment within two weeks. A total of 727 (59.0%) were either discharged or remained in the virtual pathway following their first visit.

CONCLUSION

This study encourages the use of a digital model that efficiently assesses suitable newly referred medical retina patients in both complex and local eye unit settings. This decreased the need for F2F clinics and resources. Further patient satisfaction surveys for digital services are currently being evaluated to guide long-term sustainability of this model.

Outpatients' perception of their preoperative information regarding their health literacy skills and their preoperative anxiety level: Protocol for a prospective multicenter cross-sectional study

Despite the benefits related to ambulatory surgery such as cost reduction due to lack of accommodation and patient satisfaction due to early home return, it may not lead to these expected benefits. Indeed, this kind of practice can increase responsibility for the person being treated and his or her relatives. It is therefore essential to inform them as well as possible to obtain their adherence to the proposed care protocol. Nevertheless, patients' failures to comply with preoperative instructions or the non-attendance of the patient may result in late cancellation of the scheduled surgery. In order to reduce this kind of dysfunction, the Assistance Publique-Hôpitaux de Paris (APHP) uses a reminder system by Short Message Service (SMS).This study is a descriptive cross-sectional multicenter study that focuses on outpatients' lived experiences of their preoperative preparation and information. It aims to collect patients' perceptions of their ability to follow preoperative instructions received by SMS the day before an operation performed for ambulatory surgery, according to their level of health literacy (HL) and preoperative anxiety. Indeed, poor communication between patients and doctors can contribute to preoperative anxiety, while low health literacy (LHL) can lead to poor understanding of preoperative preparation instructions. Therefore, it seems important to take these 2 criteria into account in this study. This research is designed to interview outpatients undergoing ambulatory surgery in the establishments of APHP. A self-questionnaire will be used for this purpose. The choice of this institution is justified by its decision to use in all care units the reminder of preoperative instructions by SMS.The main outcome is the perception of outpatients with LHL skills regarding preoperative information provided by doctors.French ethics review committee (Comité d'Ethique de la Recherche) of the University of Paris has approved the study protocol (IRB 00012020-14). Results from this study will be disseminated through oral communications and a scientific article in an international peer-reviewed journal.This protocol is registered on researchregistry.com (researchregistry5834). This version number is 1.1 Protocol dated July 22, 2020.

Health system responsiveness: a systematic evidence mapping review of the global literature

BACKGROUND

The World Health Organisation framed responsiveness, fair financing and equity as intrinsic goals of health systems. However, of the three, responsiveness received significantly less attention. Responsiveness is essential to strengthen systems' functioning; provide equitable and accountable services; and to protect the rights of citizens. There is an urgency to make systems more responsive, but our understanding of responsiveness is limited. We therefore sought to map existing evidence on health system responsiveness.

METHODS

A mixed method systemized evidence mapping review was conducted. We searched PubMed, EbscoHost, and Google Scholar. Published and grey literature; conceptual and empirical publications; published between 2000 and 2020 and English language texts were included. We screened titles and abstracts of 1119 publications and 870 full texts.

RESULTS

Six hundred twenty-one publications were included in the review. Evidence mapping shows substantially more publications between 2011 and 2020 (n = 462/621) than earlier periods. Most of the publications were from Europe (n = 139), with more publications relating to High Income Countries (n = 241) than Low-to-Middle Income Countries (n = 217). Most were empirical studies (n = 424/621) utilized quantitative methodologies (n = 232), while qualitative (n = 127) and mixed methods (n = 63) were more rare. Thematic analysis revealed eight primary conceptualizations of 'health system responsiveness', which can be fitted into three dominant categorizations: 1) unidirectional user-service interface; 2) responsiveness as feedback loops between users and the health system; and 3) responsiveness as accountability between public and the system.

CONCLUSIONS

This evidence map shows a substantial body of available literature on health system responsiveness, but also reveals evidential gaps requiring further development, including: a clear definition and body of theory of responsiveness; the implementation and effectiveness of feedback loops; the systems responses to this feedback; context-specific mechanism-implementation experiences, particularly, of LMIC and fragile-and conflict affected states; and responsiveness as it relates to health equity, minority and vulnerable populations. Theoretical development is required, we suggest separating ideas of services and systems responsiveness, applying a stronger systems lens in future work. Further agenda-setting and resourcing of bridging work on health system responsiveness is suggested.

Effect of Real-Time Feedback Devices on Primary Care Patient Experience Scores: A Cluster-Randomized Trial

Patient experience is a critical measure for ambulatory primary care, although it is unclear how to best improve patient experience scores. This study aimed to determine whether use of a real-time feedback (RTF) device improved patient experience scores in a cluster-randomized trial. The primary outcomes were change from baseline in 9 Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) question and domain scores most closely related to the RTF questions asked in a linear mixed effects model. There were no observed statistically significant intervention-related differences in CG-CAHPS scores in any of the 9 CG-CAHPS questions or domains (P = .12-.99). In intervention clinics, there were no statistically significant correlation between CG-CAHPS top box scores and RTF device scores (P = .23-.98). Clinics in an urban primary care network randomized to receive RTF devices did not significantly improve related CG-CAHPS question or domain scores nor were those scores correlated with RTF device scores. More research is needed to identify effective interventions to improve ambulatory primary care patient experience.

Beyond standard rehabilitation programmes: Working with people with MS for adequate goal setting and rehabilitation treatment evaluation

Shared decision-making occurs when the decision is 'preference sensitive'. It consists of identifying the different treatment options (choice talk), considering the advantages and disadvantages of each option (option talk), and then supporting making the decision in the light of an individual's experiences and values (decision talk). It is most effective when working with an 'activated patient', that is, one who is prepared for the shared decision-making role. In rehabilitation, many decisions are preference sensitive. These decisions may be framed as 'goal setting'. Skilled clinicians can support patients to learn goal setting skills until the person has the skills to maintain health supporting behaviours most of the time, only seeing a clinical team at times of change or crisis. The steps in goal setting can be summarised as building empathy, creating a contract, identifying priorities, summarising the conversation, articulating the goal, defining actions, building coping plans, and then reviewing progress. Working with people with MS can extend beyond working with individuals to a consideration of what people with MS want from services. This can result in the co-production and co-design of services, as well as the identification of research priorities as exemplified by the James Lind Alliance.

Using patient-reported measures to drive change in healthcare: the experience of the digital, continuous and systematic PREMs observatory in Italy

BACKGROUND

The use of Patient Reported Experience Measures (PREMs) has great potential in healthcare service improvement, but a limited use. This paper presents an empirical case of PREMs innovation in Italy, to foster patient data use up to the ward level, by keeping strengths and addressing weaknesses of previous PREMs survey experiences. The paper reports key lessons learned in this ongoing experience of action research, directly involving practitioners.

METHODS

The aim of this paper is to present the results of an ongoing action research, encompassing the innovation of PREMs collection, reporting and use, currently adopted by 21 hospitals of two Italian regions. The continuous and systematic PREMs collection has been implemented between 2017 and 2019 and includes: a continuous web-based administration, using web-services; an augmented and positive questionnaire matching standard closed-ended questions with narrative sections; the inclusion and benchmarking of patient data within a shared performance evaluation system; public disclosure of aggregated anonymized data; a multi-level and real-time web-platform for reporting PREMs to professionals. The action research was carried out with practitioners in a real-life and complex context. The authors used multiple data sources and methods: observations, feedback of practitioners, collected during several workshops and meetings, and analysis of preliminary data on the survey implementation.

RESULTS

A continuous and systematic PREMs observatory was developed and adopted in two Italian regions. PREMs participation and response rates tend to increase over time, reaching stable percentages after the first months. Narrative feedback provide a 'positive narration' of episodes and behaviours that made the difference to patients and can inform quality improvement actions. Real-time reporting of quantitative and qualitative data is enabling a gratifying process of service improvement and people management at all the hospitals' levels.

CONCLUSIONS

The PREMs presented in this paper has been recognized by healthcare professionals and managers as a strategic and positive tool for improving an actual use of PREMs at system and ward levels, by measuring and highlighting positive deviances, such as compassionate behaviours.

Investigating the feasibility of a patient feedback tool to improve safety in Australian primary care: a study protocol

INTRODUCTION

Patients are a valuable source of information about ways to prevent harm in healthcare, and can provide feedback about the factors that contribute to safety incidents. The Primary Care Patient Measure of Safety (PC PMOS) is a novel and validated tool that captures patient feedback on safety and can be used by primary care practice teams to identify and prevent safety incidents. The aim of this study is to assess the feasibility of PC PMOS as a tool for data-driven safety improvement and monitoring in Australian primary care.

METHODS AND ANALYSIS

Feasibility will be assessed using a mixed-methods approach to understand the enablers, barriers, acceptability, practicability, intervention fidelity and scalability of C PMOS as a tool for safety improvement across six primary care practices in the south-west region of Victoria. Patients over the age of 18 years attending their primary care practice will be invited to complete the PC PMOS when presenting for an appointment. Staff members at each practice will form a safety improvement team. Staff will then use the patient feedback to develop and implement specific safety interventions over a 6-month period. Data collection methods during the intervention period includes audio recordings of staff meetings, overt observations at training and education workshops, reflexive researcher insights, document collection and review. Data collection postintervention includes patient completion of the PC PMOS and semistructured interviews with staff. Triangulation and thematic analysis techniques will be employed to analyse the qualitative and content data. Analysis methods will use current evidence and models of healthcare culture, safety improvement and patient involvement in safety to inform the findings.

ETHICS AND DISSEMINATION

Ethics approval was granted by Deakin University Human Ethics Advisory Group, Faculty of Health (HEAG-H 175_2017). Study results will be disseminated through local and international conferences and peer-reviewed publications.

Evaluating Digital Maturity and Patient Acceptability of Real-Time Patient Experience Feedback Systems: Systematic Review

Background

One of the essential elements of a strategic approach to improving patients’ experience is to measure and report on patients’ experiences in real time. Real-time feedback (RTF) is increasingly being collected using digital technology; however, there are several factors that may influence the success of the digital system.

Objective

The aim of this review was to evaluate the digital maturity and patient acceptability of real-time patient experience feedback systems.

Methods

We systematically searched the following databases to identify papers that used digital systems to collect RTF: The Cochrane Library, Global Health, Health Management Information Consortium, Medical Literature Analysis and Retrieval System Online, EMBASE, PsycINFO, Web of Science, and CINAHL. In addition, Google Scholar and gray literature were utilized. Studies were assessed on their digital maturity using a Digital Maturity Framework on the basis of the following 4 domains: capacity/resource, usage, interoperability, and impact. A total score of 4 indicated the highest level of digital maturity.

Results

RTF was collected primarily using touchscreens, tablets, and Web-based platforms. Implementation of digital systems showed acceptable response rates and generally positive views from patients and staff. Patient demographics according to RTF responses varied. An overrepresentation existed in females with a white predominance and in patients aged ≥65 years. Of 13 eligible studies, none had digital systems that were deemed to be of the highest level of maturity. Three studies received a score of 3, 2, and 1, respectively. Four studies scored 0 points. While 7 studies demonstrated capacity/resource, 8 demonstrated impact. None of the studies demonstrated interoperability in their digital systems.

Conclusions

Patients and staff alike are willing to engage in RTF delivered using digital technology, thereby disrupting previous paper-based feedback. However, a lack of emphasis on digital maturity may lead to ineffective RTF, thwarting improvement efforts. Therefore, given the potential benefits of RTF, health care services should ensure that their digital systems deliver across the digital maturity continuum.

An evaluation of a near real-time survey for improving patients’ experiences of the relational aspects of care: a mixed-methods evaluation

Background

The Francis Report (of 2013) provided many recommendations to improve compassionate care in NHS organisations, including more widespread use of real-time feedback (RTF) to collect patient experience data. This research directly addressed these recommendations and aimed to provide an evidence-based toolkit to support NHS quality improvements.

Objectives

To develop and validate a survey of compassionate care for use in near real time on elderly care wards and accident and emergency (A&E) departments. This research also evaluated the effectiveness of the RTF approach for improving relational aspects of care and provides suggestions for how the approach can be used by other hospitals to strengthen compassionate care.

Design

The research utilised a mixed-methods design, using quantitative, qualitative and participatory research approaches to collect patients’ experiences of relational care and the views of NHS staff in an effort to evaluate the processes and impacts of near real-time feedback (NRTF) data collection. Data sources included a NRTF patient experience survey, weekly volunteer diaries, staff interviews and surveys, workshops and meetings with case study sites.

Setting

The research was carried out across six case study sites across England, in wards that predominantly serve elderly patients and in A&E departments.

Participants

The 3928 participants in the patient experience survey were inpatients on elderly care wards, or persons who had sought medical care in A&E. Frontline staff, service leads, senior management and volunteers also took part in surveys (n = 274) and interviews (n = 82) designed to understand the staff perspectives and opinions of collecting patient experience data.

Interventions

A patient experience survey was implemented using a tablet computer-based methodology, facilitated by trained volunteers. Responses were used alongside feedback from staff to evaluate the use of a NRTF approach as a method for improving patient experiences of relational aspects of care.

Main outcome measures

The patient experience survey measured relational aspects of care. Another outcome measure was improvements to care as planned, implemented and reported by staff.

Results

A small but statistically significant improvement (p = 0.044) in relational aspects of care over the course of the study was noted overall. Staff implemented a variety of improvements to enhance communication with patients.

Limitations

Maintaining volunteer and staff engagement throughout the study was difficult. Few surveys were completed per ward or department each week. This made examining trends in patient experiences over time challenging.

Conclusions

Near real-time feedback offers an effective approach for monitoring and improving relational aspects of care.

Future work

Staff frequently expressed a view that volunteers’ interactions with patients while administering the survey were themselves beneficial to patients. Future research should examine the impact of volunteer interactions with patients on their experiences of relational aspects of care.

Study registration

The project is registered on the Clinical Research Network portfolio under the primary trial identification number 18449.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

The Friends and Family Test in general practice in England: a qualitative study of the views of staff and patients

BACKGROUND

The Friends and Family Test (FFT) was introduced into general practices in England in 2015 to provide staff with information on patients' views of their experience of care.

AIM

To examine the views of practice staff and patients of the FFT, how the results are used, and to recommend improvements.

DESIGN AND SETTING

A qualitative study of a national representative sample of 42 general practices.

METHOD

Semi-structured interviews with 43 clinicians, 48 practice managers, and 27 patient representatives. Interviews were audiotaped, transcribed, and analysed thematically.

RESULTS

Although the FFT imposed little extra work on practices, it was judged to provide little additional insight over existing methods and to have had minimal impact on improving quality. Staff lacked confidence in the accuracy of the results given the lack of a representative sample and the risk of bias. The FFT question was judged to be inappropriate as in many areas there was no alternative practice for patients to choose, patients' individual needs would not be the same as those of their friends and relatives, and an overall assessment failed to identify any specific aspects of good- or poor-quality care. Despite being intended to support local quality improvement, there was widespread unease about the FFT, with many responders perceiving it as a tool for national bodies to monitor general practices.

CONCLUSION

If the use of a single-item questionnaire is to continue, changes should be made to the wording. It should be focused on stimulating local quality improvement, and practice staff should be supported to use the results effectively.

Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience

Background

There has been an increased focus towards improving quality of care within the NHS in the last 15 years; as part of this, there has been an emphasis on the importance of patient feedback within policy, through National Service Frameworks and the Quality and Outcomes Framework. The development and administration of large-scale national patient surveys to gather representative data on patient experience, such as the national GP Patient Survey in primary care, has been one such initiative. However, it remains unclear how the survey is used by patients and what impact the data may have on practice.

Objectives

Our research aimed to gain insight into how different patients use surveys to record experiences of general practice; how primary care staff respond to feedback; and how to engage primary care staff in responding to feedback.

Methods

We used methods including quantitative survey analyses, focus groups, interviews, an exploratory trial and an experimental vignette study.

Results

(1)Understanding patient experience data. Patients readily criticised their care when reviewing consultations on video, although they were reluctant to be critical when completing questionnaires. When trained raters judged communication during a consultation to be poor, a substantial proportion of patients rated the doctor as ‘good’ or ‘very good’. Absolute scores on questionnaire surveys should be treated with caution; they may present an overoptimistic view of general practitioner (GP) care. However, relative rankings to identify GPs who are better or poorer at communicating may be acceptable, as long as statistically reliable figures are obtained. Most patients have a particular GP whom they prefer to see; however, up to 40% of people who have such a preference are unable regularly to see the doctor of their choice. Users of out-of-hours care reported worse experiences when the service was run by a commercial provider than when it was run by a not-for profit or NHS provider. (2)Understanding patient experience in minority ethnic groups. Asian respondents to the GP Patient Survey tend to be registered with practices with generally low scores, explaining about half of the difference in the poorer reported experiences of South Asian patients than white British patients. We found no evidence that South Asian patients used response scales differently. When viewing the same consultation in an experimental vignette study, South Asian respondents gave higher scores than white British respondents. This suggests that the low scores given by South Asian respondents in patient experience surveys reflect care that is genuinely worse than that experienced by their white British counterparts. We also found that service users of mixed or Asian ethnicity reported lower scores than white respondents when rating out-of-hours services. (3)Using patient experience data. We found that measuring GP–patient communication at practice level masks variation between how good individual doctors are within a practice. In general practices and in out-of-hours centres, staff were sceptical about the value of patient surveys and their ability to support service reconfiguration and quality improvement. In both settings, surveys were deemed necessary but not sufficient. Staff expressed a preference for free-text comments, as these provided more tangible, actionable data. An exploratory trial of real-time feedback (RTF) found that only 2.5% of consulting patients left feedback using touch screens in the waiting room, although more did so when reminded by staff. The representativeness of responding patients remains to be evaluated. Staff were broadly positive about using RTF, and practices valued the ability to include their own questions. Staff benefited from having a facilitated session and protected time to discuss patient feedback.

Conclusions

Our findings demonstrate the importance of patient experience feedback as a means of informing NHS care, and confirm that surveys are a valuable resource for monitoring national trends in quality of care. However, surveys may be insufficient in themselves to fully capture patient feedback, and in practice GPs rarely used the results of surveys for quality improvement. The impact of patient surveys appears to be limited and effort should be invested in making the results of surveys more meaningful to practice staff. There were several limitations of this programme of research. Practice recruitment for our in-hours studies took place in two broad geographical areas, which may not be fully representative of practices nationally. Our focus was on patient experience in primary care; secondary care settings may face different challenges in implementing quality improvement initiatives driven by patient feedback. Recommendations for future research include consideration of alternative feedback methods to better support patients to identify poor care; investigation into the factors driving poorer experiences of communication in South Asian patient groups; further investigation of how best to deliver patient feedback to clinicians to engage them and to foster quality improvement; and further research to support the development and implementation of interventions aiming to improve care when deficiencies in patient experience of care are identified.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Barriers and facilitators of a near real-time feedback approach for measuring patient experiences of hospital care

Objectives

To contribute towards the current policy directive and recommendations outlined in the Francis Report (1) to strengthen relational aspects of hospital care and increase the use of a near real-time feedback (RTF) approach. This article offers insight into the challenges and enablers faced when collecting near real-time feedback of patient experiences with trained volunteers; and using the data to facilitate improvements.

Methods

Feedback was collected from staff and volunteers before, during and after a patient experience data collection. This took the form of both formal mixed methods data collections via interviews, surveys and a diary; and informal anecdotal evidence, collected from meetings, workshops, support calls and a networking event.

Results

Various challenges and enablers associated with the RTF approach were identified. These related to technology, the setting, volunteer engagement and staff engagement. This article presents the key barriers experienced followed by methods suggested and utilised by staff and volunteers in order to counteract the difficulties faced.

Conclusions

The results from this evaluation suggest that a near real-time feedback approach, when used in a hospital setting with trained volunteers, benefits from various support structures or systems to minimise the complications or burden placed on both staff and volunteers.

•

A near real-time feedback approach to collect patient experience data in hospitals using trained volunteers.

•

Empirical and anecdotal evidence collected from hospitals to understand the success of the near real-time feedback approach.

•

Feedback from volunteers and staff explores barriers and facilitators of the approach and subsequent use of the results.

•

Various support systems and structures can mitigate challenges associated with a near real-time feedback approach.

Capturing patient experience: a qualitative study of implementing real-time feedback in primary care

BACKGROUND

In recent years, hospitals have made use of new technologies, such as real-time feedback, to collect patient experience information. This approach is currently rarely used in primary care settings, but may provide practices with a useful tool that enables them to take prompt, focused action to improve their services.

AIM

To identify the factors inhibiting and enabling the implementation of real-time feedback in general practices.

DESIGN AND SETTING

Qualitative study embedded within an exploratory trial (July 2014 to February 2015) of a real-time feedback intervention targeting patient experience in general practices in south-west England and Cambridgeshire.

METHOD

Semi-structured interviews (n = 22) and focus groups (n = 4, total of 28 attendees) with practice staff were audiorecorded, transcribed, and analysed thematically, using a framework based on constructs from normalisation process theory.

RESULTS

Staff engagement with real-time feedback varied considerably, and staff made sense of real-time feedback by comparing it with more familiar feedback modalities. Effective within-team communication was associated with positive attitudes towards real-time feedback. Timing of requests for feedback was important in relation to patient engagement. Real-time feedback may offer potential as a means of informing practice development, perhaps as a component of a wider programme of capturing and responding to patients' comments.

CONCLUSION

Successful implementation of real-time feedback requires effective communication across the practice team to engender thorough engagement. Feedback processes should be carefully introduced to fit with existing patient and practice routines. Future studies should consider making real-time feedback content relevant to specific practice needs, and support participation by all patient groups.