Ethical considerations for mystery shopper studies of pharmaceutical sales

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Secret shopper studies: an unorthodox design that measures inequities in healthcare access

Secret shopper studies are particularly potent study designs that allow for the gathering of objective data for a variety of research hypotheses, including but not limited to, healthcare delivery, equity of healthcare, and potential barriers to care. Of particular interest during the COVID-19 pandemic, secret shopper study designs allow for the gathering of data over the phone. However, there is a dearth of literature available on appropriate methodological practices for these types of studies. To make these study designs more widely accessible, here we outline the case for using the secret shopper methodology and detail best practices for designing and implementing them.

Improving Data Surveillance Resilience Beyond COVID-19: Experiences of Primary heAlth Care quAlity Cohort In ChinA (ACACIA) Using Unannounced Standardized Patients

We analyzed COVID-19 influences on the design, implementation, and validity of assessing the quality of primary health care using unannounced standardized patients (USPs) in China. Because of the pandemic, we crowdsourced our funding, removed tuberculosis from the USP case roster, adjusted common cold and asthma cases, used hybrid online-offline training for USPs, shared USPs across provinces, and strengthened ethical considerations. With those changes, we were able to conduct fieldwork despite frequent COVID-19 interruptions. Furthermore, the USP assessment tool maintained high validity in the quality checklist (criteria), USP role fidelity, checklist completion, and physician detection of USPs. Our experiences suggest that the pandemic created not only barriers but also opportunities to innovate ways to build a resilient data collection system. To build data system reliance, we recommend harnessing the power of technology for a hybrid model of remote and in-person work, learning from the sharing economy to pool strengths and optimize resources, and dedicating individual and group leadership to problem-solving and results. (Am J Public Health. 2022;112(6):913-922. https://doi.org/10.2105/AJPH.2022.306779).

The simulated patient method: Design and application in health services research

The simulated patient method is becoming increasingly popular in health services research to observe the behaviour of healthcare practitioners in a naturalistic setting. This method involves sending a trained individual (simulated patient among other names), who is indistinguishable from a regular consumer, into a healthcare setting with a standardised scripted request. This paper provides an overview of the method, a brief history of its use in health services research, comparisons with other methods, ethical considerations, and considerations for the development of studies using the simulated patient method in health services research, with examples from pharmacy and other fields. Methods of analysis, mixed-methods, and the use of simulated patients with feedback are also discussed.