1
|
Hegde S, McKee S, Cole D, Wainer Z. Experiences and learnings from developing and implementing a co-designed value-based healthcare framework within Victorian public oral health sector. AUST HEALTH REV 2024; 48:134-141. [PMID: 38537306 DOI: 10.1071/ah24017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 02/29/2024] [Indexed: 04/05/2024]
Abstract
Objective This study aimed to describe the development and implementation of a co-designed value-based healthcare (VBHC) framework within the public dental sector in Victoria. Methods A mixed-method study was employed. Explorative qualitative design was used to examine patient, workforce and stakeholder perspectives of implementing VBHC. Participatory action research was used to bring together qualitative narrative-based research and service design methods. An experience-based co-design approach was used to enable staff and patients to co-design services. Quantitative data was sourced from Titanium (online patient management system). Results Building a case for VBHC implementation required intensive work. It included co-designing, collaborating, planning and designing services based on patient needs. Evidence reviews, value-stream mapping and development of patient reported outcomes (PROMs) and patient reported experience measures (PREMs) were fundamental to VBHC implementation. Following VBHC implementation, a 44% lower failure to attend rate and 60% increase in preventive interventions was reported. A higher proportion of clinicians worked across their top scope of practice within a multi-disciplinary team. Approximately 80% of services previously provided by dentists were shifted to oral health therapists and dental assistants, thereby releasing the capacity of dentists to undertake complex treatments. Patients completed baseline International Consortium for Health Outcomes Measurement PROMs (n = 44,408), which have been used for social/clinical triaging, determining urgency of care based on risk, segmentation and tracking health outcomes. Following their care, patients completed a PREMs questionnaire (n = 15,402). Patients agreed or strongly agreed that: the care they received met their needs (87%); they received clear answers to their questions (93%); they left their visit knowing what is next (91%); they felt taken care of during their visit (94%); and they felt involved in their treatment and care (94%). Conclusion The potential for health system transformation through implementation of VBHC is significant, however, its implementation needs to extend beyond organisational approaches and focus on sustaining the principles of VBHC across healthcare systems, policy and practice.
Collapse
Affiliation(s)
- Shalika Hegde
- Dental Health Services Victoria, Melbourne, Vic., Australia
| | - Susan McKee
- Dental Health Services Victoria, Melbourne, Vic., Australia
| | | | - Zoe Wainer
- Department of Health Victoria, Melbourne, Vic., Australia
| |
Collapse
|
2
|
Gilfoyle M, Melro C, Koskinas E, Salsberg J. Recruitment of patients, carers and members of the public to advisory boards, groups and panels in public and patient involved health research: a scoping review. BMJ Open 2023; 13:e072918. [PMID: 37832980 PMCID: PMC10582988 DOI: 10.1136/bmjopen-2023-072918] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Accepted: 09/28/2023] [Indexed: 10/15/2023] Open
Abstract
OBJECTIVES The objectives of this scoping review are to: (1) identify the distribution of and context of the recruitment strategies used, (2) explore the facilitators, benefits, barriers and ethical issues of the identified recruitment strategies, (3) distinguish the varying terminology for involvement (ie, panels, boards, individual) and (4) determine if the individual recruitment strategies used were to address issues of representation or bias. DESIGN A scoping review. SETTING This scoping review follows the framework by Peters et al. Seven electronic databases were explored including Scopus, Medline, PubMed, Web of Science, CINAHL, Cochrane Library and PsycINFO (conducted July 2021). The search strategy was codeveloped among the research team, PPI research experts and a faculty librarian. Two independent reviewers screened articles by title and abstract and then at full text based on predetermined criteria. PRIMARY AND SECONDARY OUTCOME MEASURES Explore recruitment strategies used, facilitators, benefits, barriers and ethical issues of the identified recruitment strategies. Identify terminology for involvement. Explore recruitment strategies used to address issues of representation or bias. RESULTS The final sample was from 51 sources. A large portion of the extracted empirical literature had a clinical focus (37%, n=13) but was not a randomised control trial. The most common recruitment strategies used were human networks (78%, n=40), such as word of mouth, foundation affiliation, existing networks, clinics or personal contacts. Within the reviewed literature, there was a lack of discussion pertaining to facilitators, benefits, barriers and ethical considerations of recruitment strategies was apparent. Finally, 41% (n=21) of studies employed or proposed recruitment strategies or considerations to address issues of representation or bias. CONCLUSION We conclude with four key recommendations that researchers can use to better understand appropriate routes to meaningfully involve patients, carers and members of the public to cocreate the evidence informing their care.
Collapse
Affiliation(s)
- Meghan Gilfoyle
- School of Medicine, University of Limerick, Limerick, Ireland
- McMaster University, Hamilton, Ontario, Canada
| | | | - Elena Koskinas
- School of Medicine, University of Limerick, Limerick, Ireland
| | - Jon Salsberg
- School of Medicine, University of Limerick, Limerick, Ireland
| |
Collapse
|
3
|
Merner B, Schonfeld L, Virgona A, Lowe D, Walsh L, Wardrope C, Graham-Wisener L, Xafis V, Colombo C, Refahi N, Bryden P, Chmielewski R, Martin F, Messino NM, Mussared A, Smith L, Biggar S, Gill M, Menzies D, Gaulden CM, Earnshaw L, Arnott L, Poole N, Ryan RE, Hill S. Consumers' and health providers' views and perceptions of partnering to improve health services design, delivery and evaluation: a co-produced qualitative evidence synthesis. Cochrane Database Syst Rev 2023; 3:CD013274. [PMID: 36917094 PMCID: PMC10065807 DOI: 10.1002/14651858.cd013274.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/16/2023]
Abstract
BACKGROUND Partnering with consumers in the planning, delivery and evaluation of health services is an essential component of person-centred care. There are many ways to partner with consumers to improve health services, including formal group partnerships (such as committees, boards or steering groups). However, consumers' and health providers' views and experiences of formal group partnerships remain unclear. In this qualitative evidence synthesis (QES), we focus specifically on formal group partnerships where health providers and consumers share decision-making about planning, delivering and/or evaluating health services. Formal group partnerships were selected because they are widely used throughout the world to improve person-centred care. For the purposes of this QES, the term 'consumer' refers to a person who is a patient, carer or community member who brings their perspective to health service partnerships. 'Health provider' refers to a person with a health policy, management, administrative or clinical role who participates in formal partnerships in an advisory or representative capacity. This QES was co-produced with a Stakeholder Panel of consumers and health providers. The QES was undertaken concurrently with a Cochrane intervention review entitled Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation. OBJECTIVES 1. To synthesise the views and experiences of consumers and health providers of formal partnership approaches that aimed to improve planning, delivery or evaluation of health services. 2. To identify best practice principles for formal partnership approaches in health services by understanding consumers' and health providers' views and experiences. SEARCH METHODS We searched MEDLINE, Embase, PsycINFO and CINAHL for studies published between January 2000 and October 2018. We also searched grey literature sources including websites of relevant research and policy organisations involved in promoting person-centred care. SELECTION CRITERIA We included qualitative studies that explored consumers' and health providers' perceptions and experiences of partnering in formal group formats to improve the planning, delivery or evaluation of health services. DATA COLLECTION AND ANALYSIS Following completion of abstract and full-text screening, we used purposive sampling to select a sample of eligible studies that covered a range of pre-defined criteria, including rich data, range of countries and country income level, settings, participants, and types of partnership activities. A Framework Synthesis approach was used to synthesise the findings of the sample. We appraised the quality of each study using the CASP (Critical Appraisal Skill Program) tool. We assessed our confidence in the findings using the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach. The Stakeholder Panel was involved in each stage of the review from development of the protocol to development of the best practice principles. MAIN RESULTS We found 182 studies that were eligible for inclusion. From this group, we selected 33 studies to include in the final synthesis. These studies came from a wide range of countries including 28 from high-income countries and five from low- or middle-income countries (LMICs). Each of the studies included the experiences and views of consumers and/or health providers of partnering in formal group formats. The results were divided into the following categories. Contextual factors influencing partnerships: government policy, policy implementation processes and funding, as well as the organisational context of the health service, could facilitate or impede partnering (moderate level of confidence). Consumer recruitment: consumer recruitment occurred in different ways and consumers managed the recruitment process in a minority of studies only (high level of confidence). Recruiting a range of consumers who were reflective of the clinic's demographic population was considered desirable, particularly by health providers (high level of confidence). Some health providers perceived that individual consumers' experiences were not generalisable to the broader population whereas consumers perceived it could be problematic to aim to represent a broad range of community views (high level of confidence). Partnership dynamics and processes: positive interpersonal dynamics between health providers and consumers facilitated partnerships (high level of confidence). However, formal meeting formats and lack of clarity about the consumer role could constrain consumers' involvement (high level of confidence). Health providers' professional status, technical knowledge and use of jargon were intimidating for some consumers (high level of confidence) and consumers could feel their experiential knowledge was not valued (moderate level of confidence). Consumers could also become frustrated when health providers dominated the meeting agenda (moderate level of confidence) and when they experienced token involvement, such as a lack of decision-making power (high level of confidence) Perceived impacts on partnership participants: partnering could affect health provider and consumer participants in both positive and negative ways (high level of confidence). Perceived impacts on health service planning, delivery and evaluation: partnering was perceived to improve the person-centredness of health service culture (high level of confidence), improve the built environment of the health service (high level of confidence), improve health service design and delivery e.g. facilitate 'out of hours' services or treatment closer to home (high level of confidence), enhance community ownership of health services, particularly in LMICs (moderate level of confidence), and improve consumer involvement in strategic decision-making, under certain conditions (moderate level of confidence). There was limited evidence suggesting partnering may improve health service evaluation (very low level of confidence). Best practice principles for formal partnering to promote person-centred care were developed from these findings. The principles were developed collaboratively with the Stakeholder Panel and included leadership and health service culture; diversity; equity; mutual respect; shared vision and regular communication; shared agendas and decision-making; influence and sustainability. AUTHORS' CONCLUSIONS Successful formal group partnerships with consumers require health providers to continually reflect and address power imbalances that may constrain consumers' participation. Such imbalances may be particularly acute in recruitment procedures, meeting structure and content and decision-making processes. Formal group partnerships were perceived to improve the physical environment of health services, the person-centredness of health service culture and health service design and delivery. Implementing the best practice principles may help to address power imbalances, strengthen formal partnering, improve the experiences of consumers and health providers and positively affect partnership outcomes.
Collapse
Affiliation(s)
- Bronwen Merner
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| | - Lina Schonfeld
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| | - Ariane Virgona
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| | - Dianne Lowe
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
- Child and Family Evidence, Australian Institute of Family Studies, Melbourne, Australia
| | - Louisa Walsh
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| | - Cheryl Wardrope
- Clinical Governance, Metro South Hospital and Health Service, Eight Mile Plains, Australia
| | | | - Vicki Xafis
- The Sydney Children's Hospitals Network, Sydney, Australia
| | - Cinzia Colombo
- Laboratory for medical research and consumer involvement, Department of Public Health, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milano, Italy
| | - Nora Refahi
- Consumer Representative, Melbourne, Australia
| | - Paul Bryden
- Consumer Representative, Caboolture, Australia
| | - Renee Chmielewski
- Planning and Patient Experience, The Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
| | | | | | | | - Lorraine Smith
- School of Pharmacy, Faculty of Medicine and Health, University of Sydney, Camperdown, Australia
| | - Susan Biggar
- Consumer Representative, Melbourne, Australia
- Australian Health Practitioner Regulation Agency (AHPRA), Melbourne, Australia
| | - Marie Gill
- Gill and Wilcox Consultancy, Melbourne, Australia
| | - David Menzies
- Chronic Disease Programs, South Eastern Melbourne Primary Health Network, Heatherton, Australia
| | - Carolyn M Gaulden
- Detroit Wayne County Authority Health Residency Program, Michigan State University, Providence Hospital, Southfield, Michigan, USA
| | | | | | - Naomi Poole
- Strategy and Innovation, Australian Commission on Safety and Quality in Health Care, Sydney, Australia
| | - Rebecca E Ryan
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| | - Sophie Hill
- Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Bundoora, Australia
| |
Collapse
|
4
|
Saut AM, Ho LL, Berger S, Berssaneti FT. How did healthcare professionals define patient engagement in quality management? A survey study. BMC Health Serv Res 2023; 23:173. [PMID: 36803492 PMCID: PMC9942306 DOI: 10.1186/s12913-023-09098-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Accepted: 01/24/2023] [Indexed: 02/22/2023] Open
Abstract
BACKGROUND Patient and family engagement (PFE) can positively impact the patient experience and care process outcomes. There is no unique type of PFE, and the process is usually defined by the quality management department or professionals responsible for this process in the hospital. The objective of this study is to define PFE in quality management based on the professional's perspective. METHOD A survey was carried out with 90 professionals from Brazilian hospitals. There were two questions aimed at understanding the concept. The first was a multiple-choice question to identify synonyms. The second was an open-ended question to develop the definition. A content analysis methodology was employed by applying techniques for thematic and inferential analysis. RESULTS Three words were classified as synonyms by more than 60% of respondents: involvement, participation, and centered care. The participants described patient participation at both the individual (related to the treatment) and organizational levels (related to quality improvement). The PFE in the treatment is related to the development, discussion and decision-making about the therapeutic plan, participation in each step of care, and knowledge of the institution's quality and safety processes. At the organizational level, engagement in quality improvement includes the involvement of the P/F in all processes of the institution, from strategic planning to the design or improvement processes, as well as active participation in institutional committees or commissions. CONCLUSION The professionals defined engagement in two levels (individual and organizational) and the results suggest that their point of view can influence the practice in the hospitals. Professionals of hospitals that implemented mechanisms of consult defined PFE more in the individual level. On the other hand, professionals of hospitals that implemented mechanisms of involvement considered PFE more focus in the organizational level.
Collapse
Affiliation(s)
- Ana Maria Saut
- Polytechnic School - Production Engineering Department, University of São Paulo (USP), Av. Prof. Luciano Gualberto, 1380, São Paulo, 05508-010, Brazil. .,Engineering School - Production Engineering Coordination, Mackenzie Presbyterian University, São Paulo, Brazil.
| | - Linda Lee Ho
- grid.11899.380000 0004 1937 0722Polytechnic School - Production Engineering Department, University of São Paulo (USP), Av. Prof. Luciano Gualberto, 1380, São Paulo, 05508-010 Brazil
| | - Simone Berger
- grid.11899.380000 0004 1937 0722Polytechnic School - Production Engineering Department, University of São Paulo (USP), Av. Prof. Luciano Gualberto, 1380, São Paulo, 05508-010 Brazil
| | - Fernando Tobal Berssaneti
- grid.11899.380000 0004 1937 0722Polytechnic School - Production Engineering Department, University of São Paulo (USP), Av. Prof. Luciano Gualberto, 1380, São Paulo, 05508-010 Brazil
| |
Collapse
|
5
|
Saut AM, Berssaneti FT, Ho LL, Berger S. How do hospitals engage patients and family members in quality management? A grounded theory study of hospitals in Brazil. BMJ Open 2022; 12:e055926. [PMID: 35985775 PMCID: PMC9396118 DOI: 10.1136/bmjopen-2021-055926] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND Patient and family engagement (PFE) is considered an essential element of the transformation of the healthcare system. However, it is characterised by its complexity and a small number of institutions that have implemented the mechanisms of engagement. OBJECTIVE To understand PFE in quality management (QM) in the hospital environment. DESIGN A qualitative approach was guided by the grounded theory based in Straussian perspective. Data were gathered using semistructured interviews. The coding was performed by excerpts, using an inductive approach and the constant comparison technique. SETTING AND PARTICIPANTS A total of seven Brazilian hospitals were selected based on the theoretical sampling technique. RESULTS A total of five categories emerged, namely: patient partner, mechanisms of engagement, internal structure for engagement, maturity of the QM system and openness to change. Externally, three contextual factors can impact the engagement: the local health system, the profile of the community and the change in access to the information. At the centre of the change is the balance in power relations between patients and professionals, the sharing of information from the hospital and a proactive attitude towards improving services. CONCLUSIONS The PFE involves a cultural and process change. Cultural change is represented by 'openness', that is, openness to learn, to listen and to consider new perspectives. The change in processes is in turn characterised by the phrase 'test and venture' because the model to be adopted may be different between hospitals. The patient's perspective allows actions to be driven towards what really matters to them, ensuring quality of service and safety, obtaining a new perspective to understand and solve problems, and stimulating a sense of urgency, more empathy and compassion in professionals.
Collapse
Affiliation(s)
- Ana Maria Saut
- Polytechnic School, Production Engineering Department, University of Sao Paulo, Sao Paulo, Brazil
| | | | - Linda Lee Ho
- Polytechnic School, Production Engineering Department, University of Sao Paulo, Sao Paulo, Brazil
| | - Simone Berger
- Polytechnic School, Production Engineering Department, University of Sao Paulo, Sao Paulo, Brazil
| |
Collapse
|
6
|
Lee YJ, Jang H, Campbell G, Carenini G, Thomas T, Donovan H. Identifying Language Features Associated With Needs of Ovarian Cancer Patients and Caregivers Using Social Media. Cancer Nurs 2022; 45:E639-E645. [PMID: 33577203 DOI: 10.1097/ncc.0000000000000928] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
BACKGROUND Online health communities (OHCs) can be a source for clinicians to learn the needs of cancer patients and caregivers. Ovarian cancer (OvCa) patients and caregivers deal with a wide range of unmet needs, many of which are expressed in OHCs. An automated need classification model could help clinicians more easily understand and prioritize information available in the OHCs. OBJECTIVE The aim of this study was to use initial OHC postings to develop an automated model for the classification of OvCa patient and caregiver needs. METHODS We collected data from the OvCa OHC and analyzed the initial postings of patients and caregivers (n = 853). Two annotators coded each posting with 12 types of needs. Then, we applied the machine learning approach with bag-of-words features to build a model to classify needs. F1 score, an indicator of model accuracy, was used to evaluate the model. RESULTS The most reported needs were information, social, psychological/emotional, and physical. Thirty-nine percent of postings described information and social needs in the same posting. Our model reported a high level of accuracy for classifying those top needs. Psychological terms were important for classifying psychological/emotional and social needs. Medical terms were important for physical and information needs. CONCLUSIONS We demonstrate the potential of using OHCs to supplement traditional needs assessment. Further research would incorporate additional information (eg, trajectory, stage) for more sophisticated models. IMPLICATIONS FOR PRACTICE This study shows the potential of automated classification to leverage OHCs for needs assessment. Our approach can be applied to different types of cancer and enhanced by using domain-specific information.
Collapse
Affiliation(s)
- Young Ji Lee
- Author Affiliations: School of Nursing (Drs Lee, Campbell, Thomas, and Donovan) and School of Medicine (Drs Lee and Donovan), University of Pittsburgh, Pennsylvania; Department of Computer Science, University of British Columbia (Drs Jang and Carenini), Vancouver, Canada; and School of Health and Rehabilitation Sciences, University of Pittsburgh (Dr Campbell), Pennsylvania
| | | | | | | | | | | |
Collapse
|
7
|
Bergerum C, Wolmesjö M, Thor J. Organising and managing patient and public involvement to enhance quality improvement – comparing a Swedish and a Dutch hospital. Health Policy 2022; 126:603-612. [DOI: 10.1016/j.healthpol.2022.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 03/07/2022] [Accepted: 04/06/2022] [Indexed: 11/26/2022]
|
8
|
Hamilton CB, Dehnadi M, Snow ME, Clark N, Lui M, McLean J, Mamdani H, Kooijman AL, Bubber V, Hoefer T, Li LC. Themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making in healthcare systems: a scoping review. BMJ Open 2021; 11:e050208. [PMID: 34635521 PMCID: PMC8506891 DOI: 10.1136/bmjopen-2021-050208] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/09/2023] Open
Abstract
OBJECTIVE To identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making across the organisation and system domains of healthcare systems. METHODS We conducted a scoping review. Seven databases of journal articles were searched from their inception to June 2019. Eligible articles were literature reviews published in English and provided useful information for determining aspects of engaging patients and family caregivers in decision-making to evaluate. We extracted text under three predetermined categories: structure, process and outcomes that were adapted from the Donabedian conceptual framework. These excerpts were then independently open-coded among four researchers. The subsequent themes and their corresponding excerpts were summarised to provide a rich description of each theme. RESULTS Of 7747 unique articles identified, 366 were potentially relevant, from which we selected the 42 literature reviews. 18 unique themes were identified across the three predetermined categories. There were six structure themes: engagement plan, level of engagement, time and timing of engagement, format and composition, commitment to support and environment. There were four process themes: objectives, engagement approach, communication and engagement activities. There were eight outcome themes: decision-making process, stakeholder relationship, capacity development, stakeholder experience, shape policy/service/programme, health status, healthcare quality, and cost-effectiveness. CONCLUSIONS The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness.
Collapse
Affiliation(s)
- Clayon B Hamilton
- Evaluation and Research Services, Fraser Health Authority, Surrey, British Columbia, Canada
- Primary Care Division, Ministry of Health, Victoria, British Columbia, Canada
- Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada
- Arthritis Research Canada, Vancouver, British Columbia, Canada
- Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada
| | - Maryam Dehnadi
- Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada
- Arthritis Research Canada, Vancouver, British Columbia, Canada
| | - M Elizabeth Snow
- Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada
- Centre for Health Evaluation and Outcomes Sciences, Vancouver, British Columbia, Canada
| | - Nancy Clark
- Faculty of Human and Social Development, University of Victoria, Victoria, British Columbia, Canada
| | - Michelle Lui
- Arthritis Research Canada, Vancouver, British Columbia, Canada
| | - Janet McLean
- Family Caregivers of British Columbia, Victoria, British Columbia, Canada
| | - Hussein Mamdani
- Arthritis Research Canada, Vancouver, British Columbia, Canada
| | - Allison L Kooijman
- Patient Voices Network, BC Patient Safety and Quality Council, Vancouver, British Columbia, Canada
- School of Interdisciplinary Studies, University of British Columbia, Vancouver, British Columbia, Canada
| | - Vikram Bubber
- Patient Voices Network, BC Patient Safety and Quality Council, Vancouver, British Columbia, Canada
| | - Tammy Hoefer
- BC Patient Safety and Quality Council, Vancouver, British Columbia, Canada
| | - Linda C Li
- Arthritis Research Canada, Vancouver, British Columbia, Canada
- Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
9
|
Lloyd N, Kenny A, Hyett N. Evaluating health service outcomes of public involvement in health service design in high-income countries: a systematic review. BMC Health Serv Res 2021; 21:364. [PMID: 33879149 PMCID: PMC8056601 DOI: 10.1186/s12913-021-06319-1] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Accepted: 03/26/2021] [Indexed: 12/11/2022] Open
Abstract
BACKGROUND Internationally, it is expected that health services will involve the public in health service design. Evaluation of public involvement has typically focused on the process and experiences for participants. Less is known about outcomes for health services. The aim of this systematic review was to a) identify and synthesise what is known about health service outcomes of public involvement and b) document how outcomes were evaluated. METHODS Searches were undertaken in MEDLINE, EMBASE, The Cochrane Library, PsycINFO, Web of Science, and CINAHL for studies that reported health service outcomes from public involvement in health service design. The review was limited to high-income countries and studies in English. Study quality was assessed using the Mixed Methods Appraisal Tool and critical appraisal guidelines for assessing the quality and impact of user involvement in health research. Content analysis was used to determine the outcomes of public involvement in health service design and how outcomes were evaluated. RESULTS A total of 93 articles were included. The majority were published in the last 5 years, were qualitative, and were located in the United Kingdom. A range of health service outcomes (discrete products, improvements to health services and system/policy level changes) were reported at various levels (service level, across services, and across organisations). However, evaluations of outcomes were reported in less than half of studies. In studies where outcomes were evaluated, a range of methods were used; most frequent were mixed methods. The quality of study design and reporting was inconsistent. CONCLUSION When reporting public involvement in health service design authors outline a range of outcomes for health services, but it is challenging to determine the extent of outcomes due to inadequate descriptions of study design and poor reporting. There is an urgent need for evaluations, including longitudinal study designs and cost-benefit analyses, to fully understand outcomes from public involvement in health service design.
Collapse
Affiliation(s)
- Nicola Lloyd
- Violet Vines Marshman Centre for Rural Health Research, La Trobe Rural Health School, La Trobe University, Bendigo, Australia
| | - Amanda Kenny
- Violet Vines Marshman Centre for Rural Health Research, La Trobe Rural Health School, La Trobe University, Bendigo, Australia
| | - Nerida Hyett
- Violet Vines Marshman Centre for Rural Health Research, La Trobe Rural Health School, La Trobe University, Bendigo, Australia
| |
Collapse
|
10
|
Novilla LK, Broadbent E, Glade R, Crandall A. Supporting and Engaging Families: An Examination of Publicly-Funded Health Promotion Programs in the Intermountain West, USA. Front Public Health 2020; 8:573003. [PMID: 33178662 PMCID: PMC7593609 DOI: 10.3389/fpubh.2020.573003] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 08/28/2020] [Indexed: 11/24/2022] Open
Abstract
Public health programming efforts have traditionally focused on either an individualistic or population approach, neglecting the family as a setting for or partner in health promotion efforts. Due to the multi-faceted influence of families on individual health, family-focused, and family-friendly public health interventions are important to making lasting changes for individual and community health. The purpose of this study was to examine the degree to which health promotion programs in a state in the US Intermountain West involve and support families across four family impact principles: family engagement, family stability, family responsibility, and family diversity. A survey was completed by 67 health promotion administrators and practitioners from 12 out of 13 county health departments with additional responses from public health practitioners at the State Health Department. The results of the survey indicated that health promotion efforts were best at supporting family responsibility and a diverse group of families but were weaker in family engagement and family stability. Applying a more family-centered and family-focused approach to health promotion efforts can be achieved by employing interdisciplinary efforts and by taking advantage of tools like the Public Health Family Impact Checklist to intentionally engage and support families in programs and interventions.
Collapse
Affiliation(s)
| | - Eliza Broadbent
- Department of Public Health, Brigham Young University, Provo, UT, United States
| | - Rozalyn Glade
- Department of Public Health, Brigham Young University, Provo, UT, United States
| | - AliceAnn Crandall
- Department of Public Health, Brigham Young University, Provo, UT, United States
| |
Collapse
|
11
|
Sibbald SL, Ziegler BR, Maskell R, Schouten K. Implementation of interprofessional team-based care: A cross-case analysis. J Interprof Care 2020; 35:654-661. [PMID: 32835539 DOI: 10.1080/13561820.2020.1803228] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Two out of five Canadians have at least one chronic disease and four out of five are at risk of developing a chronic disease. Successful disease management relies on interprofessional team-based approaches, yet lack of purposeful cultivation and patient engagement has led to systematic inefficiencies. Two primary care teams in Southwestern Ontario implementing interprofessional chronic care programs for patients with chronic obstructive pulmonary disease were compared. A mixed-methods cross-case analysis was conducted including interviews, focus groups, observations and document analysis. Cases (n = 2) were chosen based on intrinsic and unique value. Participants (n = 46) were sampled using a combination of purposive and multi-level sampling. Data was analyzed using an iterative process; inductive coding was used to gain a sense of context followed by a deductive cross-case analysis to compare and contrast themes across sites. Kompier's five-step framework was used to assess factors contributing to successful implementation and to provide insight into interactions between teams, providers and patients. Both cases satisfied all five factors (systemic and gradual approach, identification of risk factors, theory-driven, participatory approach and sustained committed support). However, one case was more successful at fully implementing their model, attributed to a flexible implementation, plans to mitigate risks, theory use, a supportive team and continued buy-in from leadership. By better understanding key facilitators and barriers, we can support the implementation of chronic disease management programs, foster sustainability of high-performing interprofessional teams, and engage patients in the development and maintenance of team-based chronic disease management.
Collapse
Affiliation(s)
- Shannon L Sibbald
- School of Health Studies, Western University, London, ON, Canada.,Department of Family Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.,The Schulich Interfaculty Program in Public Health, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.,Lawson Health Research Institute, London, ON, Canada
| | - Bianca R Ziegler
- Environment Health and Hazards Laboratory, Department of Geography, Western University, London, ON, Canada
| | - Rachelle Maskell
- School of Health Studies, Western University, London, ON, Canada
| | - Karen Schouten
- School of Health Studies, Western University, London, ON, Canada
| |
Collapse
|
12
|
Ramazani S, Bayer ND, Gottfried JA, Wagner J, Leonard MS, Lynn J, Schriefer J. The Value of Family Advisors as Coleaders in Pediatric Quality Improvement Efforts: A Qualitative Theme Analysis. J Patient Exp 2020; 7:1708-1714. [PMID: 33457634 PMCID: PMC7786654 DOI: 10.1177/2374373520939827] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Engaging family advisors in pediatric quality improvement (QI) efforts is well-studied in intensive care but less understood in other settings. The purpose of this study was to assess the perceived impact of including a family advisor as a colead on a QI initiative that successfully improved the family-centered timing of routine morning blood tests performed on pediatric inpatients. Five structured written reflections from core QI team members were analyzed using inductive thematic analysis and 3 major themes were identified. The first found that a family advisor's presence from the beginning of a QI initiative helps inform project design. The second determined that family partners working with residents fostered a better shared understanding of the role of trainees and caregivers in improving the quality of care. The third found that a family partner is an effective change agent to enact practice improvement, support professional development, and enhance resident education. Our qualitative analysis showed that engaging a family advisor as a colead influenced the design, implementation, and post-intervention impact of the initiative and improved family-centered outcomes.
Collapse
Affiliation(s)
- Suzanne Ramazani
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| | - Nathaniel D Bayer
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| | | | - Jenna Wagner
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| | - Michael S Leonard
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| | - Justin Lynn
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| | - Jan Schriefer
- Golisano Children's Hospital, University of Rochester Medical Center, Rochester, NY, USA
| |
Collapse
|
13
|
Khayatzadeh-Mahani A, Wittevrongel K, Petermann L, Graham ID, Zwicker JD. Stakeholders' engagement in co-producing policy-relevant knowledge to facilitate employment for persons with developmental disabilities. Health Res Policy Syst 2020; 18:39. [PMID: 32303228 PMCID: PMC7164207 DOI: 10.1186/s12961-020-00548-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2019] [Accepted: 03/17/2020] [Indexed: 11/24/2022] Open
Abstract
Background Persons with developmental disabilities (PWDD) face a number of individual, environmental and societal barriers when seeking employment. Integrated knowledge translation (IKT) involves ongoing and dynamic interactions between researchers and stakeholders for the purpose of engaging in mutually beneficial research to address these types of multi-faceted barriers. There is a knowledge gap in the IKT literature on effective stakeholder engagement strategies outside of the dissemination stage to inform policy. In this paper, we report on a number of engagement strategies employed over a 2-year period to engage a wide range of stakeholders in different stages of an IKT project that aimed to investigate the ‘wicked’ problem of employment for PWDD. Method Our engagement plan included multiple linked strategies and was designed to ensure the meaningful engagement of, and knowledge co-production with, stakeholders. We held two participatory consensus-building stakeholder policy dialogue events to co-produce knowledge utilising the nominal group technique and the modified Delphi technique. A total of 31 and 49 stakeholders engaged in the first and second events, respectively, from six key stakeholder groups. Focused engagement strategies were employed to build on the stakeholder dialogues for knowledge mobilisation and included a focus group attended only by PWDD, a stakeholder workshop attended only by policy/decision-makers, a webinar attended by human resources professionals and employers, and a current affairs panel attended by the general public. Results Our findings suggest that the level of engagement for each stakeholder group varies depending on the goal and need of the project. Our stakeholder dialogue findings highlight the inherent challenges in co-framing and knowledge co-production through the meaningful engagement of multiple stakeholders who hold different ideas and interests. Focused outreach is needed to foster relationships and trust for meaningful engagement. Conclusions In addition to providing guidance on how to implement adaptable meaningful engagement strategies, these findings contribute to discussions on how IKT projects are planned and funded. More studies to explore effective mechanisms for engaging a wide range of stakeholders in IKT research are needed. More evidence of successful engagement strategies employed by researchers to achieve meaningful knowledge co-production is also key to advancing the discipline.
Collapse
Affiliation(s)
- Akram Khayatzadeh-Mahani
- School of Public Policy, University of Calgary, Downtown Campus, 906 8th Avenue S.W., 5th Floor, Calgary, Alberta, T2P 1H9, Canada.,Health Services Management Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Krystle Wittevrongel
- School of Public Policy, University of Calgary, Downtown Campus, 906 8th Avenue S.W., 5th Floor, Calgary, Alberta, T2P 1H9, Canada
| | - Lisa Petermann
- Libin Cardiovascular Institute of Alberta, Calgary, Canada
| | - Ian D Graham
- Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.,School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Jennifer D Zwicker
- School of Public Policy, University of Calgary, Downtown Campus, 906 8th Avenue S.W., 5th Floor, Calgary, Alberta, T2P 1H9, Canada.
| |
Collapse
|
14
|
Jabalera M, Pons M, Gómez E, Del Castillo M. [Towards excellence in hospital management. A description of strategical management model]. J Healthc Qual Res 2019; 34:148-153. [PMID: 31164322 DOI: 10.1016/j.jhqr.2019.02.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2018] [Revised: 02/08/2019] [Accepted: 02/18/2019] [Indexed: 11/26/2022]
Abstract
BACKGROUND AND OBJECTIVE Health services are evolving from the traditional view of paternalistic medicine towards a more informed and participatory patient-provider relationship. This situation includes the empowerment and participation of patients and their families in health management, including their joint responsibility in the development of health services. The objective of this article is to describe the experience in a large teaching hospital in Spain on the application of a model that focuses on a three-fold approach to healthcare management that includes the experience of the patient. MATERIAL AND METHODS This approach is identified by the initials EMC2 transforming the E of excellence into 3main areas of improvement: methodology, clinical knowledge and client, or patient, experience. the model has been implemented in different areas: childbirth, asthma care, outpatient clinics, and operating theatre. RESULTS In the case of childbirth, there was a reduction in caesarean sections, episiotomies, induction of delivery, mean hospital stay, and number of deliveries with instrumentation. The Net Promoter Score went from 82 to 86. There was also an improvement in the areas of patient experience, hospital procedures, and clinical knowledge. Qualitative improvement results were obtained in other areas such as, asthma management, outpatient visits, and the operating room. CONCLUSIONS This model can be applied to different areas. It promotes patient-focused practice, as well as system sustainability, efficiency, effectiveness, and level of satisfaction with the health care process.
Collapse
Affiliation(s)
- M Jabalera
- Hospital Sant Joan de Déu, Esplugues de Llobregat (Barcelona), España.
| | - M Pons
- Hospital Sant Joan de Déu, Esplugues de Llobregat (Barcelona), España
| | - E Gómez
- Hospital Sant Joan de Déu, Esplugues de Llobregat (Barcelona), España
| | - M Del Castillo
- Hospital Sant Joan de Déu, Esplugues de Llobregat (Barcelona), España
| | -
- Hospital Sant Joan de Déu, Esplugues de Llobregat (Barcelona), España
| |
Collapse
|
15
|
Leslie M, Khayatzadeh-Mahani A, MacKean G. Recruitment of caregivers into health services research: lessons from a user-centred design study. RESEARCH INVOLVEMENT AND ENGAGEMENT 2019; 5:17. [PMID: 31139432 PMCID: PMC6528243 DOI: 10.1186/s40900-019-0150-6] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/19/2019] [Accepted: 05/08/2019] [Indexed: 05/23/2023]
Abstract
BACKGROUND With patient and public engagement in many aspects of the healthcare system becoming an imperative, the recruitment of patients and members of the public into service and research roles has emerged as a challenge. The existing literature carries few reports of the methods - successful and unsuccessful - that researchers engaged in user-centred design (UCD) projects are using to recruit participants as equal partners in co-design research. This paper uses the recruitment experiences of a specific UCD project to provide a road map for other investigators, and to make general recommendations for funding agencies interested in supporting co-design research. METHODS We used a case study methodology and employed Nominal Group Technique (NGT) and Focus Group discussions to collect data. We recruited 25 family caregivers. RESULTS Employing various strategies to recruit unpaid family caregivers in a UCD project aimed at co-designing an assistive technology for family caregivers, we found that recruitment through caregiver agencies is the most efficient (least costly) and effective mechanism. The nature of this recruitment work - the time and compromises it requires - has, we believe, implications for funding agencies who need to understand that working with caregivers agencies, requires a considerable amount of time for building relationships, aligning values, and establishing trust. CONCLUSIONS In addition to providing adaptable strategies, the paper contributes to discussions surrounding how projects seeking effective, meaningful, and ethical patient and public engagement are planned and funded. We call for more evidence to explore effective mechanisms to recruit family caregivers into qualitative research. We also call for reports of successful strategies that other researchers have employed to recruit and retain family caregivers in their research.
Collapse
Affiliation(s)
- Myles Leslie
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta Canada
- School of Public Policy, University of Calgary, Downtown Campus, 906 8th Avenue S.W., 5th Floor, Calgary, Alberta T2P 1H9 Canada
| | - Akram Khayatzadeh-Mahani
- School of Public Policy, University of Calgary, Downtown Campus, 906 8th Avenue S.W., 5th Floor, Calgary, Alberta T2P 1H9 Canada
- Health Services Management Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Gail MacKean
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Alberta Canada
- IMAGINE Citizens Collaborating for Health, Calgary, Alberta Canada
| |
Collapse
|
16
|
Nguyen T, Palisano RJ, Graham I. Perspectives and Experiences with Engaging Youth and Families in Research. Phys Occup Ther Pediatr 2019; 39:310-323. [PMID: 30216094 DOI: 10.1080/01942638.2018.1496966] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Engaging youth with disabilities and families in research is critical in facilitating knowledge utilization to impact processes and outcomes of services and interventions. Organizations such as INVOLVE in the United Kingdom, the Canadian Institutes of Health Research and the Patient-Centered Outcomes Research Institute in the United States advocate for engaging stakeholders in all aspects of the research process. Despite these efforts, there is a need for research to identify best practices and approaches for stakeholder engagement in research as current descriptions around processes are lacking. In this perspective paper, we share our insights and experiences with engaging youth with disabilities and families as members of the research team. We offer considerations and strategies for stakeholder engagement, as well as identify relevant frameworks to facilitate stakeholder engagement in research. We encourage researchers to share and document their experiences with stakeholder engagement in research to provide guidance for others with shared interests. We hope this paper will initiate conversations among researchers about best practices to maximize the full potential of stakeholders' time and input.
Collapse
Affiliation(s)
- Tram Nguyen
- a School of Epidemiology and Public Health , University of Ottawa , Ottawa , Ontario , Canada.,b CanChild Centre for Childhood Disability Research , McMaster University , Hamilton , Ontario , Canada.,c School of Rehabilitation Science , McMaster University , Hamilton , Ontario , Canada
| | - Robert J Palisano
- b CanChild Centre for Childhood Disability Research , McMaster University , Hamilton , Ontario , Canada.,d Department of Physical Therapy and Rehabilitation Sciences , Drexel University , Philadelphia , Pennsylvania , USA
| | - Ian Graham
- a School of Epidemiology and Public Health , University of Ottawa , Ottawa , Ontario , Canada.,e Clinical Epidemiology Program , Ottawa Hospital Research Institute , Ottawa , Ontario , Canada
| |
Collapse
|
17
|
Kohler G, Holland T, Sharpe A, Irwin M, Sampalli T, MacDonell K, Kidd N, Edwards L, Gibson R, Legate A, Ampi Kanakam R. The Newcomer Health Clinic in Nova Scotia: A Beacon Clinic to Support the Health Needs of the Refugee Population. Int J Health Policy Manag 2018; 7:1085-1089. [PMID: 30709083 PMCID: PMC6358655 DOI: 10.15171/ijhpm.2018.54] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2018] [Accepted: 06/03/2018] [Indexed: 11/16/2022] Open
Abstract
Refugees tend to have greater vulnerability compared to the general population reporting greater need for physical,
emotional, or dental problems compared to the general population. Despite the importance of creating strong
primary care supports for these patients, it has been demonstrated that there is a significant gap in accessing
primary care providers who are willing to accept the refugee population. These have resulted in bottlenecks in the
transition or bridge clinics and have left patients orphaned without a primary care provider. This in turn results
in higher use of emergency service and other unnecessary costs to the healthcare system. Currently there are few
studies that have explored these challenges from primary care provider perspectives and very few to none from
patient perspectives. A novel collaborative implementation initiative in primary healthcare (PHC) is seeking to
improve primary medical care for the refugee population by creating a globally recommended transition or beacon
clinic to support care needs of new arrivals and transitions to primary care providers. We discuss the innovative
elements of the clinic model in this paper.
Collapse
Affiliation(s)
- Graeme Kohler
- Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Timothy Holland
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Ashley Sharpe
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Mandi Irwin
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Tara Sampalli
- Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Kolten MacDonell
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Natalie Kidd
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Lynn Edwards
- Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Rick Gibson
- Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Amy Legate
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| | - Ruth Ampi Kanakam
- Newcomer Health Clinic, Primary Health Care, Nova Scotia Health Authority, Halifax, NS, Canada
| |
Collapse
|