Co-designing the InnoWell Platform to deliver the right mental health care first time to regional youth

Using a Rapid Learning Health System for Stratified Care in Emerging Adult Mental Health Services: Protocol for the Implementation of Patient-Reported Outcome Measures

BACKGROUND

Mental illness among emerging adults is often difficult to ameliorate due to fluctuating symptoms and heterogeneity. Recently, innovative approaches have been developed to improve mental health care for emerging adults, including (1) implementing patient-reported outcome measures (PROMs) to assess illness severity and inform stratified care to assign emerging adults to a treatment modality commensurate with their level of impairment and (2) implementing a rapid learning health system in which data are continuously collected and analyzed to generate new insights, which are then translated to clinical practice, including collaboration among clients, health care providers, and researchers to co-design and coevaluate assessment and treatment strategies.

OBJECTIVE

The aim of the study is to determine the feasibility and acceptability of implementing a rapid learning health system to enable a measurement-based, stratified care treatment strategy for emerging adults.

METHODS

This study takes place at a specialty clinic serving emerging adults (age 16-24 years) in Calgary, Canada, and involves extensive collaboration among researchers, providers, and youth. The study design includes six phases: (1) developing a transdiagnostic platform for PROMs, (2) designing an initial stratified care model, (3) combining the implementation of PROMs with stratified care, (4) evaluating outcomes and disseminating results, (5) modification of stratified care based on data derived from PROMs, and (6) spread and scale to new sites. Qualitative and quantitative feedback will be collected from health care providers and youth throughout the implementation process. These data will be analyzed at regular intervals and used to modify the way future services are delivered. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is used to organize and evaluate implementation according to 3 key objectives: improving treatment selection, reducing average wait time and treatment duration, and increasing the value of services.

RESULTS

This project was funded through a program grant running from 2021 to 2026. Ethics approval for this study was received in February 2023. Presently, we have developed a system of PROMs and organized clinical services into strata of care. We will soon begin using PROMs to assign clients to a stratum of care and using feedback from youth and clinicians to understand how to improve experiences and outcomes.

CONCLUSIONS

This study has key implications for researchers and clinicians looking to understand how to customize emerging adult mental health services to improve the quality of care and satisfaction with care. This study has significant implications for mental health care systems as part of a movement toward value-based health care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51667.

Monitoring Australian parents' shifting receptiveness to digital mental health interventions during the COVID-19 pandemic

BACKGROUND

Nascent evidence indicates that the mental health of parents and children has markedly declined during the COVID-19 pandemic. Considering disruptions to traditional face-to-face mental health services resultant from stay-at-home orders, the potential value of digital mental health interventions has become extremely apparent. Despite this, uptake of digital interventions remains poor, indicating that a better understanding is needed of factors that determine a willingness to use digital platforms.

METHOD

The present multi-wave, longitudinal study of 2365 Australian parents explored between-person and within-person predictors of intentions to use digital interventions during the pandemic.

RESULTS

More than one-third of parents reported likely use of a self-guided and therapist-guided digital intervention, with the most endorsed reason for use being to support their child's mental health. Between-person baseline predictors of higher intention ratings were parent's prior mental illness, not living with a partner and recent environmental stressors. Within-person predictors of higher intention ratings were endorsement of mindful parenting strategies, child access to the Internet, better perceived management of child's education, lower social support and financial hardship.

CONCLUSION

Findings demonstrate that willingness to engage in digital interventions fluctuates in response to changing circumstances. Identifying novel ways to increase acceptance and uptake of digital interventions based on modifiable predictors established here is needed to realize the full potential of these modes of care in times of need.

Introduction of Technology to Support Young People’s Care and Mental Health—A Rapid Evidence Review

Background

Technology and its use within mental health services has advanced dramatically over recent years. Opportunities for mental health services to utilise technology to introduce novel, effective, and more efficient means of delivering assessment, and treatment are increasing.

Objective

The current rapid-evidence paper reviews evidence regarding the introduction of novel technology to support young people’s mental health and psychological well-being.

Methods

A rapid evidence review was conducted. PSYCHINFO and CINAHL were searched for research articles between 2016 and 2021 that were specific to young people, mental health, and technology developments within this domain. N = 27 studies which explored the introduction, feasibility, and value of technology for mental health purposes were included in a narrative synthesis. Quality or risk of bias analyses were not completed.

Results

Overall, technological advancements in young people’s care were considered positive and engaging for young people. Factors including resources, efficiency of care, engagement, therapeutic effectiveness, ethical considerations, therapeutic alliance, and flexibility were considered within this review. Nevertheless, potential barriers include clinician concerns, socioeconomic factors, and motivation.

Conclusion

Effective and sustained use of technology within young people’s mental health services will depend on the technology’s usability, efficiency, and ability to engage young people. This paper expands on existing research by reviewing a broader range of technology proposed to support young people’s mental health and well-being. This will assist in the application of novel technological advancements by indicating effectiveness, preferences, potential barriers, and recommendations for the feasibility and efficacy of introducing technology into young people’s services.

eHealth Tools That Assess and Track Health and Well-being in Children and Young People: Systematic Review

Background

eHealth tools that assess and track health outcomes in children or young people are an emerging type of technology that has the potential to reform health service delivery and facilitate integrated, interdisciplinary care.

Objective

The aim of this review is to summarize eHealth tools that have assessed and tracked health in children or young people to provide greater clarity around the populations and settings in which they have been used, characteristics of digital devices (eg, health domains, respondents, presence of tracking, and connection to care), primary outcomes, and risks and challenges of implementation.

Methods

A search was conducted in PsycINFO, PubMed or MEDLINE, and Embase in April 2020. Studies were included if they evaluated a digital device whose primary purpose was to assess and track health, focused on children or young people (birth to the age of 24 years), reported original research, and were published in peer-reviewed journals in English.

Results

A total of 39 papers were included in this review. The sample sizes ranged from 7 to 149,329 participants (median 163, mean 5155). More studies were conducted in urban (18/39, 46%) regions than in rural (3/39, 8%) regions or a combination of urban and rural areas (8/39, 21%). Devices were implemented in three main settings: outpatient health clinics (12/39, 31%), hospitals (14/39, 36%), community outreach (10/39, 26%), or a combination of these settings (3/39, 8%). Mental and general health were the most common health domains assessed, with a single study assessing multiple health domains. Just under half of the devices tracked children’s health over time (16/39, 41%), and two-thirds (25/39, 64%) connected children or young people to clinical care. It was more common for information to be collected from a single informant (ie, the child or young person, trained health worker, clinician, and parent or caregiver) than from multiple informants. The health of children or young people was assessed as a primary or secondary outcome in 36% (14/39) of studies; however, only 3% (1/39) of studies assessed whether using the digital tool improved the health of users. Most papers reported early phase research (formative or process evaluations), with fewer outcome evaluations and only 3 randomized controlled trials. Identified challenges or risks were related to accessibility, clinical utility and safety, uptake, data quality, user interface or design aspects of the device, language proficiency or literacy, sociocultural barriers, and privacy or confidentiality concerns; ways to address these barriers were not thoroughly explored.

Conclusions

eHealth tools that assess and track health in children or young people have the potential to enhance health service delivery; however, a strong evidence base validating the clinical utility, efficacy, and safety of tools is lacking, and more thorough investigation is needed to address the risks and challenges of using these emerging technologies in clinical care. At present, there is greater potential for the tools to facilitate multi-informant, multidomain assessments and longitudinally track health over time and room for further implementation in rural or remote regions and community settings around the world.

Informing the Future of Integrated Digital and Clinical Mental Health Care: Synthesis of the Outcomes From Project Synergy

BACKGROUND

Globally, there are fundamental shortcomings in mental health care systems, including restricted access, siloed services, interventions that are poorly matched to service users' needs, underuse of personal outcome monitoring to track progress, exclusion of family and carers, and suboptimal experiences of care. Health information technologies (HITs) hold great potential to improve these aspects that underpin the enhanced quality of mental health care.

OBJECTIVE

Project Synergy aimed to co-design, implement, and evaluate novel HITs, as exemplified by the InnoWell Platform, to work with standard health care organizations. The goals were to deliver improved outcomes for specific populations under focus and support organizations to enact significant system-level reforms.

METHODS

Participating health care organizations included the following: Open Arms-Veterans & Families Counselling (in Sydney and Lismore, New South Wales [NSW]); NSW North Coast headspace centers for youth (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads); the Butterfly Foundation's National Helpline for eating disorders; Kildare Road Medical Centre for enhanced primary care; and Connect to Wellbeing North Coast NSW (administered by Neami National), for population-based intake and assessment. Service users, families and carers, health professionals, and administrators of services across Australia were actively engaged in the configuration of the InnoWell Platform to meet service needs, identify barriers to and facilitators of quality mental health care, and highlight potentially the best points in the service pathway to integrate the InnoWell Platform. The locally configured InnoWell Platform was then implemented within the respective services. A mixed methods approach, including surveys, semistructured interviews, and workshops, was used to evaluate the impact of the InnoWell Platform. A participatory systems modeling approach involving co-design with local stakeholders was also undertaken to simulate the likely impact of the platform in combination with other services being considered for implementation within the North Coast Primary Health Network to explore resulting impacts on mental health outcomes, including suicide prevention.

RESULTS

Despite overwhelming support for integrating digital health solutions into mental health service settings and promising impacts of the platform simulated under idealized implementation conditions, our results emphasized that successful implementation is dependent on health professional and service readiness for change, leadership at the local service level, the appropriateness and responsiveness of the technology for the target end users, and, critically, funding models being available to support implementation. The key places of interoperability of digital solutions and a willingness to use technology to coordinate health care system use were also highlighted.

CONCLUSIONS

Although the COVID-19 pandemic has resulted in the widespread acceptance of very basic digital health solutions, Project Synergy highlights the critical need to support equity of access to HITs, provide funding for digital infrastructure and digital mental health care, and actively promote the use of technology-enabled, coordinated systems of care.

Mapping the role of patient and public involvement during the different stages of healthcare innovation: A scoping review

BACKGROUND

Patient and public involvement (PPI) has become increasingly important in the development, delivery and improvement of healthcare. PPI is used in healthcare innovation; yet, how it is used has been under-reported. The aim of this scoping review is to identify and map the current available empirical evidence on the role of PPI during different stages of healthcare innovation.

METHODS

The scoping review was conducted in accordance with PRISMAScR and included any study published in a peer-reviewed journal between 2004 and 2021 that reported on PPI in healthcare innovation within any healthcare setting or context in any country. The following databases were searched: Medline, EMBASE, CINAHL, PsycInfo, HMIC and Google Scholar. We included any study type, including quantitative, qualitative and mixed-method studies. We excluded theoretical frameworks, conceptual, scientific or grey literature as well as discussion and opinion papers.

RESULTS

Of the 87 included studies, 81 (93%) focused on or were conducted by authors in developed countries. A wide range of conditions were considered, with more studies focusing on mental health (n = 18, 21%) and cancer care (n = 8, 9%). The vast majority of the studies focused on process and service innovations (n = 62, 71%). Seven studies focused on technological and clinical innovations (8%), while 12 looked at both technological and service innovations (14%). Only five studies examined systems innovation (5%) and one study looked across all types of innovations (1%). PPI is more common in the earlier stages of innovation, particularly problem identification and invention, in comparison to adoption and diffusion.

CONCLUSION

Healthcare innovation tends to be a lengthy process. Yet, our study highlights that PPI is more common across earlier stages of innovation and focuses mostly on service innovation. Stronger PPI in later stages could support the adoption and diffusion of innovation.

PATIENT OR PUBLIC CONTRIBUTION

One of the coauthors of the paper (S. S.) is a service user with extensive experience in PPI research. S. S. supported the analysis and writing up of the paper.

The Impact of Technology-Enabled Care Coordination in a Complex Mental Health System: A Local System Dynamics Model

Background

Prior to the COVID-19 pandemic, major shortcomings in the way mental health care systems were organized were impairing the delivery of effective care. The mental health impacts of the pandemic, the recession, and the resulting social dislocation will depend on the extent to which care systems will become overwhelmed and on the strategic investments made across the system to effectively respond.

Objective

This study aimed to explore the impact of strengthening the mental health system through technology-enabled care coordination on mental health and suicide outcomes.

Methods

A system dynamics model for the regional population catchment of North Coast New South Wales, Australia, was developed that incorporated defined pathways from social determinants of mental health to psychological distress, mental health care, and suicidal behavior. The model reproduced historic time series data across a range of outcomes and was used to evaluate the relative impact of a set of scenarios on attempted suicide (ie, self-harm hospitalizations), suicide deaths, mental health–related emergency department (ED) presentations, and psychological distress over the period from 2021 to 2030. These scenarios include (1) business as usual, (2) increase in service capacity growth rate by 20%, (3) standard telehealth, and (4) technology-enabled care coordination. Each scenario was tested using both pre– and post–COVID-19 social and economic conditions.

Results

Technology-enabled care coordination was forecast to deliver a reduction in self-harm hospitalizations and suicide deaths by 6.71% (95% interval 5.63%-7.87%), mental health–related ED presentations by 10.33% (95% interval 8.58%-12.19%), and the prevalence of high psychological distress by 1.76 percentage points (95% interval 1.35-2.32 percentage points). Scenario testing demonstrated that increasing service capacity growth rate by 20% or standard telehealth had substantially lower impacts. This pattern of results was replicated under post–COVID-19 conditions with technology-enabled care coordination being the only tested scenario, which was forecast to reduce the negative impact of the pandemic on mental health and suicide.

Conclusions

The use of technology-enabled care coordination is likely to improve mental health and suicide outcomes. The substantially lower effectiveness of targeting individual components of the mental health system (ie, increasing service capacity growth rate by 20% or standard telehealth) reiterates that strengthening the whole system has the greatest impact on patient outcomes. Investments into more of the same types of programs and services alone will not be enough to improve outcomes; instead, new models of care and the digital infrastructure to support them and their integration are needed.

Implementing a digital health model of care in Australian youth mental health services: protocol for impact evaluation

Background

The World Economic Forum has recently highlighted substantial problems in mental health service provision and called for the rapid deployment of smarter, digitally-enhanced health services as a means to facilitate effective care coordination and address issues of demand. In mental health, the biggest enabler of digital solutions is the implementation of an effective model of care that is facilitated by integrated health information technologies (HITs); the latter ensuring the solution is easily accessible, scalable and sustainable. The University of Sydney’s Brain and Mind Centre (BMC) has developed an innovative digital health solution – delivered through the Youth Mental Health and Technology Program – which incorporates two components: 1) a highly personalised and measurement-based (data-driven) model of youth mental health care; and 2) an industrial grade HIT registered on the Australian Register of Therapeutic Goods. This paper describes a research protocol to evaluate the impact of implementing the BMC’s digital health solution into youth mental health services (i.e. headspace - a highly accessible, youth-friendly integrated service that responds to the mental health, physical health, alcohol or other substance use, and vocational concerns of young people aged 12 to 25 years) within urban and regional areas of Australia.

Methods

The digital health solution will be implemented into participating headspace centres using a naturalistic research design. Quantitative and qualitative data will be collected from headspace health professionals, service managers and administrators, as well as from lead agency and local Primary Health Network (PHN) staff, via service audits, Implementation Officer logs, online surveys, and semi-structured interviews, at baseline and then three-monthly intervals over the course of 12 months.

Discussion

At the time of publication, six headspace centres had been recruited to this study and had commenced implementation and impact evaluation. The first results are expected to be submitted for publication in 2021. This study will focus on the impact of implementing a digital health solution at both a service and staff level, and will evaluate digital readiness of service and staff adoption; quality, usability and acceptability of the solution by staff; staff self-reported clinical competency; overall impact on headspace centres as well as their lead agencies and local PHNs; and social return on investment.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06394-4.

Evaluating the quality and safety of health-related apps and e-tools: Adapting the Mobile App Rating Scale and developing a quality assurance protocol

BACKGROUND

Whilst apps and e-tools have tremendous potential as low-cost, scalable mental health intervention and prevention tools, it is essential that consumers and health professionals have a means by which to evaluate their quality and safety.

OBJECTIVE

This study aimed to: 1) adapt the original Mobile App Rating Scale (MARS) in order to be appropriate for the evaluation of both mobile phone applications as well as e-tools; 2) test the reliability of the revised scale; and 3) develop a quality assurance protocol for identifying and rating new apps and e-tools to determine appropriateness for use in clinical practice.

METHODS

The MARS was adapted to include items specific to health-related apps and e-tools, such as the availability of resources, strategies for self-management, and quality information. The 41 apps and e-tools in the standard youth configuration of the InnoWell Platform, a digital tool designed to support or enhance mental health service delivery, were independently rated by two expert raters using the A-MARS. Cronbach's alpha was used to calculate the internal consistency and interclass correlation coefficients were used to calculate interrater reliability.

RESULTS

The A-MARS was shown to be a reliable scale with acceptable to excellent internal consistency and moderate to excellent interrater reliability across the subscales. Given the ever-increasing number of health information technologies on the market, a protocol to identify and rate new apps and e-tools for potential clinical use is presented.

CONCLUSIONS

Whilst the A-MARS is a useful tool to guide health professionals as they explore available apps and e-tools for potential clinical use, the training, time, and skill required to use it effectively may be prohibitive. As such, health professionals and services are likely to benefit from including a digital navigator as part of the care team to assist in selecting and rating apps and e-tools, increasing the usability of the data, and technology troubleshooting. When selecting, evaluating and/or recommending apps and e-tools to consumers, it is important to consider: 1) the availability of explicit strategies to set, monitor and review SMART goals; 2) the accessibility of credible, user friendly information and resources from reputable sources; 3) evidence of effectiveness; and 4) interoperability with other health information technologies.

Technology-Enabled Reform in a Nontraditional Mental Health Service for Eating Disorders: Participatory Design Study

BACKGROUND

The recent Australian National Agenda for Eating Disorders highlights the role technology can play in improving accessibility and service development through web-based prevention, early access pathways, self-help, and recovery assistance. However, engagement with the eating disorders community to co-design, build, and evaluate these much-needed technology solutions through participatory design processes has been lacking and, until recently, underresourced.

OBJECTIVE

This study aims to customize and configure a technology solution for a nontraditional (web-based, phone, email) mental health service that provides support for eating disorders and body image issues through the use of participatory design processes.

METHODS

Participants were recruited chiefly through the Butterfly National Helpline 1800 ED HOPE (Butterfly's National Helpline), an Australian-wide helpline supporting anyone concerned by an eating disorder or body image issue. Participants included individuals with lived experience of eating disorders and body image issues, their supportive others (such as family, health professionals, support workers), and staff of the Butterfly Foundation. Participants took part in participatory design workshops, running up to four hours, which were held nationally in urban and regional locations. The workshop agenda followed an established process of discovery, evaluation, and prototyping. Workshop activities included open and prompted discussion, reviewing working prototypes, creating descriptive artifacts, and developing user journeys. Workshop artifacts were used in a knowledge translation process, which identified key learnings to inform user journeys, user personas, and the customization and configuration of the InnoWell Platform for Butterfly's National Helpline. Further, key themes were identified using thematic techniques and coded in NVivo 12 software.

RESULTS

Six participatory design workshops were held, of which 45 participants took part. Participants highlighted that there is a critical need to address some of the barriers to care, particularly in regional and rural areas. The workshops highlighted seven overarching qualitative themes: identified barriers to care within the current system; need for people to be able to access the right care anywhere, anytime; recommendations for the technological solution (ie, InnoWell Platform features and functionality); need for communication, coordination, and integration of a technological solution embedded in Butterfly's National Helpline; need to consider engagement and tone within the technological solution; identified challenges and areas to consider when implementing a technological solution in the Helpline; and potential outcomes of the technological solution embedded in the Helpline relating to system and service reform. Ultimately, this technology solution should ensure that the right care is provided to individuals the first time.

CONCLUSIONS

Our findings highlight the value of actively engaging stakeholders in participatory design processes for the customization and configuration of new technologies. End users can highlight the critical areas of need, which can be used as a catalyst for reform through the implementation of these technologies in nontraditional services.

Recommendations for Designing Health Information Technologies for Mental Health Drawn From Self-Determination Theory and Co-design With Culturally Diverse Populations: Template Analysis

Background

Culturally diverse populations (including Aboriginal and Torres Strait Islander people, people of diverse genders and sexualities, and culturally and linguistically diverse people) in nonurban areas face compounded barriers to accessing mental health care. Health information technologies (HITs) show promising potential to overcome these barriers.

Objective

This study aims to identify how best to improve a mental health and well-being HIT for culturally diverse Australians in nonurban areas.

Methods

We conducted 10 co-design workshops (N=105 participants) in primary youth mental health services across predominantly nonurban areas of Australia and conducted template analysis on the workshop outputs. Owing to local (including service) demographics, the workshop participants naturalistically reflected culturally diverse groups.

Results

We identified 4 main themes: control, usability, affirmation, and health service delivery factors. The first 3 themes overlap with the 3 basic needs postulated by self-determination theory (autonomy, competence, and relatedness) and describe participant recommendations on how to design an HIT. The final theme includes barriers to adopting HITs for mental health care and how HITs can be used to support care coordination and delivery. Hence, it describes participant recommendations on how to use an HIT.

Conclusions

Although culturally diverse groups have specific concerns, their expressed needs fall broadly within the relatively universal design principles identified in this study. The findings of this study provide further support for applying self-determination theory to the design of HITs and reflect the tension in designing technologies for complex problems that overlap multiple medical, regulatory, and social domains, such as mental health care. Finally, we synthesize the identified themes into general recommendations for designing HITs for mental health and provide concrete examples of design features recommended by participants.

Optimising the integration of technology-enabled solutions to enhance primary mental health care: a service mapping study

Background

Despite the widely acknowledged potential for health information technologies to improve the accessibility, quality and clinical safety of mental health care, implementation of such technologies in services is frequently unsuccessful due to varying consumer, health professional, and service-level factors. The objective of this co-design study was to use process mapping (i.e. service mapping) to illustrate the current consumer journey through primary mental health services, identify barriers to and facilitators of quality mental health care, and highlight potential points at which to integrate the technology-enabled solution to optimise the provision of care based on key service performance indicators.

Methods

Interactive, discussion-based workshops of up to six hours were conducted with representative stakeholders from each participating service, including health professionals, service managers and administrators from Open Arms – Veterans & Families Counselling Service (Sydney), a counselling service for veterans and their families, and five headspace centres in the North Coast Primary Health Network, primary youth mental health services. Service maps were drafted and refined in real time during the workshops. Through both group discussion and the use of post-it notes, participants worked together to evaluate performance indicators (e.g. safety) at each point in the consumer journey (e.g. intake) to indicate points of impact for the technology-enabled solution, reviewing and evaluating differing opinions in order to reach consensus.

Results

Participants (n=84 across participating services) created service maps illustrating the current consumer journey through the respective services and highlighting barriers to and facilitators of quality mental health care. By consensus, the technology-enabled solution as facilitated by the InnoWell Platform was noted to enable the early identification of risk, reduce or eliminate lengthy intake processes, enable routine outcome monitoring to revise treatment plans in relation to consumer response, and serve as a personal data record for consumers, driving person-centred, coordinated care.

Conclusions

Service mapping was shown to be an effective methodology to understand the consumer’s journey through a service and served to highlight how the co-designed technology-enabled solution can optimise service pathways to improve the accessibility, quality and clinical safety of care relative to key service performance indicators, facilitating the delivery of the right care.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06069-0.

Flip the Clinic: A Digital Health Approach to Youth Mental Health Service Delivery During the COVID-19 Pandemic and Beyond

The demand for mental health services is projected to rapidly increase as a direct and indirect result of the COVID-19 pandemic. Given that young people are disproportionately disadvantaged by mental illness and will face further challenges related to the COVID-19 pandemic, it is crucial to deliver appropriate mental health care to young people as early as possible. Integrating digital health solutions into mental health service delivery pathways has the potential to greatly increase efficiencies, enabling the provision of "right care, first time." We propose an innovative digital health solution for demand management intended for use by primary youth mental health services, comprised of (1) a youth mental health model of care (ie, the Brain and Mind Centre Youth Model) and (2) a health information technology specifically designed to deliver this model of care (eg, the InnoWell Platform). We also propose an operational protocol of how this solution could be applied to primary youth mental health service delivery processes. By "flipping" the conventional service delivery models of majority in-clinic and minority web-delivered care to a model where web-delivered care is the default, this digital health solution offers a scalable way of delivering quality youth mental health care both in response to public health crises (such as the COVID-19 pandemic) and on an ongoing basis in the future.