Evaluation of sustainable hydrogen production technologies on an industrial scale using comparative analysis of decision-making methods

Due to its various advantages in different industrial fields, hydrogen can provide energy based on sustainability goals and recreates a critical function in the economy of countries. In this regard, evaluating hydrogen production technologies on an industrial scale is necessary for industrial development and economic growth. Therefore, this study proposes a comprehensive, integrated framework of hybrid fuzzy decision-making for assessing hydrogen production technologies in Iran. In addition to considering sustainability factors, political, technical, and reliability indicators are also assessed in this research to make a comprehensive assessment. The Fuzzy Step-wise Weight Assessment Ratio Analysis (F-SWARA) technique determines the importance of indicators, and the Fuzzy Weighted Aggregates Sum-Product Assessment (F-WASPAS) approach ranks technologies. The weighing findings indicated that the sub-indices of investment cost, technical infrastructure development, and implementation costs were introduced as the most significant sub-indices with weights of 0.226, 0.151, and 0.126, respectively. The evaluation findings with the F-WASPAS method and comparative analysis with various decision-making methods revealed that electrolysis based on solar energy and electrolysis based on wind energy technologies had the highest preference. In this regard, the infrastructure and costs of hydrogen production can be improved by presenting various incentives, such as improving financial conditions while attracting investment and increasing cooperation with top companies. So, sustainable development, economic growth, and industrial development are provided.

Topic identification, selection, and prioritization for health technology assessment in selected countries: a mixed study design

BACKGROUND

There is limited evidence-informed guidance on TISP processes for countries where health technology assessment (HTA) is in a nascent phase. We aimed to explore the range of topic identification, selection and prioritization (TISP) processes and practices for HTA in selected countries and identify aspects relevant to emerging HTA systems.

METHODS

This mixed design study included a systematic literature review, an electronic survey, and individual interviews. We conducted a systematic literature review with criteria that were developed a priori to identify countries deemed to have a recently formalized HTA system. Based on the literature review, a twenty-three item online survey was shared with the identified countries, we completed follow-up interviews with ten participants who have experience with HTA. We analyzed documents, survey responses and interview transcripts thematically to identify lessons related to TISP processes and practices.

RESULTS

The literature review identified 29 nine candidate countries as having a "potential" recently formalized HTA system. Twenty-one survey responses were analyzed and supplemented with ten individual interviews. We found variation in countries' approaches to TISP - particularly between pharmaceutical and non-pharmaceutical interventions. Results indicate that TISP is heavily driven by policy makers, expert involvement, and to a lesser extent, relevant stakeholders. The use of horizon-scanning and early warning systems is uncommon. Interviewee participants provided further insight to the survey data, reporting that political awareness and an institutional framework were important to support TISP. TISP can be optimized by stronger national regulations and legislative structures, in addition to education and advocacy about HTA among politicians and decision-makers. In some settings regional networks have been useful, particularly in the development of TISP guidelines and methodologies. Additionally, the technical capacity to conduct TISP, and access to relevant local data were factors limiting TISP in national settings. Increased network collaboration and capacity building were reported as future needs.

CONCLUSIONS

This study provides current insights into a topic where there is limited published peer reviewed literature. TISP is an important first step of HTA, and topics should be selected and prioritized based on local need and relevance. The limited capacity for TISP in settings where HTA is emerging may be supported by local and international collaboration to increase capacity and knowledge. To succeed, both TISP and HTA need to be embedded within national health care priority setting and decision-making. More in-depth understanding of where countries are situtated in formalizing the TISP process may help others to overcome factors that facilitate or hinder progress.

Research gaps identified in Iran's health technology assessment reports

INTRODUCTION

Identifying gaps in the evidence is a useful byproduct of conducting a health technology assessment (HTA). This study aims to identify research gaps in Iran's HTA reports.

METHOD

We reviewed the HTA reports published between 2014 and 2016. Then, we developed two separate questionnaires for principal investigators (PIs) and independent HTA researchers. The questionnaire for independent HTA researchers consisted of four main parts. However, the PIs' questionnaire consisted of two main parts. We also conducted a literature search in the PubMed database in November 2017 to find frameworks for prioritizing research gaps. We also conducted a semi-structured interview with the head of the Iran's HTA Office at that time and sought feedback based on his expert opinion about questionnaires, the priority-setting tool and our process for extracting research gaps.

RESULTS

A total of 11 HTA reports published between 2014 and 2016 by Iran's HTA Office were selected for the study. Of these 11 reports, 5 involved technologies related to medical equipment, while 6 involved medical and surgical interventions. Assessing the outcomes of technology use in various indications and updating HTAs when new evidence arises; evaluating the viewpoints of patients, clinicians and key technology users; conducting post-marketing evaluations of technology; comparing the impact of the technology in question to other treatments for the same condition; and requesting long-term clinical and cost-effectiveness data for technologies with limited follow-up periods were identified as the main gaps by independent HTA researchers and PIs.

CONCLUSIONS

The research gaps identified from Iran's HTAs could be utilized by research funding agencies.

A tailored approach to horizon scanning for cancer medicines

BACKGROUND

Horizon scanning (HS) is the systematic identification of emerging therapies to inform policy and decision-makers. We developed an agile and tailored HS methodology that combined multi-criteria decision analysis weighting and Delphi rounds. As secondary objectives, we aimed to identify new medicines in melanoma, non-small cell lung cancer and colorectal cancer most likely to impact the Australian government's pharmaceutical budget by 2025 and to compare clinician and consumer priorities in cancer medicine reimbursement.

METHOD

Three cancer-specific clinician panels (total n = 27) and a consumer panel (n = 7) were formed. Six prioritisation criteria were developed with consumer input. Criteria weightings were elicited using the Analytic Hierarchy Process (AHP). Candidate medicines were identified and filtered from a primary database and validated against secondary and tertiary sources. Clinician panels participated in a three-round Delphi survey to identify and score the top five medicines in each cancer type.

RESULTS

The AHP and Delphi process was completed in eight weeks. Prioritisation criteria focused on toxicity, quality of life (QoL), cost savings, strength of evidence, survival, and unmet need. In both curative and non-curative settings, consumers prioritised toxicity and QoL over survival gains, whereas clinicians prioritised survival. HS results project the ongoing prevalence of high-cost medicines. Since completion in October 2021, the HS has identified 70 % of relevant medicines submitted for Pharmaceutical Benefit Advisory Committee assessment and 60% of the medicines that received a positive recommendation.

CONCLUSION

Tested in the Australian context, our method appears to be an efficient and flexible approach to HS that can be tailored to address specific disease types by using elicited weights to prioritise according to incremental value from both a consumer and clinical perspective.

POLICY SUMMARY

Since HS is of global interest, our example provides a reproducible blueprint for adaptation to other healthcare settings that integrates consumer input and priorities.

Can Weight of Evidence, Quantitative Bias, and Bounding Methods Evaluate Robustness of Real-world Evidence for Regulator and Health Technology Assessment Decisions on Medical Interventions?

PURPOSE

High-quality evidence is crucial for health care intervention decision-making. These decisions frequently use nonrandomized data, which can be more vulnerable to biases than randomized trials. Accordingly, methods to quantify biases and weigh available evidence could elucidate the robustness of findings, giving regulators more confidence in making approval and reimbursement decisions.

METHODS

We conducted an integrative literature review to identify methods for determining probability of causation, evaluating weight of evidence, and conducting quantitative bias analysis as related to health care interventions. Eligible studies were published from 2012 to 2021, applicable to pharmacoepidemiology, and presented a method that met our objective.

FINDINGS

Twenty-two eligible studies were classified into 4 categories: (1) quantitative bias analysis; (2) weight of evidence methods; (3) Bayesian networks; and (4) miscellaneous. All of the methods have strengths, limitations, and situations in which they are more well suited than others. Some methods seem to lend themselves more to applications of health care evidence on medical interventions than others.

IMPLICATIONS

To provide robust evidence for and improve confidence in regulatory or reimbursement decisions, we recommend applying multiple methods to triangulate associations of medical interventions, accounting for biases in different ways. This approach could lead to well-defined robustness assessments of study findings and appropriate science-driven decisions by regulators and payers for public health.

Avoid equipment graveyards: rigorous process to improve identification and procurement of effective, affordable, and usable newborn devices in low-resource hospital settings

BACKGROUND

Millions of newborns die annually from preventable causes, with the highest rates occurring in Africa. Reducing neonatal mortality requires investment to scale hospital care, which includes providing hospitals with appropriate technology to care for small and sick newborns. Expensive medical devices designed for high-resource settings often fail to withstand conditions in low-resource hospitals, including humidity, dust, frequent user turnover, complex maintenance, lack of stable power, or difficulty sourcing expensive consumables. Rigorous evaluation protocols are needed to identify effective, affordable, rugged, and easy-to-use medical devices appropriate for quality hospital-based newborn care in low-resource hospitals.

METHODS

We developed an evidence-based technology review process to identify medical devices suitable for small and sick newborn care in low-resource hospitals. The eight-step process consists of: identifying devices needed for effective newborn care; defining Target Product Profiles (TPPs); identifying commercially-available products that may meet TPPs; conducting desk research to evaluate technologies against TPPs; performing technical performance verification testing under laboratory conditions; verifying technical performance after exposure to heat, humidity, dust, and power loss; performing usability evaluations with nurses, and qualifying devices that pass all steps. Devices were purchased, installed, and monitored in newborn wards across Kenya, Malawi, Nigeria, and Tanzania.

RESULTS

Of 271 devices considered, only 45 (16.6%) met corresponding TPPs based on desk research. Thirty-nine were purchased and evaluated in the laboratory; five (12.8%) failed to meet TPPs. Thirty-four products passing laboratory evaluation underwent short-term environmental testing; only one (2.9%) device failed. Thirty-seven products underwent usability testing with 127 clinicians; surprisingly, 14 (37.8%) failed to meet TPPs. Twenty-three products passed all evaluations, and 2457 devices were installed across 65 newborn wards in Kenya, Malawi, Nigeria, and Tanzania. Continuous device monitoring reported minimal device failures, with failed devices typically returned to service within two days, resulting in an average uptime (service days divided by days installed) of 99%.

CONCLUSION

An evidence-based device selection process can improve procurement of effective, affordable, rugged, usable newborn care devices for low-resource hospitals, and feedback to manufacturers can improve device quality. Similar processes could be adapted beyond newborn care to identify medical devices suitable for implementation in any low-resource setting.

Patient-reported outcomes with hypoglossal nerve stimulation for treatment of obstructive sleep apnea: a systematic review and meta-analysis

INTRODUCTION

Hypoglossal nerve stimulation (HNS) has recently been introduced as an alternative treatment for patients with OSA. A large number of studies have demonstrated substantial changes in OSA with this therapy by reducing respiratory events and improving symptoms such as daytime sleepiness and quality of life. The objective of this review was to conduct a systematic review and meta-analysis to evaluate patient-reported outcomes and experience with HNS therapy.

METHODS

A systematic literature search of MEDLINE, Cochrane, and Web of Science was performed to identify randomized controlled and observational studies reporting subjective outcomes with different HNS systems in patients with OSA. Abstracts of 406 articles were screened and a subset of 55 articles were reviewed for eligibility. Risk of bias was assessed using the ROBINS-I tool. Meta-analysis using RevMan was performed when > 2 studies were identified that reported data on a specific outcome.

RESULTS

Thirty-four publications reporting data on 3785 patients with a mean follow-up of 11.8 ± 12.2 months were identified and included in the meta-analysis. The analysis revealed a pooled effect of 4.59 points improvement in daytime sleepiness as measured by the ESS questionnaire (Z = 42.82, p < .001), 2.84 points improvement in daytime functioning as measured by the FOSQ score (Z = 28.38, p < .001), and 1.77 points improvement in sleep quality as measured by the PSQI questionnaire (Z = 2.53, p = .010). Patient-reported experience was consistently positive and revealed additional relevant aspects from this perspective.

CONCLUSION

HNS therapy significantly improves quality of life in patients with OSA and reliably produces clinically meaningful effects on daytime sleepiness, daytime functioning, and sleep quality. Treatment regularly meets or exceeds the minimum clinically important differences defined for the respective instruments. Additional research is needed to further investigate effects on quality of life beyond improvements in daytime sleepiness and daytime functioning.

Management of medical equipment in the Brazilian public health system (SUS), historical situation and the context of the pandemic COVID-19: a cut for lung ventilators

Purpose

The main objective of this paper is to analyze the Brazilian Ministry of Health (MoH) efforts in the management of medical equipment, with a specific approach for lung ventilators in the pandemic scenario of COVID-19.

Methods

The methodology included a review of the normative framework and literature on technological management and research on the database of the Ministry of Health.

Results

As a promoter for acquiring medical equipment, the MoH role is highlighted and added to this competence; its function as the coordinator of the National Policy on Health Technology Management (PNGTS). According to the PNGTS the MoH has to support health managers in the implementing, monitoring, and maintaining health technologies. The scenario of lung ventilators in the pandemic was discussed, with research to verify demands, offers, installed capacity, and investments. In less than one year, the Ministry of Health acquired several pulmonary ventilators, 8.55 times greater than the annual averages of equipment acquired from 2016 to 2019. So far, there is still no maintenance plans or strategy of management for that equipment, especially in a post-pandemic scenario. Conclusion: It is possible to conclude that the Ministry of Health needs to improve health technology management systems. On the scale of the Policy, it is necessary to commit to permanent and long-term actions to ensure sustainability and reduce the technological vulnerabilities of the SUS.

Ramping down a clinical 3 T scanner: a journey into MRI and MRS at 0.75 T

OBJECT

Lower-field MR is reemerging as a viable, potentially cost-effective alternative to high-field MR, thanks to advances in hardware, sequence design, and reconstruction over the past decades. Evaluation of lower field strengths, however, is limited by the availability of lower-field systems on the market and their considerable procurement costs. In this work, we demonstrate a low-cost, temporary alternative to purchasing a dedicated lower-field MR system.

MATERIALS AND METHODS

By ramping down an existing clinical 3 T MRI system to 0.75 T, proton signals can be acquired using repurposed ¹³C transmit/receive hardware and the multi-nuclei spectrometer interface. We describe the ramp-down procedure and necessary software and hardware changes to the system.

RESULTS

Apart from presenting system characterization results, we show in vivo examples of cardiac cine imaging, abdominal two- and three-point Dixon-type water/fat separation, water/fat-separated MR Fingerprinting, and point-resolved spectroscopy. In addition, the ramp-down approach allows unique comparisons of, e.g., gradient fidelity of the same MR system operated at different field strengths using the same receive chain, gradient coils, and amplifiers.

DISCUSSION

Ramping down an existing MR system may be seen as a viable alternative for lower-field MR research in groups that already own multi-nuclei hardware and can also serve as a testing platform for custom-made multi-nuclei transmit/receive coils.

Can mixed reality technologies teach surgical skills better than traditional methods? A prospective randomised feasibility study

BACKGROUND

Basic surgical skills teaching is often delivered with didactic audio-visual content, and new digital technologies may allow more engaging and effective ways of teaching to be developed. The Microsoft HoloLens 2 (HL2) is a multi-functional mixed reality headset. This prospective feasibility study sought to assess the device as a tool for enhancing technical surgical skills training.

METHODS

A prospective randomised feasibility study was conducted. 36 novice medical students were trained to perform a basic arteriotomy and closure using a synthetic model. Participants were randomised to receive a structured surgical skills tutorial via a bespoke mixed reality HL2 tutorial (n = 18), or via a standard video-based tutorial (n = 18). Proficiency scores were assessed by blinded examiners using a validated objective scoring system and participant feedback collected.

RESULTS

The HL2 group showed significantly greater improvement in overall technical proficiency compared to the video group (10.1 vs. 6.89, p = 0.0076), and a greater consistency in skill progression with a significantly narrower range of scores (SD 2.48 vs. 4.03, p = 0.026). Participant feedback showed the HL2 technology to be more interactive and engaging with minimal device related problems experienced.

CONCLUSIONS

This study has demonstrated that mixed reality technology may provide a higher quality educational experience, improved skill progression and greater consistency in learning when compared to traditional teaching methodologies for basic surgical skills. Further work is required to refine, translate, and evaluate the scalability and applicability of the technology across a broad range of skills-based disciplines.

The role of accelerator programmes in supporting the adoption of digital health technologies: A qualitative study of the perspectives of small- and medium-sized enterprises

Objective

Evidence-based digital health technologies are increasingly important in delivering care to an ageing population with constrained resources. In the United Kingdom, accelerator programmes (APs) have been developed to support the adoption of digital health technologies within the National Health Service. This study aims to explore the perspectives of stakeholders using APs.

Methods

Stakeholders representing nine small -and medium-sized enterprises (SMEs) that were engaged with three different APs (n = 9). Semi-structured interviews were conducted with key informants between April and September 2018. Framework analysis of the data was performed to explore their perspectives on APs.

Results

Four key themes were generated. Informants reported the need to generate evidence before and during the programme, appreciating different types of evidence and their importance. Informants identified several key factors that were a catalyst for success, including involvement in the programme and access to individuals and organisations that were crucial for support. However, several barriers were identified at the programme and system levels. Finally, informants identified key supporting processes that enhanced the adoption of their innovations.

Conclusion

SMEs that develop digital health technologies report that, while APs are useful in supporting the adoption of these technologies, some issues remain. These relate to the emphasis on traditional research evidence that remains a challenge for SMEs to generate. Also, several system-level barriers to innovation in healthcare persist. As APs and SMEs continue to create an entrepreneurial ecosystem, there is increased potential for the development of supporting processes and infrastructure to accelerate the efficient and timely adoption of new digital health technologies.

Insect allies-Assessment of a viral approach to plant genome editing

The Insect Allies program of the Defense Advanced Research Projects Agency has already sparked scientific debate concerning technology assessment-related issues, among which the most prevalent is that of dual use. Apart from the issues concerning peaceful applications, the technology also provides the blueprint for a potential bioweapon. However, the combination of a virus-induced genetic modification of crop plants in the field using genetically modified insect vectors poses a greater risk than the hitherto existing use of genetically modified organisms. The technology's great depth of intervention allows a number of sources for hazard and a tendency towards high exposure, but it is also encumbered with notable deficits in knowledge. These issues call for a thorough technology assessment. This article aims to provide an initial characterization from a technology assessment perspective, focusing on potential sources of risk for this novel invasive environmental biotechnology at an early stage of research and development. Integr Environ Assess Manag 2022;18:1488-1499. © 2022 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

Systems analysis of digestate primary processing techniques

In this paper, we performed technology assessment and systems analysis of primary digestate processing techniques to provide a comprehensive analysis of their environmental and cost performance. We compiled more than 100 observations from large-scale biogas plants and considered digestate based on manure, crops and agro-wastes, and food waste under the geographical contexts of Sweden and Belgium. Centrifuge, screw press, and rotary drum were identified as suitable primary processing techniques. We analyzed the climate impact, energy use, and operational cost of digestate management under these scenarios: no processing, partial processing (solid-liquid separation) and full processing (solid-liquid separation followed by ammonia stripping). As expected, the suitable digestate processing varied with the context, transport was often the most critical cost factor, and emissions from storage reduced the climate savings from the use of biofertilizers. However, treating liquid fraction became a main contributor to cost and climate impact under the Belgian conditions. Consequently, the possibility for local application of liquid fraction as biofertilizer could prevent costs and impacts associated with its further treatment. The main novelty of this work is in its integrative and comprehensive approach toward the choices and impacts of primary processing of digestate. We tried to bridge many individual case studies, drew from experiences of biogas plants in different geographical contexts, assessed suitable processing techniques for different digestate types, and analyzed the environmental impacts and cost of digestate management from a life cycle perspective. We believe that such integrated approaches would help decision-making for increased sustainability of the biogas sector.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Assessing clinical utility of machine learning and artificial intelligence approaches to analyze speech recordings in multiple sclerosis: A pilot study

BACKGROUND

An early diagnosis together with an accurate disease progression monitoring of multiple sclerosis is an important component of successful disease management. Prior studies have established that multiple sclerosis is correlated with speech discrepancies. Early research using objective acoustic measurements has discovered measurable dysarthria.

METHOD

The objective was to determine the potential clinical utility of machine learning and deep learning/AI approaches for the aiding of diagnosis, biomarker extraction and progression monitoring of multiple sclerosis using speech recordings. A corpus of 65 MS-positive and 66 healthy individuals reading the same text aloud was used for targeted acoustic feature extraction utilizing automatic phoneme segmentation. A series of binary classification models was trained, tuned, and evaluated regarding their Accuracy and area-under-the-curve.

RESULTS

The Random Forest model performed best, achieving an Accuracy of 0.82 on the validation dataset and an area-under-the-curve of 0.76 across 5 k-fold cycles on the training dataset. 5 out of 7 acoustic features were statistically significant.

CONCLUSION

Machine learning and artificial intelligence in automatic analyses of voice recordings for aiding multiple sclerosis diagnosis and progression tracking seems promising. Further clinical validation of these methods and their mapping onto multiple sclerosis progression is needed, as well as a validating utility for English-speaking populations.

Development, testing, and implementation of a new procedure to assess the clinical added benefit of pharmaceuticals

OBJECTIVES

The reimbursement process for innovative health technologies in Hungary lacks any formalized assessment of clinical added benefit (CAB). The aim of this research is to present the development, retrospective testing, and implementation of a local assessment framework for determining the CAB of cancer treatments at the Department of Health Technology Assessment of the National Institute of Pharmacy and Nutrition in Hungary.

METHODS

The assessment framework was drafted after screening existing methods and a retrospective comparison of local reimbursement dossiers to that of German and French methods. The Magnitude of Clinical Benefit Scale of the European Society for Medical Oncology was chosen to rate the extent of CAB in oncology, as part of a conclusion complemented by the assessment of endpoint relevance and the quality of evidence. Several rounds of retrospective assessments have been conducted involving all clinical assessors, iterated with semistructured discussions to consolidate divergence between assessors. External stakeholders were consulted to provide feedback on the framework.

RESULTS

Retrospective assessments resulted in average more than 75 percent concordance between assessors on each element of the conclusion. Input from ten stakeholders was also incorporated; stakeholders were generally supportive, and they mostly commented on the concept, the elements of the framework, and its implementation.

CONCLUSIONS

The procedure is suitable for routine use in the decision-making process to describe the CAB of antineoplastic technologies in Hungary. Further extension of the framework is required to cover more disease areas for structured and comparable conclusions on CAB of innovative health technologies.

Review of oilfield produced water treatment technologies

Produced water is the wastewater formed when water is brought from subsurface reservoirs during oil or gas extraction. Currently, produced water is mainly treated using conventional trains that contain adsorbates, membrane filters, phase separators and cyclones. This paper reviewed the detailed characteristics of oilfield-produced water and the assessment of multiple technologies at primary, secondary, and tertiary treatments stages. The effectiveness of the treatment technology from the production of waste, energy requirements, usage of chemicals and the treatment effect of contaminants has been discussed. Then a qualitative assessment was presented in terms of energy requirements, robustness, flexibility, waste generation, modularity, and mobility, which has become critical to the development and application prospects of any technology.

"What's Keeping Me Up at Night?" Reflections on the COVID-19 Pandemic in Asia

OBJECTIVES

The 2021 Health Technology Assessment International (HTAi) Asia Policy Forum (APF) aimed to explore the challenges and opportunities presented by the global COVID-19 pandemic for health systems and industry in the Asia region, to discuss how HTA changed during the pandemic, and what its role may be moving forward into a postpandemic era.

METHODS

Discussions during the virtual 2021 APF, informed by a premeeting survey of HTA agencies and industry attendees from the region describing their experience during the pandemic, form the basis of this paper.

RESULTS

During the pandemic, regulatory approval for COVID-related technologies was fast-tracked with fewer evidentiary requirements, and little or no HTA being conducted before these technologies were implemented in health systems in the region. "Living HTA" incorporating real-world evidence (RWE) as it was generated became part of the HTA landscape. In comparison, non-COVID technologies experienced regulatory approval and HTA delays. A major concern of APF members was future pandemic preparedness, and to ensure that lessons are learned from COVID-19. Governments need to continue to invest in innovation and allow early regulatory approvals with the increased use of RWE. Capacity building was identified as a key investment, including capacity in HTA, modeling, as well as local production of necessary supplies and equipment. Finally, collaboration at all levels of the health system was viewed as vital.

CONCLUSIONS

Post-COVID, different forms of HTA must be embraced as part of the new HTA landscape in addition to increased intra- and intercountry collaboration across all sectors of the health system, including regulatory bodies.

Values, principles, strategies, and frameworks underlying patient and public involvement in health technology assessment and guideline development: a scoping review

The importance of patient and public involvement (PPI) is recognized by agencies involved in health technology assessment (HTA) and guideline development. However, a comprehensive overview of the underlying PPI principles, values, strategies, and frameworks is lacking. This scoping review aimed to summarize the available evidence on principles, values, frameworks, and strategies underpinning PPI carried out by agencies involved in HTA and guideline development. A total of twelve records were included, of which four referred to guidelines and eight to HTA. Overall, this review demonstrated a lack of consistency in the definition and application of the concepts of values and principles to PPI in the context of guideline development and HTA. There was significant overlap between values and principles, with some broad themes emerging, such as representation, transparency, relevance, equity, fairness, and reconciling different types of knowledge. Frameworks were typically based on the stages of guideline development or HTA, despite heterogeneity in how stages were labeled and described. Strategies were also mapped to the stages of guideline development and HTA and varied substantially depending on the context and setting. Both strategies and frameworks demonstrated patients and the public can be involved, albeit to a variable extent, throughout the stages of guideline development and HTA. However, frameworks often failed to explicitly link the values and principles with the HTA and guideline development stages through actionable PPI strategies. Further research is warranted to better understand the values, principles, and frameworks underpinning PPI in guideline development and HTA.

Stem and Somatic Cell Monotherapy for the Treatment of Diabetic Foot Ulcers: Review of Clinical Studies and Mechanisms of Action

Diabetic foot ulcer (DFU) is one of the most severe complications of diabetes mellitus, often resulting in a limb amputation. A cell-based therapy is a highly promising approach for an effective DFU treatment. However, there is no consensus regarding the most effective cell type for DFU treatment. Various cell types contribute to chronic wound healing via different mechanisms. For example, application of keratinocytes can stimulate migration of native keratinocytes from the wound edge, while mesenchymal stem cells can correct limb ischemia. To assess the effectiveness of a certain cell type, it should be administered as a monotherapy without other substances and procedures that have additional therapeutic effects. In the present review, we described therapeutic effects of various cells and provided an overview of clinical studies in which stem and somatic cell-based therapy was administered as a monotherapy. Topical application of somatic cells contributes to DFU healing only, while injection of mesenchymal stem cells and mononuclear cells can break a pathophysiological chain leading from insufficient blood supply to DFU development. At the same time, the systemic use of mesenchymal stem cells carries greater risks. Undoubtedly, cell therapy is a potent tool for the treatment of DFU. However, it is vital to conduct further high-quality clinical research to determine the most effective cell type, dosage and way of administration for DFU treatment. Ischemia, neuropathy and neuro-ischemia are underlying factors of diabetic foot ulcer. Stem and somatic cells monotherapy can improve chronic wound healing via different mechanisms.

The development process of economic evaluation guidelines in low- and middle-income countries: a systematic review

OBJECTIVES

To review the development of economic evaluation guidelines (EEGs) in low- and middle-income countries (LMICs), with the goal of assisting those developing EEGs in LMICs.

METHODS

We conducted a systematic search in MEDLINE (Ovid), PubMed, EconLit, Embase (Ovid), the Cochrane Library, and the gray literature until March 2021. We extracted data on the methods used in the EEG development process, the responsible party engaged, and the development team's composition. We conducted a quality assessment, using the Appraisal of Guidelines for Research and Evaluation-Health Systems tool, and then carried out a relative comparison.

RESULTS

Fourteen EEGs and nine studies were identified. In ten countries, the Ministry of Health was responsible for handling the development process. The majority of LMICs who developed EEGs did not explicitly report the discipline of those involved in the process. The developers of EEGs followed four main steps: conducting a review on national guidelines, organizing workshops, and getting support from international experts or from organizations. One-third of the identified EEGs failed to engage multisectoral or multidisciplinary developers, and approximately 14 percent did not follow or report any recommended step.

CONCLUSIONS

This study identified a scarcity of published information related to the development process and the suboptimal quality of included studies. It provides relevant material to support international organizations and developers of guidelines in LMICs in developing EEGs that fit their national context. In addition, this paper recommends a transparent approach to the design of guidelines and to reporting on the methods for developing them.

Innovative and conventional "conservative" technologies for the treatment of uterine fibroids in Italy: a multidimensional assessment

BACKGROUND

To evaluate the potential benefits of the Magnetic Resonance-guided high intensity Focused Ultrasound (MRgFUS) introduction in the clinical practice, for the treatment of uterine fibroids, in comparison with the standard "conservative" procedures, devoted to women who wish to preserve their uterus or enhance fertility: myomectomy and uterine artery embolization (UAE).

METHODS

A Health Technology Assessment was conducted, assuming the payer's perspective (Italian National Healthcare Service). The nine EUnetHTA Core Model dimensions were deeply investigated, by means of i) a literature review; ii) the implementation of health economics tools (useful for uterine fibroids patients' clinical pathway economic evaluation, and budget impact analysis), to define MRgFUS economic and organizational sustainability, and iii) administration of specific questionnaires filled by uterine fibroids' experts, to gather their perceptions on the three possible conservative approaches (MRgFUS, UAE and myomectomy).

RESULTS

Literature revealed that MRgFUS would generate several benefits, from a safety and an efficacy profile, with significant improvement in symptoms relief. Advantages emerged concerning the patients' perspective, thus leading to a decrease both in the length of hospital stay (p-value< 0.001), and in patients' productivity loss (p-value = 0.024). From an economic point of view, the Italian NHS would present an economic saving of - 6.42%. A positive organizational and equity impact emerged regarding the capability to treat a larger number of women, thus performing, on average, 131.852 additional DRGs.

CONCLUSIONS

Results suggest that MRgFUS could be considered an advantageous technological alternative to adopt within the target population affected by uterine fibroids, demonstrating its economic and organisational feasibility and sustainability, with consequent social benefits.

Acceptability of Manufacturer-Proposed Utility Values for NICE Cancer Medicine Appraisals: Analysis of Manufacturers' Information Sources

OBJECTIVES

The National Institute for Health and Care Excellence's (NICE) method guide for technology appraisals (TAs) encourages medicine manufacturers to use the EuroQol 5 Dimensions (EQ-5D) in relevant clinical trials to obtain utility values; however, the EQ-5D may have low sensitivity when compared to disease-specific measures. This study investigated whether the NICE TA committee's acceptance of manufacturer-proposed utility values is dependent on the manufacturers' sources of the utility values.

METHODS

Using publicly available data for 2011-2020, we identified 136 single TAs of cancer medicines, the health-related quality-of-life-measures used in relevant clinical trials, manufacturers' sources of utility values, and the NICE TA committee's acceptance of these values. Fisher's exact tests were performed to compare the acceptability of different value sources and reasons for non-acceptance.

RESULTS

The number of appraisals for which the EQ-5D in the relevant clinical trials was the source of the manufacturer-proposed utility values increased continuously over time. The TA committee's acceptance of values was not dependent on the information source. In cases where a submission for which the information source was the EQ-5D was rejected, the reason was generally related to inappropriate values for the UK population or inappropriate data adjustment, not data reliability.

CONCLUSIONS

Our results demonstrated that according with the NICE's method guide regarding utility values does not guarantee acceptance by the TA committee. Manufacturers must consider in advance possible differences between their clinical trials and clinical practice in the UK and refine plans for EQ-5D measurement in order to obtain convincing evidence.

Applying reflective multicriteria decision analysis to understand the value of therapeutic alternatives in the management of gestational and peripartum anaemia in Spain

BACKGROUND

The objective of the FeminFER project was to assess the value of ferric carboxymaltose following a multicriteria decision analysis in obstetrics and gynaecology in Spain.

METHODS

Ferric carboxymaltose (FCM) and ferrous sulphate were evaluated using the EVIDEM framework. Ten stakeholders participated to collect different perspectives. The framework was adapted considering evidence retrieved with a PICO-S search strategy and grey literature. Criteria/subcriteria were weighted by level of relevance and an evidence-based decision-making exercise was developed in each criterion; weights and scores were combined to obtain the value of intervention relative to each criterion/subcriterion, that were further combined into the Modulated Relative Benefit-Risk Balance (MRBRB).

RESULTS

The most important criterion favouring FCM was Compared Efficacy/Effectiveness (0.183 ± 0.07), followed by Patient Preferences (0.059 ± 0.10). Only Direct medical costs criterion favoured FS (-0.003 ± 0.03). MRBRB favoured FCM; 0.45 ± 0.19; in a scale from -1 to + 1.

CONCLUSIONS

In conclusion, considering the several criteria involved in the decision-making process, participants agreed with the use of FCM according to its MRBRB.

Analysis of consumer comments into PBAC decision-making (2014-9)

OBJECTIVES

The Pharmaceutical Benefits Advisory Committee (PBAC) is an independent expert body that recommends new technologies for listing on the Pharmaceutical Benefits Scheme. Its decision-making process is evidence-based and considers a technology's clinical effectiveness, safety, and cost-effectiveness compared with other technologies. Since 2014, the PBAC has formally taken into account input from those impacted by the technology via an online consumer comments portal and has also reported on received comments in the Public Summary Documents (PSDs). Comments are welcomed from those whose health the technology is trying to improve, as well as carers, clinicians, and organizations. Our objective was to analyze and review consumer comments in the PBAC's decision-making process.

METHODS

We extracted information about consumer comments from the PBAC PSDs from 2014-9. We conducted simple descriptive analyses.

RESULTS

Our findings reveal that two thirds of all submissions did not receive a single consumer comment. Of the remaining third, eight submissions (less than 1 percent) had a substantial number of consumer comments (>500). For these technologies, multiple submissions were required before a recommendation was issued. Submissions spanned multiple therapeutic areas, the therapeutic areas with the most consumer comments were genetic disease, pediatrics, and oncology.

CONCLUSIONS

In the light of our review, we have identified limitations to the current consumer comments process, and after an examination of the processes of other comparable health technology assessment agencies, we have identified a number of improvements that could be made to the PBAC's process to increase consumer engagement.

Building a model of health technology assessment cooperation: lessons learned from EUnetHTA joint action 3

OBJECTIVE

The European Network for Health Technology Assessment (EUnetHTA) was established in 2006. During its final project phase (joint action 3 [JA3]), it undertook an activity to define the scientific and technical principles of a model of health technology assessment (HTA) cooperation in Europe. This policy article presents the key learnings from JA3 partners about developing a model of HTA cooperation.

METHODS

There were two phases to the activity: (i) A descriptive phase to describe the elements of HTA cooperation that were already in place in EUnetHTA JA3 and to identify which elements could be improved or were missing. (ii) An analytic phase synthesizing the data collected to identify learnings from the JA3 and to define the scientific and technical principles for a future model of HTA cooperation.

RESULTS

Learnings for developing HTA cooperation were identified in regard to the framework used to support the cooperation, the HTA activities undertaken, the involvement of internal and external actors, managing decision making and the required human resources and support services needed to undertake HTA activities and to coordinate collaboration.

CONCLUSIONS

These learnings coming from the experiences of the EUnetHTA JA3 are useful to inform discussions on a European Union regulation for HTA cooperation as well as subsequent work to set up the structures that will be defined in the regulation. The findings also have broader applicability and are relevant to individuals, groups, and organizations setting up HTA programs or establishing their own international collaborations.

Economic evaluation guidelines in low- and middle-income countries: a systematic review

OBJECTIVES

To systematically identify the latest versions of official economic evaluation guidelines (EEGs) in low- and middle-income countries (LMICs) and explore similarities and differences in their content.

METHODS

We conducted a systematic search in MEDLINE (Ovid), PubMed, EconLit, Embase (Ovid), the Cochrane Library, and the gray literature. Using a predefined checklist, we extracted the key features of economic evaluation and the general characteristics of EEGs. We conducted a comparative analysis, including a summary of similarities and differences across EEGs.

RESULTS

Thirteen EEGs were identified, three pertaining to lower-middle-income countries (Bhutan, Egypt, and Indonesia), nine to upper-middle-income countries (Brazil, China, Colombia, Cuba, Malaysia, Mexico, Russian Federation, South Africa, and Thailand), in addition to Mercosur, and none to low-income countries. The majority (n = 12) considered cost-utility analysis and health-related quality-of-life outcome. Half of the EEGs recommended the societal perspective, whereas the other half recommended the healthcare perspective. Equity considerations were required in ten EEGs. Most EEGs (n = 11) required the incremental cost-effectiveness ratio and recommended sensitivity analysis, as well as the presentation of a budget impact analysis (n = 10). Seven of the identified EEGs were mandatory for pharmacoeconomics submission. Methodological gaps, contradictions, and heterogeneity in terminologies used were identified within the guidelines.

CONCLUSION

As the importance of health technology assessment is increasing in LMICs, this systematic review could help researchers explore key aspects of existing EEGs in LMICs and explore differences among them. It could also support international organizations in guiding LMICs to develop their own EEGs and improve the methodological framework of existing ones.

Validity and reliability of a smartphone application for knee posture quantification and the effects of external markers on the precision of this measure

BACKGROUND

Technological resources, such as smartphones can contribute to the quantitative assessment of posture.

PURPOSES

Test the validity and reliability of using a postural assessment application to quantify the frontal plane knee posture in orthostatism and to test the influence of the use of external markers on the precision of this measure.

DESIGN

Methodological study.

METHODS

The knee frontal plane posture of 30 volunteers were analyzed by two independent examiners. The photographs were taken with different external marker arrangements. The photographs were analyzed at two moments using the Kinovea software and PhysioCode Posture (PCP) application. Reliability was analyzed using the intraclass correlation coefficient (ICC) between measures with each instrument conducted at two moments with a 7-day interval. Concurrent validity of PCP with Kinovea measure was analyzed using Pearson's correlation coefficient. Standard error measurement (SEM), minimum detectable change (MDC) and Bland Altman plots were analyzed.

RESULTS

PCP demonstrated excellent intra-rater [ICC = 0.92 (95% confidence interval [CI] 0.90-0.93)] and inter-rater [ICC = 0.88 (95%CI 0.85-0.90)] reliability. Concurrent validity analysis showed excellent agreement between PCP and Kinovea software (r = 0.88). The use of markers, independent of positioning, did not influence the measurement properties of measures with both softwares. The SEM was inferior to 1.2°, and the MDC was below 2.85°. No systematic errors were observed in the Bland Altman graphs.

CONCLUSIONS

The use of PCP application to measure knee posture was valid and demonstrated excellent intra- and inter-rater reliability levels. The use of external markers did not influence the measurement.

Identification and selection of health technologies for assessment by agencies in support of reimbursement decisions in Latin America

OBJECTIVE

There is no health system that has the resources to evaluate all technologies. The presence of a clear process to prioritize health technologies for assessment by health technology assessment (HTA) agencies is a good practice principle recognized at the international level. The objective of Health Technology Assessment International's 2020 Latin American Policy Forum (LatamPF) was to explore how to improve the way HTA agencies in Latin America identify and prioritize technologies for assessment.

METHODS

This paper is based on a background document, a survey, and the deliberations of the members of the LatamPF (forty-six participants from eleven countries) using a design thinking methodology.

RESULTS

Participants agreed that a lack of clear prioritization mechanisms results in HTA processes and decisions that are perceived to be of low transparency and overly exposed to political or interest group pressures. The LatamPF identified barriers and recommended actions to improve HTA prioritization mechanisms in Latin America. The criteria identified as the most important to be taken into consideration by HTA agencies in the region when prioritizing a technology for assessment were: the burden of illness, the potential clinical benefit, the alignment with national health priorities, the potential impact on equity, a lack of treatment alternatives for patients, and the potential economic impact.

CONCLUSIONS

Forum participants agreed that the establishment of transparent prioritization processes is a key element for all health systems. Improvements in these processes will strengthen HTA and provide greater legitimacy to decision making.

Assessing the Impact of an Original Soft Communicative Robot in a Nursing Home in Japan: Will Softness or Conversations Bring more Smiles to Older People?

It has been reported that robotics-aided care can contribute to enhancing older people's social participation and quality of life in nursing homes, while simultaneously reducing the burden on care professionals at nighttime. Due to increasing demand for social care and the relative workforce shortage, it is likely that a greater number and variety of robots will be introduced and implemented in the future. While the benefits of applying robots and assistive technologies are recognized, the current limitations and weaknesses have also been identified. One of these is the difficulty associated with a user-centered design, involving older adults with impaired cognitive and sensory abilities in nursing homes. In order to overcome this challenge, a project was carried out to develop a soft and compact bedside communication robot with an input/output device, connected to existing technologies (e.g. monitoring camera, biological sensor). Drawing on the principle of gemba (deference to frontline professionals' experience, expertise and skills), users' feedback was reflected in the iterative steps of robot development. The original soft and communicative robot was introduced and its effectiveness was tested by measuring older people's reactions and changes in their behaviors and engagement levels. The article reports the development process and results of a small-scale evaluation study, comparing the impact of this original soft-type robot with and without its communicative functions. The human-robot interactions were captured on video, and the analysis revealed that while communicative robots reduced the psychosocial burden on older adults, positive emotional, verbal, visual and behavioral engagement was generated with the help of the non-verbal plush toy.

Using INTEGRATE-HTA for the assessment of videorasterstereography in idiopathic scoliosis

OBJECTIVE

For the assessment of videorasterstereography compared with radiography for monitoring idiopathic scoliosis, new tools given by the INTEGRATE-HTA methodology were included within the frame set by the authority. In contrast to the usual linear approach by covering the single domains' safety and efficacy, costs, and ethical, legal, socio-cultural and organizational issues (ELSOI) separately, the methodology is based on the assumption of interactions of different aspects of the domains. From the very beginning, these interactions are captured systematically using various tools and are repeatedly discussed with stakeholders.

METHODS

Relevant databases were systematically searched for studies and question catalogues were processed in order to be able to search for specific aspects in orienting bibliographic research. An initial logic model was developed and successively expanded. A synoptic table showing multiple mapping of aspects to domains, a complexity checklist, and a semantic complex highlighting the socio-cultural impact of the disease were newly developed.

RESULTS

Four diagnostic studies with high risk of bias were included. For the clinical outcome, Cobb angle standard deviations up to 8.2 degrees and average measurement differences up to 8.8 degrees were determined. These differences in measurement accuracy correspond to those observed for purely radiographic measurements in other studies. In addition to freedom from radiation as the greatest advantage, there were clear arguments in favor of videorasterstereography when considering ELSOI, for example less harm.

CONCLUSIONS

Using INTEGRATE-HTA contributes to identify interactions between domains and to significantly expand the perspective on a technology. It improves patient-centered understanding and facilitates discussions.

Implementation of an Electronic Medication Management System in a large tertiary hospital: a case of qualitative inquiry

BACKGROUND

Hospitals across Australia are implementing Clinical Information Systems, e.g. Electronic Medication Management Systems (EMMS) at a rapid pace to moderate health services. The benefits of the EMMS depend on the acceptance of the system by the clinicians. The study hospital used a unique patient-centric implementation strategy that was based on the guiding principle of "one patient, one chart" to avoid a patient being on a hybrid medication chart. This paper aims to study the factors facilitating or hindering the adoption of the EMMS as viewed by clinicians and the implementation team.

METHODS

Four focus groups (FG), one each for (1) doctors, (2) nurses, (3) pharmacists, and (4) implementation team, were conducted. A guide for the FG was based on the Unified Theory of Acceptance and Use of Technology (UTAUT).

RESULTS

A total of 23 unique subthemes were identified and were grouped into five main themes (1) implementation strategy, (2) organisational outcome of EMMS, (3) individual impact of EMMS, (4) IT product, and (5) organisational culture. Clinicians reported improvement in their workflow efficiency post-EMMS implementation. They also reported some challenges in using the EMMS that centered around the area of infrastructure, technical and design issues. Additionally, the implementation team highlighted two crucial factors influencing the success of EMMS implementation, namely: (1) the patient-centric implementation strategy, and (2) the organisation readiness.

CONCLUSION

Overall, this study outlines the implementation process of the EMMS in a large healthcare facility from the clinicians' and the implementation team's perspectives using UTAUT model. The result suggests that clinicians' acceptance of the EMMS was highly influenced by the unique implementation strategy (namely, patient-centric approach and clinical leadership in the implementation team). Whereas the level of adoption of EMMS by clinicians was determined by their level of perceived and realised benefits. On the other hand, a number of barriers to the adoption of EMMS were discovered, namely, general training instead of customised training based on local needs, technical and design issues and lack of availability of computer systems. It is suggested that promptly resolving these issues can improve the adoption of the EMMS.

How to integrate evidence from patient preference studies into health technology assessment: a critical review and recommendations

Health technology assessment (HTA) agencies vary in their use of quantitative patient preference data (PP) and the extent to which they have formalized this use in their guidelines. Based on the authors' knowledge of the literature, we identified six different PP "use cases" that integrate PP into HTA in five different ways: through endpoint selection, clinical benefit rating, predicting uptake, input into economic evaluation, and a means to weight all HTA criteria. Five types of insight are distinguished across the use cases: understanding what matters to patients, predicting patient choices, estimating the utility generated by treatment benefits, estimating the willingness to pay for treatment benefits, and informing distributional considerations. Summarizing the literature on these use cases, we recommend circumstances in which PP can add value to HTA and the further research and guidance that is required to support the integration of PP in HTA. Where HTA places more emphasis on clinical outcomes, novel endpoints are available; or where there are already many treatment options, PP can add value by helping decision makers to understand what matters to patients. Where uptake is uncertain, PP can be used to estimate uptake probability. Where indication-specific utility functions are required or where existing utility measures fail to capture the value of treatments, PP can be used to generate or supplement existing utility estimates. Where patients are paying out of pocket, PP can be used to estimate willingness to pay.

Systems thinking in health technology assessment: a scoping review

OBJECTIVE

Our objective was to assess how, and to what extent, a systems-level perspective is considered in decision-making processes for health interventions by illustrating how studies define the boundaries of the system in their analyses and by defining the decision-making context in which a systems-level perspective is undertaken.

METHOD

We conducted a scoping review following the Joanna Briggs Institute methodology. MEDLINE, EMBASE, Cochrane Library, and EconLit were searched and key search concepts included decision making, system, and integration. Studies were classified according to an interpretation of the "system" of analysis used in each study based on a four-level model of the health system (patient, care team, organization, and/or policy environment) and using categories (based on intervention type and system impacts considered) to describe the decision-making context.

RESULTS

A total of 2,664 articles were identified and 29 were included for analysis. Most studies (16/29; 55%) considered multiple levels of the health system (i.e., patient, care team, organization, environment) in their analysis and assessed multiple classes of interventions versus a single class of intervention (e.g., pharmaceuticals, screening programs). Approximately half (15/29; 52%) of the studies assessed the influence of policy options on the system as a whole, and the other half assessed the impact of interventions on other phases of the disease pathway or life trajectory (14/29; 48%).

CONCLUSIONS

We found that systems thinking is not common in areas where health technology assessments (HTAs) are typically conducted. Against this background, our study demonstrates the need for future conceptualizations and interpretations of systems thinking in HTA.

Integrating Early Economic Evaluation into Target Product Profile development for medical tests: advantages and potential applications

Target Product Profiles (TPPs) outline the characteristics that new health technologies require to address an unmet clinical need. To date, published TPPs for medical tests have focused on infectious diseases, mostly in the context of low- and middle-income countries. Recently, there have been calls for a broader use of TPPs as a mechanism to ensure that diagnostic innovation is aligned with clinical needs, yet the methodology underpinning TPP development remains suboptimal. Here, we propose that early economic evaluation (EEE) should be integrated within the TPP methodology to create a more rigorous framework for the development of "fit-for-purpose" tests. We discuss the potential benefits that EEE could bring to the core activities underpinning TPP development-scoping, drafting, consensus building, and updating-and argue that using EEE to help inform TPPs provides a more objective, evidence-based, and transparent approach to defining test specifications.

Postlaunch evidence-generation studies for medical devices in Spain: the RedETS approach to integrate real-world evidence into decision making

The Monitoring Studies (MS) program, the approach developed by RedETS to generate postlaunch real-world evidence (RWE), is intended to complement and enhance the conventional health technology assessment process to support health policy decision making in Spain, besides informing other interested stakeholders, including clinicians and patients. The MS program is focused on specific uncertainties about the real effect, safety, costs, and routine use of new and insufficiently assessed relevant medical devices carefully selected to ensure the value of the additional research needed, by means of structured, controlled, participative, and transparent procedures. However, despite a clear political commitment and economic support from national and regional health authorities, several difficulties were identified along the development and implementation of the first wave of MS, delaying its execution and final reporting. Resolution of these difficulties at the regional and national levels and a greater collaborative impulse in the European Union, given the availability of an appropriate methodological framework already provided by EUnetHTA, might provide a faster and more efficient comparative RWE of improved quality and reliability at the national and international levels.

Overview of information retrieval in a hospital-based health technology assessment center in a Swedish region

OBJECTIVES

This article aims to provide a brief review of information retrieval and hospital-based health technology assessment (HB-HTA) and describe library experiences and working methods at a regional HB-HTA center from the center's inception to the present day.

METHODS

For this brief literature review, searches in PubMed and LISTA were conducted to identify studies reporting on HB-HTA and information retrieval. The description of the library's involvement in the HTA center and its working methods is based on the authors' experience and internal and/or unpublished documents.

RESULTS

Region Västra Götaland is the second largest healthcare region in Sweden and has had a regional HB-HTA center since 2007 (HTA-centrum). Assessments are performed by clinicians supported by HTA methodologists. The medical library at Sahlgrenska University Hospital works closely with HTA-centrum, with one HTA librarian responsible for coordinating the work.

CONCLUSION

In the literature on HB-HTA, we found limited descriptions of the role librarians and information specialists play in different units. The librarians at HTA-centrum play an important role, not only in literature searching but also in abstract and full-text screening.

Development of an international template to support patient submissions in Health Technology Assessments

OBJECTIVES

To develop an international template to support patient submissions in Health Technology Assessments (HTAs). This was to be based on the experience and feedback from the implementation and use of the Scottish Medicines Consortium's (SMC) Summary Information for Patient Groups (SIP).

METHODS

To gather feedback on the SMC experience, web-based surveys were conducted with pharmaceutical companies and patient groups familiar with the SMC SIP. Semistructured interviews with representatives from HTA bodies were undertaken, along with patient group discussions with those less familiar with the SIP, to explore issues around the approach. These qualitative data informed the development of an international SIP template.

RESULTS

Survey data indicated that 82 percent (18 of 22 respondents) of pharmaceutical company representatives felt that the SIP was worthwhile; 88 percent (15/17) of patient group respondents found the SIP helpful. Both groups highlighted the need for additional support and guidance around plain language summaries. Further suggestions included provision of a glossary of terms and cost-effectiveness information. Patient group interviews supported the survey findings and led to the development of a new template. HTA bodies raised potential challenges around buy-in, timing, and bias connected to the SIP approach.

CONCLUSIONS

The international SIP template is another approach to support deliberative processes in HTA. Although challenges remain around writing summaries for lay audiences, along with feasibility considerations for HTA bodies, the SIP approach should support more meaningful patient involvement in HTAs.

A voice to be heard: patient and public involvement in health technology assessment and clinical practice guidelines in Malaysia

Patient and public involvement (PPI) in health technology assessment (HTA) is widely promoted to ensure that all health-related decisions are made after taking into consideration the viewpoints of important stakeholders. In Malaysia, patients or their representatives have been involved in the development of HTA and Clinical Practice Guidelines (CPG) since 2009 and their influences have been growing steadily over the years. This paper aimed to describe the journey, achievements, challenges, and future direction of the PPI throughout all stages of the development and implementation of HTA and CPG in Malaysia. Currently, in Malaysia, patients or their representatives are mainly involved during the initial development of HTA and CPG drafts as well as during the internal and external reviews. Additionally, they are also encouraged to be involved during the implementation of HTA and CPG recommendations. Although their involvement in this aspect has slowly increased over time, challenges remain in the form of limited representativeness of selected patients or carers, uncertainty on the level of patient involvement allowed during the HTA/CPG development processes, and limited health literacy, which affect their ability to contribute meaningfully throughout the processes. Continuous improvement in these processes is important as patients or their representatives play a pivotal role in ensuring transparency, accountability, and credibility throughout the HTA/CPG development and decision-making processes.

Neoadjuvant and adjuvant end-points in health technology assessment in oncology

Health technology assessment (HTA) of clinical and economic value of a new intervention is an integral step in providing the access of patients to innovative cancer care and treatment. Overall survival (OS) is the preferred criterion for demonstrating the therapeutic efficacy in HTA given its direct clinical and patient relevance. However, with often long life expectancy of patients with early cancer, analysis of OS becomes less practical. Partially due to this reason, pathological complete response (pCR) and time-to-event end-points like disease-free survival are frequently incorporated into the pivotal clinical trials in the neoadjuvant and adjuvant settings. However, there exists a discrepancy between different national HTA bodies regarding the acknowledgement of patient relevance of these end-points. In this article, we analysed the perspectives of patients on different aspects of end-points used in clinical trials in early cancer. Gathered evidence strongly suggests that complete tumour eradication and reduced risk of recurrence provide important psychological benefits thus signifying that pCR and time-to-event end-points are directly relevant to patients. Additionally, we reviewed opinions on patient relevance of neoadjuvant and adjuvant therapy end-points adopted by HTA bodies during the recent evaluations. We found that improvements in end-points used in the adjuvant setting were commonly considered as valuable to patients. In contrast, opinions on patient relevance of neoadjuvant therapy end-points varied between the national HTA bodies. Universal acknowledgement of patient relevance of therapeutic end-points for early cancer by HTA bodies is necessary to balance the inequality in uptake of innovative therapies into national healthcare systems.

An institutional ethnographic analysis of public and patient engagement activities at a national health technology assessment agency

OBJECTIVE

The practice of public and patient engagement (PPE) in health technology assessment (HTA) has spread worldwide, yet gaps in knowledge remain. We carried out an institutional ethnography of the Canadian Agency for Drugs and Technologies in Health (CADTH) public and patient involvement in HTA.

METHODS

The research took place over 15 months and included observational work in the institutional settings, text review, and interviews with individuals working for or involved with the agency.

RESULTS

We found that despite demonstrated commitment to PPE, organizational history, governance structure, and practices were impediments to a unified approach to PPE. Unclear role descriptions for committee members and differences in philosophy and priority given to PPE across the organization presented challenges to effective participation. The high degree of value given to evidence-based principles at times conflicted with meaningful integration of patient input. A lack of clear goals and processes, roles, and differential treatment of evidence in PPE served to minimize the importance of patient experiences and to displace their validity. An acknowledgment of conflicts between multiple epistemic traditions at work within HTA activities may strengthen organizational approaches to PPE.

CONCLUSION

HTA organizations can learn from this study by reflecting on the challenges described and the recommendations offered to address them. We suggest solidifying CADTH's commitment to PPE with clear agency-wide roles and direction, values, and outcomes, a comprehensive framework, and policy and procedures. An acknowledgment of diverse epistemic traditions, as well as leadership and expertise in PPE, will strengthen CADTH's PPE activities and sustain its leadership position in the HTA field.

Evaluation of the impact of patient input in health technology assessments at NICE

OBJECTIVE

Accounts of patient experiences are increasingly used in health technology assessment (HTA) processes. However, we know little about their impact on the decision-making process. This study aims to assess the level and the type of impact of patient input to highly specialised technologies (HSTs) and interventional procedures (IPs) guidance at the National Institute for Health and Care Excellence (NICE).

METHODS

A questionnaire was developed to capture quantitative and qualitative data on the amount and type of impact of patient input into NICE HTAs. It was completed by committee members of the guidance-producing programs after a discussion of the considered topics. The data were analyzed by topic and overall, for each program, and compared across programs.

RESULTS

Patient input was assessed on ten pieces of HST guidance published between January 2015 and November 2019, and on twenty-six pieces of IP guidance scoped between February 2016 and October 2018. A total of 96 responses were collected for HST and 440 for IP. The level of impact of patient input was higher for HST than for IP. For HST, no respondents stated that it had no impact, whereas in IP, 35 percent of respondents did. The most common types of impact found for HST and IP were that it helped interpret the other evidence and that it provided new evidence.

CONCLUSIONS

The impact of patient input is not necessarily explicit in changing recommendations, but it provides context, reassurance, and new information to the committee for the decision-making process in HTAs.

Creating a patient and community advisory committee at the Canadian Agency for Drugs and Technologies in Health

In recognition of patients' roles using, and contributing to, a publicly funded health system, the Canadian Agency for Drugs and Technologies in Health (CADTH) created a Patient and Community Advisory Committee. Twelve members bring lived experiences of chronic illness, progressive illness, mental illness, trauma, traveling long distances for treatment, and caregiving to an ill child, parent, or spouse. Members contribute their own insights and ideas but do not represent specific organizations or viewpoints. This paper explores how CADTH determined the committee's role, whether to have individuals or organizations as members, and how to recruit for diversity. The creation of this committee is changing how CADTH engages with patients.

Role of patients' organizations in Health Technology Assessment: a Habermasian system and lifeworld perspective

Patient and public involvement in Health Technology Assessment (HTA) is gaining increased interest among research and policy communities. Patients' organizations represent an important link between individual patients and the health system. Social theories are increasingly being used to explain doctor-patient-system interactions, expanding understanding beyond the mere clinical perspective. In this sense, patient involvement in HTA can also be considered through the Habermas's theory of communicative action. From a Habermasian perspective, HTA as part of the instrumental rationality contributes to an increased efficiency of resource use within the system; however, such rationalization threatens to colonize the lifeworld by making it "increasingly state administered with attenuated possibilities for communicative action as a result of the commercialization and rationalization in terms of immediate returns." Using Habermasian system/lifeworld framework, this paper explores opportunities and obstacles to patient involvement in HTA, whereby trying to understand current and possible roles of patients' organizations as a mediating force between HTA as a function of the system and the lifeworld represented by patients.

Patient and caregiver engagement in the Patient-Centered Outcomes Research Institute (PCORI) Health Care Horizon Scanning System (HCHSS) process

OBJECTIVE

The Patient-Centered Outcomes Research Institute (PCORI) horizon scanning system is an early warning system for healthcare interventions in development that could disrupt standard care. We report preliminary findings from the patient engagement process.

METHODS

The system involves broadly scanning many resources to identify and monitor interventions up to 3 years before anticipated entry into U.S. health care. Topic profiles are written on included interventions with late-phase trial data and circulated with a structured review form for stakeholder comment to determine disruption potential. Stakeholders include patients and caregivers recruited from credible community sources. They view an orientation video, comment on topic profiles, and take a survey about their experience.

RESULTS

As of March 2020, 312 monitored topics (some of which were archived) were derived from 3,500 information leads; 121 met the criteria for topic profile development and stakeholder comment. We invited fifty-four patients and caregivers to participate; thirty-nine reviewed at least one report. Their perspectives informed analyst nominations for fourteen topics in two 2019 High Potential Disruption Reports. Thirty-four patient stakeholders completed the user-experience survey. Most agreed (68 percent) or somewhat agreed (26 percent) that they were confident they could provide useful comments. Ninety-four percent would recommend others to participate.

CONCLUSIONS

The system has successfully engaged patients and caregivers, who contributed unique and important perspectives that informed the selection of topics deemed to have high potential to disrupt clinical care. Most participants would recommend others to participate in this process. More research is needed to inform optimal patient and caregiver stakeholder recruitment and engagement methods and reduce barriers to participation.

Receiver Operating Characteristic (ROC) Analysis of Image Search-and-Localize Tasks

RATIONALE AND OBJECTIVES

Receiver operating characteristic (ROC) analysis for the common image search-and-localize task, in which readers search an image for lesion or lesions not knowing a priori any exists, has been studied for over four decades. However, a satisfactory solution seems elusive.

MATERIALS AND METHODS

We show that the ROC curve predictive of clinical outcomes where readers are penalized appropriately for not correctly localizing known lesions cannot be obtained because it is a missing data problem. Further, this ROC curve is between the case-based ROC curve where readers are not penalized and the lesion-based ROC curve where penalty applies. Moreover, the lesion-based ROC curve is the LROC curve proposed by Starr et al. We show maximum-likelihood (ML) estimation of the LROC curve, validation of this procedure with Monte Carlo simulations, and its application to reader ROC datasets.

RESULTS

Monte Carlo simulations validated ML estimation of area under the LROC curve (AUC) and its variance. Example applications showed that ML estimate of LROC curve fits experimental datasets.

CONCLUSION

The ROC curve predictive of clinical performance cannot be estimated from reader ROC data alone because it is a missing data problem, and is between the case-based ROC curve where readers are not penalized for not correctly identifying known lesions and the lesion-based ROC curve where penalty applies. The lesion-based ROC curve is the LROC curve proposed by Starr et al. and can be estimated via ML estimation.

Twenty years of patient involvement in health technology assessment at NICE: an evolving story

From its inception in 1999, the National Institute for Health and Care Excellence (NICE) committed to including the expertise, experiences, and perspectives of lay people, patients and carers, and patient organizations in its health technology assessments (HTAs). This is our story of patient involvement in HTA: from early methods designed for use when assessing medicines, widening to address the different requirements of HTAs for interventional procedures, medical technologies, and diagnostic technologies. We also chart the evolution and development of all our patient involvement methods over the past 20 years through regular evaluation and by responding to external challenge. However, we know that processes and methods alone are not enough. Through case studies we demonstrate the value of patient involvement in HTA and highlight the unique perspectives and experiences that patients bring to HTA committees. Finally, we discuss the underpinning principles and commitments that have made NICE a world leader in delivering meaningful and legitimate patient involvement.

Demonstrating the influence of HTA: INAHTA member stories of HTA impact

A central function of health technology assessment (HTA) agencies is the production of HTA reports to support evidence-informed policy and decision making. HTA agencies are interested in understanding the mechanisms of HTA impact, which can be understood as the influence or impact of HTA report findings on decision making at various levels of the health system. The members of the International Network of Agencies for HTA (INAHTA) meet at their annual Congress where impact story sharing is one important activity. This paper summarizes four stories of HTA impact that were finalists for the David Hailey Award for Best Impact Story.The methods to measure impact include: document review; claims analysis and review of reimbursement status; citation analysis; qualitative evaluation of stakeholders' views; and review of media response. HTA agency staff also observed changes in government activities and priorities based on the HTA. Impact assessment can provide information to improve the HTA process, for example, the value of patient and clinician engagement in the HTA process to better define the assessment question and literature reviews in a more holistic and balanced way.HTA reports produced by publicly funded HTA agencies are valued by health systems around the globe as they support decision making regarding the appropriate use, pricing, reimbursement, and disinvestment of health technologies. HTAs can also have a positive impact on information sharing between different levels of government and across stakeholder groups. These stories show how HTA can have a significant impact, irrespective of the health system and health technology being assessed.

Evaluation of patient involvement strategies in health technology assessment in Spain: the viewpoint of HTA researchers

OBJECTIVE

The Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS) defined a patient involvement (PI) framework for health technology assessment (HTA) activities in 2016. The aim of this study is to evaluate the process and impact of those PI initiatives that were implemented in the first year following the publication of this new framework.

METHODS

A survey was sent to those HTA researchers who implemented PI in RedETS projects. Responses were reviewed by two authors. An adapted thematic analysis was performed and the results were later discussed by all authors.

RESULTS

Six responses from six agencies/units were analyzed. The objectives of PI initiatives were the following: inclusion of patient perspectives, preferences and values; elicitation of important health outcomes measures; and barriers, facilitators, or suggestions for implementation. Different methods were used for PI: surveys, focus groups, in depth interviews, and participation in an expert panel. Five main themes emerged: (i) challenges with the recruitment process, (ii) needs identified, (iii) impact of PI, (iv) lessons learned, and (v) suggestions for the future.

CONCLUSIONS

PI initiatives within the RedETS framework were tailored to each HTA project, its specific goals and the individual needs and resources of each HTA agency. The results also pointed out how PI has a relevant impact that has enriched RedETS products providing key information on experiences, values, and preferences of patients, contributions that benefit the HTA and the process of drawing up recommendations. The main challenges were related to recruitment processes and capacity building.

Health technology assessment of biosimilars worldwide: a scoping review

BACKGROUND

Health technology assessment (HTA) should provide an assessment of a technology's effects on health and of the related social, economic, organisational and ethical issues. HTA reports on biosimilars can specifically assess their immunogenicity, their extrapolation to one or more conditions, and the risks of interchangeability and substitution. We aimed to complete a scoping review within the context of HTA organisations to synthesise HTA reports on biosimilars and to map the extension, scope and methodological practices.

MAIN BODY

A scoping review methodology was applied. The sources for biosimilars HTA reports were database searches and grey literature from HTA organisation websites up to June 2019. HTA reports of biosimilars were classified as full HTA, mini-HTA or rapid reviews. Data were extracted and recorded on a calibrated predefined data form. We identified 70 HTA reports of biosimilars of 16 biologic products (65.71% in 2015-2018) produced by 13 HTA organisations from 10 countries; 2 full HTAs, 4 mini-HTAs and 64 rapid reviews met the inclusion criteria. Almost all the rapid reviews gave no information regarding any evidence synthesis method and approximately half of the rapid reviews did not appraise the risk of bias of primary studies or the overall quality of evidence. All full-HTAs and mini-HTAs addressed organisational, ethical, social and legal considerations, while these factors were assessed in less than half of the rapid reviews. The immunogenicity and extrapolation of one or more conditions were often considered. The majority of full-HTAs and mini-HTAs contained an assessment of switching and a discussion of an educational approach about biosimilars. No HTA report rejected the adoption/reimbursement of the biosimilar assessed.

CONCLUSION

HTA of biosimilars are emerging in the context of HTA organisations and those that exist often duplicate reports of the same biosimilar. Most HTA reports of biosimilars do not conduct a systematic literature review or consider economic issues. No report has rejected the adoption/reimbursement of biosimilars. There is a need to standardise the minimum criteria for the development of HTA on biosimilars to ensure a better understanding and better decision-making.