Collective health research assessment: developing a tool to measure the impact of multistakeholder research initiatives

Background

The need to more collaboratively measure the impact of health research and to do so from multidimensional perspectives has been acknowledged. A scorecard was developed as part of the Collective Research Impact Framework (CRIF), to engage stakeholders in the assessment of the impacts of health research and innovations. The purpose of this study was to describe the developmental process of the MULTI-ACT Master Scorecard (MSC) and how it can be used as a workable tool for collectively assessing future responsible research and innovation measures.

Methods

An extensive review of the health research impact literature and of multistakeholder initiatives resulted in a database of 1556 impact indicators. The MSC was then cocreated by engaging key stakeholders and conducting semi-structured interviews of experts in the field.

Results

The MSC consists of five accountability dimensions: excellence, efficacy, economic, social and patient-reported outcomes. The tool contains 125 potential indicators, classified into 53 impact measurement aspects that are considered the most relevant topics for multistakeholder research and innovation initiatives when assessing their impact on the basis of their mission and their stakeholders’ interests. The scorecard allows the strategic management of multistakeholder research initiatives to demonstrate their impact on people and society. The value of the tool is that it is comprehensive, customizable and easy to use.

Conclusions

The MSC is an example of how the views of society can be taken into account when research impacts are assessed in a more sustainable and balanced way. The engagement of patients and other stakeholders is an integral part of the CRIF, facilitating collaborative decision-making in the design of policies and research agendas. In policy making, the collective approach allows the evaluation perspective to be extended to the needs of society and towards responsible research and innovation. Multidimensionality makes research and innovations more responsive to systemic challenges, and developing more equitable and sustainable health services.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00856-9.

Suppressors of cytokine signaling-1 and -6 associate with and inhibit the insulin receptor. A potential mechanism for cytokine-mediated insulin resistance

Insulin resistance contributes to a number of metabolic disorders, including type II diabetes, hypertension, and atherosclerosis. Cytokines, such as tumor necrosis factor-alpha, interleukin-1 beta, and interleukin-6, and hormones, such as growth hormone, are known to cause insulin resistance, but the mechanisms by which they inhibit the cellular response to insulin have not been elucidated. One mechanism by which these agents could cause insulin resistance is by inducing the expression of cellular proteins that inhibit insulin receptor (IR) signaling. Suppressors of cytokine signaling (SOCS) proteins are negative regulators of cytokine signaling pathways, the expression of which is regulated by certain cytokines. SOCS proteins are therefore attractive candidates as mediators of cytokine-induced insulin resistance. We have found that SOCS-1 and SOCS-6 interact with the IR when expressed in human hepatoma cells (HepG2) or in rat hepatoma cells overexpressing the human IR. In SOCS-1-expressing cells, insulin treatment increases the extent of interaction with the IR, whereas in SOCS-6-expressing cells the association with the IR appears to require insulin treatment. SOCS-1 and SOCS-6 do not inhibit insulin-dependent IR autophosphorylation, but both proteins inhibit insulin-dependent activation of ERK1/2 and protein kinase B in vivo and IR-directed phosphorylation of IRS-1 in vitro. These results suggest that SOCS proteins may be inhibitors of IR signaling and could mediate cytokine-induced insulin resistance and contribute to the pathogenesis of type II diabetes.

Chloroplast microsatellites and mitochondrial nad1 intron 2 sequences indicate congruent phylogenetic relationships among Swiss stone pine (Pinus cembra), Siberian stone pine (Pinus sibirica), and Siberian dwarf pine (Pinus pumila)

We studied the phylogenetic relationships among the three stone pine species, Pinus cembra, P. sibirica, and P. pumila, using chloroplast microsatellites and mitochondrial nad1 intron 2 sequences. The three chloroplast microsatellite loci combined into a total of 18 haplotypes. Fourteen haplotypes were detected in 15 populations of P. cembra and one population of P. sibirica, five of which were shared between the two species, and the two populations of P. pumila comprised four species-specific haplotypes. Mitochondrial intron sequences confirmed this grouping of species. Sequences of P. cembra and P. sibirica were identical, but P. pumila differed by several nucleotide substitutions and insertions/deletions. A repeat region found in the former two species showed no intraspecific variation. These results indicate a relatively recent evolutionary separation of P. cembra and P. sibirica, despite their currently disjunct distributions. The species-specific chloroplast and mitochondrial markers of P. sibirica and P. pumila should help to trace the hybridization in their overlapping distribution area and to identify fossil remains with respect to the still unresolved postglacial re-colonization history of these two species.

Risk-adapted therapy for clinical stage I-II Hodgkin's disease: 7-years results of radiotherapy alone for low-risk disease, and ABVD and radiotherapy for high-risk disease

Treatment outcomes were documented for 204 adult patients with clinical Stage I-II Hodgkin's disease who were treated with risk-adapted ABVD (doxorubicin, bleomycin, vinblastine and dacarbazine) and radiotherapy (RT) at the Toronto-Sunnybrook Regional Cancer Centre between 1984 and 1994. Forty-nine patients with clinical Stage I disease (excluding bulky mediastinal presentations) and 50 patients with a combination of clinical Stage IIA disease, age 50 years or less, and favourable pathology (lymphocyte predominant or nodular sclerosing histology) were identified as low risk and treated with RT alone to 35 Gy. One hundred and five high-risk patients were treated with chemotherapy (86 with ABVD) followed by RT to 25 Gy. The 7-year cause-specific, overall and disease-free survivals were 95%, 90% and 75% respectively for the low-risk cohort, and 91%, 90% and 88% respectively for the high-risk cohort. In-field relapses accounted for 50% of the failures in both groups. Sixteen of 24 (67%) patients with RT failure and 6/14 (43%) with combined modality therapy (CMT) failure were salvaged. Twenty-eight per cent of the patients treated with RT and 21% of those treated with CMT developed hypothyroidism by 7 years. Fatal complications were recorded in 6% of the low-risk patients managed with RT and 8% of high-risk patients managed with CMT. Septic death and second malignancy accounted for the majority of treatment-related fatalities. Risk-adapted therapy emphasizing RT alone for selected patients with favourable prognostic factors and CMT based on ABVD provides excellent long-term disease control. Further treatment refinements, including the wider application of CMT with lower doses of chemotherapy and RT, will be required to reduce the rate of fatal complications to more acceptable levels.

Haplotype variation in a mitochondrial tandem repeat of Norway spruce (Picea abies) populations suggests a serious founder effect during postglacial re-colonization of the western Alps

Populations from 13 elevational transects of Norway spruce [Picea abies (L.) Karst] across the Alpine range were sampled to elucidate the geographical pattern of genetic variation in relation to postglacial re-colonization and to study elevational effects on haplotypic diversity. We assessed fragment length variation in a tandem repeat region of the mitochondrial (mt) nad1 intron 2. This maternally inherited genetic marker is suited to infer migration as it is dispersed by seed only. A total of 10 haplotypes was found, most of which were due to repeat copy number variation. An analysis of molecular variance (amova) showed that overall population differentiation was high (F(ST)=0.41), and it revealed a significant differentiation between monomorphic western and moderately to highly variable eastern Alpine populations. This phylogeographic pattern may be explained by a founder effect during postglacial re-colonization. An early arriving haplotype, assumed to originate from a western Carpathian refugium, could expand into suitable habitats, reducing the chances for establishment of subsequently arriving haplotypes. On the other hand, the high variation in populations within an Italian transect of the south-eastern Alps may be the consequence of merging migration pathways from and close distance to putative glacial refugia, most likely those assumed in the Carpathian mountains and on the Balkan peninsula or possibly in the central plains of Italy. An effect of elevation on haplotypic diversity was not evident, though a low, but significant, partition of total genetic variation was attributed to among-population variation in one Italian transect. Various factors, such as vertical seed dispersal and forest management, may account for blurring an otherwise established pattern of genetic variation on a small geographical scale.

Personal Problem-Solving Scoring of TAT Responses: Known-Groups Validation

Two studies employed a known-groups validation strategy to evaluate a Thematic Apperception Test (TAT) scoring system purported to measure personal problem-solving skills, the Personal Problem-Solving System (PPSS). In Study 1 clinicians rated the records of 46 mental health outpatients for the presence of personal problem-solving skills deficits. Verbatim TAT transcripts from these patients were blindly scored using the PPSS. Participants predicted to demonstrate problem-solving deficits obtained lower PPSS scores. In Study 2 a psychiatric sample (n = 47) and a community-based comparison group (n = 47) completed a life history questionnaire, a checklist of problems currently experienced, a measure of psychiatric symptoms, and responded to 3 TAT cards. TAT responses were again blindly scored using the PPSS. In contrast to the comparison group, psychiatric patients checked a greater number of current problems, endorsed more psychiatric symptoms, and obtained lower scores on the PPSS. A discriminant function analysis using PPSS scores correctly classified 72% of these participants. PPSS scores predicted group membership even after controlling for differences in age, education, the number of problems experienced, and psychiatric symptoms. This combination of variables correctly classified 92% of the participants. Results of both studies are interpreted as supporting the discriminant validity of the PPSS.

Acquisition of additional primary chromosome abnormalities in the course of karyotype evolution in a case of FAB-M2 acute leukemia

Cytogenetic studies of bone marrow metaphases from a 17-year-old woman with acute myeloid leukemia revealed a leukemic clone characterized by the t(8;21)(q22;q22) characteristic of FAB-M2. The patient was treated and achieved transient remissions. On relapse, her leukemic clone had acquired, in addition to the t(8;21), the inv(16)(p13q22) characteristic of FAB-M4Eo and a 5q- of the type seen in various acute myeloid leukemias and myelodysplastic syndromes. This cell line persisted throughout the remainder of the patient's clinical course. There were no other clonal chromosome abnormalities observed. The observation of multiple chromosome mutations, usually regarded separately as primary, in a single leukemic clone is most unusual and raises questions about our concepts of the nature of primary acquired chromosome mutations in cancer.