Establishment of a Quantitative Medical Technology Evaluation System and Indicators within Medical Institutions

The construction of Chinese indicator system on public health field investigation and short-term study hub: experience and implications

Background

The increasing of exchange activities among public health institutes and experts globally calls for a standardized operation to construct public health field investigation and short-term study hub (Field Study Hub). This can funcion as a platform to share experience in public health development in an accurate and comprehensive manner that would benefit global practices. This research aims to establish a supportive indicator system to guide the construction work.

Methods

Delphi method including two rounds of surveys were conducted among 82 senior public health experts. A structured questionnaire was designed to collect the opinions of the experts on the necessity of setting and feasibility of measurement for proposed 5 dimensions of 49 indicators and 7 additionally proposed ones. Percentage and score were used to describe the assessments, χ² and t tests to compare differences, Kappa and Cronbach’s alpha values to assess intra-rater and inter-rater reliabilities. Significance level α was 0.05. Bonferroni adjustment was used in the comparison of experts’ judgment basis.

Results

The percentages of experts choosing “Very good” or “Good” for necessity and feasibility in rounds 1 and 2 were 73.1–97.6% (85.8% ± 7.5%), 64.6–93.9% (82.8% ± 6.7%), 73.8–100% (91.0% ± 6.2%) and 72.5–100% (89.2% ± 7.3%) respectively. The scores of necessity were higher than those of feasibility, and the differences in the dimensions of “Key experience”, “Capacity for logistic support” and the total were statistically significant (t₁₁ = 2.920, t₁₂ = 3.035, t₃₁ = 4.448, t₃₂ = 2.664, t_t1 = 3.794, t_t2 = 3.007, P < 0.05). The fourteen most necessary indicators were identified. The judgment bases of “Theory” and “Experience” were higher than “Knowledge” and “Intuition” statistically significantly (round 2: χ_TK² = 39.020, χ_EK² = 67.692, χ_TI² = 45.823, χ_EI² = 76.515, P < 0.0125). The Kappa values exceeded 40 with the maximum as 75 and the Cronbach’s alphas exceeded 0.8000 with the maximum as 0.9732.

Conclusions

A set of 5 dimensions of 56 indicators with good necessity and feasibility were developed to technically support and well evaluate the construction of field study hub in public health institutions. This was of high significance because it tended to provide a preliminary baseline for the standardized practice in global health. Also, the present research might serve as a methodological reference for the development of other indicator sets.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41256-022-00273-z.