Content Validity and Cognitive Testing in the Development of a Motivational Interviewing Self-Assessment Questionnaire

Aim

To develop and content validate a self-assessment questionnaire on motivational interviewing (MI) practice as the first stages in forming the questionnaire to be used in cross-sectional studies involving practitioners conducting the MI-based alcohol screening and brief intervention (ASBI).

Methods

A comprehensive mixed methods approach included a literature review, 3 rounds of expert panel (EP) opinions (n=10), cognitive testing (CT) with 10 MI-based ASBI practitioners, and questionnaire piloting with 31 MI-based ASBI practitioners. Based on the EP opinions in the second round, content validity indices (CVIs) and the modified kappa coefficient (k*) were calculated, focusing on the relevance and understandability of questions and comprehensiveness and meaningfulness of the response options. This analysis was performed in 2020, at the conclusion of the national "Together for a Responsible Attitude Towards Alcohol Consumption" ("Skupaj za odgovoren odnos do pitja alkohola", SOPA) project's pilot implementation.

Results

On a scale level, CVI values based on universal agreement for the entire questionnaire were high for 3/4 categories (S-CVI-UA>0.80), and CVI values based on average agreement were high across all categories (S-CVI-Ave>0.90). At the item level, CVI values (I-CVI) were never <0.50 (automatic item rejection), and the modified kappa value (k*) indicated poor validity for two items in the understandability category (k*=0.33). All problematic parts of the questionnaire were further tested and successfully modified based on the results of CT, and accepted in the third round of testing.

Conclusions

The final version of the questionnaire demonstrated appropriate content validity for use in studies among Slovenian MI-based ASBI practitioners and is now ready for further psychometric testing.

Facilitatory Effect of Extending the Course Duration on Dissemination of Educational Content

The physiological practice course at Saitama Medical University provides students with the opportunity to learn physiological principles through wet labs and discussions. To develop a more effective method for maximizing learning outcomes, we extended the course's schedule from one day (1d) to two days (2d) per theme, evaluated self-administered questionnaires between two different years (pre and post-change), and examined whether the increased course length affected learning outcomes. Within the 2018 curriculum year, every theme of the course was completed in a day, including experiments in the wet lab and discussions. In 2019, each theme was assessed for two days. The second-year undergraduate medical students anonymously submitted the self-assessment questionnaire that addressed several aspects, such as understanding of the theme, through a 5-point Likert scale. The average Likert scores varied from 4 to 4.5 point for all questions, and significant differences were not found between the 1d and 2d courses. However, the ratio of students with the highest points increased for one question of the 2d course: 43.6% (1d) to 53.4% (2d) for understanding. Further, the standard deviation (SD) values decreased in the 2d course for every question: 0.29 (1d) to 0.15 (2d) for interest, 0.33 (1d) to 0.19 (2d) for understanding, 0.30 (d) to 0.17 (d) for communication, 0.34 (1d) to 0.19 (2d) for general evaluation. This reduction in the SD values indicated that the educational content was imparted more efficiently to students in the 2d course. Thus, we concluded that extending the course time facilitated dissemination of educational content for every theme.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01563-4.

Self-Assessment Questionnaire on Patient-Physician Concordance on Nevus Self-Count and Models Development to Predict High-Risk Phenotype >50 Nevi

BACKGROUND

Cutaneous melanoma accounts for the majority of skin cancer-related deaths. Readily identifiable phenotypic characteristics and total body nevus count (TBNC) >50 are among the most important risk factors for cutaneous melanoma. Implementation of nevus self-count procedures and self-assessment of phenotypic traits as part of skin self-examination could be an excellent screening tool for identifying an at-risk target population.

OBJECTIVES

Objectives of the study were to assess the skills of a central Italian and eastern Spanish population sample to recognize their skin lesions via the submission of a self-assessment questionnaire and to explore which self-assessment questionnaire item combination best predicts the high-risk condition of TBNC >50.

METHODS

Patients aged ≥18 years filled a self-assessment questionnaire, autonomously and prior to the dermatological visit. Subsequently, dermatologists performed total body skin examination and reported patients' skin lesions on a separate questionnaire.

RESULTS

We reported fair to moderate patient-dermatologist agreement for skin lesion self-assessment. The item number of nevi on the back was the single questionnaire item most accurately predicting TBNC >50. The high-sensitivity and high-specificity classification and regression tree models for the prediction of TBNC >50 displayed different items combinations; the item nevus on the back was always the first and most important predictor in both our models.

CONCLUSIONS

Patients were partially able to provide correct estimation of their whole-body nevus self-count. The item nevi on the back seems to be the first and most important predictor of TBNC >50 across our models. Delivery of high-sensitivity and high-specificity prediction models based on our questionnaire item combination may help defining a high-risk target population.

Validation of the French version of HHIE-S (Hearing Handicap Inventory for the Elderly - Screening) questionnaire in French over-60 year-olds

INTRODUCTION

The HHIE-S (Hearing Handicap Inventory for the Elderly - Screening) is widely used for hearing-loss disorder in the elderly. The main objective of the present study was to validate a French version. The secondary objective was to determinate a cut-off score as indication for hearing rehabilitation.

METHODS

We translated the HHIE-S into French, respecting the cross-cultural adaptation process for medical questionnaires. An observational study assessed the translation (10 questions, scored from 0 to 40) used for screening purposes in a prospective cohort, aged ≥60 years, with comparison to pure tone, speech-in-silence and speech-in-noise audiometry. Subjects were considered hearing-impaired if the pure-tone average at 500, 1,000, 2,000 and 4,000 Hz was >20 dB HL in one or both ears.

RESULTS

We tested 294 subjects (mean age =67±6 years). Hearing loss prevalence was 34.7 %. Cronbach's alpha (test reliability) was high (0.84). Taking HHIE-S score >8/40 as cut-off defining hearing loss, sensitivity was 80.4%, specificity 85.4 %, positive predictive value 74.5 % and negative predictive value 89.1 %. Seventy-three subjects (24.8 %) had theoretic indications for hearing aids, optimally detected by HHIE-S score >16/40 (88,4 %).

CONCLUSION

Our study validated the French version of the HHIE-S. This tool could be useful in screening for age-induced hearing loss in the elderly French population.

Translational and Randomized Study of 5-HT3 Receptor Antagonists for Evaluation of Chemotherapy-induced Nausea and Vomiting Related Biomarkers

Purpose We assessed the efficacy of palonosetron (PAL) in comparison to granisetron (GRA) for the treatment of CINV using the self-assessment questionnaires. In addition, we analyzed the serum levels of emetic various biomarkers. Methods We conducted a randomized study of 70 patients naïve to chemotherapy. The primary endpoint was the late phase score on the MAT questionnaire. The plasma concentrations of the biomarkers were measured on days 1 and 3. Results There were no statistical differences in the scores on the questionnaires, but the mean values in response to PAL were higher than those in response to GRA. The value of ghrelin on day 1 was significantly higher for GRA than for PAL. Conclusions For the primary endpoint, the score of the late phase on the MAT questionnaire was not statistically different between the PAL and GRA treatment groups. Further studies are needed to clarify the role of ghrelin for the treatment of CINV. J. Med. Invest. 66 : 269-274, August, 2019.

Reliability and validity of the Eating Assessment Tool-10 (Greek adaptation) in neurogenic and head and neck cancer-related oropharyngeal dysphagia

PURPOSE

Dysphagia is a symptom associated with significant morbidity and mortality, with profound impact on physical ability and quality of life. Many questionnaires have been used to assess patient-reported dysphagia, but issues related to developmental and measurement properties affect their wide applicability. The purpose of this study was to assess the validity and reliability of the Eating Assessment Tool-10 (EAT-10, Greek adaptation) in neurogenic and head and neck cancer-related dysphagia.

METHODS

The study consisted of: item generation in the Greek language, internal consistency and reliability analysis, normative data generation, and validity analysis. Data were collected prospectively from 421 participants: 144 asymptomatic subjects, 146 patients with dysphagia, and 131 patients with dysphagia-related diagnoses. Validity was assessed by comparing scores of healthy and dysphagic participants, by comparing pre- and post-treatment scores, and by correlating the Greek-EAT-10 with fibreoptic endoscopic evaluation of swallowing (FEES).

RESULTS

The mean participants' age was 52.85 years (ranging from 18 to 85 years). All questionnaires were completed in less than 3 min. The overall internal consistency (assessed with Cronbach's alpha) was 0.963. The test-retest reliability was excellent with Spearman's rho ranging from 0.937 to 1. Dysphagic patients had a significantly higher score compared to healthy participants (p < 0.001). The mean EAT-10 improved significantly after treatment (Wilcoxon signed rank, p < 0.001). The Greek-EAT-10 and FEES scores were significantly correlated (Spearman's rho = 0.69).

CONCLUSIONS

The EAT-10 is a valid, reliable, symptom-specific tool for the assessment of dysphagia, easily self-administered, and practical for clinical use.

Reliability and validity of the DYMUS questionnaire for the assessment of dysphagia in multiple sclerosis (Greek version) and proposed modification

BACKGROUND

Recent research confirmed that at least a third of people with multiple sclerosis (MS) are suffering from swallowing difficulties. Dysphagia is associated with significant morbidity and mortality, and has profound impact on physical ability and quality of life. Dysphagia related complications can be prevented through an effective screening protocol. The Dysphagia in Multiple Sclerosis (DYMUS) questionnaire is the first dysphagia questionnaire developed specifically for patients with MS. The aim of the present study was the cultural adaptation of DYMUS for the Greek population, evaluation of the questionnaire's reliability and validity and normative data generation for DYMUS, which has not been published before.

METHODS

DYMUS was completed by 200 participants: 108 MS patients and 92 asymptomatic subjects (60 male and 140 female). Patients with MS were consecutively recruited from the Multiple Sclerosis Centre during regular visits. Asymptomatic participants were community-dwelling healthy persons. All data were collected prospectively. MS patients were invited to complete DYMUS and the Greek versions of the Eating Assessment Tool-10 (EAT-10), and the Swallowing-Quality of Life (SWAL-QoL) questionnaires. Healthy subjects completed DYMUS and the Greek EAT-10. The study consisted of item generation for the Greek DYMUS, internal consistency and reliability analysis, normative data generation, and validity analysis. Criterion validity was assessed by comparing scores between groups with expected differences: MS patients and healthy participants. Construct validity was assessed by comparison of DYMUS scores of dysphagic and non dysphagic patients. DYMUS was also validated against EAT-10, and SWAL-QoL to assess its convergent validity.

RESULTS

DYMUS was completed by all participants in less than 3 min. The internal consistency was excellent (Cronbach's alpha was 0.866). Test-retest reliability was good (Pearson's correlation coefficient was 0.637). The mean DYMUS score for the healthy cohort was 0.23 ± 0.471. The upper limit of normal was 1.172. MS patients had statistically significantly higher score than controls (Mann Whitney test, p < 0.001). DYMUS mean score was statistically significantly higher in the dysphagic compared to the non dysphagic MS patients (Mann Whitney test, p < 0.001). There was a strong positive and statistically significant correlation between DYMUS and EAT-10 (Pearson's Correlation coefficient, r = 0.754, p = 0). In the MS patients cohort 25.9% reported themselves as dysphagic, 34.3% were classified as dysphagic according to EAT-10 and 44.4% according to DYMUS. The DYMUS score had a statistically significant positive correlation with the EDSS score, (Mann Whitney, p < 0.001) whereas age, sex, type of MS and disease duration were not significantly correlated. Based on our data analysis we propose the modification of DYMUS to a 9-items tool eliminating the question about weight loss. A DYMUS score of 2 or higher is indicating dysphagia for both the original DYMUS and the modified DYMUS.

CONCLUSIONS

The Greek version of DYMUS was found to be a valid, reliable and practical for clinical use questionnaire for the detection of dysphagia in Multiple Sclerosis. The first reported normative data for DYMUS suggest a cut-off for the diagnosis of dysphagia at 2 and our findings support a modification of DYMUS eliminating the question about weight loss.

Development and psychometric testing of inflammatory bowel disease fatigue (IBD-F) patient self-assessment scale

BACKGROUND AND AIMS

Fatigue is one of the main symptoms of inflammatory bowel disease (IBD) and is frequently reported by people in both active and quiescent disease. Many different fatigue assessment scales have been used to measure fatigue, but none has been developed or tested in IBD. This study aimed to develop a fatigue scale specific to the needs and experiences of people with IBD.

METHODS

A five-step sequential mixed method design was used: a qualitative phase to assess patients' experience of fatigue and its impact on their lives, and four mixed qualitative-quantitative phases to refine the scale and to assess its psychometric properties.

RESULTS

567 people participated in five phases. The resulting questionnaire has 3 sections: 5 questions assessing frequency and severity of fatigue; 30 questions rating the experience and impact of fatigue; and a free-text section asking for patients' comments and additional issues related to fatigue. Initial validation suggests that the questionnaire has good face and content validity, acceptable to excellent test-retest stability (ICC 0.74 for Section 1 and 0.83 for Section 2) and a high degree of internal consistency (Cronbach's alpha>0.9).

CONCLUSIONS

Participants in the study confirmed that fatigue in IBD is burdensome. Items generated and refined by people with IBD-fatigue reflect their experience and form the basis of this new IBD-fatigue scale, which is psychometrically robust with reliability estimates which fall within statistically acceptable ranges. The scale can be used by patients and practitioners to assess severity and impact of fatigue in people with IBD.

Assessment of the psychometric properties of the Spanish language version of questionnaire ICIQ-Male Lower Urinary Tract Symptoms (ICIQ-MLUTS)

OBJECTIVES

To evaluate the psychometric properties of the Spanish version of the ICIQ-Male Lower Urinary Tract Symptoms Questionnaire (ICIQ-MLUTS): Feasibility (% of completion and ceiling/ground effects), reliability (Test-retest), convergent validity (vs Bladder Control Self-Assessment Questionnaire [BSAQ] and vs International Prostate Symptom Score [I-PSS]) and criterion validity (according to presence or absence of symptoms).

MATERIAL AND METHODS

This was an observational, non-interventionist and multicenter study. 223 male patients with lower urinary tract symptoms (LUTS), predominantly storage symptoms and aged 18-65, took part in the study. Patients completed the ICIQ-MLUTS (test), I-PSS and BSAQ questionnaires and referred their urinary symptoms in a single visit, with the exception of a subgroup composed by 49 patients that completed the questionnaire again 15 days after initial visit to evaluate test-retest reliability. The questionnaire includes 13 items divided in 2 sub-scales: Voiding symptoms (V) from 0-20 and Incontinence symptoms (I) from 0-24.

RESULTS

Percentage of patients that completed all items: 98.84%. Ground effect is 0 and ceiling effect was under 6% in both sub-scales. Test-retest reliability: Intraclass correlation coefficient (ICC) ranged from 0.68 to 0.88, except on Delay. Kappa shows a good agreement, between 0.60 and 0.81, except for Nocturia. Convergent validity: Correlation (Spearman) between the questionnaire sub-scales scores and the rest of measures is statistically significant (P < .01 and P < .05). Criterion validity: Statistically significant differences (P < .05) between scores on ICIQ-MLUTS, from patients that refer experiencing symptoms and those who do not.

CONCLUSION

The Spanish version of the ICIQ-MLUTS questionnaire shows adequate feasibility, reliability and validity.