Incremental Model Fit Assessment in the Case of Categorical Data: Tucker-Lewis Index for Item Response Theory Modeling

Reliability and Validity of the Chinese Version of the Nurses' Cancer Pain Management Competency Scale

BACKGROUND

The Nurses' Cancer Pain Management Competency Scale (NCPMCS) is a tool to explore nurses' competencies and subjective experiences in cancer pain management, and to help nurses understand their current shortcomings in cancer pain management. The scale, currently available only in English and translated into Chinese for wider adoption abroad, provides a tool for Chinese nurses to assess their level of cancer pain management. Furthermore, based on the scale's specific score, they can evaluate their lack of understanding about cancer pain management, advance research into this area, and enhance their capacity to control cancer pain while providing patient care.

OBJECTIVE

The purpose of this study was to translate and localize the new scale, and to measure its reliability and validity. The study was also to provide a way to quickly and accurately measure the competency of cancer pain management among nursing staff in China.

METHODS

The Bristling translation approach was used to translate, translate back, and culturally modify the English version of the cancer pain management competency scale for nurses to create the Chinese version. A convenience sample was used for the study, 220 clinical nurses from three Grade III hospitals in Zhengzhou, Henan Province, China, were chosen by convenience sampling. The Chinese version of the scale was used for this investigation.

RESULTS

The Cancer Pain Management Competency Scale for Nurses has 14 items over 4 dimensions in its Chinese form. From the exploratory factor analysis, four common components were recovered; the cumulative variance rate was 81.994%, the scale's Cronbach's α coefficient was 0.902, and the Cronbach's α coefficient for each dimension ranged from 0.800 to 0.938. Retest reliability was 0.915, scale content validity was 0.865, and Spearman-Brown's broken half reliability was 0.808.

CONCLUSION

Nurses' cancer pain management competency in clinics can be assessed using the Chinese version of the Nurses' Cancer Pain Management Competency Scale, which has strong validity and reliability.

An item response theory approach to the measurement of working memory capacity

Complex span tasks are perhaps the most widely used paradigm to measure working memory capacity (WMC). Researchers assume that all types of complex span tasks assess domain-general WM. However, most research supporting this claim comes from factor analysis approaches that do not examine task performance at the item level, thus not allowing comparison of the characteristics of verbal and spatial complex span tasks. Item response theory (IRT) can help determine the extent to which different complex span tasks assess domain-general WM. In the current study, spatial and verbal complex span tasks were examined using IRT. The results revealed differences between verbal and spatial tasks in terms of item difficulty and block difficulty, and showed that most subjects with below-average ability were able to answer most items correctly across all tasks. In line with previous research, the findings suggest that examining domain-general WM by using only one task might elicit skewed scores based on task domain. Further, visuospatial complex span tasks should be prioritized as a measure of WMC if resources are limited.

Theory of trust and acceptance of artificial intelligence technology (TrAAIT): An instrument to assess clinician trust and acceptance of artificial intelligence

BACKGROUND

Artificial intelligence and machine learning (AI/ML) technologies like generative and ambient AI solutions are proliferating in real-world healthcare settings. Clinician trust affects adoption and impact of these systems. Organizations need a validated method to assess factors underlying trust and acceptance of AI for clinical workflows in order to improve adoption and the impact of AI.

OBJECTIVE

Our study set out to develop and assess a novel clinician-centered model to measure and explain trust and adoption of AI technology. We hypothesized that clinicians' system-specific Trust in AI is the primary predictor of both Acceptance (i.e., willingness to adopt), and post-adoption Trusting Stance (i.e., general stance towards any AI system). We validated the new model at an urban comprehensive cancer center. We produced an easily implemented survey tool for measuring clinician trust and adoption of AI.

METHODS

This survey-based, cross-sectional, psychometric study included a model development phase and validation phase. Measurement was done with five-point ascending unidirectional Likert scales. The development sample included N = 93 clinicians (physicians, advanced practice providers, nurses) that used an AI-based communication application. The validation sample included N = 73 clinicians that used a commercially available AI-powered speech-to-text application for note-writing in an electronic health record (EHR). Analytical procedures included exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and partial least squares structural equation modeling (PLS-SEM). The Johnson-Neyman (JN) methodology was used to determine moderator effects.

RESULTS

In the fully moderated causal model, clinician trust explained a large amount of variance in their acceptance of a specific AI application (56%) and their post-adoption general trusting stance towards AI in general (36%). Moderators included organizational assurances, length of time using the application, and clinician age. The final validated instrument has 20 items and takes 5 min to complete on average.

CONCLUSIONS

We found that clinician acceptance of AI is determined by their degree of trust formed via information credibility, perceived application value, and reliability. The novel model, TrAAIT, explains factors underlying AI trustworthiness and acceptance for clinicians. With its easy-to-use instrument and Summative Score Dashboard, TrAAIT can help organizations implementing AI to identify and intercept barriers to clinician adoption in real-world settings.

Translation and validation of the satisfaction with life scale in the native Quechua (Collao variant) language of southern Perú

Research on life satisfaction in indigenous populations is limited due to language barriers. Therefore, this paper aimed to translate and validate the Life Satisfaction Scale into the original Quechua language (collao variant) of southern Peru. The research was classified as instrumental and transversal and was conducted with the voluntary participation of 242 Quechua adults speaking the collao variant. The instrument that was translated was the 5-item SWLS, previously adapted to Peruvian Spanish. The internal structure was analyzed under an analytical-factorial approach, and the discrimination and difficulty of the items were evaluated from the item response theory (IRT). Expert judgment was favorable for all items (V > 0.70), confirming the 1-dimensional structure of the scale (χ2 = 8.972, df = 5, p = .000; CFI = 0.985; TLI = 0.970 and RMSEA = 0.057), with acceptable reliability (ω = 0.65). All the items of the scale presented adequate discrimination indices; in addition, the results of the evaluation of factorial invariance as a function of gender demonstrated configurational equivalence but an absence of metric invariance. In conclusion, the SWLS translated into Collao Quechua (collao variant) has a stable factorial structure and adequate internal consistency, although it was not possible to completely demonstrate the invariance by gender, it can be used for initial investigations to measure satisfaction with life of the Quechua-speaking indigenous population of southern Peru.

Visualizing Cross-Sections of 3D Objects: Developing Efficient Measures Using Item Response Theory

Spatial ability is important for success in STEM fields but is typically measured using a small number of tests that were not developed in the STEM context, have not been normed with recent samples, or have not been subjected to modern psychometric analyses. Here, an approach to developing valid, reliable, and efficient computer-based tests of spatial skills is proposed and illustrated via the development of an efficient test of the ability to visualize cross-sections of three-dimensional (3D) objects. After pilot testing, three measures of this ability were administered online to 498 participants (256 females, aged 18-20). Two of the measures, the Santa Barbara Solids and Planes of Reference tests had good psychometric properties and measured a domain-general ability to visualize cross-sections, with sub-factors related to item difficulty. Item-level statistics informed the development of the refined versions of these tests and a combined measure composed of the most informative test items. Sex and ethnicity had no significant effects on the combined measure after controlling for mathematics education, verbal ability, and age. The measures ofcross-sectioning ability developed in the context of geology education were found to be too difficult, likely because they measured domain knowledge in addition to cross-sectioning ability. Recommendations are made for the use of cross-section tests in selection and training and for the more general development of spatial ability measures.

A Psychometric Analysis of the Spanish Version of the Grief Impairment Scale: A Screening Tool of Biopsychosocial Grief-Related Functional Impairment in a Salvadoran Sample

The objective of this study was to translate and psychometrically evaluate a Spanish version of the Grief Impairment Scale (GIS) using a sample of bereaved adults from El Salvador (N = 579). The results confirm the unidimensional structure of the GIS, and solid reliability, item characteristics, and criterion-related validity, where the GIS scale significantly and positively predicts depression. However, this instrument only showed evidence of configural and metric invariance between different sex groups. Overall, these results support the Spanish version of the GIS as a psychometrically sound screening tool for health professionals and researchers to use in their clinical work.

New Frontiers in Prevention Research Models: Commentary on the Special Issue

Models considered in the current special issue represent valuable additions to the statistical toolbox of prevention researchers for many types of research questions and designs. Their appropriate use, however, depends on critical evaluation relative to previously existing techniques. This evaluation includes (a.) model choice involving "right-sizing" of the model relative to the amount and quality of data at hand, (b.) examination of the external validity of identified associations relative to observed or latent subgroups, (c.) confirmation of the reasonableness of the functional form assumed by the model, and (d.) identification of influential or outlying observations which unduly affect model fit or parameter estimates. Models in this issue allow for testing of new types of hypotheses in prevention research, and can constitute counterarguments to existing statistical practice. These models may, however, in turn be the object of critical examination of counterarguments a reasonable skeptic may offer.

Advanced Categorical Data Analysis in Prevention Science

A variety of health and social problems are routinely measured in the form of categorical outcome data (such as presence/absence of a problem behavior or stages of disease progression). Therefore, proper quantitative analysis of categorical data lies at the heart of the empirical work conducted in prevention science. Categorical data analysis constitutes a broad dynamic field of methods research and data analysts in prevention science can benefit from incorporating recent advances and developments in the statistical evaluation of categorical outcomes in their methodological repertoire. The present Special Issue, Advanced Categorical Data Analysis in Prevention Science, highlights recent methods developments and illustrates their application in the context of prevention science. Contributions of the Special Issue cover a wide variety of areas ranging from statistical models for binary as well as multi-categorical data, advances in the statistical evaluation of moderation and mediation effects for categorical data, developments in model evaluation and measurement, as well as methods that integrate variable- and person-oriented categorical data analysis. The articles of this Special issue make methodological advances in these areas accessible to the audience of prevention scientists to maintain rigorous statistical practice and decision making. The current paper provides background and rationale for this Special Issue, an overview of the articles, and a brief discussion of some potential future directions for prevention research involving categorical data analysis.

Differential Item Functioning of the Jaw Functional Limitation Scale

AIMS

To assess the differential item functioning (DIF) of the Jaw Functional Limitation Scale (JFLS) due to gender, age, and language (English vs Spanish).

METHODS

JFLS data were collected from a consecutive sample of 2,115 adult dental patients from HealthPartners dental clinics in Minnesota. Participants with missing data were excluded, and analyses were performed using data from 1,678 participants. Whether the item response theory (IRT) model assumptions of essential unidimensionality and local independence held up for the JFLS was examined. Then, using Samejima's graded response model, the IRT log-likelihood ratio approach was used to detect DIF. The magnitude and impact of DIF based on Raju's noncompensatory DIF (NCDIF) cutoff value of 0.096, Cohen's effect sizes, and test (or scale) characteristic curves were also assessed.

RESULTS

Essential unidimensionality was confirmed, but locally dependent items were found on the JFLS. A few items were flagged with statistically significant DIF after adjustment for multiple comparisons. The NCDIF indices associated with all DIF items were < 0.096, and they had small effect sizes of ≤ 0.2. The differences between the expected scores shown in the test characteristic curves were little to none.

CONCLUSION

The present results support the use of the JFLS summary score to obtain psychometrically robust score comparisons across English- and Spanish-speaking, male and female, and younger and older dental patients. Overall, the magnitude of DIF was relatively small, and the practical impact minimal.

Measurement of Risk Factors Associated With bereavement Severity and Deterioration by COVID-19: A Spanish Validation Study of the Pandemic Grief Risk Factors

The present study translated and evaluated the psychometric evidence of the Pandemic Grief Risk Factors (PGRF) in a sample of 363 people from the general population of Peru who suffered the death of a loved one by COVID-19 (63-4% women and 36.6% men, where 78.5% were between 18 and 29 years old). The findings indicated that the PGRF is a unidimensional and reliable measure. The PGRF items can differentiate between individuals with different levels of risk factors and thus cover a wide range of the latent construct. Also, a greater sense of distress for each of the risk factors for pandemic grief is necessary to answer the higher response categories. Risk factors significantly and positively predict COVID-19-associated dysfunctional grief. The results indicated that the PGRF in Spanish is a measure with adequate psychometric properties to measure risk factors for pandemic grief.

Psychometric validation of a brief self-report measure of misophonia symptoms and functional impairment: The duke-vanderbilt misophonia screening questionnaire

Misophonia is a newly described disorder of sound tolerance characterized by strong negative emotional reactions to specific "trigger" sounds, resulting in significant distress, pathological avoidance, and impairment in daily life. Research on misophonia is still in its infancy, and most existing psychometric tools for assessing misophonia symptoms have not been extensively validated. The purpose of the current study was to introduce and psychometrically validate the duke-vanderbilt Misophonia Screening Questionnaire (DVMSQ), a novel self-report measure of misophonia symptoms that can be used to determine misophonia "caseness" in clinical and research settings. Employing large online samples of general population adults (n = 1403) and adults on the autism spectrum (n = 936), we rigorously evaluated the internal structure, reliability, validity, and measurement invariance of the DVMSQ. Results indicated that 17 of the 20 original DVMSQ items fit well to a bifactor structure with one "general misophonia" factor and four specific factors (anger/aggression, distress/avoidance, impairment, and global impact). DVMSQ total and subscale scores were highly reliable in both general population and autistic adult samples, and the measure was found to be approximately invariant across age, sex, education level, and autism status. DVMSQ total scores also correlated strongly with another measure of misophonia symptoms (Duke Misophonia Questionnaire-Symptom Scale), with correlations between these two measures being significantly stronger than correlations between the DVMSQ and scales measuring other types of sound intolerance (Inventory of Hyperacusis Symptoms [General Loudness subscale] and DSM-5 Severity Measure for Specific Phobia [modified for phonophobia]). Additionally, DVMSQ items were used to operationalize diagnostic criteria for misophonia derived from the Revised Amsterdam Criteria, which were further updated to reflect a recent consensus definition of misophonia (published after the development of the DVMSQ). Using the new DVMSQ algorithm, 7.3% of general population adults and 35.5% of autistic adults met criteria for clinically significant misophonia. Although additional work is needed to further investigate the psychometric properties of the DVMSQ and validate its theory-based screening algorithm using best-estimate clinical diagnoses, this novel measure represents a potentially useful tool to screen for misophonia and quantify symptom severity and impairment in both autistic adults and the general population.