Conceptualization of the term "ecological validity" in neuropsychological research on executive function assessment: a systematic review and call to action

OBJECTIVE

"Ecological validity" (EV) is classically defined as test's ability to predict real-world functioning, either alone or together with test's similarity to real-world tasks. In neuropsychological literature on assessment of executive functions (EF), EV is conceptualized inconsistently, leading to misconceptions about the utility of tests. The goal of this systematic review was to examine how EV is conceptualized in studies of EF tests described as ecologically valid.

METHOD

MEDLINE and PsychINFO Databases were searched. PRISMA guidelines were observed. After applying inclusion and exclusion criteria, this search yielded 90 articles. Deductive content analysis was employed to determine how the term EV was used.

RESULTS

About 1/3 of the studies conceptualized EV as the test's ability to predict functional outcomes, 1/3 as both the ability to predict functional outcome and similarity to real-world tasks, and 1/3 were either unclear about the meaning of the term or relied on notions unrelated to classical definitions (e.g., similarity to real-world tasks alone, association with other tests, or the ability to discriminate between populations).

CONCLUSIONS

Conceptualizations of the term EV in literature on EF assessment vary grossly, subsuming the notions of criterion, construct, and face validity, as well as sensitivity/specificity. Such inconsistency makes it difficult to interpret clinical utility of tests that are described as ecologically valid. We call on the field to require that, at minimum, the term EV be clearly defined in all publications, or replaced with more concrete terminology (e.g., criterion validity).

Knowledge, Awareness, and Practice of Safety and Emergency Response Among Scuba Divers in Malaysia: Questionnaire Development and Validation

INTRODUCTION

Safety has become paramount to ensure that scuba diving continues to expand into new markets, with minimal risks, and that scuba diving translates into a safe and relaxed adventure in response to growing market demands. Research focusing on assessing the knowledge, awareness, and practices (KAP) regarding safety and emergency response among scuba divers has been limited, and there is a need for validated assessment tools in this area.

METHODS

This study involved 555 scuba divers in Malaysia, and the questionnaire underwent a thorough development and validation process, including content and face validity assessments, as well as exploratory factor analysis. The validation of the knowledge domain was conducted using Item Response Theory (IRT) analysis, while awareness and practice were validated through exploratory and confirmatory factor analyses (EFA and CFA).

RESULTS

The content validity of the instrument was confirmed, with all items scoring over 80% for Item Content Validity Index (I-CVI), Scale Content Validity Index (S-CVI), and Scale Content Validity Index/Average (S-CVI/AVE). The initial draft comprises three domains: knowledge, awareness, and practice. Knowledge items were analyzed using IRT and demonstrated acceptable difficulty and discrimination levels. For the awareness and practice domains, EFA showed a Kaiser-Meyer-Olkin measure (KMO) of 0.83 and 0.79, respectively, with a significant Bartlett's test of Sphericity (P < 0.001). EFA yielded three factors for both awareness and practice domains with a total of 12 items for awareness and 13 items for practice, with satisfactory factor loadings (≥0.3). The final model of CFA showed good fits for both awareness and practice domains in terms of absolute, parsimonious, and comparative measures. The composite reliability of awareness was acceptable with Raykov's rho of 0.71, whereas the practice domain fell slightly below the acceptable threshold at 0.55. This was attributed to low factor loading and a limited number of items within each factor. The final questionnaire now consists of 20 items for knowledge, 10 items for awareness, and 12 items for the practice domain.

CONCLUSION

The results of this validation and reliability study indicate that the newly developed questionnaire possesses favorable psychometric properties for assessing KAP related to safety and emergency response in the scuba diving context. This research is essential for harmonizing the perspectives of crucial stakeholders within the recreational scuba diving industry.

The bruxism screener (BruxScreen): Development, pilot testing and face validity

BACKGROUND

To assess awake and sleep bruxism, the Standardised Tool for the Assessment of Bruxism (STAB) is currently being developed. The STAB will be a comprehensive tool for the detailed assessment of bruxism behaviour itself as well as of its possible consequences, causes, and comorbid conditions.

OBJECTIVE

Since the STAB cannot fully meet the 'A4 principle' for a bruxism assessment tool, i.e., being Accurate (reliable and valid), Applicable (feasible), Affordable (cost-effective), and Accessible (suitable for everyday clinical use), the Bruxism Screener (BruxScreen) has been developed to be used in large-scale epidemiological research projects and, especially, in general, dental practices.

METHODS

The BruxScreen consists of two parts: a questionnaire (BruxScreen-Q) to be completed by patients, and a clinical assessment form (BruxScreen-C) to be completed by dentists.

RESULTS

This paper describes the development of the BruxScreen and provides the outcomes of the pilot testing phase and the face validity assessment (i.e. that the first impressions of the tool indicate that it adequately reflects the construct to be measured).

CONCLUSION

The resulting BruxScreen is considered ready for more profound psychometric testing in the general dental setting.

Development of the Canadian Eating Practices Screener to assess eating practices based on 2019 Canada's Food Guide recommendations

In 2019, Health Canada released a new iteration of Canada's Food Guide (2019-CFG), which, for the first time, highlighted recommendations regarding eating practices, i.e., guidance on where, when, why, and how to eat. The objective of this study was to develop a brief self-administered screener to assess eating practices recommended in the 2019-CFG among adults aged 18-65 years. Development of the screener items was informed by a review of existing tools and mapping of items onto 2019-CFG recommendations. Face and content validity were assessed with experts in public health nutrition and/or dietary assessment (n = 16) and individuals from Government of Canada (n = 14). Cognitive interviews were conducted with English-speaking (n = 16) and French-speaking (n = 16) adults living in Canada to assess face validity and understanding of the screener items. While some modifications were identified to improve relevance or clarity, overall, the screener items were found to be relevant, well-constructed, and clearly worded. This comprehensive process resulted in the Canadian Eating Practices Screener/Questionnaire court canadien sur les pratiques alimentaires, which includes 21 items that assess eating practices recommended in the 2019-CFG. This screener can facilitate monitoring and surveillance efforts of the 2019-CFG eating practices as well as research exploring how these practices are associated with various health outcomes.

Adaptation and content validation of a patient-reported measure of treatment burden for use in stroke survivors: the patient experience with treatment and self-management in stroke (PETS-stroke) measure

PURPOSE

Stroke survivors often live with significant treatment burden yet our ability to examine this is limited by a lack of validated measurement instruments. We aimed to adapt the 60-item, 12-domain Patient Experience with Treatment and Self-Management (PETS) (version 2.0, English) patient-reported measure to create a stroke-specific measure (PETS-stroke) and to conduct content validity testing with stroke survivors.

MATERIALS AND METHODS

Step 1 - Adaptation of PETS to create PETS-stroke: a conceptual model of treatment burden in stroke was utilised to amend, remove or add items. Step 2 - Content validation: Fifteen stroke survivors in Scotland were recruited through stroke groups and primary care. Three rounds of five cognitive interviews were audio recorded and transcribed. Framework analysis was used to explore importance/relevance/clarity of PETS-stroke content. COSMIN reporting guidelines were followed.

RESULTS

The adapted PETS-stroke had 34 items, spanning 13 domains; 10 items unchanged from PETS, 6 new and 18 amended. Interviews (n = 15) resulted in further changes to 19 items, including: instructions; wording; item location; answer options; and recall period.

CONCLUSIONS

PETS-stroke has content that is relevant, meaningful and comprehensible to stroke survivors. Content validity and reliability testing are now required. The validated tool will aid testing of tailored interventions to lessen treatment burden.IMPLICATIONS FOR REHABILITATIONTreatment burden is reported by stroke survivors but no stroke-specific measure of treatment burden exists.We adapted an existing measure of treatment burden for use in multimorbid patients (PETS) to create a stroke specific version (PETS-stroke).The items in PETS-stroke are relevant and meaningful to people with stroke.Further testing will examine construct validity, reliability, and useability.This measure will be useful in future RCTs to measure treatment burden and to identify stroke patients who are at high risk of treatment burden.

Development of the Canadian Food Intake Screener to assess alignment of adults' dietary intake with the 2019 Canada's Food Guide healthy food choices recommendations

NOVELTY

The Canadian Food Intake Screener was developed to rapidly assess alignment of adults' dietary intake over the past month with the Food Guide's healthy food choices recommendations. The screener was developed and evaluated through an iterative process that included three rounds of cognitive interviews in each of English and French, along with ongoing feedback from external advisors and face and content validity testing with a separate panel of content experts. The 16-question screener is intended for use with adults, aged 18-65 years, with marginal and higher health literacy in research and surveillance contexts in which comprehensive dietary assessment is not possible.

Face Validity, Content Validity, and Psychometric Testing of the Hospital Survey on Patient Safety Culture Among Undergraduate Nursing Students

Background and Purpose: Nursing students have an essential role in patient safety. The purpose of this study was to evaluate the face validity, content validity, and psychometric properties of the Hospital Survey on Patient Safety Culture for Nursing Students (HSOPS-NS). Methods: The cross-sectional study was carried out between January and October 2021. The participants were undergraduate nursing students (N = 482) from 16 Czech nursing faculties. Results: Exploratory factor analysis revealed an eight-factor structure, which was verified by confirmatory factor analysis using the optimization process that results in adequate goodness-of-fit indices (root mean squared error approximation = .037; standardized root mean squared residuals = .056; comparative fit index = .935; Tucker-Lewis index = .926; incremental fit index = .936). The internal consistency of a new model was excellent (α = .914). Conclusion: The results indicate that the HSOPS-NS shows evidence of reliability and validity and is a valuable measure of safety culture as perceived by nursing students.

Updated Decision Aid Enabling Women to Choose between with or without Epidural Analgesia during Childbirth, and Confirmation of Validity

BACKGROUND

The use of a decision aid for choosing whether to have or not have anesthesia during childbirth has been shown to increase both knowledge about birth and the proportion of women who made their own decisions compared with women who did not use a decision aid. Herein, we updated the first version of our decision aid into a second version and evaluated this updated decision aid. We evaluated the face validity and content appropriateness of the updated decision aid developed to enhance the ability of women to choose between childbirth with or without epidural analgesia.

METHODS

This was a descriptive study based on a literature review of updated information for addition to the first version. PubMed and Cochrane Library were searched from 2003 to May 2021. Thereafter, obstetricians, anesthesiologists, and midwives were asked to respond to a questionnaire regarding the face validity and content appropriateness of the updated decision aid regarding whether it meets the IPDASi (Version 4.0) quality standards.

RESULTS

One obstetrician, one anesthesiologist, and three midwives who had performed epidural anesthesia for at least three years responded to the questionnaire. The responses to the evaluation items of face validity (i.e., style and clarity) were positive. There were 38 specific comments regarding content appropriateness classified into seven categories: "addition or revision of text", "unification of expressions", "need for explanation/information", "lack of evidence", "potential to mislead", "questionable", and "structure".

CONCLUSION

The face validity and content appropriateness of the updated decision aid was confirmed. The next step is evaluation of the updated decision aid by pregnant women who give birth.

Content, face, and construct validity of the Irritable Bowel Syndrome Quality of Life (IBS-QOL) as a measure of bowel-related quality of life in spinal cord injury

INTRODUCTION

Neurogenic bowel dysfunction is commonly encountered after a spinal cord injury (SCI), leading to a tremendous impact on quality of life (QOL). The neurogenic bowel dysfunction score (NBDS) is commonly used to measure the severity of bowel dysfunction and predict QOL. However, there is no comprehensive instrument to assess bowel-specific QOL for SCI patients. Instead, the Irritable Bowel Syndrome-Quality of Life (IBS-QOL) questionnaire evaluates the impact of bowel dysfunction on several aspects of QOL, although this questionnaire has not been validated for the SCI population. Motivated by the compelling need of instruments to accurately evaluate the QOL in patients who develope NBD after SCI, we aimed to assess the construct, content, and face validity of IBS-QOL in this population.

METHODS

Adult SCI patients with at least 3 months after their injury were recruited from the outpatient clinic of a national rehabilitation hospital. Patients completed the NBDS and IBS-QOL via telephone interview or paper survey in the clinic. Content and face validity were assessed via interviews with professionals with expertise in providing chronic care for SCI, as well as a subgroup of patients. Construct validity was assessed using the hypotheses testing method. Internal consistency was assessed using Cronbach's ⍺. Factor analysis was performed to assess the dimensionality of the IBS-QOL in the SCI population.

RESULTS

A total of 106 patients with a median age of 45.5 years (interquartile range: 21-79) participated in the study. The majority of the sample were men (n = 82, 77%) and had endured thoracolumbar injuries (n = 74, 71.2%). Twelve patients (seven English- and five Spanish speakers) and six professionals took part in content/face validation interviews. The median IBS-QOL total score was 15.91/100 (interquartile range: 4.55-33.14). IBS-QOL differentiated the subgroups of patients with severe bowel symptoms in terms of uneasiness, sweating, or headaches during bowel emptying (p = 0.0003), time spent on bowel emptying (p = 0.0065), flatus incontinence (p = 0.0076), and overall satisfaction with bowel function (p < 0.001), demonstrating its adequate construct validity. Interviews with the patients and professionals supported the comprehensiveness, comprehensibility, and relevance of IBS-QOL for assessment of bowel-related QOL in the SCI population. Item-level analysis of professional responses showed that 97% of questions were relevant to the construct and population of interest. Internal consistency analysis yielded a Cronbach's ⍺ of 0.9684. Exploratory factor analysis yielded six underlying factors which cumulatively accounted for 72.21% of the total variance, reflecting the dimensionality of bowel-related QOL in SCI population.

DISCUSSION

IBS-QOL questionnaire is a comprehensive measure of bowel-related QOL which encompasses the concerns of SCI patients. Our findings support the content, face and construct validity of IBS-QOL as a measure of bowel-related QOL in SCI. Further studies are warranted to assess the reliability and responsiveness of IBS-QOL, and to evaluate its performance across different patient populations.

Developing and testing inter-rater reliability of a data collection tool for patient health records on end-of-life care of neurological patients in an acute hospital ward

AIM

Develop and test a data collection tool-Neurological End-Of-Life Care Assessment Tool (NEOLCAT)-for extracting data from patient health records (PHRs) on end-of-life care of neurological patients in an acute hospital ward.

DESIGN

Instrument development and inter-rater reliability (IRR) assessment.

METHOD

NEOLCAT was constructed from patient care items obtained from clinical guidelines and literature on end-of-life care. Expert clinicians reviewed the items. Using percentage agreement and Fleiss' kappa we calculated IRR on 32 nominal items, out of 76 items.

RESULTS

IRR of NEOLCAT showed 89% (range 83%-95%) overall categorical percentage agreement. The Fleiss' kappa categorical coefficient was 0.84 (range 0.71-0.91). There was fair or moderate agreement on six items, and moderate or almost perfect agreement on 26 items.

CONCLUSION

The NEOLCAT shows promising psychometric properties for studying clinical components of care of neurological patients at the end-of-life on an acute hospital ward but could be further developed in future studies.

Lay Definitions of Intelligence, Knowledge, and Memory: Inter- and Independence of Constructs

The present study examined how lay participants define the following concepts used widely in psychology: being intelligent, knowing, and remembering. In the scientific community, knowledge overlaps with the contents of semantic memory, crystallized intelligence reflects the accumulation of knowledge, knowledge and event memory interact, and fluid intelligence and working memory correlate. Naturally, the lay public has implicit theories of these constructs. These theories mainly distinguish between intelligent and unintelligent behaviors and tend to include characteristics outside psychometric studies of intelligence, such as emotional intelligence. Here, we asked lay participants from the online platform Prolific to explain "what does being intelligent mean to you?" as well as "knowing" and "remembering" to understand their degree of alignment with theoretical conceptualizations in the research community. Qualitative coding of participant definitions showed that intelligence and knowledge are closely related, but asymmetrically-when defining what it means to be intelligent, participants reference knowledge, but intelligence is not considered in explaining knowing. Although participants note that intelligence is multi-faceted and related to problem-solving, there is an emphasis (in terms of frequency of mentions) on the crystallized side of intelligence (i.e., knowledge). A deeper understanding of lay participants' mental models of these constructs (i.e., their metacognitions) is essential for bridging gaps between experts and the general public.

Determination of Relative Weightings for Sacroiliac Joint Pathologies in the OMERACT Juvenile Arthritis Magnetic Resonance Imaging Sacroiliac Joint Score

This study aims to determine the relative weights (point value) of items of the juvenile idiopathic arthritis magnetic resonance imaging-sacroiliac joint scoring system (JAMRIS-SIJ). An adaptive multicriteria decision analysis was performed using the 1000Minds web application to determine the relative weights of the items in the JAMRIS-SIJ inflammation and damage domains. Experts in imaging and rheumatology independently completed a conjoint analysis survey (CAS) to determine the point value of the measurement items of the JAMRIS-SIJ. Each CAS survey question asked the expert to compare two hypothetical patient profiles, which were otherwise similar but different at two items at a time, and to select which item showed a more severe stage of inflammation or osteochondral damage. In addition, experts ranked 14 JAMRIS-SIJ grade only or image + grade patient vignettes while blinded to the CAS-derived weights. The validity of the weighted JAMRIS-SIJ was tested by comparing the expert CAS-weighted score and the image + grade ranking method. Seventeen experts completed the CAS (11 radiologists and 6 rheumatologists). Considering the point value for inflammation domain items, osteitis (24.7%) and bone marrow edema (24.3%) had higher group-averaged percentage weights compared to inflammation in erosion cavity (16.9%), joint space enhancement (13.1%), joint space fluid (9.1%), capsulitis (7.3%), and enthesitis (4.6%). Similarly, concerning the damage domain, ankylosis (41.3%) and erosion (25.1%) showed higher group-averaged weights compared to backfill (13.9%), sclerosis (10.7%), and fat metaplasia lesion (9.1%). The Spearman correlation coefficients of the CAS-weighted vignette order and unweighted JAMRIS-SIJ grade only order vignettes for all experts were 0.79 for inflammation and 0.80 for damage. The correlations of image vignettes among imaging experts to CAS were 0.75 for inflammation and 0.90 for damage. The multicriteria decision analysis identified differences in relative weights among the JAMRIS-SIJ measurement items. The determination of the relative weights provided expert-driven score scaling and face validity for the JAMRIS-SIJ, enabling the future evaluation of its longitudinal construct validity.

Development and Validation of the Tuberculosis Risk Score for Smokers (TBRSS)

Tuberculosis (TB) remains a significant public health issue worldwide. However, the effectiveness of TB screening programmes among smokers is still questionable. There is a need for a simple, reliable, and validated screening system for this risk population. This study aimed to develop and validate the tuberculosis risk score for smokers (TBRSS) in Kelantan, Malaysia. A case–control study was conducted on 159 patients (smokers with and without TB) between January and July 2020. Simple and multiple logistic regressions were applied to determine the variables to be included in the risk score. The cut-off points to determine a score indicating low or high risk for TB disease were obtained based on the receiver operating characteristics curve. Content validation was carried out through interviews with eight experts to measure each variable′s relevancy. The face validation was conducted among 20 health clinic staff. Seven variables were selected for inclusion in the risk score. The chosen cut-off point was 16 (out of 43), with 91% and 78% sensitivity and specificity, respectively. The scale-level content validity index was 0.83, while the face validity index scores for each element ranged between 0.85 and 1.00. The TBRSS can be considered a validated screening tool for use in screening TB disease risk among smokers, which potentially may lead to an increased detection of TB disease in the community.

The Development of a New International Generic Measure (EQ-HWB): Face Validity and Psychometric Stages in Argentina

OBJECTIVES

This study aimed to present the face validity and psychometric stages performed in Spanish in Argentina, the only Spanish-speaking country of an international collaboration that undertook the construction of a new measure that can be used in economic evaluation across health, social care, and public health, the EQ EQ-HWB (EQ Health and Wellbeing). We also explored the relationship among 3-level version EQ-5D (EQ-5D-3L), 5-level version EQ-5D (EQ-5D-5L), and EQ-HWB.

METHODS

Face validity was based on semistructured face to face interviews of a purposive sample to explore translatability of language and concepts of 97 candidate items, translated into Argentina Spanish. The psychometric evaluation using an online panel assessed the psychometric properties of 64 items that were carried forward (floor and ceiling effects, item correlations, known-group differences in relevant prespecified subgroups by the international and local teams, exploratory and confirmatory factor analysis, and item response theory). EQ-5D-3L, EQ-5D-5L, and EQ-HWB correlations were explored.

RESULTS

In the face validity stage, 24 interviews with carers, general public, patients, and users of social services were included. Most items showed adequate face validity. In the psychometric assessment, 497 participants were recruited (64% reporting a long-term health condition). Most of the items showed adequate psychometrics in an Argentinian context. EQ-5D-3L and EQ-5D-5L had strong correlations, and EQ-HWB was moderately correlated to EQ visual analog scale. The Argentina team recommended 23 of the final 25 items.

CONCLUSIONS

The assessment of Spanish items contributed to the overall development of EQ-HWB and helped inform the design of an internationally relevant 25-item and a short 9-item measure intended to be used in economic evaluations.

Generation, Selection, and Face Validation of Items for a New Generic Measure of Quality of Life: The EQ-HWB

OBJECTIVES

This article aims to describe the generation and selection of items (stage 2) and face validation (stage 3) of a large international (multilingual) project to develop a new generic measure, the EQ-HWB (EQ Health and Wellbeing), for use in economic evaluation across health, social care, and public health to estimate quality-adjusted life-years.

METHODS

Items from commonly used generic, carer, social care, and mental health quality of life measures were mapped onto domains or subdomains identified from a literature review. Potential terms and items were reviewed and refined to ensure coverage of the construct of the domains/subdomain (stage 2). Input on the potential item pool, response options, and recall period was sought from 3 key stakeholder groups. The pool of candidate items was tested in qualitative interviews with potential future users in an international face validation study (stage 3).

RESULTS

Stage 2 resulted in the generation of 687 items. Predetermined selection criteria were applied by the research team resulting in 598 items being dropped, leaving 89 items that were reviewed by key stakeholder groups. Face validation (stage 3) tested 97 draft items and 4 response scales. A total of 47 items were retained and 14 were modified, whereas 3 were added to the candidate pool of items. This resulted in a 64-item set.

CONCLUSIONS

This international multiculture, multilingual study with a common methodology identified many items that performed well across all countries. These were taken to the psychometric testing along with modified and new items for the EQ-HWB.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Translation, Cultural Adaptation, and Content Validation of the Palliative Care Self-Efficacy Scale for Use in the Swedish Context

The Palliative Care Self-Efficacy Scale (PCSE) is a valid instrument in English for assessing healthcare professionals’ self-efficacy in providing palliative care; it has not been translated into Swedish. The aim of this study was to describe the translation, cultural adaptation, and content-validation process of the PCSE scale. In this study, forward and backward translations, pretesting including an expert panel (n = 7), and cognitive interviewing with possible healthcare professionals (physicians, nurses, and assistant nurses) (n = 10) were performed. Experts in palliative care rated items on a Likert scale based on their understandability, clarity, sensitivity, and relevance. The item-level content validity index (I-CVI) and modified kappa statistics were calculated. Healthcare professionals were interviewed using the think-aloud method. The translation and validation process resulted in the final version of the PCSE scale. The average I-CVI for sensitivity was evaluated and approved. Most of the items were approved for clarity, some items lacked understandability, but a majority of the items were considered relevant. The healthcare professionals agreed that the items in the questionnaire evoked emotions, but were relevant for healthcare professionals. Thus, the Palliative Care Self-Efficacy scale is relevant for assessing healthcare professionals’ self-efficacy in palliative care in a Swedish care context. Further research using psychometric tests is required.

Content validity and reliability of the modified Japanese version of the Healthcare Providers Patient-Activation Scale: self-reported adherence to patient-centered physical therapy scale

BACKGROUND

Modifications to the Healthcare Providers Patient-Activation Scale (HP-PAS), which evaluates attitudes toward the importance of using a patient-centered approach (PCA), are required.

PURPOSE

Content validity and reliabilities of a scale developed from the HP-PAS to evaluate self-reported adherence to patient-centered physical therapy were preliminarily investigated.

METHODS

A total of 86 Japanese physical therapists completed the validity assessments in an anonymous survey. Among them, 53 completed the reliability assessments in a named survey. The HP-PAS was modified to mark the most suitable frequency for each item, creating an 11-point scale. For the validity assessments, the participants assigned each item to a therapist-centered approach or PCA using a 5-point scale. Content validity was considered when the proportion of scores 4 and 5 ("possibly" or "definitely" a PCA) with respect to the total number of scores was > 50%. Ceiling and flooring effects were considered when the response proportions of scores of 10 and 0, respectively, on an 11-point scale were > 20%. Items with an intraclass correlation coefficient (ICC) < 0.4 in the test-retest reliability from the item list were excluded.

RESULTS

Of the 20 items, 6 were excluded, developing a 14-item self-reported adherence to a patient-centered physical therapy scale (SAPCPTS), which exhibited good internal consistency (Cronbach's α = 0.95, 95% confident intervals [CIs] = 0.93-0.96) and test-retest reliability (ICC = 0.84, 95% CIs = 0.75-0.91).

CONCLUSION

This study identified that the 14-item SAPCPTS demonstrated preliminary evidence of content validity and reliability.

Content and Face Validation of a Novel, Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) to Evaluate Registered Dietitians' Competence: A Delphi Consensus from the United States

The nutrition-focused physical examination (NFPE) is an integral component of nutrition assessment performed by registered dietitian nutritionists (RDNs) to determine signs of malnutrition and other nutrition-related complications. Increased use of this essential skill among RDNs and the transformation of dietetics education to a competency-based model in the near future calls for appropriately validated tools to measure RDNs' NFPE competence. To fill the need for a validated competency tool, this study developed an Interactive Nutrition-Specific Physical Exam Competency Tool (INSPECT) utilizing the initial 70 items identified in the first phase of the study. The second phase of this study aimed to test the preliminary version of the INSPECT for content and face validity. An expert panel of 17 members provided consensus recommendations through the Delphi process. Internal consistency of the consensus was measured with Cronbach's alpha (α) and α of ≥0.70 was defined as acceptable a priori. Inter-rater agreement among the expert panel was determined using the intraclass correlation coefficient (ICC) and an a priori ICC of 0.75 to 0.9 was established as good and >0.9 as excellent agreement. The results showed acceptable face validity (α = 0.71) and excellent content validity for the INSPECT, with an internal consistency of α = 0.97 in the first round and α = 0.96 in the second round. The inter-rater agreement was also excellent with ICC = 0.95 for each of the Delphi rounds. A total of 52 items were retained from the preliminary version of the INSPECT. Open feedback from the experts allowed for the consolidation of 11 similar items for better scoring and evaluation and thus, a total of 41 items were included in the final version of the INSPECT. The final version of the INSPECT is currently being studied in real-life, multi-site clinical settings among practicing RDNs to examine construct validity, reliability, and item-level psychometric properties. Ultimately, the validated INSPECT will be available for the competency evaluation of RDNs practicing in clinical settings.

Development of the First Health-Related Quality of Life Questionnaires in Arabic for Women with Polycystic Ovary Syndrome (Part I): The Creation and Reliability Analysis of PCOSQoL-47 and PCOSQoL-42 Questionnaires

Background and objective

We lack a reliable and validated health-related quality of life (HRQoL) questionnaire to measure the negative impact of polycystic ovary syndrome (PCOS) on the various aspects of the lives of Arabic women, which addresses sexuality in married women only. Hence, we aimed to develop two separate, simple, reliable, validated, and easily interpretable HRQoL questionnaires in Arabic for married and unmarried women with PCOS for effective QoL evaluation.

Materials and methods

The development and reliability analysis of the Polycystic Ovary Syndrome Quality of Life (PCOSQoL)-47 and PCOSQoL-42 involved two quantitative and qualitative phases. Phase 1 included retrieval of 158 items from 584 PubMed articles, item reduction, Arabic translation, content and face validity testing, creation of a five-domain draft (53 items for married and 45 items for unmarried women), with no sexuality domain for unmarried women. Phase 2 involved test-retest reliability, which involved using the Spearman's correlation, Wilcoxon nonparametric signed-rank, and internal consistency using Cronbach's-alpha, inter-item, and intraclass correlation (ICC) coefficients, as well as creating a second draft (47 items for married and 42 items for unmarried women).

Results

The content validity indices testing by 26 healthcare experts decreased the item pool to 57 items for married and 45 items for unmarried women. Face validity by another 30 experts and 30 women from each group resulted in a further reduction to 53 items for married and 45 items for unmarried women, to be tested in a pilot study, which included another 30 women from each group. Test-retest reliability analysis by 195 married and 173 unmarried women revealed significantly excellent redundancy, reliability, and stability of items (highly significant Cronbach's alpha and ICC by internal consistency testing), and reduced the item pool to 47 items for married and 42 items for unmarried women.

Conclusions

Both questionnaires were found to be highly reliable for the HRQoL evaluation among both married and unmarried Arabic women with different phenotypes of PCOS.

Validity and reliability of the Danish version of the pregnancy physical activity questionnaire to assess levels of physical activity during pregnancy

INTRODUCTION

Physical inactivity during pregnancy is associated with complications on the mother and child. Thus, assessments of physical activity in pregnant women is relevant. The American validated questionnaire, Pregnancy Physical Activity Questionnaire (PPAQ), measures physical activity during pregnancy, and a translated Danish version of the questionnaire is needed.

OBJECTIVES

To translate the original version of PPAQ into Danish (PPAQ-DK) and to determine face and content validity, and reliability of PPAQ-DK in pregnant Danish women.

MATERIALS AND METHODS

PPAQ was translated into Danish through a systematic process. Subsequently, face and content validity was examined on pregnant women. PPAQ-DK was revised, and the test-retest reliability of PPAQ-DK was investigated in another sample of pregnant women, where the respondents completed the questionnaire twice with one week apart, using an Intraclass Correlation Coefficient (ICC). Internal consistency reliability was determined using Cronbach´s α.

RESULTS

The face validity of the original PPAQ was achieved by interviews of 10 pregnant women (age (mean ± SD): 31.0 ± 3.0 years; 50% in second and 50% in third trimester) and minor changes in the revised version of PPAQ-DK were performed. The content validity was accepted by an expert panel. In 66 pregnant women (aged 30.2 ± 4.8 years; 16.7, 63.6, and 19.7% in first, second and third trimester, respectively), the ICC in the test-retest reliability was 0.79 (0.68-0.87) of the overall PPAQ score. The Cronbach's α in the internal consistency reliability analysis was 0.7 in the overall PPAQ score.

CONCLUSIONS

The PPAQ-DK is within acceptable reliability and is a valid and reliable tool to measure overall physical activity level of pregnant Danish women.

How to develop a condition-specific PROM

Developing new patient-reported outcome measures (PROMs) for application in clinical studies can be necessary if an adequate PROM does not exist. For adequate measurement, it is essential that the PROM has face validity (ie, is perceived to be relevant by clinicians and researchers) and has high content validity (ie, content relevance and content coverage for the targeted patient group). The steps needed to create PROMs that possess face and content validity for a specific condition are described in this paper. Face validity is achieved by item identification and generation through literature review. Content validity is confirmed through repetitive cognitive interviews of patients from the targeted patient group in order to generate a consensus-based pilot-version of the new PROM. This qualitative process ensures that items are appropriately worded, understandable, and minimizes doubts about how items should be answered. A practical example of this process is presented, which shows the development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL), a condition-specific PROM for patients with deficiency of the anterior cruciate ligament (ACL).

Rationale, Relevance, and Limits of Stress-Induced Psychopathology in Rodents as Models for Psychiatry Research: An Introductory Overview

Emotional and cognitive information processing represent higher-order brain functions. They require coordinated interaction of specialized brain areas via a complex spatial and temporal equilibrium among neuronal cell-autonomous, circuitry, and network mechanisms. The delicate balance can be corrupted by stressful experiences, increasing the risk of developing psychopathologies in vulnerable individuals. Neuropsychiatric disorders affect twenty percent of the western world population, but therapies are still not effective for some patients. Elusive knowledge of molecular pathomechanisms and scarcity of objective biomarkers in humans present complex challenges, while the adoption of rodent models helps to improve our understanding of disease correlate and aids the search for novel pharmacological targets. Stress administration represents a strategy to induce, trace, and modify molecular and behavioral endophenotypes of mood disorders in animals. However, a mouse or rat model will only display one or a few endophenotypes of a specific human psychopathology, which cannot be in any case recapitulated as a whole. To override this issue, shared criteria have been adopted to deconstruct neuropsychiatric disorders, i.e., depression, into specific behavioral aspects, and inherent neurobiological substrates, also recognizable in lower mammals. In this work, we provide a rationale for rodent models of stress administration. In particular, comparing each rodent model with a real-life human traumatic experience, we intend to suggest an introductive guide to better comprehend and interpret these paradigms.

Validation of Texas Christian University Psychosocial Functioning and Motivation Scales in Iranian Persons Who Use Drugs

BACKGROUND

The psychosocial functioning and motivation scales of the Texas Cristian University (TCU) were translated and validated for the Persons Who Use Drugs (PWUDs).

METHODS

This study was carried out as a methodological research. The scales were translated forward and backward and then, the PWUDs in Kermanshah-based (west of Iran) substance abuse treatment centers in 2019 filled out the scales. Participants included 250 individuals who were selected by convenience sampling method. A demographics form and psychosocial functioning and motivation scales were used to collect data. For data analysis, content validity, face validity, testretest, internal consistency, and construct validity were applied by using exploratory analysis methods in SPSS (v.20) and confirmatory factor analysis in LISREL software.

RESULTS

Exploratory Factor Analysis (EFA) results were obtained by using the main component analysis and Varimax rotation of three factors (psychological function, social function, treatment motivation). Extracted factors, along with the eigenvalues, attributed variance percentage, and cumulative attributed variances by each factor, by 68.56%, showed a total variance of the questionnaire. Moreover, the fitness indexes of the Confirmatory Factor Analysis (CFA) of psychological function, social function, and treatment motivation were 0.059, 0.075 and 0.072, respectively. Therefore, the questionnaire fitness was at an acceptable level. The test-retest reliability coefficient was 0.875, and the Cronbachs alpha coefficient for all the components was higher than 0.71.

CONCLUSION

The results showed that psychosocial functioning, motivation scales have good psychometric properties, and they can be used in research and treatment fields.

Preliminary test-retest reliability of the Wheelchair Satisfaction Questionnaire

PURPOSE

Economic realities in lower-and-middle-income countries (LMICs) present an increased need for outcome measures for wheelchair efficacy, as these measures enable optimized use of funds. As the provision level of wheelchairs is low in these areas, and many wheelchairs are inappropriate for their intended users, use of funds based on evidence is especially necessary. The Wheelchair Satisfaction Questionnaire (WSQ) was designed to be a snapshot of a wheelchair user's level of satisfaction with their wheelchair. The WSQ is comprised of 16 visual analogue scale questions. Each question includes an option for a qualitative explanatory comment. The current study examined initial test-retest reliability of the WSQ.

MATERIALS AND METHODS

The WSQ was administered twice to the same set of wheelchair users who were secondary students at a school for students with disabilities. A demonstration was given to the participants to explain how to mark the analogue scale. Participants were instructed to answer each item honestly and without peer input. A one-week time span separated test and retest. Scores for both sessions were entered into SPSS. An interclass coefficient of 0.70 or above indicates acceptable test-retest reliability.

RESULTS

The ages of participants ranged from 13 to 24, with a mean age of 17.86 years. Sixty-five participants completed the questionnaire fully in both test and retest and were included in this study. A high degree of reliability was found between scores from both tests. The interclass coefficient was r(63) = .863, p = .01, indicating statistically significant agreement between test and retest.

CONCLUSION

The results support the WSQ as a reliable measure, confirming the WSQ as a reliable tool for user feedback on wheelchair function. Because the WSQ is designed to provide user feedback with enough granularity to give data on particular aspects of wheelchair structure and function, data can facilitate repair and modifications to wheelchair parts. Studies using the WSQ to assess specific wheelchair types could indicate consistent patterns of user satisfaction and dissatisfaction, revealing relevant design issues. The WSQ is designed to give wheelchair users a voice that can empower user centred modification and design changes to facilitate improved health, opportunity, and social interaction.Implications for RehabilitationResults from the WSQ could enable manufacturers to utilize user feedback to improve the design of wheelchairs for use in LMIC's, and providers could be better-informed in the selection of wheelchair types for specific environments.The WSQ could provide immediate user feedback to inform wheelchair modification and selection to best serve a particular user.The WSQ could be used in clinical settings over time to collect longitudinal data from wheelchair users, which could identify the most commonly perceived reasons for user dissatisfaction in a particular clinical setting.

Validation of a Sensor-Fitted Simulator for Upper Airway Examination

OBJECTIVE

To validate a simulator for upper airway examination, fitted with sensors, for use as an academic tool for learning how to conduct examination of the upper airway and for evaluation of that learning.

STUDY DESIGN

Validation study.

SETTING

Undergraduate medical education.

SUBJECTS AND METHODS

A group of 18 fifth-year medical students and another of 6 otorhinolaryngology specialists conducted 6 examinations each with the simulator. To investigate concurrent validity, we calculated the correlation between damage scores provided by the simulator and damage assessment by a specialist. To evaluate construct validity, we compared both groups with regard to damage scores, technical procedure, and time spent. To examine content and face validity, we used questionnaires based on a 5-point Likert scale.

RESULTS

For concurrent validity, the correlation between the simulator's damage scores and the specialist's damage assessment was high: Spearman's ρ was 0.828 (P < .001). For construct validity, the group of students differed from the group of specialists in damage scores (P = .027) and in technical procedures (P < .001) but not in time spent. For content validity, all questionnaire statements were scored highly, and both groups had similar average scores. For face validity, the group of specialists considered the simulator to be realistic, and all statements on the questionnaire were rated with at least 4/5.

CONCLUSION

Concurrent, construct, content, and face validity have been demonstrated for a sensor-fitted simulator for upper airway examination, which is therefore accurate enough to be used as an academic tool for learning and evaluation of learning.

Quantifying Spatiotemporal Gait Parameters with HoloLens in Healthy Adults and People with Parkinson's Disease: Test-Retest Reliability, Concurrent Validity, and Face Validity

Microsoft’s HoloLens, a mixed-reality headset, provides, besides holograms, rich position data of the head, which can be used to quantify what the wearer is doing (e.g., walking) and to parameterize such acts (e.g., speed). The aim of the current study is to determine test-retest reliability, concurrent validity, and face validity of HoloLens 1 for quantifying spatiotemporal gait parameters. This was done in a group of 23 healthy young adults (mean age 21 years) walking at slow, comfortable, and fast speeds, as well as in a group of 24 people with Parkinson’s disease (mean age 67 years) walking at comfortable speed. Walking was concurrently measured with HoloLens 1 and a previously validated markerless reference motion-registration system. We comprehensively evaluated HoloLens 1 for parameterizing walking (i.e., walking speed, step length and cadence) in terms of test-retest reliability (i.e., consistency over repetitions) and concurrent validity (i.e., between-systems agreement), using the intraclass correlation coefficient (ICC) and Bland–Altman’s bias and limits of agreement. Test-retest reliability and between-systems agreement were excellent for walking speed (ICC ≥ 0.861), step length (ICC ≥ 0.884), and cadence (ICC ≥ 0.765), with narrower between-systems than over-repetitions limits of agreement. Face validity was demonstrated with significantly different walking speeds, step lengths and cadences over walking-speed conditions. To conclude, walking speed, step length, and cadence can be reliably and validly quantified from the position data of the wearable HoloLens 1 measurement system, not only for a broad range of speeds in healthy young adults, but also for self-selected comfortable speed in people with Parkinson’s disease.

Face validity evaluation of screening tools for gaming disorder: Scope, language, and overpathologizing issues

AIM

Critics of gaming disorder (GD; i.e., Internet gaming disorder in the DSM-5; Gaming disorder in the ICD-11) have expressed concerns about the potential risks of misclassification (e.g., false positives). An important consideration of relevance to this discussion is the extent to which commonly used screening instruments contain appropriate, sensible, and relevant items. The aim of this review was to evaluate the face validity of items within current tools for GD.

METHODS

A systematic review of databases identified 29 instruments. An item bank (n = 417 items) was independently evaluated by three professional raters (i.e., a senior academic in clinical psychology, a senior psychometrician, and an academic/clinical psychologist) according to guidelines for defining and measuring addiction and gaming disorder.

FINDINGS

Evaluation of the item bank identified issues related to: scope (i.e., "scope creep" or items of questionable relevance); language (i.e., confusing language, unusual wording or syntax); and overpathologizing (i.e., pathologizing typical and/or beneficial aspects or consequences of gaming). A total of 71 items across 23 tools had at least one face validity issue.

CONCLUSIONS

Most items (83%) demonstrated satisfactory face validity and were consistent with either the DSM-5 or ICD-11 GD classification. However, many tests contain at least one item that may pathologize normal gaming behaviors. Such items refer to basic changes in mood when gaming, a desire to play or continue playing games, and experiencing immersion when gaming. This analysis highlights the challenges of screening for problematic behaviors that are thought to arise within the context of normal recreational activities.

Development of a new tool for the early identification of communication-support needs in people living with dementia: An Australian face-validation study

Language and communication difficulties are common in dementia but limited tools are available for a timely assessment of those individuals who experience these difficulties. The Communication-Support Needs Assessment Tool in Dementia (CoSNAT-D) was developed to assist in (a) the early identification of communication difficulties among people with dementia in the community context; and (b) determining the level of their communication support needs to guide appropriate service access. The CoSNAT-D was developed based on a literature review. The face validity was tested with end-users (people with dementia and support persons of people with dementia) regarding relevance, wording, syntax, appropriateness and comprehensiveness of the items of the tool. Data were collected using semi-structured phone interviews. Seven people with dementia and 15 carers participated in the study. Feedback regarding items' appropriateness was largely positive and minor changes were made to improve comprehensiveness. A new item was added to the original item pool. The interviews highlighted the importance of including people with dementia in the assessment processes and person-centered approaches in this context. The CoSNAT-D presents a first step for the early identification of individual support needs of people living with dementia and communication difficulties who live in the community. Items of the CoSNAT-D have been determined face-valid by end-users. The face-validated version of the tool is currently undergoing further testing to determine additional relevant psychometric properties.

Development and Validation of the Tele-Pulmonary Rehabilitation Acceptance Scale

BACKGROUND

Using telehealth in pulmonary rehabilitation (telerehabilitation) is a new field of health-care practice. To successfully implement a telerehabilitation program, measures of acceptance of this new type of program need to be assessed among potential users. The purpose of this study was to develop a scale to measure acceptance of using telerehabilitation by health-care practitioners and patients.

METHODS

Three objectives were met (a) constructing a modified scale of the technology acceptance model, (b) judging the items for content validity, and (c) judging the scale for face validity. Nine experts agreed to participate and evaluate item relevance to theoretical definitions of domains. To establish face validity, 7 health-care practitioners and 5 patients were interviewed to provide feedback about the scale's clarity and ease of reading.

RESULTS

The final items were divided into 2 scales that reflected the health-care practitioner and patient responses. Each scale included 3 subscales: perceived usefulness, perceived ease of use, and behavioral intention.

CONCLUSIONS

The 2 scales, each with 3 subscales, exhibited evidence of content validity and face validity. The 17-item telerehabilitation acceptance scale for health-care practitioners and the 13-item telerehabilitation acceptance scale among patients warrant further psychometric testing as valuable measures for pulmonary rehabilitation programs.

Development, reliability, and piloting of a wheelchair caster failure inspection tool (C-FIT)

Introduction: Wheelchair casters fail frequently in the field causing multiple user consequences and wheelchair breakdowns. To inform caster design improvement, there exists no validated tools that can collect caster failures. This need motivated the development of a user-reported, caster failure inspection tool (C-FIT).Methods: To develop C-FIT, a multistep design and testing approach was used which included face validity testing, test-retest reliability testing and expert review. Reliability testing was conducted with two independent cohorts of wheelchair professionals who inspected caster failures physically and online through pictures. The tool was revised based on testing outcomes and expert feedback. For preliminary data collection and evaluating usability, C-FIT was piloted at wheelchair service centers in Scotland, Indonesia and Mexico.Results: Caster failure items reported in the literature were screened to develop the initial list of C-FIT items. Face validity testing conducted through surveys with wheelchair experts (n = 6) provided 14 items for C-FIT inclusion. The test-retest reliability was found to be high for 10 items with physical failure inspections (n = 12). For each of these items, 75% or more participants had substantial to almost perfect agreement scores (κ = 0.6-1.0). Lower reliability scores were found with online failure inspections (n = 11). C-FIT received positive usability feedback from study participants and data collectors in the field. Pilot field data (n = 31) included comprehensive details about failures useful for manufacturers, designers and researchers to improve caster designs.Conclusions: The C-FIT tool developed in this study has substantial reliability and can be used for documenting caster failures at wheelchair service centers.Implications for rehabilitationCollecting data on caster failures is an important first step to inform design improvements and caster quality testing methods.The caster failure inspection tool is a reliable tool that can be used during wheelchair repair and servicing to collect caster failures in a standardized way.The failure data can be used by wheelchair manufacturers, designers, technicians and researchers to develop reliable caster designs. Wheelchair providers can select caster designs based on context of use.

Enhancing validity through cognitive interviewing. A methodological example using the Epilepsy Monitoring Unit Comfort Questionnaire

Aims

To identify problematic items, assess completeness and user‐friendliness, and undertake modifications to enhance face validity of the newly developed Epilepsy Monitoring Unit Comfort Questionnaire.

Design

Qualitative‐descriptive cross‐sectional study.

Methods

Five iterative rounds of cognitive interviewing were conducted with members of the target population between July ‐ November 2017. Think‐aloud technique, verbal probing, and observation were used, to assess how respondents understood and answered questions. Data analysis was based on the framework method; an eight‐column framework matrix was created for this purpose.

Results

In 25 interviews, problems regarding completeness of the item pool, comprehension of items, retrieval of information, judgment while finding answers and reporting the appropriate responses could be detected. According to the results, 27 items remained unchanged, 11 items were reworded, and six items were added. Instructions section of the questionnaire was reworked, too.

Conclusion

Although time‐consuming, cognitive interviewing turned out to be a valuable approach for revealing problems in an instrument, which would, otherwise, remain undetected and threaten validity.

Animal models of addiction

In recent years, animal models in psychiatric research have been criticized for their limited translational value to the clinical situation. Failures in clinical trials have thus often been attributed to the lack of predictive power of preclinical animal models. Here, I argue that animal models of voluntary drug intake—under nonoperant and operant conditions—and addiction models based on the Diagnostic and Statistical Manual of Mental Disorders are crucial and informative tools for the identification of pathological mechanisms, target identification, and drug development. These models provide excellent face validity, and it is assumed that the neurochemical and neuroanatomical substrates involved in drug-intake behavior are similar in laboratory rodents and humans. Consequently, animal models of drug consumption and addiction provide predictive validity. This predictive power is best illustrated in alcohol research, in which three approved medications—acamprosate, naltrexone, and nalmefene—were developed by means of animal models and then successfully translated into the clinical situation.

Robot Assisted Surgical Ward Rounds: Virtually Always There

BACKGROUND

 While an explosion in technological sophistication has revolutionized surgery within the operating theatre, delivery of surgical ward-based care has seen little innovation.  Use of telepresence allowing off-site clinicians communicate with patients has been largely restricted to outpatient settings or use of complex, expensive, static devices.  We designed a prospective study to ascertain feasibility and face validity of a remotely controlled mobile audiovisual drone (LUCY) to access inpatients.  This device is, uniquely, lightweight, freely mobile and emulates 'human' interaction by swiveling and adjusting height to patients' eye-level.   METHODS: Robot-assisted ward rounds(RASWR) were conducted over 3 months. A remotely located consultant surgeon communicated with patients/bedside teams via encrypted audiovisual telepresence robot (DoubleRoboticstm, California USA).  Likert-scale satisfaction questionnaires, incorporating free-text sections for mixed-methods data collection, were disseminated to patient and staff volunteers following RASWRs.  The same cohort completed a linked questionnaire following conventional (gold-standard) rounds, acting as control group. Data were paired, and non-parametric analysis performed.  RESULTS: RASWRs are feasible (>90% completed without technical difficulty). The RASWR(n=52 observations) demonstrated face validity with strong correlations (r>0.7; Spearman, p-value <0.05) between robotic and conventional ward rounds among patients and staff on core themes, including dignity/confidentiality/communication/satisfaction with management plan. Patients (96.08%, n=25) agreed RASWR were a satisfactory alternative when consultant physical presence was not possible. There was acceptance of nursing/NCHD cohort (100% (n=11) willing to regularly partake in RASWR).  CONCLUSION: RASWRs receive high levels of patient and staff acceptance, and offer a valid alternative to conventional ward rounds when a consultant cannot be physically present.

Animal models of amyotrophic lateral sclerosis: A comparison of model validity

Animal models are necessary to investigate the pathogenic features underlying motor neuron degeneration and for therapeutic development in amyotrophic lateral sclerosis (ALS). Measures of model validity allow for a critical interpretation of results from each model and caution from over-interpretation of experimental models. Face and construct validity refer to the similarity in phenotype and the proposed causal factor to the human disease, respectively. More recently developed models are restricted by limited phenotype characterization, yet new models hold promise for novel disease insights, thus highlighting their importance. In this article, we evaluate the features of face and construct validity of our new zebrafish model of environmentally-induced motor neuron degeneration and discuss this in the context of current environmental and genetic ALS models, including C9orf72, mutant Cu/Zn superoxide dismutase 1 and TAR DNA-binding protein 43 mouse and zebrafish models. In this mini-review, we discuss the pros and cons to validity criteria in each model. Our zebrafish model of environmentally-induced motor neuron degeneration displays convincing features of face validity with many hallmarks of ALS-like features, and weakness in construct validity. However, the value of this model may lie in its potential to be more representative of the pathogenic features underlying sporadic ALS cases, where environmental factors may be more likely to be involved in disease etiology than single dominant gene mutations. It may be necessary to compare findings between different strains and species modeling specific genes or environmental factors to confirm findings from ALS animal models and tease out arbitrary strain- and overexpression-specific effects.

Initial development of a patient-reported outcome measure of experience with cognitive impairment associated with schizophrenia

Cognitive impairment is a serious, often distressing aspect of schizophrenia that affects patients' day-to-day lives. Although several interview-based instruments exist to assess cognitive functioning, a reliable measure developed based on the experiences of patients facing cognitive difficulties is needed to complement the objective performance-based assessments. The present article describes the initial development of a patient-reported outcome (PRO) measure to assess the subjective experience of cognitive impairment among patients with schizophrenia, the Patient-Reported Experience of Cognitive Impairment in Schizophrenia (PRECIS). The phases of development included the construction of a conceptual model based on the existing knowledge and two sets of qualitative interviews with patients: 1) concept elicitation interviews to ensure face and content validity from the perspective of people with schizophrenia and 2) cognitive debriefing of the initial item pool. Input from experts was elicited throughout the process. The initial conceptual model included seven domains. The results from concept elicitation interviews (n=80) supported these domains but yielded substantive changes to concepts within domains and to terminology. Based on these results, an initial pool of 53 items was developed to reflect the most common descriptions and languages used by the study participants. Cognitive debriefing interviews (n=22) resulted in the removal of 18 items and modification of 22 other items. The remaining 35 items represented 23 concepts within six domains plus two items assessing bother. The draft PRO measure is currently undergoing psychometric testing as a precursor to broad-based clinical and research use.

How adults with cardiac conditions in Singapore understand the Patient Activation Measure (PAM-13) items: a cognitive interviewing study

PURPOSE

Validation studies of the PAM-13 have found differences in scale performance, suggesting that health beliefs embedded in different cultures and/or self-management needs of different client groups influence how people respond to the items. The purpose of this study was to examine how adults with cardiac conditions in Singapore interpreted and responded to the PAM-13, to investigate possible reasons for differences in responses and to propose solutions to overcome them.

METHODS

We conducted retrospective cognitive interviews with 13 participants in an out-patient heart center. Interviews were transcribed and analyzed based on the framework approach to qualitative analysis. The four stages from Tourangeau's cognitive model were used as a framework to index the data from each item.

RESULTS

There was variation in comprehension of questions leading to variation in responses. Comprehension issues were due to terms perceived by participants to be vague and the use of English terms uncommon in Singapore. Cultural influences impacted decision processes and problems with response processes of the self-rating Likert scale surfaced.

CONCLUSIONS

This study reinforces the need to culturally adapt the tool, even when language translation is not necessary. Providing Likert scales with a larger number of may widen the relevance of PAM-13 in Singapore. Implications for rehabilitation Need to culturally adapt assessment tool, even when language translation is not necessary. Consider using Likert scales with a larger number of categories when using in Asian countries such as Singapore. Caution must be taken when using PAM-13 levels to decide interventions for each individual.

Validity and reliability of a pictorial instrument for assessing perceived motor competence in Portuguese children

BACKGROUND

It is important to assess young children's perceived Fundamental Movement Skill (FMS) competence in order to examine the role of perceived FMS competence in motivation toward physical activity. Children's perceptions of motor competence may vary according to the culture/country of origin; therefore, it is also important to measure perceptions in different cultural contexts. The purpose was to assess the face validity, internal consistency, test-retest reliability and construct validity of the 12 FMS items in the Pictorial Scale for Perceived Movement Skill Competence for Young Children (PMSC) in a Portuguese sample.

METHODS

Two hundred one Portuguese children (girls, n = 112), 5 to 10 years of age (7.6 ± 1.4), participated. All children completed the PMSC once. Ordinal alpha assessed internal consistency. A random subsamples (n = 47) were reassessed one week later to determine test-retest reliability with Bland-Altman method. Children were asked questions after the second administration to determine face validity. Construct validity was assessed on the whole sample with a Bayesian Structural Equation Modelling (BSEM) approach. The hypothesized theoretical model used the 12 items and two hypothesized factors: object control and locomotor skills.

RESULTS

The majority of children correctly identified the skills and could understand most of the pictures. Test-retest reliability analysis was good, with an agreement ration between 0.99 and 1.02. Ordinal alpha values ranged from acceptable (object control 0.73, locomotor 0.68) to good (all FMS 0.81). The hypothesized BSEM model had an adequate fit.

CONCLUSIONS

The PMSC can be used to investigate perceptions of children's FMS competence. This instrument can also be satisfactorily used among Portuguese children.

Exploring the Validity of Valproic Acid Animal Model of Autism

The valproic acid (VPA) animal model of autism spectrum disorder (ASD) is one of the most widely used animal model in the field. Like any other disease models, it can't model the totality of the features seen in autism. Then, is it valid to model autism? This model demonstrates many of the structural and behavioral features that can be observed in individuals with autism. These similarities enable the model to define relevant pathways of developmental dysregulation resulting from environmental manipulation. The uncovering of these complex pathways resulted to the growing pool of potential therapeutic candidates addressing the core symptoms of ASD. Here, we summarize the validity points of VPA that may or may not qualify it as a valid animal model of ASD.

Assessing Technology in the Absence of Proof: Trust Based on the Interplay of Others' Opinions and the Interaction Process

OBJECTIVE

The present research addresses the question of how trust in systems is formed when unequivocal information about system accuracy and reliability is absent, and focuses on the interaction of indirect information (others' evaluations) and direct (experiential) information stemming from the interaction process.

BACKGROUND

Trust in decision-supporting technology, such as route planners, is important for satisfactory user interactions. Little is known, however, about trust formation in the absence of outcome feedback, that is, when users have not yet had opportunity to verify actual outcomes.

METHOD

Three experiments manipulated others' evaluations ("endorsement cues") and various forms of experience-based information ("process feedback") in interactions with a route planner and measured resulting trust using rating scales and credits staked on the outcome. Subsequently, an overall analysis was conducted.

RESULTS

Study 1 showed that effectiveness of endorsement cues on trust is moderated by mere process feedback. In Study 2, consistent (i.e., nonrandom) process feedback overruled the effect of endorsement cues on trust, whereas inconsistent process feedback did not. Study 3 showed that although the effects of consistent and inconsistent process feedback largely remained regardless of face validity, high face validity in process feedback caused higher trust than those with low face validity. An overall analysis confirmed these findings.

CONCLUSION

Experiential information impacts trust even if outcome feedback is not available, and, moreover, overrules indirect trust cues-depending on the nature of the former.

APPLICATION

Designing systems so that they allow novice users to make inferences about their inner workings may foster initial trust.

Face and content validity of a virtual-reality simulator for myringotomy with tube placement

BACKGROUND

Myringotomy with tube insertion can be challenging for junior Otolaryngology residents as it is one of the first microscopic procedures they encounter. The Western myringotomy simulator was developed to allow trainees to practice microscope positioning, myringotomy, and tube placement. This virtual-reality simulator is viewed in stereoscopic 3D, and a haptic device is used to manipulate the digital ear model and surgical tools.

OBJECTIVE

To assess the face and content validity of the Western myringotomy simulator.

METHODS

The myringotomy simulator was integrated with new modules to allow speculum placement, manipulation of an operative microscope, and insertion of the ventilation tube through a deformable tympanic membrane. A questionnaire was developed in consultation with instructing surgeons. Fourteen face validity questions focused on the anatomy of the ear, simulation of the operative microscope, appearance and movement of the surgical instruments, deformation and cutting of the eardrum, and myringotomy tube insertion. Six content validity questions focused on training potential on surgical tasks such as speculum placement, microscope positioning, tool navigation, ear anatomy, myringotomy creation and tube insertion. A total of 12 participants from the Department of Otolaryngology-Head and Neck Surgery were recruited for the study. Prior to completing the questionnaire, participants were oriented to the simulator and given unlimited time to practice until they were comfortable with all of its aspects.

RESULTS

Responses to 12 of the 14 questions on face validity were predominantly positive. One issue of concern was with contact modeling related to tube insertion into the eardrum, and the second was with the movement of the blade and forceps. The former could be resolved by using a higher resolution digital model for the eardrum to improve contact localization. The latter could be resolved by using a higher fidelity haptic device. With regard to content validity, 64% of the responses were positive, 21% were neutral, and 15% were negative.

CONCLUSIONS

The Western myringotomy simulator appears to have sufficient face and content validity. Further development with automated metrics and skills transference testing is planned.

A Methionine-Induced Animal Model of Schizophrenia: Face and Predictive Validity

BACKGROUND

Modulating the methylation process induces broad biochemical changes, some of which may be involved in schizophrenia. Methylation is in particular central to epigenesis, which is also recognized as a factor in the etiology of schizophrenia. Because methionine administration to patients with schizophrenia has been reported to exacerbate their psychotic symptoms and because mice treated with methionine exhibited social deficits and prepulse inhibition impairment, we investigated whether methionine administration could lead to behavioral changes that reflect schizophrenic symptoms in mice.

METHODS

l-Methionine was administered to mice twice a day for 7 days.

RESULTS

We found that this treatment induces behavioral responses that reflect the 3 types of schizophrenia-like symptoms (positive, negative, or cognitive deficits) as monitored in a battery of behavioral assays (locomotion, stereotypy, social interaction, forced swimming, prepulse inhibition, novel object recognition, and inhibitory avoidance). Moreover, these responses were differentially reversed by typical haloperidol and atypical clozapine antipsychotics in ways that parallel their effects in schizophrenics.

CONCLUSION

We thus propose the l-methionine treatment as an animal model recapitulating several symptoms of schizophrenia. We have established the face and predictive validity for this model. Our model relies on an essential natural amino acid and on an intervention that is relatively simple and time effective and may offer an additional tool for assessing novel antipsychotics.

The development and initial validation of a questionnaire to measure help-seeking behaviour in patients with new onset rheumatoid arthritis

BACKGROUND

Early treatment for rheumatoid arthritis (RA) is vital. However, people often delay in seeking help at symptom onset. An assessment of the reasons behind patient delay is necessary to develop interventions to promote rapid consultation.

OBJECTIVE

Using a mixed methods design, we aimed to develop and test a questionnaire to assess the barriers to help seeking at RA onset.

DESIGN

Questionnaire items were extracted from previous qualitative studies. Fifteen people with a lived experience of arthritis participated in focus groups to enhance the questionnaire's face validity. The questionnaire was also reviewed by groups of multidisciplinary health-care professionals. A test-retest survey of 41 patients with newly presenting RA or unclassified arthritis assessed the questionnaire items' intraclass correlations.

RESULTS

During focus groups, participants rephrased questions, added questions and deleted items not relevant to the questionnaire's aims. Participants organized items into themes: early symptom experience, initial reactions to symptoms, self-management behaviours, causal beliefs, involvement of significant others, pre-diagnosis knowledge about RA, direct barriers to seeking help and relationship with GP. The test-retest survey identified seven items (out of 79) with low intraclass correlations which were removed from the final questionnaire.

CONCLUSION

The involvement of people with a lived experience of arthritis and multidisciplinary health-care professionals in the preliminary validation of the DELAY (delays in evaluating arthritis early) questionnaire has enriched its development. Preliminary assessment established its reliability. The DELAY questionnaire provides a tool for researchers to evaluate individual, cultural and health service barriers to help-seeking behaviour at RA onset.

Schedule of factors influencing access to psychiatric treatment in persons with schizophrenia: validity and pilot testing

BACKGROUND

A substantial proportion of patients with severe mental disorders remain untreated in India. Qualitative research has highlighted the complex interplay of multiple factors that preclude schizophrenia patients in rural Indian settings from accessing treatment.

AIMS

(a) To establish the face and content validities of an interview schedule titled "Schedule of Factors Influencing Access (SOFIAc) to Psychiatric Treatment in Persons with Schizophrenia", which comprehensively assesses the factors that prevent schizophrenia patients from accessing psychiatric treatment. (b) To assess the feasibility of its administration.

MATERIALS AND METHODS

SOFIAc contains 15 items. This schedule involves three phases of interviewing patients and family members. This was given to 12 experts. They used Likert scales (1=not at all satisfactory to 5=very much satisfactory) to rate each item of the schedule. In addition, the experts rated (the same way as above) the following five dimensions of the schedule (as a whole) separately: A (comprehensiveness of the factors), B (scoring system), C (interviewing method), D (general instructions given to the raters), E (overall schedule). Later on, 10 persons with schizophrenia were interviewed with SOFIAc to test the feasibility of administration.

RESULTS

Thirteen items were rated as either satisfactory (score=4) or very much satisfactory (score=5) by all 12 experts; remaining two were rated as 4 or 5 by 11 experts. Regarding comprehensiveness of the factors, scoring methods and general instructions given to the interviewers, all provided scores >4; regarding the method of interviewing, 11 provided the score of >4; with regard to overall interview schedule, all experts provided scores >4. Pilot testing revealed that it took 60 min to administer SOFIAc.

CONCLUSION

SOFIAc has satisfactory face and content validities. It is also feasible to administer SOFIAc.

Face and construct validation of a virtual peg transfer simulator

BACKGROUND

The Fundamentals of Laparoscopic Surgery (FLS) trainer box is now established as a standard for evaluating minimally invasive surgical skills. A particularly simple task in this trainer box is the peg transfer task which is aimed at testing the surgeon's bimanual dexterity, hand-eye coordination, speed, and precision. The Virtual Basic Laparoscopic Skill Trainer (VBLaST) is a virtual version of the FLS tasks which allows automatic scoring and real-time, subjective quantification of performance without the need of a human proctor. In this article we report validation studies of the VBLaST peg transfer (VBLaST-PT) simulator.

METHODS

Thirty-five subjects with medical background were divided into two groups: experts (PGY 4-5, fellows, and practicing surgeons) and novices (PGY 1-3). The subjects were asked to perform the peg transfer task on both the FLS trainer box and the VBLaST-PT simulator; their performance was evaluated based on established metrics of error and time. A new length of trajectory (LOT) metric has also been introduced for offline analysis. A questionnaire was used to rate the realism of the virtual system on a 5-point Likert scale.

RESULTS

Preliminary face validation of the VBLaST-PT with 34 subjects rated on a 5-point Likert scale questionnaire revealed high scores for all aspects of simulation, with 3.53 being the lowest mean score across all questions. A two-tailed Mann-Whitney test performed on the total scores showed significant (p = 0.001) difference between the groups. A similar test performed on the task time (p = 0.002) and the LOT (p = 0.004) separately showed statistically significant differences between the experts and the novices (p < 0.05). The experts appear to be traversing shorter overall trajectories in less time than the novices.

CONCLUSION

VBLaST-PT showed both face and construct validity and has promise as a substitute for the FLS for training peg transfer skills.

A brief review on recent developments in animal models of schizophrenia

Number of patients suffering from schizophrenia is increasing daily, subsequently, increasing the need of proper medication to treat the symptoms and eventually improve the patients' condition. However, all the progress for designing or discovering medication comes to a standstill, as the symptomatic treatment can only be done in the patients, but performing clinical trials with all the possible candidate drugs in human beings and patients is unethical. Thus, the need arises for proper animal and non-human primate animal models of the disease, which would not only serve the purpose of understanding the disease in a better physiological setting, but also would allow the scientists to focus on developing a therapeutically effective and potent medication for treating this hazardous disease. This brief review article focuses on a few animal models which are generally used for carrying out studies on schizophrenic symptoms in research labs and industry worldwide. The paper also tries to validate the pre-clinically available models based on certain specified criteria like the predictive constructive and face validity. Thus, the paper gives guidance toward the mechanistic and traditional models of schizophrenia applying some of the newer principles and helps researchers in deciding a particular relevant model for their own purpose.