Defining "Ethical Mathematical Practice" Through Engagement with Discipline-Adjacent Practice Standards and the Mathematical Community

This project explored what constitutes "ethical practice of mathematics". Thematic analysis of ethical practice standards from mathematics-adjacent disciplines (statistics and computing), were combined with two organizational codes of conduct and community input resulting in over 100 items. These analyses identified 29 of the 52 items in the 2018 American Statistical Association Ethical Guidelines for Statistical Practice, and 15 of the 24 additional (unique) items from the 2018 Association of Computing Machinery Code of Ethics for inclusion. Three of the 29 items synthesized from the 2019 American Mathematical Society Code of Ethics, and zero of the Mathematical Association of America Code of Ethics, were identified as reflective of "ethical mathematical practice" beyond items already identified from the other two codes. The community contributed six unique items. Item stems were standardized to, "The ethical mathematics practitioner…". Invitations to complete the 30-min online survey were shared nationally (US) via Mathematics organization listservs and other widespread emails and announcements. We received 142 individual responses to the national survey, 75% of whom endorsed 41/52 items, with 90-100% endorsing 20/52 items on the survey. Items from different sources were endorsed at both high and low rates. A final thematic analysis yielded 44 items, grouped into "General" (12 items), "Profession" (10 items) and "Scholarship" (11 items). Moreover, for the practitioner in a leader/mentor/supervisor/instructor role, there are an additional 11 items (4 General/7 Professional). These results suggest that the community perceives a much wider range of behaviors by mathematicians to be subject to ethical practice standards than had been previously included in professional organization codes. The results provide evidence against the argument that mathematics practitioners engaged in "pure" or "theoretical" work have minimal, small, or no ethical obligations.

An international consensus on effective, inclusive, and career-spanning short-format training in the life sciences and beyond

Science, technology, engineering, mathematics, and medicine (STEMM) fields change rapidly and are increasingly interdisciplinary. Commonly, STEMM practitioners use short-format training (SFT) such as workshops and short courses for upskilling and reskilling, but unaddressed challenges limit SFT's effectiveness and inclusiveness. Education researchers, students in SFT courses, and organizations have called for research and strategies that can strengthen SFT in terms of effectiveness, inclusiveness, and accessibility across multiple dimensions. This paper describes the project that resulted in a consensus set of 14 actionable recommendations to systematically strengthen SFT. A diverse international group of 30 experts in education, accessibility, and life sciences came together from 10 countries to develop recommendations that can help strengthen SFT globally. Participants, including representation from some of the largest life science training programs globally, assembled findings in the educational sciences and encompassed the experiences of several of the largest life science SFT programs. The 14 recommendations were derived through a Delphi method, where consensus was achieved in real time as the group completed a series of meetings and tasks designed to elicit specific recommendations. Recommendations cover the breadth of SFT contexts and stakeholder groups and include actions for instructors (e.g., make equity and inclusion an ethical obligation), programs (e.g., centralize infrastructure for assessment and evaluation), as well as organizations and funders (e.g., professionalize training SFT instructors; deploy SFT to counter inequity). Recommendations are aligned with a purpose-built framework-"The Bicycle Principles"-that prioritizes evidenced-based teaching, inclusiveness, and equity, as well as the ability to scale, share, and sustain SFT. We also describe how the Bicycle Principles and recommendations are consistent with educational change theories and can overcome systemic barriers to delivering consistently effective, inclusive, and career-spanning SFT.

Urinary Symptoms Among People With Neurogenic Lower Urinary Tract Dysfunction (NLUTD) Vary by Bladder Management

Objectives

To determine whether assessment and decision-making around urinary symptoms in people with neurogenic lower urinary tract dysfunction (NLUTD) should depend on bladder management.

Methods

Three surveys of urinary symptoms associated with NLUTD (USQNBs) were designed specific to bladder management method for those who manage their bladders with indwelling catheter (IDC), intermittent catheter (IC), or voiding (V). Each was deployed one time to a national sample. Subject matter experts qualitatively assessed the wording of validated items to identify potential duplicates. Clustering by unsupervised structural learning was used to analyze duplicates. Each item was classified into mutually exclusive and exhaustive categories: clinically actionable ("fever"), bladder-specific ("suprapubic pain"), urine quality ("cloudy urine"), or constitutional ("leg pain").

Results

A core of 10 "NLUTD urinary symptoms" contains three clinically actionable, bladder-specific, and urine quality items plus one constitutional item. There are 9 (IDC), 11 (IC), and 8 (V) items unique to these instruments. One decision-making protocol applies to all instruments.

Conclusion

Ten urinary symptoms in NLUTD are independent of bladder management, whereas a similar number depend on bladder management. We conclude that assessment of urinary symptoms for persons with NLUTD should be specific to bladder management method, like the USQNBs are.

Urinary Symptoms Are Unrelated to Leukocyte Esterase and Nitrite Among Indwelling Catheter Users

Objectives

To explore the association between dipstick results and urinary symptoms.

Method

This was a prospective 12-month observational study of real-time self-administered urine dipstick results and symptoms in a community setting that included 52 spinal cord injury/disease (SCI/D) participants with neurogenic lower urinary tract dysfunction (NLUTD) who use an indwelling catheter. Symptoms were collected using the Urinary Symptom Questionnaire for Neurogenic Bladder-Indwelling Catheter (USQNB-IDC). The USQNB-IDC includes actionable (A), bladder (B1), urine quality (B2), and other (C) symptoms; analyses focused on A, B1, and B2 symptoms. Dipstick results include nitrite (NIT +/-), and leukocyte esterase (LE; negative, trace, small, moderate, or large). Dipstick outcomes were defined as strong positive (LE = moderate/large and NIT+), inflammation positive (LE = moderate/large and NIT-), negative (LE = negative/trace and NIT-), and indeterminate (all others).

Results

Nitrite positive dipsticks and moderate or large LE positive dipsticks were each observed in over 50% of the sample in every week. Strong positive dipstick results were observed in 35% to 60% of participants in every week. A, B1, or B2 symptoms co-occurred less than 50% of the time with strong positive dipsticks, but they also co-occurred with negative dipsticks. Participants were asymptomatic with a strong positive dipstick an average of 30.2% of the weeks. On average, 73% of the time a person had a negative dipstick, they also had no key symptoms (95% CI, .597-.865).

Conclusion

No association was observed between A, B1, and B2 symptoms and positive dipstick. A negative dipstick with the absence of key symptoms may better support clinical decision-making.

Intravesical Lactobacillus rhamnosus GG versus Saline Bladder Wash: Protocol for a Randomized, Controlled, Comparative Effectiveness Clinical Trial

Background

Complicated urinary tract infection (cUTI) is pervasive and costly among people with spinal cord injury (SCI) and neurogenic lower urinary tract dysfunction (NLUTD).

Objectives

To describe the protocol for a comparative effectiveness randomized controlled trial of intravesical Lactobacillus rhamnosus GG (LGG) versus saline bladder wash (BW) for self-management of urinary symptoms.

Methods

Comparative effectiveness trial of self-administered LGG versus saline bladder wash among 120 participants with SCI+NLUTD at least 6 months post SCI. The study has both treatment and prophylaxis phases. After predictive enrichment at screening, randomized participants will enter the treatment phase (6 months) in which they instill either LGG or normal saline after trigger symptoms occur (more cloudy or more foul-smelling urine). In the prophylaxis phase (6 months), participants will instill their respective intervention every 3 days after the first occurrence of trigger symptoms.

Results

Study results will provide a comparison of effects on Urinary Symptom Questionnaire for Neurogenic Bladder (USQNB) bladder and urine symptoms and episodes of "presumed UTIs"; number of days antibiotics were used (both self-reported); days of work, school, rehabilitation, or other activity lost due to urinary symptoms; engagement with the health care system; number of instillations; satisfaction; and safety.

Conclusion

cUTI is a variable clinical entity. Unlike clinical trials that assume a single, simple entity (UTI) in inclusion or outcome criteria, this protocol targets the mechanisms underlying cUTI causes and phenotypes. Featuring reliable and valid outcome measures with analytic methods specifically appropriate for quantifying self-report, patient self-management, inclusion of both intervention and prophylactic phases, and predictive enrichment, these design elements may be adopted for future research.

Development and Assessment of SCI Model Systems Complicated UTI Consensus Guidelines: A Psychometrically Designed Mixed-Methods Protocol

Background

Complicated UTI (cUTI) is highly prevalent among people with spinal cord injury and disease (SCI/D), but neither consistent nor evidence-based guidelines exist.

Objectives

We propose a two-phase, mixed-methods study to develop consensus around diagnostic and decision-making criteria for cUTI among people with SCI/D and the clinicians who treat them.

Methods

In phase 1 (qualitative), we will engage Spinal Cord Injury Model Systems (SCIMS) clinicians in focus groups to refine existing cUTI-related decision making using three reliable and validated Urinary Symptom Questionnaires for Neurogenic Bladder (USQNBs; intermittent catheterization, indwelling catheterization, and voiding) as points of departure, and then we will conduct a Delphi survey to explore and achieve consensus on cUTI diagnostic criteria among a nationally representative sample of clinicians from physical medicine and rehabilitation, infectious disease, urology, primary care, and emergency medicine. We will develop training materials based on these new guidelines and will deploy the training to both clinicians and consumers nationally. In phase 2 (quantitative), we will assess clinicians' uptake and use of the guidelines, and the impact of the guidelines training on consumers' self-management habits, engagement with the health care system, and antibiotic use over the 12 months after training.

Results

The output of this study will be diagnostic guidelines for cUTI among people with neurogenic lower urinary tract dysfunction (NLUTD) due to SCI/D, with data on uptake (clinicians) and impact (patients).

Conclusion

This mixed-methods protocol integrates formal psychometric methods with large-scale evidence gathering to derive consensus around diagnostic guidelines for cUTI among people with NLUTD due to SCI/D and provides information on uptake (clinicians) and impact (patients).

Independence of Urinary Symptoms and Urinary Dipstick Results in Voiders With Neurogenic Bladder

Background

Urinary symptoms and urinary tract infection (UTI) are frequent and burdensome problems associated with neurogenic lower urinary tract dysfunction.

Objectives

To determine whether an association exists between urinary symptoms and urine dipstick results among individuals with spinal cord injury (SCI) or multiple sclerosis (MS).

Methods

Prospective 12-month cohort study of 76 participants with SCI or MS who manage their bladders by voiding. Eligibility criteria included adults ≥18 years old, at least three UTIs since diagnosis, and residence in the United States. Participants completed the Urinary Symptoms Questionnaire for Neurogenic Bladder-Voider version (USQNB-V) biweekly (26 assessments) and tested their urine by dipstick at the same time. Symptom burden was estimated based on endorsements of USQNB-V symptoms classified as clinically actionable (9), bladder function (8), and urine quality (4). Urine dipstick results assessed were leukocyte esterase (LE) and nitrite (NIT).

Results

Participants were stratified into four groups based on etiology of neurologic dysfunction and whether they ever experienced any urinary symptoms (USx): SCI+USx (n = 14), SCI+NoUSx (n = 5), MS+USx (n = 32), and MS+NoUSx (n = 25). In descending order, symptom burden was greatest for the MS+USx group, followed by both SCI groups; it was lowest for MS+NoUSx. We assessed multiple definitions of "positive" dipstick and found evidence of independence of USQNB-V symptoms and urinary dipstick results with each definition. In each group, the median (and majority) of strong positive dipsticks did not coincide with any symptoms.

Conclusion

Among people with SCI or MS who void, self-administered urine dipstick results and urinary symptom reporting contribute independent information for clinical decision making.

Race-Dependent Association of Single-Nucleotide Polymorphisms in TrkB Receptor in People Living with HIV and Depression

Human immunodeficiency virus (HIV)-associated cognitive disorders (HAND) is characterized by impaired motor and intellectual functions, as well as mood disorders. Brain-derived neurotrophic factor and its receptor TrkB (or NTRK2) mediate the efficacy of antidepressant drugs. Genomic studies of BDNF/TrkB have implicated common single-nucleotide polymorphisms in the pathology of depression. In the current study, we investigated whether single-nucleotide polymorphisms (SNPs) (rs1212171, rs1439050, rs1187352, rs1778933, rs1443445, rs3780645, rs2378672, and rs11140800) in the NTRK2 has a functional impact on depression in HIV-positive subjects. We have utilized the Central Nervous System (CNS) HIV Antiretroviral Therapy Effects Research (CHARTER) cohort. Our methods explored the univariate associations of these SNPs with clinical (current and lifetime) diagnosis of depression via chi-square. The distribution of alleles was significantly different for African-Americans and Caucasians (non-Hispanic) for several SNPs, so our regression analyses included both "statistical controls" for race group and models for each group separately. Finally, we applied a method of simultaneous analysis of associations, estimating the mutually shared information across a system of variables, separately by race group. Our results indicate that there is no significant association between clinical diagnosis of major depression and these SNPs for either race group in any analysis. However, we identified that the SNP associations varied by race group and sex.

Reliability of the Urinary Symptom Questionnaires for people with neurogenic bladder (USQNB) who void or use indwelling catheters

STUDY DESIGN

This is a descriptive psychometrics study.

OBJECTIVES

Neurogenic lower urinary tract dysfunction (NLUTD), also called Neurogenic Bladder (NB), is a common and disruptive condition in a variety of neurologic diagnoses. Our team developed patient-centered instruments, Urinary Symptom Questionnaires for people with neurogenic bladder (USQNB), specific to people with NLUTD who manage their bladders with intermittent catheterization (IC), indwelling catheters (IDC), or who void (V). This article reports evidence of reliability of the IDC and V instruments.

SETTING

Online surveys completed by individuals in the United States with NLUTD due to spinal cord injury (SCI), or multiple sclerosis (MS) who manage their bladder with IDC (SCI, n = 306), or by voiding (SCI, n = 103; MS, n = 383).

METHODS

Reliability estimates were based on endorsement of the items on the USQNB-IDC and USQNB-V. Reliability evidence was representativeness of these symptoms for a national sample (by determining if endorsement > 10%); internal consistency estimates (by Cronbach's alpha and item correlation coefficient, ICC); and interrelatedness of the items (by inferred Bayesian network, BN). We also tested whether a one-factor conceptualization of "urinary symptoms in NLUTD" was supportable for either instrument.

RESULTS

All items were endorsed by >20% of our samples. Urine quality symptoms tended to be the most commonly endorsed on both instruments. Cronbach's alpha and ICC estimates were high (>0.74), but not suggestive of redundancy. BNs showed interpretable associations among the items, and did not discover uninterpretable or unexpected associations. Neither instrument fit a one-factor model, as expected.

CONCLUSIONS

The USQNB-IDC and USQNB-V instruments show sufficient, multidimensional reliability for implementation and further study.

Clinical Profiles and Symptom Burden Estimates to Support Decision-Making Using the Urinary Symptom Questionnaire for People with Neurogenic Bladder (USQNB) using Intermittent Catheters

OBJECTIVE

To describe the scoring approach, considering interpretability, validity, and use, of a new patient-centered patient reported outcome (PRO), the Urinary Symptom Questionnaire for Neurogenic Bladder-Intermittent Catheter version (USQNB-IC).

DESIGN

Subject matter experts (researchers, clinicians, a consumer, a psychometrician) classified USQNB-IC items. Profiles were then composed based on self-management decisions made by patients; patient management decisions made by clinicians; and research-oriented decisions made by investigators. Participants in an 18-month pilot study completed the USQNB-IC every week. Differences in decisions based on traditional 'total scores' and profiles were examined. Validity was defined based on alignment of scoring method with decisions.

SETTING

A new set of patient-centered PROs enable monitoring and decision-making around urinary signs and symptoms among people with neurogenic bladder (NB).

PARTICIPANTS

Classifications of USQNB-IC items by subject matter experts. Utility of the classifications and profiles that were created was assessed using weekly responses from the 6-month baseline period from 103 participants in a pilot study.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Classification of the 29 symptoms resulted in four categories with exchangeability within-category and nonexchangeability across categories. The burden of each symptom type is one approach to scoring the USQNB-IC. Five profiles, based on these categories, emerged based on, and supportive of, decisions to be made according to symptoms, representing a categorical approach to scoring the USQNB-IC.

RESULTS

USQNB-IC items are not all exchangeable. Four symptom classifications comprise within-class exchangeable items. Five profiles emerged to summarize these items to promote decision-making and identification of change over time. Both ways to "score" the USQNB-IC are described and discussed.

CONCLUSIONS

"Profiling" promotes valid and interpretable decisions by patients and clinicians, based on a patient's urinary symptoms with the USQNB-IC cross-sectionally and longitudinally. Alternatively, four subsets of the 29 USQNB-IC symptoms can be used as continuous outcomes representing "burden" in clinical management or research.

Effects of Intravesical Lactobacillus Rhamnosus GG on Urinary Symptom Burden in People with Neurogenic Lower Urinary Tract Dysfunction

OBJECTIVE

To test the effectiveness of intravesical Lactobacillus rhamnosus GG (LGG) to reduce the burden of urinary symptoms for individuals with spinal cord injury and disease (SCI/D) with neurogenic lower urinary tract dysfunction (NLUTD) who manage their bladders with intermittent catheterization (IC).

DESIGN

A three-phase study (6 months each in baseline, intervention, and washout). Participants self-managed following the Self-Management Protocol using Probiotics (SMP-Pro), completing the online Urinary Symptom Questionnaire for Neurogenic Bladder-IC version (USQNB-IC) weekly.

SETTING

Nationwide (United States).

PARTICIPANTS

Ninety-six adults and seven children with SCI/D.

INTERVENTIONS

In response to one or both of the SMP-Pro trigger urinary symptoms, "cloudier" or "foul smelling" urine, participants self-administered using a clean urinary catheter an LGG+ Normal Saline instillate once or twice in a 30-hour period.

MAIN OUTCOME MEASURES

Change in USQNB-IC burden was adjusted individually according to the previous phase for four symptom types. Adjusted changes in burden between the intervention and washout phases were analyzed using one-sample t-tests. Holm correction was applied for the four types of symptoms: A, clinically actionable; B1, bladder function; B2, urine quality; and C, other.

RESULTS

During the intervention phase, participants met SMP-Pro instillation criteria 3.83 times on average (range 1-20). An average of 5.6 doses of LGG were instilled. For those who instilled at least once, burdens of type A and B2 symptoms were significantly improved at washout (both adjusted P < .05).

CONCLUSIONS

Self-instilled LGG seemed to improve "clinically actionable" (A) and "urine quality" (B2) symptom burden. No changes were observed for those who did not instill. This first-in-human clinical trial supports ongoing research of intravesical LGG, and the SMP-Pro for urinary symptoms.

Effects of Intravesical Lactobacillus Rhamnosus GG on Urinary Symptom Burden in People with Neurogenic Lower Urinary Tract Dysfunction

OBJECTIVE

To test the effectiveness of intravesical Lactobacillus rhamnosus GG (LGG) to reduce the burden of urinary symptoms for individuals with spinal cord injury and disease (SCI/D) with neurogenic lower urinary tract dysfunction (NLUTD) who manage their bladders with intermittent catheterization (IC).

DESIGN

A three-phase study (6 months each in baseline, intervention, and washout). Participants self-managed following the Self-Management Protocol using Probiotics (SMP-Pro), completing the online Urinary Symptom Questionnaire for Neurogenic Bladder-IC version (USQNB-IC) weekly.

SETTING

Nationwide (United States).

PARTICIPANTS

Ninety-six adults and seven children with SCI/D.

INTERVENTIONS

In response to one or both of the SMP-Pro trigger urinary symptoms, "cloudier" or "foul smelling" urine, participants self-administered using a clean urinary catheter an LGG+ Normal Saline instillate once or twice in a 30-hour period.

MAIN OUTCOME MEASURES

Change in USQNB-IC burden was adjusted individually according to the previous phase for four symptom types. Adjusted changes in burden between the intervention and washout phases were analyzed using one-sample t-tests. Holm correction was applied for the four types of symptoms: A, clinically actionable; B1, bladder function; B2, urine quality; and C, other.

RESULTS

During the intervention phase, participants met SMP-Pro instillation criteria 3.83 times on average (range 1-20). An average of 5.6 doses of LGG were instilled. For those who instilled at least once, burdens of type A and B2 symptoms were significantly improved at washout (both adjusted P < .05).

CONCLUSIONS

Self-instilled LGG seemed to improve "clinically actionable" (A) and "urine quality" (B2) symptom burden. No changes were observed for those who did not instill. This first-in-human clinical trial supports ongoing research of intravesical LGG, and the SMP-Pro for urinary symptoms.

An Actigraphy-Based Validation Study of the Sleep Disorder Inventory in the Nursing Home

Background: Disrupted sleep is common among nursing home patients with dementia and is associated with increased agitation, depression, and cognitive impairment. Detecting and treating sleep problems in this population are therefore of great importance, albeit challenging. Systematic observation and objective recordings of sleep are time-consuming and resource intensive and self-report is often unreliable. Commonly used proxy-rated scales contain few sleep items, which affects the reliability of the raters' reports. The present study aimed to adapt the proxy-rated Sleep Disorder Inventory (SDI) to a nursing home context and validate it against actigraphy. Methods: Cross-sectional study of 69 nursing home patients, 68% women, mean age 83.5 (SD 7.1). Sleep was assessed with the SDI, completed by nursing home staff, and with actigraphy (Actiwatch II, Philips Respironics). The SDI evaluates the frequency, severity, and distress of seven sleep-related behaviors. Internal consistency of the SDI was evaluated by Cronbach's alpha. Spearman correlations were used to evaluate the convergent validity between actigraphy and the SDI. Test performance was assessed by calculating the sensitivity, specificity, and predictive values, and by ROC curve analyses. The Youden's Index was used to determine the most appropriate cut-off against objectively measured sleep disturbance defined as <6 h nocturnal total sleep time (TST) during 8 h nocturnal bed rest (corresponding to SE <75%). Results: The SDI had high internal consistency and convergent validity. Three SDI summary scores correlated moderately and significantly with actigraphically measured TST and wake-after-sleep-onset. A cut-off score of five or more on the SDI summed product score (sum of the products of the frequency and severity of each item) yielded the best sensitivity, specificity, predictive values, and Youden's Index. Conclusion: We suggest a clinical cut-off for the presence of disturbed sleep in institutionalized dementia patients to be a SDI summed product score of five or more. The results suggest that the SDI can be clinically useful for the identification of disrupted sleep when administered by daytime staff in a nursing home context. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT03357328.

The Mastery Rubric for Bioinformatics: A tool to support design and evaluation of career-spanning education and training

As the life sciences have become more data intensive, the pressure to incorporate the requisite training into life-science education and training programs has increased. To facilitate curriculum development, various sets of (bio)informatics competencies have been articulated; however, these have proved difficult to implement in practice. Addressing this issue, we have created a curriculum-design and -evaluation tool to support the development of specific Knowledge, Skills and Abilities (KSAs) that reflect the scientific method and promote both bioinformatics practice and the achievement of competencies. Twelve KSAs were extracted via formal analysis, and stages along a developmental trajectory, from uninitiated student to independent practitioner, were identified. Demonstration of each KSA by a performer at each stage was initially described (Performance Level Descriptors, PLDs), evaluated, and revised at an international workshop. This work was subsequently extended and further refined to yield the Mastery Rubric for Bioinformatics (MR-Bi). The MR-Bi was validated by demonstrating alignment between the KSAs and competencies, and its consistency with principles of adult learning. The MR-Bi tool provides a formal framework to support curriculum building, training, and self-directed learning. It prioritizes the development of independence and scientific reasoning, and is structured to allow individuals (regardless of career stage, disciplinary background, or skill level) to locate themselves within the framework. The KSAs and their PLDs promote scientific problem formulation and problem solving, lending the MR-Bi durability and flexibility. With its explicit developmental trajectory, the tool can be used by developing or practicing scientists to direct their (and their team’s) acquisition of new, or to deepen existing, bioinformatics KSAs. The MR-Bi is a tool that can contribute to the cultivation of a next generation of bioinformaticians who are able to design reproducible and rigorous research, and to critically analyze results from their own, and others’, work.

A developmental trajectory supporting the evaluation and achievement of competencies: Articulating the Mastery Rubric for the nurse practitioner (MR-NP) program curriculum

Background

Advanced practice registered nursing (APRN) competencies exist, but there is no structure supporting the operationalization of the competencies by APRN educators. The development of a Mastery Rubric (MR) for APRNs provides a developmental trajectory that supports educational institutions, educators, students, and APRNs. A MR describes the explicit knowledge, skills, and abilities as performed by the individual moving from novice (student) through graduation and into the APRN career.

Method

A curriculum development tool, the Mastery Rubric (MR), was created to structure the curriculum and career of the nurse practitioner (NP), the MR-NP. Cognitive task analysis (CTA) yielded the first of the three required elements for any MR: a list of knowledge, skills, and abilities (KSAs) to be established through the curriculum. The European guild structure and Bloom’s taxonomy of cognitive behaviors provided the second element of the MR, the specific developmental stages that are relevant for the curriculum. The Body of Work method of standard setting was used to create the third required element of the MR, performance level descriptors (PLDs) for each KSA at each of these stages. Although the CTA was informed by the competencies, it was still necessary to formally assess the alignment of competencies with the resulting KSAs; this was achieved via Degrees of Freedom Analysis (DoFA). Validity evidence was obtained from this Analysis and from the DoFA of the KSAs’ alignment with principles of andragogy, and with learning outcomes assessment criteria. These analyses are the first time the national competencies for the NP have been evaluated in this manner.

Results

CTA of the 43 NP Competencies led to seven KSAs that support a developmental trajectory for instruction and documenting achievement towards independent performance on the competencies. The Competencies were objectively evaluable for the first time since their publication due to the psychometric validity attributes of the PLD-derived developmental trajectory. Three qualitatively distinct performance levels for the independent practitioner make the previously implicit developmental requirements of the competencies explicit for the first time.

Discussion

The MR-NP provides the first articulated and observable developmental trajectory for the NP competencies, during and beyond the formal curriculum. A focus on psychometric validity was brought to bear on how learners would demonstrate their development, and ultimately their achievement, of the competencies. The MR-NP goes beyond the competencies with trajectories and PLDs that can engage both learner and instructor in this developmental process throughout the career.

A new pan-European Train-the-Trainer programme for bioinformatics: pilot results on feasibility, utility and sustainability of learning

Demand for training life scientists in bioinformatics methods, tools and resources and computational approaches is urgent and growing. To meet this demand, new trainers must be prepared with effective teaching practices for delivering short hands-on training sessions—a specific type of education that is not typically part of professional preparation of life scientists in many countries. A new Train-the-Trainer (TtT) programme was created by adapting existing models, using input from experienced trainers and experts in bioinformatics, and from educational and cognitive sciences. This programme was piloted across Europe from May 2016 to January 2017. Preparation included drafting the training materials, organizing sessions to pilot them and studying this paradigm for its potential to support the development and delivery of future bioinformatics training by participants. Seven pilot TtT sessions were carried out, and this manuscript describes the results of the pilot year. Lessons learned include (i) support is required for logistics, so that new instructors can focus on their teaching; (ii) institutions must provide incentives to include training opportunities for those who want/need to become new or better instructors; (iii) formal evaluation of the TtT materials is now a priority; (iv) a strategy is needed to recruit, train and certify new instructor trainers (faculty); and (v) future evaluations must assess utility. Additionally, defining a flexible but rigorous and reliable process of TtT ‘certification’ may incentivize participants and will be considered in future.

Degrees of freedom analysis in educational research and decision-making: leveraging qualitative data to promote excellence in bioinformatics training and education

Qualitative data are commonly collected in higher, graduate and postgraduate education; however, perhaps especially in the quantitative sciences, utilization of these qualitative data for decision-making can be challenging. A method for the analysis of qualitative data is the degrees of freedom analysis (DoFA), published in 1975. Given its origins in political science and its application in mainly business contexts, the DoFA method is unlikely to be discoverable or used to understand survey or other educational data obtained from teaching, training or evaluation. This article therefore introduces and demonstrates the DoFA with modifications specifically to support educational research and decision-making with examples in bioinformatics. DoFA identifies and aligns theoretical or applied principles with qualitative evidence. The demonstrations include two hypothetical examples, and a case study of the role of scaffolding in an independent project ('capstone') of a graduate course in biostatistics. Included to promote inquiry, inquiry-based learning and the development of research skills, the capstone is often scaffolded (instructor-supported and therefore, formative), although it is intended to be summative. The case analysis addresses the question of whether the scaffolding provided for a capstone assignment affects its utility for formative or summative assessment. The DoFA is also used to evaluate the relative efficacies of other models for scaffolding the capstone project. These examples are intended to both explain this method and to demonstrate how it can be used to make decisions within a curriculum or for bioinformatics training.

Intravesical Lactobacillus rhamnosus GG is safe and well tolerated in adults and children with neurogenic lower urinary tract dysfunction: first-in-human trial

BACKGROUND

Urinary symptoms are common for people with neurogenic lower urinary tract dysfunction (NLUTD). No nonprescription approach has been proven safe and effective for self-management of urinary symptoms. Our objective was to describe the safety and tolerability of Lactobacillus rhamnosus GG (LGG®) instilled intravesically for self-management of inflammatory urinary symptoms in adults and children with NLUTD due to spinal cord injury or disease (SCI/D) and who use intermittent catheterization (IC).

METHODS

A total of 103 individuals with SCI/D enrolled in an 18-month study consisting of three 6-month stages: baseline (weekly observation of urinary symptoms); intervention (self-instilled intravesical LGG® in response to more cloudy or foul-smelling urine); and washout (weekly observation of urinary symptoms). Urinary symptoms were assessed using the Urinary Symptom Questionnaire for people with neurogenic bladder using intermittent catheters (USQNB-IC). Safety was based on serious adverse events and adverse events (S/AEs) and trends in symptoms. Tolerability was defined as the independence of AE experience and willingness to use/pay for this intervention.

RESULTS

A total of 74 (77%) adults and 6 (86%) of children completed the study, of whom 64 instilled LGG® for a total of 357 instillations (range 1-41 per person). There were 59 S/AEs, 44% (26/59) of which were categorized as infectious genitourinary. There was no statistical relationship between S/AEs and use or dose of the intervention.

CONCLUSIONS

One or two doses of self-instilled intravesical LGG® in response to more cloudy or foul-smelling urine was safe and well tolerated among this sample of adults and children with SCI/D who have NLUTD and use IC.

Preliminary validation of a Urinary Symptom Questionnaire for individuals with Neuropathic Bladder using Intermittent Catheterization (USQNB-IC): A patient-centered patient reported outcome

BACKGROUND

We developed a Urinary Symptom Questionnaire for individuals with neurogenic bladder due to spinal cord injury (SCI) and spina bifida (SB) who manage their bladders with intermittent catheterization, the USQNB-IC. This project followed an approach to patient-centered patient reported outcomes development that we created and published in 2017, specifically to ensure the primacy of the patient's perspective and experience.

PARTICIPANTS

Two sets of responses were collected from individuals with neurogenic bladder due to either SCI (n = 336) and SB (patients, n = 179; and caregivers of patients with NB, n = 66), and three sets of "controls", individuals with neurogenic bladder who do not have a history of UTIs (n = 49) individuals with chronic mobility impairments (neither SCI nor SB) and without neurogenic bladder (n = 46), and those with no mobility impairment, no neurogenic bladder, and no history of UTIs (n = 64).

METHOD

Data were collected from all respondents to estimate these psychometric or measurement domains characterizing a health related PRO: Reliability (minimization of measurement error; internal consistency or interrelatedness of the items; and maximization of variability that is due to "true" difference between levels of the symptoms across patients), and validity (content, reflection of the construct to be measured; face, recognizability of the contents as representing the construct to be measured; structural, the extent to which the instrument captures recognizable dimensions of the construct to be measured; and criterion, association with a gold standard).

RESULTS

Evidence from these five groups of respondents suggest the instrument has face, content, criterion, convergent, and divergent validity, as well as reliability. The items were all more descriptive of our patient (focus) groups and were only weakly endorsed by the control groups.

CONCLUSIONS

The instrument is unique in its emphasis on, and origination from, the lived experiences of patients with neurogenic bladder who use intermittent catheterization; this preliminary psychometric evidence suggests the instrument could be useful for research and in the clinic. These results justify further development of the instrument, including formal exploration of the scoring and estimation of responsivity of these items to clinical interventions as well as patient-directed self care.

Supporting Evidence-Informed Teaching in Biomedical and Health Professions Education Through Knowledge Translation: An Interdisciplinary Literature Review

Phenomenon: The purpose of "systematic" reviews/reviewers of medical and health professions educational research is to identify best practices. This qualitative article explores the question of whether systematic reviews can support "evidence informed" teaching and contrasts traditional systematic reviewing with a knowledge translation (KT) approach to this objective.

APPROACH

Degrees of freedom analysis (DOFA) is used to examine the alignment of systematic review methods with educational research and the pedagogical strategies and approaches that might be considered with a decision-making framework developed to support valid assessment. This method is also used to explore how KT can be used to inform teaching and learning.

FINDINGS

The nature of educational research is not compatible with most (11/14) methods for systematic review. The inconsistency of systematic reviewing with the nature of educational research impedes both the identification and implementation of "best-evidence" pedagogy and teaching. This is primarily because research questions that do support the purposes of review do not support educational decision making. By contrast to systematic reviews of the literature, both a DOFA and KT are fully compatible with informing teaching using evidence. A DOFA supports the translation of theory to a specific teaching or learning case, so could be considered a type of KT. The DOFA results in a test of alignment of decision options with relevant educational theory, and KT leads to interventions in teaching or learning that can be evaluated. Examples of how to structure evaluable interventions are derived from a KT approach that are simply not available from a systematic review. Insights: Systematic reviewing of current empirical educational research is not suitable for deriving or supporting best practices in education. However, both "evidence-informed" and scholarly approaches to teaching can be supported as KT projects, which are inherently evaluable and can generate actionable evidence about whether the decision or intervention worked for students, instructors, and the institution. A DOFA can also support evidence- and theory-informed teaching to develop an understanding of what works, why, and for whom. Thus KT, but not systematic reviewing, can support decision making around pedagogy (and pedagogical innovation) that can also inform new teaching and learning initiatives; it can also point to new avenues of empirical research in education that are informed by, and can inform, theory.

Maintaining primacy of the patient perspective in the development of patient-centered patient reported outcomes

The objectives of this study were to describe and demonstrate a new model of developing patient reported outcomes (PROs) that are patient-centered, and to test the hypothesis that following this model would result in a qualitatively different PRO than if the typical PRO development model were followed. The typical process of developing PROs begins with an initial list of signs or symptoms originating from clinicians or PRO developers; patient validation of this list ensures that the list (i.e., the new PRO) is interpretable by patients, but not that patient perspectives are central or even represented. The new model begins with elicitation from clinicians and patients independently and separately. These perspectives are formally analyzed qualitatively, and the results are iteratively integrated by researchers, supporting clinical relevance and patient centeredness. We describe the application of this new model to the development of a PRO for urinary signs and symptoms in individuals with neuropathic bladder, and test the hypothesis that the two processes generate qualitatively different instruments using a national validation sample of 300 respondents. Of its 29 items, the new instrument included 13 signs/symptoms derived from existing clinical practice guidelines, with 16 others derived from the patient/focus groups. The three most-endorsed items came from the patients, and the three least-endorsed items came from clinical guidelines. Thematic qualitative analysis of the elicitation process, as well as the results from our national sample, support the conclusion that the new model yields an instrument that is clinically interpretable, but more patient-centered, than the typical model would have done in this context.

Cardiometabolic Syndrome in People With Spinal Cord Injury/Disease: Guideline-Derived and Nonguideline Risk Components in a Pooled Sample

OBJECTIVE

To assess cardiometabolic syndrome (CMS) risk definitions in spinal cord injury/disease (SCI/D).

DESIGN

Cross-sectional analysis of a pooled sample.

SETTING

Two SCI/D academic medical and rehabilitation centers.

PARTICIPANTS

Baseline data from subjects in 7 clinical studies were pooled; not all variables were collected in all studies; therefore, participant numbers varied from 119 to 389. The pooled sample included men (79%) and women (21%) with SCI/D >1 year at spinal cord levels spanning C3-T2 (American Spinal Injury Association Impairment Scale [AIS] grades A-D).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

We computed the prevalence of CMS using the American Heart Association/National Heart, Lung, and Blood Institute guideline (CMS diagnosis as sum of risks ≥3 method) for the following risk components: overweight/obesity, insulin resistance, hypertension, and dyslipidemia. We compared this prevalence with the risk calculated from 2 routinely used nonguideline CMS risk assessments: (1) key cut scores identifying insulin resistance derived from the homeostatic model 2 (HOMA2) method or quantitative insulin sensitivity check index (QUICKI), and (2) a cardioendocrine risk ratio based on an inflammation (C-reactive protein [CRP])-adjusted total cholesterol/high-density lipoprotein cholesterol ratio.

RESULTS

After adjustment for multiple comparisons, injury level and AIS grade were unrelated to CMS or risk factors. Of the participants, 13% and 32.1% had CMS when using the sum of risks or HOMA2/QUICKI model, respectively. Overweight/obesity and (pre)hypertension were highly prevalent (83% and 62.1%, respectively), with risk for overweight/obesity being significantly associated with CMS diagnosis (sum of risks, χ(2)=10.105; adjusted P=.008). Insulin resistance was significantly associated with CMS when using the HOMA2/QUICKI model (χ(2)2=21.23, adjusted P<.001). Of the subjects, 76.4% were at moderate to high risk from elevated CRP, which was significantly associated with CMS determination (both methods; sum of risks, χ(2)2=10.198; adjusted P=.048 and HOMA2/QUICKI, χ(2)2=10.532; adjusted P=.04).

CONCLUSIONS

As expected, guideline-derived CMS risk factors were prevalent in individuals with SCI/D. Overweight/obesity, hypertension, and elevated CRP were common in SCI/D and, because they may compound risks associated with CMS, should be considered population-specific risk determinants. Heightened surveillance for risk, and adoption of healthy living recommendations specifically directed toward weight reduction, hypertension management, and inflammation control, should be incorporated as a priority for disease prevention and management.

The Mastery Rubric for Evidence-Based Medicine: Institutional Validation via Multidimensional Scaling

CONSTRUCT: In this study we describe a multidimensional scaling (MDS) exercise to validate the curricular elements composing a new Mastery Rubric (MR) for a curriculum in evidence-based medicine (EBM). This MR-EBM comprises 10 elements of knowledge, skills, and abilities (KSAs) representing our institutional learning goals of career-spanning engagement with EBM. An MR also includes developmental trajectories for each KSA, beginning with medical school coursework, including residency training, and outlining the qualifications of individuals to teach and mentor in EBM. The development was not part of the validation effort, as our curriculum is focused at a single stage (undergraduate medical students).

BACKGROUND

An MR comprises the desired KSAs for an entire curriculum, together with descriptions of a learner's performance and/or capabilities as they develop from novice to proficiency of the curricular target(s). The MR construct is intended to support curriculum development or refinement by capturing the KSAs that support the articulation of concrete learning goals; it also promotes assessment that demonstrates development in the target KSAs and encourages reflection and self-directed learning throughout the learner's career. Two other MRs have been published, and this is the first one specific to teaching and learning in medicine; this is also the first one created specifically to evaluate an existing curriculum.

APPROACH

To validate the dispersion of the elements of the EBM curriculum, the nine clinical instructors in the EBM two-course curriculum completed an MDS exercise, rating the similarities of the 10 curricular elements. MDS is a mathematical approach to understanding relationships among concepts/objects when these relationships are difficult to quantify. Eliciting similarity ratings biased the responses toward the null hypothesis (that the elements are not different).

RESULTS

MDS results suggested that the MR represents 10 different, although related, facets of the construct "evidence-based medicine." The results support the makeup of the MR-EBM, and its use to revise our EBM curriculum so that it is more closely aligned with this MR.

CONCLUSIONS

An MR is a tool, and the MR-EBM that we describe can be useful to develop or evaluate a curriculum in EBM. The MR tool is particularly compatible with the objectives of training for EBM and practice and can be applied to create or evaluate a curriculum using any topical KSA framework. The MR-EBM we describe could be adopted or adapted to represent other institutional objectives for EBM training.

Using Ethical Reasoning to Amplify the Reach and Resonance of Professional Codes of Conduct in Training Big Data Scientists

The use of Big Data--however the term is defined--involves a wide array of issues and stakeholders, thereby increasing numbers of complex decisions around issues including data acquisition, use, and sharing. Big Data is becoming a significant component of practice in an ever-increasing range of disciplines; however, since it is not a coherent "discipline" itself, specific codes of conduct for Big Data users and researchers do not exist. While many institutions have created, or will create, training opportunities (e.g., degree programs, workshops) to prepare people to work in and around Big Data, insufficient time, space, and thought have been dedicated to training these people to engage with the ethical, legal, and social issues in this new domain. Since Big Data practitioners come from, and work in, diverse contexts, neither a relevant professional code of conduct nor specific formal ethics training are likely to be readily available. This normative paper describes an approach to conceptualizing ethical reasoning and integrating it into training for Big Data use and research. Our approach is based on a published framework that emphasizes ethical reasoning rather than topical knowledge. We describe the formation of professional community norms from two key disciplines that contribute to the emergent field of Big Data: computer science and statistics. Historical analogies from these professions suggest strategies for introducing trainees and orienting practitioners both to ethical reasoning and to a code of professional conduct itself. We include two semester course syllabi to strengthen our thesis that codes of conduct (including and beyond those we describe) can be harnessed to support the development of ethical reasoning in, and a sense of professional identity among, Big Data practitioners.

A multicenter intervention to improve ambulatory care handoffs at the end of residency

BACKGROUND

Graduating residents transition their continuity clinic patients to junior colleagues every year, creating a vulnerable transition period for about 1 million patients nationally.

OBJECTIVE

We examined a standardized, electronic template for handing off high-risk ambulatory patients by outgoing residents from 7 residencies within a large health care system, and compared handoff quantity and provider satisfaction for handoffs with and without that template.

METHODS

Residents graduating in 2011 from 5 internal medicine, 1 family medicine, and 1 internal medicine-pediatrics residency programs in 1 health care system were randomized to a new electronic handoff process with a standardized intervention template or a free-text handoff. Expert reviewers independently evaluated all handoff notes, and providers were surveyed after follow-up appointments regarding use, helpfulness, and overall satisfaction with the handoffs.

RESULTS

Fifty-two of 79 residents (66%) participated, performing 278 handoffs. Eighty-four patients (30%, 17 of 57) failed to follow up within the study period. For patients who followed up, providers read 61% (101 of 165) of the handoffs at the time of the visit. No significant difference existed between groups in the satisfaction of the follow up provider or the quality measure of the handoffs in our process. Expert agreement on which features make the handoff "helpful" was fair (κ  =  0.34).

CONCLUSIONS

A standardized template did not improve handoff quantity or satisfaction compared with a free-text handoff. Practical handoff programs can be instituted into diverse residencies within a short time frame, with most residents taking part in creating the handoff formats.

Systematic training in internal medicine-pediatrics end of residency handoffs: residency director attitudes and perceived barriers

BACKGROUND

It is unclear why systematic training in end-of-residency clinic handoffs is not universal.

PURPOSES

We assessed Internal Medicine-Pediatrics (Med-Peds) residency program directors' attitudes regarding end-of-residency clinic handoff systems and perceived barriers to their implementation.

METHODS

We surveyed all Med-Peds program directors in the United States about end-of-residency outpatient handoff systems.

RESULTS

Program directors rated systems as important (81.5%), but only 31 programs (46.3%) utilized them. Nearly all programs with (29/31 [93.5%]), and most programs without systems (24/33 [72.7%]) rated them as important. Programs were more likely to have a system if the program director rated it important (p = .049), and less likely if they cited a lack of faculty interest (p = .023) or difficulty identifying residents as primary providers (p = .04).

CONCLUSIONS

Most program directors believe it important to formally hand off outpatients. Barriers to establishing handoff systems can be overcome with modest curricular and cultural changes.

Multiple choice questions can be designed or revised to challenge learners' critical thinking

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

Detecting When "Quality of Life" Has Been "Enhanced": Estimating Change in Quality of Life Ratings

OBJECTIVE

To demonstrate challenges in the estimation of change in quality of life (QOL).

METHODS

Data were taken from a completed clinical trial with negative results. Responses to 13 QOL items were obtained 12 months apart from 258 persons with Alzheimer's disease (AD) participating in a randomized, placebo-controlled clinical trial with two treatment arms. Two analyses to estimate whether "change" in QOL occurred over 12 months are described. A simple difference (later - earlier) was calculated from total scores (standard approach). A Qualified Change algorithm (novel approach) was applied to each item: differences in ratings were classified as either: improved, worsened, stayed poor, or stayed "positive" (fair, good, excellent). The strengths of evidence supporting a claim that "QOL changed", derived from the two analyses, were compared by considering plausible alternative explanations for, and interpretations of, results obtained under each approach.

RESULTS

Total score approach: QOL total scores decreased, on average, in the two treatment (both -1.0, p < 0.05), but not the placebo (=-0.59, p > 0.3) groups. Qualified change approach: Roughly 60% of all change in QOL items was worsening in every arm; 17% - 42% of all subjects experienced change in each item.

CONCLUSIONS

Totalling the subjective QOL item ratings collapses over items, and suggests a potentially misleading "overall" level of change (or no change, as in the placebo arm). Leaving the items as individual components of "quality" of life they were intended to capture, and qualifying the direction and amount of change in each, suggests that at least 17% of any group experienced change on every item, with 60% of all observed change being worsening.

DISCUSSION

Summarizing QOL item ratings as a total "score" collapses over the face-valid, multi-dimensional components of the construct "quality of life". Qualified Change provides robust evidence of changes to QOL or "enhancements of" life quality.

Current Practice in End-of-Residency Handoffs: A Survey of Internal Medicine-Pediatrics Program Directors

BACKGROUND

End-of-residency outpatient handoffs affect at least 1 million patients per year, yet there is no consensus on best practices.

OBJECTIVE

To explore the use of formal systems for end-of-residency clinic handoffs in internal medicine-pediatrics residency (Med-Peds) programs, and their associated categorical internal medicine and pediatrics programs.

METHODS

We surveyed Med-Peds program directors about their programs' system for handing off ambulatory continuity patients.

RESULTS

Our response rate was 85% (67 of 79 programs). Thirty-one programs (46%) reported having a system for end-of-residency handoffs. Of the 30 that offered detailed information, 22 (73%) formally introduced the program to residents, 12 (40%) standardized the handoff, and 14 (47%) used multiple methods for information exchange, with the electronic health record and oral transfer of information (15 of 30, 50%) the most common. Six programs (20%) indicated they did not offer residents protected time to complete end-of-residency handoffs, and 13 programs (43%) did not identify a specific postgraduate year level for residents to whom patients were handed off. Programs were more likely to have a system for end-of-residency handoffs if another categorical program at their institution also had one (P < .001).

CONCLUSIONS

Fewer than half of responding Med-Peds programs have outpatient handoff systems in place. Inclusion of end-of-residency handoff information in the electronic health record may represent a best practice that has the potential of enhancing continuity and safety of care for patients in resident continuity clinics.

Using the Guttman scale to define and estimate measurement error in items over time: the case of cognitive decline and the meaning of "points lost"

We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as "points lost" in studies of cognitive decline or interventions. To capture this meaning of "point loss", over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of "true score", which are two key consequences of the definition of "measurement error"--and thereby, reliability--under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of "cognitive loss" was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.

Intra-individual variability in Alzheimer's disease and cognitive aging: definitions, context, and effect sizes

BACKGROUND/AIMS

To explore different definitions of intra-individual variability (IIV) to summarize performance on commonly utilized cognitive tests (Mini Mental State Exam; Clock Drawing Test); compare them and their potential to differentiate clinically-defined populations; and to examine their utility in predicting clinical change in individuals from the Alzheimer's Disease Neuroimaging Initiative (ADNI).

METHODS

Sample statistics were computed from ADNI cohorts with no cognitive diagnosis, a diagnosis of mild cognitive impairment (MCI), and a diagnosis of possible or probable Alzheimer's disease (AD). Nine different definitions of IIV were computed for each sample, and standardized effect sizes (Cohen's d) were computed for each of these definitions in 500 simulated replicates using scores on the Mini Mental State Exam and Clock Drawing Test. IIV was computed based on test items separately ('within test' IIV) and the two tests together ('across test' IIV). The best performing definition was then used to compute IIV for a third test, the Alzheimer's Disease Assessment Scale-Cognitive, and the simulations and effect sizes were again computed. All effect size estimates based on simulated data were compared to those computed based on the total scores in the observed data. Association between total score and IIV summaries of the tests and the Clinician's Dementia Rating were estimated to test the utility of IIV in predicting clinically meaningful changes in the cohorts over 12- and 24-month intervals.

RESULTS

ES estimates differed substantially depending on the definition of IIV and the test(s) on which IIV was based. IIV (coefficient of variation) summaries of MMSE and Clock-Drawing performed similarly to their total scores, the ADAS total performed better than its IIV summary.

CONCLUSION

IIV can be computed within (items) or across (totals) items on commonly-utilized cognitive tests, and may provide a useful additional summary measure of neuropsychological test performance.

Independent contributions of neural and "higher-order" deficits to symptoms in Alzheimer's disease: a latent variable modeling approach

BACKGROUND

Analytic models of Alzheimer's disease (AD) tend to focus on one type of symptom and assume implicitly that no measurement error is present. These tendencies render changes in symptom domains difficult to model mathematically, although latent variable methods can accommodate both multiple symptom domains and error. This study formulated and compared underlying (latent) factor structures representing previously reported dependence and independence of symptoms of cognitive decline, functional impairment, and behavioral disturbance in AD.

METHODS

In confirmatory factor analyses of data from 2 cohorts of AD patients, 2 levels of latent variables were conceptualized. One general neurologic factor represented disease, and symptom factors represented cognition, function, and behavior. Two "null" models had either a single factor or 3 symptom factors. Two 2-level models treated the general factor as underlying both the observed variables and the symptom factors or treated the symptom factors as explaining variability in the observed variables after taking the general factor into account ("residualized").

RESULTS/CONCLUSIONS

The residualized model fit the data in both cohorts significantly better than the other models, and relations in this model between some observed and latent variables were different across cohorts. Neither cohort supported a single factor model; both cohorts independently supported a residualized model that may permit differentiation of symptom- from disease-modifying effects of treatment.

Incidence of atypical handedness in epilepsy and its association with clinical factors

The incidence of atypical handedness (left-handedness and ambidexterity) in patients with epilepsy, particularly its association with major clinical factors, is not well established. We evaluated a full range of clinical variables in 478 patients with epilepsy from the United States and Korea. With the Edinburgh Handedness Inventory, handedness was established as both a categorical variable (right-handed, left-handed, ambidextrous) and a continuous variable. Seizures were classified as complex or simple partial, primary generalized, or generalized tonic-clonic. The relationship between handedness and a range of clinical findings was explored. The overall incidence of atypical handedness in our patients was higher than in the general population (13.6%) and significantly higher in the U.S. patient group (17.6%) than in the Korean patients (8.8%). Handedness was not associated with sex; age; seizure type; age at onset; type, side, or site of EEG or brain imaging abnormalities; family history of seizures; refractory epilepsy; or history of epilepsy surgery.

Agreement in cognitive and clinical assessments in Alzheimer's disease

AIMS

To estimate agreement among scores on three common assessments of cognitive function.

METHOD

Baseline responses on the Alzheimer's Disease Assessment Scale - Cognitive, Clinical Dementia Rating, and the Mini-Mental State Examination were obtained from two clinical trials (n = 138 and n = 351). A graphical method of examining agreement, the means-difference or Bland-Altman plot, was followed by Levene's test of the equality of variance corrected for multiple comparison within each sample.

RESULTS

70-78% of variability was shared by one factor, suggesting that all three instruments reflect cognitive impairment. However, agreement among tests was significantly worse for individuals with greater-than-average, relative to individuals with less-than-average, cognitive impairment.

CONCLUSIONS

Worse agreement between tests, as a function of increasing cognitive impairment, implies that interpretation of these tests and selection of coprimary cognitive impairment outcomes may depend on impairment level.

Analytic methods for factors, dimensions and endpoints in clinical trials for Alzheimer's disease

Alzheimer's disease (AD) is a complex disease process, so finding a single biomarker to track in clinical trials has proven difficult. This paper describes and contrasts statistical methods that might be used with biomarkers in clinical trials for AD, highlighting their differences, limitations and interpretations. The first method is traditional regression, within which one dependent variable, the Best Empirically Supported Indicator (BESI), must be identified. In this approach one biomarker (e.g., the ratio of tau to Abeta42 from CSF) is the indicator for an individual's disease status, and change in that status. The second approach is an exploratory factor analysis (EFA) to consolidate a multitude of candidate dependent variables into a sample-dependent, mathematically-optimized smaller set of 'factors'. The third method is latent variable (LV) modeling of multiple indicators of an entity (e.g., "disease burden"). The LV approach can yield a complex 'dependent variable', the Best Measurement Model Indicator (BMMI). A measurement model represents an entity that several dependent variables reflect or measure, and so can include many 'dependent variables', and estimate their relative contributions to the underlying entity. The selection of a single BESI is an artifact of regression that limits the investigator's ability to utilize all relevant variables representing the entity of interest. EFA results in sample-specific combination of biomarkers that might not generalize to a new sample - and fit of the EFA results cannot be tested. Latent variable methods can be useful to construct powerful, efficient statistical models that optimally combine diverse biomarkers into a single, multidimensional dependent variable that can generalize across samples when they are theory-driven and not sample-dependent. This paper shows that EFA can work to uncover underlying structure, but that it does not always yield solutions that 'fit' the data. It is not recommended as a method to build BMMIs, which will be useful in establishing diagnostic criteria, creating and evaluating benchmarks, and monitoring progression in clinical trials.

A Mastery Rubric: guiding curriculum design, admissions and development of course objectives

This article describes a 'Mastery Rubric' (MR) used to design both the curriculum and the assessments in a new two-year certificate programme intended to train physicians in clinical research skills. The MR for clinical research skills is built around a set of core research skills: critical review of literature; articulation of research objective; development of research design; development of analysis plan; implementation of the study; implementation of the analysis plan and presentation of results. Four distinct levels of performance are described for each skill: beginning, novice, competent and proficient. This rubric outlines and provides a path to mastery of the clinical research skills the certificate programme was designed and funded to target. Using the rubric to design the curriculum ensures that courses will provide instruction in key domains, promotes assessment that demonstrates development in the target skills and knowledge, and encourages reflection and cognitive self-monitoring in the students. It is a flexible, criterion-referenced definition of 'success' for students as well as the programme itself. The criteria are characterised in terms of the skills, habits of mind and organisational principles that can foster excellence in clinical research, but the approach can be generalised.

Towards the integration of medical informatics education for clinicians into the medical curriculum

In the context of an existing first year, one-semester mandatory course of medical informatics (MI) for medical students, we tested an interactive teaching approach in parallel with the traditional academic program. After six semesters (at the beginning of the clinical stage) we collected feedback from the former students in the two parallel programs (with anonymous questionnaires comprising both subjectively-rated items and open-ended questions). We conclude that an introductory course on information and communication technology and information skills can be useful at the beginning of the medical curriculum, while an interactive, problem-based-learning-type MI course should be included during the clinical stage. Early development of these skills, and their use/utility across the curriculum, are important aspects of integrating MI education into clinical training.

Oral gabapentin for the treatment of postoperative pain after photorefractive keratectomy

PURPOSE

To evaluate oral gabapentin for postoperative pain after photorefractive keratectomy (PRK).

DESIGN

Prospective, nonrandomized clinical trial.

METHODS

In additional to a standard regimen of topical antibiotics, topical steroids, and topical tetracaine as required, all PRK patients at our laser vision center were treated after surgery for pain for a two-month period with Percocet (oxycodone/acetaminophen) [Endo Pharmaceuticals; Chadds Ford, Pennsylvania, USA] 5 mg/325 mg as required for three days (control group). Patients completed a pain assessment survey using a faces pain scale (from zero through 6) on the evening of surgery and each subsequent morning and evening until postoperative day 3. A successive cohort of patients received Neurontin (gabapentin) [Pfizer, New York, New York, USA] 300 mg thrice daily (first dose administered two hours or more before the procedure) as an oral pain medication for three days, and the same survey data were collected.

RESULTS

Data were collected on 141 patients in each cohort. Successful pain management score (defined as faces zero through 2 on the scale) differences did not reach statistical significance between the two cohorts except on the morning of the second postoperative day, when gabapentin was superior. On all postoperative days, patients in the oxycodone/acetaminophen cohort used significantly less tetracaine eye drops as required. The percent of patients rating overall pain experience as better than expected was 35% and 36%, those rating pain experience as about what was expected was 50% and 49%, and those rating pain experience as worse than expected was 15% and 15% in the oxycodone/acetaminophen and gabapentin cohorts, respectively.

CONCLUSIONS

We found no difference in overall subjective pain management ratings between gabapentin and oxycodone/acetaminophen for postoperative PRK pain, although gabapentin was associated with significantly more frequent use of anesthetic eye drops as required.

No cross-sectional influence of APOE epsilon4 dose on clinical tests in Alzheimer's disease

This study sought to determine if there are detectible influences on the symptoms of Alzheimer's disease (AD) from the genetic risk factor for AD, the epsilon4 allele of apolipoprotein-E (APOE). Using data from two cohorts of AD patients, a cross-sectional latent variable model of AD was tested with three symptom factors explaining variability in the observed variables after taking a general neurological factor into account. No significant influence of epsilon4 was detected. APOE's effect in AD may occur prior to clinical symptoms, or may simply be more subtle than these instruments can detect.

Promoting self-awareness and reflection through an experiential mind-body skills course for first year medical students

BACKGROUND

This research examines student evaluations of their experience and attitudes in an 11 week mind-body skills course for first year medical students.

AIMS

The aim is to understand the impact of this course on students' self-awareness, self-reflection, and self-care as part of their medical education experience.

METHODS

This study uses a qualitative content analysis approach to data analysis. The data are 492 verbatim responses from 82 students to six open-ended questions about the students' experiences and attitudes after a mind-body skills course. These questions queried students' attitudes about mind-body medicine, complementary medicine, and their future as physicians using these approaches.

RESULTS

The data revealed five central themes in students' responses: connections, self discovery, stress relief, learning, and medical education.

CONCLUSIONS

Mind-body skills groups represent an experiential approach to teaching mind-body techniques that can enable students to achieve self-awareness and self-reflection in order to engage in self-care and to gain exposure to mind-body medicine while in medical school.

Assessing and analyzing change in attitudes in the classroom

We explore three analytic methods that can be used to quantify and qualify changes in attitude and similar outcomes that may be encountered in the educational context. These methods can be used or adapted whenever the outcome of interest is change in a generally unmeasurable attribute, such as attitude. The analyses we describe focus on: (1) change in total 'attitude score'; (2) item-level changes in attitudes towards different topics; and (3) 'attitude shift' that is defined based on a qualified change algorithm. In our example data, the total-score approach gives a general index to the level of positive attitude; the item-level approach gives the median level of positive attitude and indicates items with the most positive/negative attitude (i.e., items to target in future iterations). The qualified change approach provides an objective measure of whether a shift in attitude has occurred. Each analysis is described with its advantages and disadvantages using the data from a survey of 70 preclinical first and second year medical students before and after an elective 11-week interactive seminar (22 contact hours) which introduced elements of complementary and alternative medicine (CAM) into programmed medical school training. When assessing changes that are more qualitative than quantitative, any of these methods can be employed to derive either descriptive or inferential statistics. The methods are straightforward and are appropriate when measurements are imperfect, ratings are subjective and differences are not necessarily absolute.

Use of steroid profiles in determining the cause of adrenal insufficiency

HYPOTHESIS

A cortisol response to adrenocorticotropin injection is the standard test for diagnosing adrenal insufficiency. Multiple steroid hormones can now be accurately measured by tandem mass spectrometry in a single sample. The study objective was to determine whether a steroid profile, created by simultaneous measurement of 10 steroid hormones by tandem mass spectrometry, would help determine the cause of adrenal insufficiency.

DESIGN

A 10-steroid profile was measured by tandem mass spectrometry during the performance of a standard high dose cortrosyn stimulation test. The steroids were measured at baseline, 30, and 60min following synthetic adrenocorticotropin injection. Adrenal insufficiency was defined as a peak cortisol level of less than 20microg/dL. Testing was conducted in the general clinical research center of a university medical center. Normal volunteers, patients suspected of having adrenal insufficiency, and patients with known adrenal insufficiency participated.

RESULTS

Our results showed that adrenal insufficiency of any cause was adequately diagnosed using the response of 11-deoxycortisol, dehydroepiandrosterone, or these analytes combined in a two-steroid profile. A three-steroid profile yielded a test with 100% accuracy for discriminating primary adrenal insufficiency from normal status. Primary adrenal insufficiency was well separated from secondary adrenal insufficiency using only a single aldosterone value. 11-Deoxycortisol, dehydroepiandrosterone, and a two-steroid profile each provided fair discrimination between secondary adrenal insufficiency and normal status.

CONCLUSIONS

We conclude that stimulated levels of aldosterone, 11-deoxycortisol, dehydroepiandrosterone, and a two- or three-steroid profile provided additional discrimination between states of adrenal sufficiency and insufficiency. It is proposed that a steroid profile measuring cortisol, aldosterone, 11-deoxycortisol, and dehydroepiandrosterone would potentially improve the ability to determine the cause of adrenal insufficiency.

A large-sample survey of first- and second-year medical student attitudes toward complementary and alternative medicine in the curriculum and in practice

PURPOSE

To assess attitudes toward complementary and alternative medicine (CAM) and its place in the medical school curriculum and medical practice among preclinical students at Georgetown University School of Medicine (GUSOM), Washington, DC.

METHOD

Two-hundred sixty-six first-year (n=111) and second-year (n=155) medical students rated their attitudes toward CAM and 15 CAM modalities in terms of personal use, inclusion in the curriculum, and use/utility in clinical practice.

RESULTS

Nearly all (91%) students agreed that "CAM includes ideas and methods from which Western medicine could benefit"; more than 85% agreed that "knowledge about CAM is important to me as a student/future practicing health professional"; and more than 75% felt that CAM should be included in the curriculum. Among all students, the most frequently indicated level of desired training was "sufficient to advise patients about use," for 11 of the 15 modalities. The greatest level of training was wanted for acupuncture, chiropractic, herbal medicine, and nutritional supplements. The descriptions of CAM in future clinical practice that occurred most frequently were endorsement, referral, or provision of acupuncture, biofeedback, chiropractic, herbal medicine, massage, nutritional supplements, prayer, and meditation.

CONCLUSIONS

Interest in and enthusiasm about CAM modalities was high in this sample; personal experience was much less prevalent. Students were in favor of CAM training in the curriculum to the extent that they could provide advice to patients; the largest proportions of the sample planned to endorse, refer patients for, or provide 8 of the 15 modalities surveyed in their future practice.

Modeling dimensions of choice in accepting the smallpox vaccine

OBJECTIVE

To model the dimensionality of understanding refusal to be vaccinated against smallpox.

METHOD

The Smallpox Vaccination Choice Questionnaire was administered to a convenience sample of students. Through a series of factor analyses, responses were subjected to a statistical evaluation of the underlying factor structure of potential influences on choice.

RESULTS

The analyses suggest 5 dimensions: knowledge, perceptions of risk and worry, psychological distress, general beliefs about vaccines, and age.

CONCLUSION

This model will be used to help identify targets of intervention to increase acceptance of the vaccine and satisfaction with choice.

Characterizing sleep problems in persons with Alzheimer's disease and normal elderly

We retrospectively analyzed sleep disturbance symptoms and estimated time in bed from the intake interviews of 399 healthy, non-demented elderly (NDE) and 263 persons with a diagnosis of possible (n = 53) or probable (n = 210) Alzheimer's disease (AD). Our primary objective was to identify what symptoms might underlie an individual's perception of 'sleep problems' and to determine if these were consistent within, and across, our two cohorts. We stratified each cohort according to whether or not they (or their caregiver) indicated that they had a 'sleep problem', and compared the frequency and endorsement rates of each of 21 sleep disturbance symptoms across those who did or did not endorse 'sleep problem'. For less than half of the symptoms in persons with AD, and a quarter of those in NDE, endorsement rates were significantly different depending on whether the reporter (or their sleep partner) did or did not report a sleep problem. Differences in mean frequency ratings between individuals reporting sleep problems relative to those not reporting were observed on 10 symptoms in both cohorts; six of these were the same symptom for both cohorts. When persons with subjective sleep problems in the AD and NDE cohorts were compared, only four of 21 symptoms were endorsed in one and not the other; two symptoms were significantly more frequent in one cohort than the other. Thus, within cohorts, the differences between persons with and without 'sleep problems' were relatively pronounced while the main differences in specific sleep-related symptoms between AD and NDE were not. Observed between-cohort differences appear to be driven by who is reporting, and the high prevalence of daytime sleeping in AD. Within-cohort differences reflect a clear distinction between persons with and without sleep problems, regardless of the reporter.