Estimated Prevalence of Resident-to-Resident Aggression in Assisted Living

Importance

Resident-to-resident aggression in assisted living facilities can result in physical and psychological harm, but its prevalence is unknown.

Objective

To estimate the prevalence of resident-to-resident aggression, including physical, verbal, and sexual, among residents in assisted living facilities.

Design, Setting, and Participants

This study used cross-sectional, observational data from a clinical trial, in which residents of assisted living facilities were monitored for events over a 1-month period. All residents of 14 large facilities randomly selected from 2 geographic locations (N = 1067), except those receiving hospice care (n = 11), were invited to participate; 93 died or moved prior to enrollment. There were 33 family and resident refusals; 930 residents were enrolled. Data were collected between May 30, 2018, and August 11, 2022.

Main Outcomes and Measures

The data are from a clinical trial testing the effectiveness of an intervention to reduce resident-to-resident aggression. In addition, the study was designed to assess prevalence using the Time 1 (baseline) data, using a probability sample of facilities to allow for this analysis. Resident-to-resident aggression was identified using a mixed-method, case-finding strategy involving 6 sources: (1) cognitively capable resident reports regarding 22 possible events, (2) direct care staff report, (3) staff member reports collected from event-reporting forms, (4) research assistant observation of events in real time, (5) facility accident or incident reports, and (6) resident records.

Results

The prevalence of resident-to-resident aggression among the 930 participants (mean [SD] age, 88.0 [7.2] years; 738 women [79.4%]) during the past month was estimated to be 15.2% (141 of 930 residents; 95% CI, 12.1%-18.8%). The most common forms of aggression included verbal (11.2% [104 of 930 residents; 95% CI, 8.8%-14.2%]), physical (41 of 930 residents; 4.4% [95% CI, 3.1%-6.3%]), sexual (0.8% [7 of 930 residents; 95% CI, 0.4%-1.6%]), and other (70 of 930 residents; 7.5% [95% CI, 5.5%-10.2%]). These categories are not mutually exclusive as residents could be involved with more than 1 type of aggressive behavior.

Conclusions and Relevance

In this cross-sectional, observational prevalence study, resident-to-resident aggression in assisted living facilities was highly prevalent. Verbal aggression was the most common form, and physical aggression also occurred frequently. The effects of resident-to-resident aggression can be both morbid and mortal; therefore, intervention research is needed to prevent it and to treat it when it occurs.

The use of subjective cognitive complaints for detecting mild cognitive impairment in older adults across cultural and linguistic groups: A comparison of the Cognitive Function Instrument to the Montreal Cognitive Assessment

INTRODUCTION

This pilot study aims to explore the psychometric properties of the Cognitive Function Instrument (CFI) as a measure of subjective cognitive complaints (SCC) and its performance in distinguishing mild cognitive impairment (MCI) from normal control (NC) compared to an objective cognitive screen (Montreal Cognitive Assessment [MoCA]).

METHODS

One hundred ninety-four community-dwelling non-demented older adults with racial/ethnic diversity were included. Unidimensionality and internal consistency of the CFI were examined using factor analysis, Cronbach's alpha, and McDonald's omega. Logistic regression models and receiver operating characteristic (ROC) analysis were used to examine the performance of CFI.

RESULTS

The CFI demonstrated adequate internal consistency; however, the fit for a unidimensional model was suboptimal. The CFI distinguished MCI from NC alone or in combination with MoCA. ROC analysis showed comparable performance of the CFI and the MoCA.

DISCUSSION

Our findings support the use of CFI as a brief and easy-to-use screen to detect MCI in culturally/linguistically diverse older adults.

HIGHLIGHT

What is the key scientific question or problem of central interest of the paper? Subjective cognitive complaints (SCCs) are considered the earliest sign of dementia in older adults. However, it is unclear if SCC are equivalent in different cultures. The Cognitive Function Instrument (CFI) is a 14-item measure of SCC. This study provides pilot data suggesting that CFI is sensitive for detecting mild cognitive impairment in a cohort of older adults with racial/ethnic diversity. Comparing performance, CFI demonstrates comparable sensitivity to the Montreal Cognitive Assessment, an objective cognitive screening test. Overall, SCC may provide a non-invasive, easy-to-use method to flag possible cognitive impairment in both research and clinical settings.

Improving the Delivery of Function-Directed Care During Acute Hospitalizations: Methods to Develop and Validate the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT)

OBJECTIVE

To (1) develop a patient-reported, multidomain functional assessment tool focused on medically ill patients in acute care settings; (2) characterize the measure's psychometric performance; and (3) establish clinically actionable score strata that link to easily implemented mobility preservation plans.

DESIGN

This article describes the approach that our team pursued to develop and characterize this tool, the Functional Assessment in Acute Care Multidimensional Computer Adaptive Test (FAMCAT). Development involved a multistep process that included (1) expanding and refining existing item banks to optimize their salience for hospitalized patients; (2) administering candidate items to a calibration cohort; (3) estimating multidimensional item response theory models; (4) calibrating the item banks; (5) evaluating potential multidimensional computerized adaptive testing (MCAT) enhancements; (6) parameterizing the MCAT; (7) administering it to patients in a validation cohort; and (8) estimating its predictive and psychometric characteristics.

SETTING

A large (2000-bed) Midwestern Medical Center.

PARTICIPANTS

The overall sample included 4495 adults (2341 in a calibration cohort, 2154 in a validation cohort) who were admitted either to medical services with at least 1 chronic condition or to surgical/medical services if they required readmission after a hospitalization for surgery (N=4495).

INTERVENTION

Not applicable.

MAIN OUTCOME MEASURES

Not applicable.

RESULTS

The FAMCAT is an instrument designed to permit the efficient, precise, low-burden, multidomain functional assessment of hospitalized patients. We tried to optimize the FAMCAT's efficiency and precision, as well as its ability to perform multiple assessments during a hospital stay, by applying cutting edge methods such as the adaptive measure of change (AMC), differential item functioning computerized adaptive testing, and integration of collateral test-taking information, particularly item response times. Evaluation of these candidate methods suggested that all may enhance MCAT performance, but none were integrated into initial MCAT parameterization.

CONCLUSIONS

The FAMCAT has the potential to address a longstanding need for structured, frequent, and accurate functional assessment among patients hospitalized with medical diagnoses and complications of surgery.

Challenges in Measuring Applied Cognition: Measurement Properties and Equivalence of the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT) Applied Cognition Item Bank

OBJECTIVE

To present challenges in assessment of applied cognition and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT).

DESIGN

Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and sex groups. DIF hypotheses informed the evaluation of the results.

SETTING

Hospital-based rehabilitation from a single hospital system.

PARTICIPANTS

A total of 2216 hospitalized patients (N=2216).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Applied cognition item pool from multiple sources.

RESULTS

Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (>0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude and 19 with accompanying supportive hypotheses.

CONCLUSIONS

A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one-third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and sex. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; 4 were deleted from the bank and 2 had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.

Evaluation of measurement equivalence of the Family Satisfaction with the End-of-Life Care (FAMCARE): Tests of differential item functioning between Hispanic and non-Hispanic White caregivers

OBJECTIVE

Although the psychometric properties of the Family Satisfaction with End-of-Life Care measure have been examined in diverse settings internationally; little evidence exists regarding measurement equivalence in Hispanic caregivers. The aim was to examine the psychometric properties of a short-form of the FAMCARE in Hispanics using latent variable models and place information on differential item functioning (DIF) in an existing family satisfaction item bank.

METHOD

The graded form of the item response theory model was used for the analyses of DIF; sensitivity analyses were performed using a latent variable logistic regression approach. Exploratory and confirmatory factor analyses to examine dimensionality were performed within each subgroup studied. The sample included 1,834 respondents: 317 Hispanic and 1,517 non-Hispanic White caregivers of patients with Alzheimer's disease and cancer, respectively.

RESULTS

There was strong support for essential unidimensionality for both Hispanic and non-Hispanic White subgroups. Modest DIF of low magnitude and impact was observed; flagged items related to information sharing. Only 1 item was flagged with significant DIF by both a primary and sensitivity method after correction for multiple comparisons: "The way the family is included in treatment and care decisions." This item was more discriminating for the non-Hispanic, White responders than for the Hispanic subsample, and was also a more severe indicator at some levels of the trait; the Hispanic respondents located at higher satisfaction levels were more likely than White non-Hispanic respondents to report satisfaction.

SIGNIFICANCE OF RESULTS

The magnitude of DIF was below the salience threshold for all items. Evidence supported the measurement equivalence and use for cross-cultural comparisons of the short-form FAMCARE among Hispanic caregivers, including those interviewed in Spanish.

Evaluation of the measurement properties of the Perceived Stress Scale (PSS) in Hispanic caregivers to patients with Alzheimer's disease and related disorders

OBJECTIVES

The Perceived Stress Scale (PSS) is the most widely used measure of perceived stress; however, minimal psychometric evaluation has been performed among Hispanic respondents, and even less among Hispanic caregivers to persons with Alzheimer's disease and related disorders (ADRDs).

DESIGN

Secondary data analysis.

SETTING

New York City, NY, USA.

PARTICIPANTS

A sample of 453 community dwelling Hispanic caregivers to patients with ADRD.

MEASUREMENTS

Latent variable models were used to evaluate the PSS. Exploratory and confirmatory factor analyses were used to examine unidimensionality. Differential item functioning (DIF) was examined for age, education, and language using the graded item response model.

RESULTS

The factor and bifactor analyses results supported essential unidimensionality of the item set; however, positively worded items were observed using response item theory to be less informative than the negatively worded items. Reliability estimates were high. Salient DIF was not observed for age, education, or language of interview using the primary DIF detection method. Sensitivity analyses using a second DIF detection method identified uniform language-DIF for the item, "In the last month, how often have you felt that you were on top of things?" However, the non-compensatory DIF value was below the threshold considered salient.

CONCLUSIONS

In summary, the 10-item PSS performed well in a sample of English- and Spanish-speaking Hispanic caregivers to patients with ADRD. Very little DIF, and none of high magnitude and impact, was observed. However, the negatively worded items, perhaps because they are more directly reflective of stress, were more informative. In the context of a short-form measure or computerized adaptive test, more informative items are those that would be selected for inclusion.

Psychometric Properties of a Spanish-Language Version of a Short-Form FAMCARE: Applications to Caregivers of Patients With Alzheimer's Disease and Related Dementias

Although family satisfaction is recognized as a critical indicator of quality care for persons with serious illness, Spanish-language measures are limited. The study aims were to develop a Spanish translation of the short-form Family Satisfaction With End-of-Life Care (FAMCARE), investigate its psychometric properties in Hispanic caregivers to patients with Alzheimer's disease and related dementias (ADRD; N = 317; 209 interviewed in Spanish), and add parameters to an existing item bank. Based on factor analyses, the measure was found to be essentially unidimensional. Reliabilities from a graded item response theory model were high; the average estimate was 0.93 for the total and Spanish-language subsample. Discrimination parameters were high, and the model fit adequate. This is the first study to examine the performance of the short-form FAMCARE measure among Hispanics and caregivers to patients with ADRD. The short-form measure can be recommended for Hispanics and caregivers to patients with ADRD.

Development of a Short-Form of the Medication Management Test: Evaluation of Dimensionality, Reliability, Information and Measurement Equivalence Using Latent Variable Models

BACKGROUND AND PURPOSE

The Medication Management Test (MMT) measures higher cognitive functioning. The aim of the analyses presented was to reduce assessment burden by developing a short-form version, and describe its psychometric properties.

METHODS

Factor analyses, item response theory (IRT), and differential item functioning (DIF) were performed to examine the dimensionality, reliability information, and measurement equivalence.

RESULTS

The ratio of the first two extracted eigenvalues from the exploratory principal component analysis was 7.62, indicating essential unidimensionality. Although one item "needs prompting for pill regime" evidenced DIF above the threshold for education and race/ethnicity, the magnitude was relatively small and the impact minimal. IRT-based reliability estimates were high (>0.80) across all subgroups.

CONCLUSIONS

Because medication management is an important task associated with independent living, it is critical to assess whether medications can be self-administered safely.

Assessment of financial decision making: an informant scale

Older adults with cognitive impairment are a population at great risk for financial exploitation. At-risk older adults often have difficulty reporting on their own financial abilities. Collecting information from trusted others is vital for professionals investigating the financial exploitation older adults. There are few reliable, valid, and standardized informant-report measures of financial capacity, and none that assess decisional abilities for an ongoing, real-world financial transaction. The present study sought to examine the psychometric properties of a new informant-report scale of financial decisional abilities in older adults. One hundred fifty participants were recruited to complete the Family and Friends and Interview regarding a known older adult's financial decisional abilities. A factor analysis identified two subscales. The full scale had adequate sensitivity and specificity to detect an informant's current concerns regarding financial exploitation. The Family and Friends Scale is a useful tool for collecting informant-report regarding an older adult's ability to make financial transactions.

Methodological approaches to the analyses of elder abuse screening measures: Application of latent variable measurement modeling to the WC-RAPS

While several elder abuse screens exist, few measure risk and none target long-term support services. The aims were to examine the psychometric properties of the Weinberg Center Risk and Abuse Prevention Screen (WC-RAPS), comparing approaches to modeling self-reported risk and abuse in relation to reported Adult Protective Services contacts.

METHODS

The sample (n = 7,035), admissions to managed long-term care (79%) and short-term rehabilitation (20%), was primarily (66%) female, with mean age 77.6 (SD = 9.10); 7% each were African American and Latino and 12% Asian. Latent variable models were used to examine measurement properties of six indicators of abuse and five of risk.

RESULTS

Good model fit and stable subscale measurement models were observed across analyses. Reliability was >0.80 across methods, and concurrent criterion validity estimates were as expected.

CONCLUSION

Evidence supported the reliability and concurrent criterion validity of the risk and abuse subscales in an ethnically diverse cohort.

Conceptual and Empirical Approaches to Financial Decision-making by Older Adults: Results from a Financial Decision-making Rating Scale

OBJECTIVES

The objectives of this study were threefold: (1) to empirically test the conceptual model proposed by the Lichtenberg Financial Decision-making Rating Scale (LFDRS); (2) to examine the psychometric properties of the LFDRS contextual factors in financial decision-making by investigating both the reliability and convergent validity of the subscales and total scale, and (3) extending previous work on the scale through the collection of normative data on financial decision-making.

METHODS

A convenience sample of 200 independent function and community dwelling older adults underwent cognitive and financial management testing and were interviewed using the LFDRS. Confirmatory factor analysis, internal consistency measures, and hierarchical regression were used in a sample of 200 community-dwelling older adults, all of whom were making or had recently made a significant financial decision.

RESULTS

Results confirmed the scale's reliability and supported the conceptual model. Convergent validity analyses indicate that as hypothesized, cognition is a significant predictor of risk scores. Financial management scores, however, were not predictive of decision-making risk scores.

CONCLUSIONS

The psychometric properties of the LFDRS support the scale's use as it was proposed.

CLINICAL IMPLICATIONS

The LFDRS instructions and scale are provided for clinicians to use in financial capacity assessments.

Item response theory analysis of the Lichtenberg Financial Decision Screening Scale

The focus of these analyses was to examine the psychometric properties of the Lichtenberg Financial Decision Screening Scale (LFDSS). The purpose of the screen was to evaluate the decisional abilities and vulnerability to exploitation of older adults. Adults aged 60 and over were interviewed by social, legal, financial, or health services professionals who underwent in-person training on the administration and scoring of the scale. Professionals provided a rating of the decision-making abilities of the older adult. The analytic sample included 213 individuals with an average age of 76.9 (SD = 10.1). The majority (57%) were female. Data were analyzed using item response theory (IRT) methodology. The results supported the unidimensionality of the item set. Several IRT models were tested. Ten ordinal and binary items evidenced a slightly higher reliability estimate (0.85) than other versions and better coverage in terms of the range of reliable measurement across the continuum of financial incapacity.

Methodological issues in measuring subjective well-being and quality-of-life: Applications to assessment of affect in older, chronically and cognitively impaired, ethnically diverse groups using the Feeling Tone Questionnaire

Quality of life assessment includes measurement of positive affect. Methods artifacts associated with positively and negatively worded items can manifest as negative items loading on a second factor, despite the conceptual view that the items are measuring one underlying latent construct. Negatively worded items may elicit biased responses. Additionally, item-level response bias across ethnically diverse groups may compromise group comparisons. The aim was to illustrate methodological approaches to examining method factors and measurement equivalence in an affect measure with 9 positively and 7 negatively worded items: The Feeling Tone Questionnaire (FTQ). The sample included 4,960 non-Hispanic White, 1,144 non-Hispanic Black, and 517 Hispanic community and institutional residents receiving long-term supportive services. The mean age was 82 (s.d.=11.0); 73% were female. Two thirds were cognitively impaired. Methods effects were assessed using confirmatory factor analyses (CFA), and reliability with McDonald's omega and item response theory (IRT) generated estimates. Measurement equivalence was examined using IRT-based Wald tests. Methods effects associated with negatively worded items were observed; these provided little IRT information, and as a composite evidenced lower reliability. Both 13 and 9 item positive affect scales performed well in terms of model fit, reliability, IRT information, and evidenced little differential item functioning of high magnitude or impact. Both CFA and IRT approaches provided complementary methodological information about scale performance. The 9-item affect scale based on the FTQ can be recommended as a brief quality-of-life measure among frail and cognitively impaired individuals in palliative and long-term care settings.

Reliability and Validity of the Lichtenberg Financial Decision Screening Scale

The scarcity of empirically validated assessment instruments continues to impede the work of professionals in a number of fields, including medicine, finance, and estate planning; adult protective services; and criminal justice—and, more importantly, it impedes their ability to effectively assist and, in some case, protect their clients. Other professionals (e.g., legal, financial, medical, mental health services) are in a position to prevent financial exploitation and would benefit from access to new instruments. The Lichtenberg Financial Decision Screening Scale (LFDSS) was introduced in 2016, along with evidence for its convergent validity (Lichtenberg, P. A., Fickern, L., Rahman-Filipiak, A., Tatro, R., Farrell, C., Speir, J. J., … Jackman, J. D. (2016b). The Lichtenberg Financial Decision Screening Scale: A new tool for assessing financial decision making and preventing financial exploitation (2016). Journal of Elder Abuse and Neglect, 28, 134–151. doi:10.1080/08946566.2016.1168333). Using a sample of 213 participants, this study investigated the internal consistency of the LFDSS and its criterion validity based on ratings by professionals using the scale. Results demonstrate that the LFDSS has excellent internal consistency and clinical utility properties. This paper provides support for use of the LFDSS as a reliable and valid instrument. The LFDSS and instructions for its use are included in the article, along with information about online tools and support.

Applications of Item Response Theory to the Examination of the Psychometric Properties and Differential Item Functioning of the Comprehensive Assessment and Referral Evaluation Dementia Diagnostic Scale among Samples of Latino, African American, and White Non-Latino Elderly

Item response theory was used to examine the psychometric properties of a cognitive screening measure used in several epidemiological surveys among Latino, African American, and White non-Latino elderly. Estimates of precision (reliability) examined across several values of Θ (the estimate of degree of cognitive impairment) were good (.70s to .90s) in the range representing most respondents. Overall reliability, although adequate, was lower among the White non-Latino and high-education subgroups relative to other racial/ethnic and education subgroups. Differential item functioning (DIF) was examined using several area- and model-based tests. Tests of the magnitude of DIF showed the measure to be relatively free of DIF for the racial/ethnic subgroup and education subgroups examined. However, one item related to remembering the telephone number was more difficult for Latinos than for other racial/ethnic subgroups. Several other items evidenced mild DIF, and one (difficulty remembering words or names) was a poorly discriminating item. The subjective memory items, intended to provide additional information at the earlier, “borderzone” stages of cognitive impairment, did provide more information at the mild to moderate levels of impairment, although maximum information was not provided at these borderzone ranges.

Measurement Equivalence of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Applied Cognition - General Concerns, Short Forms in Ethnically Diverse Groups

AIMS

The goals of these analyses were to examine the psychometric properties and measurement equivalence of a self-reported cognition measure, the Patient Reported Outcome Measurement Information System^® (PROMIS^®) Applied Cognition - General Concerns short form. These items are also found in the PROMIS Cognitive Function (version 2) item bank. This scale consists of eight items related to subjective cognitive concerns. Differential item functioning (DIF) analyses of gender, education, race, age, and (Spanish) language were performed using an ethnically diverse sample (n = 5,477) of individuals with cancer. This is the first analysis examining DIF in this item set across ethnic and racial groups.

METHODS

DIF hypotheses were derived by asking content experts to indicate whether they posited DIF for each item and to specify the direction. The principal DIF analytic model was item response theory (IRT) using the graded response model for polytomous data, with accompanying Wald tests and measures of magnitude. Sensitivity analyses were conducted using ordinal logistic regression (OLR) with a latent conditioning variable. IRT-based reliability, precision and information indices were estimated.

RESULTS

DIF was identified consistently only for the item, brain not working as well as usual. After correction for multiple comparisons, this item showed significant DIF for both the primary and sensitivity analyses. Black respondents and Hispanics in comparison to White non-Hispanic respondents evidenced a lower conditional probability of endorsing the item, brain not working as well as usual. The same pattern was observed for the education grouping variable: as compared to those with a graduate degree, conditioning on overall level of subjective cognitive concerns, those with less than high school education also had a lower probability of endorsing this item. DIF was also observed for age for two items after correction for multiple comparisons for both the IRT and OLR-based models: "I have had to work really hard to pay attention or I would make a mistake" and "I have had trouble shifting back and forth between different activities that require thinking". For both items, conditional on cognitive complaints, older respondents had a higher likelihood than younger respondents of endorsing the item in the cognitive complaints direction. The magnitude and impact of DIF was minimal. The scale showed high precision along much of the subjective cognitive concerns continuum; the overall IRT-based reliability estimate for the total sample was 0.88 and the estimates for subgroups ranged from 0.87 to 0.92.

CONCLUSION

Little DIF of high magnitude or impact was observed in the PROMIS Applied Cognition - General Concerns short form item set. One item, "It has seemed like my brain was not working as well as usual" might be singled out for further study. However, in general the short form item set was highly reliable, informative, and invariant across differing race/ethnic, educational, age, gender, and language groups.

Psychometric Properties and Performance of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Depression Short Forms in Ethnically Diverse Groups

Short form measures from the Patient Reported Outcomes Measurement Information System^® (PROMIS^®) are used widely. The present study was among the first to examine differential item functioning (DIF) in the PROMIS Depression short form scales in a sample of over 5000 racially/ethnically diverse patients with cancer. DIF analyses were conducted across different racial/ethnic, educational, age, gender and language groups.

METHODS

DIF hypotheses, generated by content experts, informed the evaluation of the DIF analyses. The graded item response theory (IRT) model was used to evaluate the five-level ordinal items. The primary tests of DIF were Wald tests; sensitivity analyses were conducted using the IRT ordinal logistic regression procedure. Magnitude was evaluated using expected item score functions, and the non-compensatory differential item functioning (NCDIF) and T1 indexes, both based on group differences in the item curves. Aggregate impact was evaluated with expected scale score (test) response functions; individual impact was assessed through examination of differences in DIF adjusted and unadjusted depression estimates.

RESULTS

Many items evidenced DIF; however, only a few had slightly elevated magnitude. No items evidenced salient DIF with respect to NCDIF and the scale-level impact was minimal for all group comparisons. The following short form items might be targeted for further study because they were also hypothesized to evidence DIF. One item showed slightly higher magnitude of DIF for age: nothing to look forward to; conditional on depression, this item was more likely to be endorsed in the depressed direction by individuals in older groups as contrasted with the cohort aged 21 to 49. This item was also hypothesized to show age DIF. Only one item (failure) showed DIF of slightly higher magnitude (just above threshold) for Whites vs. Asians/Pacific Islanders in the direction of higher likelihood of endorsement for Asians/Pacific Islanders. This item was also hypothesized to show DIF for minority groups. The impact of DIF was negligible. Conditional on depression, the items, worthless and hopeless were more likely to be endorsed in the depressed direction by respondents with less than high school education vs. those with a graduate degree; the magnitude of DIF was slightly above the T1 threshold, but not that of NCDIF. These items were also hypothesized to show DIF in the direction of more feelings of worthlessness by groups with lower education. While the magnitude and aggregate impact of DIF was small, in a few instances, individual impact was observed. Information provided was relatively high, particularly in the middle upper (depressed) tail of the distribution. Reliability estimates were high (> 0.90) across all studied groups, regardless of estimation method.

CONCLUSIONS

This was the first study to evaluate measurement equivalence of the PROMIS Depression short forms across large samples of ethnically diverse groups. There were few items with DIF, and none of high magnitude, thus supporting the use of PROMIS Depression short form measures across such groups. These results could be informative for those using the short forms in minority populations or clinicians evaluating individuals with the depression short forms.

Measurement Equivalence of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Pain Interference Short Form Items: Application to Ethnically Diverse Cancer and Palliative Care Populations

Reducing the response burden of standardized pain measures is desirable, particularly for individuals who are frail or live with chronic illness, e.g., those suffering from cancer and those in palliative care. The Patient Reported Outcome Measurement Information System^® (PROMIS^®) project addressed this issue with the provision of computerized adaptive tests (CAT) and short form measures that can be used clinically and in research. Although there has been substantial evaluation of PROMIS item banks, little is known about the performance of PROMIS short forms, particularly in ethnically diverse groups. Reviewed in this article are findings related to the differential item functioning (DIF) and reliability of the PROMIS pain interference short forms across diverse sociodemographic groups.

METHODS

DIF hypotheses were generated for the PROMIS short form pain interference items. Initial analyses tested item response theory (IRT) model assumptions of unidimensionality and local independence. Dimensionality was evaluated using factor analytic methods; local dependence (LD) was tested using IRT-based LD indices. Wald tests were used to examine group differences in IRT parameters, and to test DIF hypotheses. A second DIF-detection method used in sensitivity analyses was based on ordinal logistic regression with a latent IRT-derived conditioning variable. Magnitude and impact of DIF were investigated, and reliability and item and scale information statistics were estimated.

RESULTS

The reliability of the short form item set was excellent. However, there were a few items with high local dependency, which affected the estimation of the final discrimination parameters. As a result, the item, "How much did pain interfere with enjoyment of social activities?" was excluded in the DIF analyses for all subgroup comparisons. No items were hypothesized to show DIF for race and ethnicity; however, five items showed DIF after adjustment for multiple comparisons in both primary and sensitivity analyses: ability to concentrate, enjoyment of recreational activities, tasks away from home, participation in social activities, and socializing with others. The magnitude of DIF was small and the impact negligible. Three items were consistently identified with DIF for education: enjoyment of life, ability to concentrate, and enjoyment of recreational activities. No item showed DIF above the magnitude threshold and the impact of DIF on the overall measure was minimal. No item showed gender DIF after correction for multiple comparisons in the primary analyses. Four items showed consistent age DIF: enjoyment of life, ability to concentrate, day to day activities, and enjoyment of recreational activities, none with primary magnitude values above threshold. Conditional on the pain state, Spanish speakers were hypothesized to report less pain interference on one item, enjoyment of life. The DIF findings confirmed the hypothesis; however, the magnitude was small. Using an arbitrary cutoff point of theta (θ) ≥ 1.0 to classify respondents with acute pain interference, the highest number of changes were for the education groups analyses. There were 231 respondents (4% of the total sample) who changed from the designation of no acute pain interference to acute interference after the DIF adjustment. There was no change in the designations for race/ethnic subgroups, and a small number of changes for respondents aged 65 to 84.

CONCLUSIONS

Although significant DIF was observed after correction for multiple comparisons, all DIF was of low magnitude and impact. However, some individual-level impact was observed for low education groups. Reliability estimates were high. Thus, the PROMIS short form pain items examined in this ethnically diverse sample performed relatively well; although one item was problematic and removed from the analyses. It is concluded that the majority of the PROMIS pain interference short form items can be recommended for use among ethnically diverse groups, including those in palliative care and with cancer and chronic illness.

Use of an Item Bank to Develop Two Short-Form FAMCARE Scales to Measure Family Satisfaction With Care in the Setting of Serious Illness

CONTEXT

Family satisfaction is an important and commonly used research measure. Yet current measures of family satisfaction are lengthy and may be unnecessarily burdensome--particularly in the setting of serious illness.

OBJECTIVES

To use an item bank to develop short forms of the Family Satisfaction with End-of-Life Care (FAMCARE) scale, which measures family satisfaction with care.

METHODS

To shorten the existing 20-item FAMCARE measure, item response theory parameters from an item bank were used to select the most informative items. The psychometric properties of the new short-form scales were examined. The item bank was based on data from family members from an ethnically diverse sample of 1983 patients with advanced cancer.

RESULTS

Evidence for the new short-form scales supported essential unidimensionality. Reliability estimates from several methods were relatively high, ranging from 0.84 for the five-item scale to 0.94 for the 10-item scale across different age, gender, education, ethnic, and relationship groups.

CONCLUSION

The FAMCARE-10 and FAMCARE-5 short-form scales evidenced high reliability across sociodemographic subgroups and are potentially less burdensome and time-consuming scales for monitoring family satisfaction among seriously ill patients.

Evaluation of measurement equivalence of the Family Satisfaction with the End-of-Life Care in an ethnically diverse cohort: tests of differential item functioning

BACKGROUND

The Family Satisfaction with End-of-Life Care is an internationally used measure of satisfaction with cancer care. However, the Family Satisfaction with End-of-Life Care has not been studied for equivalence of item endorsement across different socio-demographic groups using differential item functioning.

AIMS

The aims of this secondary data analysis were (1) to examine potential differential item functioning in the family satisfaction item set with respect to type of caregiver, race, and patient age, gender, and education and (2) to provide parameters and documentation of differential item functioning for an item bank.

DESIGN

A mixed qualitative and quantitative analysis was conducted. A priori hypotheses regarding potential group differences in item response were established. Item response theory and Wald tests were used for the analyses of differential item functioning, accompanied by magnitude and impact measures.

RESULTS

Very little significant differential item functioning was observed for patient's age and gender. For race, 13 items showed differential item functioning after multiple comparison adjustment, 10 with non-uniform differential item functioning. No items evidenced differential item functioning of high magnitude, and the impact was negligible. For education, 5 items evidenced uniform differential item functioning after adjustment, none of high magnitude. Differential item functioning impact was trivial. One item evidenced differential item functioning for the caregiver relationship variable.

CONCLUSION

Differential item functioning was observed primarily for race and education. No differential item functioning of high magnitude was observed for any item, and the overall impact of differential item functioning was negligible. One item, satisfaction with "the patient's pain relief," might be singled out for further study, given that this item was both hypothesized and observed to show differential item functioning for race and education.

Development of an instrument to measure staff-reported resident-to-resident elder mistreatment (R-REM) using item response theory and other latent variable models

OBJECTIVES

Although numerous studies have measured behaviors among individuals in congregate settings, few have focused on resident-to-resident elder mistreatment (R-REM). To our knowledge, there is no psychometrically developed measure of R-REM extant. The quantitative development of a measure of staff-reported R-REM is described.

METHODS

The design was a prevalent cohort study of residents of 5 long-term care facilities. The primary certified nursing assistant was interviewed about R-REM. Advanced measurement methods were used to develop a measure of R-REM.

RESULTS

The loadings on the general factor for the final 11-item scale were greater than those on the group factor except for the item "other physical behavior" (0.63 vs. 0.74), suggesting essential unidimensionality. Although the bifactor model fit was slightly better than that of the unidimensional model, the difference was trivial (bifactor comparative fit index [CFI] = 0.997, root mean square error of approximation [RMSEA] = 0.013, unidimensional CFI = 0.979, and RMSEA = 0.030). However, modest support was provided for use of verbal and physical subscales. The explained common variance statistics were 0.76 for the bifactor model compared with 0.63 for the unidimensional model.

DISCUSSION

The development of this R-REM measure will help to advance the measurement and ultimately evaluation of interventions associated with this important and under recognized problem facing residents in long-term care settings.

A comparison of item response theory-based methods for examining differential item functioning in object naming test by language of assessment among older Latinos

Object naming tests are commonly included in neuropsychological test batteries. Differential item functioning (DIF) in these tests due to cultural and language differences may compromise the validity of cognitive measures in diverse populations. We evaluated 26 object naming items for DIF due to Spanish and English language translations among Latinos (n=1,159), mean age of 70.5 years old (Standard Deviation (SD)±7.2), using the following four item response theory-based approaches: Mplus/Multiple Indicator, Multiple Causes (Mplus/MIMIC; Muthén & Muthén, 1998-2011), Item Response Theory Likelihood Ratio Differential Item Functioning (IRTLRDIF/MULTILOG; Thissen, 1991, 2001), difwithpar/Parscale (Crane, Gibbons, Jolley, & van Belle, 2006; Muraki & Bock, 2003), and Differential Functioning of Items and Tests/MULTILOG (DFIT/MULTILOG; Flowers, Oshima, & Raju, 1999; Thissen, 1991). Overall, there was moderate to near perfect agreement across methods. Fourteen items were found to exhibit DIF and 5 items observed consistently across all methods, which were more likely to be answered correctly by individuals tested in Spanish after controlling for overall ability.

Analysis of differential item functioning in the depression item bank from the Patient Reported Outcome Measurement Information System (PROMIS): An item response theory approach

The aims of this paper are to present findings related to differential item functioning (DIF) in the Patient Reported Outcome Measurement Information System (PROMIS) depression item bank, and to discuss potential threats to the validity of results from studies of DIF. The 32 depression items studied were modified from several widely used instruments. DIF analyses of gender, age and education were performed using a sample of 735 individuals recruited by a survey polling firm. DIF hypotheses were generated by asking content experts to indicate whether or not they expected DIF to be present, and the direction of the DIF with respect to the studied comparison groups. Primary analyses were conducted using the graded item response model (for polytomous, ordered response category data) with likelihood ratio tests of DIF, accompanied by magnitude measures. Sensitivity analyses were performed using other item response models and approaches to DIF detection. Despite some caveats, the items that are recommended for exclusion or for separate calibration were "I felt like crying" and "I had trouble enjoying things that I used to enjoy." The item, "I felt I had no energy," was also flagged as evidencing DIF, and recommended for additional review. On the one hand, false DIF detection (Type 1 error) was controlled to the extent possible by ensuring model fit and purification. On the other hand, power for DIF detection might have been compromised by several factors, including sparse data and small sample sizes. Nonetheless, practical and not just statistical significance should be considered. In this case the overall magnitude and impact of DIF was small for the groups studied, although impact was relatively large for some individuals.

A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression

BACKGROUND

Several techniques have been developed to detect differential item functioning (DIF), including ordinal logistic regression (OLR). This study compared different criteria for determining whether items have DIF using OLR.

OBJECTIVES

To compare and contrast findings from three different sets of criteria for detecting DIF using OLR. General distress and physical functioning items were evaluated for DIF related to five covariates: age, marital status, gender, race, and Hispanic origin.

RESEARCH DESIGN

Cross-sectional study.

SUBJECTS

1,714 patients with cancer or HIV/AIDS.

MEASURES

A total of 23 items addressing physical functioning and 15 items addressing general distress were selected from a pool of 154 items from four different health-related quality of life questionnaires.

RESULTS

The three sets of criteria produced qualitatively and quantitatively different results. Criteria based on statistical significance alone detected DIF in almost all the items, while alternative criteria based on magnitude detected DIF in far fewer items. Accounting for DIF by using demographic-group specific item parameters had negligible effects on individual scores, except for race.

CONCLUSIONS

Specific criteria chosen to determine whether items have DIF have an impact on the findings. Criteria based entirely on statistical significance may detect small differences that are clinically negligible.

Evaluating measurement equivalence using the item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): applications (with illustrations) to measures of physical functioning ability and general distress

BACKGROUND

Methods based on item response theory (IRT) that can be used to examine differential item functioning (DIF) are illustrated. An IRT-based approach to the detection of DIF was applied to physical function and general distress item sets. DIF was examined with respect to gender, age and race. The method used for DIF detection was the item response theory log-likelihood ratio (IRTLR) approach. DIF magnitude was measured using the differences in the expected item scores, expressed as the unsigned probability differences, and calculated using the non-compensatory DIF index (NCDIF). Finally, impact was assessed using expected scale scores, expressed as group differences in the total test (measure) response functions.

METHODS

The example for the illustration of the methods came from a study of 1,714 patients with cancer or HIV/AIDS. The measure contained 23 items measuring physical functioning ability and 15 items addressing general distress, scored in the positive direction.

RESULTS

The substantive findings were of relatively small magnitude DIF. In total, six items showed relatively larger magnitude (expected item score differences greater than the cutoff) of DIF with respect to physical function across the three comparisons: "trouble with a long walk" (race), "vigorous activities" (race, age), "bending, kneeling stooping" (age), "lifting or carrying groceries" (race), "limited in hobbies, leisure" (age), "lack of energy" (race). None of the general distress items evidenced high magnitude DIF; although "worrying about dying" showed some DIF with respect to both age and race, after adjustment.

CONCLUSIONS

The fact that many physical function items showed DIF with respect to age, even after adjustment for multiple comparisons, indicates that the instrument may be performing differently for these groups. While the magnitude and impact of DIF at the item and scale level was minimal, caution should be exercised in the use of subsets of these items, as might occur with selection for clinical decisions or computerized adaptive testing. The issues of selection of anchor items, and of criteria for DIF detection, including the integration of significance and magnitude measures remain as issues requiring investigation. Further research is needed regarding the criteria and guidelines appropriate for DIF detection in the context of health-related items.

IRT health outcomes data analysis project: an overview and summary

BACKGROUND

In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, "Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment." A component of the conference was presentation of a psychometric and content analysis of a secondary dataset.

OBJECTIVES

A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset.

RESEARCH DESIGN

HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared.

SUBJECTS

The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites.

MEASURES

Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System-Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey.

RESULTS AND CONCLUSIONS

Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed.

Identification of differential item functioning using item response theory and the likelihood-based model comparison approach. Application to the Mini-Mental State Examination

BACKGROUND

An important part of examining the adequacy of measures for use in ethnically diverse populations is the evaluation of differential item functioning (DIF) among subpopulations such as those administered the measure in different languages. A number of methods exist for this purpose.

OBJECTIVE

The objective of this study was to introduce and demonstrate the identification of DIF using item response theory (IRT) and the likelihood-based model comparison approach.

METHODS

Data come from a sample of community-residing elderly who were part of a dementia case registry. A total of 1578 participants were administered either an English (n = 913) or Spanish (n = 665) version of the 21-item Mini-Mental State Examination. IRT was used to identify language DIF in these items with the likelihood-based model comparison approach.

RESULTS

: Fourteen of the 21 items exhibited significant DIF according to language of administration. However, because the direction of the identified DIF was not consistent for one language version over the other, the impact at the scale level was negligible.

CONCLUSIONS

IRT and the likelihood-based model comparison approach comprise a powerful tool for DIF detection that can aid in the development, refinement, and evaluation of measures for use in ethnically diverse populations.

A program to reduce the disparity in the rate of biennial lipid profiles between African-American and white Medicare beneficiaries with diabetes mellitus in New York City

The burden of diabetes is more severe for minority populations than for the white population in the United States. Analysis of Medicare fee-for-service claims data revealed a significant disparity regarding screening for lipids between white and African-American Medicare beneficiaries with diabetes as measured by the biennial lipid profile indicator in New York State and the disparity was even greater in New York City. Thus IPRO, the Medicare New York State Quality Improvement Organization, launched a multifaceted quality improvement project to reduce the disparity, targeting African-American Medicare beneficiaries with diabetes in New York City and the providers who serve them. There was an absolute increase of 16.7% in the proportion of African-American beneficiaries with diabetes receiving a biennial lipid profile in the intervention areas and the disparity reduced by 9.8% between African-American and all eligible white beneficiaries in the intervention areas. Although it was not feasible to determine the direct impact of selected interventions on reducing the disparity, the interventions collectively appeared to be effective.

Effect of inpatient quality of care on functional outcomes in patients with hip fracture

OBJECTIVES

We sought to examine the relationship between functional outcome and process of care for patients with hip fracture.

RESEARCH DESIGN AND PARTICIPANTS

We undertook a prospective cohort study in 4 hospitals of 554 patients treated with surgery for hip fracture.

MEASUREMENTS

Information on patient characteristics and processes of hospital care collected from the medical record, interviews, and bedside observations. Follow-up information obtained at 6 months on function (using the Functional Independence Measure [FIM]), survival, and readmission.

RESULTS

Individual processes of care were generally not associated with adjusted outcomes. A scale of 9 processes related to mobilization was associated with improved adjusted locomotion (P = 0.006), self care (P = 0.022), and transferring (P = 0.007) at 2 months, but the benefits were smaller and not significant by 6 months. These processes were not associated with mortality. The predicted value for the FIM locomotion measure (range, 2-14) at 2 months was 5.9 (95% confidence interval 5.4-6.4) for patients at the 10th percentile of performance on these processes compared with 7.1 (95% confidence interval 6.6, 7.6) at the 90th percentile. Patients who experienced no hospital complications and no readmissions retained the benefits in locomotion at 6 months. Anticoagulation processes were associated with improved transferring at 2 months (P = 0.046) but anticoagulation and other processes of care were not otherwise associated with improved function.

DISCUSSION

Our findings indicate the need to attend to all steps in the care of patients with hip fracture. Additionally, functional outcomes were more sensitive markers of improved process of care, compared with 6-month mortality, in the case of hip fracture.

The development and psychometric analyses of ADEPT: an instrument for assessing the interactions between doctors and their elderly patients

BACKGROUND

The lack of instruments and methodologies designed specifically for assessing doctor-elderly patient interactions has constricted research on effective communication in the medical care of older adults.

PURPOSE

This article reports on the development, qualitative analyses, and psychometric testing of the Assessment of Doctor-Elderly Patient Transactions (ADEPT), an instrument for assessing interactions between doctors and their elderly patients.

METHODS

The ADEPT was based on the recommendations of an expert panel and designed around the three-function model of the medical interview. The ADEPT is meant to operationalize the research findings of interactional analysis studies of doctor-patient interaction. Following preliminary testing with standardized patients, the ADEPT was applied to videotaped visits of 433 patients 65 years of age and older to the doctor (n = 40) identified as their primary source of care.

RESULTS

Four final scales derived from exploratory and confirmatory factor analyses were scored: Supporting, comprised of the 12 items from the first factor; Eliciting Needs, containing the 5 items from the second factor; and Informing, based on the final 6-item factor. Individual Cronbach's alphas across raters for this sample ranged from .71 to .79 for the first scale, from .83 to .88 for the second scale, and from .64 to .81 for the third scale. The reliability estimates for the total scale (23 items) ranged from .80 to .86 across raters. A fifth summed index composed of 46 binary checklist items also was computed.

CONCLUSIONS

The findings indicate that credible scales can be developed for assessing communication behaviors.

The relationship between organizational systems and clinical quality in diabetes care

OBJECTIVE

To assess the clinical quality of diabetes care and the systems of care in place in Medicare managed care organizations (MCOs) to determine which systems are associated with the quality of care.

STUDY DESIGN

Cross-sectional, observational study that included a retrospective review of 2001 diabetes Health Plan Employer and Data Information Set (HEDIS) measures and a mailed survey to MCOs.

METHODS

One hundred and thirty-four plans received systems surveys. Data on clinical quality were obtained from HEDIS reports of diabetes measures.

RESULTS

Ninety plans returned the survey. Composite diabetes quality scores (CDSs) were based on averaging scores for the 6 HEDIS diabetes measures. For the upper quartile of responding plans, the average score was 77.6. The average score for the bottom quartile was 53.9 (P < .001). The mean number of systems or interventions for the upper-quartile group and the bottom-quartile group was 17.5 and 12.5 (P < .01), respectively. There were significant differences in the 2 groups in the following areas: computer-generated reminders, physician champions, practitioner quality-improvement work groups, clinical guidelines, academic detailing, self-management education, availability of laboratory results, and registry use. After adjusting for structural and geographic variables, practitioner input and use of clinical-guidelines software remained as independent predictors of CDS. Structural variables that were independent predictors were nonprofit status and increasing number of Medicare beneficiaries in the MCO.

CONCLUSIONS

MCO structure and greater use of systems/interventions are associated with higher-quality diabetes care. These relationships require further exploration.

Modern psychometric methods for detection of differential item functioning: application to cognitive assessment measures

Cognitive screening tests and items have been found to perform differently across groups that differ in terms of education, ethnicity and race. Despite the profound implications that such bias holds for studies in the epidemiology of dementia, little research has been conducted in this area. Using the methods of modern psychometric theory (in addition to those of classical test theory), we examined the performance of the Attention subscale of the Mattis Dementia Rating Scale. Several item response theory models, including the two- and three-parameter dichotomous response logistic model, as well as a polytomous response model were compared. (Log-likelihood ratio tests showed that the three-parameter model was not an improvement over the two-parameter model.) Data were collected as part of the ten-study National Institute on Aging Collaborative investigation of special dementia care in institutional settings. The subscale KR-20 estimate for this sample was 0.92. IRT model-based reliability estimates, provided at several points along the latent attribute, ranged from 0.65 to 0.97; the measure was least precise at the less disabled tail of the distribution. Most items performed in similar fashion across education groups; the item characteristic curves were almost identical, indicating little or no differential item functioning (DIF). However, four items were problematic. One item (digit span backwards) demonstrated a large error term in the confirmatory factor analysis; item-fit chi-square statistics developed using BIMAIN confirm this result for the IRT models. Further, the discrimination parameter for that item was low for all education subgroups. Generally, persons with the highest education had a greater probability of passing the item for most levels of theta. Model-based tests of DIF using MULTILOG identified three other items with significant, albeit small, DIF. One item, for example, showed non-uniform DIF in that at the impaired tail of the latent distribution, persons with higher education had a higher probability of correctly responding to the item than did lower education groups, but at less impaired levels, they had a lower probability of a correct response than did lower education groups. Another method of detection identified this item as having DIF (unsigned area statistic=3.05, p<0.01, and 2.96, p<0.01). On average, across the entire score range, the lower education group's probability of answering the item correctly was 0.11 higher than the higher education group's probability. A cross-validation with larger subgroups confirmed the overall result of little DIF for this measure. The methods used for detecting differential item functioning (which may, in turn, be indicative of bias) were applied to a neuropsychological subtest. These methods have been used previously to examine bias in screening measures across education and ethnic and racial subgroups. In addition to the important epidemiological applications of ensuring that screening measures and neuropsychological tests used in diagnoses are free of bias so that more culture-fair classifications will result, these methods are also useful for the examination of site differences in large multi-site clinical trials. It is recommended that these methods receive wider attention in the medical statistical literature.

Placebo run-in period in studies of depressive disorders. Clinical, heuristic and research implications

BACKGROUND

In spite of the virtually ubiquitous nature of the initial 10-day placebo run-in period (IPR) in drug trials, there is little empirical data establishing its relevance.

METHOD

Data from 593 subjects were examined retrospectively to determine whether or not the prognosis of subjects minimally improved during the IPR was different to those who were unimproved. The IPR period was single-blind and was followed by a six-week double-blind phase in all studies.

RESULTS

Twenty-six per cent of the subjects were minimally improved and 74% were unimproved. Approximately 10% of the subjects who were much improved were not followed systematically. Across a range of diagnosis, severity and chronicity subjects minimally improved (versus unimproved) after IPR had a more favourable prognosis whether assigned to drug or placebo.

CONCLUSIONS

Change during IPR appears to be a meaningful predictor. Stratification should be considered in future antidepressant studies.

Imipramine treatment of opiate-dependent patients with depressive disorders. A placebo-controlled trial

BACKGROUND

The literature is inconclusive on the role of antidepressant medications in treating drug dependence. Studies have either not focused on depressed patients or have selected patients with depressive disorders based on cross-sectional symptoms rather than a syndromal diagnosis. A clinical trial of an antidepressant was, therefore, conducted on drug-dependent patients with syndromal depression.

METHODS

Patients receiving methadone hydrochloride maintenance treatment were selected if they met the criteria for a DSM-III-R depressive disorder that was chronologically primary, had persisted during a past abstinent period or was long-standing, and persisted during at least 1 month of stable methadone treatment. Subjects were randomized to a 12-week, double-blind, placebo-controlled trial of imipramine hydrochloride. A favorable response was defined as a Clinical Global Impression scale score for depression of 2 ("much improved") or 1 ("very much improved") and at least a 75% reduction in self-reported drug or alcohol use or abstinence.

RESULTS

One hundred thirty-seven patients were randomized, and 84 completed a minimum adequate trial of at least 6 weeks. Among the 84 adequately treated patients, 57% (24/42) receiving imipramine were rated as responders compared with 7% (3/42) receiving placebo (P < .001). On measures of mood, there was a robust effect of imipramine. Imipramine was superior to placebo on some self-reported measures of substance use and craving, and mood improvement was associated with improvement in self-reported substance use. However, few patients achieved urine-confirmed abstinence.

CONCLUSIONS

Imipramine was an effective antidepressant in patients receiving methadone who were selected via syndromal criteria for depressive illness. Imipramine may reduce substance abuse among patients whose mood improves; however, this effect was less robust.

The efficacy of imipramine and psychotherapy in early-onset chronic depression: a reanalysis of the National Institute of Mental health Treatment of Depression Collaborative Research Program

The authors compared the effectiveness of Cognitive Behavioral Therapy (CBT), Interpersonal Psychotherapy (IPT), Imipramine Clinical Management (ICM) to Placebo Clinical Management (PCM) for outpatients with early-onset chronic depression (N = 65) in the National Institute of Mental Health (NIMH) Treatment of Depression Collaborative Research Program (TDRP). The post-treatment depression scores of the CBT. IPT, and ICM groups were not significantly different from the PCM group. We did not find a relationship between the duration of Major Depression and response to a specific treatment. Studies are needed to determine if combining psychotherapy with medication improves social functioning and enhances the quality of life for patients with chronic depression.

Chronological milestones to guide drug change. When should clinicians switch antidepressants?

BACKGROUND

We attempt to identify the time when patients whose conditions are unimproved while receiving antidepressants are unlikely to respond and should have their treatment changed.

METHODS

A total of 593 patients were studied. The course of treatment for patients was examined to determine the weeks at which patients who received drug therapy had a better chance of being rated as responders at the study end (week 6) vs patients who received placebo.

RESULTS

At the end of week 3, 19 (32%) of the 59 patients who received drug therapy and 6 (10%) of the 57 patients who received placebo and who never minimally improved were rated as responders at week 6. For those who showed no improvement by week 4, the effects of drug therapy and the placebo were equal. Patients who received drug therapy and whose conditions were unimproved but who had been minimally improved at some point had a superior prognosis with drug therapy vs placebo until week 4. Of those unimproved at week 4 but minimally improved at some point previously, 20 (39%) of the 51 patients who received drug therapy vs 3 (8%) of the 36 patients who received placebo were rated as responders at week 6. Of the 75 patients who minimally improved while receiving drug therapy at the end of week 5, 33 (44%) had a chance of being rated a responder at the end of week 6 vs 9 (26%) of the 35 patients receiving placebo.

CONCLUSIONS

Patients tolerant of an adequate dose, whose conditions have never been at least minimally improved by the end of week 4, should have their treatment regimen altered. These patients represented a minority of drug-treated patients in the sample studied (ie, 39/392 [10%]). Patients whose conditions minimally improve at some prior week but not after week 5 should have their treatment changed. Patients whose conditions minimally improve in week 5 should continue treatment until week 6.

Imipramine treatment of alcoholics with primary depression: A placebo-controlled clinical trial

BACKGROUND

Depressive disorders are commonly comorbid with alcoholism, particularly in treatment-seeking samples. If antidepressant treatment were safe and improved the treatment outcome in the subset of actively drinking alcoholics with depression, this would be of clinical importance.

METHODS

We conducted a randomized, 12-week placebo-controlled trial of imipramine hydrochloride combined with weekly relapse prevention psychotherapy. The subjects were 69 actively drinking alcoholic outpatients with current depressive disorders. The first onset of depression was either antecedent to the abuse of alcohol or occurred during prolonged periods of sobriety. Depression and drinking outcomes at 12 weeks, as well as their relationship, were measured.

RESULTS

Imipramine treatment was safe and associated with improvement in depression in both adequately treated and intention-to-treat samples. While there was no overall effect on drinking outcome, patients whose mood improved showed decreased alcohol consumption that was more marked in those treated with imipramine.

CONCLUSIONS

Imipramine treatment is effective for primary depression among actively drinking alcoholic outpatients, and may improve alcoholic outcome for those whose depression responds to treatment.

Imipramine treatment of cocaine abuse: possible boundaries of efficacy

A 12-week placebo-controlled, randomized clinical trial was undertaken to evaluate imipramine as a treatment for cocaine abuse, and to examine whether its effect may be limited to subgroups defined by route of use or by diagnosis of depression. One-hundred thirteen patients were randomized, stratified by route of use and depression. All patients received weekly individual counseling. Compared to placebo the imipramine group showed greater reductions in cocaine craving, cocaine euphoria, and depression, but the effect of imipramine on cocaine use was less clear. A favorable response, defined as at least 3 consecutive, urine-confirmed, cocaine-free weeks was achieved by 19% (11/59) of patients on imipramine compared to 7% (4/54) on placebo (P < 0.09). The imipramine effect was greater among nasal users--33% (9/27) response on imipramine vs. 5% (1/22) on placebo (P < 0.02). Response was also more frequent, but not significantly so, among depressed users on imipramine (26%, 10/38) than on placebo (13%, 4/31) (P < 0.19). Response rates were low in intravenous and freebase users and those without depression. Considered together with the literature on desipramine, these data suggest tricyclic antidepressants are not promising as a mainstay of treatment for unselected cocaine abusers. However, tricyclics may be useful for selected cocaine abusers with comorbid depression or intranasal use, or in conjunction with a more potent psychosocial intervention.

Duration of antidepressant trials: clinical and research implications

The objective of our study was to demonstrate that additional antidepressant benefit occurs between weeks 4 and 6 in adult outpatients, even when dose is not increased. Response between weeks 4 and 6 was studied among depressed outpatients randomly assigned to imipramine, phenelzine, or placebo under double-blind conditions. Patients were selected for analysis only if they did not have a dose increase after the start of the fourth week of treatment (day 22). Eighty-eight patients met this condition. Conditional probability analysis was performed. Nonresponders to 4 weeks (28 days) of treatment had a significantly greater likelihood of responding by week 6 if they were on phenelzine rather than placebo. The same is probably true for patients on imipramine. In research and clinical care, 4 weeks is too short a trial of phenelzine to conclude a lack of efficacy. Four weeks is probably also too short a trial of imipramine.

A double-blind placebo-controlled comparison of phenelzine and imipramine in the treatment of bulimia in atypical depressives

Although antidepressants have been found to be superior to placebo in 12 of 14 studies, the relationship between improvement in the depressive diathesis and bulimia is unclear. In this study, the efficacy of placebo, imipramine, and phenelzine is examined in patients comorbid for atypical depression and bulimia. Greater improvement was observed for both depressive and bulimic symptoms with phenelzine than with either imipramine or placebo. Consistent with its poor antidepressant effects in atypical depression, imipramine seemed to have minimal efficacy for the bulimic symptoms of atypical depressives. These data suggest that the presence of bulimia does not alter the treatment response of atypically depressed patients. Furthermore, the data may suggest a link between depression and bulimia in atypical depressives. Demonstrating a statistical difference with a small sample suggests the effect size is robust, however conclusions are limited by a small sample size.

Chronic depression: response to placebo, imipramine, and phenelzine

We reanalyzed data from a larger, previously published study in order to directly address whether very chronically depressed patients could benefit from antidepressant medications. This study entered 598 depressed patients into a study randomizing patients to 6 weeks of double-blind treatment with imipramine, phenelzine, or placebo. Patients were assessed for chronicity on a four-point scale from "mostly well" to "virtually always depressed." The current analyses include only the 153 study completers who were rated as "virtually always depressed." In these patients, imipramine was effective for significantly more patients than was placebo (22 [46%] of 48 responding to imipramine vs. 9 [17%] of 52 responding to placebo; chi 2 = 9.50; p = 0.002), whereas phenelzine was significantly more effective than imipramine (37 [70%] of 53 responding to phenelzine; chi 2 = 5.96; p = .015). Patients with mild depression, early onset, or histories of panic attacks did not have substantially different outcomes than patients without these characteristics. These findings suggest that some chronically depressed patients may be good candidates for treatment with antidepressant medication. Because the majority (80%) of the sample met Columbia criteria for definite or probable atypical depression, too few chronic depressives were available to evaluate separately antidepressant efficacy in chronically depressed outpatients who did not have atypical depression. Hence, these results may be applicable only to patients with atypical depression.

Columbia atypical depression. A subgroup of depressives with better response to MAOI than to tricyclic antidepressants or placebo

We summarise a series of studies using a MAOI to help establish the validity of a subgroup of depressives referred to as atypical depressives. Patients with reactive mood meeting DSM-III criteria for depressive illness who had associated atypical features (which include hyperphagia, hypersomnolence, leaden paralysis, and rejection sensitivity) were randomised to imipramine, phenelzine and placebo. Non-responders were crossed over, and in all there were over 400 patient trials. Phenelzine consistently was found to be superior to imipramine. Only in trials which included patients lacking atypical, vegetative symptoms was imipramine found to equal phenelzine. We conclude that the researcher and the clinician should consider the relevance of the atypical depressive syndrome.

Imipramine treatment of alcoholism with comorbid depression

Of 60 depressed alcoholics who completed an open trial of imipramine, 27 (45%) responded with improvement in both mood and drinking behavior, and eight (13%) responded after further dosage increases or treatment with disulfiram. In a subsequent 6-month, randomized discontinuation trial, four of 13 subjects (31%) relapsed during imipramine treatment and seven of 10 (70%) relapsed while taking placebo. This suggests a potential treatment approach for a high-risk subgroup of alcoholics.

How symptomatic do depressed patients remain after benefiting from medication treatment?

Symptom Check List-90 (SCL-90) scores of 318 moderately depressed, nonmelancholic outpatients treated with phenelzine, imipramine, or placebo were compared with scores obtained from a demographically matched control group of 69 "normal" subjects. Pretreatment scores of depressed patients were significantly higher than those of the control group; after 6 weeks of treatment, they were indistinguishable from scores of the control group. These results suggest that symptoms of moderately depressed patients who benefit from treatment not only decrease from baseline, but are reduced to a level of symptomatology that is psychiatrically normal.

Loss of drug effects during continuation therapy

OBJECTIVE

This study sought to determine what proportion of relapses during continuation therapy with antidepressants can be attributed to loss of nonspecific placebo effects while the patients are taking the drugs.

METHOD

Depressed patients were studied over a 12-week period. One hundred sixty-four patients were randomly assigned to placebo, 174 to imipramine, and 169 to phenelzine. At 6 weeks 35 were judged to be responders to placebo, 70 to imipramine, and 96 to phenelzine. These patients continued their double-blind treatment for weeks 7-12.

RESULTS

Thirty-one percent of the patients who were taking placebo, approximately 12% who were taking imipramine, and approximately 9% who were taking phenelzine relapsed in the 7- to 12-week phase. Two different methods of estimating relapses suggested that during the first 3 months of treatment, a large percentage of the relapses of patients taking drugs was attributable to the loss of nonspecific placebo effects rather than true drug effects.

CONCLUSIONS

A considerable proportion of relapses in the first 3 months of treatment with antidepressants appears to be due to loss of placebo effects. These clinically relevant data may be used to encourage patients who relapse during this period, and who erroneously conclude that anti-depressant effects are temporary, to try another medication.

Further evidence that a placebo response to antidepressants can be identified

OBJECTIVE

The authors' goal was to analyze the acute phase of antidepressant drug treatment to identify placebo responses.

METHOD

Patients rated as improved after 6 weeks of double-blind treatment with imipramine or phenelzine were followed for an additional 6 weeks of double-blind treatment. Initial responses were classified according to the speed of improvement (abrupt or gradual), the persistence or nonpersistence of improvement, and the timing of improvement (early or late onset).

RESULTS

It was predicted that patients with nonpersistent, abrupt responses to the drugs were actually experiencing a placebo response and would have the worst prognosis. In fact, this group accounted for a disproportionate number of the relapses. Nonpersistent responders to a drug had a 23.7% relapse rate, but persistent responders had only a 9.0% relapse rate, a significant difference.

CONCLUSIONS

The authors conclude that a significant proportion of relapses within the first 6 weeks of treatment with an active drug are not related to loss of a true drug effect. Rather, some are related to loss of nonspecific placebo effects, and abrupt nonpersistent responses during drug treatment are most likely the result of placebo effects.

Demoralization Predicts Nonresponse to Cognitive Therapy in Depressed Outpatients

Thirty-nine depressed outpatients meeting DSM-III criteria for nonmelancholic major depression or dysthymic disorder were treated with 16 weekly individual cognitive therapy sessions. Prior to treatment, they completed the Beck Depression Inventory, the Hopelessness Scale, and the Dysfunctional Attitudes Scale. Independent of knowledge of outcome, the authors chose from these scales items indicating demoralization, that is, that patients perceived their ability to positively affect their own future as too likely to be ineffectual to warrant efforts at change. After cognitive therapy, 20 patients were considered responders (51%) although three quickly relapsed (44% responded and maintained). Nonresponders had significantly higher pretreatment demoralization scores than did responders. These results suggest that high levels of demoralization may predict poor response of depression to cognitive therapy, although the small sample size precluded differentiation of demoralization from hopelessness.

A double-blind crossover trial of imipramine and phenelzine for outpatients with treatment-refractory depression

OBJECTIVE

Data from controlled studies concerning the response rates of patients to a second antidepressant medication after they have been unresponsive to a systematic trial of another antidepressant are extremely useful to clinicians for rational prescription of pharmacotherapy. Such information allows making an accurate prognosis, sustaining realistic hope in the patient, and achieving the best possible therapeutic outcome. This study was designed to add to the scanty literature available on this subject.

METHOD

Eighty-nine mood-reactive, nonmelancholic, mainly chronically depressed outpatients at a university research clinic who were unresponsive to vigorous double-blind trials of imipramine or phenelzine were crossed over to treatment with the other drug under double-blind conditions.

RESULTS

Of 46 patients previously unresponsive to imipramine who completed phenelzine treatment, 31 (67%) responded to phenelzine. Of 22 patients previously unresponsive to phenelzine who completed imipramine treatment, nine (41%) responded to imipramine. The difference in response rates was statistically significant. Even after they had shown no response to 7 weeks of placebo and 6 weeks of imipramine treatment, 10 (83%) of 12 patients who then completed treatment with phenelzine responded.

CONCLUSIONS

These data suggest that among chronically ill, mood-reactive depressed patients with many symptoms of atypical depression, phenelzine is strikingly effective in those who have been nonresponders to imipramine and should be tried in such patients.