An Item Response Theory Model for Incorporating Response Times in Forced-Choice Measures

Forced-choice (FC) measures have been widely used in many personality or attitude tests as an alternative to rating scales, which employ comparative rather than absolute judgments. Several response biases, such as social desirability, response styles, and acquiescence bias, can be reduced effectively. Another type of data linked with comparative judgments is response time (RT), which contains potential information concerning respondents' decision-making process. It would be challenging but exciting to combine RT into FC measures better to reveal respondents' behaviors or preferences in personality measurement. Given this situation, this study aims to propose a new item response theory (IRT) model that incorporates RT into FC measures to improve personality assessment. Simulation studies show that the proposed model can effectively improve the estimation accuracy of personality traits with the ancillary information contained in RT. Also, an application on a real data set reveals that the proposed model estimates similar but different parameter values compared with the conventional Thurstonian IRT model. The RT information can explain these differences.

Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

The ability to rapidly provide examinees with detailed and effective diagnostic information is a critical topic in psychology. Knowing what diagnostic criteria the examinees have met enables the practitioner to seek the solution to help them in a timely manner, and this can be achieved by cognitive diagnostic computerized adaptive testing (CD-CAT). However, the pervasive challenge of replenishing items in the CD-CAT item bank limits its practical application. Online calibration is a means to address item replenishment, but in CD-CAT, most existing online calibration methods that jointly calibrate the Q-matrix and item parameters of the new items are developed only for dichotomous responses and are time-consuming. Notably, previous studies pay no attention to polytomously scored items that are frequently observed in testing, even though they can offer additional evidence for the examinees' diagnosis. To fill this gap, we propose a SCAD-based method (SCAD-EM) to calibrate the Q-matrix and item parameters of the new items with polytomous response data in order to promote the application of CD-CAT in practice. The performance of the SCAD-EM was investigated in two comprehensive simulation studies and compared against the revised single-item estimation method (SIE-BIC). Results indicated that the SCAD-EM produces a higher calibration accuracy for the category-level Q-matrix and is computationally more efficient across all conditions, but it produces a lower calibration accuracy for the item-level Q-matrix. An empirical study further demonstrated the utility of the SCAD-EM and the SIE-BIC methods in calibrating new items with a real dataset. The advantages of the proposed method, its limitations, and possible future research directions are offered at the end.

Item selection methods in multidimensional computerized adaptive testing for forced-choice items using Thurstonian IRT model

Multidimensional computerized adaptive testing for forced-choice items (MFC-CAT) combines the benefits of multidimensional forced-choice (MFC) items and computerized adaptive testing (CAT) in that it eliminates response biases and reduces administration time. Previous studies that explored designs of MFC-CAT only discussed item selection methods based on the Fisher information (FI), which is known to perform unstably at early stages of CAT. This study proposes a set of new item selection methods based on the KL information for MFC-CAT (namely MFC-KI, MFC-KB, and MFC-KLP) based on the Thurstonian IRT (TIRT) model. Three simulation studies, including one based on real data, were conducted to compare the performance of the proposed KL-based item selection methods against the existing FI-based methods in three- and five-dimensional MFC-CAT scenarios with various test lengths and inter-trait correlations. Results demonstrate that the proposed KL-based item selection methods are feasible for MFC-CAT and generate acceptable trait estimation accuracy and uniformity of item pool usage. Among the three proposed methods, MFC-KB and MFC-KLP outperformed the existing FI-based item selection methods and resulted in the most accurate trait estimation and relatively even utilization of the item pool.

A general nonparametric classification method for multiple strategies in cognitive diagnostic assessment

Cognitive diagnosis models (CDMs) have been used as psychometric tools in educational assessments to estimate students' strengths and weaknesses in terms of cognitive skills learned and skills that need study. In practice, it is not uncommon that questions can often be solved using more than one strategy, which requires CDMs capable of accommodating multiple strategies. However, existing parametric multi-strategy CDMs need a large sample size to produce a reliable estimation of item parameters and examinees' proficiency class memberships, which obstructs their practical applications. This article proposes a general nonparametric multi-strategy classification method with promising classification accuracy in small samples for dichotomous response data. The method can accommodate different strategy selection approaches and different condensation rules. Simulation studies showed that the proposed method outperformed the parametric CDMs when sample sizes were small. A set of real data was analyzed as well to illustrate the application of the proposed method in practice.

Using Process Data to Improve Classification Accuracy of Cognitive Diagnosis Model

With the advance of computer-based assessments, many process data, such as response times (RTs), action sequences, Eye-tracking data, the log data for collaborative problem-solving (CPS) and mouse click/drag becomes readily available. Findings from previous studies (e.g., Peng et al., Multivariate Behavioral Research, 1-20, 2021; Xu, The British Journal of Mathematical and Statistical Psychology, 73(3), 474-505, 2020; He & von Davier, Handbook of research on technology tools for real-world skill development (pp. 750-777). IGI Global, 2016; Man & Harring, Educational and Psychological Measurement, 81(3), 441-465, 2021) suggest a substantial relationship between this human-computer interactive process information and proficiency, which means these process data were potentially useful variables for psychological and educational measurement. To make full use of the process data, this paper aims to combine two useful and easily available types of process data, including the mouse click/drag traces and the response times, to the conventional cognitive diagnostic model (CDM) to better understand individual's response behavior and improve the classification accuracy of existing CDM. Then the full Bayesian analysis using Markov chain Monte Carlo (MCMC) was employed to estimate the proposed model parameters. The viability of the proposed model was investigated by an empirical data and two simulation studies. Results indicated the proposed model combing both types of process data could not only improve the attribute classification reliability in real data analysis, but also provide an improvement on item parameters recovery and person classification accuracy.

Psychometric Properties and Measurement Invariance of the LIVES Daily Hassles Scale in Chinese Samples

Daily hassles have a larger effect on our health and well-being than those major events in daily life. The present study aimed to evaluate the psychometric properties and measurement invariance of the LIVES Daily Hassles Scale (LIVES-DHS) in Chinese samples, which consisted of 815 people at work aged between 20 and 60 years old. The results of both Explanatory Factor Analysis and Confirmatory Factor Analysis showed that the five-factor model solution was better than other solutions, which supported the original structure of LIVES-DHS. The Cronbach's alpha coefficients of the five subdimensions varied between.721 and.818, with the entire scale of.920, and McDonald's ω values of the five subdimensions varied between.716 and.821, with the entire scale of.936. The results also showed the support for measurement invariance of the five-factor model across different groups, which is the first to offer evidence for configural, metric, scalar and strict invariance of LIVES-DHS across gender, age and educational groups.

Measurement invariance and latent mean differences of the morbid curiosity scale (MCS) across the United States and China

In recent years, there has been a growing body of research focusing on morbid curiosity. However, the development of measurement tools has been slow, with only two scales available. Compared to the unidimensional scale of Curiosity About Morbid Events (CAME) proposed by Zuckerman and Little (1986), the recently developed four-factor Morbid Curiosity Scale (MCS) by Scrivner (2021) demonstrates a stable factor structure and good reliability and validity. As the time since the development of this scale is relatively short, its measurement properties have not been widely evaluated. Therefore, this study used exploratory and confirmatory factor analyses to validate the factor structure of the MCS in the Chinese cultural context, and the results supported the four-factor structure of the MCS. Additionally, we established partial scalar invariance of the MCS between Chinese (N = 663) and American (N = 330) cultures, and further analyzed cultural differences in morbid curiosity using latent mean comparison. The results revealed that Chinese individuals had a lower motivation for understanding the minds of dangerous people. This study validated the four-factor Morbid Curiosity Scale across different cultures for the first time, promoting the generalizability of the four-factor MCS and suggesting its potential for use in a wide range of cultural backgrounds. These findings contribute to enriching cross-cultural research on morbid curiosity and its associated psychological factors.

A Mixed Sequential IRT Model for Mixed-Format Items

To provide more insight into an individual's response process and cognitive process, this study proposed three mixed sequential item response models (MS-IRMs) for mixed-format items consisting of a mixture of a multiple-choice item and an open-ended item that emphasize a sequential response process and are scored sequentially. Relative to existing polytomous models such as the graded response model (GRM), generalized partial credit model (GPCM), or traditional sequential Rasch model (SRM), the proposed models employ an appropriate processing function for each task to improve conventional polytomous models. Simulation studies were carried out to investigate the performance of the proposed models, and the results indicated that all proposed models outperformed the SRM, GRM, and GPCM in terms of parameter recovery and model fit. An application illustration of the MS-IRMs in comparison with traditional models was demonstrated by using real data from TIMSS 2007.

Two efficient selection methods for high-dimensional CD-CAT utilizing max-marginals factor from MAP query and ensemble learning approach

Computerized adaptive testing for cognitive diagnosis (CD-CAT) needs to be efficient and responsive in real time to meet practical applications' requirements. For high-dimensional data, the number of categories to be recognized in a test grows exponentially as the number of attributes increases, which can easily cause system reaction time to be too long such that it adversely affects the examinees and thus seriously impacts the measurement efficiency. More importantly, the long-time CPU operations and memory usage of item selection in CD-CAT due to intensive computation are impractical and cannot wholly meet practice needs. This paper proposed two new efficient selection strategies (HIA and CEL) for high-dimensional CD-CAT to address this issue by incorporating the max-marginals from the maximum a posteriori query and integrating the ensemble learning approach into the previous efficient selection methods, respectively. The performance of the proposed selection method was compared with the conventional selection method using simulated and real item pools. The results showed that the proposed methods could significantly improve the measurement efficiency with about 1/2-1/200 of the conventional methods' computation time while retaining similar measurement accuracy. With increasing number of attributes and size of the item pool, the computation time advantage of the proposed methods becomes more significant.

A Generalized Diagnostic Classification Modeling Framework Integrating Differential Speediness: Advantages and Illustrations in Psychological and Educational Testing

To advance the theoretical foundation of incorporating response times (RTs) into diagnostic classification models (DCMs), this study attempts to further derive, test and illustrate a generalized modeling framework (known as the JVRT-LCDM) that can simultaneously analyze response accuracy and differential speediness based on an existing method (Zhan et al., British Journal of Mathematical and Statistical Psychology, 71(2), 262-286, 2018). The JVRT-LCDM not only provides fine-grained diagnostic feedback without strict model constraints but also clarifies the specific speed trajectory of individuals. Moreover, some existing models from psychometric literatures are included in the JVRT-LCDM as special cases. The feasibility of the JVRT-LCDM is investigated via a Monte Carlo simulation study using a Bayesian estimation scheme, and two empirical datasets are then analyzed to illustrate the applicability of the JVRT-LCDM in practice. The results indicate that (1) as a generalized and flexible model, the JVRT-LCDM realizes high correct classification rates and accurate speed parameter recovery; (2) the JVRT-LCDM outperforms the existing models in terms of model-data fit, diagnostic consistency, and estimation of specific individuals in practical cognitive diagnosis assessments; and (3) the JVRT-LCDM provides reliable evidence for nonconstant speed modeling.

Dual-Objective Item Selection Methods in Computerized Adaptive Test Using the Higher-Order Cognitive Diagnostic Models

To efficiently obtain information about both the general abilities and detailed cognitive profiles of examinees from a single model that uses a single-calibration process, higher-order cognitive diagnostic computerized adaptive testing (CD-CAT) that employ higher-order cognitive diagnostic models have been developed. However, the current item selection methods used in higher-order CD-CAT adaptively select items according to only the attribute profiles, which might lead to low precision regarding general abilities; hence, an appropriate method was proposed for this CAT system in this study. Under the framework of the higher-order models, the responses were affected by attribute profiles, which were governed by general abilities. It is reasonable to hold that the item responses were affected by a combination of general abilities and attribute profiles. Based on the logic of Shannon entropy and the generalized deterministic, inputs, noisy "and" gate (G-DINA) model discrimination index (GDI), two new item selection methods were proposed for higher-order CD-CAT by considering the above combination in this study. The simulation results demonstrated that the new methods achieved more accurate estimations of both general abilities and cognitive profiles than the existing methods and maintained distinct advantages in terms of item pool usage.

Combining Cognitive Diagnostic Computerized Adaptive Testing With Multidimensional Item Response Theory

The new generation of tests not only focuses on the general ability but also the process of finer-grained skills. Under the guidance of this thought, researchers have developed a dual-purpose CD-CAT (Dual-CAT). In the existing Dual-CAT, the models used in overall ability estimation are unidimensional IRT models, which cannot apply to the multidimensional tests. This article intends to develop a multidimensional Dual-CAT to improve its applicability. To achieve this goal, this article firstly proposes some item selection methods for the multidimensional Dual-CAT, and then verifies the estimation accuracy and exposure rate of these methods through both simulation study and a real item bank study. The results show that the established multidimensional Dual-CAT is effective and the new proposed methods outperform the traditional methods. Finally, this article discusses the future direction of the Dual-CAT.

Computerized Adaptive Testing for Sleep Disorders

Abstract. Background: As more and more people suffer from sleep disorders, the need to develop an efficient, inexpensive, and accurate assessment tool for screening sleep disorders has become more urgent. Aim: The aim of the current study was to develop a system allowing computerized adaptive testing for sleep disorders (CAT-SD). Methods: A large sample ( N = 1,304) was recruited to construct an item bank for CAT-SD and to investigate the psychometric characteristics of CAT-SD. First, analyses of unidimensionality, model fit, item fit, item discrimination parameters, and differential item functioning (DIF) were conducted to construct a final item pool to meet the requirements of item response theory measurement. Then, a simulated CAT study with real data was performed to investigate the psychometric characteristics of CAT-SD, including the reliability, validity, and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD had good item fit, high discrimination, and no DIF. Moreover, it had acceptable reliability, validity, and predictive utility. Limitations: Non-statistical assembly constraints, execution environment, construction of item bank, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-SD, and sample representativeness are discussed. Conclusions: The CAT-SD could be used as an effective and accurate assessment tool for measuring the sleep disorders in individuals and offers a novel approach to the screening of sleep disorders utilizing psychological scales.

Comparison of Psychometric Characteristics for Five Versions of the Interpersonal Needs Questionnaire in Teenagers Sample

Interpersonal Needs Questionnaire (INQ) is a self-report measure of perceived burdensomeness and thwarted belongingness with five versions in recent studies. There are five versions of INQ. But results from studies using different versions are quite different. Current suicide behavior among teenagers has attracted much attention. But which version is more suitable for teenage samples is still uncertain. It is important to compare the potential differences in different versions of INQ to identify the most psychometrically available version to predict teenagers' acquired capability for suicide and provide them with timely help to reduce teenagers' suicide rates. This study compared the construct validity, internal consistency, validity, and average test information of each version in the sample of teenagers. Results showed the 10-item version provided the most average test information in both thwarted belongingness subscale and perceived burdensomeness subscale, and the INQ-10 is more suitable for teenage samples.

Automated Test Assembly for Multistage Testing With Cognitive Diagnosis

Computer multistage adaptive test (MST) combines the advantages of paper and pencil-based test (P&P) and computer-adaptive test (CAT). As CAT, MST is adaptive based on modules; as P&P, MST can meet the need of test developers to manage test forms and keep test forms parallel. Cognitive diagnosis (CD) can accurately measure students' knowledge states (KSs) and provide diagnostic information, which is conducive to student's self-learning and teacher's targeted teaching. Although MST and CD have a lot of advantages, many factors prevent MST from applying to CD. In this study, we first attempt to employ automated test assembly (ATA) to achieve the objectives of MST in the application of CD (called CD-MST) via heuristic algorithms. The mean correct response probability of all KSs for each item is used to describe the item difficulty of CD. The attribute reliability in CD is defined as the test quantitative target. A simulation study with the G-DINA model (generalized deterministic input noisy "and" gate model) was carried out to investigate the proposed CD-MST, and the results showed that the assembled panels of CD-MST satisfied the statistical and the non-statistical constraints.

Computerized Adaptive Testing for Schizotypal Personality Disorder: Detecting Individuals at Risk

As schizotypal personality disorder (SPD) increasingly prevails in the general population, a rapid and comprehensive measurement instrument is imperative to screen individuals at risk for SPD. To address this issue, we aimed to develop a computerized adaptive testing for SPD (CAT-SPD) using a non-clinical Chinese sample (N = 999), consisting of a calibration sample (N₁ = 497) and a validation sample (N₂ = 502). The item pool of SPD was constructed from several widely used SPD scales and statistical analyses based on the item response theory (IRT) via a calibration sample using a graded response model (GRM). Finally, 90 items, which measured at least one symptom of diagnostic criteria of SPD in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) and had local independence, good item fit, high slope, and no differential item functioning (DIF), composed the final item pool for the CAT-SPD. In addition, a simulated CAT was conducted in an independent validation sample to assess the performance of the CAT-SPD. Results showed that the CAT-SPD not only had acceptable reliability, validity, and predictive utility but also had shorter but efficient assessment of SPD which can save significant time and reduce the test burden of individuals with less information loss.

Developing and Validating an Item Bank for Alcohol Use Disorder Screening in the Chinese Population by Using the Computerized Adaptive Testing

Objective

To detect the individual’s severity of alcohol use disorder (AUD) in an effective and accurate manner, this study aimed to build an item bank for AUD screening and derive the computerized adaptive testing (CAT) version of AUD (CAT-AUD).

Methods

The initial CAT-AUD item bank was selected from the Chinese version of the questionnaires related to AUD according to the DSM-5 criteria. Then 915 valid Chinese samples, covering the healthy individuals and the AUD high-risk individuals, completed the initial CAT-AUD item bank. By testing the unidimensionality, test fit, item fit, discrimination parameter and differential item functioning of the initial item bank, the final CAT-AUD item bank confirming to the requirements of the item response theory (IRT) were obtained. Subsequently, the CAT-AUD simulation study based on the real data of the final item bank conducted to detect characteristics, reliability, validity, and predictive utility (sensitivity and specificity) of CAT-AUD.

Results

The CAT-AUD item bank meeting the IRT psychometric measurement requirements could be well geared into the graded response model. The Pearson’s correlation between the estimated theta via CAT-AUD and the estimated theta via the full-length item bank reached 0.95, and the criterion-related validity was 0.63. CAT-AUD can provide highly reliable test results for subjects whose theta above −0.8 with an average of 16 items. Besides, the predictive utility of CAT-AUD was better than AUDIT and AUDIT-C.

Conclusion

In brief, the CAT-AUD developed in this study can effectively screen the AUD high-risk group and accurately measure the AUD severity of individuals.

Procedures to Develop a Computerized Adaptive Testing to Advance the Measurement of Narcissistic Personality

Narcissistic personality (NP) has recently attracted a great deal of attention. In this study, we mainly investigated the feasibility and effectiveness of using computerized adaptive testing (CAT) to measure NP (CAT-NP). The CAT in this study was simulated by the responses of several NP questionnaires of 1,013 university students as if their responses were collected adaptively. The item bank (85 items) that met the requirements of the psychometric properties of Item Response Theory (IRT) was first established, and then the CAT dynamically selected items according to the estimates of current trait level until the prespecified measurement precision is achieved. Finally, the efficiency and validity of the CAT were verified. The results showed that the proposed CAT-NP had reasonable reliability, validity, predictive utility, and high correlation. In addition, the CAT-NP could significantly save item usage without losing measurement accuracy, which greatly improves the test efficiency. The advantages and limitations of CAT in measuring NP and other psychological tests are discussed in the final section.

Development of a Computerized Adaptive Test for Separation Anxiety Disorder Among Adolescents

BACKGROUND

Separation anxiety disorder (SAD) is one of the most common mental disorders among children and adolescents, and it may seriously affect their growth, daily life, and learning. Self-report scales have been used for diagnosis, which require lengthy testing and personnel.

METHODS

A total of 1,241 adolescents were recruited from 16 junior- and senior-high schools in China. The initial item bank was selected from classical SAD scales according to the DSM-5. First, the optimal model was selected using item response theory (IRT) according to data fit. Then, per the IRT analysis, items that did not meet the psychometric requirements were deleted (e.g., discriminating values < 0.2). Consequently, a computerized adaptive test (CAT) for SAD was formed (CAT-SAD).

RESULTS

An average of 17 items per participant was required to achieve and maintain a 0.3 standard error of measurement in the SAD severity estimate. The estimated correlation of the CAT-SAD with the total 68-item test score was 0.955. CAT-SAD scores were strongly related to the probability of a SAD diagnosis with the Separation Anxiety Assessment Scale-Child and Adolescent Version. Therefore, SAD could be accurately predicted by the CAT-SAD.

CONCLUSIONS

Exploratory factor analyses revealed that SAD was unidimensional. The CAT-SAD, which has good reliability and validity and high sensitivity and specificity, provides an efficient test for adolescents with SAD as compared to standard paper-and-pencil tests. It can be used to diagnose varying degrees of SAD quickly and reliably and ease the burden on adolescents. Potential applications for inexpensive, efficient, and accurate screening of SAD are discussed.

Q-Matrix Estimation Methods for Cognitive Diagnosis Models: Based on Partial Known Q-Matrix

Different from the item response models that postulate a single underlying proficiency, cognitive diagnostic assessments (CDAs) can provide fine-grained diagnostic information about students' knowledge state to aid classroom instructions. In CDAs, a Q-matrix that associates each item in a test with the cognitive skills is required to infer students' knowledge states. In practice, the Q-matrix is typically performed by domain experts, which is certainly affected by the subjective tendency of experts and, to a large extent, may consist of some misspecifications. In addition, if the number of items increases, the expert-based Q-matrix specification will be time-consuming and costly. To address this concern, this paper proposed several approaches based on the likelihood ratio test to estimate Q-matrix with partial known Q-matrix and the response data, which can be used with a wide class of cognitive diagnosis models (CDMs). The feasibility and effectiveness of the proposed methods were evaluated by simulated data generated under various conditions and an example to real data. Results show that new methods can estimate Q-matrix correctly and outperforms the existing method in most conditions.

A diagnostic classification version of Schizotypal Personality Questionnaire using diagnostic classification models

OBJECTIVE

To obtain more precise and rich information from the measurements for schizotypal personality disorder (SPD), a cutting-edge psychometric theory called diagnostic classification models (DCMs) was first employed in the present study to develop a diagnostic classification version of the Schizotypal Personality Questionnaire (DC-SPQ) based on the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders.

METHODS

Under the framework of DCMs, 980 college students were recruited to calibrate item parameters of the Schizotypal Personality Questionnaire. Items that fit the psychometric characteristic would be selected to compose the DC-SPQ, prior to an analysis of its indexes.

RESULTS

Results showed that the DC-SPQ had high reliability and validity in both the classical test theory and DCMs, in addition to showing a sensitivity of 0.921 and a specificity of 0.841 with area under receiver operating characteristic curve = 0.936. Meanwhile, the four-factor model proposed adequately fits with the data. More importantly, the DC-SPQ provides not only the general-level information similar to traditional questionnaires but also the symptom-level information with the posterior probability, which provides an insight into delivering the individual-specific intervention that is tailor made to schizotypal personality disorder.

CONCLUSIONS

This study demonstrates that the DC-SPQ is very valuable for psychometric detection in that it can clarify the symptom being measured and provide more reasonable estimates.

A Canadian cancer trials group phase IB study of durvalumab (anti-PD-L1) plus tremelimumab (anti-CTLA-4) given concurrently or sequentially in patients with advanced, incurable solid malignancies

Background The IND.226 study was a phase Ib study to determine the recommended phase II dose of durvalumab + tremelimumab in combination with standard platinum-doublet chemotherapy. Sequential administration of multiple agents increases total chair time adding costs overall and inconvenience for patients. This cohort of the IND.226 study evaluated the safety and tolerability of durvalumab + tremelimumab given either sequentially (SEQ) or concurrently (CON). Methods Patients with advanced solid tumours were enrolled and randomised to either SEQ tremelimumab 75 mg IV over 1 h followed by durvalumab 1500 mg IV over 1 h q4wks on the same day, or CON administration over 1 h. The serum pharmacokinetic profile of SEQ versus CON of durvalumab and tremelimumab administration was also evaluated. Results 14 patients either received SEQ (n = 7pts) or CON (n = 7 pts). There were no infusion related reactions. Drug related adverse events (AEs) were mainly low grade and manageable, and comparable in frequency between SEQ/CON- fatigue (43%/57%), rash (43%/43%), pruritus (43%/29%) and nausea (14%/29%). One patient in each cohort discontinued treatment due to toxicity. The PK profiles of durvalumab and tremelimumab were similar between CON and SEQ, and to historical reference data. Conclusions Concurrent administration of durvalumab and tremelimumab over 1 h is safe with a comparable PK profile to sequential administration.

Development and Validation of an Item Bank for Drug Dependence Measurement Using Computer Adaptive Testing

Currently, measurement tools to assess patient-reported outcomes for drug dependence are limited in their latent trait to adapt to the needs of individual patients while also maintaining comparability of scores across patients. Purpose/Objectives: To develop an item bank for computer adaptive testing (CAT) to measure severity of drug dependence. Methods: There were four phases: (1) review the literature of drug dependence measurement; (2) formulate an item list to be assessed by experts; (3) pretest our item list in two substance dependence treatment centers; and (4) field-test and conduct psychometric performance analysis with the final item bank. Additionally, based on our response data, a CAT simulation was used to validate the item bank, Drug Dependence CAT (DD-CAT). Results: The final drug dependence item bank - with a unidimensional configuration - contained 56 items with good item-fit, high discrimination, no differential item functioning, and covered all symptoms of diagnostic criteria for drug dependence. These results revealed that the final item bank was of good quality. Additionally, the results of a simulation CAT procedure with real response data indicated that the DD-CAT item bank exhibited acceptable and reasonable test reliability, content validity, and criterion-related validity. Conclusions/Importance: The proposed item bank for DD-CAT contained acceptable reliability and validity, and exhibited a shorter but efficient assessment of drug dependence. These psychometric properties can result in shorter test times, less information loss, and a reduction in the testing burden of patients.

Development of a New Instrument for Depression With Cognitive Diagnosis Models

Most existing instruments for depression are developed based on classical test theory, factor analysis, or sometimes, item response theory, and focus on the accurate measurement of the severity of depressive disorder. Nevertheless, they tend to be less useful in supporting the decision based on ICD-10 or DSM-5 because of the lack of detailed information for symptoms. To gain rich and valid information at the symptom level, this article developed a depression test under the framework of cognitive diagnosis models (CDMs), referred to as CDMs-D. A total of 1,181 individuals were finally recruited and their responses were used to examine the psychometric properties of CDMs-D. After excluding poor items for statistical reasons (e.g., low discrimination, poor model-fit or having DIF), 56 items were included in the CDMs-D. The CDMs-D measures all ten symptom criteria for depression defined in ICD-10 and covers five domains of depression defined by Gibbons et al. (2012). Comparing with the existing self-report measures (such as PHQ-9, SDS, CES-D and so on), a distinguishing feature of the CDMs-D is that it can provide both overall information about the severity of depressive disorder and the assessment information about specific symptoms, which could be useful for diagnostic and interventional purposes.

Cognitive Diagnostic Models With Attribute Hierarchies: Model Estimation With a Restricted Q-Matrix Design

Attribute hierarchy is a common assumption in the educational context, where the mastery of one attribute is assumed to be a prerequisite to the mastery of another one. The attribute hierarchy can be incorporated through a restricted Q matrix that implies the specified structure. The latent class-based cognitive diagnostic models (CDMs) usually do not assume a hierarchical structure among attributes, which means all profiles of attributes are possible in a population of interest. This study investigates different estimation methods to the classification accuracy for a family of CDMs when they are combined with a restricted Q-matrix design. A simulation study is used to explain the misclassification caused by an unrestricted estimation procedure. The advantages of the restricted estimation procedure utilizing attribute hierarchies for increased classification accuracy are also further illustrated through a real data analysis on a syllogistic reasoning diagnostic assessment. This research can provide guidelines for educational and psychological researchers and practitioners when they use CDMs to analyze the data with a restricted Q-matrix design and make them be aware of the potentially contaminated classification results if ignoring attribute hierarchies.

Development of a Computerized Adaptive Testing for Internet Addiction

Internet addiction disorder has become one of the most popular forms of addiction in psychological and behavioral areas, and measuring it is growing increasingly important in practice. This study aimed to develop a computerized adaptive testing to measure and assess internet addiction (CAT-IA) efficiently. Four standardized scales were used to build the original item bank. A total of 59 polytomously scored items were finally chosen after excluding 42 items for failing the psychometric evaluation. For the final 59-item bank of CAT-IA, two simulation studies were conducted to investigate the psychometric properties, efficiency, reliability, concurrent validity, and predictive validity of CAT-IA under different stopping rules. The results showed that (1) the final 59 items met IRT assumptions, had high discrimination, showed good item-model fit, and were without DIF; and (2) the CAT-IA not only had high measurement accuracy in psychometric properties but also sufficient efficiency, reliability, concurrent validity, and predictive validity. The impact and limitations of CAT-IA were discussed, and several suggestions for future research were provided.

The economic impact of the transition from branded to generic oncology drugs

Background

Economic evaluations are an integral component of many clinical trials. Costs used in those analyses are based on the prices of branded drugs when they first enter the market. The effect of genericization on the cost-effectiveness (ce) or cost-utility (cu) of an intervention is unknown because economic analyses are rarely updated using the costs of generic drugs.

Methods

We re-examined the ce or cu of regimens previously evaluated in Canadian Cancer Trials Group (cctg) studies that included prospective economic evaluations and where genericization has occurred or is anticipated in Canada. We incorporated the new costs of generic drugs to characterize changes in ce or cu. We also determined acceptable cost levels of generic drugs that would make regimens reimbursable in a publicly funded health care system.

Results

The four randomized controlled trials included (representing 1979 patients) were cctg br.10 (early lung cancer, adjuvant vinorelbine-cisplatin vs. observation, n = 172), cctg br.21 (metastatic lung cancer, erlotinib vs. placebo, n = 731), cctg co.17 (metastatic colon cancer, cetuximab vs. best supportive care, n = 557), and cctg ly.12 (relapsed or refractory lymphoma, gemcitabine-dexamethasone-cisplatin vs. cytarabine-dexamethasone-cisplatin, n = 619). Since the initial publication of those trials, the genericization of vinorelbine, erlotinib, cetuximab, and cisplatin has taken place or is expected in Canada. Costs of generics improved the ces and cus of treatment significantly. For example, genericization of erlotinib ($1460.25 per 30 days) resulted in an incremental cost-effectiveness ratio (icer) of $45,746 per life-year gained compared with $94,638 for branded erlotinib. Likewise, genericization of cetuximab ($275.80 per 100 mg) produced an icer of $261,126 per quality-adjusted life-year (qaly) gained compared with $299,613 for branded cetuximab. Decreases in the cost of generic cetuximab to $129.39 and $63.51 would further improve the icer to $150,000 and $100,000 per QALY respectively.

Conclusions

Genericization of a costly oncology drug can modify the ce and cu of a regimen significantly. Failure to revisit economic analyses with the costs of generics could be a missed opportunity for funding bodies to optimize value-based allocation of health care resources. At current levels, the costs of generics might not be sufficiently low to sustain publicly funded health care systems.

Abstract P4-14-03: Influence of competing risks of death on the interpretation of adjuvant endocrine therapy trials for breast cancer

Background: Early stage, hormone sensitive breast cancer is associated generally with a good prognosis, with only a minority of patients expected to die of breast cancer. Death from causes other than breast cancer can dilute the patients at risk of breast cancer events and result in over-estimation of risk of recurrence and consequently the benefit from breast cancer therapy, a so-called immortal time bias. The MA.17R trial (Goss et al 2016) evaluated the role of extending adjuvant treatment with letrozole from 5 to 10 years. Here we determine the effect of analyzing the MA.17R trial using methods accounting for competing risks.

Methods: We compared conventional and competing risk methods for disease-free survival (DFS) and for distant recurrence-free survival (DRFS). In Kaplan-Meier analyses death from any cause was considered an event while cumulative incidence functions (CIFs) assumed death without recurrence to be a competing risk. The complement of the survival function (one minus the survival function) was used to estimate incidence of the primary event of interest. This was compared to estimates obtained using CIFs accounting for the occurrence of competing events.

Results: Non-breast cancer death was the most common event defining DFS and DRFS. Over the course of follow-up, there was increasing discrepancy between the risk of disease recurrence measured using Kaplan-Meier and CIF. Among letrozole treated patients the estimated distant recurrence at 5 years of follow-up was 5.4% using CIF and 9.6% using Kaplan-Meier. At 10 years of follow-up, the estimated distant recurrence was 8.4% using CIF and 20.0% using Kaplan-Meier. Similar results were observed for the placebo group (8.5% vs 12.1% at 5 years and 14.8% vs 27.3% at 10 years), and in patients with baseline cardiovascular disease (see Table). Benefit from letrozole on DFS and DRFS was greater when accounting for competing risk (hazard ratio [HR] for DFS 0.66, 95%CI 0.48-0.90; DFRS HR 0.75, 0.50-1.14) compared to the conventional method (DFS HR 0.79, 0.62-0.99; DRFS HR 0.91, 0.70-1.18). In women with baseline cardiovascular risk, the benefits of extended adjuvant letrozole when considering competing risk (DFS HR 0.38, 0.16-0.89; DRFS HR 0.46, 0.16-1.35) were also greater than those observed in the conventional analysis (DFS HR 0.55, 0.32-0.93; DRFS HR 0.59, 0.33-1.04). Treatment with extended letrozole did not influence non-breast cancer death in women who died with disease recurrence (HR 1.06, 0.74 -1.50) or in those with competing risk or censored from the analysis (HR 1.05, 0.73 -1.49).

Conclusion: Over the course of follow-up, estimates of DFS and DRFS differ increasingly if measured using Kaplan-Meier or CIF, with CIF estimates of risk being substantially lower. Using a competing risk model, the reduction in distant recurrence at 8 years with extended letrozole is less than 1%. Additional competing risk analyses of the MA.17 (Goss 2006) and MA.27 (Goss 2013) trials are ongoing.

Cumulative incidence of disease recurrence in patients with baseline cardiovascular riskTime (years)CIF (%)1-KM (%)Letrozole11.51.534.46.555.813.8Placebo13.53.538.311.8512.520.3

Citation Format: Ethier J-L, Parulekar W, Shepherd L, Summers L, Strasser-Weippl K, Tu D, Amir E. Influence of competing risks of death on the interpretation of adjuvant endocrine therapy trials for breast cancer [abstract]. In: Proceedings of the 2018 San Antonio Breast Cancer Symposium; 2018 Dec 4-8; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2019;79(4 Suppl):Abstract nr P4-14-03.

Psychometric Properties of the SAS, BAI, and S-AI in Chinese University Students

Three widely-used self-report anxiety scales, including the Self-Rating Anxiety Scale (SAS), the Beck Anxiety Inventory (BAI), and the State Anxiety Inventory (S-AI), were used to simultaneously compare the psychometric properties via an item response theory (IRT) model with Chinese university students as the sample. Although these scales were probably to measure the same underlying construct, namely, anxiety, their psychometric properties were different. Results showed that the BAI's measurement error was fewer than that of the other scales, with their anxiety severity ranging approximately from the 0.8 standard deviations below the mean to 3 standard deviations above the mean, while the S-AI's measurement error was fewer than that of the other degrees of anxiety. The S-AI provided more information than the other scales when the student's scale was less than approximately 0.8 standard deviations below the mean of anxiety severity. In general, the BAI showed better, for it provided more information than the other scales at the broadest range of anxiety severity. The SAS provided less information than the other scales at all anxiety severity range. In conclusion, BAI shows good psychometric quality. Finally, the three instruments were combined on a common scale by using IRT model and a conversion table was provided so as to achieve the transformation of each scale score.

Mobilization of iron from ferritin: new steps and details

Much evidence indicates that iron stored in ferritin is mobilized through protein degradation in lysosomes, but concerns about this process have lingered, and the mechanistic details of its aspects are lacking. In the studies presented here, ⁵⁹Fe-labeled ferritin was induced by preloading hepatic (HepG2) cells with radiolabeled Fe. Placing these cells in a medium containing desferrioxamine resulted in the loss of ferritin-⁵⁹Fe, but adding high concentrations of reducing agents or modulating the internal GSH concentration failed to alter the rates of ferritin-⁵⁹Fe release. Confocal microscopy showed that Fe deprivation increased the movement of ferritin into lysosomes and hyperaccumulation was observed when lysosomal proteolysis was inhibited. It also resulted in the rapid movement of DMT1 to lysosomes, which was inhibited by bafilomycin. Ferrihydrite crystals isolated from purified rat liver/spleen ferritin were solubilized at pH 5 and 7 by GSH, ascorbate, citrate and lysosomal fluids obtained from livers and J774a.1 macrophages. The inhibition of DMT1/Nramp2 and siRNA knockdown of Nramp1 each reduced the transfer of ⁵⁹Fe from lysosomes to the cytosol; and hepatocyte-specific knockout of DMT1 in mice prevented the release of Fe from the liver responding to EPO treatment, but did not inhibit lysosomal ferritin degradation. We conclude that ferritin-Fe mobilization does not occur through changes in cellular concentrations of reducing/chelating agents but by the coordinated movement of ferritin and DMT1 to lysosomes, where the ferrihydrite crystals exposed by ferritin degradation dissolve in the lysosomal fluid, and the reduced iron is transported back to the cytosol via DMT1 in hepatocytes, and by both DMT1 and Nramp1 in macrophages, prior to release into the blood or storage in ferritin.

Structure of Arabic Scale of Death Anxiety With Chinese College Students: A Bifactor Approach

The Arabic Scale of Death Anxiety (ASDA), as one of the most widely used measures of death anxiety (DA), has increasingly been applied in many studies. However, the structures derived from different studies are highly inconsistent. In this study, both traditional and novel (bifactor) modeling approaches were used, to investigate the most optimal structure of the ASDA in a sample of 984 Chinese college students. After a series of comparisons, the results showed that the bifactor model, with a dominant general DA factor and three distinct sub-dimensions, was the most optimal measurement structure, and measurement invariance of this bifactor model between sexes was also confirmed. Based on the implications of this bifactor model, the discussion was focused mainly on whether distinct dimensions should be interpreted or not. Some strengths and limitations of the study were also discussed at the end of the paper.

Hypertension as a predictor of advanced colorectal cancer outcome and cetuximab treatment response

Background

Adrenergic receptor stimulation is involved in the development of hypertension (htn) and has been implicated in cancer progression and dissemination of metastases in various tumours, including colon cancer. Adrenergic antagonists such as beta-blockers (bbs) demonstrate inhibition of invasion and migration in colon cancer cell lines and have been associated with decreased mortality in colorectal cancer (crc). We examined the association of baseline htn and bb use with overall (os) and progression-free survival (pfs) in patients with pretreated, chemotherapy refractory, metastatic crc (mcrc). We also examined baseline htn as a predictor of cetuximab efficacy.

Methods

Using data from the Canadian Cancer Trials Group co.17 study [cetuximab vs. best supportive care (bsc)], we coded baseline htn and use of anti-htn medications, including bbs, for 572 patients. The chi-square test was used to assess the associations between those variables and baseline characteristics. Cox regression models were used for univariate and multivariate analyses of os and pfs by htn diagnosis and bb use.

Results

Baseline htn, bb use, and anti-htn medication use were not found to be prognostic for improved os. Baseline htn and bb use were not significant predictors of cetuximab benefit.

Conclusions

In chemorefractory mcrc, neither baseline htn nor bb use is a significant prognostic factor. Baseline htn and bb use are not predictive of cetuximab benefit. Further investigation to determine whether baseline htn or bb use have a similarly insignificant impact on prognosis in patients receiving earlier lines of treatment remains warranted.

Item Selection Methods in Multidimensional Computerized Adaptive Testing With Polytomously Scored Items

Multidimensional computerized adaptive testing (MCAT) has been developed over the past decades, and most of them can only deal with dichotomously scored items. However, polytomously scored items have been broadly used in a variety of tests for their advantages of providing more information and testing complicated abilities and skills. The purpose of this study is to discuss the item selection algorithms used in MCAT with polytomously scored items (PMCAT). Several promising item selection algorithms used in MCAT are extended to PMCAT, and two new item selection methods are proposed to improve the existing selection strategies. Two simulation studies are conducted to demonstrate the feasibility of the extended and proposed methods. The simulation results show that most of the extended item selection methods for PMCAT are feasible and the new proposed item selection methods perform well. Combined with the security of the pool, when two dimensions are considered (Study 1), the proposed modified continuous entropy method (MCEM) is the ideal of all in that it gains the lowest item exposure rate and has a relatively high accuracy. As for high dimensions (Study 2), results show that mutual information (MUI) and MCEM keep relatively high estimation accuracy, and the item exposure rates decrease as the correlation increases.

Theorems and Methods of a Complete Q Matrix With Attribute Hierarchies Under Restricted Q-Matrix Design

The design of test Q matrix can directly influence the classification accuracy of a cognitive diagnostic assessment. In this paper, we focus on Q matrix design when attribute hierarchies are known prior to test development. A complete Q matrix design is proposed and theorems are presented to demonstrate that it is a necessary and sufficient condition to guarantee the identifiability of ideal response patterns. A simulation study is also conducted to detect the effects of the proposed design on a family of conjunctive diagnostic models. The results revealed that the proposed Q matrix design is the key condition for guaranteeing classification accuracy. When only one type of item pattern in R matrix is missing from the associated test Q matrix, the related attribute-wise agreement rate will decrease dramatically. When the entire R matrix is missing, both the pattern-wise and attribute-wise agreement rates will decrease sharply. This indicates that the proposed procedures for complete Q matrix design with attribute hierarchies can serve as guidelines for test blueprint development prior to item writing in a cognitive diagnostic assessment.

Development and Validation of an Item Bank for Depression Screening in the Chinese Population Using Computer Adaptive Testing: A Simulation Study

With the increasing prevalence of depression, creating a simple and precise tool for measuring depression is becoming more important. This study developed a computer adaptive testing for depression (CAT-Depression) from a Chinese sample. The depression item bank was constructed from a sample of 1,135 participants with or without depression using the Graded Response Model (GRM; Samejima, 1969). The final depression item bank with strict unidimensionality comprised 68 items, which had local independence, good item-fit, high discrimination, no differential item functioning (DIF), and each item measured at least one symptom of diagnostic criteria for depression in ICD-10. In addition, the mean IRT discrimination of the item bank reached 1.784, which clearly showed that the item bank of CAT-Depression was high-quality. Moreover, a simulation CAT study with real response data was conducted to investigate the characteristics, marginal reliability, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-Depression. The results revealed that the proposed CAT-Depression had acceptable and reasonable marginal reliability, criterion-related validity, and sensitivity and specificity.

OV21/PETROC: a randomized Gynecologic Cancer Intergroup phase II study of intraperitoneal versus intravenous chemotherapy following neoadjuvant chemotherapy and optimal debulking surgery in epithelial ovarian cancer

Background

The purpose of this multistage, adaptively, designed randomized phase II study was to evaluate the role of intraperitoneal (i.p.) chemotherapy following neoadjuvant chemotherapy (NACT) and optimal debulking surgery in women with epithelial ovarian cancer (EOC).

Patients and methods

We carried out a multicenter, two-stage, phase II trial. Eligible patients with stage IIB-IVA EOC treated with platinum-based intravenous (i.v.) NACT followed by optimal (<1 cm) debulking surgery were randomized to one of the three treatment arms: (i) i.v. carboplatin/paclitaxel, (ii) i.p. cisplatin plus i.v./i.p. paclitaxel, or (iii) i.p. carboplatin plus i.v./i.p. paclitaxel. The primary end point was 9-month progressive disease rate (PD9). Secondary end points included progression-free survival (PFS), overall survival (OS), toxicity, and quality of life (QOL).

Results

Between 2009 and 2015, 275 patients were randomized; i.p. cisplatin containing arm did not progress beyond the first stage of the study after failing to meet the pre-set superiority rule. The final analysis compared i.v. carboplatin/paclitaxel (n = 101) with i.p. carboplatin, i.v./i.p. paclitaxel (n = 102). The intention to treat PD9 was lower in the i.p. carboplatin arm compared with the i.v. carboplatin arm: 24.5% (95% CI 16.2% to 32.9%) versus 38.6% (95% CI 29.1% to 48.1%) P = 0.065. The study was underpowered to detect differences in PFS: HR PFS 0.82 (95% CI 0.57-1.17); P = 0.27 and OS HR 0.80 (95% CI 0.47-1.35) P = 0.40. The i.p. carboplatin-based regimen was well tolerated with no reduction in QOL or increase in toxicity compared with i.v. administration alone.

Conclusion

In women with stage IIIC or IVA EOC treated with NACT and optimal debulking surgery, i.p. carboplatin-based chemotherapy is well tolerated and associated with an improved PD9 compared with i.v. carboplatin-based chemotherapy.

Clinical trial number

clinicaltrials.gov, NCT01622543.

OV21/PETROC: a randomized Gynecologic Cancer Intergroup phase II study of intraperitoneal versus intravenous chemotherapy following neoadjuvant chemotherapy and optimal debulking surgery in epithelial ovarian cancer

Background

The purpose of this multistage, adaptively, designed randomized phase II study was to evaluate the role of intraperitoneal (i.p.) chemotherapy following neoadjuvant chemotherapy (NACT) and optimal debulking surgery in women with epithelial ovarian cancer (EOC).

Patients and methods

We carried out a multicenter, two-stage, phase II trial. Eligible patients with stage IIB-IVA EOC treated with platinum-based intravenous (i.v.) NACT followed by optimal (<1 cm) debulking surgery were randomized to one of the three treatment arms: (i) i.v. carboplatin/paclitaxel, (ii) i.p. cisplatin plus i.v./i.p. paclitaxel, or (iii) i.p. carboplatin plus i.v./i.p. paclitaxel. The primary end point was 9-month progressive disease rate (PD9). Secondary end points included progression-free survival (PFS), overall survival (OS), toxicity, and quality of life (QOL).

Results

Between 2009 and 2015, 275 patients were randomized; i.p. cisplatin containing arm did not progress beyond the first stage of the study after failing to meet the pre-set superiority rule. The final analysis compared i.v. carboplatin/paclitaxel (n = 101) with i.p. carboplatin, i.v./i.p. paclitaxel (n = 102). The intention to treat PD9 was lower in the i.p. carboplatin arm compared with the i.v. carboplatin arm: 24.5% (95% CI 16.2% to 32.9%) versus 38.6% (95% CI 29.1% to 48.1%) P = 0.065. The study was underpowered to detect differences in PFS: HR PFS 0.82 (95% CI 0.57-1.17); P = 0.27 and OS HR 0.80 (95% CI 0.47-1.35) P = 0.40. The i.p. carboplatin-based regimen was well tolerated with no reduction in QOL or increase in toxicity compared with i.v. administration alone.

Conclusion

In women with stage IIIC or IVA EOC treated with NACT and optimal debulking surgery, i.p. carboplatin-based chemotherapy is well tolerated and associated with an improved PD9 compared with i.v. carboplatin-based chemotherapy.

Clinical trial number

clinicaltrials.gov, NCT01622543.