Multivariate disease progression modeling with longitudinal ordinal data

Disease modeling is an essential tool to describe disease progression and its heterogeneity across patients. Usual approaches use continuous data such as biomarkers to assess progression. Nevertheless, categorical or ordinal data such as item responses in questionnaires also provide insightful information about disease progression. In this work, we propose a disease progression model for ordinal and categorical data. We built it on the principles of disease course mapping, a technique that uniquely describes the variability in both the dynamics of progression and disease heterogeneity from multivariate longitudinal data. This extension can also be seen as an attempt to bridge the gap between longitudinal multivariate models and the field of item response theory. Application to the Parkinson's progression markers initiative cohort illustrates the benefits of our approach: a fine-grained description of disease progression at the item level, as compared to the aggregated total score, together with improved predictions of the patient's future visits. The analysis of the heterogeneity across individual trajectories highlights known disease trends such as tremor dominant or postural instability and gait difficulties subtypes of Parkinson's disease.

Partial Identification of Latent Correlations with Ordinal Data

The polychoric correlation is a popular measure of association for ordinal data. It estimates a latent correlation, i.e., the correlation of a latent vector. This vector is assumed to be bivariate normal, an assumption that cannot always be justified. When bivariate normality does not hold, the polychoric correlation will not necessarily approximate the true latent correlation, even when the observed variables have many categories. We calculate the sets of possible values of the latent correlation when latent bivariate normality is not necessarily true, but at least the latent marginals are known. The resulting sets are called partial identification sets, and are shown to shrink to the true latent correlation as the number of categories increase. Moreover, we investigate partial identification under the additional assumption that the latent copula is symmetric, and calculate the partial identification set when one variable is ordinal and another is continuous. We show that little can be said about latent correlations, unless we have impractically many categories or we know a great deal about the distribution of the latent vector. An open-source R package is available for applying our results.

Bayesian analysis of Ecological Momentary Assessment (EMA) data collected in adults before and after hearing rehabilitation

This paper presents a new Bayesian method for analyzing Ecological Momentary Assessment (EMA) data and applies this method in a re-analysis of data from a previous EMA study. The analysis method has been implemented as a freely available Python package EmaCalc, RRID:SCR 022943. The analysis model can use EMA input data including nominal categories in one or more situation dimensions, and ordinal ratings of several perceptual attributes. The analysis uses a variant of ordinal regression to estimate the statistical relation between these variables. The Bayesian method has no requirements related to the number of participants or the number of assessments by each participant. Instead, the method automatically includes measures of the statistical credibility of all analysis results, for the given amount of data. For the previously collected EMA data, the analysis results demonstrate how the new tool can handle heavily skewed, scarce, and clustered data that were collected on ordinal scales, and present results on interval scales. The new method revealed results for the population mean that were similar to those obtained in the previous analysis by an advanced regression model. The Bayesian approach automatically estimated the inter-individual variability in the population, based on the study sample, and could show some statistically credible intervention results also for an unseen random individual in the population. Such results may be interesting, for example, if the EMA methodology is used by a hearing-aid manufacturer in a study to predict the success of a new signal-processing method among future potential customers.

Analysis of the Weighted Kappa and Its Maximum with Markov Moves

In this paper, the notion of Markov move from algebraic statistics is used to analyze the weighted kappa indices in rater agreement problems. In particular, the problem of the maximum kappa and its dependence on the choice of the weighting schemes are discussed. The Markov moves are also used in a simulated annealing algorithm to actually find the configuration of maximum agreement.

Comparing healthcare quality: A common framework for both ordinal and cardinal data with an application to primary care variation in England

The paper proposes a framework for comparing the quality of healthcare providers and assessing the variation in quality between them, which is directly applicable to both ordinal and cardinal quality data on a comparable basis. The resultant measures are sensitive to the full distribution of quality scores for each provider, not just the mean or the proportion meeting some binary quality threshold, thereby making full use of the multicategory response data increasingly available from patient experience surveys. The measures can also be standardized for factors such as age, sex, ethnicity, health and deprivation using a distribution regression model. We illustrate by measuring the quality of primary care services in England in 2019 using three different sources of publicly available, general practice-level information: multicategory response patient experience data, ordinal inspection ratings and cardinal clinical achievement scores. We find considerable variation at both local and regional levels using all three data sources. However, the correlation between the comparative quality indices calculated using the alternative data sources is weak, suggesting that they capture different aspects of general practice quality.

Robust estimates of regional treatment effects in multiregional randomized clinical trials with ordinal responses

Global clinical trials involving multiple regions are common in current drug development processes. Determining the regional treatment effects of a new therapy over an existing therapy is important to both the sponsors and the regulatory agencies in the regions. Existing methods are mainly for continuous primary endpoints and use subjectively specified models, which may deviate from the true model. Here, we consider trials that have ordinal responses as the primary endpoint. This article extends the recently developed robust semiparametric ordinal regression model to estimate regional treatment effects, in which the regression coefficients and regional effects are modeled parametrically for ease of interpretation, and the regression link function is specified nonparametrically for robustness. The model parameters are estimated by semiparametric maximum likelihood estimation, and the null hypothesis of no regional effect is tested by the Wald test. Simulation studies are conducted to evaluate the performance of the proposed method and compare it with the commonly used parametric model. The results of the former show an improved overall performance over the latter. In particular, the model yields much higher precision in estimation and prediction than the fixed-link model. This result is especially appealing since our interest is to estimate the treatment effect more efficiently and the estimand is of particular interest in multiregional clinical trials. We then apply the method by analyzing real multiregional clinical trials with ordinal responses as their primary endpoint.

Factor Retention Using Machine Learning With Ordinal Data

Determining the number of factors in exploratory factor analysis is probably the most crucial decision when conducting the analysis as it clearly influences the meaningfulness of the results (i.e., factorial validity). A new method called the Factor Forest that combines data simulation and machine learning has been developed recently. This method based on simulated data reached very high accuracy for multivariate normal data, but it has not yet been tested with ordinal data. Hence, in this simulation study, we evaluated the Factor Forest with ordinal data based on different numbers of categories (2-6 categories) and compared it to common factor retention criteria. It showed higher overall accuracy for all types of ordinal data than all common factor retention criteria that were used for comparison (Parallel Analysis, Comparison Data, the Empirical Kaiser Criterion and the Kaiser Guttman Rule). The results indicate that the Factor Forest is applicable to ordinal data with at least five categories (typical scale in questionnaire research) in the majority of conditions and to binary or ordinal data based on items with less categories when the sample size is large.

A Critical Appraisal of the Evidence Supporting the Factor Structure of Extant Coping Instruments

This paper evaluated the evidence supporting the factor structure of extant coping instruments based on modern psychometric standards. Our literature search identified nine coping instruments that are routinely used to measure coping strategies in adult populations. While nearly 10 thousand papers have been published using these instruments, only 39 studies have investigated their psychometric validity. Our findings revealed that the majority of these studies did not follow current psychometric recommendations for establishing internal validity in part because they did not account for the ordinal nature of the data. Further, studies employing exploratory factor analysis used methods for identifying the number of factors to retain that have been found to have a low accuracy in a simulation study while those employing confirmatory factor analysis reported model fit statistics that did not meet widely accepted benchmarks. Hence, conflicting results were found within and across the nine coping instruments. Recommendations are made for improving future validation studies.

The importance of using ordinal scores for patient classification based on health-related quality of life trajectories

Changes in health-related quality of life (HRQoL) over time are not necessarily homogeneous within a population of interest. Our study aim was twofold: to determine homogeneous patient subpopulations distinguished by HRQoL trajectories, and to identify the particular patient profile associated with each subpopulation. To classify patients according to HRQoL dimension scores, we compared mixtures of linear mixed models (LMMs) classically applied to scores defined by the EORTC procedure, and mixtures of random effect cumulative models (CMs) applied to scores treated as ordinal variables. A simulation study showed that the mixture of LMMs overestimated the number of subpopulations and was less able to correctly classify patients than the mixture of CMs. Considering HRQoL scores as ordinal rather than continuous variables is relevant when classifying patients. The mixture of CMs for ordinal scores is able to identify homogeneous subpopulations and their associated trajectories. The application focused on changes over time in HRQoL data (collected using the EORTC QLQ-C30 questionnaire) from 132 breast cancer patients from the Moral study. Once the classification is obtained only from HRQoL scores, class membership was then explained through a logistic regression model, given a large panel of variables collected at baseline. Analysis of data revealed that deterioration over time of role functioning and insomnia was closely related to patient anxiety: anxiety at baseline is a prognostic factor for a poor level and/or a deterioration over time of HRQoL. For functional dimensions, large tumor size and high education level were associated with worse HRQoL scores.

Estimating Finite Mixtures of Ordinal Graphical Models

Graphical models have received an increasing amount of attention in network psychometrics as a promising probabilistic approach to study the conditional relations among variables using graph theory. Despite recent advances, existing methods on graphical models usually assume a homogeneous population and focus on binary or continuous variables. However, ordinal variables are very popular in many areas of psychological science, and the population often consists of several different groups based on the heterogeneity in ordinal data. Driven by these needs, we introduce the finite mixture of ordinal graphical models to effectively study the heterogeneous conditional dependence relationships of ordinal data. We develop a penalized likelihood approach for model estimation, and design a generalized expectation-maximization (EM) algorithm to solve the significant computational challenges. We examine the performance of the proposed method and algorithm in simulation studies. Moreover, we demonstrate the potential usefulness of the proposed method in psychological science through a real application concerning the interests and attitudes related to fan avidity for students in a large public university in the United States.

Intuitionistic Fuzzy Synthetic Measure on the Basis of Survey Responses and Aggregated Ordinal Data

The paper addresses the problem of complex socio-economic phenomena assessment using questionnaire surveys. The data are represented on an ordinal scale; the object assessments may contain positive, negative, no answers, a “difficult to say” or “no opinion” answers. The general framework for Intuitionistic Fuzzy Synthetic Measure (IFSM) based on distances to the pattern object (ideal solution) is used to analyze the survey data. First, Euclidean and Hamming distances are applied in the procedure. Second, two pattern object constructions are proposed in the procedure: one based on maximum values from the survey data, and the second on maximum intuitionistic values. Third, the method for criteria comparison with the Intuitionistic Fuzzy Synthetic Measure is presented. Finally, a case study solving the problem of rank-ordering of the cities in terms of satisfaction from local public administration obtained using different variants of the proposed method is discussed. Additionally, the comparative analysis results using the Intuitionistic Fuzzy Synthetic Measure and the Intuitionistic Fuzzy TOPSIS (IFT) framework are presented.

Bayesian approaches to the weighted kappa-like inter-rater agreement measures

Inter-rater agreement measures are used to estimate the degree of agreement between two or more assessors. When the agreement table is ordinal, different weight functions that incorporate row and column scores are used along with the agreement measures. The selection of row and column scores is effectual on the estimated degree of agreement. The weighted measures are prone to the anomalies frequently seen in agreement tables such as unbalanced table structures or grey zones due to the assessment behaviour of the raters. In this study, Bayesian approaches for the estimation of inter-rater agreement measures are proposed. The Bayesian approaches make it possible to include prior information on the assessment behaviour of the raters in the analysis and impose order restrictions on the row and column scores. In this way, we improve the accuracy of the agreement measures and mitigate the impact of the anomalies in the estimation of the strength of agreement between the raters. The elicitation of prior distributions is described theoretically and practically for the Bayesian estimation of five agreement measures with three different weights using an agreement table having two grey zones. A Monte Carlo simulation study is conducted to assess the classification accuracy of the Bayesian and classical approaches for the considered agreement measures for a given level of agreement. Recommendations for the selection of the highest performing agreement measure and weight combination are made in the breakdown of the table structure and sample size.

Ordinal SuStaIn: Subtype and Stage Inference for Clinical Scores, Visual Ratings, and Other Ordinal Data

Subtype and Stage Inference (SuStaIn) is an unsupervised learning algorithm that uniquely enables the identification of subgroups of individuals with distinct pseudo-temporal disease progression patterns from cross-sectional datasets. SuStaIn has been used to identify data-driven subgroups and perform patient stratification in neurodegenerative diseases and in lung diseases from continuous biomarker measurements predominantly obtained from imaging. However, the SuStaIn algorithm is not currently applicable to discrete ordinal data, such as visual ratings of images, neuropathological ratings, and clinical and neuropsychological test scores, restricting the applicability of SuStaIn to a narrower range of settings. Here we propose 'Ordinal SuStaIn', an ordinal version of the SuStaIn algorithm that uses a scored events model of disease progression to enable the application of SuStaIn to ordinal data. We demonstrate the validity of Ordinal SuStaIn by benchmarking the performance of the algorithm on simulated data. We further demonstrate that Ordinal SuStaIn out-performs the existing continuous version of SuStaIn (Z-score SuStaIn) on discrete scored data, providing much more accurate subtype progression patterns, better subtyping and staging of individuals, and accurate uncertainty estimates. We then apply Ordinal SuStaIn to six different sub-scales of the Clinical Dementia Rating scale (CDR) using data from the Alzheimer's disease Neuroimaging Initiative (ADNI) study to identify individuals with distinct patterns of functional decline. Using data from 819 ADNI1 participants we identified three distinct CDR subtype progression patterns, which were independently verified using data from 790 ADNI2 participants. Our results provide insight into patterns of decline in daily activities in Alzheimer's disease and a mechanism for stratifying individuals into groups with difficulties in different domains. Ordinal SuStaIn is broadly applicable across different types of ratings data, including visual ratings from imaging, neuropathological ratings and clinical or behavioural ratings data.

Latent trait models for perceived risk assessment using a Covid-19 data survey

Aim of the contribution is analyzing potential events that may negatively impact individuals, assets, and/or the environment, and making judgments about the perceived personal and social riskiness of Covid-19 compared to other hazards belonging to health (AIDS, cancer, infarction), environmental (climate change), behavioral (serious car accidents), and technological (nuclear weapons) domains. The comparative risk analysis has been performed on a survey data collected during the first Italian Covid-19 lockdown. An item response theory model for polytomously scored items has been implemented for the analysis of the positioning of Covid-19 with respect to the other hazards in terms of perceived risk. Among the attributes determining the hazard's perceived risk, Covid-19 distinguishes for the knowledge of risks from the hazard, media attention, and fear caused by the hazard in the peers. Besides, through a latent regression analysis, the role of some individual characteristics on the perceived risk for Covid-19 has been examined. Our contribution allows us to disentangle among several aspects of hazards and describe the main factors affecting the perceived risk. It also contributes to determine if existing control measures are perceived as adequate and the interest for new media with related impact on a person's reaction.

Sinonasal Outcome Scores and Imaging: A Concurrent Assessment of Factors Influencing Their Association

OBJECTIVE

The overall discriminatory ability of validated instrument scores for computed tomography (CT) findings of chronic rhinosinusitis has limitations and may be modified by multiple factors. To support optimal methods for assessment, we studied which factors could influence this relationship, including the concurrent impact of multiple discrete CT scoring mechanisms, colocalized imaging findings, and nasal comorbid conditions.

STUDY DESIGN

Observational outcomes study.

SETTING

Academic medical center.

METHODS

Patients with sinonasal complaints who completed the 22-item Sinonasal Outcome Test (SNOT-22) and underwent CT were included. Multivariate ordinal regression was utilized to assess associations. CT data were quantified with the Lund-Mackay system, Zinreich system, and a direct measure of maximal mucosal thickness. The impact of incidental findings (mucous retention cysts, periapical dental disease) and nasal comorbid conditions was also assessed.

RESULTS

A total of 233 patients were included. SNOT-22 nasal scores were significantly associated with CT results when those with incidental findings were excluded, regardless of the radiologic scoring mechanism utilized: Lund-Mackay regression coefficient, 0.321 (P = .046); Zinreich, 0.340 (P = .033); and maximum mucosal thickness, 0.316 (P = .040). This relationship subsided when incidental findings were present. SNOT-22 overall scores, sleep scores, and psychological domain scores had no significant association with imaging results, regardless of radiologic scoring system utilized. Nasal comorbid conditions had inconsistent associations.

CONCLUSIONS

SNOT-22 nasal domain scores were associated with all 3 radiologic scoring systems when incidental findings were absent but not when they were present. Delineating the presence or absence of these colocalized findings affected the relationship between SNOT-22 scores and radiological results, beyond other concurrent factors.

Uncertainty in Latent Trait Models

A model that extends the Rasch model and the Partial Credit Model to account for subject-specific uncertainty when responding to items is proposed. It is demonstrated that ignoring the subject-specific uncertainty may yield biased estimates of model parameters. In the extended version of the model, uncertainty and the underlying trait are linked to explanatory variables. The parameterization allows to identify subgroups that differ in uncertainty and the underlying trait. The modeling approach is illustrated using data on the confidence of citizens in public institutions.

Upper bound estimators of the population size based on ordinal models for capture-recapture experiments

Capture-recapture studies have attracted a lot of attention over the past few decades, especially in applied disciplines where a direct estimate for the size of a population of interest is not available. Epidemiology, ecology, public health, and biodiversity are just a few examples. The estimation of the number of unseen units has been a challenge for theoretical statisticians, and considerable progress has been made in providing lower bound estimators for the population size. In fact, it is well known that consistent estimators for this cannot be provided in the very general case. Considering a case where capture-recapture studies are summarized by a frequency of frequencies distribution, we derive a simple upper bound of the population size based on the cumulative distribution function. We introduce two estimators of this bound, without any specific parametric assumption on the distribution of the observed frequency counts. The behavior of the proposed estimators is investigated using several benchmark datasets and a large-scale simulation experiment based on the scheme discussed by Pledger.

Factors affecting injury severity of single-vehicle rollover crashes in the United States

Objective: This study investigated the confounding effects of factors on injury outcomes for all occupants in fatal single-vehicle crashes that involved a rollover event.Method: A generalized ordered logit model was used to investigate the role of roadway attributes, environmental factors, driver characteristics, and vehicle features in injury severity outcomes for occupants. Five years of single-vehicle rollover crash data for the United States were studied.Results: Results showed that the likelihood of serious and fatal injuries increases in rollover crashes with partial or complete ejection of the occupant, no seat belt use, speeding, higher posted speed limits, roadside and median rollovers, undulating terrain, blacktop road surface, and rural roads. We also found that evening, weekdays, previous driver crash, careless or inattentive driving, driver-passenger engagement, aggressive driving, and vehicle type affect injury severity. The deployment of airbags was associated with fewer serious and fatal injuries. Regional differences were found for injury severity outcomes in rollover crashes.Conclusions: The study provides valuable insight for reducing injury severity in single-vehicle crashes where a rollover occurs. Several proven countermeasures may prevent rollovers or reduce injury severity. These strategies include increasing seatbelt use, posting lower speed limits and installing speed enforcement cameras in high-risk areas, flattening roadside embankments, and promoting in-vehicle stability enhancement systems such as electronic stability control and rollover-activated side curtain airbags.

On Identification and Non-normal Simulation in Ordinal Covariance and Item Response Models

A standard approach for handling ordinal data in covariance analysis such as structural equation modeling is to assume that the data were produced by discretizing a multivariate normal vector. Recently, concern has been raised that this approach may be less robust to violation of the normality assumption than previously reported. We propose a new perspective for studying the robustness toward distributional misspecification in ordinal models using a class of non-normal ordinal covariance models. We show how to simulate data from such models, and our simulation results indicate that standard methodology is sensitive to violation of normality. This emphasizes the importance of testing distributional assumptions in empirical studies. We include simulation results on the performance of such tests.

A method for ordinal outcomes: The ordered stereotype model

OBJECTIVE

The collection and use of ordinal variables are common in many psychological and psychiatric studies. Although the models for continuous variables have similarities to those for ordinal variables, there are advantages when a model developed for modeling ordinal data is used such as avoiding "floor" and "ceiling" effects and avoiding to assign scores, as it happens in continuous models, which can produce results sensitive to the score assigned. This paper introduces and focuses on the application of the ordered stereotype model, which was developed for modeling ordinal outcomes and is not so popular as other models such as linear regression and proportional odds models. This paper aims to compare the performance of the ordered stereotype model with other more commonly used models among researchers and practitioners.

METHODS

This article compares the performance of the stereotype model against the proportional odd and linear regression models, with three, four, and five levels of ordinal categories and sample sizes 100, 500, and 1000. This paper also discusses the problem of treating ordinal responses as continuous using a simulation study. The trend odds model is also presented in the application.

RESULTS

Three types of models were fitted in one real-life example, including ordered stereotype, proportional odds, and trend odds models. They reached similar conclusions in terms of the significance of covariates. The simulation study evaluated the performance of the ordered stereotype model under four cases. The performance varies depending on the scenarios.

CONCLUSIONS

The method presented can be applied to several areas of psychiatry dealing with ordinal outcomes. One of the main advantages of this model is that it breaks with the assumption of levels of the ordinal response are equally spaced, which might be not true.

A Measurement Is a Choice and Stevens' Scales of Measurement Do Not Help Make It: A Response to Chalmers

Chalmers recently published a critique of the use of ordinal α proposed in Zumbo et al. as a measure of test reliability in certain research settings. In this response, we take up the task of refuting Chalmers' critique. We identify three broad misconceptions that characterize Chalmers' criticisms: (1) confusing assumptions with consequences of mathematical models, and confusing both with definitions, (2) confusion about the definitions and relevance of Stevens' scales of measurement, and (3) a failure to recognize that a measurement for a true quantity is a choice, not an absolute. On dissection of these misconceptions, we argue that Chalmers' critique of ordinal α is unfounded.

Model-based goodness-of-fit tests for the ordered stereotype model

This paper presents two new model-based goodness-of-fit tests for the ordered stereotype model applied to an ordinal response variable. The proposed tests are based on the Lipsitz test, which partitions the subjects into G groups following the popular Hosmer-Lemeshow test for binary data. The tests construct an alternative model where group effects are added into the null model. If the model fits the data well then the null model is correct, and there should be no group effects. One of the main advantages of the ordered stereotype model is that it allows us to determine a new uneven spacing of the ordinal response categories, dictated by the data. The two proposed tests use this new adjusted spacing. One test uses the form of the original ordered stereotype model, and the other uses an ordinary linear model. We demonstrate the performance of both tests under a variety of scenarios. Finally, the results of the application in three examples are presented.

Fitting Large Factor Analysis Models With Ordinal Data

A simulation study was conducted to investigate the model size effect when confirmatory factor analysis (CFA) models include many ordinal items. CFA models including between 15 and 120 ordinal items were analyzed with mean- and variance-adjusted weighted least squares to determine how varying sample size, number of ordered categories, and misspecification affect parameter estimates, standard errors of parameter estimates, and selected fit indices. As the number of items increased, the number of admissible solutions and accuracy of parameter estimates improved, even when models were misspecified. Also, standard errors of parameter estimates were closer to empirical standard deviation values as the number of items increased. When evaluating goodness-of-fit for ordinal CFA with many observed indicators, researchers should be cautious in interpreting the root mean square error of approximation, as this value appeared overly optimistic under misspecified conditions.

A Problem with Discretizing Vale-Maurelli in Simulation Studies

Previous influential simulation studies investigate the effect of underlying non-normality in ordinal data using the Vale-Maurelli (VM) simulation method. We show that discretized data stemming from the VM method with a prescribed target covariance matrix are usually numerically equal to data stemming from discretizing a multivariate normal vector. This normal vector has, however, a different covariance matrix than the target. It follows that these simulation studies have in fact studied data stemming from normal data with a possibly misspecified covariance structure. This observation affects the interpretation of previous simulation studies.

Validity of a questionnaire developed to measure the impact of a high-fidelity simulation intervention: A feasibility study

AIM

To evaluate the validity and responsiveness of a questionnaire developed to measure the impact of a high-fidelity simulation intervention.

DESIGN

A pre- and postintervention design.

METHODS

In August 2017, 107 participants completed a questionnaire measuring knowledge and perceived self-confidence pre- and postintervention. Validity of the questionnaire was determined by expert reviews, individual interviews and estimates of the changes in knowledge and perceived self-confidence. The changes were estimated by the differences between paired proportions of participants. The responsiveness of the ordered categorical item scores on self-confidence was evaluated by the measure of systematic group change and individual variations.

RESULTS

The analysis of the interviews resulted in three themes: item content, item style and the administration of the questionnaire. An intervention effect on knowledge assessments was shown by the changes in paired proportions of participants with increased or decreased correct assessments (ranging from -25.5 - 24.8 percentage units). The responsiveness of the self-confidence scale was confirmed by evidence of post-intervention systematic group changes towards higher levels.

CONCLUSION

This study provides useful experience for a forthcoming randomized controlled study to evaluate the effect of high-fidelity simulation on undergraduate nursing students' knowledge and self-confidence when assessing patient deterioration.

IMPACT

Cause-and-effect relationship between simulation and learning is required to improve nursing education. A statistically significant rise in students' knowledge and levels of self-confidence after simulation were identified in this study. The study provided important aspects of future research study designs.

Fruit production is influenced by tree size and size-asymmetric crowding in a wet tropical forest

In tropical forest communities, seedling recruitment can be limited by the number of fruit produced by adults. Fruit production tends to be highly unequal among trees of the same species, which may be due to environmental factors. We observed fruit production for ~2,000 trees of 17 species across 3 years in a wet tropical forest in Costa Rica. Fruit production was modeled as a function of tree size, nutrient availability, and neighborhood crowding. Following model selection, tree size and neighborhood crowding predicted both the probability of reproduction and the number of fruit produced. Nutrient availability only predicted only the probability of reproduction. In all species, larger trees were more likely to be reproductive and produce more fruit. In addition, number of fruit was strongly negatively related to presence of larger neighboring trees in 13 species; presence of all neighboring trees had a weak-to-moderate negative influence on reproductive status in 16 species. Among various metrics of soil nutrient availability, only sum of base cations was positively associated with reproductive status, and for only four species. Synthesis Overall, these results suggest that direct influences on fruit production tend to be mediated through tree size and crowding from neighboring trees, rather than soil nutrients. However, we found variation in the effects of neighbors and nutrients among species; mechanistic studies of allocation to fruit production are needed to explain these differences.

Response Styles in the Partial Credit Model

In the modeling of ordinal responses in psychological measurement and survey-based research, response styles that represent specific answering patterns of respondents are typically ignored. One consequence is that estimates of item parameters can be poor and considerably biased. The focus here is on the modeling of a tendency to extreme or middle categories. An extension of the partial credit model is proposed that explicitly accounts for this specific response style. In contrast to existing approaches, which are based on finite mixtures, explicit person-specific response style parameters are introduced. The resulting model can be estimated within the framework of generalized mixed linear models. It is shown that estimates can be seriously biased if the response style is ignored. In applications, it is demonstrated that a tendency to extreme or middle categories is not uncommon. A software tool is developed that makes the model easy to apply.

Selecting polychoric instrumental variables in confirmatory factor analysis: An alternative specification test and effects of instrumental variables

The polychoric instrumental variable (PIV) approach is a recently proposed method to fit a confirmatory factor analysis model with ordinal data. In this paper, we first examine the small-sample properties of the specification tests for testing the validity of instrumental variables (IVs). Second, we investigate the effects of using different numbers of IVs. Our results show that specification tests derived for continuous data are extremely oversized at all sample sizes when applied to ordinal variables. Possible modifications for ordinal data are proposed in the present study. Simulation results show that the modified specification tests with all available IVs are able to detect model misspecification. In terms of estimation accuracy, the PIV approach where the IVs outnumber the endogenous variables by one produces a lower bias but a higher variation than the PIV approach with more IVs for correctly specified factor loadings at small samples.

Superior outcomes following cervical fusion vs. multimodal rehabilitation in a subgroup of randomized Whiplash-Associated-Disorders (WAD) patients indicating somatic pain origin-Comparison of outcome assessments made by four examiners from different disciplines

BACKGROUND AND AIMS

Whiplash-Associated Disorders (WAD) are characterized by great variability in long-term symptoms. Patients with central neck and movement-induced stabbing pain participated in a randomized study comparing cervical fusion and multimodal rehabilitation. As reported in our previous paper, more patients treated by cervical fusion than by rehabilitation experienced pain relief. Although patient reported outcome measures are a core component of outcome evaluation, independent examiner has been recommended. Because of the heterogeneity of WAD complaints the patients in our study were examined at baseline and follow-up by four experts representing neurology, orthopedics, psychology and physical medicine. The aim was to compare the professional assessments of change both regarding the possible impact of the different examiners' perspectives on individual patient's outcome, and also on the analysis of possible outcome differences between the treatment groups.

METHODS

WAD patients with long-term neck pain as the predominant symptom after a traffic accident were eligible. The neck pain origin should be in the midline and perceived as dull and aching, with sudden movement inducing midline stabbing pain. Of the 1,052 patients in contact with our team, 49 were eligible. The overall treatment effect was evaluated on a global outcome transitional scale. The criteria for the scale categories were defined by each expert's professional perspective on change in the whiplash complaints. Statistical methods that take account of the non-metric properties of ordered categorical data were used. Observed inter-expert disagreement was evaluated by the Svensson method that identifies and measures systematic group-related disagreement separately from disagreement caused by individual variation. Possible differences in the distributions of assessments on the expert-specific outcome scales between the treatment groups were analyzed by the Kruskal-Wallis test.

RESULTS

The per-protocol evaluation showed that a majority of the 18 patients who underwent fusion surgery were assessed as somewhat or much better, ranging from 67% to 78% depending on the expert. Corresponding proportions of improvement in the 17 patients treated by multimodal rehabilitation ranged from 29% to 53%. The statistical analyses confirmed better outcomes in the patients treated by fusion surgery, with p-values ranging from 0.003 to 0.04. The experts' assessments of intra-patient change disagreed more or less for all patients. The analyses of the paired comparisons confirmed that these disagreements could most probably be explained by the different profession-specific operational definitions of the outcome scales rather than by individual variations in data.

CONCLUSIONS

The multi-dimensional complexity of WAD-related complaints was comprehensively demonstrated by the inter-disciplinary disagreements in assessing intra-patient outcomes. The superiority of positive treatment effects in patients who underwent cervical fusion compared with multimodal rehabilitation was evident to all experts.

IMPLICATIONS

The results strengthen our previous opinion that neck pain in this subgroup of WAD patients has a somatic origin. More than one examiner is recommended for multi-dimensional outcome assessments.

Testing Equality of Treatments under an Incomplete Block Crossover Design with Ordinal Responses

The generalized odds ratio (GOR) for paired sample is considered to measure the relative treatment effect on patient responses in ordinal data. Under a three-treatment two-period incomplete block crossover design, both asymptotic and exact procedures are developed for testing equality between treatments with ordinal responses. Monte Carlo simulation is employed to evaluate and compare the finite-sample performance of these test procedures. A discussion on advantages and disadvantages of the proposed test procedures based on the GOR versus those based on Wald's tests under the normal random effects proportional odds model is provided. The data taken as a part of a crossover trial studying the effects of low and high doses of an analgesic versus a placebo for the relief of pain in primary dysmenorrhea over the first two periods are applied to illustrate the use of these test procedures.

Correlated probit analysis of repeatedly measured ordinal and continuous outcomes with application to the Health and Retirement Study

The Health and Retirement Study was designed to evaluate changes in health and labor force participation during and after the transition from working to retirement. Every 2 years, participants provided information about their self-rated health (SRH), body mass index (BMI), smoking status, and other characteristics. Our goal was to assess the effects of smoking and gender on trajectories of change in BMI and SRH over time. Joint longitudinal analysis of outcome measures is preferable to separate analyses because it allows to account for the correlation between the measures, to test the effects of predictors while controlling type I error, and potentially to improve efficiency. However, because SRH is an ordinal measure while BMI is continuous, formulating a joint model and parameter estimation is challenging. A joint correlated probit model allowed us to seamlessly account for the correlations between the measures over time. Established estimating procedures for such models are based on quasi-likelihood or numerical approximations that may be biased or fail to converge. Therefore, we proposed a novel expectation-maximization algorithm for parameter estimation and a Monte Carlo bootstrap approach for standard errors approximation. Expectation-maximization algorithms have been previously considered for combinations of binary and/or continuous repeated measures; however, modifications were needed to handle combinations of ordinal and continuous responses. A simulation study demonstrated that the algorithm converged and provided approximately unbiased estimates with sufficiently large sample sizes. In the Health and Retirement Study, male gender and smoking were independently associated with steeper deterioration in self-rated health and with lower average BMI. Copyright © 2016 John Wiley & Sons, Ltd.

OpenMx 2.0: Extended Structural Equation and Statistical Modeling

The new software package OpenMx 2.0 for structural equation and other statistical modeling is introduced and its features are described. OpenMx is evolving in a modular direction and now allows a mix-and-match computational approach that separates model expectations from fit functions and optimizers. Major backend architectural improvements include a move to swappable open-source optimizers such as the newly written CSOLNP. Entire new methodologies such as item factor analysis and state space modeling have been implemented. New model expectation functions including support for the expression of models in LISREL syntax and a simplified multigroup expectation function are available. Ease-of-use improvements include helper functions to standardize model parameters and compute their Jacobian-based standard errors, access to model components through standard R $ mechanisms, and improved tab completion from within the R Graphical User Interface.

Numeric score-based conditional and overall change-in-status indices for ordered categorical data

Planned interventions and/or natural conditions often effect change on an ordinal categorical outcome (e.g., symptom severity). In such scenarios, it is sometimes desirable to assign a priori scores to observed changes in status, typically giving higher weight to changes of greater magnitude. We define change indices for such data based upon a multinomial model for each row of a c × c table, where the rows represent the baseline status categories. We distinguish an index designed to assess conditional changes within each baseline category from two others designed to capture overall change. One of these overall indices measures expected change across a target population. The other is scaled to capture the proportion of total possible change in the direction indicated by the data, so that it ranges from -1 (when all subjects finish in the least favorable category) to +1 (when all finish in the most favorable category). The conditional assessment of change can be informative regardless of how subjects are sampled into the baseline categories. In contrast, the overall indices become relevant when subjects are randomly sampled at baseline from the target population of interest, or when the investigator is able to make certain assumptions about the baseline status distribution in that population. We use a Dirichlet-multinomial model to obtain Bayesian credible intervals for the conditional change index that exhibit favorable small-sample frequentist properties. Simulation studies illustrate the methods, and we apply them to examples involving changes in ordinal responses for studies of sleep deprivation and activities of daily living.

A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables

This article compares a variety of imputation strategies for ordinal missing data on Likert scale variables (number of categories = 2, 3, 5, or 7) in recovering reliability coefficients, mean scale scores, and regression coefficients of predicting one scale score from another. The examined strategies include imputing using normal data models with naïve rounding/without rounding, using latent variable models, and using categorical data models such as discriminant analysis and binary logistic regression (for dichotomous data only), multinomial and proportional odds logistic regression (for polytomous data only). The result suggests that both the normal model approach without rounding and the latent variable model approach perform well for either dichotomous or polytomous data regardless of sample size, missing data proportion, and asymmetry of item distributions. The discriminant analysis approach also performs well for dichotomous data. Naïvely rounding normal imputations or using logistic regression models to impute ordinal data are not recommended as they can potentially lead to substantial bias in all or some of the parameters.

A novel modeling framework for ordinal data defined by collapsed counts

Adolescent alcohol use is a serious public health concern. Despite advances in the theoretical conceptualization of pathways to alcohol use, researchers are limited by the statistical techniques currently available. Researchers often fit linear models and restrictive categorical models (e.g., proportional odds models) to ordinal data with many response categories defined by collapsed count data (0 drinking days, 1-2 days, 3-6 days, etc.). Consequently, existing models ignore the underlying count process, resulting in disjoint between the construct of interest and the models being fitted. Our proposed ordinal modeling approach overcomes this limitation by explicitly linking ordinal responses to a suitable underlying count distribution. In doing so, researchers can use maximum likelihood estimation to fit count models to ordinal data as if they had directly observed the underlying discrete counts. The usefulness of our ordinal negative binomial and ordinal zero-inflated negative binomial models is verified by simulation studies. We also demonstrate our approach using real empirical data from the 2010 National Survey of Drug Use and Health. Results show the benefit of the proposed ordinal modeling framework compared with existing methods.

Testing equality and interval estimation of the generalized odds ratio in ordinal data under a three-period crossover design

The crossover design can be of use to save the number of patients or improve power of a parallel groups design in studying treatments to noncurable chronic diseases. We propose using the generalized odds ratio for paired sample data to measure the relative effects in ordinal data between treatments and between periods. We show that one can apply the commonly used asymptotic and exact test procedures for stratified analysis in epidemiology to test non-equality of treatments in ordinal data, as well as obtain asymptotic and exact interval estimators for the generalized odds ratio under a three-period crossover design. We further show that one can apply procedures for testing the homogeneity of the odds ratio under stratified sampling to examine whether there are treatment-by-period interactions. We use the data taken from a three-period crossover trial studying the effects of low and high doses of an analgesic versus a placebo for the relief of pain in primary dysmenorrhea to illustrate the use of these test procedures and estimators proposed here.

Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation

This research is concerned with two topics in assessing model fit for categorical data analysis. The first topic involves the application of a limited-information overall test, introduced in the item response theory literature, to structural equation modeling (SEM) of categorical outcome variables. Most popular SEM test statistics assess how well the model reproduces estimated polychoric correlations. In contrast, limited-information test statistics assess how well the underlying categorical data are reproduced. Here, the recently introduced C2 statistic of Cai and Monroe (2014) is applied. The second topic concerns how the root mean square error of approximation (RMSEA) fit index can be affected by the number of categories in the outcome variable. This relationship creates challenges for interpreting RMSEA. While the two topics initially appear unrelated, they may conveniently be studied in tandem since RMSEA is based on an overall test statistic, such as C2. The results are illustrated with an empirical application to data from a large-scale educational survey.

Comparison of ordinal and nominal classification trees to predict ordinal expert-based occupational exposure estimates in a case-control study

OBJECTIVES

To evaluate occupational exposures in case-control studies, exposure assessors typically review each job individually to assign exposure estimates. This process lacks transparency and does not provide a mechanism for recreating the decision rules in other studies. In our previous work, nominal (unordered categorical) classification trees (CTs) generally successfully predicted expert-assessed ordinal exposure estimates (i.e. none, low, medium, high) derived from occupational questionnaire responses, but room for improvement remained. Our objective was to determine if using recently developed ordinal CTs would improve the performance of nominal trees in predicting ordinal occupational diesel exhaust exposure estimates in a case-control study.

METHODS

We used one nominal and four ordinal CT methods to predict expert-assessed probability, intensity, and frequency estimates of occupational diesel exhaust exposure (each categorized as none, low, medium, or high) derived from questionnaire responses for the 14983 jobs in the New England Bladder Cancer Study. To replicate the common use of a single tree, we applied each method to a single sample of 70% of the jobs, using 15% to test and 15% to validate each method. To characterize variability in performance, we conducted a resampling analysis that repeated the sample draws 100 times. We evaluated agreement between the tree predictions and expert estimates using Somers' d, which measures differences in terms of ordinal association between predicted and observed scores and can be interpreted similarly to a correlation coefficient.

RESULTS

From the resampling analysis, compared with the nominal tree, an ordinal CT method that used a quadratic misclassification function and controlled tree size based on total misclassification cost had a slightly better predictive performance that was statistically significant for the frequency metric (Somers' d: nominal tree = 0.61; ordinal tree = 0.63) and similar performance for the probability (nominal = 0.65; ordinal = 0.66) and intensity (nominal = 0.65; ordinal = 0.65) metrics. The best ordinal CT predicted fewer cases of large disagreement with the expert assessments (i.e. no exposure predicted for a job with high exposure and vice versa) compared with the nominal tree across all of the exposure metrics. For example, the percent of jobs with expert-assigned high intensity of exposure that the model predicted as no exposure was 29% for the nominal tree and 22% for the best ordinal tree.

CONCLUSIONS

The overall agreements were similar across CT models; however, the use of ordinal models reduced the magnitude of the discrepancy when disagreements occurred. As the best performing model can vary by situation, researchers should consider evaluating multiple CT methods to maximize the predictive performance within their data.

Bounded influence function based inference in joint modelling of ordinal partial linear model and accelerated failure time model

A common objective in longitudinal studies is to characterize the relationship between a longitudinal response process and a time-to-event data. Ordinal nature of the response and possible missing information on covariates add complications to the joint model. In such circumstances, some influential observations often present in the data may upset the analysis. In this paper, a joint model based on ordinal partial mixed model and an accelerated failure time model is used, to account for the repeated ordered response and time-to-event data, respectively. Here, we propose an influence function-based robust estimation method. Monte Carlo expectation maximization method-based algorithm is used for parameter estimation. A detailed simulation study has been done to evaluate the performance of the proposed method. As an application, a data on muscular dystrophy among children is used. Robust estimates are then compared with classical maximum likelihood estimates.

Likelihoods for fixed rank nomination networks

Many studies that gather social network data use survey methods that lead to censored, missing, or otherwise incomplete information. For example, the popular fixed rank nomination (FRN) scheme, often used in studies of schools and businesses, asks study participants to nominate and rank at most a small number of contacts or friends, leaving the existence of other relations uncertain. However, most statistical models are formulated in terms of completely observed binary networks. Statistical analyses of FRN data with such models ignore the censored and ranked nature of the data and could potentially result in misleading statistical inference. To investigate this possibility, we compare Bayesian parameter estimates obtained from a likelihood for complete binary networks with those obtained from likelihoods that are derived from the FRN scheme, and therefore accommodate the ranked and censored nature of the data. We show analytically and via simulation that the binary likelihood can provide misleading inference, particularly for certain model parameters that relate network ties to characteristics of individuals and pairs of individuals. We also compare these different likelihoods in a data analysis of several adolescent social networks. For some of these networks, the parameter estimates from the binary and FRN likelihoods lead to different conclusions, indicating the importance of analyzing FRN data with a method that accounts for the FRN survey design.

Causal inference for bivariate longitudinal quality of life data in presence of death by using global odds ratios

In longitudinal clinical trials, if a subject drops out due to death, certain responses, such as those measuring quality of life (QoL), will not be defined after the time of death. Thus, standard missing data analyses, e.g., under ignorable dropout, are problematic because these approaches implicitly 'impute' values of the response after death. In this paper we define a new survivor average causal effect for a bivariate response in a longitudinal quality of life study that had a high dropout rate with the dropout often due to death (or tumor progression). We show how principal stratification, with a few sensitivity parameters, can be used to draw causal inferences about the joint distribution of these two ordinal quality of life measures.

A Composite Likelihood Inference in Latent Variable Models for Ordinal Longitudinal Responses

The paper proposes a composite likelihood estimation approach that uses bivariate instead of multivariate marginal probabilities for ordinal longitudinal responses using a latent variable model. The model considers time-dependent latent variables and item-specific random effects to be accountable for the interdependencies of the multivariate ordinal items. Time-dependent latent variables are linked with an autoregressive model. Simulation results have shown composite likelihood estimators to have a small amount of bias and mean square error and as such they are feasible alternatives to full maximum likelihood. Model selection criteria developed for composite likelihood estimation are used in the applications. Furthermore, lower-order residuals are used as measures-of-fit for the selected models.

Statistical analysis of the National Institutes of Health peer review system

A statistical model is proposed for the analysis of peer-review ratings of R01 grant applications submitted to the National Institutes of Health. Innovations of this model include parameters that reflect differences in reviewer scoring patterns, a mechanism to account for the transfer of information from an application's preliminary ratings and group discussion to final ratings provided by all panel members and posterior estimates of the uncertainty associated with proposal ratings. Application of this model to recent R01 rating data suggests that statistical adjustments to panel rating data would lead to a 25% change in the pool of funded proposals. Viewed more broadly, the methodology proposed in this article provides a general framework for the analysis of data collected interactively from expert panels through the use of the Delphi method and related procedures.

Model-based clustering using S-PLUS

Cluster analysis can be used to identify homogenous subgroups in many fields, including psychology and psychiatry. However, most clustering methods implemented in general-purpose statistical packages are heuristic and can be criticized in principle for their lack of an underlying statistical model. Furthermore correlations between variables are generally ignored by standard methods. The question addressed here is whether currently available commercial software (S-PLUS), which provides model-based methods for clustering correlated continuous data, should be used for clustering data derived from questionnaires. Such data may be either continuous or ordinal in nature and typically exhibit correlations. Performance is assessed in this study on simulated data sets containing distinct multivariate normal subpopulations, both before and after mapping the simulated data onto an ordinal scale. A practical example showing how correlated data can be cluster-analysed using these methods is given. The conclusion is that model-based methods are certainly worthwhile for continuous data. However, their benefit, in particular their ability to deal with correlated data, is not marked for ordinal data. Simpler methods such as Ward's method may be almost as effective in this situation.