Know your population and know your model: Using model-based regression and poststratification to generalize findings beyond the observed sample

On the Selection of Item Scores or Composite Scores for Clinical Prediction

Recent shifts to prioritize prediction, rather than explanation, in psychological science have increased applications of predictive modeling methods. However, composite predictors, such as sum scores, are still commonly used in practice. The motivations behind composite test scores are largely intertwined with reducing the influence of measurement error in answering explanatory questions. But this may be detrimental for predictive aims. The present paper examines the impact of utilizing composite or item-level predictors in linear regression. A mathematical examination of the bias-variance decomposition of prediction error in the presence of measurement error is provided. It is shown that prediction bias, which may be exacerbated by composite scoring, drives prediction error for linear regression. This may be particularly salient when composite scores are comprised of heterogeneous items such as in clinical scales where items correspond to symptoms. With sufficiently large training samples, the increased prediction variance associated with item scores becomes negligible even when composite scores are sufficient. Practical implications of predictor scoring are examined in an empirical example predicting suicidal ideation from various depression scales. Results show that item scores can markedly improve prediction particularly for symptom-based scales. Cross-validation methods can be used to empirically justify predictor scoring decisions.

Material insecurity and religiosity: A causal analysis

Some cultural evolutionary models predict that under stressful reductions of well-being, individuals will be more attracted and fastidiously adhere to traditional systems of norms that promote solidarity and cooperation. As religious systems can bolster human relationships with a variety of mechanisms, the material insecurity hypothesis of religion posits that individual religiosity will increase under conditions of material insecurity. The bulk of the literature up to this point has been correlational and cross-national. Here, across 14 field sites, we examine the causal role that educational attainment and food insecurity play in religiosity. We find that years of formal education and food insecurity do not consistently contribute to individual religiosity cross-culturally. We conclude with a discussion of some theoretical and methodological implications. As a general workflow for cross-cultural causal research in the quantitative social sciences, the present work is a modest but necessary first step in reliably estimating causation in the material insecurity hypothesis of religiosity.

Quantifying and reducing inequity in average treatment effect estimation

BACKGROUND

Across studies of average treatment effects, some population subgroups consistently have lower representation than others which can lead to discrepancies in how well results generalize.

METHODS

We develop a framework for quantifying inequity due to systemic disparities in sample representation and a method for mitigation during data analysis. Assuming subgroup treatment effects are exchangeable, an unbiased sample average treatment effect estimator will have lower mean-squared error, on average across studies, for subgroups with less representation when treatment effects vary. We present a method for estimating average treatment effects in representation-adjusted samples which enables subgroups to optimally leverage information from the full sample rather than only their own subgroup's data. Two approaches for specifying representation adjustment are offered-one minimizes average mean-squared error for each subgroup separately and the other balances minimization of mean-squared error and equal representation. We conduct simulation studies to compare the performance of the proposed estimators to several subgroup-specific estimators.

RESULTS

We find that the proposed estimators generally provide lower mean squared error, particularly for smaller subgroups, relative to the other estimators. As a case study, we apply this method to a subgroup analysis from a published study.

CONCLUSIONS

We recommend the use of the proposed estimators to mitigate the impact of disparities in representation, though structural change is ultimately needed.

Reducing the Bias of Norm Scores in Non-Representative Samples: Weighting as an Adjunct to Continuous Norming Methods

We investigated whether the accuracy of normed test scores derived from non-demographically representative samples can be improved by combining continuous norming methods with compensatory weighting of test results. To this end, we introduce Raking, a method from social sciences, to psychometrics. In a simulated reference population, we modeled a latent cognitive ability with a typical developmental gradient, along with three demographic variables that were correlated to varying degrees with the latent ability. We simulated five additional populations representing patterns of non-representativeness that might be encountered in the real world. We subsequently drew smaller normative samples from each population and used an one-parameter logistic Item Response Theory (IRT) model to generate simulated test results for each individual. Using these simulated data, we applied norming techniques, both with and without compensatory weighting. Weighting reduced the bias of the norm scores when the degree of non-representativeness was moderate, with only a small risk of generating new biases.

Rigorous exploration in a model-centric science via epistemic iteration

Urgent attention is needed to address generalizability problems in psychology. However, the current dominant paradigm centered on dichotomous results and rapid discoveries cannot provide the solution because of its theoretical inadequacies. We propose a paradigm shift towards a model-centric science, which provides the sophistication to understanding the sources of generalizability and promote systematic exploration. In a model-centric paradigm, scientific activity involves iteratively building and refining theoretical, empirical, and statistical models that communicate with each other. This approach is transparent, and efficient in addressing generalizability issues. We illustrate the nature of scientific activity in a model-centric system and its potential for advancing the field of psychology.

Exercise improves the social and behavioral skills of children and adolescent with autism spectrum disorders

Background

Currently, there is no standard treatment for Autism Spectrum Disorders (ASD), but there are many ways to minimize the symptoms and maximize abilities. Some studies suggest that exercise and other physical activities with children with ASD may be beneficial. In this study, we hypothesized that a physical exercise program (48-week exercise-intervention) could improve symptomatology dyad among children and adolescents with ASD. Our main aim was to examine the effects of physical activity on the primary clinical symptoms and associated comorbidities in children and adolescents with ASD.

Methods

We allocated 229 children with ASD, ranging in age from 2.3-17.3 years (M = 7.8, SD = 3.2), into three groups: (a) exercise- intervention group, (b) control group from the same institution, and (c) control group from another institution. The exercise program was performed at moderate intensity in a 30 min section twice a week for 48 weeks. We used Bayesian multilevel regression modeling to examine participant outcomes and responses to the exercise-intervention.

Results

Our results showed that a 48-week exercise-intervention substantially decreased ASD social interaction problems, attention deficit, emotional reactivity, stereotypical verbal and motor behavior, and sleep disturbances. However, physical exercise did not affect eye contact and food selectivity. We also observed that ASD severity and socioeconomic status influence eye contact, attention deficit, and sleep disturbance responses.

Conclusion

In conclusion, children and adolescents with ASD exposed to a 48-week physical exercise-intervention program had important improvements in ASD symptoms. This study highlights that structured exercise programs can be a powerful complementary therapy for the ASD population.

Bayesian Multistate Life Table Methods for Large and Complex State Spaces: Development and Illustration of a New Method

Multistate life table methods are an important tool for producing easily understood measures of population health. Most contemporary uses of these methods involve sample data, thus requiring techniques for capturing uncertainty in estimates. In recent decades, several methods have been developed to do so. Among these methods, the Bayesian approach proposed by Lynch and Brown has several unique advantages. However, the approach is limited to estimating years to be spent in only two living states, such as "healthy" and "unhealthy." In this article, the authors extend this method to allow for large state spaces with "quasi-absorbing" states. The authors illustrate the new method and show its advantages using data from the Health and Retirement Study to investigate U.S. regional differences in years of remaining life to be spent with diabetes, chronic conditions, and disabilities. The method works well and yields rich output for reporting and subsequent analyses. The expanded method also should facilitate the use of multi-state life tables to address a wider array of social science research questions.

Meal preparation and consumption before and during the COVID-19 pandemic: The relationship with cooking skills of Brazilian university students

OBJECTIVE

This study aimed to analyze meal preparation and the place of its consumption by university students before and during the COVID-19 pandemic, according to their individual characteristics and cooking skills.

METHODS

Data were collected with 1919 Brazilian university students using the Brazilian Cooking Skills Questionnaire (BCSQ) and questions about gender, living arrangement, knowing how to cook and location of meal consumption. Bayesian multilevel ordinal regression models were used to estimate the probabilities of meal preparation and local location of its consumption by Brazilian university students before and during the COVID-19 pandemic, according to their individual characteristics and cooking skills.

RESULTS

Most of participants showed reported a high level of cooking skills (70.7%). Also, they reported a decrease in ordering delivery of take-away food and eating fast-food, while increasing homemade meals with fresh ingredients, ultra-processed food or both during the pandemic compared to the period before the quarantine. Lastly, they increased their cooking frequency during the pandemic, regardless of their cooking skills level. However, the group with high cooking skills had a greater increase in the use of fresh ingredients.

CONCLUSIONS

The study showed that the pandemic was an opportunity for university students to increase their cooking skills and frequency of cooking at home with a variety of foods. Particular attention should be given to those with low cooking skills and that use ultra-processed foods, promoting interventions to disseminate information and with behavior change components to teach about healthy cooking to increase cooking knowledge and skills, focusing on preparation of fresh ingredients.

Validity and usefulness of the student-athletes' motivation toward sport and academics questionnaire: a Bayesian multilevel approach

BACKGROUND

Reliable assessment and understanding of student-athletes' motivation for dual careers are crucial to support their career development and transitions. The purpose of this research was to examine the validity and usefulness of the student-athletes' motivation toward sport and academics questionnaire (SAMSAQ-PT) in the Brazilian higher education context. Four studies were performed.

METHODS

In study one, conceptually and semantic translation of the questionnaire and Bayesian exploratory factor analysis were conducted. In study two, a Bayesian confirmatory factor analysis with an independent sample was performed. In study three, Bayesian multilevel modeling was applied to examine the construct validity of the questionnaire in a cross-sectional sample. In study four, the SAMSAQ-PT sensitiveness was examined in a longitudinal sample, and the results were interpreted based on multilevel regression and poststratification.

RESULTS

Altogether the results provided evidence validity and usefulness of the SAMSAQ-PT in Brazilian student-athletes. The Brazilian student-athletes' motivation scores were sensitive to the influence of sex, sport level, and type of university on career and sport motivation. SAMSAQ-PT estimate scores across an academic year showed a trend of stability in the scores, adjusting for sex, sport level, type of university, and student-athlete status.

CONCLUSION

The SAMSAQ-PT proved to be a robust and valuable questionnaire, which could be used in Portuguese-speaking countries. The findings of the cross-sectional and longitudinal surveys urge to consider individual and contextual characteristics when investigating motivation of dual-career of athletes, also concerning the sex-related opportunities in university sports. Furthermore, there is a need for a call for action to promote and nurture the student-athletes motivation to remain engaged in both sports and educational commitments.