Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

German Language Acquisition of Refugee Children-The Role of Preschools and Language Instruction

Mastering the language of the destination country is key to immigrant and refugee children's educational success. Refugee children typically face the challenge of starting or continuing their educational carrier in a completely new context and in a completely new language. In this study, we examine the role of preschool attendance and formal language instruction in supporting young refugee children to acquire destination language competencies. We pursue three research objectives: First, we aim at identifying relevant conditions associated with German language acquisition in general. Second, we examine the (relative) importance of institutional learning support in preschool and language instruction. Third, we investigate whether the benefits of attending preschool are more pronounced for refugee children who have only limited exposure to the destination language outside of the institutional context, as compared to children who have more exposure to the language outside of preschool. Using data from the ReGES study, we analyze the early processes of destination language acquisition among a large population of refugee children of preschool age in Germany. Our findings indicate that conditions associated with motivation, exposure and efficiency of learning that were found in prior research to determine destination language competencies of children from other immigrant groups apply to refugee children in a similar manner. Additional conditions associated with the specific circumstances that refugees often experience, including possible consequences of insecure residence status, risk of post-traumatic stress disorders, and living in collective accommodation, do not significantly contribute to this outcome in our analysis. Furthermore, we find that there is a positive relationship between children's German language competency levels and both preschool attendance and formal language instruction. The findings indicate that the benefits of attending preschool are largely related to additional language instruction that refugee children receive within this context. Moreover, these benefits are particularly pronounced among refugee children who have only limited exposure to German at home and in their everyday lives. Overall, our findings emphasize the importance of preschool attendance and formal language instruction for refugee children's destination language acquisition.

Why bother with testing? The validity of immigrants' self-assessed language proficiency

Due to its central role in social integration, immigrants' language proficiency is a matter of considerable societal concern and scientific interest. This study examines whether commonly applied self-assessments of linguistic skills yield results that are similar to those of competence tests and thus whether these self-assessments are valid measures of language proficiency. Analyses of data for immigrant youth reveal moderate correlations between language test scores and two types of self-assessments (general ability estimates and concrete performance estimates) for the participants' first and second languages. More importantly, multiple regression models using self-assessments and models using test scores yield different results. This finding holds true for a variety of analyses and for both types of self-assessments. Our findings further suggest that self-assessed language skills are systematically biased in certain groups. Subjective measures thus seem to be inadequate estimates of language skills, and future research should use them with caution when research questions pertain to actual language skills rather than self-perceptions.