The Challenge of Quantification: An Interdisciplinary Reading

Socio-environmental modeling shows physics-like confidence with water modeling surpassing it in numerical claims

Several modern scientific fields rely on computationally intensive mathematical models to study uncertain, complex socio-environmental phenomena such as the spread of a virus, climate change, or the water cycle. However, the degree of epistemic commitment of these fields is unclear. By using machine learning to extract the knowledge claims of around 755,000 abstracts from 14 scientific fields spanning the human and physical sciences, we show that epidemic, integrated assessment, and water modeling display a degree of linguistic assertiveness akin to physics. Water modeling surpasses even the most accurate physical sciences in substantiating knowledge claims with numbers, which are largely produced without accompanying uncertainty and sensitivity analysis. By exploring the balance between doubt and certainty in academic writing, our study reflects on whether the strong conviction and quantification of fields modeling socio-environmental processes, especially water modeling, are epistemically justified.

The COVID-19 quandemic

BACKGROUND

The terms syndemic and infodemic have both been applied to the COVID-19 pandemic, and emphasize concurrent socio-cultural dynamics that are distinct from the epidemiological outbreak itself. We argue that the COVID-19 pandemic has exposed yet another important socio-political dynamic that can best be captured by the concept of a quandemic - a portmanteau of "quantification" and "pandemic".

MAIN TEXT

The use of quantifiable metrics in policymaking and evaluation has increased throughout the last decades, and is driven by a synergetic relationship between increases in supply and advances in demand for data. In most regards this is a welcome development. However, a quandemic, refers to a situation where a small subset of quantifiable metrics dominate policymaking and the public debate, at the expense of more nuanced and multi-disciplinary discourse. We therefore pose that a quandemic reduces a complex pandemic to a few metrics that present an overly simplified picture. During COVID-19, these metrics were different iterations of case numbers, deaths, hospitalizations, diagnostic tests, bed occupancy rates, the R-number and vaccination coverage. These limited metrics came to constitute the internationally recognized benchmarks for effective pandemic management. Based on experience from the Nordic region, we propose four distinct dynamics that characterize a quandemic: 1) A limited number of metrics tend to dominate both political, expert, and public spheres and exhibit a great deal of rigidity over time. 2) These few metrics crowd-out other forms of evidence relevant to pandemic response. 3) The metrics tend to favour certain outcomes of pandemic management, such as reducing hospitalization rates, while not capturing potential adverse effects such as social isolation and loneliness. 4) Finally, the metrics are easily standardized across countries, and give rise to competitive dynamics based on international comparisons and benchmarking.

CONCLUSION

A quandemic is not inevitable. While metrics are an indispensable part of evidence-informed policymaking, being attentive to quandemic dynamics also means identifying relevant evidence that might not be captured by these few but dominant metrics. Pandemic responses need to account for and consider multilayered vulnerabilities and risks, including socioeconomic inequities and comorbidities.

What can mathematical modelling contribute to a sociology of quantification?

Sociology of quantification has spent relatively less energies investigating mathematical modelling than it has on other forms of quantification such as statistics, metrics, or algorithms based on artificial intelligence. Here we investigate whether concepts and approaches from mathematical modelling can provide sociology of quantification with nuanced tools to ensure the methodological soundness, normative adequacy and fairness of numbers. We suggest that methodological adequacy can be upheld by techniques in the field of sensitivity analysis, while normative adequacy and fairness are targeted by the different dimensions of sensitivity auditing. We also investigate in which ways modelling can inform other instances of quantification as to promote political agency.