1
|
Dedhe AM, Chowkase AA, Gogate NV, Kshirsagar MM, Naphade R, Naphade A, Kulkarni P, Naik M, Dharm A, Raste S, Patankar S, Jogdeo CM, Sathe A, Kulkarni S, Bapat V, Joshi R, Deshmukh K, Lele S, Manke-Miller KJ, Cantlon JF, Pandit PS. Conventional and frugal methods of estimating COVID-19-related excess deaths and undercount factors. Sci Rep 2024; 14:10378. [PMID: 38710715 DOI: 10.1038/s41598-024-57634-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 03/20/2024] [Indexed: 05/08/2024] Open
Abstract
Across the world, the officially reported number of COVID-19 deaths is likely an undercount. Establishing true mortality is key to improving data transparency and strengthening public health systems to tackle future disease outbreaks. In this study, we estimated excess deaths during the COVID-19 pandemic in the Pune region of India. Excess deaths are defined as the number of additional deaths relative to those expected from pre-COVID-19-pandemic trends. We integrated data from: (a) epidemiological modeling using pre-pandemic all-cause mortality data, (b) discrepancies between media-reported death compensation claims and official reported mortality, and (c) the "wisdom of crowds" public surveying. Our results point to an estimated 14,770 excess deaths [95% CI 9820-22,790] in Pune from March 2020 to December 2021, of which 9093 were officially counted as COVID-19 deaths. We further calculated the undercount factor-the ratio of excess deaths to officially reported COVID-19 deaths. Our results point to an estimated undercount factor of 1.6 [95% CI 1.1-2.5]. Besides providing similar conclusions about excess deaths estimates across different methods, our study demonstrates the utility of frugal methods such as the analysis of death compensation claims and the wisdom of crowds in estimating excess mortality.
Collapse
Affiliation(s)
- Abhishek M Dedhe
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA.
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA.
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, USA.
| | - Aakash A Chowkase
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Psychology, University of California, Berkeley, CA, USA
| | - Niramay V Gogate
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Physics and Astronomy, Texas Tech University, Lubbock, TX, USA
| | - Manas M Kshirsagar
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Institute of Clinical Neurobiology, University Hospital Würzburg, Würzburg, Germany
| | - Rohan Naphade
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
| | - Atharv Naphade
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
| | - Pranav Kulkarni
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Mrunmayi Naik
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
| | - Aarya Dharm
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
| | - Soham Raste
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
| | - Shravan Patankar
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Mathematics, University of Illinois, Chicago, IL, USA
| | - Chinmay M Jogdeo
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- College of Pharmacy, University of Nebraska Medical Center, Omaha, NE, USA
| | - Aalok Sathe
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Soham Kulkarni
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Troy High School, Fullerton, CA, USA
| | - Vibha Bapat
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Biology, Indian Institute of Science Education and Research, Pune, Maharashtra, India
| | - Rohinee Joshi
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Mathematics, Indian Institute of Technology Bombay, Mumbai, Maharashtra, India
| | - Kshitij Deshmukh
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Division of Molecular and Cellular Function, School of Biological Sciences, University of Manchester, Manchester, Greater Manchester, UK
- Department of Molecular Physiology and Biophysics, Pappajohn Biomedical Discovery Building (PBDB), University of Iowa, Iowa City, IA, USA
| | - Subhash Lele
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA
- Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB, Canada
| | | | - Jessica F Cantlon
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA, USA
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, PA, USA
| | - Pranav S Pandit
- JPF Analytics, Jnana Prabodhini Foundation, Murrieta, CA, USA.
- Department of Population Health and Reproduction, School of Veterinary Medicine, University of California, Davis, Davis, CA, USA.
| |
Collapse
|
2
|
Gruen A, Mattingly KR, Morwitch E, Bossaerts F, Clifford M, Nash C, Ioannidis JPA, Ponsonby AL. Machine learning augmentation reduces prediction error in collective forecasting: development and validation across prediction markets with application to COVID events. EBioMedicine 2023; 96:104783. [PMID: 37708701 PMCID: PMC10502359 DOI: 10.1016/j.ebiom.2023.104783] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 08/18/2023] [Accepted: 08/18/2023] [Indexed: 09/16/2023] Open
Abstract
BACKGROUND The recent COVID-19 pandemic highlighted the challenges for traditional forecasting. Prediction markets are a promising way to generate collective forecasts and could potentially be enhanced if high-quality crowdsourced inputs were identified and preferentially weighted for likely accuracy in real-time with machine learning. METHODS We aim to leverage human prediction markets with real-time machine weighting of likely higher accuracy trades to improve performance. The crowd sourced Almanis prediction market longitudinal platform (n = 1822) and Next Generation Social Science (NGS2) platform (n = 103) were utilised. FINDINGS A 43-feature model predicted accurate forecasters, those with top quintile relative Brier accuracy, with subsequent replication in two out-of-sample datasets (pboth <1 × 10-9). Trades graded by this model as having higher accuracy scores than others produced a greater AUC temporal gain in the overall market after vs before trade. Accuracy score-weighted forecasts had higher accuracy than market forecasts alone, particularly when the two systems disagreed by 5% or more for binary event prediction: the hybrid system demonstrating substantial % AUC gains of 13.2%, p = 1.35 × 10-14 and 13.8%, p = 0.003 in two out-of-sample datasets. When discordant, the hybrid model was correct for COVID-19 event occurrence 72.7% of the time vs 27.3% for market models, p = 0.007. This net classification benefit was replicated in the separate Almanis B dataset, p = 2.4 × 10-7. INTERPRETATION Real-time machine classification followed by weighting human trades according to likely accuracy improves collective forecasting performance. This could provide improved anticipation of and thus response to emerging risks. FUNDING This work was supported by an AusIndustry R and D tax incentive program from the Department of Industry, Science, Energy and Resources, Australia, to SlowVoice Pty Ltd. (IR 2101990) and Fellowship (GNT 1110200) and Investigator grant (GNT 1197234) to A-L Ponsonby by the National Health and Medical Research Council of Australia.
Collapse
Affiliation(s)
- Alexander Gruen
- The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia
| | | | - Ellen Morwitch
- The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia
| | | | | | - Chad Nash
- Dysrupt Labs by SlowVoice, Melbourne, Australia
| | - John P A Ioannidis
- Stanford Prevention Research Center, Department of Medicine, and Departments of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, Stanford University, Meta-Research Innovation Center at Stanford, Stanford, CA, USA
| | - Anne-Louise Ponsonby
- The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia; Murdoch Children's Research Institute, Royal Children's Hospital, Melbourne, Australia; Centre of Epidemiology and Biostatistics, School of Population and Global Health, University of Melbourne, Australia.
| |
Collapse
|
3
|
Mavragani A, Eysenbach G, Ingram D, Khan B, Marsh J, McAndrew T. Crowdsourced Perceptions of Human Behavior to Improve Computational Forecasts of US National Incident Cases of COVID-19: Survey Study. JMIR Public Health Surveill 2022; 8:e39336. [PMID: 36219845 PMCID: PMC9822568 DOI: 10.2196/39336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Revised: 10/08/2022] [Accepted: 10/09/2022] [Indexed: 11/07/2022] Open
Abstract
BACKGROUND Past research has shown that various signals associated with human behavior (eg, social media engagement) can benefit computational forecasts of COVID-19. One behavior that has been shown to reduce the spread of infectious agents is compliance with nonpharmaceutical interventions (NPIs). However, the extent to which the public adheres to NPIs is difficult to measure and consequently difficult to incorporate into computational forecasts of infectious diseases. Soliciting judgments from many individuals (ie, crowdsourcing) can lead to surprisingly accurate estimates of both current and future targets of interest. Therefore, asking a crowd to estimate community-level compliance with NPIs may prove to be an accurate and predictive signal of an infectious disease such as COVID-19. OBJECTIVE We aimed to show that crowdsourced perceptions of compliance with NPIs can be a fast and reliable signal that can predict the spread of an infectious agent. We showed this by measuring the correlation between crowdsourced perceptions of NPIs and US incident cases of COVID-19 1-4 weeks ahead, and evaluating whether incorporating crowdsourced perceptions improves the predictive performance of a computational forecast of incident cases. METHODS For 36 weeks from September 2020 to April 2021, we asked 2 crowds 21 questions about their perceptions of community adherence to NPIs and public health guidelines, and collected 10,120 responses. Self-reported state residency was compared to estimates from the US census to determine the representativeness of the crowds. Crowdsourced NPI signals were mapped to 21 mean perceived adherence (MEPA) signals and analyzed descriptively to investigate features, such as how MEPA signals changed over time and whether MEPA time series could be clustered into groups based on response patterns. We investigated whether MEPA signals were associated with incident cases of COVID-19 1-4 weeks ahead by (1) estimating correlations between MEPA and incident cases, and (2) including MEPA into computational forecasts. RESULTS The crowds were mostly geographically representative of the US population with slight overrepresentation in the Northeast. MEPA signals tended to converge toward moderate levels of compliance throughout the survey period, and an unsupervised analysis revealed signals clustered into 4 groups roughly based on the type of question being asked. Several MEPA signals linearly correlated with incident cases of COVID-19 1-4 weeks ahead at the US national level. Including questions related to social distancing, testing, and limiting large gatherings increased out-of-sample predictive performance for probabilistic forecasts of incident cases of COVID-19 1-3 weeks ahead when compared to a model that was trained on only past incident cases. CONCLUSIONS Crowdsourced perceptions of nonpharmaceutical adherence may be an important signal to improve forecasts of the trajectory of an infectious agent and increase public health situational awareness.
Collapse
Affiliation(s)
| | | | - David Ingram
- Actuarial Risk Management, Austin, TX, United States
| | - Bilal Khan
- Computer Science and Engineering, Lehigh University, Bethlehem, PA, United States
| | - Jessecae Marsh
- Department of Psychology, Lehigh University, Bethlehem, PA, United States
| | - Thomas McAndrew
- College of Health, Lehigh University, Bethlehem, PA, United States
| |
Collapse
|