1
|
Li Y, Viswaroopan D, He W, Li J, Zuo X, Xu H, Tao C. Improving entity recognition using ensembles of deep learning and fine-tuned large language models: A case study on adverse event extraction from VAERS and social media. J Biomed Inform 2025; 163:104789. [PMID: 39923968 DOI: 10.1016/j.jbi.2025.104789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2024] [Revised: 01/07/2025] [Accepted: 02/05/2025] [Indexed: 02/11/2025]
Abstract
OBJECTIVE Adverse event (AE) extraction following COVID-19 vaccines from text data is crucial for monitoring and analyzing the safety profiles of immunizations, identifying potential risks and ensuring the safe use of these products. Traditional deep learning models are adept at learning intricate feature representations and dependencies in sequential data, but often require extensive labeled data. In contrast, large language models (LLMs) excel in understanding contextual information, but exhibit unstable performance on named entity recognition (NER) tasks, possibly due to their broad but unspecific training. This study aims to evaluate the effectiveness of LLMs and traditional deep learning models in AE extraction, and to assess the impact of ensembling these models on performance. METHODS In this study, we utilized reports and posts from the Vaccine Adverse Event Reporting System (VAERS) (n = 230), Twitter (n = 3,383), and Reddit (n = 49) as our corpora. Our goal was to extract three types of entities: vaccine, shot, and adverse event (ae). We explored and fine-tuned (except GPT-4) multiple LLMs, including GPT-2, GPT-3.5, GPT-4, Llama-2 7b, and Llama-2 13b, as well as traditional deep learning models like Recurrent neural network (RNN) and Bidirectional Encoder Representations from Transformers for Biomedical Text Mining (BioBERT). To enhance performance, we created ensembles of the three models with the best performance. For evaluation, we used strict and relaxed F1 scores to evaluate the performance for each entity type, and micro-average F1 was used to assess the overall performance. RESULTS The ensemble demonstrated the best performance in identifying the entities "vaccine," "shot," and "ae," achieving strict F1-scores of 0.878, 0.930, and 0.925, respectively, and a micro-average score of 0.903. These results underscore the significance of fine-tuning models for specific tasks and demonstrate the effectiveness of ensemble methods in enhancing performance. CONCLUSION In conclusion, this study demonstrates the effectiveness and robustness of ensembling fine-tuned traditional deep learning models and LLMs, for extracting AE-related information following COVID-19 vaccination. This study contributes to the advancement of natural language processing in the biomedical domain, providing valuable insights into improving AE extraction from text data for pharmacovigilance and public health surveillance.
Collapse
Affiliation(s)
- Yiming Li
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Deepthi Viswaroopan
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - William He
- Department of Electrical & Computer Engineering, Pratt School of Engineering, Duke University, 305 Tower Engineering Building, Durham, NC 27708, USA
| | - Jianfu Li
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL 32224, USA
| | - Xu Zuo
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Hua Xu
- Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT 06510, USA
| | - Cui Tao
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL 32224, USA.
| |
Collapse
|
2
|
Meng X, Fan Y, Qiao Y, Lin J, Cai Z, Si S. Evolutionary analysis of a coupled epidemic-voluntary vaccination behavior model with immunity waning on complex networks. RISK ANALYSIS : AN OFFICIAL PUBLICATION OF THE SOCIETY FOR RISK ANALYSIS 2025. [PMID: 39826914 DOI: 10.1111/risa.17699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 11/11/2024] [Accepted: 12/13/2024] [Indexed: 01/22/2025]
Abstract
Vaccination is the most effective method of preventing and controlling the transmission of infectious diseases within populations. However, the phenomenon of waning immunity can induce periodic fluctuations in epidemic spreading. This study proposes a coupled epidemic-vaccination dynamic model to analyze the influence of immunity waning on the epidemic spreading within the context of voluntary vaccination. First, we establish an SIRSV (susceptible-infected-recovered-susceptible-vaccinated) compartment model to describe the transmission mechanism of infectious diseases based on the mean-field theory. Within this model, we incorporate a nonlinear infection rate with network topology and consider the waning natural and vaccine-induced immunity at the individual level. The evolutionary model of voluntary vaccination strategy is integrated into the SIRSV model to characterize the impact of vaccination behavior on the infectious disease transmission. We also consider two individual risk assessment methods, namely, the individual-based risk assessment (IB-RA) method and the society-based risk assessment (SB-RA) method, originating from local and global perspectives, respectively. Then, utilizing the next-generation matrix method, we derive the time-varying effective reproduction numbers of the model. Also, the theoretical analysis of optimal strategy thresholds in the individual decision-making process is also conducted. The results indicate that the thresholds obtained from the agent-based model (ABM) simulation method are consistent with the theoretical analysis, demonstrating the effectiveness of our model. Finally, we apply the coupled model to the COVID-19 pandemic in France, Germany, Italy, and the United Kingdom. This study analyzes the impact of waning immunity and provides early warning for the outbreak of the epidemics.
Collapse
Affiliation(s)
- Xueyu Meng
- Department of Industrial Engineering, School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, China
- Ministry of Industry and Information Technology Key Laboratory of Industrial Engineering and Intelligent Manufacturing, Northwestern Polytechnical University, Xi'an, China
- Department of Physics, University of Fribourg, Fribourg, Switzerland
| | - Yufei Fan
- Department of Industrial Engineering, School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, China
- Ministry of Industry and Information Technology Key Laboratory of Industrial Engineering and Intelligent Manufacturing, Northwestern Polytechnical University, Xi'an, China
| | - Yanan Qiao
- Department of Physics, University of Fribourg, Fribourg, Switzerland
- State Key Laboratory for Manufacturing Systems Engineering, School of Management, Xian Jiaotong University, Xian, China
| | - Jianhong Lin
- Blockchain & Distributed Ledger Technologies Group, Department of Informatics, University of Zurich, Zurich, Switzerland
- UZH Blockchain Center, University of Zurich, Zurich, Switzerland
| | - Zhiqiang Cai
- Department of Industrial Engineering, School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, China
- Ministry of Industry and Information Technology Key Laboratory of Industrial Engineering and Intelligent Manufacturing, Northwestern Polytechnical University, Xi'an, China
| | - Shubin Si
- Department of Industrial Engineering, School of Mechanical Engineering, Northwestern Polytechnical University, Xi'an, China
- Ministry of Industry and Information Technology Key Laboratory of Industrial Engineering and Intelligent Manufacturing, Northwestern Polytechnical University, Xi'an, China
| |
Collapse
|
3
|
Aldila D, Dhanendra RP, Khoshnaw SHA, Wijayanti Puspita J, Kamalia PZ, Shahzad M. Understanding HIV/AIDS dynamics: insights from CD4+T cells, antiretroviral treatment, and country-specific analysis. Front Public Health 2024; 12:1324858. [PMID: 38665242 PMCID: PMC11043473 DOI: 10.3389/fpubh.2024.1324858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Accepted: 03/14/2024] [Indexed: 04/28/2024] Open
Abstract
In this article, we present a mathematical model for human immunodeficiency virus (HIV)/Acquired immune deficiency syndrome (AIDS), taking into account the number of CD4+T cells and antiretroviral treatment. This model is developed based on the susceptible, infected, treated, AIDS (SITA) framework, wherein the infected and treated compartments are divided based on the number of CD4+T cells. Additionally, we consider the possibility of treatment failure, which can exacerbate the condition of the treated individual. Initially, we analyze a simplified HIV/AIDS model without differentiation between the infected and treated classes. Our findings reveal that the global stability of the HIV/AIDS-free equilibrium point is contingent upon the basic reproduction number being less than one. Furthermore, a bifurcation analysis demonstrates that our simplified model consistently exhibits a transcritical bifurcation at a reproduction number equal to one. In the complete model, we elucidate how the control reproduction number determines the stability of the HIV/AIDS-free equilibrium point. To align our model with the empirical data, we estimate its parameters using prevalence data from the top four countries affected by HIV/AIDS, namely, Eswatini, Lesotho, Botswana, and South Africa. We employ numerical simulations and conduct elasticity and sensitivity analyses to examine how our model parameters influence the control reproduction number and the dynamics of each model compartment. Our findings reveal that each country displays distinct sensitivities to the model parameters, implying the need for tailored strategies depending on the target country. Autonomous simulations highlight the potential of case detection and condom use in reducing HIV/AIDS prevalence. Furthermore, we identify that the quality of condoms plays a crucial role: with higher quality condoms, a smaller proportion of infected individuals need to use them for the potential eradication of HIV/AIDS from the population. In our optimal control simulations, we assess population behavior when control interventions are treated as time-dependent variables. Our analysis demonstrates that a combination of condom use and case detection, as time-dependent variables, can significantly curtail the spread of HIV while maintaining an optimal cost of intervention. Moreover, our cost-effectiveness analysis indicates that the condom use intervention alone emerges as the most cost-effective strategy, followed by a combination of case detection and condom use, and finally, case detection as a standalone strategy.
Collapse
Affiliation(s)
- Dipo Aldila
- Department of Mathematics, Universitas Indonesia, Depok, Indonesia
| | | | | | | | | | - Muhammad Shahzad
- Department of Mathematics and Statistics, The University of Haripur, Haripur, KP, Pakistan
| |
Collapse
|
4
|
Majumder M, Pal S, Kumar Tiwari P. Vaccination impact on impending HIV-COVID-19 dual epidemic with autogenous behavior modification: Hill-type functional response and premeditated optimization technique. CHAOS (WOODBURY, N.Y.) 2024; 34:033104. [PMID: 38427935 DOI: 10.1063/5.0186156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2023] [Accepted: 02/09/2024] [Indexed: 03/03/2024]
Abstract
An HIV-COVID-19 co-infection dynamics is modeled mathematically assimilating the vaccination mechanism that incorporates endogenous modification of human practices generated by the COVID-19 prevalence, absorbing the relevance of the treatment mechanism in suppressing the co-infection burden. Envisaging a COVID-19 situation, the HIV-subsystem is analyzed by introducing COVID-19 vaccination for the HIV-infected population as a prevention, and the "vaccination influenced basic reproduction number" of HIV is derived. The mono-infection systems experience forward bifurcation that evidences the persistence of diseases above unit epidemic thresholds. Delicate simulation methodologies are employed to explore the impacts of baseline vaccination, prevalence-dependent spontaneous behavioral change that induces supplementary vaccination, and medication on the dual epidemic. Captivatingly, a paradox is revealed showing that people start to get vaccinated at an additional rate with the increased COVID-19 prevalence, which ultimately diminishes the dual epidemic load. It suggests increasing the baseline vaccination rate and the potency of propagated awareness. Co-infection treatment needs to be emphasized parallelly with single infection medication under dual epidemic situations. Further, an optimization technique is introduced to the co-infection model integrating vaccination and treatment control mechanisms, which approves the strategy combining vaccination with awareness and medication as the ideal one for epidemic and economic gain. Conclusively, it is manifested that waiting frivolously for any anticipated outbreak, depending on autogenous behavior modification generated by the increased COVID-19 prevalence, instead of elevating vaccination campaigns and the efficacy of awareness beforehand, may cause devastation to the population under future co-epidemic conditions.
Collapse
Affiliation(s)
- Madhuri Majumder
- Department of Mathematics, University of Kalyani, Kalyani 741235, India
| | - Samares Pal
- Department of Mathematics, University of Kalyani, Kalyani 741235, India
| | - Pankaj Kumar Tiwari
- Department of Basic Science and Humanities, Indian Institute of Information Technology, Bhagalpur 813210, India
| |
Collapse
|
5
|
Mahadhika CK, Aldila D. A deterministic transmission model for analytics-driven optimization of COVID-19 post-pandemic vaccination and quarantine strategies. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:4956-4988. [PMID: 38872522 DOI: 10.3934/mbe.2024219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2024]
Abstract
This study developed a deterministic transmission model for the coronavirus disease of 2019 (COVID-19), considering various factors such as vaccination, awareness, quarantine, and treatment resource limitations for infected individuals in quarantine facilities. The proposed model comprised five compartments: susceptible, vaccinated, quarantined, infected, and recovery. It also considered awareness and limited resources by using a saturated function. Dynamic analyses, including equilibrium points, control reproduction numbers, and bifurcation analyses, were conducted in this research, employing analytics to derive insights. Our results indicated the possibility of an endemic equilibrium even if the reproduction number for control was less than one. Using incidence data from West Java, Indonesia, we estimated our model parameter values to calibrate them with the real situation in the field. Elasticity analysis highlighted the crucial role of contact restrictions in reducing the spread of COVID-19, especially when combined with community awareness. This emphasized the analytics-driven nature of our approach. We transformed our model into an optimal control framework due to budget constraints. Leveraging Pontriagin's maximum principle, we meticulously formulated and solved our optimal control problem using the forward-backward sweep method. Our experiments underscored the pivotal role of vaccination in infection containment. Vaccination effectively reduces the risk of infection among vaccinated individuals, leading to a lower overall infection rate. However, combining vaccination and quarantine measures yields even more promising results than vaccination alone. A second crucial finding emphasized the need for early intervention during outbreaks rather than delayed responses. Early interventions significantly reduce the number of preventable infections, underscoring their importance.
Collapse
Affiliation(s)
- C K Mahadhika
- Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Indonesia, Depok 16424, Indonesia
| | - Dipo Aldila
- Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Indonesia, Depok 16424, Indonesia
| |
Collapse
|
6
|
Liu L, Wang X, Li Y. Mathematical analysis and optimal control of an epidemic model with vaccination and different infectivity. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:20914-20938. [PMID: 38124581 DOI: 10.3934/mbe.2023925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2023]
Abstract
This paper aims to explore the complex dynamics and impact of vaccinations on controlling epidemic outbreaks. An epidemic transmission model which considers vaccinations and two different infection statuses with different infectivity is developed. In terms of a dynamic analysis, we calculate the basic reproduction number and control reproduction number and discuss the stability of the disease-free equilibrium. Additionally, a numerical simulation is performed to explore the effects of vaccination rate, immune waning rate and vaccine ineffective rate on the epidemic transmission. Finally, a sensitivity analysis revealed three factors that can influence the threshold: transmission rate, vaccination rate, and the hospitalized rate. In terms of optimal control, the following three time-related control variables are introduced to reconstruct the corresponding control problem: reducing social distance, enhancing vaccination rates, and enhancing the hospitalized rates. Moreover, the characteristic expression of optimal control problem. Four different control combinations are designed, and comparative studies on control effectiveness and cost effectiveness are conducted by numerical simulations. The results showed that Strategy C (including all the three controls) is the most effective strategy to reduce the number of symptomatic infections and Strategy A (including reducing social distance and enhancing vaccination rate) is the most cost-effective among the three strategies.
Collapse
Affiliation(s)
- Lili Liu
- Shanxi Key Laboratory of Mathematical Techniques and Big Data Analysis on Disease Control and Prevention, Complex Systems Research Center, Shanxi University, Taiyuan 030006, China
| | - Xi Wang
- Shanxi Key Laboratory of Mathematical Techniques and Big Data Analysis on Disease Control and Prevention, Complex Systems Research Center, Shanxi University, Taiyuan 030006, China
| | - Yazhi Li
- School of Mathematics and Statistics, Qiannan Normal University for Nationalities, Duyun 558000, China
| |
Collapse
|
7
|
Kim JE, Choi H, Lee M, Lee CH. The effect of shortening the quarantine period and lifting the indoor mask mandate on the spread of COVID-19: a mathematical modeling approach. Front Public Health 2023; 11:1166528. [PMID: 37546304 PMCID: PMC10401846 DOI: 10.3389/fpubh.2023.1166528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 07/06/2023] [Indexed: 08/08/2023] Open
Abstract
In this paper, we present a mathematical model to assess the impact of reducing the quarantine period and lifting the indoor mask mandate on the spread of Coronavirus Disease 2019 (COVID-19) in Korea. The model incorporates important epidemiological parameters, such as transmission rates and mortality rates, to simulate the transmission of the virus under different scenarios. Our findings reveal that the impact of mask wearing fades in the long term, which highlights the crucial role of quarantine in controlling the spread of the disease. In addition, balancing the confirmed cases and costs, the lifting of mandatory indoor mask wearing is cost-effective; however, maintaining the quarantine period remains essential. A relationship between the disease transmission rate and vaccine efficiency was also apparent, with higher transmission rates leading to a greater impact of the vaccine efficiency. Moreover, our findings indicate that a higher disease transmission rate exacerbates the consequences of early quarantine release.
Collapse
Affiliation(s)
- Jung Eun Kim
- Department of Mathematics and Computer Science, Korea Science Academy of KAIST, Busan, Republic of Korea
| | - Heejin Choi
- Department of Mathematical Sciences, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Minji Lee
- Department of Mathematical Sciences, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| | - Chang Hyeong Lee
- Department of Mathematical Sciences, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea
| |
Collapse
|