All Issue

2025 Vol.55, Issue 3 Preview Page

Original Article

Survice-BERT: A BERT Model for Named Entity Recognition in Infectious Disease Surveillance Reports
Saeyeon Cheon12
,
Insung Ahn12*
1Research center for datacentric problem solving, Korea Institute of Science and Technology Information, Yuseong-gu, Daejeon 34141, Republic of Korea
2Department of Applied AI, University of Science & Technology, Yuseong-gu, Daejeon 34113, Republic of Korea
*isahn@kisti.re.kr
30 September 2025. pp. 235-247
PDF XML Citation
Abstract

Since the onset of the COVID-19 pandemic, research on infectious disease surveillance has advanced rapidly, driven by the convergence of computing technologies with biology, medicine, and public health. A persistent challenge in the early stages of outbreaks is the lack of structured data, which hinders timely analysis and response. To address this issue, we present a domain-specific named entity recognition (NER) dataset derived from infectious disease surveillance reports and introduce Survice-BERT, a bidirectional encoder representations from transformers (BERT)-based model fine-tuned on this dataset for pandemic-related information extraction. Survice-BERT achieves an average F1-score of 0.99, demonstrating its ability to extract information with high accuracy and support forecasting for early warning of communicable diseases. We anticipate that the model will be valuable to practitioners across multiple fields. Survice-BERT is publicly available at https://github.com/csanny/Survice-BERT.

Keywords
Language model
Named entity recognition
Fine-tuning
Infectious disease
Surveillance report
References
1

Bogoch II, Watts A, Thomas-Bachli A, Huber C, Kraemer MUG, Khan K. Pneumonia of unknown aetiology in Wuhan, China: potential for international spread via commercial air travel. J Travel Med. 2020;27(2):taaa008.

10.1093/jtm/taaa00831943059PMC7107534
2

Undiagnosed pneumonia - China (Hubei): request for information. ProMED-mail. 2019 Dec 30. Available at https://www.promedmail.org/alert/6864153 [accessed on 18 September 2025].

3

Undiagnosed pneumonia - China (Hubei) (07): official confirmation of novel coronavirus. ProMED-mail. 2020 Jan 9. Available at https://www.promedmail.org/alert/6878869 [accessed on 18 September 2025].

4

Debata B, Patnaik P, Mishra A. COVID‐19 pandemic! It’s impact on people, economy, and environment. J Public Aff. 2020;20(4):e2372.

10.1002/pa.2372
5

Riccaboni M, Verginer L. The impact of the COVID-19 pandemic on scientific research in the life sciences. PLoS One. 2022;17(2):e0263001.

10.1371/journal.pone.026300135139089PMC8827464
6

Ioannidis JPA. The end of the COVID‐19 pandemic. Eur J Clin Invest. 2022;52(6):e13782.

10.1111/eci.1378235342941PMC9111437
7

Xu J, Xu K, Li Z, Meng F, Tu T, Xu L, et al. Forecast of dengue cases in 20 Chinese cities based on the deep learning method. Int J Environ Res Public Health. 2020;17(2):453.

10.3390/ijerph1702045331936708PMC7014037
8

Ribeiro MHDM, da Silva RG, Mariani VC, Coelho LDS. Short-term forecasting COVID-19 cumulative confirmed cases: Perspectives for Brazil. Chaos Solitons Fractals. 2020;135:109853.

10.1016/j.chaos.2020.10985332501370PMC7252162
9

Yeung AY, Roewer-Despres F, Rosella L, Rudzicz F. Machine learning–based prediction of growth in confirmed COVID-19 infection cases in 114 countries using metrics of nonpharmaceutical interventions and cultural dimensions: model development and validation. J Med Internet Res. 2021;23(4):e26628.

10.2196/2662833844636PMC8074952
10

Wakamiya S, Kawai Y, Aramaki E. Twitter-based influenza detection after flu peak via tweets with indirect information: text mining study. JMIR Public Health Surveill. 2018;4(3):e65.

10.2196/publichealth.862730274968PMC6231889
11

Dixon S, Keshavamurthy R, Farber DH, Stevens A, Pazdernik KT, Charles LE. A comparison of infectious disease forecasting methods across locations, diseases, and time. Pathogens. 2022;11(2):185.

10.3390/pathogens1102018535215129PMC8875569
12

Barua L, Zou B, Zhou Y. Machine learning for international freight transportation management: a comprehensive review. Res Transp Bus Manag. 2020;34:100453.

10.1016/j.rtbm.2020.100453
13

Buchmann CM, Grossmann K, Schwarz N. How agent heterogeneity, model structure and input data determine the performance of an empirical ABM–A real-world case study on residential mobility. Environ Model Softw. 2016;75:77-93.

10.1016/j.envsoft.2015.10.005
14

Domingos P. A few useful things to know about machine learning. Commun ACM. 2012;55(10):78-87.

10.1145/2347736.2347755
15

Haq MA, Ahmed A, Khan I, Gyani J, Mohamed A, Attia EA, et al. Analysis of environmental factors using AI and ML methods. Sci Rep. 2022;12(1):13267.

10.1038/s41598-022-16665-735918395PMC9345871
16

Cho H, Choi UJ, Park H. Deep learning application to time-series prediction of daily chlorophyll-a concentration. WIT Trans Ecol Environ. 2018;215:157-163.

10.2495/EID180141
17

Garg R, Barpanda S, Salanke NSGR, S R. Machine Learning Algorithms for Time Series Analysis and Forecasting. 2022;arXiv: arXiv:2211.14387.2022.

18

Cao H, Goh YM. Analyzing construction safety through time series methods. Front Eng Manag. 2019;6:262-274.

10.1007/s42524-019-0015-6
19

Chan EH, Brewer TF, Madoff LC, Pollack MP, Sonricker AL, Keller M, et al. Global capacity for emerging infectious disease detection. Proc Natl Acad Sci U S A. 2010;107(50):21701-21706.

10.1073/pnas.100621910721115835PMC3003006
20

Desai AN, Kraemer MUG, Bhatia S, Cori A, Nouvellet P, Herringer M, et al. Real-time epidemic forecasting: challenges and opportunities. Health Secur. 2019;17(4):268-275.

10.1089/hs.2019.002231433279PMC6708259
21

Semenza JC, Sudre B, Miniota J, Rossi M, Hu W, Kossowsky D, et al. International dispersal of dengue through air travel: importation risk for Europe. PLoS Negl Trop Dis. 2014;8(12):e3278.

10.1371/journal.pntd.000327825474491PMC4256202
22

Bhatia S, Lassmann B, Cohn E, Desai AN, Carrion M, Kraemer MUG, et al. Using digital surveillance tools for near real-time mapping of the risk of infectious disease spread. NPJ Digit Med. 2021;4(1):73.

10.1038/s41746-021-00442-333864009PMC8052406
23

Perera N, Dehmer M, Emmert-Streib F. Named entity recognition and relation detection for biomedical information extraction. Front Cell Dev Biol. 2020;8:673.

10.3389/fcell.2020.0067332984300PMC7485218
24

Aggarwal CC, Zhai C. Mining text data. Berlin/Heidelberg: Springer Science & Business Media; 2012.

10.1007/978-1-4614-3223-4
25

Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. J Mach Learn Res. 2011;12:2493-2537.

26

Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. (2016). Neural architectures for named entity recognition. 2016 Conf. North Am. Chapter Assoc. Comput. Linguist: Hum. Lang. Technol. San Diego, USA.

10.18653/v1/N16-1030
27

Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee L, et al. (2018). Deep contextualized word representations. 2018 Annu. Conf. North Am. Chapter Assoc. Comput. Linguist: Hum. Lang. Technol. New Orleans, USA.

10.18653/v1/N18-1202
28

Yang Y, Lin H, Yang Z, Zhang Y, Zhao D, Huai S. ADPG: Biomedical entity recognition based on Automatic Dependency Parsing Graph. J Biomed Inform. 2023;140:104317.

10.1016/j.jbi.2023.104317
29

Liu M, Tu Z, Zhang T, Su T, Wang Z. LTP: a new active learning strategy for CRF- based named entity recognition. Neural Process Lett. 2022;54(3):2433-2454.

10.1007/s11063-021-10737-x
30

Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:4171-4186.

31

Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Blog. 2019.

32

Raj Kanakarajan K, Kundumani B, Sankarasubbu M. (2021). BioELECTRA: pretrained biomedical text encoder using discriminators. 20th Workshop Biomed. Lang. Process. Online.

10.18653/v1/2021.bionlp-1.16
33

Trieu HL, Miwa M, Ananiadou S. BioVAE: a pre-trained latent variable language model for biomedical text mining. Bioinformatics. 2022;38(3):872-874.

10.1093/bioinformatics/btab70234636886PMC8756089
34

Gurulingappa H, Mateen-Rajput A, Toldo L. Extraction of potential adverse drug events from medical case reports. J Biomed Semantics. 2012;3(1):15.

10.1186/2041-1480-3-1523256479PMC3599676
35

Szklarczyk D, Franceschini A, Wyder S, Forslund K, Heller D, Huerta-Cepas J, et al. STRING v10: Protein-protein interaction networks, integrated over the tree of life. Nucleic Acids Res. 2015;43:D447-D452.

10.1093/nar/gku100325352553PMC4383874
36

Smith L, Tanabe LK, Ando RJ, Kuo CJ, Chung IF, Hsu CN, et al. Overview of BioCreative II gene mention recognition. Genome Biol. 2008;9 Suppl 2(Suppl 2): S2.

10.1186/gb-2008-9-s2-s218834493PMC2559986
37

Sun C, Yang Z, Wang L, Zhang Y, Lin H, Wang J. Biomedical named entity recognition using BERT in the machine reading comprehension framework. J Biomed Inform. 2021;118:103799.

10.1016/j.jbi.2021.103799
38

Yang X, Zhang H, He X, Bian J, Wu Y. Extracting family history of patients from clinical narratives: exploring an end-to-end solution with deep learning models. JMIR Med Inform. 2020;8(12):e22982.

10.2196/2298233320104PMC7772072
39

Caskey J, McConnell IL, Oguss M, Dligach D, Kulikoff R, Grogan B, et al. Identifying COVID-19 outbreaks from contact-tracing interview forms for public health departments: development of a natural language processing pipeline. JMIR Public Health Surveill. 2022;8(3):e36119.

10.2196/3611935144241PMC8906835
40

Joy M, Krishnaveni M. (2022). A review of epidemic surveillance systems for infectious diseases. 2022 Int. Conf. Comput. Commun. Secur. Intell. Syst. (IC3SIS). Kochi, India.

10.1109/IC3SIS54991.2022.9885291
41

Dellanzo A, Cotik V, Lozano Barriga DY, Mollapaza Apaza JJ, Palomino D, Schiaffino F, et al. Digital surveillance in Latin American diseases outbreaks: information extraction from a novel Spanish corpus. BMC Bioinformatics. 2022;23(1):558.

10.1186/s12859-022-05094-y36564712PMC9780622
42

Gu D, He J, Sun J, Shi X, Ye Y, Zhang Z, et al. The global infectious diseases epidemic information monitoring system: development and usability study of an effective tool for travel health management in China. JMIR Public Health Surveill. 2021;7(2):e24204.

10.2196/2420433591286PMC7925143
43

Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234-1240.

10.1093/bioinformatics/btz68231501885PMC7703786
44

Doğan RI, Leaman R, Lu Z. NCBI disease corpus: a resource for disease name recognition and concept normalization. J Biomed Inform. 2014;47:1-10.

10.1016/j.jbi.2013.12.00624393765PMC3951655
45

Li J, Sun Y, Johnson RJ, Sciaky D, Wei CH, Leaman R, et al. BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database. 2016;2016:baw068.

10.1093/database/baw06827161011PMC4860626
46

Kim J, Ohta T, Tsuruoka Y, Tateisi Y, Collier N. (2004). Introduction to the bio-entity recognition task at JNLPBA. Int. Joint Workshop Nat. Lang. Process. Biomed. Appl. Geneva, Switzerland.

10.3115/1567594.1567610
47

Gerner M, Nenadic G, Bergman CM. Linnaeus: a species name identification system for biomedical literature. BMC Bioinformatics. 2010;11:85.

10.1186/1471-2105-11-8520149233PMC2836304
48

Beltagy I, Cohan A, Lo K. (2019). SciBERT: A pre-trained language model for scientific text. 2019 Conf. Empir. Methods Nat. Lang. Process. & 9th Int. Joint Conf. Nat. Lang. Process. Hong Kong.

10.18653/v1/D19-1371
49

Ammar W, Groeneveld D, Bhagavatula C, Beltagy I, Crawford M, Downey D, et al. (2018). Construction of the literature graph in Semantic Scholar. 2018 Conf. North Am. Chapter Assoc. Comput. Linguist: Hum. Lang. Technol. New Orleans, USA.

10.18653/v1/N18-3011
50

Krallinger M, Rabal O, Akhondi SA, Pérez MP, Santamaría J, Rodríguez GP, et al. (2017). Overview of the BioCreative VI chemical-protein interaction track. 6th BioCreative Challenge Eval. Workshop. Bethesda, USA.

51

Nye B, Li J, Patel R, Yang Y, Marshall I, Nenkova A, et al. (2018). A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. 56th Annu. Meet. Assoc. Comput. Linguist. Melbourne, Australia.

10.18653/v1/P18-1019
52

Ji Y, Zhou Z, Liu H, Davuluri RV. DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome. Bioinformatics. 2021;37(15):2112-2120.

10.1093/bioinformatics/btab08333538820PMC11025658
53

Huang K, Altosaar J, Ranganath R. (2020). ClinicalBERT: modeling clinical notes and predicting hospital readmission. ACM Conf. Health, Inference, Learn. Toronto, Canada.

54

Mulyar A, Uzuner O, McInnes B. MT-clinical BERT: scaling clinical information extraction with multitask learning. J Am Med Inform Assoc. 2021;28(10):2108-2115.

10.1093/jamia/ocab12634333635PMC8449623
55

Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput Healthc. 2021;3(1):1-23.

10.1145/3458754
56

Kjeldgaard L, Nielsen LC. NERDA GitHub repository. NERDA. Available at https://github.com/ebanalyse/NERDA [accessed on 18 September 2025].

57

Sang ETK, De Meulder F. (2003). Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. 7th Conf. Nat. Lang. Learn. at HLT-NAACL. Edmonton, Canada.

58

European Centre for Disease Prevention and Control. ECDC Communicable Disease Threats Report (CDTR). Available at https://www.ecdc.europa.eu/en/publications-data [accessed on 18 September 2025].

59

European Centre for Disease Prevention and Control. ECDC communicable disease threats report (CDTR). Week 18, 30 April – 6 May 2023. Available at https://www.ecdc.europa.eu/en/publications-data/communicable-disease-threats- report-30-april-6-may-2023-week-18 [accessed on 18 September 2025].

60

Pan American Health Organization, World Health Organization. COVID-19 region of the Americas update. Available at https://www.paho.org/en/covid-19-global-and-regional-daily-update [accessed on 18 September 2025].

61

Pan American Health Organization, World Health Organization. COVID-19 Daily Update: 4 Feb 2022. Available at https://iris.paho.org/handle/10665.2/55823 [accessed on 18 September 2025].

62

World Health Organization. Regional Office for the Western pacific. (2015). Dengue Situation Updates 2015. WHO Regional Office for the Western Pacific. Available at https://iris.who.int/handle/10665/274097 [accessed on 18 September 2025].

63

World Health Organization. Regional Office for the Western Pacific. (2023)‎. Dengue Situation Updates 2023. WHO Regional Office for the Western Pacific. Available at https://apps.who.int/iris/handle/10665/365676 [accessed on 18 September 2025].

64

Rasmy L, Xiang Y, Xie Z, Tao C, Zhi D. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digit Med. 2021;4(1):86.

10.1038/s41746-021-00455-y34017034PMC8137882
65

Murray Valley encephalitis - Australia (04): (VICTORIA) fatal. ProMED-mail. 2023 Mar 7. Available at https://www.promedmail.org/alert/8708785promedmail.org/alert/8708785 [accessed on 18 September 2025].

66

Meningitis - Niger. World Health Organization. 2023 Feb 8. Available at https://www.who.int/emergencies/disease-outbreak-news/item/2023-DON439 [accessed on 18 September 2025].

67

Lahariya C, Thakur A, Dudeja N. Monkeypox disease outbreak (2022): epidemiology, challenges, and the way forward. Indian Pediatr. 2022;59(8):636-642.

10.1007/s13312-022-2578-235762024PMC9419123
68

Multi-country monkeypox outbreak: situation update. World Health Organization. 2022 Jun 10. Available at https://www.who.int/emergencies/disease-outbreak-news/item/2022-DON392 [accessed on 18 September 2025].

69

Tambo E, Al-Nazawi AM. Combating the global spread of poverty-related Monkeypox outbreaks and beyond. Infect Dis Poverty. 2022;11(1):80.

10.1186/s40249-022-01004-935794644PMC9261034
70

Patel M, Surti M, Adnan M. Artificial intelligence (AI) in Monkeypox infection prevention. J Biomol Struct Dyn. 2023;41(17):8629-8633.

10.1080/07391102.2022.213421436218112PMC9627635
71

Survice-BERT. GitHub. Available at https://github.com/csanny/Survice-BERT [accessed on 18 September 2025].

Information
  • Publisher :The Korean Society for Microbiology and The Korean Society of Virology
  • Publisher(Ko) :대한미생물학회‧대한바이러스학회
  • Journal Title :JOURNAL OF BACTERIOLOGY AND VIROLOGY
  • Volume : 55
  • No :3
  • Pages :235-247
  • Received Date : 2025-04-15
  • Revised Date : 2025-09-01
  • Accepted Date : 2025-09-19
  • DOI :https://doi.org/10.4167/jbv.2025.55.3.235
Journal Informaiton JOURNAL OF BACTERIOLOGY AND VIROLOGY JOURNAL OF BACTERIOLOGY AND VIROLOGY
  • NRF
  • 임플란트학회 전용 배너
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close