Abstract
Since the onset of the COVID-19 pandemic, social media users have shared their personal experiences related to the viral infection. Their posts contain rich information of symptoms that may provide useful hints to advancing the knowledge body of medical research and supplement the discoveries from clinical settings. Identification of symptom expressions in social media text is challenging, partially due to lack of annotated data. In this study, we investigate utilizing few-shot learning with generative pre-trained transformer language models to identify COVID-19 symptoms in Twitter posts. The results of our approach show that large language models are promising in more accurately identifying symptom expressions in Twitter posts with small amount of annotation effort, and our method can be applied to other medical and health applications where abundant of unlabeled data is available.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, E., Lerman, K., Ferrara, E.: Tracking social media discourse about the covid-19 pandemic: development of a public coronavirus twitter data set. JMIR Public Health Surveill. 6(2), e19273 (2020)
Müller, M., Salathé, M., Kummervold, P.E.: COVID-Twitter-BERT: A natural language processing model to analyse covid-19 content on twitter. ar**v preprint ar**v:2005.07503 (2020)
Wijeratne, S., et al.: Feature engineering for Twitter-based applications. In Feature Engineering for Machine Learning and Data Analytics, pp. 359–393 (2018)
Guo, J.W., Radloff, C.L., Wawrzynski, S.E., Cloyes, K.G.: Mining twitter to explore the emergence of COVID-19 symptoms. Public Health Nurs. 37(6), 934–940 (2020)
Krittanawong, C., Narasimhan, B., Virk, H.U.H., Narasimhan, H., Wang, Z., Tang, W.W.: Insights from Twitter about novel COVID-19 symptoms. Eur. Heart J. Digital Health 1(1), 4–5 (2020)
Sarker, A., Lakamana, S., HoggBremer, W., **e, A., AlGaradi, M.A., Yang, Y.C.: Self-reported COVID-19 symptoms on Twitter: an analysis and a research resource. J. Am. Med. Inform. Assoc. 27(8), 1310–1315 (2020)
Jiang, K., Zhu, M., Bernard, G.R.: Discovery of COVID-19 symptomatic experience reported by twitter users. Stud. Health Technol. Inform. 294, 664–668 (2022)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Black, S., et al.: GPT-NeoX-20b: an open-source autoregressive language model. ar**v preprint ar**v:2204.06745 (2022)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2018)
Gao, L., et al.: The pile: an 800gb dataset of diverse text for language modeling. ar**v preprint ar**v:2101.00027 (2020)
Logan IV, R.L., Balažević, I., Wallace, E., Petroni, F., Singh, S., Riedel, S.: Cutting down on prompts and parameters: simple few-shot learning with language models. ar**v preprint ar**v:2106.13353 (2021)
Zhu, M., Song, Y., **, G., Jiang, K.: Identifying personal experience tweets of medication effects using pre-trained RoBERTa language model and its updating. In Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis, pp. 127–137 (2020)
Liu, Y., et al.: RoBERTa: A robustly optimized BERT pretraining approach. ar**v preprint ar**v:1907.11692 (2019)
Demner-Fushman, D., Rogers, W.J., Aronson, A.R.: MetaMap lite: an evaluation of a new Java implementation of MetaMap. J. Am. Med. Inform. Assoc. 24(4), 841–844 (2017)
World Health Organization: Diagnostic testing for SARS-CoV-2 (2020). https://apps.who.int/iris/bitstream/handle/10665/334254/WHO-2019-nCoV-laboratory-2020.6-eng.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, K., Zhu, M., Bernard, G.R. (2023). Few-Shot Learning for Identification of COVID-19 Symptoms Using Generative Pre-trained Transformer Language Models. In: Koprinska, I., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2022. Communications in Computer and Information Science, vol 1753. Springer, Cham. https://doi.org/10.1007/978-3-031-23633-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-23633-4_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23632-7
Online ISBN: 978-3-031-23633-4
eBook Packages: Computer ScienceComputer Science (R0)