Abstract
In this paper, we propose a simple and effective few-shot named entity recognition (NER) method for biomedical domain, called PEMRC (Positive Enhanced Machine Reading Comprehension). PEMRC is based on the idea of using machine reading comprehension reading comprehension (MRC) framework to perfome few-shot NER and fully exploit the prior knowledge implied in the label information. On one hand, we design three different query templates to better induce knowledge from pre-trained language models (PLMs). On the other hand, we design a positive enhanced loss function to improve the model’s accuracy in identifying the start and end positions of entities under low-resources scenarios. Extensive experimental results on eight benchmark datasets in biomedical domain show that PEMRC significantly improves the performance of few-shot NER.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, T.B., et al.: Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20, Curran Associates Inc., Red Hook, NY, USA (2020)
Chen, X., et al.: LightNER: A lightweight tuning paradigm for low-resource NER via pluggable prompting. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2374–2387. Gyeongju, Republic of Korea (2022)
Chiu, J.P., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. Trans. Assoc. Comput. linguist. 4, 357–370 (2016)
Cui, L., Wu, Y., Liu, J., Yang, S., Zhang, Y.: Template-based named entity recognition using BART. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1835–1845. Association for Computational Linguistics (2021)
Das, S.S.S., Katiyar, A., Passonneau, R., Zhang, R.: CONTaiNER: few-shot named entity recognition via contrastive learning. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6338–6353. Association for Computational Linguistics, Dublin, Ireland (2022)
Ding, N., et al.: Few-NERD: a few-shot named entity recognition dataset. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3198–3213. Association for Computational Linguistics (2021)
Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993–1000 (2019)
Gao, T., Fisch, A., Chen, D.: Making pre-trained language models better few-shot learners. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3816–3830. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-long.295. https://aclanthology.org/2021.acl-long.295
Han, X., Zhao, W., Ding, N., Liu, Z., Sun, M.: PTR: prompt tuning with rules for text classification. Ar**v abs/2105.11259 (2021). https://api.semanticscholar.org/CorpusID:235166723
Hou, Y., et al.: Few-shot slot tagging with collapsed dependency transfer and label-enhanced task-adaptive projection network. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1381–1393. Association for Computational Linguistics (2020)
Huang, Y., et al.: COPNER: contrastive learning with prompt guiding for few-shot named entity recognition. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2515–2527. International Committee on Computational Linguistics, Gyeongju, Republic of Korea (2022)
Lee, J., et al: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2019)
Li, J., Shang, S., Shao, L.: MetaNER: named entity recognition with meta-learning. In: Proceedings of The Web Conference 2020, pp. 429–440. WWW ’20, Association for Computing Machinery, New York, NY, USA (2020)
Li, J., Sun, A., Han, J., Li, C.: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 34(1), 50–70 (2020)
Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582–4597. Association for Computational Linguistics (2021)
Liu, S., Zhang, X., Zhang, S., Wang, H., Zhang, W.: Neural machine reading comprehension: methods and trends. Ar**v abs/1907.01118 (2019)
Ma, R., et al.: Template-free prompt tuning for few-shot NER. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5721–5732. Association for Computational Linguistics, Seattle, United States (2022)
Nguyen, N.D., Du, L., Buntine, W., Chen, C., Beare, R.: Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 4063–4071. Abu Dhabi, United Arab Emirates (2022)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)
Schick, T., Schütze, H.: Exploiting cloze-questions for few-shot text classification and natural language inference. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 255–269. Association for Computational Linguistics (2021)
Schick, T., Schütze, H.: It’s not just size that matters: small language models are also few-shot learners. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2339–2352. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.naacl-main.185. https://aclanthology.org/2021.naacl-main.185
Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: AutoPrompt: eliciting knowledge from language models with automatically generated prompts. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222–4235. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.emnlp-main.346. https://aclanthology.org/2020.emnlp-main.346
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning, pp. 4080–4090. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (2017)
Wiseman, S., Stratos, K.: Label-agnostic sequence labeling by copying nearest neighbors. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5363–5369. Association for Computational Linguistics, Florence, Italy (2019)
Yadav, V., Bethard, S.: A survey on recent advances in named entity recognition from deep learning models. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 2145–2158. Association for Computational Linguistics, Santa Fe, New Mexico, USA (2018)
Yang, Y., Katiyar, A.: Simple and effective few-shot named entity recognition with structured nearest neighbor learning. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6365–6375. Association for Computational Linguistics (2020)
Zhang, Y., Fei, H., Li, D., Li, P.: PromptGen: automatically generate prompts using generative models. In: Findings of the Association for Computational Linguistics: NAACL 2022, pp. 30–37. Association for Computational Linguistics, Seattle, United States (2022). https://doi.org/10.18653/v1/2022.findings-naacl.3. https://aclanthology.org/2022.findings-naacl.3
Funding
This research is supported by the National Natural Science Foundation of China [61976147] and the research grant of The Hong Kong Polytechnical University Projects [# 1-W182].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Dong, Y., Li, D., Gu, J., Qian, L., Zhou, G. (2024). PEMRC: A Positive Enhanced Machine Reading Comprehension Method for Few-Shot Named Entity Recognition in Biomedical Domain. In: Xu, H., et al. Health Information Processing. CHIP 2023. Communications in Computer and Information Science, vol 1993. Springer, Singapore. https://doi.org/10.1007/978-981-99-9864-7_2
Download citation
DOI: https://doi.org/10.1007/978-981-99-9864-7_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9863-0
Online ISBN: 978-981-99-9864-7
eBook Packages: Computer ScienceComputer Science (R0)