SpringerLink
Log in

Enhancing Document-Level Relation Extraction with Attention-Convolutional Hybrid Networks and Evidence Extraction

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Document-level relation extraction aims at extracting relations between entities in a document. In contrast to sentence-level correspondences, document-level relation extraction requires reasoning over multiple sentences to extract complex relational triples. Recent work has found that adding additional evidence extraction tasks and using the extracted evidence sentences to help predict can improve the performance of document-level relation extraction tasks, however, these approaches still face the problem of inadequate modeling of the interactions between entity pairs. In this paper, based on the review of human cognitive processes, we propose a hybrid network HIMAC applied to the entity pair feature matrix, in which the multi-head attention sub-module can fuse global entity-pair information on a specific inference path, while the convolution sub-module is able to obtain local information of adjacent entity pairs. Then we incorporate the contextual interaction information learned by the entity pairs into the relation prediction and evidence extraction tasks. Finally, the extracted evidence sentences are used to further correct the relation extraction results. We conduct extensive experiments on two document-level relation extraction benchmark datasets (DocRED/Re-DocRED), and the experimental results demonstrate that our method achieves state-of-the-art performance (62.84/75.89 F1). Experiments demonstrate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The data that support the findings of this study are openly available in DocRED at https://github.com/thunlp/DocRED.

References

  1. Zeng D, Liu K, Lai S, Zhou G, Zhao J. Relation classification via convolutional deep neural network. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers; 2014 p. 2335-44.

  2. Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, et al. Attention-based bidirectional long short-term memory networks for relation classification; 2016 p. 207-12.

  3. Zhang Y, Qi P, Manning C. Graph convolution over pruned dependency trees improves relation extraction. Assoc Comput Linguist; 2018 p. 2205-15.

  4. Wei Z, Su J, Wang Y, Tian Y, Chang Y. A novel cascade binary tagging framework for relational triple extraction. Assoc Comput Linguist; 2020 p. 1476-88.

  5. Wang H, Qin K, Lu G, Luo G, Liu G. Direction-sensitive relation extraction using Bi-SDP attention model. Knowl-Based Syst. 2020;21(198): 105928.

    Article  Google Scholar 

  6. Tang R, Chen Y, Huang R, Qin Y. Enhancing interaction representation for joint entity and relation extraction. Cogn Syst Res. 2023;1(82): 101153.

    Article  Google Scholar 

  7. Cheng Q, Liu J, Qu X, Zhao J, Liang J, Wang Z, et al. HacRED: a large-scale relation extraction dataset toward hard cases in practical applications; 2021 p. 2819-31.

  8. Wang H, Qin K, Zakari RY, Lu G, Yin J. Deep neural network-based relation extraction: an overview. Neural Comput Appl. 2022;34(6):4781–801.

    Article  Google Scholar 

  9. Nayak T, Majumder N, Goyal P, Poria S. Deep neural approaches to relation triplets extraction: a comprehensive survey. Cogn Comput. 2021;13(5):1215–32.

    Article  Google Scholar 

  10. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, et al. DocRED: a large-scale document-level relation extraction dataset. ACLWeb. Florence, Italy: Assoc Comput Linguist; 2019. p. 764-77.

  11. Nan G, Guo Z, Sekulić I, Lu W. Reasoning with latent structure refinement for document-level relation extraction. Assoc Comp Linguist; 2020 p. 1546-57.

  12. Wang D, Hu W, Cao E. Global-to-local neural networks for document-level relation extraction. Assoc Comput Linguist; 2020 p. 3711-21.

  13. Peng N, Poon H, Quirk C, Toutanova K, Yih W. Cross-sentence N-ary relation extraction with graph LSTMs. Trans Assoc Comput Linguist. 2017;5(5):101–15.

    Article  Google Scholar 

  14. Christopoulou F, Miwa M. Ananiadou S. Connecting the dots: document-level neural relation extraction with edge-oriented graphs; 2019. p. 4925–36.

  15. Guo Z, Zhang Y, Lu W. Attention guided graph convolutional networks for relation extraction. 2019 p. 241-51.

  16. Wang H, Focke C, Sylvester R, Mishra NK, Wang WY. Fine-tune Bert for DocRED with two-step process. ar**v:1706.02677 [cs.CL]. 2019 Sep 26;

  17. Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, et al. HIN: hierarchical inference network for document-level relation extraction. Advances in Knowledge Discovery and Data Mining. 2020;197-209.

  18. Zhou W, Huang K, Ma T, Huang J. Document-level relation extraction with adaptive thresholding and localized context pooling. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14612–20.

    Article  Google Scholar 

  19. Zhang N, Chen X, **e X, Deng S, Tan C, Chen M, et al. Document-level relation extraction as semantic segmentation. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence; 2021.

  20. **e Y, Shen J, Li S, Mao Y, Han J. Eider: empowering document-level relation extraction with efficient evidence extraction and inference-stage fusion. Findings of the Association for Computational Linguistics: ACL. 2022;2022(1):257–68.

    Google Scholar 

  21. Tan Q, Li X, Bing L, Hwee Tou Ng, Aljunied SM. Revisiting DocRED - addressing the false negative problem in relation extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022;8472-87.

  22. Sahu SK, Christopoulou F, Miwa M, Ananiadou S. Inter-sentence relation extraction with document-level graph convolutional neural network. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019 Jul;4309-16.

  23. Quirk C, Poon H. Distant supervision for relation extraction beyond the sentence boundary. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017 Apr;1171-82.

  24. Jia R, Wong C, Poon H. Document-level N-ary relation extraction with multiscale representation learning. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019 Jun;1:3693-704.

  25. Zeng S, Wu Y, Chang B. SIRE: separate intra-and inter-sentential reasoning for document-level relation extraction. Findings of the Association for Computational Linguistics: ACL-IJCNLP. 2021;2021:524–34.

    Google Scholar 

  26. Zeng S, Xu R, Chang B, Li L. Double graph based reasoning for document-level relation extraction. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020 Nov;1630-40.

  27. Xu W, Chen K, Zhao T. Document-level relation extraction with reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14167–75.

    Article  Google Scholar 

  28. Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z. Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(16):14149–57.

  29. Tan Q, He R, Bing L, Hwee Tou Ng. Document-level relation extraction with adaptive focal loss and knowledge distillation. Findings of the Association for Computational Linguistics: ACL 2022. 2022 May;1672-81.

  30. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30.

  31. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI. 2015;2015(9351):234–41.

    Google Scholar 

  32. Pan X, Ge C, Lu R, Song S, Chen G, Huang Z, et al. On the integration of self-attention and convolution. https://openaccess.thecvf.com/. 2022. p. 815-25.

  33. Wolpert DH. Stacked generalization. Neural Netw. 1992;5(2):241–59.

    Article  Google Scholar 

  34. Devlin J, Chang MW, Lee K, Google K, Language A. BERT: pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT. 2019;2019:4171–86.

    Google Scholar 

  35. Loshchilov I, Hutter F. Decoupled weight decay regularization. International Conference on Learning Representations.; 2019.

  36. Goyal P, Dollár P, Girshick R, Noordhuis P, Wesolowski L, Kyrola A, et al. Accurate, large minibatch SGD: training ImageNet in 1 Hour. ar**v:1706.02677 [cs]. 2018 Apr 30;

  37. Li B, Ye W, Sheng Z, **e R, ** X, Zhang S. Graph enhanced dual attention network for document-level relation extraction. Proceedings of the 28th International Conference on Computational Linguistics; 2020 Dec p. 1551-60.

  38. Xu W, Chen K, Zhao T. Discriminative reasoning for document-level relation extraction. Findings of the Association for Computational Linguistics: ACL-IJCNLP. 2021;2021:1653–63.

    Google Scholar 

  39. Huang K, Qi P, Wang G, Ma T, Huang J. Entity and evidence guided document-level relation extraction. Rogers A, Calixto I, Vulić I, Saphra N, Kassner N, Camburu OM, et al., editors. Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021). 2021 Aug 1;307-15.

  40. Ye D, Lin Y, Du J, Liu Z, Li P, Sun M, et al. Coreferential reasoning learning for language representation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020 Nov;7170-86.

  41. Jiang F, Niu J, Mo S, Fan S. Key mention pairs guided docu-ment-level relation extraction. Calzolari N, Huang CR, Kim H, Pustejovsky J, Wanner L, Choi KS, et al., editors. Proceedings of the 29th International Conference on Computational Linguistics. 2022 Oct;1904-14.

Download references

Funding

This work was supported by the National Natural Science Foundation of China (No. U19A2059) and the 2022 Chengdu Textile College Scientific Research Foundation (No. X22032161).

Author information

Authors and Affiliations

  1. Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, China

    Feiyu Zhang & Ruiming Hu

  2. Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China, Chengdu, China

    Feiyu Zhang, Ruiming Hu & Guiduo Duan

  3. Trusted Cloud Computing and Big Data Key Laboratory of Sichuan Province, Chengdu, China

    Guiduo Duan

  4. Department of Fundamental Courses, Chengdu Textile College, Chengdu, China

    Tianxi Huang

Authors
  1. Feiyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruiming Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guiduo Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianxi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Guiduo Duan or Tianxi Huang.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interest

Guiduo Duan has received research grants from the National Natural Science Foundation of China (No. U19A2059). Tianxi Huang has received research grants from the 2022 Chengdu Textile College Scientific Research Foundation(No. X22032 161). Feiyu Zhang and Ruiming Hu declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, F., Hu, R., Duan, G. et al. Enhancing Document-Level Relation Extraction with Attention-Convolutional Hybrid Networks and Evidence Extraction. Cogn Comput 16, 1113–1124 (2024). https://doi.org/10.1007/s12559-024-10269-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-024-10269-1

Keywords

Search

Navigation