Abstract
Table-to-text generation task refers to converting tabular data into language text to facilitate easier understanding and analysis of the table. Recently, pre-trained models have made significant advancements in this kind of tasks. However, the inherent structural differences between tabular data and text, and the lack of domain-specific knowledge in few-shot datasets, make it challenging for pre-trained models to generate faithful text. To solve these problems, we proposed a framework that encodes tables by obtaining structural bias attention through pruning full self-attention, distinguishing the importance of cells from a structural perspective. We use the pre-trained model with the structural bias framework to the generation component of Prototype-to-Generation. To encourage prototype memory to adhere to the table content and generate more accurate and aligned sentences, we employ Reinforcement Learning. We conducted extensive experiments on three few-shot table datasets. Compared to previous advanced methods, our model achieved superior performance across multiple evaluation metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lebret, R., Grangier, D., Auli, M.: Neural text generation from structured data with application to the biography domain. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2016)
Hasan, S.A., Farri, O.: Clinical natural language processing with deep learning. Data Sci. Healthcare: Methodol. Appli., 147–171 (2019)
Li, Y., Li, W., Nie, L.: MMCoQA: conversational question answering over text, tables, and images. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4220–4231 (2022)
Liu, T., Wang, K., Sha, L., et al.: Table-to-text generation by structure-aware seq2seq learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32(1) (2018)
Su, Y., Meng, Z., Baker, S., et al.: Few-shot table-to-text generation with prototype memory. In: Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 910–917 (2021)
Vaswani, A., Shazeer, N,, Parmar, N., et al.: Attention is all you need. Adv. Neural Inform. Proc. Syst. 30 (2017)
Papineni, K., Roukos, S., Ward, T., et al.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318 (2002)
Chen Z, Eavani H, Chen W, et al. Few-Shot NLG with Pre-trained language model[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 183–190
Gong, H., Sun, Y., Feng, X., et al.: Tablegpt: few-shot table-to-text generation with table structure reconstruction and content matching. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 1978–1988 (2020)
Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582–4597 (2021)
Zhao, W., Liu, Y., Wan, Y., Yu, P.: Attend, memorize and generate: towards faithful table-to-text generation in few shots. In: Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, pp. 4106–4117. Association for Computational Linguistics (2021)
Luo, Y., Lu, M., Liu, G., Wang, S.: Few-shot table-to-text generation with prefix-controlled generator. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 6493–6504 (2022)
Raffel, C., Shazeer, N., Roberts, A., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)
Dhingra, B., Faruqui, M., Parikh, A., Chang, M.-W., Das, D., Cohen, W.: Handling divergent reference texts when evaluating table-to-text generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4884–4895 (2019)
Acknowledgments
This work is supported by National Natural Science Foundation of China (Nos.62066033, 61966025); Inner Mongolia Key Research and Development Fund Project (No. 2023YFSW0001); Inner Mongolia Autonomous Region Over-seas Students Innovation and Entrepreneurship Startup Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, D., Wang, W., Bao, F., Gaov, G. (2024). Few-Shot Table-to-Text Generation with Structural Bias Attention. In: Liu, F., Sadanandan, A.A., Pham, D.N., Mursanto, P., Lukose, D. (eds) PRICAI 2023: Trends in Artificial Intelligence. PRICAI 2023. Lecture Notes in Computer Science(), vol 14326. Springer, Singapore. https://doi.org/10.1007/978-981-99-7022-3_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-7022-3_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7021-6
Online ISBN: 978-981-99-7022-3
eBook Packages: Computer ScienceComputer Science (R0)