Abstract
Since the launch of various generative AI tools, scientists have been striving to evaluate their capabilities and contents, in the hope of establishing trust in their generative abilities. Regulations and guidelines are emerging to verify generated contents and identify novel uses. we aspire to demonstrate how ChatGPT claims are checked computationally using the rigor of network models. We aim to achieve fact-checking of the knowledge embedded in biological graphs that were contrived from ChatGPT contents at the aggregate level. We adopted a biological networks approach that enables the systematic interrogation of ChatGPT’s linked entities. We designed an ontology-driven fact-checking algorithm that compares biological graphs constructed from approximately 200,000 PubMed abstracts with counterparts constructed from a dataset generated using the ChatGPT-3.5 Turbo model. In 10-samples of 250 randomly selected records a ChatGPT dataset of 1000 “simulated” articles , the fact-checking link accuracy ranged from 70% to 86%. This study demonstrated high accuracy of aggregate disease-gene links relationships found in ChatGPT-generated texts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
OpenAI. ChatGPT: Conversational ai assistant. OpenAI Platform (2023). Accessed 14 Aug 2023
Van Dis, E.A., Bollen, J., Zuidema, W., Van Rooij, R., Bockting, C.L.: ChatGPT: five priorities for research. Nature 614(7947), 224–226 (2023)
Przymusinski, T.C.: An algorithm to compute circumscription. Artif. Intell. 38(1), 49–73 (1989)
Query rewriting for ontology-mediated conditional answers (2020)
Torralba, A., Efros, A.A.: Unbiased look at dataset bias (2011)
Minker, J.: On indefinite databases and the closed world assumption. In: Loveland, D.W. (ed.) CADE 1982. LNCS, vol. 138, pp. 292–308. Springer, Heidelberg (1982). https://doi.org/10.1007/BFb0000066
Tchechmedjiev, A., et al.: CLAIMSKG: a knowledge graph of fact-checked claims. Semant. Web - ISWC 11779, 2019 (2019)
Vedula, N., Parthasarathy, S.: Face-keg: fact checking explained using knowledge graphs. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 526–534 (2021)
Lin, P., Song, Q., Shen, J., Wu, Y.: Discovering graph patterns for fact checking in knowledge graphs. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds.) DASFAA 2018. LNCS, vol. 10827, pp. 783–801. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91452-7_50
Lin, P., Song, Q., Yanhong, W.: Fact checking in knowledge graphs with ontological subgraph patterns. Data Sci. Eng. 3, 341–358 (2018)
Lin, P., Song, Q., Yinghui, W., Pi, J.: Discovering patterns for fact checking in knowledge graphs. J. Data Inf. Qual. (JDIQ) 11(3), 1–27 (2019)
Ciampaglia, G.L., Shiralkar, P., Rocha, L.M., Bollen, J., Menczer, F., Flammini, A.: Computational fact checking from knowledge networks. PloS One 10(6), e0128193 (2015)
Wang, S., Wang, L., Mao, W.: A kg-based enhancement framework for fact checking using category information. In: 2020 IEEE International Conference on Intelligence and Security Informatics (ISI), pp. 1–6. IEEE (2020)
Khandelwal, S., Kumar, D.: Computational fact validation from knowledge graph using structured and unstructured information. In: Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp. 204–208 (2020)
Orthlieb, T., Abdessalem, H.B., Frasson, C.: Checking method for fake news to avoid the twitter effect. In: Cristea, A.I., Troussas, C. (eds.) ITS 2021. LNCS, vol. 12677, pp. 68–72. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-80421-3_8
Shi, B., Weninger, T.: Proje: embedding projection for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI 2017, pp. 1236–1242. AAAI Press (2017)
Wang, S., Mao, W., Wei, P., Zeng, D.D.: Knowledge structure driven prototype learning and verification for fact checking. Knowl.-Based Syst. 238, 107910 (2022)
Mengoni, P., Yang, J.: Empowering covid-19 fact-checking with extended knowledge graphs. In: International Conference on Computational Science and its Applications, pp. 138–150. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-10536-4_10
Kim, J., Choi, K.S.: Unsupervised fact checking by counter-weighted positive and negative evidential paths in a knowledge graph. In: Proceedings of the 28th International Conference on Computational Linguistics (2020)
Kim, J.S., Choi, K.S.: Fact checking in knowledge graphs by logical consistency. Semantic Web J. swj2721 (2021)
Zhu, B., Zhang, X., Gu, M., Deng, Y.: Knowledge enhanced fact checking and verification. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 3132–3143 (2021)
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., Wu, X.: A roadmap, Unifying large language models and knowledge graphs (2023)
Yang, L., Chen, H., Li, Z., Ding, X., Wu, X.: Chatgpt is not enough: enhancing large language models with knowledge graphs for fact-aware language modeling (2023)
Pubmed central (pmc). Accessed 2 Sept 2023
Hofer, P., Neururer, S., Goebel, G.: Semi-automated annotation of biobank data using standard medical terminologies in a graph database, vol. 228 (2017)
Sow, A., Guissé, A., Niang, O.: Enrichment of medical ontologies from textual clinical reports: towards improving linking human diseases and signs. In: Bassioni, G., Kebe, C.M.F., Gueye, A., Ndiaye, A. (eds.) InterSol 2019. LNICST, vol. 296, pp. 104–115. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34863-2_10
Huntley, R.P., et al.: The GOA database: gene ontology annotation updates for 2015. Nucleic Acids Res. 43(D1), D1057–D1063 (2015)
Gene ontology annotations and resources. Nucleic Acids Res. 41 (2013)
Camon, E., et al.: The gene ontology annotation (goa) project: implementation of go in swiss-prot, trembl, and interpro (2003)
Acknowledgments
This research is supported by the European Union’s Horizon 2020 research and innovation programme under grant agreement Sano No 857533 and carried out within the International Research Agendas programme of the Foundation for Polish Science, co-financed by the European Union under the European Regional Development Fund, Additionally, is created as part of the Ministry of Science and Higher Education’s initiative to support the activities of Excellence Centers established in Poland under the Horizon 2020 program based on the agreement No MEiN/2023/DIR/3796’
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hamed, A.A., Crimi, A., Lee, B.S., Misiak, M.M. (2024). Fact-Checking Generative AI: Ontology-Driven Biological Graphs for Disease-Gene Link Verification. In: Franco, L., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2024. ICCS 2024. Lecture Notes in Computer Science, vol 14835. Springer, Cham. https://doi.org/10.1007/978-3-031-63772-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-63772-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-63771-1
Online ISBN: 978-3-031-63772-8
eBook Packages: Computer ScienceComputer Science (R0)