Evaluating AI-Generated Citation Accuracy: An Empirical Framework and Verification Efficiency Matrix for Academic Librarians
DOI:
https://doi.org/10.63300/arjst0503062026.01Keywords:
AI Hallucinations, Academic Integrity, Generative AI, Information Literacy, SIFT Method, Ghost Citations, Academic LibrarianshipAbstract
Using the widespread adoption of Generative AI tools, there is a risk of “Reference Hallucination” which threatens academic integrity. This study evaluates the structural correctness of citations generated by Large Language Models (LLMs) and presents an empirical verification matrix for academic librarians. The study investigates the hallucination rate and maps the verification efficiency of traditional databases (Scopus, JSTOR) against search-augmented AI tools (Elicit, Perplexity) through the analysis of a 100 AI-generated citation simulation matrix across four multidisciplinary fields.. The findings suggest a “Verify-and-Validate” structure workflow using the SIFT method to reduce the invisible workload on the library services.
Downloads
References
[1]. The Economics Times Online. Last Updated: May 26, 2026, 05:56:00 PM IST Nearly 1.46 lakh AI-hallucinated references entered scientific papers in 2025: Study https://economictimes.indiatimes.com/news/new-updates/nearly-1-46-lakh-ai-hallucinated-references-entered-scientific-papers-in-2025-study/articleshow/131329519.cms
[2]. Research Papers
[3]. Topaz M, Roguin N, Gupta P et al. Fabricated citations: an audit across 2·5 million biomedical papers. The Lancet, 407, 1779-1781 https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(26)00603-3/fulltext
[4]. Diletta Abbonato. CheckIfExist: Detecting Citation Hallucinations in the Era of AI-Generated Content.
Published in arXiv.org 27 January 2026. DOI:10.48550/arXiv.2602.15871
[5]. Association of College and Research Libraries. (2025). AI Competencies for Academic Library Workers Approved by the ACRL Board of Directors, October 2025 https://www.ala.org/sites/default/files/2025-10/acrl_ai_competencies.pdf
[6]. Bender, E. M, Gebru, T., McMillan-Major, A., & Shmitchell, S. On the Dangers of Stochastic Parrots:Can Language Models Be Too Big? https://dl.acm.org/doi/epdf/10.1145/3442188.3445922
[7]. Caulfield, M. (2017). Web literacy for student fact-checkers. pressbooks.com https://digitalcommons.liberty.edu/cgi/viewcontent.cgi?article=1004&context=textbooks
[8]. Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228–239. https://doi.org/10.1080/14703297.2023.2190148
[9]. Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Wheeler, S., Leiter, M. A., Burgess, M. M., & Checketts, J. X. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detector software and blind human reviewers. NPJ Digit Med. 2023 Apr 26;6:75. doi: 10.1038/s41746-023-00819-6
[10]. OpenAI. ChatGPT: Optimizing language models for dialogue. OpenAIhttps://openai.com/blog/chatgpt/ (2022).
[11]. Shankland, S. ChatGPT: Why everyone is obsessed this mind-blowing AI chatbot. CNEThttps://www.cnet.com/tech/computing/chatgpt-why-everyone-is-obsessed-this-mind-blowing-ai-chatbot/ (2022).
[12]. GPT-2 Output Detector. https://huggingface.co/openai-detector. Accessed December 2022.
[13]. Korngiebel DM, Mooney SD. Considering the possibilities and pitfalls of Generative Pre-trained Transformer 3 (GPT-3) in healthcare delivery. 2021, https://doi.org/10.1038/s41746-021-00464-x
[14]. Thorp HH. ChatGPT is fun, but not an author. Science. 2023;379:313. https://doi.org/10.1126/science.adg7879
[15]. Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A. & Testrow, C. The death of the short-form Physics essay in the coming AI revolution. 2022. DOI:10.48550/arXiv.2212.11661
[16]. Kung TH, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit. Health. 2023 Feb 9;2(2):e0000198 DOI: 10.1371/journal.pdig.0000198
[17]. Halaweh, M. (2023). ChatGPT in education: Strategies for responsible implementation. Contemporary Educational Technology, 15(2), ep421. https://doi.org/10.30935/cedtech/13036
[18]. Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26-29. https://doi.org/10.1108/LHTN-01-2023-0009
[19]. Jamaluddin J, Ga¬ar NA, Din NSS. Hallucination: A key challenge to Arti cial Intelligence-Generated writing. Malays Fam Physician. 2023;18:68. https://doi.org/10.51866/lte.527
[20]. Abbonato Diletta,CheckIfExist: Detecting Citation Hallucinations in the Era of AI-Generated Content. 2026. https://doi.org/10.48550/arXiv.2602.15871
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Articles published in the Academic Research Journal of Science and Technology (ARJST will be Open-Access articles distributed under the terms and conditions of the Creative Commons Attribution-Noncommercial 4.0 International (CC BY-NC 4.0). This allows for immediate free access to the work and permits any user to read, download, copy, distribute, print, search, or link to the full texts of articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose.
This open-access article is distributed under the terms and conditions of the Creative Commons Attribution-Noncommercial 4.0 International (CC BY-NC 4.0).
How to Cite
