JISARA

Abstract: Applications of natural language processing (NLP) for use in large language models (LLMs) continue to evolve with technological advancements in the domain Generative AI (GenAI). The massive explosion of data, availability of scalable computing capacity and machine learning innovation, LLMs, have all led towards Generative AI (GenAI) becoming increasingly popular. A major challenge involved with base model LLMs is their tendency to hallucinate. This occurs as most LLMs are trained on a large amount of generic data and must be augmented using domain specific and external data for use in GenAI tasks such as chatbots, Q&A, summarization and for text generation. To address the challenge of hallucination, this study will make use of domain specific healthcare data, in the form of PDF files, alongside an FM to create a Retrieval Augmented Generation (RAG) chatbot. This study makes use of the base foundation model, Llama 2. Our domain specific healthcare data was sourced from relevant and reliable sources. The RAG chatbot was developed using Python and colab notebook and responses were evaluated using Rouge and Meteor, evaluation metrics for automatically generated text. The evaluation was based on three scenarios: responses less than 250 characters, more than 250 characters and combined responses from multiple LLMs. Our findings provide strong evidence that augmenting FMs with domain specific data can improve the quality of the models’ responses in providing reliable medical knowledge to patients.

Download this article: JISARA - V18 N3 Page 18.pdf

Recommended Citation: Richard-Ojo, O., Wimmer, H., Rebman Jr., C.M., (2025). RAG Chatbot for Healthcare related prompts using Amazon Bedrock. Journal of Information Systems Applied Research and Analytics 18(3) pp 18-29. https://doi.org/10.62273/RQAT8911