UAJY Handbook RAG Chatbot Splits FAISS Search From Gemini

The public UAJY Academic Document RAG Chatbot repository serves as an implementation example for the 2025/2026 academic handbook of Universitas Atma Jaya Yogyakarta's Faculty of Industrial Technology. This chatbot offers a Streamlit assistant that handles document ingestion, local vector search, answer generation, refusal controls, and evaluation metrics all within a single codebase.

The ingestion pipeline begins by extracting text and tables from the handbook PDF using pdfplumber. It then splits the material into chunks, creating 3,072-dimensional embeddings with gemini-embedding-001 before storing these vectors in a FAISS index, alongside metadata about pages and headings. The index contains 350 chunks derived from 112 pages of the handbook.

Upon runtime, a Streamlit query triggers a top-K similarity retrieval from the FAISS index. The application then constructs a prompt from the retrieved context and conversation history, sends this to Gemini 2.5 Flash, and formats the response with page numbers and section titles.

The repository includes controls for unsupported questions: a similarity threshold filters retrieval results, and the system prompt instructs Gemini to refuse requests that the retrieved document context cannot answer. These controls are described as an anti-hallucination defense, emphasizing that generated answers rely solely on the retrieved PDF chunks.

The evaluation suite comprises 20 questions, with Retrieval Recall@4 reaching 100% for 15 in-scope queries and full refusal correctness for five out-of-scope queries. Retrieval latency averages 0.42 seconds, while total response latency is approximately 1.85 seconds.

However, it is important to note that this repository provides an inspectable design showcasing a concrete RAG implementation, rather than acting as proof of its broader "production-grade" or "hallucination-free" claims. The published evaluation only covers 20 prompts and does not guarantee behavior beyond this specific test suite.

Moreover, the project's own disclaimer characterizes the chatbot as an educational assistant, with final authority resting with the university administration. The trade-off between local and hosted components, with the FAISS index running locally and Gemini services accessed via API for embedding creation and answer generation, requires careful consideration for the intended deployment environment.