Building Trustworthy AI for Enterprise Support: An Empirical Study of a RAG-Based Architectural Framework

Citation: Udit Joshi, Kapil Verma, "Building Trustworthy AI for Enterprise Support: An Empirical Study of a RAG-Based Architectural Framework", Universal Library of Innovative Research and Studies, Volume 03, Issue 01.

Copyright: This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The paper considers an approach to engineering a trustworthy enterprise support service based on a Retrieval-Augmented Generation (RAG) architecture in a high-load ticket-handling environment. The study’s relevance stems from the widespread deployment of generative solutions in contact centers and users’ growing sensitivity to hallucinations, stale data, and unpredictable system behavior. The objective of the research is to obtain an empirical assessment of a full-scale RAG architecture for internal support and to identify engineering decisions that critically determine its reliability. Across five experimental series, empirical data were obtained for key metrics: MRR@10 for retrieval quality, overall refusal and correct-refusal rates, the proportion of confident yet factually incorrect answers, p50/p90/p99 end-to-end latency, the time required to incorporate a document into the index, and the share of interactions involving outdated information. The scientific contribution of the work lies in analyzing a RAG system as an integrated engineering artifact studied on a real-world corporate corpus. It is shown that semantic document segmentation yields a substantial improvement in retrieval quality over fixed-size chunking. That dense semantic search dramatically outperforms sparse keyword-based search, including in terms of the system’s ability to refuse correctly under knowledge scarcity and to reduce dangerous hallucinations. It is established that the generative component is the dominant source of latency. In contrast, a hybrid indexing strategy that combines streaming delta-indexing of critical documents with nightly re-indexing of the entire corpus enables maintaining both knowledge freshness and operational resilience. The paper is intended for researchers and practitioners designing scalable and trustworthy enterprise support systems built on large language models.

Keywords: Enterprise Support, Retrieval-Augmented Generation, Semantic Search, Trustworthy AI.

https://doi.org/10.70315/uloap.ulirs.2026.0301008