MEDINFO-LLM: A PARAMETER-EFFICIENT RETRIEVAL-AUGMENTED EXPERT SYSTEM FOR LOCALIZED ALLOPATHIC DRUG INFORMATION

Abstract

Due to the lack of healthcare resources, the variety of pharmaceutical brands, and the restrictions of reference manuals, access to reliable and contextual pharmacological information is still difficult to obtain in many resource-limited settings like Pakistan. While general large language models can be used to obtain general medical information, their tendency to produce unreliable statements hampers their use in clinical decision support applications. In this paper, we introduce MedInfo-LLM, an offline and parameter-efficient clinical decision support system for localized pharmacological information. MedInfo-LLM combines a 4-bit quantized BioMistral-7B model with a hybrid retrieval architecture consisting of a FAISS vector database and a knowledge graph built with Neo4j, which contains information about pharmaceuticals, their interactions, and drugs of the Pakistani drug registry. Parameter-efficient fine-tuning via LoRA and preference optimization via DPO, along with a three-stage safety framework against hallucinations, were applied to improve the quality of responses without exceeding a memory footprint of 3.8 GB. MedInfo-LLM was evaluated for detecting drug interactions, renal dose computation, and mapping of generic names to brand names for drugs from Pakistan, on a manually curated reviewed dataset by pharmacists consisting of 5,000 QA pairs split 70:15:15 into 3,500 training, 750 validation, and 750 testing examples, together with a separate 200-query benchmark spanning four tiers of reasoning complexity. MedInfo-LLM achieved a hallucination rate of 3.1% and an Exact Match score of 91.7%, with its observed performance on the evaluated dose-reasoning tasks higher than that of the comparison models tested. End-to-end inference latency was 60-180 seconds on an NVIDIA GTX 1650 with 4 GB VRAM.