Question Answering on Patient Medical Records with Private Fine-Tuned LLMs
Question Answering on Patient Medical Records with Private Fine-Tuned LLMs - arXiv
Summary
This study proposes using Large Language Models (LLMs) for semantic Question Answering over vast Electronic Health Records (EHRs). It demonstrates that privately hosted, fine-tuned LLMs outperform benchmark models in performance while ensuring privacy.
Details
The paper focuses on the Fast Healthcare Interoperability Resources (FHIR), a standard developed by HL7 to improve interoperability across healthcare systems. While EHR data is vast, its complexity makes it difficult for users to extract crucial insights. To address this, the research proposes a two-stage process using LLMs: first, identifying the most relevant FHIR resources for a user query (Task1), and second, generating an answer based on those resources (Task2). A critical aspect discussed is data privacy. Since sharing Personal Health Information (PHI) with cloud-hosted general-purpose LLMs risks violating regulations like HIPAA, the study emphasizes the viability of edge-deployed, privately hosted LLMs. The results show that smaller, customized models can achieve high performance (e.g., 0.55% improvement in F1 score on Task1 and 42% on Meteor Task in Task2) compared to large commercial models like GPT-4. This approach aims to simplify complex medical information for patients and providers, enhancing data accessibility and potentially improving healthcare efficiency by providing quicker, more intuitive access to relevant records.
Original content copyright by respective publishers