Reinforcement Learning for Tool-Calling Agents in Fast Healthcare Interoperability Resources (FHIR)

Details

The paper introduces a novel approach for answering clinically meaningful questions using Fast Healthcare Interoperability Resources (FHIR), which serves as the primary standard for interoperable healthcare data exchange. In FHIR, electronic health records form a directed graph of resources, requiring multi-step reasoning, filtering, and aggregation across various resource types. Prior work indicated that even tool-augmented LLM agents often select incorrect resources or violate traversal constraints. The study frames this problem within the context of FHIR-AgentBench—a benchmark using real-world hospital data—and models reasoning on FHIR as a sequential decision-making process over a queryable structured graph. They implemented a multi-turn CodeAct agent and post-trained it using Reinforcement Learning (RL) with a custom harness and tools. By utilizing an LLM Judge for execution-grounded rewards, the approach demonstrated improved performance while enforcing data-integrity constraints compared to prompt-based or closed-model baselines. Empirically, the method significantly improved answer correctness on FHIR-AgentBench from 50% (o4-mini) to 77%, even when using a smaller and more cost-effective Qwen3-8B model. The paper presents an end-to-end post-training pipeline—covering environment building, harness construction, model training, and custom evaluation—that reliably enhances multi-turn reasoning over structured clinical graphs.