From Meal Photo to FHIR: Building a Reliable Healthcare Data Pipeline
From meal photo to FHIR - Benjamin Adams
Summary
This article introduces a technical pipeline that converts meal photos into nutrition logs and transforms them into FHIR resources usable by medical systems. By adopting LLM output validation and streaming processing, it achieves both reliability and user experience.
Details
The case describes building a pipeline to extract nutritional information from user-submitted meal photos and structure this data not merely as a database entry, but as a FHIR resource usable in clinical settings. Since the process involves multiple probabilistic elements—such as vision model guesses or food database discrepancies—a robust design is critical. Specifically, output from the LLM (Claude Sonnet 4.5) is forced to conform to a strict JSON schema. Data failing validation is retried, ensuring that only reliable information enters the record. The final structured data is stored as a FHIR resource in Medplum, guaranteeing interoperability with the entire healthcare stack, including care teams and analytics. Furthermore, user experience is enhanced by streaming results via SSE (Server-Sent Events), providing real-time feedback to the user.
Original content copyright by respective publishers