Real-Time Epic FHIR Analytics: Architecture for .NET, ADF, and Power BI Integration

Details

The core challenge addressed is transforming the massive amount of clinical and operational data residing in Epic EHR into live, analytical dashboards without overwhelming Epic's FHIR API. Since standard FHIR APIs are designed for transactional point-of-care access, continuous polling for large-scale analytics quickly leads to rate limiting. The proposed solution involves a multi-layered pipeline architecture. First, the .NET extraction service utilizes Epic's Bulk Data Export API (preferred over individual FHIR calls) to generate NDJSON files asynchronously. This method significantly reduces API traffic and minimizes load on Epic. Next, Azure Data Factory (ADF) acts as the orchestration layer. It initiates the bulk export via the .NET service, continuously polls the status endpoint until completion, and then downloads the raw NDJSON files into Azure Data Lake Storage Gen2. Subsequently, data transformation and flattening are performed using tools like Azure Synapse or Databricks, converting the raw FHIR bundles into a curated Star Schema stored in Azure SQL/Synapse Serverless. Finally, Power BI connects to this structured schema for near real-time dashboarding. This separation of concerns ensures scalability and reliability while respecting Epic's operational constraints.