Scaling Healthcare Data Processing: Building a Multi-Environment FHIR Batch Workflow

Key Players

Details

This article details the technical process of solving challenges related to bulk updates of patient contact information (like phone numbers) across multiple geographical regions (e.g., Purbalingga, Lombok Barat). Initially, the team used a simple looping script, but as the record volume grew and environments multiplied, they faced issues with memory consumption, instability due to network calls, and managing load distribution across separate regional FHIR servers. The solution was to re-engineer the process from a mere 'update script' into a robust 'healthy pipeline.' The new architecture uses Google Sheets as the data source, processes data through a Flask application (the batch engine), and then accesses each region's dedicated FHIR server. Key features include strict environment isolation, ensuring that multiple regions can run concurrently without interference. Furthermore, it prevents job failure due to a single bad record. The batch engine processes records in defined batches (e.g., 100 records). This approach manages the load on the FHIR servers, preventing overload, while also providing real-time progress updates and detailed error logs for operators. This demonstrates a resilient best practice for handling increasing data volumes and complex operational environments.

Technology Note

FHIR（Fast Healthcare Interoperability Resources）は医療データ交換の国際標準。このエントリの関連技術: FHIR