A hospital network needed real-time visibility into patient flow, bed occupancy, and resource utilisation across 12 sites. Here is how we built a system processing 50,000 events per hour with sub-second dashboard updates.
The Challenge: Blindness at Scale in Critical Healthcare Operations
Our client operates a network of 12 hospitals and 40 outpatient clinics across the UK, with over 4,000 staff. Hospital operations directors were making capacity decisions — diverting ambulances, authorising overtime, requesting agency staff — based on data that was 4–6 hours old. In a system under constant pressure, this delay had measurable consequences for patient outcomes and cost efficiency.
The goal: build a real-time operational intelligence platform that gives directors and site managers visibility into patient flow, bed occupancy, A&E wait times, theatre utilisation, and staffing levels across all sites simultaneously, updating within seconds of real-world events.
Architecture: Event-Driven Real-Time Processing
Data Sources and Ingestion
The hospital network ran seven different Electronic Patient Record (EPR) systems across its 12 sites — a legacy reality of NHS trust mergers. Each system had different data models, different update mechanisms, and different data quality standards. Our ingestion layer needed to normalise all of them into a unified event model.
We built dedicated connectors for each EPR system — some via HL7 FHIR APIs, some via database change data capture (CDC), some via file-based integration with legacy systems that had no API capability. All events were published to an Apache Kafka cluster that acted as the real-time backbone of the architecture.
Stream Processing
Kafka Streams consumers processed raw EPR events and produced derived metrics: current bed occupancy (beds occupied / beds available by ward, by site), A&E 4-hour performance (patients seen within 4 hours / total A&E arrivals), theatre utilisation (booked session time / available session time), and predicted capacity pressure (time-series model predicting A&E arrival rate from historical patterns and real-time data).
Real-Time Dashboard Architecture
The dashboard used WebSockets (Socket.io on a Node.js backend) to push updates to client browsers within 500ms of the underlying event. A Redis Pub/Sub layer decoupled the stream processors from the WebSocket servers, allowing both layers to scale independently. Each hospital site had its own "room" in Socket.io — clients subscribed to their site's channel and received only relevant updates.
HIPAA and NHS Data Security Compliance
Healthcare data processing carries strict regulatory requirements. Our architecture incorporated: end-to-end encryption for all data in transit and at rest, role-based access control aligned with NHS Spine roles, comprehensive audit logging of every data access, data minimisation (the dashboard consumed only the minimum data elements required for operational metrics — no clinical data, no patient identifiers), and Business Associate Agreements with all cloud providers.
A Data Protection Impact Assessment (DPIA) was completed before architecture was finalised, ensuring privacy-by-design rather than privacy as an afterthought.
Implementation: The Phased Approach
Phase 1: Two Pilot Sites
We launched with the two largest hospitals in the network, processing real data in read-only mode while the operations teams validated dashboard accuracy against ground truth. This phase caught several data quality issues in the source EPR systems — bed closures not being recorded in real time, discharge events firing before patient physical discharge — that required process changes on the hospital operations side.
Phase 2: Network Rollout
With validated data pipelines and a refined data model, we onboarded the remaining 10 hospital sites over 8 weeks. A national view, showing network-wide capacity with drill-down to site level, launched at the end of this phase.
Results: Measurable Operational Impact
Six months after full deployment: ambulance diversion decisions were made 4 hours earlier on average, allowing better coordination with ambulance services. Bank and agency staffing requests were placed with 3-hour lead time rather than same-shift. A&E 4-hour performance improved by 6 percentage points network-wide — a significant improvement given the pressured environment. The operations team estimated annual cost savings from improved resource allocation at £2.4 million.
Expert insights on AI, software engineering, and digital transformation from the TechGeneses team of engineers and strategists.