Case study
National surveillance and surge readiness
Architecture and engineering standards for a national public-health surveillance platform designed to absorb a tenfold surge without losing trust in the data.
Evidence status: confirmedDisclosure: Anonymised
- Role
- Senior Systems Architect & Technical Lead
- Timeframe
- Multi-year programme
- Sector
- National public health
- Capabilities
- Enterprise architecture · Technical leadership · Data engineering · DevOps · AI engineering
- Evidence
- confirmed
- Disclosure
- anonymised
System view
Architecture and relationships
Diagram not yet published
A publishable system diagram is not yet available. The case study describes the architecture and organisational relationships without exposing client detail.
Decision records
The choices that shaped the work
Decision 01
Design for surge, not steady state
- Alternatives
- Tune the existing steady-state pipeline; add capacity only after a surge; or design the ingestion and processing path for a defined surge envelope.
- Chosen
- Design the pipeline for a 2–10m records/day surge envelope while keeping the current 250k records/day path operable.
- Why
- Surge capacity is an architectural property. Retrofitting it during an incident creates the highest-risk change at the worst possible time.
What would change my mind
A validated demand model showing surge volumes materially below the design envelope, or a constraint that makes the surge design uneconomic without reducing public-health readiness.
Context
Public-health surveillance is a trust exercise performed at volume. The programme receives data from multiple sources, validates it, and makes it available to analysts and downstream services. The system has to remain legible to the people accountable for it, not just operational for the people running it.
That distinction matters in a national setting. A late feed, a rejected record or a schema drift can change what a public-health team believes about the population it serves. The architecture has to make those conditions visible.
Challenge
Current throughput was around 250,000 records per day. The design target for surge conditions was 2–10 million records per day. That is not a problem solved by adding replicas. It changes the behaviour of ingestion, validation, storage and downstream processing at exactly the moment when the programme has the least room for error.
The programme also carried a serious control environment. Caldicott principles, data transfer agreements, a DPIA, CAB change control and multi-environment testing all shaped what could be changed, by whom, and with what evidence. The data path included scheduled ingestion, JSON and XML feeds, SQL Server and SSIS components, Azure services, SFTP transfers, and AWS SQS and Lambda for event-driven work.
The ambiguity was organisational as much as technical. Several teams owned parts of the path. The programme needed standards that could survive hand-offs between architecture, DevOps, frontend, backend and AI engineering without turning into a central bottleneck.
My role
I was Senior Systems Architect and Technical Lead. I worked across business analysis, enterprise architecture and technical leadership. I set and enforced programme-wide engineering standards covering architecture, DevOps, frontend, backend and AI engineering, and I was hands-on when a team needed a decision or a blocker removed.
My accountability was technical. I did not own the programme budget, financials or line management. The value I added was making the technical system coherent enough for the delivery system to move without losing control.
How it fits together
The platform is best understood as a set of controlled movements: source ingestion, validation, processing, storage and consumption. Some feeds arrive on a schedule; others arrive through managed file transfer. The pipeline has to preserve provenance and handle malformed records without silently dropping them.
The architecture spans Azure and AWS services. SQL Server and SSIS carry parts of the data path; SQS and Lambda support event-driven work. Scheduled ingestion and file transfer sit alongside service-based processing. The important design question is not which product sits in which box. It is where the system can absorb delay, reject bad data, replay a feed and still give an operator a truthful answer.
I have deliberately not published the detailed architecture. The system view above is a placeholder until a diagram can be redrawn without exposing client or control detail.
The trade-offs
The decision record accompanying this case study is the one that shaped the architecture: design for surge, not steady state. It changed how we thought about queueing, back-pressure, replay, environment parity and test data.
A steady-state design would have passed the current-volume test and failed the moment the programme needed it. Designing for the surge envelope forced the team to name the operating conditions in advance: what happens when a feed arrives late, when a validation rule rejects a batch, when a downstream consumer is unavailable, and when the volume is several times the normal day.
That decision also changed the standards conversation. A standard that only described the happy path was not a standard. It was a preference.
Trust and failure
Caldicott, DTNs, the DPIA, CAB and multi-environment testing were not paperwork bolted onto delivery. They defined what counted as a safe change. The hardest failure mode was not a crashed service. It was a pipeline that continued to look healthy while a feed was late, a schema had drifted, or a validation rule had quietly rejected records.
We treated observability and reconciliation as part of the architecture, not as a monitoring afterthought. A pipeline that cannot explain what it did with a record is not trustworthy, even if every dashboard is green.
Delivery
Delivery was through technical leadership rather than line management: architecture reviews, standards, design sessions and unblocking. The aim was to make the right thing the easy thing, so teams did not have to rediscover the same decisions at each boundary.
That meant working with delivery leads, engineers and control functions in the same conversations. It also meant being willing to make a call when the architecture had drifted into a local optimum.
Evidence
- Current throughput of around 250,000 records per day.
- A surge design target of 2–10 million records per day.
- Cross-environment pipelines and multi-environment testing.
- Caldicott, DTN, DPIA and CAB controls embedded in the change process.
These are programme facts, not a claim that the surge target was exercised at full volume in production. The distinction matters, and it is part of the evidence.
Reflection
The architecture was strongest when it made failure visible. It was weakest when the standards were treated as a document rather than an operating practice.
What did not work
Two things did not work well enough. Standards that were published without a named owner became documentation drift. And surge capacity that was described but not exercised was an assumption rather than a capability.
I would define the evidence for surge readiness earlier, including the tests and operating drills that would prove it. I would also spend more time with the teams who had to live with the standards, because adoption is a design constraint, not a communications task.
Related proof
- National Public Health Publications Service: programme delivery and service design in the same public-health context.
- Responsible AI technical assurance: assurance work that changed deployment conditions.
Evidence plate
Evidence visual not yet published
Evidence in this case study is drawn from approved programme records. Figures are attributed to their source and are not recombined into new claims.