Resilience position

How it is built

• Two application services — the interface and the API — each running multiple instances behind Azure’s load balancing, in the UK South region. Losing one instance does not interrupt the service.
• A managed Azure database, patched and kept available by Microsoft, reachable only over private networking. It is not exposed to the internet.
• Scheduled jobs run separately from the application, so a problem with the daily checks cannot take the product down, and a problem with the product does not silently stop the checks.
• Monitoring with real alerting. Eight alert rules watch availability, the database and the scheduled jobs, and they route to people rather than to a dashboard nobody opens. A job that stops running raises an alert on its own schedule — silence is treated as a fault, not as good news.

What we have measured

We run a database restore drill quarterly rather than assuming the backups work. The drill restores to a throwaway server, checks the restored copy is coherent rather than merely present, and the server is destroyed the same hour.

• The backup restores, and the restored copy is coherent — table counts, every tenant present, and the newest record sitting correctly before the restore point rather than after it.
• A restored database server reached a ready state in just over six minutes.

What that figure does not yet cover, and we would rather say so: both drills so far measured the time to a restored database server — not the time to a fully working product. Reconnecting the application over the private network path, and cutting the service over to the restored database, has not been timed end to end. So our true recovery time is not yet a number we can give you, and we are not going to give you one we have not measured. The next drill is specified to close exactly that gap.

The decision we have taken deliberately

Ceidwad runs one database instance with no hot standby, and its backups are held in one region rather than replicated to a second. Both are choices, not oversights, and the reasoning is:

• What it costs you if it fails. Ceidwad is a governance and assurance platform. If it is unavailable for an hour, a school cannot open its readiness score or generate a report — it does not stop teaching, it does not stop the school’s email working, and it does not stop an incident being handled, because the response plan is designed to be printed and used on paper when everything else is down.
• What the alternative buys. A standby database shortens a rare outage from a restore to a failover. At our current size that spend is better put into the things that reduce the chance of needing it and into the evidence that recovery works.
• It has a trigger, not an excuse. This is recorded internally with the point at which it gets revisited — it changes as the number of schools relying on the service grows, rather than when somebody remembers.

If your risk assessment needs a higher availability guarantee than that, tell us during procurement. We would rather have the conversation before you sign than have you assume something the architecture does not currently provide.

What we do not claim

• No uptime percentage. We do not publish an availability figure, because a number like “99.9%” beside a single-instance database would be a claim the architecture does not support.
• No public status page today. If the service is unavailable we email the contacts on your account.
• No independent penetration test yet, and we do not treat our own review as a substitute. See the trust centre for the full list of what we do not have.

Questions

Architecture and resilience questions go to security@ceidwad.co.uk. We will answer specifically, including where the answer is that we have not measured it.