At 16:35 UTC on 06/03/2019, we attempted to deploy version 12.5 of the diary application to our Southeast Asia server. Unfortunately the deployment process failed before all of the steps in the process had completed, meaning that the web server was running the latest version, but its database had not been updated correctly.

At the time, engineers noticed that the deployment process had failed and manually updated the web application to 12.5, without noticing that the database updates had not been run.

At 00:11 UTC one of our on-call engineers was notified that users on the Southeast Asia server were having issues accessing their diaries or performing certain operations against bookings. He investigated and realised that the deployment had failed part of the way through, without the database updated being run. At this point he rolled back to the previous version of the application, restoring functionality.

We sincerely apologise for any inconvenience caused. We are examining our processes to try to prevent something like this from happening in future.