This is a summary and analysis of an issue that occurred with the delivery of an Acquia product or service. The purpose of this document is to share details about what happened and why, so there is a common understanding of what is required to prevent a future occurrence if at all possible. Any remaining issues or risks are identified, as are recommended or pending actions.
On August 31, 2018 Acquia detected a problem with launching or relauching server instances in the Acquia Cloud. This caused Acquia to halt any provisioning or configuration change operations in Acquia Cloud Enterprise, as well as turning off any resizing or purchasing operations for Acquia Cloud Professional. After identifying downtime in a third party service that is called for all Operating System launches, Acquia disabled this service across the complete Acquia Platform and then re-enabled all provisioning, launch, and resizing operations.
The tools and processes that manage the provisioning and launching of server instances in the Acquia Cloud have a dependency on a service to verify the operating system patch level when server instances are created. This service experienced downtime and caused the Acquia tools to fail.