Amazon has concluded that an incorrectly entered command led to many popular internet sites being taken down, including Giphy and Medium.
Earlier this week, millions of sites that are hosted by Amazon Web Services (AWS) S3 were reporting widespread outages.
This was an unintended result of an effort to try to improve its S3 server speed, the e-commerce giant says. To fix the problem, Amazon tried taking a few billing servers offline. “Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended,” the company writes on its site.
Amazon says that some of its server systems haven’t been restarted in “many years,” which made getting things up and running again a bit more difficult. “The process of restarting these services and running the necessary safety checks to validate the integrity of the metadata took longer than expected,” the company says.
Amazon apologized for the incident and says a few changes are being made to help prevent it from happening again in the future. “We have modified this tool to remove capacity more slowly and added safeguards to prevent capacity from being removed when it will take any subsystem below its minimum required capacity level,” Amazon says.