AWS engineers confirmed that their fix was fully rolled out in all affected availability zones. We have been closely monitoring our services for last couple hours and can confirm everything is back to normal. We have rerouted traffic back to our regular load balancers at AWS and will monitor them further.
Posted Feb 11, 2021 - 01:23 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Feb 10, 2021 - 22:10 UTC
Update
Uploads traffic in US-EAST-1 is back to normal. AWS is rolling out their fix to load balancers in affected zones. We continue serving traffic through our backup load balancers until we confirm that the fix is in place.
Posted Feb 10, 2021 - 22:10 UTC
Update
US-EAST-1 uploads traffic was rerouted through our API servers in other regions. AWS confirmed that they have implemented fix and are deploying it to affected load balancers.
Posted Feb 10, 2021 - 21:46 UTC
Identified
AWS confirmed issue with their load balancers (https://status.aws.amazon.com/) that we are monitoring closely. In the meantime we have provisioned emergency load balancers cluster and directed CDN traffic through it. Delivery and transformations should be back to normal, however uploads are still affected.
Posted Feb 10, 2021 - 21:17 UTC
Update
We are continuing to investigate network issues. We will be updating this page with more details as we have them.
Posted Feb 10, 2021 - 20:25 UTC
Investigating
Our team is investigating an issue regarding abnormal traffic to the CDN.
Once we have further information we will be updating the status of this incident.