Preparing for Black Friday

That’s a 16% increase on Black Friday last year and a 200% increase compared to a typical day!

It’s that last statistic which really stands out for the Dotdigital technical teams. We need to be ready for that level of increased usage and so we spend the weeks prior trawling over telemetry data from our platform to scale our infrastructure accordingly. This year we already knew a lot of the work had been done due to the incredibly busy period running up to the GDPR in May.

There are four major elements to our server infrastructure: website clusters, background service clusters, database servers and email servers. Because we’re cloud-based, each can be scaled separately and so we:

• added extra web servers to handle the 37 million hits our click-and-open tracking website saw, plus another 34 million for Web Behavior Tracking.
• increased the numbers, size and performance of some of our servers which handle background tasks such as sending and importing.
• added a new email sending capacity to our fast-growing US deployment which has seen rapid growth in the last 12 months. At peak we sent over 320GB of email an hour!
• optimized email delivery throughput with improved compliance with email receiver guidelines.

Throughout the day, our engineers monitor system metrics, making notes of areas that will soon need attention. It’s busy days like this which enable us to see early warning signs of weaknesses in our different systems.

Sometimes concerns can be addressed quickly and easily with more computing power, or by altering slightly the behavior of a system via a setting. Alternatively, bottlenecks are fed into the development roadmap so systems can be overhauled for the coming year. As demand from customers continues to increase, we continue to reinvest in our platform and we’re already looking forward to next November!