This is a tough project that I need to design, build and stress test a email platform for Coca Cola coke with food TVC campaign, which protentially needs to receive email from millions of users on every Sunday during campaign period. The good news is I can estimate the high traffic only happens on Sunday night and the rest days in the week remain low traffic. This is my architecture design:
I prepared some standby mail servers so I can add then into route 53 MX records before the expected high traffic period.
After the user watches TV advertisement, he takes a photo of the combination of coke and food and submits this photo via email.
One of the email servers receives this email and need to pass the mail raw content to the worker EC2. Since the number of worker EC2 instances can be changed manually based on the estimated email traffic, I implemented a simple workload balancing logic:
worker_id = email_id MOD n, while n is the number of total workers EC2 instances
(Note that there was no autoscale group in early 2013, and this is the best I can achieve)
The worker will parse the raw mail content and then get the sender email and base64 encoded image, and the user info will be saved in RDS and raw image content will be base64 decoded then save to S3. Admin users can check the statistic of the campaign submissions as well as for lucky draw purpose.