“Async advised and helped us scale our backend server set up in preparation for our app’s showcase on Dragons’ Den. Everything ran very smoothly even with 1000’s of users using at once. Very happy with the outcome and service. Highly recommended!”
Gary McDonagh, Double’s CEO.
When Double, a London-based innovative dating startup, gets asked to join BBC Two’s famous Dragons’ Den program, they were wary of being able to handle the load as a result of the broadcast of the show on national TV.
Async immediately took a look at the task and came up with an action plan to firstly measure the load that the show was likely to produce. The first step was to install our own dashboard and insert the necessary hooks in Double’s application stack so that we could see the load on the site, on a granular controller basis. From previous experience we knew the key to successfully scaling systems like this is getting accurate, real time metrics showing where the bottlenecks are across the application.
Our metrics dashboard uses open source software, meaning no nasty recurring fees, which can be used for tracking customer activity in aggregate and is useful beyond the immediate DevOps requirements.
Once Async had installed the measurement tooling, we went about mapping the behaviour of the app from real world usage of the iOS app and building scripts that could replicate this to emulate what the likely load would be like. Initially after running the load testing scripts against the then current production backend showed a major problem: the sheer number of concurrent users brought about by the TV airing would more than likely bring down their entire system. At that point (shortly before Christmas), the stack was a good order of magnitude off where it needed to be and would require Async’s experts to make changes throughout the system.
“Seeing that every other app and website that went on the show instantly fell over, we were really concerned about not getting the maximum marketing impact from the airing, We really needed to make sure that our AWS backend could handle the load without breaking a sweat and we were short on internal resource.”
Gary McDonagh, Double’s CEO.
The next step was to take a holistic view of the needs of the backend and utilise AWS in the best way possible.
Async migrated the single RDS instance to a bigger machine with more iOPS and then split the DB into two read replicas and one write replica. We then ported the application stack to EBS (elastic beanstalk) allowing us to scale up and down the number of application servers within minutes. From this point after running the load testing scripts we saw a dramatic performance improvement in terms of number of timeouts and response times, but still we needed to go a few steps further. The application stack and PHP framework were under control and performing smoothly, however there were still issues with the DB and some of the queries.
The penultimate task to ensure the backend could handle the load was to profile and optimise the mySQL database cluster. At Async, our multidisciplinary team is helpful in these circumstances. We were able to task one of our DBA specialists to pinpoint the areas of contention in the database stack.
Once Asyncs’ experts handled that, it was time to tidy a few loose ends, including moving the product site over to a static S3 website deployment, and passed over our documentation to the Double team.
On the night of the broadcast, the application stack performed without any hiccups without any timeouts (despite being under severe load). Double secured funding from the Dragons in what was a great pitch on BBC. Here is the link to the episode on BBC Dragons’ Den.