2018 09 18 Load Testing

On 09/18/2018 we ran load tests against our Dev-Green server and observed the server managing to handle loads up to about 20 requests per second.

We ran three rounds of tests. The first was of 50 simultaneous users, launched at a rate of 5 users per second and then left running for several minutes. This ran with no failures. The second was of 100 simultaneous users, launched at a rate of 10 users per second and then left running for several minutes. This also ran with no failures. The final round was of 200 simultaneous users, launched at a rate of 20 users per second and then left running for several minutes. This one had one period where there were a handful of requests that failed over a few seconds, but otherwise had no problems. These failures represented far less than 1% of the total number of requests for that run.

The server was set up with specs to match the current state of our production server. In ECS it was set to have 4 virtual CPUs and 16 gigs of RAM. The latest version of the master branch was deployed to it.

The performance test script is built on the Locust library. When each user is launches the first thing it does is to sign up for a new account. The user then goes through different actions in random order for the rest of the duration of the test, pausing a random amount between 5 and 15 seconds between each step. These actions include creating a new bus subscription, creating a new commuter rail subscription, creating a new Red Line subscription, visiting their subscriptions index page, and visiting their account settings page.