Automate Load Testing with Armory Spinnaker and Locust

Last week we introduced Certified Pipelines, a tool to help you define and enforce policies to ensure your teams are deploying safely and with best practices. The policies you create are comprised of pipeline stages – simply list the stages required to meet your policy and any pipelines that are missing one of those stages will be blocked from deploying.

We’re adding first class support for specific types of stages that help you automate the steps we believe create a safe deployment. We’re starting with load testing because, from experience, we know the cost of load testing is high which is why they are rarely done.

We also know that there’s huge benefit to doing more frequent load testing. Our goal is to reduce the friction associated with doing load testing such that it’s easy for everyone to add to their pipelines.

Adding a Locust.io Load Testing Stage

The load testing stage currently supports Locust.io, a simple, open-source load testing utility (we’ll be adding support for other tools shortly). To add a Locust Load Testing stage, simply select “Locust Load Test” from the stages dropdown menu in your pipeline configuration view.

Automate Load Testing using Canaries

Once you’ve configured your load testing stage, the next step is automation. To accomplish that, we’ll leverage automated canary analysis using Barometer.

1. Define load tests. Configure the number of clients, hatch rate, duration, and host.
2. Add a Barometer stage. The automation is enabled by Barometer, an ACA service for Spinnaker. You’ll need to add the Barometer ACA stage directly after the Load Testing stage. When configuring the Barometer stage, just make sure you select the cluster associated with the load testing environment.
3. Define application metrics. Within Barometer, define which application metrics you want to measure, ie. response times, CPU, memory.
4. Stand up environment. Create an environment to perform the load testing in.
5. Deploy a baseline and canary. Within the load testing environment, standup a baseline (the existing code) and a canary (the new code).
6. Run load tests. We’ll run the Locust load tests against both the baseline & the canary to determine if the new code is good or bad.
7. Determine if the tests pass or fail. If the canary’s metrics are within a reasonable threshold to the baseline, continue the pipeline, otherwise fail it.

Roadmap

We have plans to add support for Gatling and Jmeter. Let us know below if there are other load testing tools you’d like to see us add support for.

Learn More

• New Spinnaker JIRA Stage
• Introducing Armory Certified Pipelines

And if you are interested in trying our Load Testing stage, just drop us a note below or install Armory Spinnaker yourself here.