More of our customers are integrating Flood tests into their build and deploy pipelines. In this post we’ll describe how you can integrate Flood with Jenkins, a popular open source automation server.
Jenkins provides hundreds of plugins to support building, deploying and automating any project.
Flood has an easy to use API which can automate the creation of grids and control of flood tests.
Combine the two together and you get continuous integration of performance tests with your build and deployment pipeline.
You don’t need anything more fancy than curl and jq to consume our API. Most often these already exist on modern linux based systems, but the same binaries can be installed and run on Windows.
In our following example, we’re running Jenkins as a docker container. Spinning up a container is as simple as:
docker run -p 8080:8080 -p 50000:50000 jenkins
We’ll incorporate a simple bash script into our source control, that can be called programatically from your Jenkins workspace in a build step as follows.
Looking at the script in more detail, we can control the following typical activities within Jenkins.
1. Launch a flood test and take note of its ID. Note that we can specify things like the number of threads, rampup and duration parameters as well as other details like the project and build number provided by Jenkins. We’ll also use the JMeter file called load.jmx which is kept under source control.
flood_uuid=$(curl --silent -u $FLOOD_API_TOKEN: -X POST https://api.flood.io/floods \
-F "flood[tool]=jmeter" \
-F "flood[threads]=100" \
-F "flood[rampup]=30" \
-F "flood[duration]=120" \
-F "flood[privacy]=public" \
-F "flood[project]=CI Pipeline" \
-F "flood[name]=Build $BUILD_NUMBER" \
-F "flood[tag_list]=ci,load" \
-F "flood_files=@$WORKSPACE/tests/load.jmx" | /tmp/jq -r ".uuid")
2. Poll the flood test status and wait for it to finish.
while [ $(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r '.status == "finished"') = "false" ]; do
3. Extract key performance metrics with things like a summary report, error rate and response times, so that we can programatically fail the build based on a performance regression.
error_rate=$(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r .error_rate)if [ "$error_rate" -gt "0" ]; then
echo "Flood test failed with error rate $error_rate%"
firesponse_time=$(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r .response_time)if [ "$response_time" -gt "3000" ]; then
echo "Flood test failed with $response_time > 3000ms"
Putting it all together
Using our powerful API and relatively simple scripts, you can incorporate load and performance testing as part of your build in your Jenkins workflow.
Using tools like Ruby-JMeter or Gatling gives you the added benefit of storing easy to read, expressive load test plans in source control, making them simple to share and version against each build e.g.:
post name: 'create session', url: '/api/oauth',
username: 'Michel Rosen',
extract json: '.access_token', name: 'access_token'
The same approach can be applied to virtually any build automation tool. Bamboo load testing or Buildkite load testing through the use of simple scripts and our powerful API is very similar process.
Keep running those performance tests as part of CI and you will soon be able to compare performance over time. Regressions or improvements will then be easy to isolate and attribute back to individual build or commit identifiers.