Jenkins provides hundreds of plugins to support building, deploying and automating any project.
Flood has an easy to use API which can automate the creation of grids and control of flood tests.
Combine the two together and you get continuous integration of performance tests with your build and deployment pipeline.
In our following example, we’re running Jenkins as a docker container. Spinning up a container is as simple as:
docker run -p 8080:8080 -p 50000:50000 jenkins
We’ll incorporate a simple bash script into our source control, that can be called programatically from your Jenkins workspace in a build step as follows.
Looking at the script in more detail, we can control the following typical activities within Jenkins.
1. Launch a flood test and take note of its ID. Note that we can specify things like the number of threads, rampup and duration parameters as well as other details like the project and build number provided by Jenkins. We’ll also use the JMeter file called load.jmx which is kept under source control.
flood_uuid=$(curl --silent -u $FLOOD_API_TOKEN: -X POST https://api.flood.io/floods \
-F "flood[tool]=jmeter" \
-F "flood[threads]=100" \
-F "flood[rampup]=30" \
-F "flood[duration]=120" \
-F "flood[privacy]=public" \
-F "flood[project]=CI Pipeline" \
-F "flood[name]=Build $BUILD_NUMBER" \
-F "flood[tag_list]=ci,load" \
-F "flood_files=@$WORKSPACE/tests/load.jmx" | /tmp/jq -r ".uuid")
2. Poll the flood test status and wait for it to finish.
while [ $(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r '.status == "finished"') = "false" ]; do
3. Extract key performance metrics with things like a summary report, error rate and response times, so that we can programatically fail the build based on a performance regression.
error_rate=$(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r .error_rate)if [ "$error_rate" -gt "0" ]; then
echo "Flood test failed with error rate $error_rate%"
firesponse_time=$(curl --silent --user $FLOOD_API_TOKEN: https://api.flood.io/floods/$flood_uuid | jq -r .response_time)if [ "$response_time" -gt "3000" ]; then
echo "Flood test failed with $response_time > 3000ms"
Putting it all together
Using our powerful API and relatively simple scripts, you can incorporate load and performance testing as part of your build in your Jenkins workflow.
Using tools like Ruby-JMeter or Gatling gives you the added benefit of storing easy to read, expressive load test plans in source control, making them simple to share and version against each build e.g.:
post name: 'create session', url: '/api/oauth',
username: 'Michel Rosen',
extract json: '.access_token', name: 'access_token'
Keep running those performance tests as part of CI and you will soon be able to compare performance over time. Regressions or improvements will then be easy to isolate and attribute back to individual build or commit identifiers.