Load Testing in DevOps Pipelines with Azure and Flood

DevOps load testing tutorial using Azure Pipelines and Flood

In this video on the Azure DevOps channel, Flood co-founder Tim Koopmans talks about how load testing with Flood can be incorporated into CI/CD pipelines.

What can load testing show you?

When most people think of load testing, they think about application performance in terms of speed: how fast the application servers can respond to a request. However, Tim points out that there are other non-functional aspects of an application that load testing can measure. He calls these qualitative aspects PEARS:

Performance - response times, throughput, concurrency, size of page resources, and other metrics that affect speed

Elasticity - how well your application can scale out or scale back in according to demand

Availability - your application's uptime, and whether it stays operational despite high load

Reliability - whether your application returns correct or expected responses especially when under load

Scalability - capacity planning and ensuring appropriately sized infrastructure for the application

All of these aspects of an application can and should be tested using load testing techniques. Incorporating load testing into a Continuous Integration/ Continuous Development pipeline allows you to see a bigger picture of your application's quality.

What does a DevOps pipeline with load testing look like?

A pipeline is essentially a list of automated tasks involved in releasing software builds that run one after another, reducing manual effort.

From Azure Pipelines documentation

Automation is critical in a pipeline because it allows for certain activities to be triggered without human intervention. This setup enables Continuous Integration within software cycles, linking software testing to development and ensuring that new code is always tested for quality.

Adding load testing to the suite of test tasks means monitoring how non-functional aspects of the application (PEARS above) change for each code commit or build. This puts performance front and center in everyone's minds and makes it easier to spot which lines of code introduce bottlenecks.

Real-time load testing results allow teams to quickly spot problems before code is deployed, and historical results can also show improvements or degradations in application quality over time.

Real-time load testing results with Flood

How to use Flood with Azure Pipelines

Writing a load testing script

First, you'll need a load testing script. In the video, Tim chose to use Flood Element, an open-source browser-level tool based on Puppeteer, but Flood also supports other tools.

Element is written in typescript, and its syntax is very similar to that of Selenium, making it easy for automation testers to get started with it.

Here are some resources for creating your first Element script:

Installing Element: video and text

Using Element with Flood: video and text

Here's the script that Tim used in the video:

js
import { step, TestSettings, Until, By } from '@flood/element'
import * as assert from 'assert'
export const settings: TestSettings = {
   loopCount: -1,
   actionDelay: 1,
   stepDelay: 2,
   waitUnti: 'visible',
}
export default () => {
   step('Start', async browser => {

       // visit instructs the browser to launch, open a page, and navigate to <https://devpops.azurewebsites.net/>
await browser.visit('<https://devpops.azurewebsites.net/>')
       await browser.takeScreenshot()
   })

   step('Visit Amazing App', async b => {
       const appLink = await b.findElement(By.id('app'))
       appLink.click()
       await b.takeScreenshot()
   })

}

Note that the application he used has since been taken down, but you can modify this script to test your own application.

You can run your Element script within your workspace to debug, but you'll want to run it on Flood for the full load test.

Adding load testing step to Azure Pipelines

Next, you can add the execution of your Element script as a step in Azure Pipelines. To do this, add a bash task to Pipelines.

Flood has an API that lets you programmatically run load tests. In your pipeline, select the "Inline" option for the bash script and paste this:

bash
curl -su ${API_TOKEN}: -X POST <https://api.flood.io/floods> \\
 -F "flood[tool]=flood-chrome" \\
 -F "flood[threads]=20" \\
 -F "flood[name]=Baseline" \\
 -F "flood[tag_list]=ci,shakeout" \\
 -F "flood_files[]=@specs/baseline.ts" \\
-F "flood[project]=Azure Devops" \\
-F "flood[duration]=300" \\
-F "flood[git_ref]=$(Build.Repository.Uri)/commit/$(Build.SourceVersion)" \\
-F "flood[region]=southcentralus"

You'll need to replace `${API_TOKEN}` above with your Flood API access token, which you can acquire here.

This script runs a load test by spinning up a load generator on the Azure cloud.

Adding a bash script task for the Flood test in Azure Pipelines

Click the "Save and queue" button, add a save comment, and click "Save and run." Your Flood load test will start, and you can view the results in real time on your Flood dashboard.

Why should you include load testing in your DevOps pipelines?

There is a misconception that load tests need to be large in scope, involving thousands of users. This type of load test scenario does yield valuable information about how much load your application can withstand. On Flood, you can run millions of users depending on the cloud regions that you select.

However, there's also a lot of value to be gained from smaller tests that are run more frequently as part of a continuous testing framework. Having a performance baseline for every build is like constantly checking the pulse of your application and being able to identify when a new bottleneck is introduced.

In the Flood results dashboard below, regular tests show a significant increase in the last build's response times.

This spike, seen even with a 20-user test, would not have been spotted using a single peak load test. Incorporating load testing into a DevOps pipeline gives you data points and insights about continuous performance, instead of just a snapshot of performance in a single moment of time.

More on integrating Flood with Azure DevOps

Start load testing now

It only takes 30 seconds to create an account, and get access to our free-tier to begin load testing without any risk.