So you've decided to jump in with both feet and test the performance of your web app? Hello, and welcome to the wonderful world of load testing!
Perhaps you have a major sales event coming up soon and you want to ensure your site can gracefully handle the projected influx of orders. Maybe you've established a link between app performance and an important business metric, such as revenue, customer signups or more general customer satisfaction and want to ensure that performance doesn’t negatively impact these. Or it could be that you just feel that the app could run better to consume fewer resources, either on your servers or your users' own devices.
Finding the Right Tool for the Job
The first thing you've probably noticed on your journey to load test is the dizzying array of tools and approaches available to get the job done. Today I'd like to tell you about my new favourite load testing tool - Flood Element.
Many popular tools operate at the protocol level - they attempt to approximate users' behaviour on your app by issuing network requests. However in today's world wide web, web apps are becoming more and more sophisticated as well as utilising many more of the browsers' ever growing list of features than ever before. Even seemingly simple apps like an online shop use a plethora of browser features to make the experience of choosing items, ordering and paying online as smooth as possible for the user.
And so, in this modern era of the web, protocol level user (PLU) testing can only give us so much of the picture of our app's performance. By contrast, new tools like Flood Element test by simulating user behaviour using real web browsers. We call this approach Browser Level Users (BLUs).
Introducing Flood Element
In Flood Element, simulated user behaviour is defined using simple, natural actions such as clicking links and buttons, filling in web forms and so on. Once you've defined the behaviour, it's a snap to run at huge scale at flood.io - and by huge scale, I mean thousands of users and from all over the globe!
Load testing with Flood Element can help you develop a holistic, intuitive view of your app’s performance as experienced by a user. Given that thinking as users is what we all naturally do when we’re users ourselves, it’s also significantly easier to get started with an Element BLU test than with traditional PLU tools.
The holistic view
Using Flood Element to measure performance on your whole app from users' perspective is a great way to develop a holistic view of your app's performance and thus defend against unexplained changes - be they temporary anomalies or perhaps regressions in the code.
To illustrate how Flood Element can work for you, let's work through a hypothetical-but-realistic scenario.
Imagine for a moment that your site is running on all cylinders, performing slickly and serving both the business’ and customers' needs to a T. Development of the site application carries on as usual with scheduled daily deployments carried out each afternoon.
Suddenly one morning you receive a call from the marketing team saying that their analytics show customer signups dropping by 20%. Examining the analytics, the graphs reveal that the decline began after the daily deploy a couple of days earlier.
How could you track this problem down in the moment?
If you thought I'd say "Flood Element!" you'd be wrong... because in this story, everything was running so smoothly up to this point that you hadn't needed to do any load testing. But all is not lost, there are definitely clues to work with. In this case the marketing analytics data might provide a continuous, predictable enough view of the rate of signups that the point at which the anomaly started was clear enough to give the development team an idea of where to start looking for the problem.
How could you proactively defend against this problem?
If you thought I'd say "Flood Element!" this time you'd be right!
PLU load testing using say, JMeter could never provide the insight necessary for avoiding this regression.
Flood Element BLU tests provide a view of performance much closer to the user experience. They take into account every part of performance that a user will experience: network performance, but also on-page script performance as well as third-party scripts such as analytics or advertisement add-ins (in fact our tests are so realistic that you need to take steps [link] not to create false data in Google Analytics.)
If running a BLU load test with Element gives you a holistic view of your app’s performance a point in time, running tests regularly allows you to build up an intuition of whether your app is getting faster or slow… perhaps suddenly as in the signup widget scenario.
Writing a test
Now let’s put the hypotheticals aside and see how easy it is to get started with Flood Element.
You can install Element using the getting started instructions here. Once installed edit and test your script locally, and when you’re ready upload it to flood.io to perform a full-scale load test.
A basic Element BLU test for the signup scenario above might look like
If you’re familiar with testing at the protocol level, you know that all timings are arranged around request-response transactions. When learning to test at the browser level, there are a few extra subtleties to take into account when finessing your tests. In particular, there are no real built in timing groupings, so we need to introduce our own in the structure of our test.
The simplest approach for grouping timings is to wait for the necessary elements to appear on the page — just like a user would.
From the scenario above, we’re interested in timing the user experience of signing up, but not so much the loading of the page. So, in the first step, we simply visit the URL and wait for the page to reach a known state — until the title is shown.
Next we perform the action we’re actually interested in keeping an eye on, namely filling in and submitting the signup.
Now it’s time to upload the script to flood.io.
- Head over and sign in (or sign up for a free trial if you haven’t already got a login).
- Create a new flood and upload your test.ts.
- Decide how long you’d like to run the script and at what scale (the defaults should be fine for now).
- You can watch the results roll in in real time, or go and get something else important done (☕️.)
- Once the test has finished running, take a look at the graph. At this point the graph contains a treasure trove of info that you’ll have to learn to interpret.
The default graph shows the performance of the site in aggregate, but in our scenario we decided that we were really most interested in the form itself. So go ahead and click on the “signup” label to drill down and see only the graph for that section.
The whole picture
So now you know why (and how) how to test with Browser Level Users using Flood Element. I hope it helped you to see how it could be useful in your own business.
Of course in an article titled “Why You Should Load Test with Browsers” You’d expect to hear a lot of positivity about the new approach. But the whole picture is that BLUs and Element are only a part of the whole picture, albeit a valuable and exciting new part. PLUs are still extremely powerful and worthwhile tools for examining the performance of your app.
My general approach for load testing:
- Work out your goals for performance testing. It’s always worth performance testing a customer facing app, but you should first work out your business or technical goals for doing so. Goals for performance testing may be built around testing capacity guarantees, observing or improving business metrics, or for optimisation.
- Start with BLUs. Use Flood Element to jump start your load testing journey. Get some regular, scheduled (or deployment-triggered) load testing happening early on, so that you can develop a baseline and an evidence-based intuition for the performance profile of the app.
- Over time, add PLU tests to critical API endpoints or server-side rendered pages using JMeter or Gatling. Use PLUs to focus your efforts on specific optimisations and to push the performance envelope of your app.