What are the different types of load testing scenarios? Our flooder friend, Leandro Melendez (aka Señor Performo), discusses some of the most common types as well as some best practices for load testing in this video. Read on for a transcript!
Hello again... Nicole and Flood amigos,
I want to begin the episode with a quick word... I want to repeat how thankful I am for you to invite me here to the Flood amigos channel! Acknowledgments aside, let's begin!
Today, I will use this space to talk to our Flooder amigos about load test scenarios--something that you all Flooders must be doing a lot of. But beware... the scenario modeling topic can be huge! So today, I will keep it down to describe the most common load scenarios and describe some details about them.
But first a word of advice, veeery important! Finding performance bottlenecks during load tests is no easy task. Often, it can feel like looking for a needle in a haystack. Make sure not to complicate your life by running multiple different load scenarios at the same time. Sadly, I have seen and been at many organizations that try to do many scenarios at once. This ends up being like looking for a needle in multiple haystacks, in different barns, and spread over multiple farms.
A general rule and best practice is to run one load scenario at a time and as segmented as possible. In other words, try to search on one tiny haystack at a time. Even if you must indeed look for needles in multiple haystacks, do not test them all at once, at least not as a first approach. Try them all separated first, starting from the smallest haystack, moving one by one until you reach the largest, and then you can start mixing haystacks. Most probably, way before you get to the last haystack, you will have found a few needles that require your attention. Well, beware: when I say needles, I mean it as an analogy for bottlenecks.
So, after that PSA (which is super important! You might be surprised by how many do not know that! Don't forget it!), let's get going and talk about the most common load test scenarios. Similar to the haystacks, I will start describing them from the tiniest and simplest, all the way down to the largest and weirdest ones.
Lone Rangers Scenario
1--The first one I am not sure if I can count it as load testing but definitely is a scenario that everyone should do as a very first step. This scenario has many names, such as single thread, unit performance, and many others. Here we are not stressing or loading the system again. Since it doesn't count as a load scenario, I guess that is why it has no official name. But I like to call it the lone processes scenario.
Where we will find the good, the bad, and the ugly, each one alone by themselves. In this scenario, you will set only one thread per script, component, or business process. You will run them all, one at a time one after the other, for a smallish amount of time. If you want, you can run them all together for a little while, but again just one thread for each one.
This scenario is a must, as it will gather base readings and identify early if there is a process in your system that is already by itself misbehaving. No point in load testing yet, those early bad and ugly boys. You could even have this scenario run at certain cycles of time to constantly monitor or ping the processes in the system.
Average Load Scenario
2--... Moving on! The second type could be counted as the most common scenario. It is called as well by multiple names such as the average load scenario, baseline scenario, day in prod, and many more. This one is even called just the load scenario. Which I try to avoid as it brings lots of obvious confusions
Probably, as you have already deduced by its name, this scenario focuses on simulating an average period of time of system load in production. Generally, we will try to simulate an average hour or probably a busy average hour. What you consider busy varies a lot, based on what you consider the most useful average load.
This average has to add up a mix of averages... a number of CONCURRENT users in that hour, doing an average number of actions in the system, and all of this, distributed per use among the most commonly used processes.
Phew, too much averaging, right? But indeed, this is the most average and common load test case. Pun intended.
As we are narrowing that average load to an hour, generally, that will be the duration of the full load of this type of scenario, one hour with about 30 minutes of ramp-up and ramp-down. All these values can vary based on the risk we are trying to mitigate, but the goal of this scenario is to clear up bottlenecks on average days or hours.
In other words, average expected loads.
Busy/Extreme Average Scenario
3--Next we have another scenario with some more confusion on its name, Some call it peak scenario, some call it the stress scenario, and even some other names. I like to call it the busy average or extreme average scenario.
This scenario is very similar to the previous one, as you noticed on my reiterated use of the word average. In this one, we will push further the average test to see if the system survives a seriously busy average hour, like a rush hour or a moment when, for some reason, everyone wants to work on the system at the same time.
This scenario can be larger than the average by fifty percent more. Double or even triple the average. It will depend on what you consider to be a very high but still realistic amount of load.
Again, the point for this one is to see how the system handles a seriously really really busy day, but while mimicking realistic super busy patterns. We commonly run this scenario for an hour and make the ramp-up and ramp-down a bit longer than the average scenario since it has to go further.
Do not try to run this scenario before running the average load scenario. This is because you might find a bottleneck in the system without knowing if it happens on a regular load.
Next time...
Phew... and speaking of bottlenecks. This episode is not meant to be a load or endurance test of your guest host (that's me).
So, since we are not endurance testing me, and the list of load scenarios is long, I guess that that will be for today, and we will go over the rest of the scenarios in the next episode. For today, that is it from me.
An endurance test? What is that?
Well, if you want to find it out about that and figure out if Señor Performo will complete the list of the types of load scenarios, don't forget to tune us on the next episode, same Flood-time, same Flood-station!
Resources for Load Testing Basics
[API Load Testing ebook] Load test scenario types
What is load testing?
6 common mistakes in performance testing