Cloud Load Testing Tutorial: Integrating Flood with New Relic

We're excited to announce that you can now pipe your Flood results into New Relic, an APM that you may already be using in order to get all your test metrics alongside your other server metrics. Antonio discusses how to do just that.

Recently, we released an exciting new integration of Flood with the leading APM tools: New Relic, DynaTrace and AppDynamics. The key idea is to create a 360-degree view of your application performance by pushing Flood’s metrics into the APM platforms to marry them up with the metrics and events aggregated by the agents.

What is an APM tool?

An APM (Application Performance Management) tool monitors the performance and availability of software applications. APM strives to detect, alert and diagnose complex behaviors and potential issues to keep an expected level of service.

Why combine Flood’s Metrics with an APM?

Flood has specific; user experienced focused information which we believe add tremendous value when combined with the metrics of an APM tool.  Some examples of how you can use Flood metrics to more quickly assess the health of your application within your APM include:

  • Follow concurrency curve to ensure response time isn’t increasing disproportionately as new users are added
  • Identify peaks in response time to set the context for where to investigate server metrics
  • Highlight increases in failed transactions to examine the source of failing logic

A Sample Using A Well Know Application: The Internet Heroku App

We want to share our results by integrating the internet Heroku app, which is a Ruby application mounted on Docker containers. We installed this app on an AWS t2.micro server with New Relic agents monitoring the infrastructure and Flood in place as the cloud load testing tool. The following breakdown may help you if you are interested in setting up a similar environment:

Application Configuration

The internet Heroku app is an open source project created in Sinatra Ruby by tourdedave. We found it perfectly suitable to be the core of this scenario, and we deployed the Docker container version into a t2.micro instance on AWS running Ubuntu 18.04

New Relic Configuration

If you don't have experience with APM tools, check out  New Relic because they provide videos, tutorials, courses, and certification at New Relic University which we found easy to get started. The installation of the infrastructure agent was quite simple, with easy instructions we were able to get it done in just a couple of minutes. However, the ruby specific application agent took a bit longer to get appropriately configured due to our lack of skills in this area, and the fact that the internet Heroku app runs on an older version of Ruby which is now less common.

Flood Configuration

Integrating Flood with New Relic is reasonably straightforward, with only a License Key Required.   Once the integration is active, you will need to choose a scripting tool (like JMeter) to create a test that you can execute in Flood.  We created a simple example test in JMeter you can upload and try yourself using this getting started guide for JMeter Load Testing in Flood.

Dashboard in New Relic Insights

The primary purpose of setting up this example was to get all the counters, metrics, and events together in connected dashboards within New Relic Insights. We start by generating load with Flood that hits the internet Heroku app with 50 users (fairly significant load for our t2.micro instance). Then the monitoring agents and aggregated application and infrastructure metrics were available in New Relic's dashboard, along with the Flood plugin counters.

If you are successful with your configuration, you should be able to create a dashboard much like the below:

Application Agent (Web transactions time, Apdex score, Throughput)

Infrastructure Agent (CPU, Load, Memory) also possible Network, Disk, and Processes.

The load testing scripts we developed in JMeter (at the protocol level) and  Flood Element (at the browser level). Once that the scripts were running, we started to get a rich influx of data from both Flood and our infrastructure, helping to identify the most time consuming scenarios:

One of our favorite features in New Relic Insights is the wealth of VM stats. Developers and architects will find it extremely valuable at the time of configuring or tuning the garbage collector algorithm to enhance the application's performance.

We also enjoyed taking a look at the thread profiling in New Relic Insights. It collects information for 5 minutes, and those results provide a clear break down of the internal processing. It's recommended to execute it with low loads so as not to collect too much information to be able to sift through. We believe this feature is beneficial for developers or architects when requesting low-level details.

Lastly, you can see Flood counters in the Plugins section of New Relic Insights. Unlike the native New Relic metrics, you will be required to build your dashboards with the values provided by Flood since they are custom, external metrics. You shouldn’t need to spend more than a few minutes to generate the chart for each metric, and once you have it you can clone it and reuse it across your future New Relic Insights dashboards.

Other Notes

A feature that we didn't test because it's out of scope but could be vital when using an APM tool is configuring alerts. This should be an easy task if you have the threshold or SLA that you wanted to monitor.

Putting It All Together

As we mentioned earlier, the primary purpose of this exercise is having all the pieces together in a single screen. The journey of configuring agents and developed scripts may seem daunting, but the benefits of tools that increase application performance will last for months if not years. In the image below you might see eight snippets, the first four were collected by the App agent, the following two linked to Flood integration counters and the last ones were aggregated by the infrastructure agent.

If you want to give this a try on your own, provision a free trial of Flood in less than 5 minutes.  Once in the trial, you can head to the integrations area to integrate your APM tool and create a complete view of your application’s performance. If you need a trial of New Relic, sign up for one so you can get the complete experience. We hope you enjoy the future of cloud load testing + APM!

Start load testing now

It only takes 30 seconds to create an account, and get access to our free-tier to begin load testing without any risk.

Keep reading: related stories
Return to the Flood Blog