API Load Testing: RESTful APIs

Using Ruby JMeter to test a RESTful API

In this post we're going to show you how to perform an API load test against a RESTful API using our popular Ruby-JMeter gem. If you only use JMeter, don't worry, the gem will also output the JMX format so you can use the examples here without the gem as well.

The Test Application

We're using an example application for this demonstration, based on code taken from a popular Railscast about API versioning.

Our application lets users create, read, update or delete products. The application can be accessed using a browser or via a RESTful API. The following product list is just a simple index view of all the products. It looks like this in HTML.

We can also make the same call using a HTTP GET to the products API endpoint with a response in JSON like this.

Using Ruby-JMeter

This gem lets you write test plans for JMeter in your favorite text editor, and optionally run them on Flood IO. It's great for users who want to skip using the JMeter GUI and express their test plans in a succinct, easy to read and shared format.

So instead of looking at something like this in JMeter:

We can write the test plan like this in Ruby-JMeter.

bash
require 'ruby-jmeter'

test do
 with_json
 threads 1, loops: 5 do
   get name: 'get_products_index',
       url: 'http://example-rest-api.herokuapp.com/api/products'
 end
end.run

HTTP Verbs / Methods

Most RESTful APIs will respond to HTTP verbs (or methods) such as POST, GET, PUT, and DELETE. These normally relate to Create, Read, Update, and Delete (CRUD) operations.

Our test application supports the following routes.

bash
GET /api/products api/v2/products#index
POST /api/products api/v2/products#create
GET /api/products/:id api/v2/products#show
PUT /api/products/:id api/v2/products#update
DELETE /api/products/:id api/v2/products#destroy

So tying that all together, we can extend our Ruby-JMeter test plan to cover some of these other methods as follows.

Show all Products using GET /api/products

The HTTP GET verb is often used to retrieve (or read) a representation of a resource. We've already demonstrated the index view of our products which lists all the products in our catalog using a GET as follows.

`get name: 'get_products_index', url: "#{base_url}/products"`

Create a Product using POST /api/products

The POST verb is often used for creation of new resources. In order for us to create a new product we need to provide some additional parameters for the product itself via the fill_in parameter as follows.

bash
post  name: 'create_new_product',
     url: "#{base_url}/products",
     fill_in: {
       "product[name]"        => 'Thomas the Tank Engine',
       "product[price]"       => 9.99,
       "product[released_on]" => Time.now
     }

We should also validate the response and extract the newly created product ID so we can use it in subsequent requests. If the request is processed without errors we should expect a HTTP/1.1 201 Created response code along with a response body in JSON that looks like this.

bash
{
 "category_id":null,
 "created_at":"2014-05-16T02:41:46Z",
 "id":30,
 "name":"Thomas the Tank Engine",
 "price":"9.99",
 "released_on":"2014-05-16T12:41:01Z",
 "updated_at":"2014-05-16T02:41:46Z"
}

We can check for the same in our test plan using the assert and extract methods on the response body like this.

bash
post  name: 'create_new_product',
     ...
     } do
       assert equals: '201', test_field: 'Assertion.response_code'
       extract name: 'product_id', regex: '"id":(\d+)'
end

We've already published a more complete guide to using JMeter Regular Expressions which might be of help. JMeter-Plugins also provide a useful JSON path extractor if you don't want to deal with regex.

Show a Product using GET /api/products/:id

Now that we've created a product, we can use its product ID extracted during the creation and use the GET verb to show the matching product in the database. We'll also assert that the product name is the same as the product we created as follows.

bash
get name: 'get_products_show', url: "#{base_url}/products/${product_id}" do
 assert substring: 'Thomas the Tank Engine'
end

Update a Product using PUT /api/products/:id

Now that we've created a product, we can use its product ID extracted during the creation and use the PUT verb to update its attributes. If the request is processed without errors we should expect a HTTP/1.1 204 No Content response code along like this.

bash
put name: 'put_products_edit',
   raw_path: true,
   url: "#{base_url}/products/${product_id}?product[name]=Salty the Steam Engine&product[released_on]=#{Time.now}" do
     assert equals: '204', test_field: 'Assertion.response_code'
end

Notice in this case we used the raw_path parameter in order to modify attributes via query parameters instead.

Delete a Product using DELETE /api/products/:id

Finally we can delete a product using the DELETE verb. It's as straightforward as this.

bash
delete name: 'delete_product', url: "#{base_url}/products/${product_id}" do
 assert equals: '204', test_field: 'Assertion.response_code'
end

Ready for Load Testing

Once you've completed your test plan, you can scale out and run the test on distributed infrastructure in the AWS cloud using Flood.

As promised, if you don't want to use Ruby-JMeter you can use the JMX formatted test plan available here.

Upload your test plan using our GUI, or use our own API to start your load test.

bash
ruby test/performance/flood_load_test.rb
I, [2014-05-16T13:48:50.572687 #54778]  INFO -- : Flood results at: https://flood.io/1YTVqUGoN1fH9hcRtCIIjg

Start load testing now

It only takes 30 seconds to create an account, and get access to our free-tier to begin load testing without any risk.