Performance Testing with JMeter

Introduction

This article is about using JMeter to perform some basic performance and concurrency testing on JasperReports Server. In this article, we will tackle some different scenarios:

• Executing canned reports
• Creating Ad Hoc Reports
• Simulating Concurrency

See also these screencasts for reference:

The numbers that the JMeter application generates could be used for benchmarking performance for different environments, basic load testing, and measuring gains after tuning.

The numbers that JMeter produces are best used and interpreted when we can compare them to another set of results, with each set of results representing a different set of conditions (e.g., different memory settings, different network throughput, different number of concurrent users, etc.)

The numbers will also vary from customer to customer, mainly because a customer's requirements vary; report size, query complexity, network latency, software versions and configuration as well as concurrency requirements all vary from customer to customer. When writing this article, I used the foodmart sample database that ships with JasperReports Server, since it gives any customer can use it if they've installed it. For systems with larger databases, the process would be the same.

Downloading and Installing JMeter

JMeter is a desktop Java application, and you can download it from http://jakarta.apache.org/jmeter. The instructions below deal with JMeter version 2.4. Ideally you would install JMeter on a development and testing machine, with that machine being able to access the instance of JasperReports Server you want to test.

Configuring JMeter

With the JMeter proxy in place, you can create a test plan by simply visiting the web site you wish you test. Clicking around the website will add entries to your test plan. This is what it looks like when you first kick it off:

Create the Thread Group

1. Start JMeter
2. Select "Test Plan" on the tree
3. Right click on the "Test Plan" and add a new thread group: Add -> Threads -> Thread Group
4. Select the Thread Group
   1. At this stage, you can also define the number of threads to run when executing the test plan, which will mimic the number of concurrent users. I suggest using 50 as the initial value.
5. Right click and choose "Add -> Config Element -> HTTP Request Defaults"
   1. Server Name or IP: Fill it in with the environment you're going to test.
      1. This can be tricky if you're testing it on your own environment (localhost); I got a local test plan working by using a Ubuntu virtual environment running on my PC. JMeter doesn't seem to like recording steps that go through localhost.
   2. Port Number: the port number of the JRS instance you're testing; usually 8080 in default JRS installations
   3. There are boxes at the bottom that you can fill out to configure a proxy server, if necessary

Configure the JMeter Proxy

We're now going to create the Proxy Server using JMeter.

1. On the JMeter application, select workbench, right-click on it, and choose "Add -> Non-test Elements -> HTTP Proxy Server". We will need to modify some of its values:
   1. Port: Change it from 8080 to some other port that's not in use
   2. Target Controller: Choose "Test Plan -> Thread Group".
   3. Click the "Add" button in "Patterns to Include". Since we want to get an idea of overall server performance, add ".*" (minus the double-quotes) to the entry. That pattern means that it will consider all file requests coming from the server. You can limit it by specifying, for instance, only HTML files, only GIF files, etc, by defining the URL pattern that matches the file you want.
   4. Click the "Start" button - this kicks off the proxy

Note: When defining the proxy server, take note that after you save your test plan, and then close down JMeter, it will often appear that you have lost your proxy server configuration; my recommendation is to do the steps outlined in this document all in one go; that way you will have recorded your test plan, which is the main point of using the proxy server and JMeter, all in one go.

Configure your browser to go through the proxy

In this step, we configure the browser to make its HTTP requests go through the proxy server we just set up on JMeter. Once we have configured the browser in this way, all the HTTP requests the browser makes will go through the JMeter proxy server, which will in turn record these steps as part of the test plan we want to create.

You can do this with Internet Explorer, Firefox or Chrome; they each have settings where we can define the proxy server.

The Firefox and Internet Explorer settings go through the same process; the Chrome settings are slightly different, but achieve the same goal.

As a developer, since you're just running the JMeter application in your own test environment, it's fine to just set the proxy server to "localhost", since you are running JMeter locally anyway.

Create the Test Plan Using the Browser

The goal of the test plan will be to go through JasperReports Server UI, login, execute a series of reports, and create a sample ad hoc report. We will then execute the test plan. We can imagine a test plan covering at least these cases:

Now that you have the proxy server set up and running, and you have configured your browser to pass all the HTTP requests via the JMeter proxy server, you can now record your test plan. The general procedure is this:

• Use your browser to login to JasperReports Server
• Perform the functionality that you wish to load test
• Quit your browser when done

When you switch back to the JMeter application, you will find that JMeter has recorded the HTTP requests you executed in your browser:

The list of items on the left-hand side show all the HTTP requests that your browser made when interacting with JasperReports Server.

Turning on the Reporting Function

Now that we have the test plan defined, we'll switch on the reporting function, so that once we execute the test plan, we'll be able to see how well the application performed.

1. On the JMeter application, select the "Thread Group"
2. Right-click on the "Thread Group", and choose "Add -> Listener -> Aggregate Report"
3. With the "Thread Group" still selected, choose the number of threads, which represent the number of concurrent users.

Run the Test Plan

Now you are ready to kick off the test plan. You do this by choosing "Run -> Start" from the JMeter top-level menu.

While the test plan is running, you will see that the aggregate report is being filled in at the same time, and a little green box on the top-right corner of the JMeter application appears, indicating that it is currently running.

Note that there is a "Save Table Data" button - this will allow you to save the data the test plan generated, which you can then use to compare with different test scenarios.

Analyze JMeter logs

Use the aggregate report to test different environments, and not just the thread count. The environment includes elements like:

• JVM settings
• Application Server Thread Count
• Database Driver versions used
• Network Latency - throughput between different elements in the software stack
• Version of JasperReports Server that is used

Any element that is a candidate for tuning can be added to that list. The idea is to be able to compare JMeter aggregate reports for environments that use the different settings, and thus determine what impact each of the elements we have tuned have had on the overall aggregate performance: did it improve performance or not?

Other Performance Tips:

Measure and Test

• Test DB Queries with DBTester
• Measure Ad Hoc Performance

Ad Hoc

• Set query limits on the "Ad Hoc Options" page
• Modify the Ad Hoc caching values if your data does not update constantly

Tomcat

• Compress HTTP responses at the Tomcat level
• Increase the maximum thread count at the Tomcat level
• Set up a cache filter for Tomcat or JBoss
• Disable automatic deployment on production servers using Tomcat

Database

• Index columns in the database to speed up domain-based reports
• Set the fetch size in the jasperreports.properties file to modify the number of rows to fetch

JasperReports Configuration Items

• Modify Virtualizer Settings for Very Large Reports
• Modify the jasperreports.properties file to disable multi-line data processing

Resources

This article was based in no small part on the document below, which outlines the step-by-step procedure we discussed in this article.

• http://jakarta.apache.org/jmeter/usermanual/jmeter_proxy_step_by_step.pdf