JMeter Performance and Tuning Tips

0saves

    Context

You want to make a Load Test that will use hundreds to thousands of Threads and you’ve read on so many blogs that :

  • JMeter consumes a lot of memory…
  • JMeter generates OutOfMemory…
  • JMeter consumes a lot of CPU…
  • JMeter can only cope with 100 threads and not much more…
  • JMeter is fine for playing but once it becomes serious use another tool :-)

This has become a kind of “Urban Legend” partly due:

  • to issues that have been fixed for a while now
  • and partly in my opinion to some default configuration parameters that lead to these issues
  • misuse of elements that are to be used during scripting and never during Load Test

Hopefully, You’re in the right place, Everything you read is WRONG. :-)

In this short article, we will see how to get the most out of JMeter.

JMeter Performance Basic concepts

Understand important concepts about JMeter:

  • The more it computes to create or process your samples, the less time it spends sampling, so when doing custom coding, ensure it is efficient
  • Listeners receive Sample Results and do some processing with it, this takes resources (memory, CPU) so during Load Testing, a very simple rule is to REMOVE ALL LISTENERS ! , you will ask How can I use my results then ? , the answer is, you will do it after the run. So NEVER EVER have one of these listeners in your Test Plan during a Load Test :
    • View Results in Table => OutOfMemory guarantee
    • View Results in Tree => OutOfMemory guarantee
    • Graph Results => Performance issues
    • Assertion Results => OutOfMemory guarantee
    • Comparison Assertion Visualizer
    • Distribution Graph (alpha) => Performance issues
    • Graph Results => Performance issues

Performance checklist

Here are our tips to get the most out of JMeter:

Always use the last version of JMeter

JMeter performances and memory usage have been highly improved in last 3 versions and upcoming 2.9 version will bring new set of improvements regarding memory and CPU:

Use NON-GUI for Load Test

JMeter has 2 modes:

  • GUI mode is for creating the test plan, checking it, debuging it BUT NOT FOR MASSIVE LOAD TEST . AWT Event Thread will disrupt your Load Test.
  • NON-GUI mode is for massive load testing, it is as simple as:
<JMETER_HOME>/bin/jmeter -t <Path to Test Plan> -n -l <path to results>/results.csv

Configure JMeter Java options to meet your requirements

Default JMeter java configuration comes with 512 Mo and very little GC tuning.

First ensure you set -Xmx option value to a reasonable value regarding your test requirements.

Then change MaxNewSize option in jmeter file to respect the original ratio between MaxNewSize and -Xmx.

Finally try tuning GC options only if you master this domain.

Use CSV as output for SaveService

XML is verbose, it takes resources to be written (CPU and memory) and for analysis, CSV is great so forget about XML. Furthermore, for massive load tests there are many result data you don’t need.

So, in user.properties, add:

jmeter.save.saveservice.output_format=csv
jmeter.save.saveservice.data_type=false
jmeter.save.saveservice.label=true
jmeter.save.saveservice.response_code=true
jmeter.save.saveservice.response_data.on_error=false
jmeter.save.saveservice.response_message=false
jmeter.save.saveservice.successful=true
jmeter.save.saveservice.thread_name=true
jmeter.save.saveservice.time=true
jmeter.save.saveservice.subresults=false
jmeter.save.saveservice.assertions=false
jmeter.save.saveservice.latency=true
jmeter.save.saveservice.bytes=true
jmeter.save.saveservice.hostname=true
jmeter.save.saveservice.thread_counts=true
jmeter.save.saveservice.sample_count=true
jmeter.save.saveservice.response_message=false
jmeter.save.saveservice.assertion_results_failure_message=false
jmeter.save.saveservice.timestamp_format=HH:mm:ss
jmeter.save.saveservice.default_delimiter=;
jmeter.save.saveservice.print_field_names=true

Use Post-Processor and Assertion efficiently

Post-Processor and Assertions have a cost.

Ensure you use them when required and use the ones that consume the less memory and CPU:

Use Regular Expression Extractor

Use Regular Expression Extractor for extracting data BUT never ever check Body (unescaped), choose among:

  • Body
  • Headers
  • URL
  • Response Code
  • Response Message

Use efficient Regular expressions and extract as less data as possible

Avoid XPath Extractor when possible

XPath builds a DOM tree so it consumes CPU and memory, prefer Regular Expression extractor

Use Response Assertion or Size assertion

These 2 implementations fit 99% of requirements.

Avoid costly ones as:

  • XML Assertion
  • XML Schema Assertion
  • XPath Assertion

Use JSR 223 + Groovy for Scripting

You have a lot of options to do scripting with JMeter:

  • Beanshell
  • BSF and all supported languages Javascript, Scala , Groovy, Java …
  • JSR223 and all supported languages Javascript, Scala , Groovy, Java …

Although you can be lazy and choose the language you know, FORGET ABOUT IT.

Use the most efficient option, which is JSR223 + Groovy + Caching (supported since JMeter 2.8 in external script and in next upcoming JMeter 2.9 also supported with embedded scripts).
See:

Using Groovy is as simple as adding groovy-VERSION-all.jar in <JMETER_HOME>/lib folder.

But of course ensure your script is necessary and efficiently written, DON’T OVERSCRIPT

Generate Reports AFTER the Run

Always generate Graphs and Reports AFTER the Load Test and NEVER EVER DURING IT

You can use JMeter reports:

  • Aggregate Graph
  • Response Time Graph
  • JMeter Plugins graphs

Distributed (Remote) Testing

Once you reach the limits of one machine, you can switch to distributed or remote testing.

But JMeter defaults are not fine for efficient remote testing, so in user.properties, add:

mode=StrippedBatch

This will:

  • remove some data from the SampleResults as the response body, but do you need response body during a High Load Test, NO, DEFINITELY NO !
  • will send Sample Results as Batches and not for every sample reducing CPU, IO and network roundtrips

When it’s not enough

  • Use one of the Cloud Solutions that use Apache JMeter

See also

Author

Philippe Mouawad

 

0saves
If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.