Magento Performance Toolkit and JMeter Best Practices

on 2 November 2014 DEVOPS, ECOMMERCE, GROOVY, JMETER, JMETER-PLUGINS, LOAD_TESTING, PERFORMANCE, TUNING and Tags: , , , , , , , , , , , with 1 comment

Magento Performance Toolkit

I recently discovered that Magento Team provided a performance toolkit based on Apache JMeter:

It is a great initiative as performance of eCommerce websites is critical nowadays and bad performance can make your customers quit your website.

I was curious to see how it was implemented so I grabbed the JMX Plan and reviewed it.

When you load test eCommerce websites you usually want to go rather high in the number of Virtual Users (Threads in JMeter) you want to run. In order to do so there is a number of easy Best-Practices you need to follow that will let you get the maximum from JMeter reducing the number of machines you need to simulate high traffic.

Magento Performance Toolkit and JMeter Best Practices : What can be improved

Script is rather well organized and some interesting features are used:

  • it uses __P function to avoid hard coding values in script and easily switch between environment
  • setUp Thread Group and tearDown Thread Group : Those fixtures let you as with JUnit run some script before and at end of Test
  • In Thread Group, “Start Next Thread Loop” is checked to avoid running requests after one has failed as usually next ones will fail especially if next requests depend on its response.

But there are some things that should be improved, here is a list that presents the improvements by their type:

  • Maintainability : Following these recommendations will make your script more readable and more resistant to changes in the application you are testing.
  • Performance : This impacts the performances of JMeter engines. The more you take care of its performance the more Virtual Users you can simulate
  • Realistic Tests : Following these recommendations will make your test simulate in a more realistic way the behaviour of users. It is critical that you follow them to ensure your tests are useful.
  1. Maintainability : Http Request should not have Implementation set, it should be left blank and set it in HTTP Request Defaults.Screen Shot 2014-11-01 at 23.37.57
  2. Performance/Realistic Tests : Better use HttpClient4 than JavaScreen Shot 2014-11-01 at 23.39.45
  3. Performance : Replace Beanshell by JSR223 + Groovy + Use cache key, it performs 10 times better in terms of throughput thanks to Pre-Compilation of Groovy scripts. Note by the way that Beanshell Sampler should be replaced by a Pre-Processor to avoid outputting it in results Screen Shot 2014-11-01 at 23.48.14
  4. Performance : In Assertions, instead of Contains, use Substring as you usually don’t need regexp, it will perform way better in terms of CPUScreen Shot 2014-11-01 at 23.50.34
  5. Performance/Maintainability : In “Simple Product 1 View(BrowsAddToCart)”, use CSS/JQuery Extractor instead of XPath Extractor (XPath Extractor: Extarct form key), it is more suitable for HTML and performs way better than XPath. Apply this everywhere you use XPath for HTMLScreen Shot 2014-11-01 at 23.56.54
  6. Performance/Maintainability : In “Checkout start(CustomerChkt)” use CSS/JQuery Extractor instead of Regexp Extractor(Extract form key), it is more maintainable for HTML and performs betterScreen Shot 2014-11-02 at 00.01.13
  7. Performance/Maintainability : Avoid XPath assertion like in Login(CustomerChkt), combine CSS/JQuery Extractor + Response Assertion using “JMeter Variable” in Response Assertion
  8. Performance/Maintainability : Disable all Listeners, keep only Simple Data Writer that writes on error and use NON GUI Mode + CSV output. For this read our previous blog:
  9. REALISTIC TEST : For a realistic test, there should always be a “Think time” to simulate user reading the page or inputting some data, to control precisely every pause your put, you can put a Gaussian Random Timer as a child of Test Action with Pause of 0, thus Test Action will not be output in the resultsScreen Shot 2014-11-02 at 00.05.30Screen Shot 2014-11-02 at 00.05.39
  10. In Test Plan check “Run tearDown thread Groups after shutdown of main threads” to ensure tearDown is run when you stop the test before it endsScreen Shot 2014-11-02 at 00.03.43

Follow up

We opened this issue on Magento Github :

About the author:

Philippe Mouawad works as an Architect and technical expert for Ubik-Ingenierie where he leads among other things the development of UbikLoadPack a set of Commercial Plugins for Apache JMeter allowing to load test different protocols like HLS, GWT, Flex.
Philippe is a committer on the Apache JMeter project and member of the PMC since October 2011.
Finally he is a co-author of a french book on JMeter called Maîriser JMeter: Du test de charge à Devops.

About us:

< Newer Post
Older Post >

1 Comment

Comments are closed.