Magento Performance Toolkit
I recently discovered that Magento Team provided a performance toolkit based on Apache JMeter:
It is a great initiative as performance of eCommerce websites is critical nowadays and bad performance can make your customers quit your website.
I was curious to see how it was implemented so I grabbed the JMX Plan and reviewed it.
When you load test eCommerce websites you usually want to go rather high in the number of Virtual Users (Threads in JMeter) you want to run. In order to do so there is a number of easy Best-Practices you need to follow that will let you get the maximum from JMeter reducing the number of machines you need to simulate high traffic.
Magento Performance Toolkit and JMeter Best Practices : What can be improved
Script is rather well organized and some interesting features are used:
- it uses __P function to avoid hard coding values in script and easily switch between environment
- setUp Thread Group and tearDown Thread Group : Those fixtures let you as with JUnit run some script before and at end of Test
- In Thread Group, “Start Next Thread Loop” is checked to avoid running requests after one has failed as usually next ones will fail especially if next requests depend on its response.
But there are some things that should be improved, here is a list that presents the improvements by their type:
- Maintainability : Following these recommendations will make your script more readable and more resistant to changes in the application you are testing.
- Performance : This impacts the performances of JMeter engines. The more you take care of its performance the more Virtual Users you can simulate
- Realistic Tests : Following these recommendations will make your test simulate in a more realistic way the behaviour of users. It is critical that you follow them to ensure your tests are useful.
- Maintainability : Http Request should not have Implementation set, it should be left blank and set it in HTTP Request Defaults.
- Performance/Realistic Tests : Better use HttpClient4 than Java
- Performance : Replace Beanshell by JSR223 + Groovy + Use cache key, it performs 10 times better in terms of throughput thanks to Pre-Compilation of Groovy scripts. Note by the way that Beanshell Sampler should be replaced by a Pre-Processor to avoid outputting it in results
- Performance : In Assertions, instead of Contains, use Substring as you usually don’t need regexp, it will perform way better in terms of CPU
- Performance/Maintainability : In “Simple Product 1 View(BrowsAddToCart)”, use CSS/JQuery Extractor instead of XPath Extractor (XPath Extractor: Extarct form key), it is more suitable for HTML and performs way better than XPath. Apply this everywhere you use XPath for HTML
- Performance/Maintainability : In “Checkout start(CustomerChkt)” use CSS/JQuery Extractor instead of Regexp Extractor(Extract form key), it is more maintainable for HTML and performs better
- Performance/Maintainability : Avoid XPath assertion like in Login(CustomerChkt), combine CSS/JQuery Extractor + Response Assertion using “JMeter Variable” in Response Assertion
- Performance/Maintainability : Disable all Listeners, keep only Simple Data Writer that writes on error and use NON GUI Mode + CSV output. For this read our previous blog:
- https://www.ubik-ingenierie.com/blog/automatically-generating-nice-graphs-at-end-of-your-load-test-with-apache-jmeter-and-jmeter-plugins/. You can possibly output full details on errors by adding Simple Data Writer:
We opened this issue on Magento Github :
Philippe Mouawad works as an Architect and technical expert for Ubik-Ingenierie where he leads among other things the development of UbikLoadPack a set of Commercial Plugins for Apache JMeter allowing to load test different protocols like HLS, GWT, Flex.
Philippe is a committer on the Apache JMeter project and member of the PMC since October 2011.
Finally he is a co-author of a french book on JMeter called Maîriser JMeter: Du test de charge à Devops.
- Ubik Load Pack is a JMeter based solution for complex protocols.
- Follow us on Twitter and StackOverflow.
- Follow Apache JMeter on Twitter
Comments are closed.
- Handy checklist for ensuring User Experience when delivering Video Streams at Large Scale 4 February 2019
- Reporting feature in JMeter 17 January 2019
- Shift left your performance tests with JMeter and Maven 15 January 2019