Video Streaming Top 5 metrics to analyze your User’s Experience

on 10 January 2019 ABR-Streaming, Apple HTTP Live Streaming, DEVOPS, ECOMMERCE, HLS, LOAD_TESTING, Microsoft-Smooth, MPEG-DASH, Neoload Alternative, OTT, PERFORMANCE, SMOOTH Streaming, VIDEO and Tags: , , , , , , , , with 0 comments
Make your audience happy !

Some insights on Video Streaming

If you think that load testing your videos streaming platform is a luxury you can do without, here are some numbers to convince you that you should absolutely do it:

  • Live video is more appealing to brand audiences: 80% would rather watch live video from a brand than read a blog, and 82% prefer live video from a brand to social posts.
  • The average conversion rate for websites using video is 4.9%, compared to 2.9% for those who don’t.
  • 4X as many consumers would rather watch a video about a product than read about it.
  • 73% of B2B organizations report positive ROI from video marketing.
  • 23% of people who have been presented with a poor quality video experience would hesitate to purchase from that brand.
  • Companies using video enjoy 41% more web traffic from search than non-users.

All the datas are from livestream and cisco.

Why load test your video streaming servers / OTT ?

This is a legitimate question. And there are several reasons for that :

  • Nowadays, the page load speed is critical for customers experience. The longer a customer waits for the page to load, the higher are the chance that he will leave your website. The problem is the same (and might be even bigger) with videos. If a customer can’t watch a video that describes your main product on your website or faces lags or very long initial buffering, because the traffic is too high on it you will loose a potential buyer. See an example on how load time impacts website abandonment here.
  • Same issue if you didn’t provide the correct format for his device.
  • Load testing your servers allows you to see what an end user will receive under circumstances that you can customize. You will see if the user will face some lags, if the initial buffering takes time, etc … And with those results, you will be able to fine tune your servers to improve the user experience.
  • Secondly, the test will tell to you whether your server infrastructure is able to handle the load incurred by the users traffic. Maybe you need to add/remove a server, increase/lower RAM to your existing servers to avoid lag …
  • As you can also see here, it’s quite important to correctly tune your server/application, our UbikLoadPack Streaming plugin can help you ensure your server serves content with optimal Lag Ratio.
  • With streaming, a positive user experience and sustained audience engagement is highly correlated with video playback performance.

Top 5 metrics to analyze your User’s Experience

Bit rate

This metric is a critical one as:

  • it reflects the quality of the video that you customers are viewing
  • it also impact the network traffic that is going out of your datacenter or hoster

Our solution allows you to define in your load test, the different bit rates you expect to deliver and ensure that you meet the expected SLA in terms of quality, lag ratio …

Buffer Fill time

This is the time user waits before video starts playing.During this time he gets a progress roller.

During this time the first seconds of the video are downloaded. It’s an important metric to check as users may not wait minutes for your video.

Our solution exposes it as ${ulp_buffer_fill}. It is also available in JMeter CSV output and can be graphed with our UbikLoadPack Streaming plugin and integrated to the HTML report

Lag Time

When you face issues, network contention or overloaded server, at some time during the play of the video, there won’t be enough data to play by the player, so it will stop playing and wait for data to be buffered.

This negatively impacts your user experience so you need to ensure that this lag time is acceptable regarding your SLA.

In ideal case, lag time should be equal to buffer fill time.

Our solution exposes it as ${ulp_lag_time}. It is also available in JMeter CSV output and can be graphed with our UbikLoadPack Streaming plugin and integrated to the HTML report

Play Time

This is the “real time” video took to play provided user waits for lags.

Our solution exposes it as ${ulp_play_time}. It is also available in JMeter CSV output and can be graphed with our UbikLoadPack Streaming plugin and integrated to the HTML report

Lag Ratio

Lag ratio is :

lag time / by video total duration

So if user waits 30s for a video of 5 minutes. Lag ratio would be 10%

Our solution exposes it as ${ulp_lag_ratio}. It is also available in JMeter CSV output and can be graphed with our UbikLoadPack Streaming plugin and integrated to the HTML report

How to compute them ?

Our UbikLoadPack Streaming plugin provides those metrics Out Of The Box in a dedicated VideoStreamingSampleResult that is emitted by default every 60s. This can be adjusted in the Video Streaming Sampler.

The response time of the VideoStreamingSampleResult is the sum of the response time of each chunks and manifests downloaded during the period set in the VideoStreamingSampleResult frequency field.

It also allows you to vary bitrate to be able to anticipate expected peaks and ensure your customer experience is a good one.

These metrics are output in the CSV formats provided by JMeter.

They are also available as a Graph over Time, in the “Custom Metric” part of JMeter HTML Report.

Try it:

Read this business case to learn more about our solution.

Request a trial version HERE.

Ask for a turnkey or consulting service at ubik-load-pack-sales@ubik-ingenierie.com.

About UbikLoadPack :

Ubik Load Pack Streaming Solution features