Case Study from the Telco industry: Performance Test video streaming services with UbikLoadPack

on 17 February 2022 AWS, CLOUD, HLS, JMETER, LOAD_TESTING, MPEG-DASH, OTT, PERFORMANCE, VIDEO and Tags: , , , , , , , , , , , with 0 comments
Bad OTT streaming performance lags

How to ensure the best quality of viewing experience during the live transmission of a football event ?

By Performance Testing video streaming services with UbikLoadPack

Recently, one of our client wanted to verify that his infrastructure could properly handle the delivery of a live sport event up to 150 000 watchers.

 

This streaming infrastructure was composed of

  • Customer CDN
  • Origin delivering Live DASH and HLS
  • 2 Datacenters

He wanted to check multiple things:

  1. Customers user experience (no lags, accuracy of stream, no delay)
  2. CDN and Origin delivery performance
  3. Bandwidth of the network
  4. Load balancing across 2 Datacenters
  5. Performance Test his video streaming servers up to 150 000 players

His event was a national one, where customers would be watching an important football match in Europe.

So he contacted our professional services for a turnkey service of a realistic performance test on a HLS & DASH live streams in order to validate or improve his streaming infrastructure before the upcoming event. 


Which tools to use and why ?

His OTT service was composed of an origin delivering Live Dash and HLS manifests using Customer solution.

Since we needed to run a Performance Test for a on premise HLS Live stream,  we used:

UbikLoadPack Video Streaming plugin + Apache JMeter  as the tool to simulate users watching Live streams delivered in HLS

JMeter Threads

JMeter Threads per Second

UbikLoadPack Video Streaming plugin fits customer requirements in the following fields:

  • Reproduce realistically and very easily a player reading Live stream delivered in HLS and MPEG-DASH in this particular case (MS Smooth is also supported). There is no need to write complex and unmaintainable test plan to reproduce the player behaviour, knowing that the multitude of formats and possibilities of Live Streaming makes this very complex and time consuming.
  • Provide custom metrics to analyze User watching video experience
  • Ability to graph those metrics
  • Availability from the Cloud. For the short term, customer wanted to run the test from the Cloud

Redline13 fits customer requirements in the following fields:

  • Easily deploy JMeter + UbikLoadPack with a single click from a web page on multiple servers in the AWS Cloud
  • Start those servers from all over the world using AWS datacenters from a simple Web Page, in this particular case, we used 5 European Datacenters and 150 Servers to spread the load
  • Live feedback on performance test
  • Merged results and metrics in one CSV accross all nodes
  • Get the JMeter report at end of test

The tools chosen were:

UbikLoadPack streaming plugin to validate OTT video streaming performances  

Let’s see how simple it is to setup a test using UbikLoadPack.

 

The metrics and results of this massive load test:

Few numbers about this massive streaming performance test:
  • 150 000 Virtual Users simulated reastically
  • More than 1.2 Million requests per minute
  • Around 4 Tera Bytes received
  • 150 AWS Servers spread across 5 regions

Average response time

live streaming performance test metrics - response time

Requests per minute

live streaming performance test metrics - request per minute

 

Summary report of load test metrics of a 150 000 Virtual Users shoot 

live streaming performance test - summary metrics

Thanks to this massive load testing campaign, our customer was able to:

football live event no lags

 

Run a load test with 150 000 players on premise hosting

Diagnose multiple configuration (Caching, TLS…), sizing problems

Optimize the configuration

Provide his end-user customer metrics showing that their services was delivering streams as per requested SLR

Interested to test ?

Request a trial version HERE.

Ask for a turnkey or consulting service at ubik-load-pack-sales@ubik-ingenierie.com.

 

About us:

You’ll probably also like: