Case Study from the Entertainment industry: Streaming load test for a live theatre show

on 30 May 2022 AWS, CLOUD, HLS, JMETER, LOAD_TESTING, MPEG-DASH, OTT, PERFORMANCE, VIDEO and Tags: , , , , , , , , , , with 0 comments

How to ensure the best quality of viewing experience during the live transmission of a theatre show ?

By performance testing video streaming services with UbikLoadPack

Recently, one of our client wanted to verify that their new streaming infrastructure could properly handle the delivery of a live theatre show up to 20 000 watchers.

Their solution had seen increased traffic and success following COVID19 pandemic and theatre being forced to close their doors.

This streaming infrastructure was composed of

  • A custom video streaming solution delivering Live HLS developed in Nods JS and built around MongoDB, Redis, Nginx, Docker, AWS ELB in the first phase
  • This application was hosted in AWS cloud initially and then moved to Scaleway Cloud

They wanted to check multiple things:

  1. Customers user experience (no lags, accuracy of stream, no delay), they had previously faced a production incident leading to bad buzz
  2. Size correctly the infrastructure
  3. Check the performance after Cloud migration
  4. Get deep analysis by our Professional Services who investigated the scalability problems, the application code to provide ways to handle more efficiently the expected load. 

There main upcoming event was a national one, where customers would be watching a live theatre show from a very popular comic showman in France.

So they contacted our professional services for a turnkey streaming load test service. 

 

Which tools we used and why ?

Since we needed to run a Performance Test for a Cloud hosted HLS Live stream,  we used:

UbikLoadPack Video Streaming plugin + Apache JMeter  as the tool to simulate users watching Live streams delivered in HLS

UbikLoadPack Video Streaming plugin fits customer requirements in the following fields:

  • Reproduce realistically and very easily a player reading Live stream delivered in HLS. There is no need to write complex and unmaintainable test plan to reproduce the player behaviour, knowing that the multitude of formats and possibilities of Live Streaming makes this very complex and time consuming.
  • It provides custom metrics to analyze User watching video experience
  • It has the ability to graph those metrics
  • It is availability from the Cloud and On Premise. In this case the customer wanted to run the test from the Cloud

Redline13 fits customer requirements in the following fields:

  • Easily deploy JMeter + UbikLoadPack with a single click from a web page on multiple servers in the AWS Cloud
  • Start those servers from all over the world using AWS datacenters from a simple Web Page, in this particular case, we used 5 European Datacenters and 150 Servers to spread the load
  • Live feedback on performance test
  • Merged results and metrics in one CSV accross all nodes
  • Get the JMeter report at end of test

The tools chosen were:

UbikLoadPack streaming plugin to validate OTT video streaming performances

Let’s see how simple it is to setup a test using UbikLoadPack.

The outcome of our Professional Service:

Our professional services consultant worked with developer of the custom solution to:

  • Diagnose the stability and performance problems
  • Advise on fixing concurrency problems in the code, since our consultants are also senior developers with knowledge of different technologies (Java, PHP, NodeJS, .NET) and scalable architectures
  • Advise on improvements that could be applied to the software architecture
  • Help sizing and tune different components:
    • Nginx 
    • System stack
    • SSL Layer

The metrics and results of this streaming load test:

Few numbers about this streaming performance test:
  • 20 000 Virtual Users simulated reastically
  • More than 175 000 requests per minute
  • Around 2 Tera Bytes received
  • Customer Platform scalability jumped from ability to hold 2 000 concurrent customers to 20 000

Average response time

streaming load test average response time theatre live show

Requests per minute

streaming load test requests per minutes theatre live show

Summary report of load test metrics of a 20 000 Virtual Users shoot 

streaming load test metrics entertainment theatre live show summary

Thanks to this load testing campaign, our customer was able to:

 streaming load test for a live theatre show up to 2500 watchers

Run a streaming load test with 20 000 players on premise hosting

Diagnose multiple configuration problems

Diagnose and fix multiple application problems and optimize it

Provide his end-user customer metrics showing that their services was delivering streams as per requested SLR

Want to test ?

Request a trial version HERE.

Ask for a turnkey or consulting streaming load test service at ubik-load-pack-sales@ubik-ingenierie.com.

 

About us:

You’ll probably also like: