CDN with Balancing and Autoscaling Powered by Amazon Web Services

In our blog, we have mentioned the practical application of CDN many times already. This includes broadcasts of auctions, horse races and sports events. As well as broadcasts of webinars, master classes and online lessons.

Indeed, the need for low-latency WebRTC video broadcasts is already well established in our lives. We propose to consider another option for deploying CDN with Elastic Load Balancing and auto scaling in the Amazon Web Services (AWS) environment.

AWS services have long been a favorite among developers of all tiers for their scalability, reliability, and ease of use. It is possible to deploy a virtual server of any desired configuration in a few clicks. Thanks to the automatic scaling system, when a resource-intensive task appears, hundreds of servers can be deployed in a short time, on which the load will be distributed, and which, upon completion of the work, will be stopped to save money on payment.

As you may probably know, Web Call Server is adapted for launching in Amazon EC2 environment in a few clicks. WCS instances support load balancing using AWS Elastic Load Balancing technology, which automatically distributes WebSocket connections across multiple Amazon EC2 instances. If the load on the server processor reaches the value defined as a trigger in the scaling policy, new WCS instances will be launched and added to the balancer.

That is, it is possible to automatically deploy a server based on a ready-made image — either personally assembled one or downloaded from the AWS Marketplace.

Running a load balancer with autoscaling based on a self-assembled WCS image will be convenient for tasks with a long period of operation (months, years). For such tasks, the cost of using an hourly WCS license, which is available when launching an image from the AWS Marketplace, will be significantly higher. Therefore, we recommned purchasing a monthly license. Then manually deploy WCS on an Amazon EC2 instance, activate the purchased license, configure the server, and create an image for automatic deployment.

On the contrary, for tasks that run for short periods (hours, days), we recommend using an image-based autoscaling load balancer launch from the AWS Marketplace. An hourly license that is automatically activated when the instance is launched will be optimal in this case.

Below, we will take a closer look at how to deploy a CDN with a load balancer and autoscaling based on a WCS image from the AWS Marketplace and test the deployed system.

CDN Deployment with Balancer and Autoscaling

The CDN configuration will be as follows:

• one origin server;
• one Edge server as part of a load balancer that allows to increase the number of Edge servers to three during peak loads.

Deployment requires configuring the following components in the Amazon EC2 console:

• WCS Origin instance;
• load balancer;
• launch template;
• scaling group.

Launching WCS Origin

Select “Instances” in the “Instances” menu section on the left side of the AWS console. Click “Launch Instance”:

In the Instance Launch Wizard that opens, select AWS Marketplace and use the search bar to find Flashphoner Web Call Server. Click “Select”:

Read the information about the image and click “Continue”:

Select the type of virtual machine for the instance and click the “Next: Configure Instance Details” button:

At the bottom of the “Configure Instance Details” page, expand the “Advanced Details” section and paste the WCS update and settings script into the “User Data” field (we will consider an example of a script below). Other settings can be left as default. After adding the script, go to the “Configure Security Group” wizard page:

On the page “Configure Security Group” create a new security group for the firewall, or use a previously created group. By default, all required ports are available from the image settings. Click “Review and Launch”:

Review the parameters of the created instance and, if there are no errors, click the “Launch” button:

A WCS server instance with the CDN Origin role will be launched. Find and write down (or remember) the Origin WCS IP address on the AWS intranet — you will need it for further configuration:

Script for updating WCS to the latest version and setting up the CDN Origin role:

#!/bin/bash

# Stop WCS before reconfiguring
PID="$(pgrep -f 'com.flashphoner.server.Server' | grep -v bash)"
if [ -n "$PID" ]; then
service webcallserver stop
fi

# Update WCS to the latest build (optionally, set to false if you don't)
UPDATE=true
if $UPDATE; then
cd /tmp
wget --timeout=10 --no-check-certificate https://flashphoner.com/download-wcs5.2-server.tar.gz -O wcs5-server.tar.gz
if [ $? -eq 0 ]; then
mkdir -p FlashphonerWebCallServer-5.2-latest && tar xzf wcs5-server.tar.gz -C FlashphonerWebCallServer-5.2-latest --strip-components 1
cd FlashphonerWebCallServer-5.2-latest
chmod +x install.sh
./install.sh -silent
cd ..
rm -rf FlashphonerWebCallServer-5.2-latest wcs5-server.tar.gz
fi
fi

# Configuration setup
WCS_CONFIG=/usr/local/FlashphonerWebCallServer/conf/flashphoner.properties
JVM_CONFIG=/usr/local/FlashphonerWebCallServer/conf/wcs-core.properties

#CDN settings
CDN_ROLE=origin
CDN_IP=0.0.0.0
echo -e "\ncdn_enabled=true" >> $WCS_CONFIG
echo -e "\ncdn_ip=$CDN_IP" >> $WCS_CONFIG
echo -e "\ncdn_role=$CDN_ROLE" >> $WCS_CONFIG
echo -e "\ncdn_nodes_resolve_ip=false" >> $WCS_CONFIG

# Request keyframes from WebRTC publishers every 5 seconds
echo -e "\nperiodic_fir_request=true" >> $WCS_CONFIG

# Disable RTMP keepalives to publish from OBS
echo -e "\nkeep_alive.enabled=websocket,rtmfp" >> $WCS_CONFIG

# Configure heap settings
HEAP_SIZE=512m
sed -i -e "s/^\(-Xmx\).*\$/\1$HEAP_SIZE/" $JVM_CONFIG

# Start WCS after reconfiguring
PID="$(pgrep -f 'com.flashphoner.server.Server' | grep -v bash)"
if [ -n "$PID" ]; then
service webcallserver restart
else
service webcallserver start
fi

# Disable internal firewall, ports are allowed/blocked on security group level
iptables -F

Load Balancer

In the AWS console, select “Load Balancers” from the menu on the left side of the page and click the “Create Load Balancer” button on the page that will appear:

Next, select the type of load balancer — “Classic Load Balancer”. This type allows to assign the necessary ports to monitor the server status. Click the “Create” button for the selected balancer type:

The balancer setup wizard will open. On the first tab, specify the name of the balancer to be created and the ports and protocols required for operation. To move to the next setting step, click “Next: Assign Security Groups”:

On the next tab, specify a security group and click “Next: Configure Security Settings” :

In the third step, you can upload SSL certificates to authenticate the servers. Select the “Upload a certificate to IAM” item in the “Certificate type“ section and upload the certificates. Then click “Next: Configure Health Check”:

The fourth step of the configuration allows to configure the status check of servers included in the load balancer. The traffic will only be directed to servers that have passed the performance check. If the server fails validation, it is automatically removed from the balancer.

Use the following URLs to monitor status:

via HTTP: http://<WCS instance DNS name>:8081/?action=stat
via HTTPS: https://<WCS instance DNS name>:8444/?action=stat

Adjust the settings and click “Next: Add EC2 Instances” :

In the fifth step of the configuration, servers are added to the balancer. At the moment, we have only launched an instance with the CDN Origin role, and we are setting up the balancer for instances with the CDN Edge role, so at this step we do not add the existing instance to the balancer. Click the “Next: Add Tags” button:

The next step is to assign tags to the balancer. If necessary, specify the tags and click the “Review and Create” button:

And finally, the final step. Check the configuration of the balancer and click the “Create” button:

After a few seconds, we will get a message about the successful creation of the balancer. Click “Close” to return to the AWS console:

After creating the balancer, we move on to creating a launch template to automatically create new instances.

Launch Template

In the left side menu, select the “Launch Templates” item in the “Instances” section and click the “Create launch template” button:

The Launch Template Wizard opens. In the “Launch template name and description” section, specify the name and description of the template:

In the next section “Amazon machine image (AMI)” select a fresh Flashphoner WCS image from the drop-down list. For speed and convenience, you can use the search:

In the sections “Instance type” “Key pair (login)” “Network settings” select the type of instance to create, a key pair for accessing the instance via SSH, set the network type, and select the security group for the firewall:

In the next section “Storage (volumes)”, configure the size and parameters of the hard disk for the instances that will be created based on this template:

Expand the section “Advanced details”. Find the “User data” field and insert the script to update WCS and configure the CDN Edge role, and then click “Create launch template”:

Script code listing for updating WCS and configuring the CDN Edge role. For the script to work correctly, specify the internal address of the CDN Origin instance in the “CDN_POINT_OF_ENTRY” variable:

#!/bin/bash

# Stop WCS before reconfiguring
PID="$(pgrep -f 'com.flashphoner.server.Server' | grep -v bash)"
if [ -n "$PID" ]; then
service webcallserver stop
fi

# Update WCS to the latest build (optionally, set to false if you don't)
UPDATE=true
if $UPDATE; then
cd /tmp
wget --timeout=10 --no-check-certificate https://flashphoner.com/download-wcs5.2-server.tar.gz -O wcs5-server.tar.gz
if [ $? -eq 0 ]; then
mkdir -p FlashphonerWebCallServer-5.2-latest && tar xzf wcs5-server.tar.gz -C FlashphonerWebCallServer-5.2-latest --strip-components 1
cd FlashphonerWebCallServer-5.2-latest
chmod +x install.sh
./install.sh -silent
cd ..
rm -rf FlashphonerWebCallServer-5.2-latest wcs5-server.tar.gz
fi
fi

# Configuration setup
WCS_CONFIG=/usr/local/FlashphonerWebCallServer/conf/flashphoner.properties
JVM_CONFIG=/usr/local/FlashphonerWebCallServer/conf/wcs-core.properties

#CDN settings
CDN_ROLE=edge
CDN_IP=0.0.0.0
CDN_POINT_OF_ENTRY=172.31.43.82 #IP address CDN Origin
echo -e "\ncdn_enabled=true" >> $WCS_CONFIG
echo -e "\ncdn_ip=$CDN_IP" >> $WCS_CONFIG
echo -e "\ncdn_role=$CDN_ROLE" >> $WCS_CONFIG
echo -e "\ncdn_point_of_entry=$CDN_POINT_OF_ENTRY" >> $WCS_CONFIG
echo -e "\ncdn_nodes_resolve_ip=false" >> $WCS_CONFIG

# Configure heap settings
HEAP_SIZE=512m
sed -i -e "s/^\(-Xmx\).*\$/\1$HEAP_SIZE/" $JVM_CONFIG

# Start WCS after reconfiguring
PID="$(pgrep -f 'com.flashphoner.server.Server' | grep -v bash)"
if [ -n "$PID" ]; then
service webcallserver restart
else
service webcallserver start
fi

# Disable internal firewall, ports are allowed/blocked on security group level
iptables -F

The next step in the deployment will be the creation of scaling groups.

Scaling Group

In the menu on the left side of the EC2 Console page in the “Auto Scaling” section select “Auto Scaling Groups”:

In the group creation wizard that opened, the first step will be to specify the launch template for new instances that we created earlier. Switch the radio button to use a template and specify the desired template. To go to the next step, click the “Next Step”:

In the next step of the group creation wizard, specify a name for the group and select the “Latest” version for the launch template. Select the “Combine purchase options and instances” item and specify the type of virtual machine for the instance. Deactivate the “Use the default settings to get started quickly” checkbox and set the ratio between the virtual machines used:

Below on this page, expand the “Advanced Details” section, activate the “Receive traffic from one or more load balancers” checkbox, and specify the balancer that we created earlier in the “Classic Load Balancers” fiald. After adjusting all the settings, click “Next: Configure scaling policies”:

On the next page of the wizard, select the “Use scaling policies to adjust the capacity of this group” item. Specify the minimum and maximum number of instances in the AutoScaling group and set the processor load values and the time during which the load should remain to start an additional instance. After specifying the settings, click the “Review” button:

On the “Review” page, check the settings and click “Create Auto Scaling Group”:

We will get a message about the successful creation of the scaling group. Click “Close” to return to the console:

This completes the deployment phase. If no instances were launched in the balancer, the new instance will start automatically when the scaling group is created. When the processor load of this instance exceeds 80%, the scaling condition will be met and additional instances will be launched.

Testing

In addition to the above-prepared instances on AWS, the following will be required for testing:

• Google Chrome;
• hardware or virtual webcam for organizing video stream broadcasting.

In the Google Chrome browser, open the WCS server web interface using the DNS name of the instance with the CDN Origin role:

https://<CDN Origin Instans DNS name>:8444

Log in to the web interface and open the “Two-way Streaming” example. Establish a WebSocket connection to the server and publish the video stream:

Then launch the web interface of the WCS CDN Edge server using the DNS name of the balancer:

https://<Load balancer DNS name>:8444

Log in to the web interface and open the “Media Devices” example. Establish a WebSocket connection to the balancer. In the right column of settings, uncheck the “default” checkbox for the Size parameter and set the values for transcoding the video stream. For example, if the stream on the Origin server is published in 320×240, set the value to 640×480. Repeat the steps in several browser tabs to simulate a large number of viewers:

Transcoding is quite a resource-intensive process and will trigger the launch of additional Edge servers.

For this testing, we lowered the scaling threshold to 10% CPU utilization. As you can see in the screenshot below, after the processor load threshold was exceeded for more than a minute, two additional instances were launched to reduce the load:

To ensure that connections are distributed between active instances in the load balancer, use the statistics page:

http://<WCS instance DNS name>:8081/?action=stat

which has to be opened for each of the instances included in the balancer. The statistics page has a “connection_websocket” value that shows the number of active WebSocket sessions on the server:

Setting Up Domain Names

For registering and managing domain names, there is Amazon Route 53, which provides users with a highly available and scalable Domain Name System (DNS). The “Route 53” service routes user requests to AWS infrastructure, such as Amazon EC2 instances or Elastic Load Balancing load balancers.

To set up domain names for the CDN that we created above, follow these steps:

Open the “Route 53” service management console:

Register a new domain name or transfer an existing one:

After registering or transferring a domain name, select “Hosted zones” from the menu on the window’s left side and go to the created domain zone:

To create new A records of servers included in the CDN, click the “Create Record Set” button:

Create entries for the Origin server and load balancer:

As a check, try to contact the servers included in the CDN using the created domain names:

Result

As a result of all the work, we created a CDN with a load balancer and automatic scaling, with which you can conduct any video broadcasts that require low latency. The balancer helps to level out peak loads, even with such resource-intensive operations as video transcoding, so that the broadcast latency remains low. As for autoscaling, it helps save money on licenses and virtual machine rentals because servers are only launched when needed. Therefore, it will not be superfluous to spend time setting up once, so that later automatic services will always be on standby.

Enjoy your streaming!

Links

Demo

WCS on Amazon EC2