Forem: Donnie Prakoso

Let's Build Serverless Apps. Wait, Why?

Donnie Prakoso — Mon, 20 Jun 2022 07:16:57 +0000

This article is part of the AWS Community ASEAN content. All presentations and ready-to-deploy codes used in this talk are available at AWS Community ASEAN Content Repo.

Beside containers, serverless technology is something I consider an integral part of modern application development. When I ask developers about "what is serverless?", the perception mostly refers to the definition of "no server". While this is not wrong, it is not entirely true.

Serverless is an operating model.

With serverless, we delegate various activities that do not provide value to our business to other parties. By delegating these things, we can focus on improving business logic and not worrying about infrastructure. At the end, serverless enables us to have less code, less liability, better integration, better application, more focus, more values to deliver. These are what I think the real values for developers.

Looking back on some of the past speaking engagements, I often get the opportunity to explain how we can build a particular solution using the AWS serverless services. Recently, I've realized that I haven't done much at explaining the "why" or various logical reasons for using serverless. And finally I decided to make this content.

Now that we've set the context, let's dive "why" using AWS serverless services might be a good approach for you:

Reason #1: No Server Management

The main reason why using serverless gives you an advantage is that there is no server management. This means that there is no need for patching, retiring aging hardware, and performing day-to-day operations or maintenance.

The most classic example is AWS Lambda. AWS Lambda is a service where we can run a function without the need to manage hardware. At minimum, all we need to do is upload or write our business logic to AWS Lambda.

Like normal functions, we can integrate with various databases, or interact with HTTP API endpoints. To trigger, we can integrate with a variety of other AWS services — such as HTTP API requests with Amazon API Gateway, or when an object is uploaded into Amazon S3.

For those who use containers, you can also use AWS Fargate — a serverless containers compute engine — which you can use to run your apps. Both of these options free you from provisioning to maintaingin your servers, and you can allocate focus, effort and time to what is more important: building features for your application.

Reason #2: Get to Market, Fast and Secure

One of the misconceptions about serverless is that serverless is synonymous with AWS Lambda. The fact is, AWS Lambda is a just one layer of AWS serverless services.

If we refer to serverless as an operational model, in practice serverless can be applied to any layer. For example, for computing, we can use AWS Lambda and AWS Fargate. For databases, we can use Amazon DynamoDB as well as Amazon Aurora Serverless. For integration from service to service, we can use Amazon EventBridge, and so on.

Another thing that is not less important is that each serverless service is rich in features that are difficult for us to replicate if we build it manually. Take for example when you want to monetize your API. You need to build a limit for different tiers, for example a maximum of 100 RPS with a limit of 5,000 requests per day for plan A, and 200 RPS with a limit of 10,000 requests per day for plan B. You can use the usage plan feature that you can use, directly within Amazon API Gateway.

The point is that by using the features of the serverless service, you can get your apps to market faster.

But even though we can deliver our apps faster to the market, we certainly need to think about security. Before the era of cloud computing, this was an option for me, to deliver apps quickly, or prioritize security which usually slows down apps delivery. Now these two are not an option, and we can do it at the same time.

The diagram below describes how we can maintain security of AWS Lambda to access Amazon CloudWatch and Amazon DynamoDB using the IAM Role.

Although your code runs fine, if you don't have an IAM role to access DynamoDB, it won't work. This is one example where we can run applications more secure.

Reason #3: Effective Scaling and Cost Efficient

Before the era of cloud computing, to run services and apps, I needed a fixed number of servers, and of course this approach is not effective and cost efficient. Cloud computing changed all of this by introducing auto scaling. There are two types of scaling, namely horizontal and vertical scaling scaling. Horizontal scaling basically means adding a number of compute engines to your fleet. Vertical scaling is more about adding power, such as CPU or RAM to the existing machines in your pool.

With serverless, the effort expended on handling requests is made easier by auto scaling. Here I take an example of auto scaling for a database using Amazon DynamoDB.

Let's be straight here, database workloads are difficult to predict and database scaling is a very challenging thing. Most of the startups I used to be involved with were in the Media and Entertainment industry. Read consumption for the database is higher between 8-10am, 1-3pm, and 6-9pm and handles approximately 1 million requests during those hours. Outside of those hours, the requests are significantly reduced. In other words, the capacity of a database to handle requests is not the same as a high traffic hours.

Then, I gave myself another thought on database. When I switched from self-managed MySQL to Amazon DynamoDB, I benefited from auto scaling. I can define the scaling configuration for read capacity, and different configuration for write capacity.

The image below is an illustration of how auto scaling from Amazon DynamoDB works.

The red line is provisioned capacity and the blue line indicates actual consumption based on item size. Here we can see that provisioned capacity adjusts the utilization rate to support application performance. In addition, this also reduces the cost that I need to pay because capacity is not fixed all the time, and adjusts to requests.

Reason #4: Flexible and Seamless Integrations

Serverless services on AWS are built with a specific purpose — Amazon DynamoDB as database, Amazon S3 as object storage, Amazon EventBridge as event bus. I'd like to think them somewhat like a Lego, building blocks. This provides flexibility for developers to build architectures using appropriate services.

Due to its modular characteristics, it makes it easier to build functionality for an application. We can be more flexible on building applications from very simple to sophisticated architecture.

"But if we are adopting various services, that also means we also need to work on more integration." That's right. The more services that we use, the more integration is needed.

Fortunately, AWS serverless services have seamless integration, which means we can integrate easily because the integration features are built-in. Take the example here on AWS Lambda, you can easily do the integration by configuring them in the dashboard.

We can also find the same thing in other serverless services, for example AWS Step Functions — one of my favorite services because it can easily create visual workflows for distributed applications. With AWS Step Functions, we can define workflow flows and seamless integration with AWS Lambda by simply defining the ARNs from AWS Lambda.

Below is an example of how we can implement distributed transactions using state machine with AWS Step Functions using Amazon ECS and AWS Fargate.

Reason #5: Faster Response with Events

AWS Lambda was first launched in late 2014 as an event-driven function. Triggering by event was something new and a bit unusual at the time, but as we move our workloads into cloud, the adoption of leveraging event increased.

Event itself is not something entirely new. Events can take any form, from changes in the state of a system — such as when data is entered into a database — to custom events — such as in the context of e-commerce, when a customer places an order.

By leveraging events, our applications can now respond to various kinds of events generated by the system, or the services we have. For example, below is a near real-time response by AWS Lambda to data changes that occur in Amazon DynamoDB. This feature is called DynamoDB Streams, where our system now has the ability to capture a time-ordered sequence of item-level modifications in any DynamoDB table.

Leveraging events is becoming more adopted in recent years, and formally defined in Event-driven architecture. Event-driven architecture is a paradigm in building architecture by using production and consumption to respond to an event.

Another example how serverless could help you is the implementation of choreography in microservices using Amazon EventBridge as a serverless event bus. By using Amazon EventBridge, we can easily define rules to route to which service we need to send data. This gives us the advantage of decoupling between services so they can run their processes independently and scale according to requests.

Below is an example on how you can build choreography for microservices in context of e-commerce application.

Wrapping up!

At this point, you already understand some of the reasons to consider whether serverless can be your development system approach. Serverless architecture can be used in almost all cases, starting from building a simple serverless API, IT automation, streaming data processing to implementing microservices.

Although this requires a change in perspective from developing with a traditional approach, but from my experience coupled with the development of features for AWS serverless services, the serverless approach will provide significant advantages in the long run.

And also, you can use presentations and ready-to-use code for this talk. All materials have been collected for you on the AWS Community ASEAN Content Repo.

Happy building!🤘🏻
—Donnie

Live Video Authorization with Private Channel on Amazon IVS

Donnie Prakoso — Mon, 09 May 2022 04:24:10 +0000

You can use the Private Channel feature to restrict video playback access using Amazon IVS. In this article, we will build a simple website, serverless API, and JWT signing to implement the Private Channel feature in Amazon IVS.

Github Repository for Demo(s)

Code for this article is available at github.com/donnieprakoso/demo-ivs

In the previous article, we learned how we can build interactive live videos using Amazon IVS. Running live streaming is now much easier and simpler, without worrying about scalability and low latency.

However, there are some use cases where you need to protect your content. In this situation, you need an authorization mechanism to allow only certain viewers — such as those already logged into your app — to access the live video and deny access to viewers who are not registered in your system.

Controlling Access with Private Channel on IVS

The Private Channel feature is one of the features within Amazon IVS that you can use to implement this use case. With the Private Channel feature, you have the flexibility to control access for playback by using a JWT (JSON Web Token) as part of the mechanism to authorize playback.

By using the Private Channel feature, you can restrict access to playback or broadcasting of live video streaming. For example, you can run live video streaming for a virtual conference that can only be accessed by logged-in users. With this, you can provide additional value for your existing customers.

What We Are Going To Build

In this article, we will build a simple application to demonstrate how we can implement a Private Channel with Amazon IVS. The diagram below describes the set of application architectures that we will build.

First, we will do provisioning for the IVS channel. The main difference between provisioning IVS channels in this tutorial and the previous one is that in this tutorial we will enable authorization mode.

Then, we will do video ingesting. In this tutorial, we will use FFMPEG to send videos to the IVS channel.
After that, we will run the webserver to host live video from the IVS channel on a web page. There are 2 buttons in this tutorial — as you can already see in the "Demo Preview" video above. The first button is to run IVS without a token — which will fail for sure. The second button is to run IVS with a token.

The next question is, how do we get the tokens? We will get this token from the AWS Lambda function which will perform the signing request and return the token in the form of a JWT. We can then use this token as one of the parameters in the IVS Playback URL, to gain access to video playback from IVS.

Requirements

This tutorial uses the following requirements and please make sure that your development environment satisfies all requirements described below:

Name	Versions	Where to get
AWS CDK	2.17.0	github.com/aws/aws-cdk
Python	3.8.13	python.org/downloads/release/python-3813/
ffmpeg	4.4	ffmpeg.org/download.html
Sample videos	N/A	peach.blender.org/download/
curl	N/A	https://curl.se/

Step 0: Clone Github repo

If you'd like to do this tutorial, you can clone this repo: github.com/donnieprakoso/demo-ivs. Otherwise, carry on reading this tutorial if you'd like to get big pictures of how everything works.

To clone the Git repo, you can run this command on your development environment:



git clone github.com/donnieprakoso/demo-ivs

We will use the 2-private-channel module inside the repo.

Step 1: Create ECDSA Private/Public Key Pair

Private channels with Amazon IVS work using the Private/Public Key Pair mechanism. We will upload the public key into IVS and we will use the private key to perform signing requests with the AWS Lambda function. The diagram below explains how this mechanism works.

To generate Public/Private key pair, we can use the openssl tool which is generally already installed on Linux. Here's the command to generate the private key:



# Working directory: 2-private-channel/keypair/
openssl ecparam -name secp384r1 -genkey -noout -out private.pem

And to generate the public key, we can use the private key with the following command:



# Working directory: 2-private-channel/keypair/
openssl ec -in private.pem -pubout -out public.pem

If you successfully ran the commands, we can see the files:



# Working directory: 2-private-channel/keypair/
dev> tree
.
├── private.pem
└── public.pem

Step 2: Deploy CDK App

In this step, you only need to run the cdk deploy command in the cdk folder. However, before we move on to deployment, let's first evaluate 2 important things: 1) an overview of the CDK app and 2) using the Lambda function for signing JWT.

CDK App Overview

The first thing defined in the CDK app is the AWS Secrets Manager. In this tutorial, we will use the Secrets Manager to store the private key that we generated in the previous step.


 python
private_key_secret = secrets_manager.Secret(self, "{}-secret-private-key".format(id),
                    secret_name="{}-secret-private-key".format(
                        en)
                    )

In addition, we will also upload the public key into Amazon IVS to be used as a playback key. The code snippet below defines how we can do this:


 python
public_key_pair_file = self.node.try_get_context(
    "publicKeyPair")

public_key_pair_string = ""
with open(public_key_pair_file, 'r') as f:
    public_key_pair_string = f.read()

playback_key = ivs.CfnPlaybackKeyPair(
    self, "{}-keypair".format(id), name="{}-keypair".format(id), public_key_material=public_key_pair_string)

Finally, we can define the IVS channel and notice the authorized parameter which has the value True. This defines an IVS channel that requires authorization to be able to perform playback.


 python
# File: 2-private-channel/cdk/app.py

        ivs_channel = ivs.CfnChannel(self, "{}-channel".format(id),
                                     authorized=True,
                                     latency_mode="LOW",
                                     name=id,
                                     recording_configuration_arn="",
                                     type="STANDARD"
                                     )

The rest of the CDK application is defining AWS Lambda functions and integration with Amazon API Gateway. For details, you can see the source code on Github.

AWS Lambda Function for Signing JWT

To get a token as playback authorization, we need to do a signing request with the public key. In this tutorial, we will implement an API with Amazon API Gateway and AWS Lambda function to perform signing requests with JWT.

The code below shows how we can use the AWS Secrets Manager to retrieve the private key.


 python
# File: 2-private-channel/lambda-functions/sign-requests/app.py

def get_private_key(secret_id):
    client = boto3.client('secretsmanager')
    response = client.get_secret_value(
        secretId=secret_id,
    )
    if not response:
        return None
    if "SecretString" not in response:
        return None
    else:
        return response['SecretString']

After getting the private key, we can now form the JWT using the ECDSA signature and the SHA-384 hash.


 python
# File: 2-private-channel/lambda-functions/sign-requests/app.py

def sign_request(private_key, channel_arn):
    payload = {
        "aws:channel-arn": channel_arn,
        "aws:access-control-allow-origin": "*",
        "exp": datetime.now() + timedelta(days=3)}
    encoded = jwt.encode(payload, private_key, algorithm="ES384")
    return encoded

The token is in base64 form and if you decoded the token, the JWT will have the following headers:



{
  "alg": "ES384",
  "typ": "JWT"
}

In addition, JWT will also have a payload that stores information channel-arn and also origin access * which means it can be accessed from any website. If you want to restrict only loading from your website, then you need to change it to your domain name. In addition, I also added exp — expiration time — as part of the JWT claim which indicates that this token should not be processed after the specified time has passed.




{
    "aws:channel-arn": "<channel_arn>",
    "aws:access-control-allow-origin": "*",
    "exp": <timestamp>
}

App Deployments

Now you understand the main components of this CDK app. The next step is to run cdk deploy to deploy the app.

In this app, it uses context to get the file path for your public key pair. Below is an example on how to run the cdk deploy command:



# Working directory: 2-private-channel/cdk/

cdk deploy --context publicKeyPair=<YOUR_PUBLIC_KEY_FOLDER>/public.pem
Synthesis time: 7.4s

demo2-private-channel: deploying...
[0%] start: Publishing c713b4752fe62d1d6d6c8a9cacd2e576039cd0c7697ea0bf0317b41aa6ec8f40:XXXXXXXXXXXXXX-us-east-1
[100%] success: Published c713b4752fe62d1d6d6c8a9cacd2e576039cd0c7697ea0bf0317b41aa6ec8f40:XXXXXXXXXXXXXX-us-east-1

 demo2-private-channel (no changes)

Deployment time: 8.32s

After the deployment is complete, you will get the following output. You will need the output below, so make sure you save it for later use.



# Working directory: 2-private-channel/cdk/

Outputs:
demo2-private-channel.demo2privatechannelapigatewayEndpoint7F53E47C = https://XXXXXXXXXXXXXX/prod/
demo2-private-channel.demo2privatechanneloutputapigateway = https://XXXXXXXXXXXXXX/prod/
demo2-private-channel.demo2privatechanneloutputchannelarn = arn:aws:ivs:us-east-1:XXXXXXXXXXXXXX:channel/XXXXXXXXXXXXXX
demo2-private-channel.demo2privatechanneloutputchannelingest = XXXXXXXXXXXXXX.global-contribute.live-video.net
demo2-private-channel.demo2privatechanneloutputchannelplayback = https://XXXXXXXXXXXXXX.us-east-1.playback.live-video.net/api/video/v1/us-east-1.XXXXXXXXXXXXXX.channel.XXXXXXXXXXXXXX.m3u8
demo2-private-channel.demo2privatechanneloutputsecretmanagerarn = arn:aws:secretsmanager:us-east-1:XXXXXXXXXXXXXX:secret:demo2-private-channel-secret-private-key-XXXXXXXXXXXXXX
demo2-private-channel.demo2privatechanneloutputstreamkey = XXXXXXXXXXXXXX

Total time: 15.72s

Step 3: Post Deployment — Deploy CDK

At this point, you have a ready-to-use architecture for Private Channel implementation with IVS. After deployment, you need to import the private key into the AWS Secrets Manager.

To import the private key, we first need to convert it into base64 form.



# Working directory: 2-private-channel/keypair/
base64 private.pem > encoded-private.pem

After that, we can import the private key into AWS Secrets Manager using --secret-id which we got in the output of CDK. Note that you also need to define the --region parameter to define the region where you are deploying your app — in this case, I used us-east-1.



# Working directory: 2-private-channel/keypair/
dev> aws secretsmanager put-secret-value --secret-id arn:aws:secretsmanager:us-east-1:XXXXXXXXXXXXXX:secret:demo2-private-channel-secret-private-key-XXXXXXXXXXXXXX --secret-string file: //encoded-private.pem --region us-east-1

{
    "ARN": "arn:aws:secretsmanager:us-east-1:XXXXXXXXXXXXXX:secret:demo2-private-channel-secret-private-key-XXXXXXXXXXXXXX",
    "Name": "demo2-private-channel-secret-private-key",
    "VersionId": "XXXXXXXXXXXXXX",
    "VersionStages": [
        "AWSCURRENT"
    ]
}

Step 4: Adjusting Variables for Web and Video Ingestion Script

Now we need to adjust the variables for the web and video ingestion script.

For the web, we need to change the location of the playback URL in the index.js file so that the Amazon IVS player knows the URL location for the video playback. You need to change this URL playback manually for 2 functions, namely playWithoutToken() and playWithToken()



// File: 2-private-channel/web/index.js
function playWithoutToken() {
  var PLAYBACK_URL = "<PLAYBACK_URL>";
  ...
}

function playWithToken() {
  var PLAYBACK_URL = "<PLAYBACK_URL>";
  ...
}

Apart from that, we also need to change the variables in the video ingestion script, which you can find at 2-private-channel/video-stream/stream-video.sh.

For the STREAM_URL variable, you will need the channel_ingest and stream_key variables which you found when you deployed the CDK app in step 2.



# File: 2-private-channel/video-stream/stream-video.sh
TEST_FILE="<YOUR VIDEO FILEPATH>"
STREAM_URL="rtmps://<YOUR_CHANNEL_INGEST>:443/app/<YOUR_STREAM_KEY>"

Step 5: Test!

Now we have everything we need for testing. Before that, we need to make sure that our API is running properly.

Using the apigateway variable from step 2 — which is the API URL of the Amazon API Gateway — we can test our API by doing a GET request with the following command:



curl -X GET https://<URL_API_ENDPOINT>/sign

Then we will get the following output.



{"token": "JWT_TOKEN_EXAMPLE"}

To finish testing, we need to run a webserver for serving HTML pages and ingesting video to the Amazon IVS channel.

To run the webserver in Python, you can use the following command:



# Working directory: 2-private-channel/web/
python -m http.server 8080

Now, you can access the website by visiting http://localhost:8080.

And to run the video ingestion script, you need to run the following command to make the script executable:



# Working directory: 2-private-channel/video-stream/
chmod +x stream-video.sh

To run the script, you can use the following command:



# Working directory: 2-private-channel/video-stream/
./stream-video.sh

And with this command, we are now ingesting the video to the IVS channel.

Now, when we open the web http://localhost:8080, there are two buttons available. When you click Play without Token, video playback will not be successful because this channel requires authorization with a playback key.

TODO: ADD Screenshot

And when you click Play with Token, you will see your video is running fine because the playback URL has been added to the token that we get from the API.

Congrats! 🥳

Now you can implement private channels with Amazon IVS.

Step 6: Cleanup

Don't forget to remove all resources once you're done with the tutorial. To do cleanup, do the following steps:



cd cd/
cdk destroy

Choose "Yes" and CDK will remove all resources created.

Wrapping Up!

I was quite surprised by how easy it is to implement access control restrictions for video playback using Amazon IVS. The token mechanism, using JWT signing and public/private keys, can be implemented in various authentication and authorization approaches.

And that's a wrap! Hope you enjoyed this tutorial and if you have any questions, please leave your comments.

Happy building!🤘🏻
— Donnie

Build Interactive Live Video Apps with Amazon IVS

Donnie Prakoso — Mon, 25 Apr 2022 02:52:20 +0000

Live streaming is becoming more popular than before as we explore new ways to connect during the restriction in a pandemic situation. In this article, I'll show you how to build your streaming channel using Amazon IVS, so you can host them on your website or applications. Adding a bit of fun, I'll also show you how to add an interactive element to your live streaming to engage your audiences even more.

Github Repository for Demo(s)

Code for this article is available at github.com/donnieprakoso/demo-ivs

Challenges on Running Live Streaming Platform

As we are constantly improving how we deliver information — especially in the current situation — live streaming has become one of the solutions. The idea of disseminating real-time information to (practically) unlimited audiences is one reason for running live streaming.

If we want to live stream into video platforms, i.e. YouTube, Vimeo, we can easily setup our OBS and start streaming. But, there's a case if you want to provide more seamless experience with your business, or to add features that aren't available in the video platform, you will need to host the live stream on your website.

The biggest challenge in this case is the complexities of managing infrastructure to deliver low-latency videos. Not to mention how hard it is to bring interactivity into the stream. I'm not talking about greeting every viewer who joined the live stream. I'm talking about getting their responses to any questions in real-time.

Wouldn't it be great if we can interact with our viewers? If only we can show some a popup dialog right in the stream and viewers can select the options without having to go to external apps or websites.

Or, in the case of e-commerce live streaming, it would be easier for viewers to shop with the presented items on the stream. They just need to click the item, added into their cart, and proceed with checkout.

Delivering the stream helps us to reach more audiences. Interactivity makes it live.

Here's Amazon IVS

So, imagine that now you can set up your live stream channel with just several clicks. Even better, you can programmatically create the live stream channels for any number of the live stream that you'd like to deliver. These channels can also be embedded into your website or applications. You can also add quizzes, polling, or anything that requires feedback from your viewers.

And the good news is, that those features are what you can expect from Amazon IVS. Amazon Interactive Video Service (IVS), is a managed live streaming solution to create interactive video experiences.

Roughly speaking, this is how it works:

First, you need to create a channel in Amazon IVS. You will get a stream key and URL as output from this step.

Second, you need to supply the stream key and URL to your favorite broadcasting channel. We have quite a few options — OBS, Streamlabs, XSplit — or you can even use an online live streaming studio such as Streamyard or Restream.

The third step is how you want to host the stream. You can use a website using SDK for Web or mobile applications — using SDK for iOS and Android — to host the stream.

The fourth step is optional if you want to add interactivity. You can send metadata payloads into the stream, and your website or mobile apps can receive and process it accordingly.

And that's it. You don't need to provision any infrastructures or expertise to run and scale the live streaming platform.

What We Are Going to Build

If you like what you read so far, the next part will help you to get started. We are going to build a live stream video channel, serve the live stream on a website and send real-time metadata into the stream.

There are 4 main steps in this tutorial.

Step 1: Provision the Amazon IVS channel.
Step 2: Broadcast a video. In this tutorial, I'm going to use the VLC app to ingest the video into Amazon IVS. You can also use OBS or other broadcasting tools, as long as it supports RTMP protocols.
Step 3: Run and host a website to serve the live video stream. A quick note in this tutorial, we are going to use a local website. In practice, you can host your website in other AWS services, such as AWS Amplify.
Step 4: Once everything is set up properly, the last step is to send metadata to demonstrate how we can add interactivity to the live stream.

Requirements

This tutorial uses the following requirements and please make sure that your development environment satisfies all requirements described below:

Name	Version	Where to get
AWS CDK	2.17.0	github.com/aws/aws-cdk
Python	3.8.13	python.org/downloads/release/python-3813/
ffmpeg	4.4	ffmpeg.org/download.html
Sample video	N/A	peach.blender.org/download/

Step 0: Clone Github repo

If you'd like to do this tutorial, you can clone this repo: github.com/donnieprakoso/demo-ivs. Otherwise, carry on reading this tutorial if you'd like to get big pictures of how everything works.

To clone the Git repo, you can run this command on your development environment:

git clone github.com/donnieprakoso/demo-ivs

Step 1: Provision IVS Channel with AWS CDK

In this step, we are going to provision the IVS channel with AWS CDK. This tutorial uses AWS Cloud Development Kit (CDK) to provide consistent deployment between this tutorial and yours.

Code Overview

The cdk/app.py code is the main file to provision your IVS channel. Below is the redacted version of the source code, to point out important parts you need to understand.

First, we need to configure the channel. The code below uses the L1 construct of IVS. This channel uses the basic configuration of the IVS channel, without auto-record enabled to Amazon S3 bucket. Noticed that the authorized parameter is set to false, which translates to disabling playback authorization.

ivs_channel = ivs.CfnChannel(self, "{}-channel".format(id),  
            authorized=False,  
            latency_mode="LOW",  
            name="demo-ivs-metadata",  
            recording_configuration_arn="",  
            type="STANDARD"  
        )

Once we have the channel configured, we need to configure the stream key so we can ingest video into the channel. To pair with the channel, we pass the IVS channel ARN into the channel_arn parameter.

ivs_stream_key = ivs.CfnStreamKey(self, "{}-streamkey".format(id),  
            channel_arn=ivs_channel.attr_arn,  
        )

Once we have the channel and stream key configured, the next step is to define and instantiates the IVS stack to produce the AWS CloudFormation template. Noticed that we are injecting the account using the environment variable CDK_DEFAULT_ACCOUNT and using the us-east-1 region.

app = App()  

env = _cdk.Environment(account=os.environ["CDK_DEFAULT_ACCOUNT"], region="us-east-1")  
IvsChannel(app, "demo1-ivs-metadata", env=env)  
app.synth()

Deployment

To deploy the IVS stack using CDK, follow these steps:

cd 1-stream-channel-timed-metadata/cdk  
pip install -r requirements.txt  
cdk deploy

This will deploy the IVS channel and you'll see the output below:

Outputs:  
demo1-ivs-metadata.demo1ivsmetadataoutputchannelarn = arn:aws:ivs:us-east-1:XXXXXXXXXX:channel/XXXXXXXXXX  
demo1-ivs-metadata.demo1ivsmetadataoutputchannelingest = XXXXXXXXXX.global-contribute.live-video.net  
demo1-ivs-metadata.demo1ivsmetadataoutputchannelplayback = https://XXXXXXXXXX.us-east-1.playback.live-video.net/api/video/v1/us-east-1.XXXXXXXXXX.channel.XXXXXXXXXX.m3u8  
demo1-ivs-metadata.demo1ivsmetadataoutputstreamkey = XXXXXXXXXX  
Stack ARN:  
arn:aws:cloudformation:us-east-1:XXXXXXXXXX:stack/demo1-ivs-metadata/XXXXXXXXXX  

✨  Total time: 15.72s

These are the information and credential that you can use to start hosting your live stream. Take note of these pieces of information, as we need them in the next steps.

Quick overview of these variables:

channelarn is the ARN of your IVS channel. This is required to programmatically interact with your channel.
channelingest is the variable to ingest video into your channel.
channelplayback is the variable you need to pass into your website so it could play the video
streamkey is the key to initiating a live stream. Remember to store your stream key in a safe place as anyone who has the stream key can ingest the video stream into the channel.

Step 2: Ingest video with FFMPEG

In this step, we are going to ingest video into the IVS channel. Once we have successfully ingested video into IVS channel, our website can start playing the video using the playback URL.

To ingest video into the IVS channel, you can use OBS, XSplit, or other broadcasting. In this tutorial, to simplify the demo, we are going to use FFMPEG to stream a video into the IVS channel.

Configure Ingest Endpoint

Before we run the FFMPEG command, we need to configure a few variables on video-stream/stream-video.sh file. You will need two variables: 1) channelingest and 2) streamkey, which you already obtained from deployment with CDK. Also, you need to have a video to ingest into your channel. A good starting point that you can use to test is "Big Buck Bunny" — which you can download from peach.blender.org/download/.

Once you have the required variables, you need to change the following lines on video-stream/stream-video.sh:

TEST_FILE="<YOUR_VIDEO_FILEPATH>.mp4"  
STREAM_URL="rtmps://<YOUR_CHANNEL_INGEST>:443/app/<YOUR_STREAM_KEY>"

Run ffmpeg

You also need to run the following command to make the script executable:

chmod +x stream-video.sh

To run the script, you can use the following command:

./stream-video.sh

And, it will start ingesting the video into your IVS channel.

Step 3: Play Video Stream on Website

At this stage, we already have the IVS channel created and configured with the stream key, and ingesting the video into IVS channel. The next part is to run the website to host the live video stream. With IVS, you can host your live stream on a website or mobile application.

In this tutorial, we use a website to host the live streaming by running the webserver on localhost. In practice, you will need to deploy the website so it can be accessible to your users. There are various ways to do this, two of my favorites approach are hosting it as a static website using AWS Amplify, or you can use Amazon ECS and AWS Fargate to host the dynamic website.

Configure Variables

The website needs to have the playback URL to identify where it should retrieve the video stream. To do this, get your playback URL variable from the CDK output and open web/index.js, and change the following line:

var PLAYBACK_URL = "https://XXXXXXXXXX.us-east-1.playback.live-video.net/api/video/v1/us-east-1.XXXXXXXXXX.channel.XXXXXXXXXX.m3u8";

Run Webserver

Once you have your app configured, the next step is to run the webserver to host the web page.

Again, there are various ways to do this, and as we are working on Python, the easiest way is to run the following command to run the webserver:

cd web/  
python -m http.server 8080

Then, open your browser and point the URL to http://localhost:8080. You will find a page below and watch the video ingested from IVS playing.

If you can see a similar page like this, Congrats! You just successfully integrate your IVS channel and your web page. If you don't, let me know in the comments below so I can help you out.

Step 4: Sending Timed Metadata

In previous steps, we configured and ingested live video, and host a website to playback the video. These steps are the starting point for you to run a live streaming platform.

Now, we'd like to take it to the next level by adding an interactive element to your live stream. The key to interactivity is the ability to send out a message (or payload) into the live video stream. With IVS, you can send this message as timed metadata. In short, timed metadata is data about other data equipped with timestamps that are sent simultaneously to every viewer in any channel.

The implementation of timed metadata isn't as complicated as its definition, and in this step, I'll explain how you can implement it.

Generator (App) Code Overview

The main application to send timed metadata into the IVS channel is located at timed-metadata/app.py. In this generator app, the payload we are going to send into the stream is a JSON data with the following structure:

{  
    "current_time":"2022-Apr-14-153617",  
    "question":"Question : 1",  
           "answers": [  
                "True",  
                "False"  
            ]  
}

This is just an example of a payload and all payloads are populated by the following code:

data = {  
            "current_time": datetime.datetime.utcnow().strftime('%Y-%b-%d-%H%M%S'),  
            "question": "Question : {}".format(i),  
            "answers": [  
                "True",  
                "False"  
            ]  
        }

Once we have the payload, we can send it to particular IVS channel by calling put_metadata API from IVS. Following lines show how to send metadata into IVS channel:

response = ivs.put_metadata(  
                channelArn=channel_arn,  
                metadata=json.dumps(data)  
            )

Once the request is successfully sent, the stream will receive the following metadata:

{  
  startTime: 3.458,  
  endTime: 3.458,  
  type: 'TextMetadataCue',  
  description: '',  
  text: '{"current_time": "2022-Apr-14-153617", "question": "Question : 8", "answers": ["True", "False"]}',  

}

With this data, you can build your interactive element by processing the metadata received on the text property.

Receiver (Web) Code Overview

In the previous step, we evaluated the generator app to send the timed metadata. We still need to receive the timed metadata on our web page. Let's review how we can instantiate the IVS player and how to receive the metadata.

The web page itself is built with 2 files: web/index.html and web/index.js. The main logic to consume the timed metadata is on web/index.js.

The first thing that you need to configure on this file is the PLAYBACK_URL. Simply replace the variable value with the value you obtained from CDK deployment outputs.

var PLAYBACK_URL =  
    "https://XXXXXXXXXX.us-east-1.playback.live-video.net/api/video/v1/us-east-1.XXXXXXXXXX.channel.XXXXXXXXXX.m3u8";

As for the IVS player, I'm using VideoJS integration, and the following lines show how to instantiate the player:

var player = videojs(  
    "amazon-ivs-videojs",  
    {  
      techOrder: ["AmazonIVS"],  
    },  
    () => {  
      player.src(PLAYBACK_URL);  
    }  
  );

To receive the timed metadata, we need to add event_listener.

player.getIVSPlayer().addEventListener(PlayerEventType.TEXT_METADATA_CUE, fn_metadata);

This way, we invoke the fn_metadata function whenever the player receives timed metadata from IVS.

Sending Timed Metadata

To demonstrate sending metadata into the stream, you will need your IVS channel ARN. Once you have the channel ARN, you can pass it as input to the timed-metadata/app.py application.

To send the metadata, you can run this command:

python app.py arn:aws:ivs:us-east-1:XXXXXXXXXX:channel/XXXXXXXXXX

To stop the application, you need to press Ctrl-C.

Once you successfully run the application, it will send the metadata into the IVS channel and it will be received by the web page. If you go to the web page, it will show the timed metadata on the right column.

What's Next?

You're probably wondering, what you can do with this metadata. This metadata is the raw form for adding interactivity. You can add pop-up dialog for running quizzes like the screenshot below using the metadata payload you configure from your application.

Step 5: Cleaning up

Don't forget to remove all resources once you're done with the tutorial. To do cleanup, do the following steps:

cd cdk/  
cdk destroy

Choose "Yes" and CDK will remove all resources created.

Wrapping Up

In conclusion, I found it's easy to run low-latency live streaming with Amazon IVS. Equipped with a timed metadata feature, it provides a simple way to add interactive elements with viewers and take your live streaming experience to the next level.

I hope this tutorial provides you with a good understanding of how Amazon IVS works. In the next post, I'll cover how to restrict playback access with authorization for a private channel.

Happy building!🤘🏻
— Donnie