Forem: Lumigo

Key Metrics for Effective ECS Monitoring

JSkog — Tue, 23 May 2023 22:37:25 +0000

What is Amazon Elastic Cloud Service (AWS ECS)?

ECS is a managed container orchestration service provided by Amazon Web Services (AWS). It allows developers to easily run and scale containerized applications in the cloud, while ECS manages the underlying infrastructure.

In this article, we will explore ECS monitoring and discuss the key metrics to monitor.

Monitoring Amazon ECS

There are several ways to monitor Amazon ECS. One of the primary tools is Amazon CloudWatch, a monitoring service provided by AWS. CloudWatch provides metrics and logs for ECS, including CPU and memory usage, the number of tasks and services running, and the deployment status of containers.

Setting up CloudWatch alarms is crucial to automatically trigger actions in response to metric changes. Alarms can help maintain the smooth operation of the ECS cluster and ensure prompt resolution of any potential issues. Additionally, the ECS Management Console provides a web-based user interface for monitoring and managing ECS clusters and tasks. It allows developers to view the status of the cluster and tasks, as well as details such as CPU and memory usage.

Monitoring ECS on EC2

When running ECS on Amazon Elastic Compute Cloud (EC2), it is essential to monitor key metrics to ensure smooth and efficient container operation. Some of the important metrics to monitor include CPU and memory usage of EC2 instances and individual containers, task and container counts, cluster and service health, and network performance.

CPU and memory usage monitoring helps identify performance bottlenecks and determine if scaling up EC2 instances or adjusting resource allocation for containers is necessary. Monitoring the number of tasks and containers ensures that all tasks run as expected and capacity is not exceeded. Monitoring cluster and service health provides an overview of task availability and reports any errors or issues. Network performance monitoring helps ensure adequate network resources for container communication.

Monitoring ECS on Fargate

Monitoring ECS on Fargate, AWS's serverless compute engine for containers, requires specific tools designed for container monitoring. Traditional server monitoring tools are not applicable as Fargate abstracts the underlying EC2 instances. Amazon CloudWatch Container Insights is one such tool that provides detailed metrics and logs for Fargate tasks and services.

CloudWatch Container Insights allows real-time monitoring of Fargate environment performance, including CPU and memory usage, network performance, and running task/container counts. It enables setting alarms and reacting to environment changes using CloudWatch.

Key Metrics to Monitor in AWS ECS

To effectively monitor ECS clusters, the following metrics should be considered:

CPUReservation: Represents the amount of CPU capacity reserved for a task or service, ensuring sufficient resources and avoiding contention.
CPUUtilization: Indicates the percentage of CPU capacity used by a task or service, aiding identification of overloading or resource optimization opportunities.
MemoryUtilization: Shows the percentage of memory used by a task or service, helping identify excessive memory usage and potential shortages.
Storage metrics: These include disk usage, read/write operations, latency, and throughput, essential for detecting storage-related issues.
I/O metrics: Monitor disk read/write operations and network bytes sent/received to troubleshoot performance problems related to input/output operations.
Network metrics: Monitor packets sent/received, network errors, and retransmits to identify and troubleshoot network-related issues.

AWS ECS Monitoring with Lumigo

For all the benefits that AWS ECS brings to developing and running containers, these distributed applications still need observability to ensure they run at the highest performance, with the greatest reliability to deliver seamless customer experiences.

Lumigo is a cloud native observability platform purpose-built for microservice applications that provides deep visibility into applications and infrastructure, enabling users to easily monitor and troubleshoot their applications running on Amazon ECS.

Trace end-to-end applications running on Amazon ECS, AWS Lambda and consuming AWS services and 3rd party APIs
Easily monitor and debug ECS clusters and underlying services and tasks in real-time
Setup automatic alerts to notify you in Slack, Pagerduty and other workflow tools

Learn more about monitoring ECS in the full article on Lumigo.io

Comparing Amazon ECS launch types: EC2 vs. Fargate

DeveloperSteve — Wed, 18 Jan 2023 02:58:33 +0000

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that enables users to easily run, manage and scale containers on AWS. With ECS, you can deploy containers either on a cluster of Amazon EC2 instances or on AWS Fargate, a serverless computing engine for containers. In this article, we’ll take a closer look at what ECS is and how it works, including the different components and launch types available.

In this article, we’ll take a closer look at what ECS is and how it works, including the different components and launch types available. We'll also briefly compare the features of EC2 and Fargate, and explore the use cases for each launch type. By the end of this article, you'll have a better understanding of how to use ECS and be able to make an informed decision on which launch type is best for your needs.

But first, let's define what ECS is. ECS is a highly scalable and powerful tool for orchestrating containers that are run on Docker. It allows you to deploy containers from your local environment via Docker and provision access to those containers from anywhere, resulting in zero downtime and no complicated configurations. With ECS, you can scale numerous containers with just a single click.

ECS provides the underlying infrastructure to create task definitions for your tasks and services; however, you still have to select a launch type that will handle running, stopping, and deploying your tasks and services.

Want to find out more?

Follow this link to read the full article and learn how to use ECS to manage and scale your containers on AWS, and take the first step towards streamlining your container orchestration process.

The Hidden Costs of Serverless Observability

DeveloperSteve — Thu, 08 Dec 2022 23:05:57 +0000

Serverless architectures are becoming increasingly popular, which has led to an increased need for solutions to the challenges of microservice observability. Observability solutions enable fast and easy debugging of applications, as well as driving optimization and cost efficiency. However, the complexity of serverless systems presents challenges to observability, and the wrong tool set can lead to unexpected costs.

In a serverless environment, not only are developers abstracted from the underlying infrastructure that runs their code, but the distributed, event-driven nature of serverless applications makes traditional approaches to monitoring and debugging ineffective. Serverless functions are often chained through asynchronous event sources such as AWS SNS, SQS, Kinesis, or DynamoDB, making tracing extremely difficult.

The distributed nature of serverless architectures makes monitoring harder, as there isn't one central, monolithic application server anymore. Instead, developers are faced with a distributed architecture and an abundance of competing tools to conduct serverless monitoring.

Serverless observability tools are purpose built to support serverless architectures. They allow developers to reap all the benefits of serverless technology without sacrificing insights into the system, allowing them to retain opportunities to catch errors and optimize for cost and responsiveness.

Building an observability solution tailored to a serverless system requires data, and a lot of it. Logs, metrics, and traces are required to ensure the system can answer as many questions as possible about how it's working, whether you're repairing or optimizing. Each source contributes a different perspective on how your application is performing and, when analyzed together, give a complete picture of its health.

There are many different serverless observability tools available, each with its own unique features and capabilities. Choosing the right one for your needs can be a daunting task. Some popular options include IOpipe, Dashbird, and Epsagon.

Follow this link to read more

Tracing and Observing AWS ECS

DeveloperSteve — Thu, 08 Sep 2022 06:02:02 +0000

It’s no secret that application containerization has revolutionized the digital world as we know it by providing a transient gateway into elastic infrastructure that can scale and grow as needed. Where traditional virtualization was all about creating a single homogenous entity, containers are self-contained units of software, able to run in just about any environment, making them extremely portable.

Like with all excellent adventures, there are a few things we need to prepare before we begin our journey. Have an AWS account set up and at least one ECS cluster deployed—extra points for also having a few Lambdas or other traceable services.

The other thing you will need is a Lumigo account that is connected to your AWS. It’s easy to do, free, and only takes a few minutes.

To get started, head to the Lumigo signup page and create your account by clicking the “Continue with Google” button, or by entering an email and password. The next step is then to connect your AWS account by clicking the “Connect to AWS” button.

Once the AWS connection process is complete, Lumigo will automatically start scanning for services and applications to trace from the connected account. This process will identify resources of interest and will start tracing them in your Lumigo instance.

Give it a few moments to complete the initial scans, then explore your newly created Lumigo account to see which applications have been observed and discovered. Clicking into the ECS tab from your Lumigo instance will display any clusters that Lumigo has detected from your connected AWS account.

It’s really important to note here that those cluster metrics are only visible if you have CloudWatch container insights enabled on your cluster.

Clicking into a monitored ECS application will display an overview for a cluster that includes metrics such as CPU and Memory utilization alongside running task metrics. These metrics combined give a useful insight into where resources are being consumed and the timing associated with their usage.

This is where things get REALLY interesting.

By adding the Lumigo tracers into our application you can gain even more insight into the inner workings of your ECS container applications.

To get started all you need to do is add a library, a few lines of code, and some environmental variables. This will slightly vary depending on what your language and codebase look like, but first, let’s check out some libraries.

Lumigo has some open-source libraries for getting trace information from your application. It currently supports [Python}(https://github.com/lumigo-io/opentelemetry-js-distro) and NodeJS languages.

Install the library with your nominated package manager and reference the library from within your application. Again, this may vary slightly depending on how your application is structured.

Finally, we need to set 2 environmental variables on the container as part of its deployment to ECS. These can be set within the code as part of the library, but it is highly recommended to set these as environment variables.

Now we will be able to see a lot more information from that ECS cluster application, right down to the application level and beyond. Clicking through from the services button on the cluster overview page will whisk you away to the Explore page with a resource filter set.

This will then let you see additional information from a trace and view the associated invocation in greater detail.

From the invocation view page you can see any associated connected services, which is really handy to being able to see the full end to end trace view from your ECS deployments.

To read more check out Observing ECS Containers

Observing AWS ECS deployments

DeveloperSteve — Tue, 06 Sep 2022 03:39:32 +0000

To look at it another way, and to build on a popular quote with ancient origins, “With great scale comes the need for great observability”. Tracing and monitoring can help surface these issues while observing the resources being utilized by your containers.

Here’s how to get started on observing and monitoring your ECS clusters with Lumigo.

The other thing you will need is a Lumigo account that is connected to your AWS. It’s easy to do, free, and only takes a few minutes.

This is where things get REALLY interesting.

By adding the Lumigo tracers into our application you can gain even more insight into the inner workings of your ECS container applications.

Lumigo has some open-source libraries for getting trace information from your application. It currently supports Python and NodeJS languages.

If you are using another language for your application, you can still connect your trace information, but you may need a few additional steps and should use one of the OpenTelemetry collector libraries.

Install the library with your nominated package manager and reference the library from within your application. Again, this may vary slightly depending on how your application is structured.

OTEL_SERVICE_NAME

This sets the application name on the service application traces and should reflect something specific to the application being traced within this container.

LUMIGO_TRACER_TOKEN

This helps identify your traces to your Lumigo instance. It can be found by visiting Settings and then Tracing from your Lumigo dashboard.

Check out tracing and observing AWS ECS to read more

Getting set up to use ABAC for Lambda Functions

DeveloperSteve — Wed, 17 Aug 2022 04:28:22 +0000

First, we need to set up a policy that has the right permissions to create Lambda functions using team-specific tagging. For this we are going to use team Alpha and team Beta.

{
   "Version":"2012-10-17",
   "Statement":{
      "Effect":"Allow",
      "Action":[
         "lambda:CreateFunction",
         "lambda:TagResource"
      ],
      "Resource":"arn:aws:lambda:*:*:function:*",
      "Condition":{
         "StringEquals":{
            "aws:RequestTag/Team":[
               "Alpha",
               "Beta"
            ]
         },
         "ForAllValues:StringEquals":{
            "aws:TagKeys":"Team"
         }
      }
   }
}

Next we create another policy to only allow Lambda API actions providing the resource tag is set to team Alpha.

{
   "Version":"2012-10-17",
   "Statement":{
      "Effect":"Allow",
      "Action":[
         "lambda:InvokeFunction"
      ],
      "Resource":"arn:aws:lambda:*:*:function:*",
      "Condition":{
         "StringEquals":{
            "aws:ResourceTag/Team":"Alpha"
         }
      }
   }
}

Then we need one more policy that gives permissions to iam:PassRole and iam:ListRoles.

It’s important to note that iam:PassRole does not support tag-based authentication, so the role will need permissions to ListRoles in order to create a function with existing execution using the AWS Console. It’s also important to note here that the Resource needs to be updated with a role-specific ARN from the setup process.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"VisualEditor0",
         "Effect":"Allow",
         "Action":[
            "iam:ListRoles",
            "iam:PassRole"
         ],
         "Resource":[
            "Role specific ARN"
         ]
      }
   ]
}

Now, we need to create an IAM role, attach all 3 of these policies, and then tag the role with “Team: Alpha”. Finally, we can create a user and assign them the newly created role.

To read more on using ABAC for Lambda, Check out Building for Scale and Traceability Using ABAC for Lambda Functions

Optimizing NodeJS Library Use in Lambda Functions

DeveloperSteve — Thu, 11 Aug 2022 04:27:37 +0000

As Dev/Ops, we build our applications with our end users in mind. Ensuring that they have a speedy and responsive application experience is integral to the application’s success. It’s equally important to make sure that the server side performance is up to the task at hand and not consuming more resources than it needs.

That’s why it’s critically important that our Lambda functions only call on the libraries and dependencies that they need to run. This is especially so when it comes to the AWS-SDK, which contains a lot of functionality that your Lambda function may not need but will load into memory.

Let’s take a look at an example using a basic NodeJS function that connects to a DynamoDB table called lambda_test.

This is the code that we want to use for our test. It’s important to note that the whole AWS-SDK library is being called on but we are only using DynamoDB.

const AWS = require('aws-sdk'); 
exports.handler = async (event, context) => {
    const documentClient = new AWS.DynamoDB.DocumentClient();
    let responseBody = "";
    let statusCode = 0;
    const params = {
        TableName: "lambda_test"
    };
    try {
        const data = await documentClient.scan(params).promise();
        responseBody = JSON.stringify(data.Items);
        statusCode = 200;
    } catch (err) {
        responseBody = `Unable to get data: ${err}`;
        statusCode = 403;
    }
    const response = {
        statusCode: statusCode,
        headers: {
            "Content-Type": "application/json"
        },
        body: responseBody
    };
    return response;
};

Checking on test invocations in the Lumigo dashboard, we can see that it does run, although it has some fairly high metrics.

Ideally, we only want to call in the relevant DynamoDB libraries because we only need that to run as part of this script. So instead of using const AWS = require(‘aws-sdk’) in our code to call on the whole SDK, we can just call the DynamoDB part of the library and save some time and resources.

By changing only two lines in the code snippet, we can improve the performance. Our new test code will look like this:

const AWS = require('aws-sdk/clients/dynamodb')

exports.handler = async (event, context) => {
const documentClient = new AWS;
let responseBody = "";
let statusCode = 0;
const params = {
TableName: "lambda_test"
};
try {
const data = await documentClient.scan(params).promise();
responseBody = JSON.stringify(data.Items);
statusCode = 200;
} catch (err) {
responseBody = `Unable to get data: ${err}`;
statusCode = 403;
}
const response = {
statusCode: statusCode,
headers: {
"Content-Type": "application/json"
},
body: responseBody
};
return response;
};

And now, if we take that for a spin to test out our changes we can see that even the cold start improved.

Check out some other ways to see how you can optimize your NodeJS lambda functions