Forem: bikash119

Deploying Docling Application on ECS with Application Load Balancer

bikash119 — Mon, 08 Sep 2025 11:02:41 +0000

This is Part 3 (Final) of our 3-part series on docling deployment to complete AWS ECS infrastructure. In Part 1, we set up the foundational networking and IAM, and in Part 2, we created the ECS cluster with Auto Scaling Groups and Launch Templates. Now we'll deploy our actual application and make it accessible through an Application Load Balancer.

Welcome to the final part of our journey to deploy docling to AWS ECS infrastructure! In this comprehensive guide, we'll deploy the docling application (a GPU-accelerated document processing service) on our ECS infrastructure and expose it to the internet using an Application Load Balancer (ALB). We'll also explore the core concepts of load balancing through an intuitive restaurant analogy.

Understanding Application Load Balancer Components

Before diving into the implementation, let's understand how Application Load Balancers work using a restaurant analogy:

Core Concepts 🍽️

Load Balancer

Think of the Load Balancer as the restaurant owner whose primary responsibility is to serve customers efficiently. The restaurant owner ensures that customers have a great dining experience and that the restaurant runs smoothly.

Listener

The Listener is like a host/hostess hired by the restaurant owner with specific instructions on which customer requests should be served where. For example:

If a customer requests ice cream → direct them to the ice cream corner
If a customer wants drinks → direct them to the bar area
If a family arrives → guide them to the family seating section

Target Group

Target Groups are like groups of waiters for each section:

Waiters at the ice cream corner
Bartenders at the bar
Waiters in the family seating area
Each specialized group of staff forms a "Target Group"

Register Targets

Registering Targets is the process where waiters register themselves with their respective target groups, letting the system know they're available to serve customers in their designated area.

This analogy helps us understand how ALB distributes incoming traffic (customers) to the right backend services (waiters) based on configured rules (host instructions).

What We're Building

In this final part, we'll:

Create and register ECS Task Definitions for the Docling application
Set up ECS Services to manage our containers
Configure an Application Load Balancer for external access
Establish proper networking and security group rules
Test our complete GPU-enabled document processing service

Prerequisites

Make sure you've completed:

Part 1: Foundation - Networking & IAM - VPC, subnets, and IAM roles
Part 2: ECS EC2 with Auto Scaling - ECS cluster, Launch Templates, and Auto Scaling Groups

You should have the following from previous parts:

$VPC_ID - VPC ID from Part 1
$PUBLIC_SUBNET and $PRIVATE_SUBNET - Subnet IDs from Part 1
$ECS_SG_ID - Security Group ID from Part 2
docling-ecs-cluster - ECS cluster from Part 2
ECS_Asg - Auto Scaling Group from Part 2

Step 1: Create ECS Task Definition

The Task Definition is like a blueprint that tells ECS how to run our Docling container. Create a file called docling-task-definition.json:

{
  "family": "docling-nvidia",
  "networkMode": "host",
  "requiresCompatibilities": ["EC2"],
  "executionRoleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/ecs_task_exec_role",
  "taskRoleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/ecs_task_role",
  "containerDefinitions": [
    {
      "name": "docling-serve",
      "image": "ghcr.io/docling-project/docling-serve-cu126:main",
      "essential": true,
      "portMappings": [
        {
          "containerPort": 5001,
          "hostPort": 5001,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "DOCLING_SERVE_ENABLE_UI",
          "value": "true"
        }
      ],
      "resourceRequirements": [
        {
          "value": "1",
          "type": "GPU"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/docling-serve-nvidia",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs",
          "awslogs-create-group": "true"
        }
      },
      "linuxParameters": {
        "capabilities": {
          "add": ["SYS_ADMIN"]
        }
      }
    }
  ],
  "cpu": "2048",
  "memory": "8192"
}

⚠️ Important: Replace <YOUR_ACCOUNT_ID> with your actual AWS account ID and ensure the role names match those created in Part 1.

Key Configuration Highlights

GPU Support: resourceRequirements specifies 1 GPU allocation
Network Mode: host mode allows direct access to the EC2 instance's network
CloudWatch Logs: Automatic log group creation for monitoring
Resource Allocation: 2 vCPUs and 8GB RAM for GPU-intensive processing

Register the Task Definition

TASK_ARN=$(aws ecs register-task-definition \
    --cli-input-json file://docling-task-definition.json \
    --query "taskDefinition.taskDefinitionArn" \
    --output text)

echo "Task Definition ARN: $TASK_ARN"

Add Tags to Task Definition

aws ecs tag-resource --resource-arn $TASK_ARN --tags key=Name,value=ECS_Docling_Task
aws ecs tag-resource --resource-arn $TASK_ARN --tags file://cluster-tags.json

Verify Tags

aws ecs list-tags-for-resource --resource-arn $TASK_ARN

Step 2: Create ECS Service

ECS Services ensure that your desired number of tasks are running and healthy. Create docling-service-definition.json:

{
  "serviceName": "docling-serve",
  "cluster": "docling-ecs-cluster",
  "taskDefinition": "docling-nvidia:2",
  "desiredCount": 0,
  "launchType": "EC2"
}

📝 Note: We start with desiredCount: 0 to prevent tasks from starting before we have EC2 instances available.

Create the Service

SERVICE_ARN=$(aws ecs create-service \
    --cli-input-json file://docling-service-definition.json \
    --query "service.serviceArn" \
    --output text)

echo "Service ARN: $SERVICE_ARN"

Add Tags to Service

aws ecs tag-resource --resource-arn $SERVICE_ARN --tags file://cluster-tags.json
aws ecs tag-resource --resource-arn $SERVICE_ARN --tags key=Name,value=ECS_Docling_Service

Verify Service Tags

aws ecs list-tags-for-resource --resource-arn $SERVICE_ARN

Step 3: Test Basic Service Deployment

Before setting up the load balancer, let's verify our service works correctly.

Launch EC2 Instance

Scale up the Auto Scaling Group to launch an instance (this uses the configuration from Part 2):

aws autoscaling update-auto-scaling-group \
    --auto-scaling-group-name ECS_Asg \
    --min-size 1 \
    --max-size 1 \
    --desired-capacity 1

Start the Service

Update the service to run one task:

aws ecs update-service \
    --cluster docling-ecs-cluster \
    --service docling-serve \
    --desired-count 1

Verify Container is Running

SSH into the instance and check the container status:

# Get the instance IP (from Part 2's command)
aws ec2 describe-instances \
    --filters "Name=instance-state-name,Values=running" "Name=tag-key,Values=Purpose" \
    --query "Reservations[*].Instances[*].[InstanceId,InstanceType,PrivateIpAddress,PublicIpAddress]" \
    --output table

# SSH into the instance
ssh -i ECSInstanceKey.pem ec2-user@<INSTANCE_PUBLIC_IP>

# Check running containers
sudo docker ps

# View container logs
sudo docker logs <container_id> -f

You should see the Docling application starting up and utilizing the GPU.

Step 4: Set Up Application Load Balancer

Now let's create the ALB infrastructure to make our application accessible from the internet.

Create ALB Security Group

ALB_SG_ID=$(aws ec2 create-security-group \
    --tag-specifications 'ResourceType=security-group,Tags=[{Key=Name,Value=ALB_SG}]' \
    --vpc-id $VPC_ID \
    --group-name ALB_SG \
    --description "SG for ALB" \
    --query "GroupId" \
    --output text)

echo "ALB Security Group ID: $ALB_SG_ID"

Allow Internet Traffic to ALB

aws ec2 authorize-security-group-ingress \
    --group-id $ALB_SG_ID \
    --protocol tcp \
    --port 5001 \
    --cidr 0.0.0.0/0

Create the Load Balancer

Remember our restaurant analogy - this creates the "restaurant owner":

DOCLING_ALB=$(aws elbv2 create-load-balancer \
    --name docling-alb \
    --subnets $PRIVATE_SUBNET $PUBLIC_SUBNET \
    --security-groups $ALB_SG_ID \
    --scheme internet-facing \
    --type application \
    --tags Key=Name,Value=DoclingALB \
    --query "LoadBalancers[].LoadBalancerArn" \
    --output text)

echo "Load Balancer ARN: $DOCLING_ALB"

📋 Note: The load balancer spans both private and public subnets from Part 1 for high availability.

Add Tags to Load Balancer

aws elbv2 add-tags --resource-arns $DOCLING_ALB --tags file://alb-tags.json

Step 5: Create Target Group

In our restaurant analogy, this creates the "group of waiters" that will serve our customers:

DOCLING_TARGET_GRP=$(aws elbv2 create-target-group \
    --name docling-targets \
    --protocol HTTP \
    --port 5001 \
    --vpc-id $VPC_ID \
    --health-check-path /docs \
    --target-type ip \
    --tags Key=Name,Value=doclingTargetGroup \
    --query "TargetGroups[].TargetGroupArn" \
    --output text)

echo "Target Group ARN: $DOCLING_TARGET_GRP"

Key Target Group Configuration

target-type ip: Uses IP addresses rather than instance IDs
health-check-path /docs: Docling's health check endpoint
port 5001: The port our application listens on

Add Tags to Target Group

aws elbv2 add-tags --resource-arns $DOCLING_TARGET_GRP --tags file://alb-tags.json

Update Service to use Target Group

Now we need to update our ECS service to integrate with the Application Load Balancer. Create a file called service-alb.json:

[
  {
    "targetGroupArn": "<TARGET_GROUP_ARN>",
    "containerName": "docling-serve",
    "containerPort": 5001
  }
]

⚠️ Important: Replace <TARGET_GROUP_ARN> with the actual Target Group ARN we created above (stored in $DOCLING_TARGET_GRP).

Update the service to use the load balancer:

aws ecs update-service \
    --cluster docling-ecs-cluster \
    --service docling-serve \
    --load-balancers file://service-alb.json

echo "Service updated to use ALB integration"

This step is crucial as it tells ECS to automatically register and deregister tasks with the target group as they start and stop, eliminating the need for manual target registration.

Step 6: Create Listener

The listener is our "host/hostess" that directs traffic to the right place:

DOCLING_LISTENER=$(aws elbv2 create-listener \
    --load-balancer-arn $DOCLING_ALB \
    --protocol HTTP \
    --port 5001 \
    --default-actions Type=forward,TargetGroupArn=$DOCLING_TARGET_GRP \
    --tags Key=Name,Value=DoclingListener \
    --query "Listeners[].ListenerArn" \
    --output text)

echo "Listener ARN: $DOCLING_LISTENER"

Add Tags to Listener

aws elbv2 add-tags --resource-arns $DOCLING_LISTENER --tags file://alb-tags.json

Step 7: Configure Security Rules

Allow ALB to Reach EC2 Instances

Update the EC2 security group (from Part 2) to allow traffic from the ALB:

aws ec2 authorize-security-group-ingress \
    --group-id $ECS_SG_ID \
    --protocol tcp \
    --port 5001 \
    --source-group $ALB_SG_ID

This creates a security rule allowing our "restaurant owner" (ALB) to communicate with our "kitchen" (EC2 instances).

Step 8: Final Testing and Verification

Check Target Health

Verify that our target is healthy:

aws elbv2 describe-target-health --target-group-arn $DOCLING_TARGET_GRP

You should see the target status as healthy once the health checks pass.

Get Load Balancer DNS Name

ALB_DNS=$(aws elbv2 describe-load-balancers \
    --load-balancer-arns $DOCLING_ALB \
    --query "LoadBalancers[].DNSName" \
    --output text)

echo "Access your application at: http://$ALB_DNS:5001"

Test the Application

Open your browser and navigate to http://<ALB_DNS>:5001/ui. You should see the Docling web interface!

You can also test the API endpoint:

curl http://$ALB_DNS:5001/docs

Monitor Application Logs

Check the application logs through CloudWatch or directly on the instance:

# On the EC2 instance
sudo docker logs <container_id> -f

# Or check CloudWatch Logs
aws logs describe-log-groups --log-group-name-prefix "/ecs/docling-serve-nvidia"

Troubleshooting Common Issues

Service Won't Start

Check if EC2 instances are running and registered with ECS cluster
Verify task definition has correct IAM roles from Part 1
Check CloudWatch logs for container startup errors

Can't Access Application

Verify security group rules allow traffic
Check target group health status
Ensure load balancer is in correct subnets from Part 1

GPU Not Available

Confirm EC2 instance type supports GPU (g4dn.xlarge)
Check ECS agent configuration includes GPU support
Verify NVIDIA drivers are installed (should be automatic with ECS GPU AMI)

Conclusion

Congratulations! 🎉 You've successfully deployed docling to AWS ECS infrastructure with GPU support. Throughout this 3-part series, we've covered:

What We've Accomplished

Part 1 Foundation: VPC networking, security groups, and IAM roles
Part 2 Infrastructure: ECS cluster, Launch Templates, and Auto Scaling Groups

Part 3 Application: Task definitions, services, and Application Load Balancer

The Complete Architecture

Your infrastructure now includes:

Scalable GPU Computing: Auto Scaling Groups with GPU-enabled instances
Container Orchestration: ECS managing Docling containers with resource requirements
High Availability: Multi-subnet deployment with health checks
Internet Accessibility: Application Load Balancer with proper security
Monitoring: CloudWatch integration for logs and metrics
Security: Layered security groups and IAM roles

Real-World Applications

This architecture is perfect for:

AI/ML Workloads: GPU-accelerated machine learning inference
Document Processing: Like our Docling example for document conversion
Video Processing: GPU-accelerated video transcoding and analysis
Scientific Computing: High-performance computing workloads

Series Navigation

Part 1: Foundation - Networking & IAM ✅ - VPC setup, subnets, security groups, and IAM roles
Part 2: ECS EC2 with Auto Scaling ✅ - Launch templates, Auto Scaling Groups, and ECS cluster setup
Part 3: Application Deployment (Current) ✅ - Task definitions, services, and load balancers

What's Next?

Consider exploring these advanced topics in future posts:

High Availability & Scaling: Multi-AZ deployments, auto scaling policies, and health monitoring
Monitoring & Observability: CloudWatch Container Insights, custom metrics, and distributed tracing
Cost Optimization: Spot instances, reserved capacity, and right-sizing strategies

Thank you for following along with this comprehensive AWS ECS series. You should now be able to setup the n8n workflow which uses docling deployed to a GPU enabled AWS ECS infrastructure 🚀

Deploying GPU-Enabled ECS EC2 Instances with Auto Scaling Groups and Launch Templates

bikash119 — Mon, 08 Sep 2025 04:34:54 +0000

This is Part 2 of a 3-part series on Deploy Docling to AWS ECS infrastructure. In Part 1, we covered the foundational networking and IAM setup required for this deployment.

Setting up Amazon ECS (Elastic Container Service) with EC2 instances can be complex, especially when you need GPU support for compute-intensive workloads. In this comprehensive guide, we'll walk through creating a robust, scalable ECS infrastructure using Auto Scaling Groups (ASG) and Launch Templates, specifically configured for GPU workloads.

Why Use Auto Scaling Groups with ECS?

Auto Scaling Groups provide several key benefits for ECS deployments:

Automatic scaling based on demand and health checks
High availability across multiple availability zones
Cost optimization by scaling down during low usage periods
Consistent tagging and configuration through Launch Templates
Integration with ECS Capacity Providers for seamless container orchestration

Prerequisites

Before we begin, ensure you have:

AWS CLI configured with appropriate permissions
A VPC with public subnets already created - If you haven't set this up yet, please refer to Part 1 of this series where we cover the complete VPC setup including subnets, internet gateways, and route tables
IAM roles properly configured - The ec2_instance_role-profile referenced in our Launch Template was created in Part 1. If you skipped Part 1, you'll need to set up the necessary IAM roles and instance profiles
Basic understanding of AWS ECS, EC2, and Auto Scaling concepts

💡 Note: This guide assumes you have the VPC ID ($VPC_ID) and public subnet ID ($PUBLIC_SUBNET) from Part 1. If you need to retrieve these values, refer to the VPC setup section in Part 1.

Step 1: Create Security Infrastructure

Generate SSH Key Pair

First, let's create a key pair for secure access to our EC2 instances:

aws ec2 create-key-pair \
    --key-name ECS_Instance_Key \
    --tag-specifications 'ResourceType=key-pair,Tags=[{Key=Name,Value=ECS_Instance_Key}]' \
    --query 'KeyMaterial' \
    --output text > ECSInstanceKey.pem

# Secure the key file
chmod 400 ECSInstanceKey.pem

Create Security Group

Next, we'll create a security group to control network access. This security group will be associated with the VPC we created in Part 1:

ECS_SG_ID=$(aws ec2 create-security-group \
    --tag-specifications 'ResourceType=security-group,Tags=[{Key=Name,Value=ECS_Instance_SG}]' \
    --vpc-id $VPC_ID \
    --group-name ECS_Instance_SG \
    --description "SG for ECS Instance" \
    --query "GroupId" \
    --output text)

# Add tags to the security group
aws ec2 create-tags --resources $ECS_SG_ID --cli-input-json file://tags.json

📝 Reminder: The $VPC_ID variable should contain the VPC ID from Part 1. If you need to find your VPC ID, you can use the command provided in Part 1 or run: aws ec2 describe-vpcs --filters "Name=tag:Name,Values=your-vpc-name"

Configure Security Group Rules

⚠️ Security Notice: The following rule allows SSH access from anywhere on the internet. For production environments, restrict this to your specific IP range or use AWS Systems Manager Session Manager for more secure access.

aws ec2 authorize-security-group-ingress \
    --group-id $ECS_SG_ID \
    --protocol tcp \
    --port 22 \
    --cidr 0.0.0.0/0

Step 2: Prepare User Data Script

Create a user data script that configures the ECS agent with GPU support:

#!/bin/bash
echo ECS_CLUSTER=docling-ecs-cluster >> /etc/ecs/ecs.config
echo ECS_BACKEND_HOST=https://ecs.us-east-1.amazonaws.com >> /etc/ecs/ecs.config
echo ECS_ENABLE_GPU_SUPPORT=true >> /etc/ecs/ecs.config

Save this as user-data.sh and encode it in base64:

base64 user-data.sh

Step 3: Get the Optimal GPU AMI

AWS provides optimized AMIs for ECS with GPU support. Let's fetch the latest recommended AMI ID:

aws ssm get-parameters \
    --names /aws/service/ecs/optimized-ami/amazon-linux-2/gpu/recommended \
    --region $(aws configure get region)

Step 4: Create Launch Template

Launch Templates provide a way to store launch parameters so that you don't have to specify them every time you launch an instance. Create a JSON file called ec2-launch-template.json:

{
  "ImageId": "ami-0372b2cc554a36da2",
  "InstanceType": "g4dn.xlarge",
  "KeyName": "ECS_Instance_Key",
  "IamInstanceProfile": {
    "Name": "ec2_instance_role-profile"
  },
  "NetworkInterfaces": [
    {
      "AssociatePublicIpAddress": true,
      "DeleteOnTermination": true,
      "DeviceIndex": 0,
      "SubnetId": "<subnet id of public subnet",
      "Groups": ["value of $ECS_SG_ID"]
    }
  ],
  "UserData": "<replace_with_base64_encoded_user-data.sh>",
  "BlockDeviceMappings": [
    {
      "DeviceName": "/dev/xvda",
      "Ebs": {
        "VolumeSize": 30,
        "VolumeType": "gp3",
        "DeleteOnTermination": true,
        "Encrypted": true
      }
    }
  ],
  "TagSpecifications": [
    {
      "ResourceType": "instance",
      "Tags": [
        {
          "Key": "Name",
          "Value": "ECS-Instance"
        }
      ]
    },
    {
      "ResourceType": "volume",
      "Tags": [
        {
          "Key": "Name",
          "Value": "ECS-Instance-Volume"
        }
      ]
    }
  ],
  "Monitoring": {
    "Enabled": true
  },
  "MetadataOptions": {
    "HttpTokens": "required",
    "HttpPutResponseHopLimit": 2,
    "HttpEndpoint": "enabled"
  }
}

⚠️ Important Configuration Notes:

Replace the SubnetId with your actual public subnet ID from Part 1

Replace the Groups array with your actual security group ID (the $ECS_SG_ID we just created)

The IamInstanceProfile name (ec2_instance_role-profile) was created in Part 1 - make sure this matches your actual IAM instance profile name

Key Configuration Highlights

GPU Instance Type: g4dn.xlarge provides NVIDIA T4 GPU support
EBS Encryption: Enabled for data security
Enhanced Monitoring: Enabled for better observability
IMDSv2: Enforced for improved instance metadata security
GP3 Storage: Latest generation EBS for better price/performance

Now create the launch template:

EC2_LAUNCH_TEMPLATE_ID=$(aws ec2 create-launch-template \
    --launch-template-name DoclingLaunchTemplate \
    --tag-specifications 'ResourceType=launch-template,Tags=[{Key=Name,Value=ECS_EC2_Launch_Template}]' \
    --launch-template-data file://ec2-launch-template.json \
    --query "LaunchTemplate.LaunchTemplateId" \
    --output text)

# Add additional tags
aws ec2 create-tags --resources $EC2_LAUNCH_TEMPLATE_ID --cli-input-json file://tags.json

Step 5: Create ECS Cluster

Important: Create the ECS cluster before launching EC2 instances. The ECS agent on the instances needs an existing cluster to register with.

aws ecs create-cluster \
    --cluster-name docling-ecs-cluster \
    --tags key=Name,value=doclingECSCluster

# Get cluster ARN for tagging
DOCLING_CLUSTER_ARN=$(aws ecs describe-clusters \
    --clusters docling-ecs-cluster \
    --query "clusters[].clusterArn" \
    --output text)

# Add additional tags
aws ecs tag-resource \
    --resource-arn $DOCLING_CLUSTER_ARN \
    --tags file://cluster-tags.json

Step 6: Create Auto Scaling Group

Create the Auto Scaling Group with zero desired capacity initially. This ASG will use the public subnet we configured in Part 1:

aws autoscaling create-auto-scaling-group \
    --auto-scaling-group-name ECS_Asg \
    --launch-template LaunchTemplateId=$EC2_LAUNCH_TEMPLATE_ID,Version='$Latest' \
    --min-size 0 \
    --max-size 0 \
    --desired-capacity 0 \
    --vpc-zone-identifier $PUBLIC_SUBNET \
    --tags Key=Name,Value=ECS_AutoScaling

📋 Note: The $PUBLIC_SUBNET variable should contain the subnet ID from Part 1. If you need to retrieve your subnet ID, refer to the VPC setup section in Part 1.

Configure Auto Scaling Group Tags

Create an asg-tags.json file for propagating tags to launched instances:

[
   {
        "ResourceId": "ECS_Asg",
        "ResourceType": "auto-scaling-group",
        "Key": "Purpose",
        "Value": "DoclingSetup",
        "PropagateAtLaunch": true
    },
    {
        "ResourceId": "ECS_Asg",
        "ResourceType": "auto-scaling-group",
        "Key": "Environment",
        "Value": "Dev",
        "PropagateAtLaunch": true
    }
]

Apply the tags:

aws autoscaling create-or-update-tags --tags file://asg-tags.json

Step 7: Testing and Verification

Launch an Instance

Scale up the Auto Scaling Group to launch an instance:

aws autoscaling update-auto-scaling-group \
    --auto-scaling-group-name ECS_Asg \
    --min-size 1 \
    --max-size 1 \
    --desired-capacity 1

Monitor Scaling Activities

Check the status of the launch activity:

aws autoscaling describe-scaling-activities \
    --auto-scaling-group-name ECS_Asg \
    --query 'Activities[?StatusCode==`InProgress`]'

Verify Tag Propagation

Confirm that tags from the ASG were propagated to the EC2 instance:

aws ec2 describe-instances --filters "Name=tag-key,Values=Purpose"

Get Instance IP Address

Find the IP address of the launched EC2 instance for SSH access:

aws ec2 describe-instances \
    --filters "Name=instance-state-name,Values=running" "Name=tag-key,Values=Purpose" \
    --query "Reservations[*].Instances[*].[InstanceId,InstanceType,PrivateIpAddress,PublicIpAddress]" \
    --output table

This command will display a table showing the Instance ID, Instance Type, Private IP Address, and Public IP Address of all running instances tagged with the "Purpose" key. Make note of the Public IP Address as you'll need it for SSH access in the next step.

Verify ECS Agent

SSH into the launched instance and check the ECS agent status:

# SSH into the instance using the generated key
ssh -i ECSInstanceKey.pem ec2-user@<INSTANCE_PUBLIC_IP>

# Check ECS agent status
sudo systemctl status ecs

# Verify ECS agent container is running
sudo docker ps

Verify Cluster Registration

Check if the instance successfully registered with the ECS cluster:

aws ecs list-container-instances --cluster docling-ecs-cluster

Best Practices and Production Considerations

Security Enhancements

Restrict SSH Access: Replace 0.0.0.0/0 with your specific IP range
Use AWS Systems Manager: Consider Session Manager for secure shell access
Enable VPC Flow Logs: Monitor network traffic for security analysis
Use Secrets Manager: Store sensitive configuration data securely

Monitoring and Logging

CloudWatch Container Insights: Enable for comprehensive ECS monitoring
Custom Metrics: Set up custom CloudWatch metrics for your applications
Log Aggregation: Use CloudWatch Logs or a centralized logging solution

Cost Optimization

Spot Instances: Consider using Spot Instances for cost savings
Mixed Instance Types: Use multiple instance types in your ASG
Scheduled Scaling: Implement time-based scaling policies
ECS Capacity Providers: Use for automatic scaling based on resource utilization

Conclusion

You've successfully created a scalable, GPU-enabled ECS infrastructure using Auto Scaling Groups and Launch Templates. This setup builds upon the foundational networking and IAM infrastructure we established in Part 1 and provides:

Automated scaling based on demand
Consistent instance configuration through Launch Templates
Proper tagging for resource management and cost allocation
GPU support for compute-intensive workloads
High availability and fault tolerance
Integration with the VPC and security architecture from Part 1

The infrastructure is now ready to host containerized applications that require GPU processing power, with the flexibility to scale automatically based on your workload demands.

What's Next?

In Part 3 of this series, we'll cover:

Creating and deploying ECS Task Definitions
Setting up ECS Services for your applications
Implementing Application Load Balancer for traffic distribution
Advanced ECS configurations and monitoring

Stay tuned for the final part where we'll bring everything together with actual application deployment!

Series Navigation

Part 1: Foundation - Networking & IAM - VPC setup, subnets, security groups, and IAM roles
Part 2: ECS EC2 with Auto Scaling (Current) - Launch templates, Auto Scaling Groups, and ECS cluster setup
Part 3: Application Deployment - Task definitions, services, and load balancers

Next Steps

Consider implementing:

ECS Services and Task Definitions for your applications
Application Load Balancer for distributing traffic

This foundation, combined with the networking setup from Part 1, provides a robust starting point for deploying docling to AWS ECS cluster as containerized applications on AWS ECS with GPU support.

Deploy docling to AWS ECS: A Complete Guide

bikash119 — Sun, 07 Sep 2025 12:37:31 +0000

I want to share key insights from my journey deploying Docling to AWS ECS (Elastic Container Service). Rather than overwhelming you with one massive tutorial, I've structured this as a three-part series that mirrors the logical flow of AWS infrastructure setup.

Series Overview

Part 1: Foundation – Networking & IAM
Setting up the essential building blocks: VPC configuration, security groups, and IAM roles. Since networking and permissions form the backbone of any AWS deployment, we'll establish these fundamentals first.

Part 2: Setting up the Workhorse – EC2
Configuring the compute infrastructure that will run our containers: Auto Scaling Groups (ASG), EC2 launch templates, and instance profiles.

Part 3: ECS & Application Load Balancer
Creating ECS tasks and services, then configuring an Application Load Balancer (ALB) to provide external access to our Docling deployment.

Why This Structure?

Cloud deployments require significant upfront investment in networking and security configuration. By tackling these foundational elements first, we create a solid platform for our application infrastructure. The EC2 layer serves as our container runtime environment, while ECS orchestrates our application containers on top of this foundation.

Prerequisites

This tutorial assumes you have:

Basic familiarity with AWS concepts and services
An active AWS account with appropriate permissions.
AWS CLI installed and configured on your local machine

If you need to set up the AWS CLI, follow the installation and configuration steps in the AWS documentation to get started. For those comfortable taking a calculated risk and wanting to get started quickly, you can use your root user AWS access key and secret for initial setup and learning purposes.

What You'll Build

By the end of this series, you'll have:

A production-ready VPC with proper security groups
Auto-scaled EC2 instances configured for container workloads
ECS services running Docling containers
Load-balanced external access to your deployment
A deep understanding of how these AWS services work together

Let's begin building our docling deployment infrastructure starting with Networking and IAM roles.

Foundation – Networking & IAM

bikash119 — Sun, 07 Sep 2025 12:35:13 +0000

This is Part 1 of a 3-part series on Deploy Docling to AWS ECS infrastructure.

AWS VPC Infrastructure Setup Guide

Overview

This guide provides step-by-step instructions for creating a secure Virtual Private Cloud (VPC) infrastructure on AWS using the AWS CLI. A VPC enables you to provision a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define.

Prerequisites

AWS CLI installed and configured with appropriate credentials
Basic understanding of networking concepts (CIDR blocks, subnets)
Appropriate IAM permissions for VPC and EC2 operations

Resource Tagging Strategy

Throughout this tutorial, we implement a consistent tagging strategy for all resources. This approach ensures:

Easy resource identification and tracking
Simplified cost allocation
Better resource organization and management

Standard tags used:

Name: Descriptive identifier for the resource
Environment: Development stage (e.g., Dev, Staging, Production)
Purpose: Project or application identifier

1. Virtual Private Cloud (VPC) Setup

Create the VPC

Create a VPC with a /16 CIDR block, providing approximately 65,536 IP addresses:

aws ec2 create-vpc \
    --cidr-block 10.0.0.0/16 \
    --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=DoclingVPC},{Key=Environment,Value=Dev}]'

Note: The CIDR block 10.0.0.0/16 means all resources within this VPC will have IP addresses starting with 10.0.x.x, where the first two octets remain constant.

Store VPC ID for Future Reference

VPC_ID=$(aws ec2 describe-vpcs \
    --filters Name=tag-key,Values=Name \
    --query "Vpcs[?Tags[?Key=='Name' && Value=='DoclingVPC']].VpcId" \
    --output text)

Apply Additional Tags

Create a tags.json file:

{
    "Tags": [
        {
            "Key": "Purpose",
            "Value": "DoclingSetup"
        }
    ]
}

Apply the tags to the VPC:

aws ec2 create-tags --resources $VPC_ID --cli-input-json file://tags.json

Verify VPC Creation

aws ec2 describe-vpcs \
    --filters Name=tag-key,Values=Purpose \
    --query "Vpcs[].[VpcId,CidrBlock,State]" \
    --output table

2. Subnet Configuration

Subnets segment your VPC into smaller networks. We'll create both public and private subnets following AWS best practices.

Create Private Subnet

Deploy a private subnet in availability zone us-east-1b:

aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.4.0/24 \
    --availability-zone us-east-1b \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=PrivateSubnet-1b}]'

Create Public Subnet

Deploy a public subnet in availability zone us-east-1a:

aws ec2 create-subnet \
    --vpc-id $VPC_ID \
    --cidr-block 10.0.3.0/24 \
    --availability-zone us-east-1a \
    --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=PublicSubnet-1a}]'

Note: Each /24 subnet provides 256 IP addresses (though AWS reserves 5 IPs per subnet for internal use).

Retrieve Subnet IDs

# Get Private Subnet ID
PRIVATE_SUBNET=$(aws ec2 describe-subnets \
    --filters Name=tag-value,Values=PrivateSubnet-1b \
    --query "Subnets[0].SubnetId" \
    --output text)

# Get Public Subnet ID
PUBLIC_SUBNET=$(aws ec2 describe-subnets \
    --filters Name=tag-value,Values=PublicSubnet-1a \
    --query "Subnets[0].SubnetId" \
    --output text)

Apply Tags to Subnets

aws ec2 create-tags --resources $PUBLIC_SUBNET --cli-input-json file://tags.json
aws ec2 create-tags --resources $PRIVATE_SUBNET --cli-input-json file://tags.json

Verify Subnet Configuration

aws ec2 describe-subnets \
    --filters Name=tag-key,Values=Purpose \
    --query "Subnets[].[SubnetId,CidrBlock,AvailabilityZone]" \
    --output table

3. Internet Gateway Configuration

An Internet Gateway (IGW) enables communication between your VPC and the internet.

Create Internet Gateway

IGW_ID=$(aws ec2 create-internet-gateway \
    --tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=DoclingIGW}]' \
    --query "InternetGateway.InternetGatewayId" \
    --output text)

Apply Tags

aws ec2 create-tags --resources $IGW_ID --cli-input-json file://tags.json

Attach Internet Gateway to VPC

aws ec2 attach-internet-gateway \
    --internet-gateway-id $IGW_ID \
    --vpc-id $VPC_ID

Verify Internet Gateway

aws ec2 describe-internet-gateways \
    --filters Name=tag-key,Values=Purpose \
    --query "InternetGateways[].[InternetGatewayId,Attachments[0].VpcId,Attachments[0].State]" \
    --output table

4. Route Table Configuration

Route tables determine where network traffic is directed. AWS creates a main route table automatically with each VPC, but best practice dictates creating custom route tables for better security control.

Create Public Route Table

PUBLIC_ROUTE_TABLE_ID=$(aws ec2 create-route-table \
    --vpc-id $VPC_ID \
    --query "RouteTable.RouteTableId" \
    --output text)

Add Internet Route

Configure the route table to direct internet-bound traffic through the Internet Gateway:

aws ec2 create-route \
    --route-table-id $PUBLIC_ROUTE_TABLE_ID \
    --destination-cidr-block 0.0.0.0/0 \
    --gateway-id $IGW_ID

Associate Public Subnet with Route Table

PUBLIC_SUBNET_ASSOCIATION_ID=$(aws ec2 associate-route-table \
    --route-table-id $PUBLIC_ROUTE_TABLE_ID \
    --subnet-id $PUBLIC_SUBNET \
    --query "AssociationId" \
    --output text)

Verify Route Table Configuration

aws ec2 describe-route-tables \
    --route-table-ids $PUBLIC_ROUTE_TABLE_ID \
    --query "RouteTables[].[RouteTableId,Routes[].DestinationCidrBlock,Associations[].SubnetId]" \
    --output table

Security Best Practices

Main Route Table: Never add internet routes to the main route table. This ensures new subnets don't accidentally become public.
Subnet Isolation: Keep private subnets truly private by not associating them with route tables containing internet gateway routes.
Multi-AZ Deployment: For production environments, deploy subnets across multiple availability zones for high availability.
Network ACLs: Consider implementing Network Access Control Lists for additional subnet-level security.

AWS IAM Roles and Instance Profiles

AWS IAM operates on three fundamental components:

Trust Relationship - Defines which entities can assume a role
Permissions - Specifies what actions the role can perform
Temporary Credentials - Provides time-limited access tokens for role assumption

IAM Implementation Process

Create Trust Policy - Define which AWS services can assume the role
Create Role - Establish the role with the trust policy
Attach Permissions - Grant specific permissions to the role
Create Instance Profile - Package the role for EC2 instances (when needed)
Associate Resources - Link the role or instance profile to AWS resources

You may want to dive into the concepts here.

EC2 IAM Configuration

Trust Policy for EC2

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

Create EC2 Instance Role

# Create the role
aws iam create-role \
  --role-name ec2_instance_role \
  --assume-role-policy-document file://ec2-trust-policy.json \
  --tags Key=Name,Value=EC2InstanceRole

# Add additional tags
aws iam tag-role \
  --role-name ec2_instance_role \
  --cli-input-json file://tags.json

# Attach managed policy
aws iam attach-role-policy \
  --role-name ec2_instance_role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role

Create Instance Profile

# Create instance profile
aws iam create-instance-profile \
  --instance-profile-name ec2_instance_role-profile \
  --tags Key=Name,Value=EC2InstanceRoleProfile

# Add tags to instance profile
aws iam tag-instance-profile \
  --instance-profile-name ec2_instance_role-profile \
  --cli-input-json file://tags.json

# Associate role with instance profile
aws iam add-role-to-instance-profile \
  --role-name ec2_instance_role \
  --instance-profile-name ec2_instance_role-profile

ECS IAM Configuration

Trust Policy for ECS Tasks

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ecs-tasks.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

ECS Task Execution Role

The task execution role is required for ECS to pull container images and write logs on your behalf.

# Create ECS task execution role
ECS_TASK_EXEC_ROLE=$(aws iam create-role \
  --role-name ecs_task_exec_role \
  --assume-role-policy-document file://ecs-trust-policy.json \
  --tags Key=Name,Value=ECSTaskExecutionRole \
  --query "Role.RoleName" \
  --output text)

# Add tags
aws iam tag-role \
  --role-name $ECS_TASK_EXEC_ROLE \
  --cli-input-json file://tags.json

# Attach AWS managed policy for task execution
aws iam attach-role-policy \
  --role-name $ECS_TASK_EXEC_ROLE \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

ECS Task Role (Optional)

The task role provides permissions for your application code running inside the container.

Note: This role can be omitted if your application doesn't need AWS API access. Test without it first to determine if it's necessary.

# Create ECS task role (if needed)
ECS_ROLE=$(aws iam create-role \
  --role-name ecs_task_role \
  --assume-role-policy-document file://ecs-trust-policy.json \
  --tags Key=Name,Value=ECSTaskRole \
  --query "Role.RoleName" \
  --output text)

# Add tags
aws iam tag-role \
  --role-name $ECS_ROLE \
  --cli-input-json file://tags.json

# Attach application-specific policies as needed
# aws iam attach-role-policy --role-name $ECS_ROLE --policy-arn <policy-arn>

Summary

You now have the essential IAM & networking foundation for our AWS ECS deployment.

Next Steps: In the next part of this series, we'll configure the EC2 infrastructure that will use these roles to run your containerized applications.

IAM Roles & Instance Profiles

bikash119 — Sun, 07 Sep 2025 12:25:17 +0000

💡 Understanding the Concepts

Understanding IAM Through a Real-World Analogy

Imagine a FedEx delivery to a house inside a gated community. This scenario perfectly illustrates how AWS IAM works:

1. Trust Policy - The Community Protocol

The gated community management creates a protocol stating: "Only verified FedEx employees can be assigned the role of package delivery within our community." This is your Trust Policy - it defines which entities (FedEx employees = AWS services like EC2, ECS) are trusted to assume a specific role.

2. Role - The Delivery Role

Using this protocol, the security team at the gate creates a "Package Delivery Role." This role exists as a defined position with specific responsibilities, but no one holds it permanently. This is your IAM Role - a set of permissions that can be temporarily assumed by trusted entities.

3. Permissions - What the Role Can Do

The role comes with specific permissions: "Can access all residential areas, cannot access the car charging stations, must use designated parking spots." These rules define what anyone with this role can or cannot do within the community. This represents your Role Policies - the specific AWS permissions attached to the role.

4. Instance Profile - The Badge Vending Machine

When a FedEx driver arrives, security verifies their identity and issues a temporary access badge with all the delivery permissions encoded into it. Once the delivery is complete, the badge expires automatically. The driver loses all community access privileges upon exit. This is your Instance Profile - it provides temporary credentials to AWS resources, ensuring access is time-limited and automatically revoked.

This system ensures that even legitimate delivery personnel only have access for exactly as long as needed, with precisely the permissions required for their task.

Why Instance Profiles? EC2 vs ECS Role Assignment

EC2 Instances Have Their Own Job in ECS

When an EC2 instance joins an ECS cluster, it becomes a worker node with specific responsibilities:

Register itself with the ECS service
Report available resources (CPU, memory, storage)
Download and start containers as instructed by ECS
Monitor container health and report status
Manage container lifecycle events

To perform these cluster management tasks, the EC2 instance needs the AmazonEC2ContainerServiceforEC2Role policy. Since EC2 is a general-purpose compute service, it uses instance profiles to securely deliver these credentials.

ECS Tasks Assume Roles Directly

ECS tasks, however, don't need instance profiles because:

They're purpose-built containerized workloads with specific, known requirements
ECS can directly inject appropriate credentials into containers
The ECS service handles credential management natively

Two Separate Layers of Permissions

Think of it as two distinct jobs:

Infrastructure Layer (EC2 + Instance Profile): "I'm a worker node that needs to communicate with ECS control plane"
Application Layer (ECS Tasks + Task Roles): "I'm a specific application that needs to access AWS services"

This separation allows the same EC2 instance to host multiple different ECS tasks, each with their own specific permissions, while the instance itself maintains its cluster management permissions.

Equity Fundamental Octo Researcher: Next-Gen Stock Research

bikash119 — Sat, 30 Aug 2025 05:26:40 +0000

This is a submission for the AI Agents Challenge powered by n8n and Bright Data

What I Built

I created an AI-powered assistant to streamline fundamental analysis for stock market investment ( India ). Investors typically spend hours reviewing annual reports, credit ratings, quarterly results, and transcripts to make investment decisions—a process that’s slow and overwhelming.

My solution uses Large Language Models (LLMs) to analyze and summarize these documents, enabling users to ask questions and instantly receive targeted, contextual answers. This tool dramatically accelerates research, reduces manual effort, and helps analysts focus on insights, not information overload.

Demo

n8n Workflow

Equity Fundamental Octo Researcher N8N Workflow

Technical Implementation

Overview

The Equity Fundamental Octo Researcher workflow automates the extraction, processing, and retrieval of fundamental stock data and reports from Screener.in. It enables interactive fundamental analysis by combining web scraping, automated document parsing (using docling-project by IBM), vector storage (Pinecone), and leading LLMs for Q&A.

Architecture/Components

Form Trigger: Accepts Screener.in company URLs from the user.
BrightData HTTP Nodes: Automate scraping of key disclosure documents (annual reports, transcripts, presentations, etc.).
Intermediary Storage & Extraction: Handles job tracking and discovery of report/document URLs.
Docling OSS: All document parsing, text extraction, and conversion (from PDF, HTML, image, etc.) are performed through the open-source docling-project library, orchestrated via a custom Gradio API deployed to AWS ECS.
Async/Task Management: Manages and monitors asynchronous parsing with job/event IDs.
Embedding and Vector Database: Processes parsed content into OpenAI embeddings and loads into Pinecone for fast semantic search.
LLM-Powered Q&A Agent: Uses Anthropic Claude for question/answer generation, with context retrieved from Pinecone.
n8n Chat Trigger: Interface for handling user questions about the company or its documents.

Execution Flow

User Input User submits a company URL from Screener.in.
Web Scraping with BrightData The workflow triggers a BrightData scraper run to find and collect links to all relevant fundamental documents.
Extract and Track Results Extraction nodes gather the discovered URLs and manage job progress.
Document Processing (Docling OSS) Every collected URL is processed by docling (invoked via a Gradio-based API deployed to AWS). docling-project extracts, parses, and converts source documents—including PDFs, scanned images, web pages—into clean markdown/text suitable for downstream NLP.
Async Task Management The workflow creates, tracks, and retrieves outputs from asynchronous Docling processing jobs using event/task IDs.
Text Embeddings and Storage Extracted text is transformed into embeddings using OpenAI and loaded into Pinecone for retrieval-augmented analysis.
Interactive Chat/Q&A The user interacts via chat; queries leverage the Anthropic Claude LLM plus context pulled via semantic search from Pinecone vector store.

Integrations

BrightData API: Web scraping for financial documents.
n8n: Workflow automation and orchestration.
Docling OSS (deployed to AWS ECS): Core document conversion and extraction.
OpenAI API: Embedding generation.
Pinecone: Vector database.
Anthropic Claude API: Question-answering language model.

Usage Instructions

Deploy the workflow with all service credentials and dependencies configured.
Expose the webform to users for them to submit Screener.in URLs.
Scraping, parsing (via Docling OSS), and embedding are automated in the pipeline.
Users chat with the agent to ask analysis or document-grounded questions.
Extend as needed—add new endpoints, sources, or Docling capabilities for more coverage.

Bright Data Verified Node

There was no scraper available for https://screener.in that could reliably capture all relevant document links (annual reports, con call transcripts, quarterly reports, investor presentations, etc.). To overcome this, I built a custom collector from scratch using the BrightData IDE. You can find the scraper here

Challenges

Building a Custom Scraper for Screener.in

There was no ready-made scraper available for Screener.in that could reliably capture all relevant document links (annual reports, con call transcripts, quarterly reports, investor presentations, etc.). To overcome this, I built a custom collector from scratch using the BrightData IDE. You can find the scraper here
Self-Hosting docling-project OSS as a Cloud Service

docling, which handles document parsing and conversion, is not available as a hosted SaaS. To make it production-ready, converted Docling’s docker-compose deployment into an AWS ECS task, set up networking with a load balancer, and ensured stable API access from n8n. This required container orchestration, secure environment set-up, persistent storage, and custom integration so that document parsing at scale could be achieved in a robust and maintainable way.

Learnings

Use Brightdata and scrape / collect whatever you are interested in.
A ton about AWS networking, IAM , ECS & LoadBalancers.My learnings here.
I have just scratched the surface of docling-project.