Forem: Learn2Skills

Microsoft Ignite 2025 Release Updates

Learn2Skills — Sat, 03 Jan 2026 05:54:36 +0000

AWS re:Invent 2025 Announcements

Learn2Skills — Thu, 25 Dec 2025 17:20:39 +0000

AWS re:Invent 2025 Announcements

Artificial Intelligence

Amazon Nova 2 Sonic: Advanced Speech-to-Speech Model for Conversational AI
Amazon launches Nova 2 Sonic, a next-generation speech-to-speech AI model designed to enhance natural voice interactions. This model supports multilingual conversations, dynamic speech control, and crossmodal inputs (integrating speech, text, images, etc.), along with improved telephony integration. It maintains conversational context across multiple tasks, enabling more fluid, human-like dialogue in applications such as virtual assistants, customer support, and real-time translation.

Amazon Nova 2 Lite: Fast, Cost-Effective Reasoning AI Model
Nova 2 Lite is introduced as a streamlined AI model optimized for everyday applications requiring quick, efficient reasoning. It offers an extended context window of up to one million tokens and comes equipped with built-in tools to facilitate complex thought processes, making it suitable for cost-sensitive deployments without sacrificing performance.

Amazon Nova Forge: Custom Frontier Model Development Program
Nova Forge empowers organizations to create bespoke frontier AI models by providing access to Nova’s training infrastructure. This program removes traditional barriers like high costs, extensive compute requirements, and lengthy development cycles, enabling companies to infuse domain-specific expertise into foundational models tailored to their unique needs.

Amazon Nova 2 Omni (Preview): Multimodal Reasoning and Image Generation
Nova 2 Omni is an all-in-one AI model preview supporting multiple input types—text, images, videos, and speech—and capable of generating both text and image outputs. This multimodal architecture facilitates sophisticated reasoning across diverse data forms, opening new possibilities for integrated AI applications.

Amazon Nova Act: Reliable AI Agents for UI Workflow Automation

Now generally available, Amazon Nova Act enables developers to build AI agents that automate complex browser-based tasks such as form filling, searching and extracting information, shopping and booking, and quality assurance testing. These agents achieve over 90% reliability, making them viable for enterprise-grade automation.

Amazon Bedrock AgentCore: Enhanced AI Agent Deployment Controls
AgentCore enhances AI agent deployment with advanced policy controls, quality evaluations, improved memory management, and natural conversational capabilities. This facilitates scalable and trustworthy AI agent implementations across organizations, ensuring compliance and operational integrity.

Amazon S3 Vectors: Scalable, High-Performance Vector Storage
Amazon S3 Vectors reaches general availability, scaling vector storage and querying to unprecedented levels—up to 2 billion vectors per index with query latencies around 100 milliseconds. It supports expanded regional availability and reduces costs by up to 90% compared to specialized vector databases, making large-scale AI workloads more accessible and economical.

Amazon Bedrock: Expanded Foundation Model Access
Amazon Bedrock now offers 18 fully managed open-weight foundation models from industry leaders including Google, NVIDIA, OpenAI, Mistral AI, Kimi AI, MiniMax AI, and Qwen. The lineup includes the latest Mistral Large 3 and Ministral 3 models in various sizes (3B, 8B, 14B parameters), providing developers with a rich selection of pre-trained models optimized for diverse AI tasks.

Amazon SageMaker AI with Serverless MLflow: Simplified AI Experimentation
SageMaker AI integrates serverless MLflow to streamline AI experimentation. This zero-infrastructure service deploys within minutes, auto-scales based on demand, and integrates seamlessly with SageMaker’s model customization and pipeline tools, accelerating development cycles and reducing operational overhead.

Amazon Bedrock Reinforcement Fine-Tuning: Smarter AI Models with Less Effort
Bedrock introduces reinforcement fine-tuning capabilities that improve model accuracy by 66% over base models using feedback-driven training. This approach eliminates the need for large labeled datasets or deep ML expertise, democratizing advanced model customization.

Checkpointless and Elastic Training on Amazon SageMaker HyperPod

SageMaker HyperPod enhances AI training with checkpointless recovery, allowing instant continuation after failures, and elastic scaling that adjusts resources dynamically. These improvements accelerate model development by reducing downtime and optimizing compute utilization.

Serverless Customization in Amazon SageMaker AI

Further expanding on SageMaker’s capabilities, serverless customization enables rapid fine-tuning with automatic failure recovery and resource scaling, boosting productivity and simplifying AI model refinement.

Compute

AWS Graviton5: Most Powerful and Efficient CPU Yet
AWS introduces Graviton5, its fifth-generation CPU chip delivering superior price-performance across a broad spectrum of workloads on Amazon EC2. The chip combines efficiency with high computational power, catering to diverse applications from web hosting to complex data processing.

Trainium3 UltraServers: Advanced AI Training and Deployment
Trainium3 UltraServers, powered by AWS’s first 3nm AI chip, provide enhanced speed and cost-efficiency for AI training and inference. These servers enable organizations to tackle ambitious AI workloads more effectively, supporting growth in AI adoption across industries.

Amazon EC2 X8aedz Instances: High-Performance Memory-Optimized Compute

The new EC2 X8aedz instances feature 5th Gen AMD EPYC processors with up to 5 GHz speeds and 3 TiB of memory. Designed for memory-intensive tasks like electronic design automation and large databases, these instances deliver exceptional single-threaded performance and scalability.

AWS Lambda Managed Instances: Serverless Benefits with EC2 Flexibility

Lambda Managed Instances allow running Lambda functions on EC2 infrastructure, combining serverless simplicity with the flexibility to use specialized hardware and cost-optimized EC2 pricing. AWS manages the underlying infrastructure, simplifying deployment of workloads requiring unique compute resources.

AWS Lambda Durable Functions: Multi-Step AI Workflows

Durable Functions extend Lambda’s capabilities by enabling orchestration of multi-step applications that can run reliably over long periods (up to one year). This feature eliminates the need to pay for idle compute time during waits for external events or human input, optimizing cost and resource use.

Containers

Amazon EKS Enhancements: Workload Orchestration and Cloud Resource Management
Amazon EKS introduces new fully managed features that streamline Kubernetes workload orchestration and cloud resource management. These enhancements reduce infrastructure maintenance burdens while offering enterprise-level reliability, security, and operational efficiency.

Database

Database Savings Plans for AWS Databases
A new pricing model, Database Savings Plans, helps organizations optimize costs while maintaining flexibility across database services and deployment options, encouraging more cost-effective database management.

Amazon RDS for SQL Server and Oracle: Cost and Scalability Improvements
Amazon RDS introduces new capabilities including SQL Server Developer Edition support, optimized CPU performance with M7i/R7i instances, and expanded storage options up to 256 TiB. These features enhance cost efficiency and scalability for development, testing, and production environments.

Amazon OpenSearch Service: GPU-Accelerated Vector Database Performance

OpenSearch Service now supports GPU acceleration and auto-optimization for vector databases, enabling workloads to run up to 10 times faster at 25% of previous costs. This advancement balances search quality, speed, and resource usage for large-scale AI search applications.

Global Infrastructure

AWS AI Factories: On-Premises AI Infrastructure Deployment

AWS AI Factories provide fully managed AI infrastructure that can be deployed within enterprise and government data centers. This solution integrates foundation models, specialized hardware, and AWS services, accelerating AI initiatives while ensuring data residency and compliance requirements are met.

Management & Governance

AWS DevOps Agent (Preview): Autonomous Incident Response

The DevOps Agent acts like an autonomous on-call engineer, analyzing data from CloudWatch, GitHub, ServiceNow, and more to identify root causes and coordinate incident response. This tool accelerates issue resolution and improves system reliability.

Enhanced AWS Support Plans: AI-Powered Expert Guidance

New AWS Support plans combine AI-driven insights with expert human guidance to proactively monitor and prevent cloud infrastructure issues. These plans offer faster response times and comprehensive coverage across performance, security, and cost.

Amazon CloudWatch: Unified Data Management and Analytics

CloudWatch introduces automatic normalization of data from multiple sources, native analytics integration, and support for standards like OCSF and Apache Iceberg. These capabilities reduce complexity, lower costs, and improve operational, security, and compliance analytics.

Migration & Modernization

AWS Transform Custom: AI-Powered Code Modernization

AWS Transform Custom leverages AI to automate code modernization at scale, learning organizational patterns to transform repositories and reduce execution time by up to 80%. This accelerates tech debt reduction and application modernization.

AWS Transform for Windows: Full-Stack Modernization

This service modernizes Windows applications up to five times faster by coordinating AI-powered transformations across code, UI frameworks, databases, and deployment configurations, enabling comprehensive modernization efforts.

AWS Transform for Mainframe: Reimagine and Automated Testing

New capabilities support mainframe modernization by transforming legacy applications into cloud-native architectures while automating complex testing. This reduces modernization timelines from years to months through intelligent analysis and automated test generation.

Networking & Content Delivery

Amazon Route 53 Global Resolver (Preview): Secure Anycast DNS Resolution
Global Resolver simplifies hybrid DNS management by resolving both public and private domains globally via secure anycast-based DNS. This unified service reduces operational complexity and maintains consistent security controls across hybrid environments.

Partner Network

AWS Partner Central: Console Integration

Partner Central is now accessible directly within the AWS Management Console, streamlining the partner journey from customer onboarding to managing solutions, opportunities, and marketplace listings with enterprise-grade security in a unified interface.

Security, Identity, & Compliance

AWS Security Agent (Preview): Proactive Application Security
The Security Agent scales AppSec expertise through AI-powered design reviews, code analysis, and contextual penetration testing tailored to unique application architectures, enhancing security from design through deployment.

Amazon GuardDuty: Extended Threat Detection
GuardDuty now offers extended threat detection across Amazon EC2 and ECS, providing unified visibility into virtual machines and containers. This helps identify complex multi-stage attacks affecting interconnected AWS workloads.

AWS Security Hub: Near Real-Time Analytics and Risk Prioritization

Security Hub is generally available with capabilities to correlate security signals in near real-time across AWS environments, enabling faster risk response and improved security posture.

IAM Policy Autopilot: Open Source MCP Server for Policy Generation

IAM Policy Autopilot accelerates policy creation by analyzing application code to generate valid IAM policies. It provides AI coding assistants with current AWS service knowledge and permission recommendations, simplifying secure development.

Storage

Amazon FSx for NetApp ONTAP Integration with Amazon S3

FSx for NetApp ONTAP now integrates seamlessly with Amazon S3, enabling direct file system data access via S3. This facilitates unified workflows with AWS analytics, ML, and generative AI services without moving or duplicating data.

Replication Support and Intelligent-Tiering for Amazon S3 Tables
New features introduce automated cost optimization through intelligent-tiered storage and simplified replication of S3 Tables across regions and accounts, enhancing data availability and cost efficiency.

Amazon S3 Storage Lens: Enhanced Performance Metrics and Scalability
Storage Lens adds advanced performance metrics, supports analysis of billions of prefixes, and enables metric exports to S3 Tables. These enhancements help optimize application performance and simplify large-scale data analytics.

This comprehensive summary covers the latest AWS launches and updates across analytics, AI, compute, containers, databases, infrastructure, management, migration, networking, partner programs, security, and storage, providing a detailed overview of innovations designed to accelerate cloud adoption, optimize cost and performance, and enhance security and manageability.

Monitor Amazon ECS containers with ECS Exec

Learn2Skills — Thu, 25 Dec 2025 14:09:45 +0000

Amazon ECS Exec enables direct interaction with running containers for troubleshooting and monitoring without needing SSH access or host-level intervention. This feature simplifies diagnostics by allowing commands or shells inside containers on EC2, Fargate, or ECS Anywhere.

Key Features
ECS Exec supports Linux and Windows containers, logging commands to CloudWatch or S3 for auditing via CloudTrail, and uses AWS KMS for encryption. Enable it at the cluster level with executeCommandConfiguration and per-task via enableExecuteCommand.

The Amazon ECS console now supports ECS Exec, enabling you to open secure, interactive shell access directly from the AWS Management Console to any running container.

ECS customers often need to access running containers to debug applications and examine running processes. ECS Exec provides easy and secure access to running containers without requiring inbound ports or SSH key management.

To get started, you can turn on ECS Exec directly in the console when creating or updating services and standalone tasks. Additional settings like encryption and logging can also be configured at the cluster level through the console. Once enabled, simply navigate to a task details page, select a container, and click "Connect" to open an interactive session through CloudShell. The console also displays the underlying AWS CLI command, which you can customize or copy to use in your local terminal.

Configuring ECS Exec

To use ECS Exec, you must first turn on the feature for your tasks and services, and then you can run commands in your containers.

Turning on ECS Exec for your tasks and services
You can turn on the ECS Exec feature for your services and standalone tasks by specifying the --enable-execute-command flag when using one of the following AWS CLI commands: create-service, update-service, start-task, or run-task.

For example, if you run the following command, the ECS Exec feature is turned on for a newly created service that runs on Fargate. For more information about creating services, see create-service.

aws ecs create-service \
    --cluster cluster-name \
    --task-definition task-definition-name \
    --enable-execute-command \
    --service-name service-name \
    --launch-type FARGATE \
     --network-configuration "awsvpcConfiguration={subnets=[subnet-12344321],securityGroups=[sg-12344321],assignPublicIp=ENABLED}" \
    --desired-count 1

After you turn on ECS Exec for a task, you can run the following command to confirm the task is ready to be used. If the lastStatus property of the ExecuteCommandAgent is listed as RUNNING and the enableExecuteCommand property is set to true, then your task is ready.

aws ecs describe-tasks \
    --cluster cluster-name \
    --tasks task-id

IAM permissions required for Amazon CloudWatch Logs or Amazon S3 Logging
To enable logging, the Amazon ECS task role that's referenced in your task definition needs to have additional permissions. These additional permissions can be added as a policy to the task role. They're different depending on if you direct your logs to Amazon CloudWatch Logs or Amazon S3.

Reference:

Creating an Amazon ECS service that uses Service Discovery

Learn2Skills — Mon, 17 Nov 2025 09:56:46 +0000

Amazon ECS services with Service Discovery enable dynamic discovery of containerized services using AWS Cloud Map, allowing tasks to register and find each other by DNS names without hardcoding IPs.

Core Concepts
Service Discovery integrates ECS with Cloud Map to automatically register service instances with custom names and health checks. Use it for microservices where tasks need to communicate via service names (e.g., api.default.local). Supports private/public namespaces and automatic deregistration on task stops.

When an ECS task associated with a service discovery-enabled ECS service starts, Amazon ECS automatically registers the task's IP address and port with AWS Cloud Map. Other services within the same Cloud Map namespace can then resolve the service's name (e.g., myservice.example.com) to discover the IP addresses of the running tasks and establish connections. This eliminates the need for manual IP address management and provides a flexible, dynamic way for services to interact.

Before you start this tutorial, make sure that the following prerequisites are met:

The latest version of the AWS CLI is installed and configured. For more information, see Installing or updating to the latest version of the AWS CLI.
The steps described in Set up to use Amazon ECS are complete.
Your IAM user has the required permissions specified in the AmazonECS_FullAccess IAM policy example.
You have created at least one VPC and one security group. For more information, see Create a virtual private cloud.

Step 1: Create the Service Discovery resources in AWS Cloud Map
Follow these steps to create your service discovery namespace and service discovery service:

Create a private Cloud Map service discovery namespace. This example creates a namespace that's called tutorial. Replace vpc-abcd1234 with the ID of one of your existing VPCs.

aws servicediscovery create-private-dns-namespace \
      --name tutorial \
      --vpc vpc-abcd1234

Using the OperationId from the output of the previous step, verify that the private namespace was created successfully. Make note of the namespace ID because you use it in subsequent commands. aws servicediscovery get-operation \ --operation-id h2qe3s6dxftvvt7riu6lfy2f6c3jlhf4-je6chs2e
Using the NAMESPACE ID from the output of the previous step, create a service discovery service. This example creates a service named myapplication. Make note of the service ID and ARN because you use them in subsequent commands.

aws servicediscovery create-service \
      --name myapplication \
      --dns-config "NamespaceId="ns-uejictsjen2i4eeg",DnsRecords=[{Type="A",TTL="300"}]" \
      --health-check-custom-config FailureThreshold=1

Step 2: Create the Amazon ECS resources
Follow these steps to create your Amazon ECS cluster, task definition, and service:

Create an Amazon ECS cluster. This example creates a cluster that's named tutorial.

aws ecs create-cluster \
      --cluster-name tutorial

a. Create a file that's named fargate-task.json with the contents of the following task definition.

{
    "family": "tutorial-task-def",
        "networkMode": "awsvpc",
        "containerDefinitions": [
            {
                "name": "sample-app",
                "image": "public.ecr.aws/docker/library/httpd:2.4",
                "portMappings": [
                    {
                        "containerPort": 80,
                        "hostPort": 80,
                        "protocol": "tcp"
                    }
                ],
                "essential": true,
                "entryPoint": [
                    "sh",
                    "-c"
                ],
                "command": [
                    "/bin/sh -c \"echo '<html> <head> <title>Amazon ECS Sample App</title> <style>body {margin-top: 40px; background-color: #333;} </style> </head><body> <div style=color:white;text-align:center> <h1>Amazon ECS Sample App</h1> <h2>Congratulations!</h2> <p>Your application is now running on a container in Amazon ECS.</p> </div></body></html>' >  /usr/local/apache2/htdocs/index.html && httpd-foreground\""
                ]
            }
        ],
        "requiresCompatibilities": [
            "FARGATE"
        ],
        "cpu": "256",
        "memory": "512"
}

b. Register the task definition using fargate-task.json.

aws ecs register-task-definition \
      --cli-input-json file://fargate-task.json

Create an ECS service by following these steps:

Create a file that's named ecs-service-discovery.json with the contents of the ECS service that you're creating. This example uses the task definition that was created in the previous step. An awsvpcConfiguration is required because the example task definition uses the awsvpc network mode.

When you create the ECS service, specify Fargate and the LATEST platform version that supports service discovery. When the service discovery service is created in AWS Cloud Map , registryArn is the ARN returned. The securityGroups and subnets must belong to the VPC that's used to create the Cloud Map namespace. You can obtain the security group and subnet IDs from the Amazon VPC Console.

{
    "cluster": "tutorial",
    "serviceName": "ecs-service-discovery",
    "taskDefinition": "tutorial-task-def",
    "serviceRegistries": [
       {
          "registryArn": "arn:aws:servicediscovery:region:aws_account_id:service/srv-utcrh6wavdkggqtk"
       }
    ],
    "launchType": "FARGATE",
    "platformVersion": "LATEST",
    "networkConfiguration": {
       "awsvpcConfiguration": {
          "assignPublicIp": "ENABLED",
          "securityGroups": [ "sg-abcd1234" ],
          "subnets": [ "subnet-abcd1234" ]
       }
    },
    "desiredCount": 1
}

b. Create your ECS service using ecs-service-discovery.json.

aws ecs create-service \
      --cli-input-json file://ecs-service-discovery.json

Step 3: Verify Service Discovery in AWS Cloud Map
You can verify that everything is created properly by querying your service discovery information. After service discovery is configured, you can either use AWS Cloud Map API operations, or call dig from an instance within your VPC. Follow these steps:

Using the service discovery service ID, list the service discovery instances. Make note of the instance ID (marked in bold) for resource cleanup.

 aws servicediscovery list-instances \
       --service-id srv-utcrh6wavdkggqtk

Use the service discovery namespace, service, and additional parameters such as ECS cluster name to query details about the service discovery instances.

aws servicediscovery discover-instances \
      --namespace-name tutorial \
      --service-name myapplication \
      --query-parameters ECS_CLUSTER_NAME=tutorial

The DNS records that are created in the Route 53 hosted zone for the service discovery service can be queried with the following AWS CLI commands:

Using the namespace ID, get information about the namespace, which includes the Route 53 hosted zone ID.

aws servicediscovery \
      get-namespace --id ns-uejictsjen2i4eeg

b. Using the Route 53 hosted zone ID from the previous step (see the text in bold), get the resource record set for the hosted zone.

aws route53 list-resource-record-sets \
      --hosted-zone-id Z35JQ4ZFDRYPLV

You can also query the DNS from an instance within your VPC using dig.

dig +short myapplication.tutorial

Step 4: Clean up
When you're finished with this tutorial, clean up the associated resources to avoid incurring charges for unused resources. Follow these steps:

When you're finished with this tutorial, clean up the associated resources to avoid incurring charges for unused resources. Follow these steps:

aws servicediscovery deregister-instance \
      --service-id srv-utcrh6wavdkggqtk \
      --instance-id 16becc26-8558-4af1-9fbd-f81be062a266

Using the OperationId from the output of the previous step, verify that the service discovery service instances were deregistered successfully.

aws servicediscovery get-operation \ 
      --operation-id xhu73bsertlyffhm3faqi7kumsmx274n-jh0zimzv

Delete the service discovery service using the service ID.

aws servicediscovery delete-service \ 
      --id srv-utcrh6wavdkggqtk

Delete the service discovery namespace using the namespace ID.

aws servicediscovery delete-namespace \ 
      --id ns-uejictsjen2i4eeg

Using the OperationId from the output of the previous step, verify that the service discovery namespace was deleted successfully.

aws servicediscovery get-operation \ 
      --operation-id c3ncqglftesw4ibgj5baz6ktaoh6cg4t-jh0ztysj

Update the desired count for the Amazon ECS service to 0. You must do this to delete the service in the next step.

aws ecs update-service \
      --cluster tutorial \
      --service ecs-service-discovery \
      --desired-count 0

Delete the Amazon ECS service.

aws ecs delete-service \
      --cluster tutorial \
      --service ecs-service-discovery

Delete the Amazon ECS cluster.

aws ecs delete-cluster \
      --cluster tutorial

Key Benefits vs Alternatives

Practice Question: What is required for ECS Service Discovery?
Answer: Cloud Map namespace/service + awsvpc networking; auto-registers task IPs as DNS records.

Service discovery pricing

Customers using Amazon ECS service discovery are charged for Route 53 resources and AWS Cloud Map discovery API operations. This involves costs for creating the Route 53 hosted zones and queries to the service registry. For more information, see AWS Cloud Map Pricing
Amazon ECS performs container level health checks and exposes them to AWS Cloud Map custom health check API operations. This is currently made available to customers at no extra cost. If you configure additional network health checks for publicly exposed tasks, you're charged for those health checks.

Flow Diagram

Reference:

Amazon EMR on EKS now supports Service Quotas

Learn2Skills — Mon, 30 Jun 2025 09:55:49 +0000

Amazon EMR on EKS now supports Service Quotas, allowing users to manage and request increases for quotas like StartJobRun API calls directly in the AWS Service Quotas console.

Key Benefits
This eliminates the need for support tickets in many cases, enabling automated approvals for eligible requests and faster scaling. Users can also set CloudWatch alarms to monitor usage against quotas, improving operational efficiency.

Previously, to request an increase for EMR on EKS quotas, such as maximum number of StartJobRun API calls per second, customers had to open a support ticket and wait for the support team to process the increase. Now, customers can view and manage their EMR on EKS quota limits directly in the Service Quotas console. This enables automated limit increase approvals for eligible requests, improving response times and reducing the number of support tickets. Customers can also set up Amazon CloudWatch alarms to get automatically notified when their usage reaches a certain percentage of a maximum quota.

How It Works
Access the Service Quotas console via the AWS Management Console to view EMR on EKS limits by region. Requests for increases are processed centrally, with some approved automatically to reduce wait times.

With Service Quotas, you can view and manage your quotas for AWS services from a central location. Quotas, also referred to as limits in AWS services, are the maximum values for the resources, actions, and items in your AWS account. Each AWS service defines its quotas and establishes default values for those quotas. If your business needs aren't met by the default limit of service resources or operations that apply to an AWS account, resource, or an AWS Region, you might need to increase your service quota values. Service Quotas enables you to look up your service quotas and to request increases. Support might approve, deny, or partially approve your requests.

AWS Management Console
The Service Quotas console is a browser-based interface that you can use to view and manage your service quotas. You can perform almost any task that's related to your service quotas by using the console. You can access Service Quotas from any AWS Management Console page by choosing it on the top navigation bar, or by searching for Service Quotas in the AWS Management Console.

Using the AWS Management Console to request an increase
To request a service quota increase

Sign in to the AWS Management Console and open the Service Quotas console at https://console.aws.amazon.com/servicequotas/home.
In the navigation pane, choose AWS services.
Choose an AWS service from the list, or enter the name of the service in the search box.
If the quota is adjustable, you can request a quota increase at either the account-level or resource-level based on the value listed in the Adjustability column.

Account-level – Request a quota increase at the account-level for an account-level quota such as Domains per Region for Amazon OpenSearch Service. To do so, select the quota from the list and choose Request increase at account-level.
Resource-level – Request a quota increase for a specific resource for a resource-level quota such as Instances per domain for Amazon OpenSearch Service. To do so, choose the quota name to view additional information about the quota. Under the Resource-level quotas section, select the resource for which you want to increase the quota value, and choose the Request increase at resource-level button.

For Increase quota value, enter the new value. The new value must be greater than the current value.
Choose Request.
To view any pending or recently resolved requests in the console, navigate to the Request history tab from the service's details page, or choose Dashboard from the navigation pane. For pending requests, choose the status of the request to open the request receipt. The initial status of a request is Pending. After the status changes to Quota requested, you'll see the case number with Support. Choose the case number to open the ticket for your request.

Service Flow Diagram

This diagram illustrates the streamlined process: workloads interact with quotas checked via the console, with monitoring and requests handled centrally for EMR on EKS resources like API rates and cluster limits.

Reference:

Amazon Redshift now supports refresh interval in a zero-ETL integration

Learn2Skills — Wed, 20 Nov 2024 11:06:14 +0000

This set of tasks walks you through setting up your first zero-ETL integration. First, you configure your integration source and set it up with the required parameters and permissions. Then, you continue to the rest of the initial setup from the Amazon Redshift console or AWS CLI.

Topics

Create and configure a target Amazon Redshift data warehouse
Turn on case sensitivity for your data warehouse
Configure authorization for your Amazon Redshift data warehouse
Create a zero-ETL integration

Creating destination databases in Amazon Redshift
To replicate data from your source into Amazon Redshift, you must create a database from your integration in Amazon Redshift.

Connect to your target Redshift Serverless workgroup or provisioned cluster and create a database with a reference to your integration identifier. This identifier is the value returned for integration_id when you query the SVV_INTEGRATION view.

Note- Before creating a database from your integration, your zero-ETL integration must be created and in the Active state on the Amazon Redshift console.

Before you can start replicating data from your source into Amazon Redshift, create a database from the integration in Amazon Redshift. You can either create the database using the Amazon Redshift console or the query editor v2.

In the left navigation pane, choose Zero-ETL integrations.
From the integration list, choose an integration.
If you're using a provisioned cluster, you must first connect to the database. Choose Connect to database. You can connect using a recent connection, or by creating a new connection.
To create a database from the integration, choose Create database from integration.
Enter a Destination database name. The Integration ID and Data warehouse name are pre-populated. For Aurora PostgreSQL sources, enter the Source named database that you specified when creating your zero-ETL integration. You can map a maximum of 100 Aurora PostgreSQL databases to Amazon Redshift databases.
Choose Create database.

Ref: Getting started with zero-ETL integrations

Amazon DynamoDB adds support for attribute-based access control

Learn2Skills — Mon, 30 Sep 2024 06:34:24 +0000

Attribute-based access control (ABAC) is an authorization technique that allows you to define fine-grained permissions based on user factors like department, job title, and team name. User attributes make permissions more intuitive and simplify the administrative process of managing access. By specifying permissions with attributes, you can reduce the number of separate permissions required to create fine-grained controls in your AWS account.

Attribute-Based Access Control for Amazon DynamoDB is now available in limited preview in the US East (Ohio), US East (Virginia), and US West (N. California) Regions. To request access to the limited preview, visit the preview page.

Use cases

Grant developers and workloads read and write access to only their project resources.

Solution: When you base permissions on user attributes, you can ensure that developers and workloads only have read and write access to resources related to their projects. If the attributes of developers or workloads match those of project resources, they are granted access. Otherwise, they are rejected. For example, you can assign two developers from different teams, Alejandro and Mary, to the same IAM role and then use the team name property to manage access. When Alejandro and Mary check in to AWS, their identity provider (IdP) transmits their team name as an attribute in the AWS session, and they are only permitted access to their team's project resources, as indicated by the tags on those resources.

As you create new resource permission and new secrets and application have automatically access, to all secrets tag and product tag.

This are the tag Governance, Tags are for access control

Configure AWS tags and keys

Create secrets that are tagged with the project tag

Ref: attribute-based access control

Assign an IAM role to a Kubernetes service account

Learn2Skills — Sat, 20 Jul 2024 16:58:08 +0000

How to set up a Kubernetes service account to take an AWS Identity and Access Management (IAM) role using EKS Pod Identity. Any Pods that are set up to use the service account can then access any AWS service that the role has permission to access.

An EKS Pod Identity association can be created in a single step using the AWS Management Console, AWS CLI, AWS SDKs, AWS CloudFormation, and other technologies. There is no data or information about the cluster's affiliations in any Kubernetes objects, and you do not annotate the service accounts.

Prerequisites:

An existing cluster.
The IAM principal that is creating the association must have iam:PassRole.
The latest version of the AWS CLI installed and configured on your device or AWS CloudShell. You can check your current version with aws --version | cut -d / -f2 | cut -d ' ' -f1. Package managers such yum, apt-get, or Homebrew for macOS are often several versions behind the latest version of the AWS CLI.
The kubectl command line tool is installed on your device or AWS CloudShell. The version can be the same as or up to one minor version earlier or later than the Kubernetes version of your cluster. For example, if your cluster version is 1.29, you can use kubectl version 1.28, 1.29, or 1.30 with it.
An existing kubectl config file that contains your cluster configuration. To create a kubectl config file

Creating the EKS Pod Identity association

Open the Amazon EKS console at https://console.aws.amazon.com/eks/home#/clusters.
In the left navigation pane, select Clusters, and then select the name of the cluster that you want to configure the EKS Pod Identity Agent add-on for.
Choose the Access tab.
In the Pod Identity associations, choose Create.
For the IAM role, select the IAM role with the permissions that you want the workload to have.

Note
The list only contains roles that have the following trust policy which allows EKS Pod Identity to use them.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
            "Effect": "Allow",
            "Principal": {
                "Service": "pods.eks.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
    ]
}

sts:AssumeRole
EKS Pod Identity uses AssumeRole to assume the IAM role before passing the temporary credentials to your pods.

sts:TagSession
EKS Pod Identity uses TagSession to include session tags in the requests to AWS STS.

You can use these tags in the condition keys in the trust policy to restrict which service accounts, namespaces, and clusters can use this role.

For a list of Amazon EKS condition keys, see Conditions defined by Amazon Elastic Kubernetes Service in the Service Authorization Reference. To learn which actions and resources you can use a condition key with, see Actions defined by Amazon Elastic Kubernetes Service.

For the Kubernetes namespace, select the Kubernetes namespace that contains the service account and workload. Optionally, you can specify a namespace by name that doesn't exist in the cluster.
For the Kubernetes service account, select the Kubernetes service account to use. The manifest for your Kubernetes workload must specify this service account. Optionally, you can specify a service account by name that doesn't exist in the cluster.
(Optional) For the Tags, choose Add tag to add metadata in a key and value pair. These tags are applied to the association and can be used in IAM policies.

You can repeat this step to add multiple tags.

Choose Create.

Confirm configuration
Confirm that the role and service account are configured correctly.

Confirm that the IAM role's trust policy is configured correctly. aws iam get-role --role-name my-role --query Role.AssumeRolePolicyDocument
Confirm that the policy that you attached to your role in a previous step is attached to the role. aws iam list-attached-role-policies --role-name my-role --query AttachedPolicies[].PolicyArn --output text
View the default version of the policy. aws iam get-policy --policy-arn $policy_arn ---

for more details: IAM role to a Kubernetes service account

Amazon Aurora PostgreSQL now supports RDS Data API

Learn2Skills — Sun, 31 Dec 2023 15:21:35 +0000

By using RDS Data API (Data API), you can work with a web-services interface to your Aurora DB cluster. Data API doesn't require a persistent connection to the DB cluster. Instead, it provides a secure HTTP endpoint and integration with AWS SDKs. You can use the endpoint to run SQL statements without managing connections.

You can enable Data API when you create the Aurora DB cluster. You can also modify the configuration later. For more information, see Enabling RDS Data API.

Note- Currently, for Aurora MySQL, Data API and query editor aren't supported for Aurora Serverless v2 or for provisioned DB clusters.

Region and version availability
RDS Data API is available for the following types of Aurora DB clusters:

Aurora PostgreSQL using specific PostgreSQL versions. These clusters can use Aurora Serverless v2 instances, provisioned instances, or a combination of both.
Aurora Serverless v1 clusters using either Aurora PostgreSQL or Aurora MySQL.

Limitations with RDS Data API

RDS Data API (Data API) has the following limitations:
You can only execute Data API queries on writer instances in a DB cluster.
With Aurora global databases, you can enable Data API on both primary and secondary DB clusters. However, until a secondary cluster is promoted to be the primary, it has no writer instance. Thus, Data API queries that you send to the secondary fail. After a promoted secondary has an available writer instance, Data API queries on that DB instance should succeed.
Performance Insights doesn't support monitoring database queries that you make using Data API.
Data API isn't supported on T DB instance classes.
For Aurora PostgreSQL Serverless v2 and provisioned DB clusters, RDS Data API doesn't support enumerated types.

Enabling RDS Data API when you create a database

While you are creating a database that supports RDS Data API (Data API), you can enable this feature. The following procedures describe how to do so when you use the AWS Management Console, the AWS CLI, or the RDS API.

Enabling or disabling Data API (Aurora PostgreSQL Serverless v2 and provisioned)

Use the following procedures to enable or disable Data API on Aurora PostgreSQL Serverless v2 and provisioned databases. To enable or disable Data API on Aurora Serverless v1 databases, use the procedures in Enabling or disabling Data API (Aurora Serverless v1 only).

more details RDS Data API for Aurora Serverless

Amazon S3 Express One Zone

Learn2Skills — Thu, 30 Nov 2023 15:45:17 +0000

S3 Express One Zone can improve data access speeds by 10x and reduce request costs by 50% compared to S3 Standard and scales to process millions of requests per minute for your most frequently accessed datasets.

S3 Express One Zone is ideal for any application where it's important to minimize the latency required to access an object. This can be human-interactive workflows, like video editing, where creative professionals need responsive access to content from their user interfaces. S3 Express One Zone also benefits analytics and machine learning workloads that have similar responsiveness requirements from their data, especially workloads with lots of smaller accesses or large numbers of random accesses. S3 Express One Zone can be used with other AWS services to support analytics and AI/ML workloads, such as Amazon EMR, Amazon SageMaker, and Amazon Athena.

When using S3 Express One Zone, you can interact with your directory bucket in an AWS virtual private cloud (VPC) by using a gateway VPC endpoint. With a gateway endpoint, you can access S3 Express One Zone directory buckets from your VPC without an internet gateway or NAT device for your VPC and at no additional cost.

You can use many of the same S3 APIs and features with directory buckets that you use with general purpose buckets and other storage classes. These include Mountpoint for Amazon S3, server-side encryption with Amazon S3 managed keys (SSE-S3), S3 Batch Operations, and S3 Block Public Access. You can access S3 Express One Zone by using the AWS Management Console, AWS Command Line Interface (AWS CLI), or AWS SDKs.

Overview

To optimize performance and reduce latency, S3 Express One Zone introduces the following new concepts.

Single Availability Zone
The Amazon S3 Express One Zone storage class is designed for 99.95% availability within a single Availability Zone and is backed by the Amazon S3 Service Level Agreement. With S3 Express One Zone, your data is redundantly stored on multiple devices within a single Availability Zone. S3 Express One Zone is designed to handle concurrent device failures by quickly detecting and repairing any lost redundancy. If the existing device encounters a failure, S3 Express One Zone automatically shifts requests to new devices within an Availability Zone. This redundancy helps ensure uninterrupted access to your data within an Availability Zone.

Directory buckets
There are two types of Amazon S3 buckets, S3 general purpose buckets and S3 directory buckets. Directory buckets use only the S3 Express One Zone storage class, which is designed for workloads or performance-critical applications that require consistent single-digit millisecond latency. General purpose buckets are the default Amazon S3 bucket type that is used for the vast majority of S3 use cases. You should choose the bucket type that best fits your application and performance requirements.

Directory buckets organize data hierarchically into directories as opposed to the flat storing structure of general purpose buckets. There aren’t prefix limits for directory buckets and individual directories can scale horizontally.

Endpoints and gateway VPC endpoints
Bucket-management API operations are available through a Regional endpoint and are referred to as Regional endpoint APIs. Examples of Regional endpoint APIs are CreateBucket and DeleteBucket. After you create a directory bucket, you can use Zonal endpoint APIs to upload and manage the objects in your directory bucket. Zonal endpoint APIs are available through a Zonal endpoint. Examples of Zonal endpoint APIs are PutObject and CopyObject.

Session-based authorization
With S3 Express One Zone, you authenticate and authorize requests through a new session-based mechanism, which is optimized to provide the lowest latency. You can use CreateSession to request temporary credentials that provide low latency access to your bucket. These temporary credentials are scoped to a specific S3 directory bucket. Session tokens are used only with Zonal (object-level) operations (with the exception of CopyObject) and are optimized to provide the lowest latency. For more information, see Create session.

Features of S3 Express One Zone
The following S3 features are available for S3 Express One Zone. For a complete list of supported APIs and unsupported features, see How is S3 Express One Zone different?.

Access management and security
With directory buckets, you can use the following features to audit and manage access. By default, directory buckets are private and can be accessed only by users who are explicitly granted access. Unlike general purpose buckets, which can set the access control boundary at the bucket, prefix, or object tag level, the access control boundary for directory buckets is set only at the bucket level. For more information, see AWS Identity and Access Management (IAM) for S3 Express One Zone.

S3 Block Public Access – All S3 Block Public access settings are enabled by default at the bucket level. This default setting can't be modified.
S3 Object Ownership (Bucket owner enforced by default) – Access control lists (ACLs) are not supported for directory buckets. Directory buckets automatically use the bucket owner enforced setting for S3 Object Ownership, which means that ACLs are disabled and the bucket owner automatically owns and has full control over every object in the bucket. This default setting can’t be modified.
AWS Identity and Access Management (IAM) – IAM helps you securely control access to your directory buckets. You can use IAM to grant access to bucket management (Regional) actions and object management (Zonal) APIs through the CreateSession action. For more information, see AWS Identity and Access Management (IAM) for S3 Express One Zone. Unlike object-management actions, bucket management actions cannot be cross-account. Only the bucket owner can perform those actions.
Bucket policies – Use IAM-based policy language to configure resource-based permissions for your directory buckets. You can also use IAM to control access to the CreateSession API which allows you to use the Zonal or object management APIs. You can grant same-account or cross-account access. For more information on S3 Express One Zone permissions and policies, see AWS Identity and Access Management (IAM) for S3 Express One Zone.
IAM Access Analyzer for S3 – Evaluate and monitor your access policies, ensuring that the policies provide only the intended access to your S3 resources.

Logging and monitoring
S3 Express One Zone uses S3 logging and monitoring tools that you can use to monitor and control how your resources are being used.

Amazon CloudWatch metrics – Monitor your AWS resources and applications using CloudWatch to collect and track metrics. S3 Express One Zone uses the same CloudWatch namespace as other Amazon S3 storage classes (AWS/S3) and supports daily storage metrics for directory buckets: BucketSizeBytes and NumberOfObjects. For more information, see Monitoring metrics with Amazon CloudWatch.
AWS CloudTrail logs – AWS CloudTrail is an AWS service that helps you enable operational and risk auditing, governance, and compliance of your AWS account by recording actions taken by a user, role, or an AWS service. For S3 Express One Zone, CloudTrail captures regional endpoint APIs (for example, CreateBucket, PutBucketPolicy) as management events. This includes actions taken in the AWS Management Console, AWS CLI, AWS SDKs, and APIs. The eventsource for CloudTrail management events for S3 Express One Zone is s3express.amazonaws.com. For more information, see Amazon S3 CloudTrail events.

Object management
After you create a directory bucket, you can manage your object storage using the S3 console, AWS SDKs, and AWS CLI. The following features are available for object management with S3 Express One Zone.

S3 Batch Operations – Use Batch Operations to perform bulk operations on objects in directory buckets, for example, Copy and Invoke AWS Lambda function. For example, you can use Batch Operations to copy objects between directory buckets and general purpose buckets. With Batch Operations, you can manage billions of objects at scale with a single S3 request using the AWS SDKs or AWS CLI or a few clicks in the Amazon S3 console.
Import – After you create a directory bucket, you can populate your bucket with objects by using the import feature in the Amazon S3 console. Import is a streamlined method for creating Batch Operations jobs to copy objects from general purpose buckets to directory buckets.

EKS Pod Identities

Learn2Skills — Thu, 30 Nov 2023 15:39:30 +0000

Applications in a Pod's containers can use the AWS SDK or the AWS CLI to make API requests to AWS services using AWS Identity and Access Management (IAM) permissions. Applications must sign their AWS API requests with AWS credentials.

EKS Pod Identities provide the ability to manage credentials for your applications, similar to the way that Amazon EC2 instance profiles provide credentials to Amazon EC2 instances. Instead of creating and distributing your AWS credentials to the containers or using the Amazon EC2 instance's role, you associate an IAM role with a Kubernetes service account and configure your Pods to use the service account.

Each EKS Pod Identity association maps a role to a service account in a namespace in the specified cluster. If you have the same application in multiple clusters, you can make identical associations in each cluster without modifying the trust policy of the role.

Benefits of EKS Pod Identities

EKS Pod Identities provide the following benefits:

Least privilege – You can scope IAM permissions to a service account, and only Pods that use that service account have access to those permissions. This feature also eliminates the need for third-party solutions such as kiam or kube2iam.

Credential isolation – A Pod's containers can only retrieve credentials for the IAM role that's associated with the service account that the container uses. A container never has access to credentials that are used by other containers in other Pods. When using Pod Identities, the Pod's containers also have the permissions assigned to the Amazon EKS node IAM role, unless you block Pod access to the Amazon EC2 Instance Metadata Service (IMDS). For more information, see Restrict access to the instance profile assigned to the worker node.

Auditability – Access and event logging is available through AWS CloudTrail to help facilitate retrospective auditing.

EKS Pod Identity is a simpler method than IAM roles for service accounts, as this method doesn't use OIDC identity providers. EKS Pod Identity has the following enhancements:

Independent operations – In many organizations, creating OIDC identity providers is a responsibility of different teams than administering the Kubernetes clusters. EKS Pod Identity has clean separation of duties, where all configuration of EKS Pod Identity associations is done in Amazon EKS and all configuration of the IAM permissions is done in IAM.

Reusability – EKS Pod Identity uses a single IAM principal instead of the separate principals for each cluster that IAM roles for service accounts use. Your IAM administrator adds the following principal to the trust policy of any role to make it usable by EKS Pod Identities.

        "Principal": {
            "Service": "pods.eks.amazonaws.com"
        }

Scalability – Each set of temporary credentials are assumed by the EKS Auth service in EKS Pod Identity, instead of each AWS SDK that you run in each pod. Then, the Amazon EKS Pod Identity Agent that runs on each node issues the credentials to the SDKs. Thus the load is reduced to once for each node and isn't duplicated in each pod. For more details of the process, see How EKS Pod Identity works.

Overview of setting up EKS Pod Identities

Turn on EKS Pod Identities by completing the following procedures:

Setting up the Amazon EKS Pod Identity Agent – You only complete this procedure once for each cluster.
Configuring a Kubernetes service account to assume an IAM role with EKS Pod Identity – Complete this procedure for each unique set of permissions that you want an application to have.
Configuring Pods to use a Kubernetes service account – Complete this procedure for each Pod that needs access to AWS services.
Using a supported AWS SDK – Confirm that the workload uses an AWS SDK of a supported version and that the workload uses the default credential chain.

EKS Pod Identity cluster versions
To use EKS Pod Identities, the cluster must have a platform version that is the same or later than the version listed in the following table, or a Kubernetes version that is later than the versions listed in the table.

Kubernetes version Platform version
1.28 eks.4
1.27 eks.8
1.26 eks.9
1.25 eks.10
1.24 eks.13
EKS Pod Identity restrictions
EKS Pod Identities are available on the following:

Amazon EKS cluster versions listed in the previous topic EKS Pod Identity cluster versions.
Worker nodes in the cluster that are Linux Amazon EC2 instances.

EKS Pod Identities aren't available on the following:

China Regions.
AWS GovCloud (US).
AWS Outposts.
Amazon EKS Anywhere.
Kubernetes clusters that you create and run on Amazon EC2. The EKS Pod Identity components are only available on Amazon EKS.

You can't use EKS Pod Identities with:

Pods that run anywhere except Linux Amazon EC2 instances. Linux and Windows pods that run on AWS Fargate (Fargate) aren't supported. Pods that run on Windows Amazon EC2 instances aren't supported.
Amazon EKS add-ons that need IAM credentials. The EKS add-ons can only use IAM roles for service accounts instead. The list of EKS add-ons that use IAM credentials include:
Amazon VPC CNI plugin for Kubernetes
AWS Load Balancer Controller
The CSI storage drivers: EBS CSI, EFS CSI, Amazon FSx for Lustre CSI driver, Amazon FSx for NetApp ONTAP CSI driver, Amazon FSx for OpenZFS CSI driver, Amazon File Cache CSI driver

EKS Fargate supports additional Ephemeral Storage

Learn2Skills — Sun, 13 Aug 2023 06:05:45 +0000

Customers can now specify the size (in GiB) of ephemeral storage that their workloads require using the storage parameter in their pod spec. 20 GiB of ephemeral storage is included with every EKS Fargate pod. Additional ephemeral storage requested, up to 175GB, is charged in GB increments for the duration that the pod is running. See the Fargate pricing page for more details. Customers with data intensive workloads, like machine learning inference, data processing, or with large container images, can now use AWS Fargate with Amazon EKS to reduce their operational burden, pay only for the resources used by their applications, and get the security benefits of AWS Fargate’s built-in workload isolation.

Fargate storage
A Pod running on Fargate automatically mounts an Amazon EFS file system. You can't use dynamic persistent volume provisioning with Fargate nodes, but you can use static provisioning. For more information, see Amazon EFS CSI Driver on GitHub.

When provisioned, each Pod running on Fargate receives a default 20 GiB of ephemeral storage. This type of storage is deleted after a Pod stops. New Pods launched onto Fargate have encryption of the ephemeral storage volume enabled by default. The ephemeral Pod storage is encrypted with an AES-256 encryption algorithm using AWS Fargate managed keys.

Note- The default usable storage for Amazon EKS Pods that run on Fargate is less than 20 GiB. This is because some space is used by the kubelet and other Kubernetes modules that are loaded inside the Pod.

You can increase the total amount of ephemeral storage up to a maximum of 175 GiB. To configure the size with Kubernetes, specify the requests of ephemeral-storage resource to each container in a Pod. When Kubernetes schedules Pods, it ensures that the sum of the resource requests for each Pod is less than the capacity of the Fargate task. For more information, see Resource Management for Pods and Containers in the Kubernetes documentation.

Amazon EKS Fargate provisions more ephemeral storage than requested for the purposes of system use. For example, a request of 100 GiB will provision a Fargate task with 115 GiB ephemeral storage.