Forem: subbaramireddy

CloudFront (CDN) for AWS S3

subbaramireddy — Tue, 23 Jan 2024 03:38:06 +0000

CloudFront

Amazon CloudFront is a web service that speeds up the distribution of your static and dynamic web content, such as .html, .css, .js, and image files, to your users. CloudFront delivers your content through a worldwide network of data centres called edge locations. When a user requests content that you’re serving with CloudFront, the request is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance. If the content is already in the edge location with the lowest latency, CloudFront delivers it immediately. If the content is not in that edge location, CloudFront retrieves it from an origin that you’ve defined — such as an Amazon S3 bucket, a MediaPackage channel, or an HTTP server (for example, a web server) that you have identified as the source for the definitive version of your content.

CloudFront speeds up the distribution of your content by routing each user request through the AWS backbone network to the edge location that can best serve your content. Typically, this is a CloudFront edge server that provides the fastest delivery to the viewer. Using the AWS network dramatically reduces the number of networks that your users’ requests must pass through, which improves performance. Users get lower latency — the time it takes to load the first byte of the file — and higher data transfer rates.

Prerequisites

hosting-portfolio-on-aws-s3

Architecture

Steps

Navigate to the AWS Management Console, type ‘CloudFront’ in the search bar, and select it.

Select ‘Create a CloudFront distribution’.

Choose the S3 bucket endpoint created for the portfolio as the origin domain and select use website endpoint.

Opt for the ‘C*aching Optimized*’ cache policy in the default cache behaviour settings.

Within the Web Application Firewall (WAF), choose ‘Do Not Enable Security Protections’.

In the settings, keep everything at default and proceed by clicking ‘Create Distribution’.

You can check the created CloudFront, copy the Distribution domain name, and paste it into the browser to access the portfolio.

When accessing the portfolio directly from the S3 bucket, the load time is observed to be 108 ms.

Accessing the portfolio through CloudFront resulted in a quicker load time, taking only 42ms.

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with fellow tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on tech, programming, and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

Hosting Portfolio on AWS S3

subbaramireddy — Sat, 25 Nov 2023 14:16:18 +0000

Simple Storage Service (S3)

Amazon Simple Storage Service (AWS S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. Customers of all sizes and industries can use Amazon S3 to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements.

Steps

Navigate to the AWS Management Console, locate the search bar, type ‘S3,’ and click on it.

Inside the S3 Dashboard, you’ll find a list of buckets. Ready to add a new one? Just hit ‘Create Bucket’.

Ensure a distinctive bucket name for successful creation. Don’t forget to choose the region as ap-south-1 (Mumbai).

Enable ACLs for added security, and in the Object Ownership section, choose ‘Object Writer’ for the desired control.

Ensure public access for the bucket by unchecking the necessary settings.

Enable Bucket Versioning to keep track of various versions of objects within the bucket.

Add tags for organization. Specify ‘Name’ as the key and ‘Portfolio’ as the value.

Opt for server-side encryption with Amazon S3-managed keys and enable the bucket key. Once set, click ‘Create Bucket’ to finalize.

Explore the list of objects and upload new ones by clicking on the ‘Upload’ button.

To upload to a specific folder, click ‘Add Folder’.

Choose the ‘Portfolio’ folder and hit ‘Upload’. Your files are ready for download here.

It will load all the files into the bucket.

Check the destination path and click on ‘Permissions’ to define access permissions for the objects.

In the Access Control List (ACL), choose from predefined ACLs and grant public-read access.

Once you’ve set the permissions for the objects, click ‘Upload’ to complete the process.

You can check the status of the uploaded files in the bucket.

You’ll find the list of uploaded objects here.

Choose the object and click on ‘Properties’.

Look for ‘Static Website Hosting’ and click on ‘Edit’.

Enable static website hosting, select ‘Host a static website’ as the hosting type, and specify ‘index.html’ as the index document. For error handling, set ‘error.html’ as the Error document.

Find the static website hosting section with the bucket website endpoint and click on it.

Append the endpoint with /Portfolio/. It will automatically redirect to the index.html page.

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with fellow tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on tech, programming, and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

Kubiya AI and AWS Integration

subbaramireddy — Thu, 23 Nov 2023 16:03:12 +0000

Integrating Kubiya AI with AWS brings a new level of automation and intelligence to your cloud computing endeavours. AWS, as a cloud platform, empowers users to access computing resources, storage, and services on-demand, fostering scalability and flexibility.

By partnering with Kubiya AI, you introduce an intelligent assistant into this dynamic environment. Kubiya AI navigates AWS seamlessly, automating routine tasks, and enhancing the user experience. Whether you’re exploring the vast capabilities of Amazon EC2 for computing power, leveraging Amazon S3 for storage solutions, or managing databases with Amazon RDS, Kubiya AI becomes your automated guide.

This integration isn’t just about convenience — it’s a synergy of cutting-edge technology that allows you to focus on your core objectives while Kubiya AI takes care of the intricate details. It’s a fusion of AWS’s robust cloud infrastructure with Kubiya AI’s automation prowess, providing you with an efficient, user-friendly, and intelligent cloud computing experience.

prerequisites

Required a Slack account
Create a channel in Slack to integrate with the Kubiya.

Install the Slack App

Installing Kubiya Slack app

The best way to get started with Kubiya is using the direct integration within Slack which ensures a streamlined and enhanced user experience, enabling you and your team to interface with the virtual assistant without requiring platform switching.

Don’t worry — after installing the Slack app you’ll still be able to access the management interface.

Features of the Slack integration

Immediacy: Engage with Kubiya directly within your established communication channels on Slack and manage all kinds of cloud resources with a blink of an eye.
**Efficiency: **All native integrations can be easily connected right from Slack.
**Collaboration: **Enhance collective decision-making by sharing and debating Kubiya’s insights within your team’s Slack channels.

Installing the Slack App

Add the Kubiya app to your Slack workspace:

Clicking on the Add to Slack button above will prompt you to approve Kubiya’s access to your workspace:

Important: Make sure the correct workspace is selected on the top right.

After adding to Slack, you should be redirected to this page, indicating that the onboarding process succeeded:

You should see the newly installed Kubiya app in your Slack workspace — with a message waiting for you to enable your first integrations:

Add connection of Amazon Web Services (AWS)

Prerequisites

Before you begin, ensure that you have the following:

Access to the Kubiya platform.
Appropriate permissions to make configurations on AWS (IAM)

Create a Role on AWS

Create a role on AWS for Kubiya to perform actions on AWS:

Login to your AWS console and navigate to the Identity and Access Management (IAM) page.
Click on the Roles section
Click the Create Role button in the top right corner
In the “Select trusted entity” dialog:
Select “AWS account”
Select “Another AWS account”
Enter the Kubiya Account ID in the identifier input box 564407622114
Click Next

In the ‘Policies’ dialog, select the access rules that you would like Kubiya to have access to, then click Next

In the “Role details” dialog:
Name the role, you will need this later.
Confirm the Kubiya Account ID in the JSON under “Step 1: Select trusted entities”
Confirm the selected permissions under “Step 2: Add permissions”
If correct, click the Create role button

Enable the integration

open the channel which is created and mention the Kubiya with a greeting. It will invite the Kubiya app to the channel and prompt you to activate the Kubiya account.

It will redirect you to the Kubiya login page and click on Continue with Slack.

Accept and Continue the sign-in to Kubiya with Slack. It will send an email for authentication.

Enter the One-time password to authenticate with the Kubiya

It will prompt two options

Chat on Slack
Kubiya Web App, Click on Kubiya Web App

It will open the Kubiya Welcome page and click on the Start.

This is the Kubiya Initial dashboard to view history.

Once you’ve activated the Kubiya account, open Slack and navigate to the integration setup. Click on “Connect AWS” and a window will appear prompting you to enter the AWS credentials.

Input the credentials to establish a connection with the AWS account.

A prompt will appear indicating the connection process between Kubiya and the AWS account.

A window will appear, indicating that the connection to AWS has been successfully established.

In the Kubiya UI, We can view the Connections which are configured.

Navigate to the Apps section of Slack to see the integrated apps within the Slack account.

Open the Channel which is created for your team. Mention Kubi to list the S3 buckets within the AWS Account.

We can view the list of s3 buckets in the AWS management console.

AI Suggestions are enabled for AWS by default

Mention Kubi to List all Ec2 Instances.

Mention Kubi to List all the Ec2 instances in the ap-south-1 region. Kubiya will give you the reply in the Thread.

Mention Kubi to Create an Ec2 Instance. It will prompt you to enter the required options to create the instance.

Enter the required values to create the Ec2 instance.

In the AWS Management Console, search for the EC2. Click on it and we can view the running instances

Mention Kubi to list the running Ec2 instance in the ap-south-1 region. Kubi will give you a reply in the thread. we can verify the EC2 Instance with the Instance ID.

Mention Kubi to terminate the EC2 instance. It will prompt you to enter the Instance ID.

Mention Kubi with the Instance ID to terminate.

In the Thread window kubi will reply as the operation completed successfully.

No instances are running in the ap-south-1 region.

Open the Kubiya Web UI dashboard.

We can view the Actions history in the Kubiya web UI.

In the Users & Groups, we can view the Slack user name.

In summary, the integration of Kubiya AI with AWS is a game-changer in cloud computing. AWS’s extensive capabilities blend effortlessly with Kubiya AI’s automation, offering users a seamless, intelligent, and personalized experience. It’s more than a partnership; it’s a revolution in how we leverage the cloud, making complex tasks simple and unlocking the full potential of AWS with ease.

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on cloud computing, tech and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

ChatGPT for DevOps — Kubiya

subbaramireddy — Sun, 19 Nov 2023 13:53:50 +0000

In the dynamic realm of tech, Artificial Intelligence (AI) plays a pivotal role in DevOps. It’s the powerhouse that turbocharges efficiency, agility, and continuous improvement for DevOps engineers.

AI’s automation eliminates repetitive tasks, freeing up time for strategic endeavours. Its predictive analytics foresee issues, preventing disruptions and enabling smarter decision-making. Moreover, AI fosters intelligent collaboration by offering insights and enhancing communication.

In the ever-evolving landscape of technology, AI isn’t just a luxury; it’s a necessity for DevOps engineers navigating the fast-paced world of software development.

Let me introduce DevOps AI — Kubiya

Kubiya stands as an internal developer platform driven by conversational AI, tapping into the power of Generative AI and Large Language Models (LLMs) to deliver a user-friendly, self-serve interface. With Kubiya, engineering teams effortlessly consume DevOps services by engaging in conversations with their dedicated DevOps assistant, Kubi. This platform empowers engineers to seamlessly access resources from a myriad of popular engineering tools. Whether it’s running CI jobs, listing deployments, managing Kubernetes namespaces, handling Jira tickets, or updating configuration files in Github, Gitlab, or Bitbucket, Kubiya is the go-to assistant for a wide array of tasks.

Understanding Kubiya AI: 🎭 Kubiya AI isn’t your typical DevOps player — it’s a next-gen maestro. Instead of just orchestrating processes, it conducts purposeful movements that align with the core principles of efficiency, collaboration, and continuous improvement. As we explore Kubiya AI, we not only understand how it works but also discover the reasons behind its outstanding performance in the world of DevOps.

Key Features of Kubiya AI:

Smart Automation with Intent 🤖: Kubiya AI, a mindful automator, uses machine learning for purposeful automation — liberating teams strategically.
Predictive Analytics for Proactive Operations 🔮: Kubiya AI foresees issues purposefully, aligning predictive analytics for proactive DevOps, minimizing downtime, and optimizing resources.
Intelligent Collaboration with Meaning 💬: Orchestrating meaningful communication, Kubiya AI automates incidents for efficient teamwork, ensuring alerts reach the right stakeholders purposefully.
*Continuous Improvement Rooted in Data-Driven Insights *📊: Kubiya AI provides purposeful data-driven insights, contributing to a continuous improvement ballet within development and operations teams.

The True Future of AI in DevOps:

*Efficiency Redefined with Strategic Intent *💡: Kubiya AI is a beacon of efficiency in DevOps, where every action, from automating tasks to predicting issues, is driven by strategic intent — to reshape team operations and unlock unprecedented productivity.
*Human-AI Synergy Anchored in Purpose *🤝: The collaboration between human expertise and Kubiya AI is intentional, not arbitrary. It aims to provide intelligent support, enhancing decision-making with a clear purpose — to drive innovation and efficiency.
*Agile and Adaptive for Strategic Advantage *🔄: Kubiya AI’s agility and adaptability go beyond technical prowess; they are strategic advantages. In the ever-changing landscape, Kubiya AI purposefully learns, adapts, and contributes to the strategic agility of DevOps processes.

Components of Kubiya

Connectors

Kubiya offers native integration with a variety of common cloud providers and solutions. For users who require a customized solution or want to extend functionality, the platform also supports integration with any RestAPI or platform through custom connections.

Connections are a pivotal part of Kubiya workflows, allowing developers to establish a centralized repository of actions that can be reused across various workflows. They offer a uniform method to access and manage actions, hence enhancing modularity and promoting code reuse.

What are connectors?

Connections are collections of actions that can be executed during a chatbot conversation. Actions are concise, independent units of code designed to accomplish a particular task and deliver a result. They can be summoned using the action type within a workflow.

For instance, a connection might comprise an action that fetches the current time or an action that lists available virtual machines.

How do connectors work?

Connections are realized as a suite of APIs provided by the Kubiya SDK, ready to be called by Kubiya. When a Kubiya workflow calls an action, it supplies any necessary parameters to the connection and obtains a result in return. This result is then utilized as input for other actions or workflow stages.

Connections can be designed for specific tasks, such as accessing a database or engaging with a cloud platform. They also support various programming languages, though currently, only Python is endorsed.

Built-in connectors

Integrations include support for Amazon Web Services, Jira and Kubernetes, allowing you to get started quickly. Using these integrations allows one to quickly enter the world of Kubiya without developing their own custom actions. A significant feature of our platform support is the continual expansion of our support to meet evolving cloud technologies.

Pre-Built Integrations vs. Custom Actions

Pre-Built Integrations:

Rapid, hassle-free setup.
Wide platform support.
Hassle-free maintenance and updates.
Ideal for most users.

Custom Actions:

Requires coding expertise.
Tailored for unique requirements.
You manage maintenance.
Best for specific technical needs.

Select the approach that matches your technical proficiency and integration goals

Supported platforms

Kubernetes
Amazon Web Services (AWS)
GitHub
Atlassian JIRA
Custom environments

Custom connections

Pydantic functions that execute actions on a designated engineering platform

Custom connections allows you to define your own logic which can be executed by the Kubiya virtual assistant. No AI knowledge is needed. Just simple programming to define the logic you need to execute

In the world of custom connections, there are a few things to take in mind:

You own the maintenance of your actions — as those are running from your own infrastructure
Very often authentication and networking to various target platforms will be managed using your own code

Extending with custom actions

Very often we can extend the built-in capabilities with our own custom use cases — such as executing a python function. Or sending a very specific HTTP request based on a query to the virtual assistant

we can tweak Kubiya to execute our code on our own infrastructure using the Custom Actions SDK which allows you to define simple Python functions which can run on your own Kubernetes cluster.

Once given a descriptive name for the function, alongside a strict Pydantic model Kubiya can respond to queries related to the function topic immediately

Action Runners allows an effective, secured tunnel to your infrastructure from Kubiya to fulfil executions

Action Runners

Action runners are a fundamental component of Kubiya, Action Runners enable the execution of custom actions using the Kubiya SDK. These runners enable users to execute action stores on any Kubernetes-based infrastructure, offering versatility for a wide range of use cases.

Action runners offer a solution for users who want to take advantage of Kubiya’s functionality but cannot open network access to the broader network. Actions can be executed on local networks, such as a virtual private cloud (VPC) or on-premise environments. Remote runners allow users to control the source code of their actions in their own repositories, giving them full control over development and deployment.

How Action Runners communicate with your Kubiya namespace

With local runners, you can execute serverless functions against a supported infrastructure with ease, making it a key feature of Kubiya’s user-friendly and customizable experience.

For a full reference on how to use the remote runners’ feature, please visit the Action Runners documentation.

Events

The Events feature empowers your virtual assistant to seamlessly respond to webhooks, enhancing its capabilities to interact with external systems and services. By leveraging events triggered by incoming webhooks, your virtual assistant can stay dynamically connected to the broader ecosystem of your applications and platforms.

Note: The events feature is still on Beta

Key Features

1. Webhook Integration

Easily integrate your virtual assistant with external services through webhook configurations.
Receive real-time data and updates from third-party systems directly into your assistant’s workflow.

2. Event Handlers

Define custom event handlers to process incoming webhook data.
Tailor your assistant’s responses based on the content and context of the received events.

3. Dynamic Responses

Enable dynamic and context-aware responses by utilizing webhook-triggered events.
Craft personalized interactions with users based on external events.

4. Configuration Flexibility

Configure event subscriptions and webhook endpoints to adapt to your specific use cases.
Fine-tune how your virtual assistant reacts to different types of events.

Use cases examples

Kubiya offers versatile applications for integrating notifications from various platforms within your technology stack.

The platform’s strength lies in its capacity to process generic webhooks and events, enabling users to respond effectively based on predefined suggestions or custom-developed actions.

This flexibility enhances the overall functionality and power of the system. To assist you in leveraging these capabilities, we will present several practical use cases, demonstrating how to effectively utilize events in Kubiya for your specific needs.

There are some Advanced features in Kubiya but those are in the Experimental

Kubiya offers Advanced Mode (Experimental)

Chat with a containerized application with custom tools

Advanced mode allows you to define a containerized application with your own set of tools. You can prepare your use cases in that containerized environment and allow your team members to chat with it from within Kubiya.

Each containerized application is defined as a custom profile, utilizing stateless container technology and AI agents to execute a wide array of tasks. This document provides a technical overview of the architecture and functionalities of Kubiya’s custom copilots.

Built-in tools

Managed containers (available in the free trial and sandbox) are equipped with the following tools:

Terraform CLI
AWS CLI
GH (GitHub CLI)
Bash
Python (3.11)

Architecture

Custom Copilots are built on top of stateless container technology. These containers serve as isolated execution environments where a set of tools and applications can be deployed.

Custom profiles can be managed using source control for easy maintenance of the core instructions for the AI on the specified profile environment

Key Components:

Stateless Containers: The containers are ephemeral, initializing with a predefined configuration and terminating after the execution is complete. This ensures a consistent environment for every execution cycle.
Can run on your own environment using a local runner (Kubernetes)
**AI Agent**: An AI-powered agent resides within the container, acting as the executor of tasks. It interprets natural language commands and interacts with the tools and applications within the container to perform the specified actions.

Functionalities

Tool Integration: Containers can be provisioned with a wide variety of tools according to the use case requirements, such as DevOps tools like Terraform or cloud interfaces like AWS CLI.
Natural Language Interface: Users can interact with the copilots using natural language commands, making the system accessible to users with varying levels of technical expertise.
Ability to plan and execute code
Easy definition of use cases using simple YAML specification

Deployment Options

On-Premises: For users requiring bespoke configurations and control over the execution environment, custom copilots can be deployed on private infrastructure.
SaaS Platform: For ease of use and maintenance, Kubiya offers a SaaS platform where custom copilots with pre-configured tools are available. This option provides streamlined access with some limitations in tool customization.

Usage

Command Execution: Users issue commands in natural language, which the AI Agent interprets and executes within the container environment after starting a Slack thread
Session Lifecycle: Each session initializes a stateless container with the necessary tools, executes the commands, and then safely terminates the container after a limited amount of time. default is 90 minutes

Limitations

Core Tool Customization: While on-premises deployment allows full customization, the SaaS version limits the ability to customize core tools within the container.

Conclusion:

As we embark on the dynamic journey of DevOps, Kubiya AI stands out not just as a participant but as a strategic leader. With its effective use of automation, insightful analytics, collaborative approach, and a strong commitment to continuous improvement, Kubiya AI navigates through the evolving landscape of software development. It leaves behind a sense of admiration and inspiration. Applause for a remarkable performance, Kubiya! 🌟

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on cloud computing, tech and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

AWS Cost Anomaly Detection — Optimizing Cloud Economics

subbaramireddy — Fri, 17 Nov 2023 18:36:12 +0000

In the dynamic world of cloud services, managing budgets and assessing cost performance is a crucial task. AWS introduced Anomaly Detection, a feature that allows users to segment spending effectively. It provides a detailed understanding of spending patterns through a comprehensive analysis, enabling more informed and strategic decision-making in cost management.

What is Anomaly?

Anomaly refers to something that deviates from what is standard, normal, or expected. In the context of AWS Cost Anomaly Detection, it involves identifying and flagging unusual or unexpected patterns in spending behaviour within cloud services.

AWS Cost Anomaly Detection uses a multi-layered state machine learning model that learns your unique spend patterns to adjust spend thresholds — this means you do not need to worry about determining appropriate thresholds (e.g. 10% increase in daily spend) and maintaining them as your usage changes over time. These machine learning layers allow Anomaly Detection to detect various types of anomalies, such as a one-time cost spike or gradual, consistent cost increases. Anomaly Detection is customizable to segment your spending data and provides tailored alerting preferences so that you are informed as soon as an anomaly is detected.

Architecture

Use cases

Reduce surprise bills

Stay informed of spending anomalies through automated detection alerts, via email or Amazon SNS topics, at the frequency of your choice. With Amazon SNS topics, you can send alerts to your Slack channel or Amazon Chime chat room to support collaboration and timely resolution of alerts.

Set alert subscription

Once you have created your cost monitor, you can choose your alerting preference by setting up a dollar threshold (e.g. only alert on anomalies with impact greater than $1,000). You don’t need to define an anomaly (e.g. percent or dollar increase) as Anomaly Detection does this automatically for you and adjusts over time. ).

Receive alerts

Once cost monitors and alert subscriptions are created, you’re all set! Anomaly Detection will begin to work within 24 hours and you will be notified if any anomaly meets your alert threshold. You can visit your Anomaly Detection dashboard to monitor the activities, including anomalies detected that are below your alert threshold.

Benefits

Anomaly Detection: AWS Cost Anomaly Detection spots irregularities in your AWS spending. This includes detecting significant spending spikes, unusual usage patterns, or deviations from historical norms.
Alerts and Notifications: The service provides customizable alerts and notifications. When an anomaly is detected, it can send notifications via email, SMS, or other channels, ensuring that you’re promptly informed about spending irregularities.
Customization Options: AWS Cost Anomaly Detection can be customized to detect anomalies specific to the organization’s needs. You can set custom threshold values, define which AWS accounts or resources to monitor, and configure alerting preferences to match your operational requirements.

Let’s take a deeper look at Cost Anomaly Detection

Anomaly Detection comes integrated with AWS Cost Explorer so you can graphically visualize the anomaly and perform further analysis, as needed. You’ll notice that the filters and dates are automatically configured based on the anomaly details.

Anomaly Detection is powered by a complex machine learning model yet it is simple and easy to use in AWS Console. It takes two steps with a few clicks to enable Anomaly Detection.

Log in to your AWS Console and navigate to AWS Cost Management Dashboard, and you will find “Cost Anomaly Detection” on your left pane.

You can check the detection history and summary of cost anomaly detection

To view the cost monitors, navigate to it. AWS automatically creates a default Service Monitor. To create a new cost monitor, click on ‘Create monitor’.

Create a monitor

Create a monitor by selecting monitor types, linked accounts, and specifying a unique name during the setup. These are the supported monitor types by AWS.

Key Monitor Types:

AWS Services Monitor:

Ideal for users not need internal organization segmentation.
Automatically adapts to new AWS services without manual configuration.

2. Linked/Member Account Monitor:

Evaluates total spending for individual or grouped linked accounts.
Useful for organizations segmenting spend by teams, products, services, or environments.

3. Cost Categories Monitor:

Tailored for users leveraging Cost Categories for organizational structure.
Allows monitoring spending for specific Cost Category values.

4. Cost Allocation Tag Monitor:

Segments spent based on Cost Allocation Tags.
Supports one tag key with multiple tag values for flexible evaluation.

Note: Only 1 AWS services monitor type can be created for an account. Remove any existing monitors to create a new AWS Service monitor.

Alert Subscriptions

As you create your monitors, you will be able to configure your alerting preference. Note that you do not have to define an anomaly (e.g. 10% increase in daily spend) — we leverage machine learning to automatically determine and continuously adjust thresholds as your spending patterns change over time. What you need to define is your alerting preferences so that you only receive alerts that are important to you based on the estimated cost impact. Anomalies that do not meet these thresholds are captured and available to you in the Anomaly Detection dashboard.

Additionally, you can determine the alerting frequency. Individual alerts allow you to be notified as soon as an anomaly is detected.

(note: this is not alerted immediately after the usage occurs) — this means that if multiple anomalies are detected throughout the day, you might receive multiple alerts. Another option is to receive Daily or Weekly summary reports, which consolidate the alerts into a single notification, either daily or weekly.

Lastly, Anomaly Detection supports SNS and email notifications.

You’ll notice that you can configure these preferences within each individual Monitor you have created. This will allow you to notify the appropriate stakeholders within your organization.

Example: let’s say your organization has two linked/member accounts -1234 and -4321. You can create a monitor that evaluates account -1234 and configure notifications to alert Subbu and Bucky. You can also create a separate monitor that evaluates account -4321 and configure notifications to alert Subbu and Sravan. In this example, Bucky and Sravan are only notified if there is an anomaly in the account that they are associated with, but Subbu gets notified of any anomaly across both accounts.

Configure alerting preferences

Configure the Alert Subscriptions, Create a new Subscription with the Subscription name. select the Alerting frequency as Daily and specify the Alert recipients to receive the notifications. Specify the Threshold which you need to get the alert whenever the amount is above the expected spend.

View the list of Cost monitors, select the created one, and click on it.

In the monitor dashboard, you can check the count of detected anomalies and view the detection history.

Click on **Alert Subscriptions **to view the added subscriptions.

We can review subscribers and attach cost monitors to the DevOpsVoyager Alert Subscription for streamlined management. This ensures timely notifications for spending anomalies.

Conclusion

AWS Cost Anomaly Detection provides you with an easy-of-use, ML-driven capability to detect unusual spending across your AWS accounts. You can configure Anomaly Detection to better reflect your organization’s business needs and/or how you manage your cost and usage. Anomaly Detection provides flexible alerting capability so that the correct stakeholders are notified of each anomaly that is detected, using their own alerting preferences. With the introduction of Anomaly Detection, AWS Cost Management is providing you yet another way to more effectively control your cost and usage while minimizing unintended spend.

https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-detection/

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on cloud computing, tech and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) **Linkedin**

AWS Cost Explorer — Your Financial Compass in the Cloud

subbaramireddy — Thu, 16 Nov 2023 18:09:43 +0000

Introduction to AWS Cost Explorer, your guide to financial clarity in the dynamic realm of cloud computing. Developed by Amazon Web Services (AWS), this tool is crafted to navigate the complexities of managing costs and usage effectively within the AWS platform.

AWS Cost Explorer is more than just a financial compass; it’s a comprehensive solution to the challenges organizations face in the ever-expanding cloud landscape. Whether you prefer a visual dashboard or a programmatic approach, this tool caters to your needs. Its user-friendly interface allows you to delve into historical data, forecast expenses for the future, and provides strategic recommendations for cost optimization, including the use of Reserved Instances.

AWS introduced Cost Explorer in response to the surging complexity of cloud spending. As organizations embraced the cloud for its flexibility and scalability, the need for effective cost management became paramount. This tool emerged as a financial advisor, simplifying the cloud financial journey by offering a consolidated view of your financial landscape on AWS.

It goes beyond just numbers; it’s about empowering users to make informed decisions and ensuring that cloud budgets remain in check. The integration of the AWS Cost Management console with the Billing console provides a holistic approach to managing ongoing payments and optimizing future costs.

Essential benefits of AWS Cost Explorer

Visibility and Transparency: Provides a clear view of AWS spending.
User-Friendly Interface: Intuitive for users at all levels.
Historical Data Analysis: Analyze costs for the past 12 months.
**Cost Forecasting: **Estimate spending for the next 12 months.
Reserved Instances Recommendations: Optimize costs with strategic recommendations.
Preconfigured Views: Quick insights into cost trends.
Cost Explorer API: Programmatic access for automation.
Data Export: Export data for custom analyses.
**Regular Data Refresh: **Data is updated at least every 24 hours.
**Cost Management Integration: **Works seamlessly with the Billing console.
Free User Interface Access: No additional charges for UI usage.

Use Cases

Cost Monitoring and Visibility: Gain insights into ongoing AWS costs for visibility and control.
Budgeting and Financial Planning: Forecast future spending to create budgets and allocate resources efficiently.
**Resource Optimization: **Utilize Reserved Instances recommendations for strategic cost optimization.
**Historical Cost Analysis: **Analyze trends and patterns in historical cost data for informed decision-making.
**Cost Attribution and Showback: **Allocate costs to projects, departments, or teams for accountability.
Custom Reporting and Analysis: Export data for tailored, in-depth analysis using external tools.
Automation and Integration: Integrate with workflows using the Cost Explorer API for automated cost management.
**Periodic Cost Reviews: **Schedule regular reviews to proactively manage and optimize costs.
**Educational and Training: **Use as a learning tool for AWS cost management practices.
**Vendor and Financial Audits: **Facilitate audits and financial reviews with comprehensive cost data.

Pricing

AWS Cost Explorer provides users with the ability to view and analyze their costs and usage on the AWS platform for free. You can explore your spending trends, and historical data for up to the last 12 months, and get insights into your projected costs for the next year — all without incurring additional charges for using the Cost Explorer user interface.

However, it’s essential to note that if you choose to access your data programmatically using the Cost Explorer API, each paginated API request incurs a minor charge of $0.01. The user interface, though, is at your service without any additional cost.AWS Cost Explorer, with its user-friendly interface and powerful features, is a valuable tool for managing and optimizing your cloud expenses without breaking the bank!

Let’s take a deeper look at Cost Explorer

open the AWS Cost Management console, In the navigation pane, choose Cost Explorer and On the Welcome to Cost Explorer page, choose Launch Cost Explorer.

AWS Cost Management console -> Cost Explorer -> Launch Cost Explorer

On the Cost Explorer page, you have the flexibility to view cost reports tailored to your specific date range. What’s even better is that you can download these reports into a handy comma-separated value (CSV) file.

It’s like having your cost data at your fingertips, ready to be analyzed or shared in a format that suits your needs. It’s all about giving you the tools to manage your cloud costs with ease!

Note: In the Filters section select the charge type as usage.

To view costs based on resources, go to the “Group by” section in Cost Explorer. Under Dimensions, choose “service”. This smart move categorizes your costs, giving a visual breakdown of expenses and showing where your spending is concentrated across AWS services.

For region-based cost reports in Cost Explorer, choose “Region” under Dimensions. Save it with a specific name for organized reference. Easy and effective!

On the Cost Explorer page, check out the region-wise cost report dashboard for a quick visual overview of your expenses across different AWS regions. Simple and insightful!

Let’s try to get the cost based on the specific AWS Resources, i.e Elastic Compute Cloud (EC2) we can use the filters to get the cost of EC2:

Service: EC2
Linked Account: subbaramireddy
Regions: ap-south-1, ap-south-2
Instance Types: t2.micro, t3.micro, t2.medium
Charge Types: usage

Cost Reports

AWS Cost Management -> Reports
Check out the list of reports on the Cost Explorer page — some made by AWS, some custom. For a new report, simply click “New Report”. It’s your ticket to personalized cost analysis!

Choose “Cost and Usage” as the report type (recommended), then click “Create Report” It’s the quick path to insightful cost analysis!

In essence, AWS Cost Explorer is your ally in simplifying the cloud financial journey, combining practical insights with aesthetic appeal. It’s not just a tool; it’s a testament to AWS’s commitment to empowering users for efficient and cost-effective cloud usage. For a deeper dive into AWS resources, understanding, and organizing your bills, refer to the AWS Billing.

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with fellow tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on tech, programming, and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

AWS Resource Explorer

subbaramireddy — Tue, 14 Nov 2023 17:49:41 +0000

When you have a large cloud estate having multiple accounts, multiple resources and multiple resources across multiple regions, it is a tedious task for one to search for a particular resource that he is looking for. For example, to find an S3 bucket and EC2 instance, we need to explore the respective resource details and search for the resource we are looking for.

This is reduced a lot by the introduction of AWS Resource Explorer. It can be used to quickly search for a cloud resource using its resource ID, metadata tags or resource groups. AWS Resource Explorer is a resource search and discovery service. With Resource Explorer, you can explore your resources, such as Amazon Elastic Compute Cloud instances, Amazon Kinesis streams, or Amazon DynamoDB tables, using an internet search engine-like experience. You can search for your resources using resource metadata like names, tags, and IDs. Resource Explorer works across AWS Regions in your account to simplify your cross-region workloads.

Resource Explorer provides fast responses to your search queries by using indexes that are created and maintained by the AWS Resource Explorer service. Resource Explorer uses a variety of data sources to gather information about resources in your AWS account. Resource Explorer stores that information in the indexes for Resource Explorer to search. For more click here

Supported services and resource types

Supported AWS services

| Service Name                               | Description                                   |
|--------------------------------------------|-----------------------------------------------|
| Amazon Elastic Compute Cloud (EC2)         | Virtual servers in the cloud                  |
| Amazon Simple Storage Service (Amazon S3) | Scalable object storage                       |
| Amazon OpenSearch Service                  | Managed Elasticsearch service                 |
| AWS Elastic Beanstalk                       | Platform for deploying applications           |
| Amazon CloudFront                           | Content delivery network (CDN)                |
| Amazon ElastiCache                          | In-memory caching service                     |
| Amazon Elastic Container Registry          | Docker container registry                     |
| Amazon Elastic File System                  | Scalable file storage service                 |
| AWS IoT                                    | Internet of Things (IoT) platform             |
| Amazon Elastic Container Service           | Container orchestration service               |
| AWS CodeBuild                              | Fully managed build service                   |
| Amazon Kinesis                              | Real-time streaming data service              |
| Elastic Load Balancing                      | Distributes incoming application traffic     |
| AWS CodePipeline                           | Continuous delivery service                   |
| AWS Resource Explorer                       | View and explore AWS resources                |
| AWS Identity and Access Management (IAM)    | Identity and access management service       |
| Amazon CloudWatch Logs                      | Monitor, store, and access log files          |
| AWS Service Catalog                         | Create, manage, and share products            |
| AWS Lambda                                 | Run code without provisioning or managing servers |
| Amazon DynamoDB                            | NoSQL database service                        |
| Amazon Simple Notification Service (SNS)    | Pub/sub messaging service                     |
| Amazon Relational Database Service (Amazon RDS) | Managed relational database service      |
| Amazon CloudWatch                           | Monitoring service for AWS resources         |
| Amazon Simple Queue Service (SQS)           | Fully managed message queuing service        |
| AWS Systems Manager                         | Gain operational insights and take action     |
| AWS Step Functions                          | Serverless function orchestration            |
| Amazon Cognito                             | Identity management for apps                 |

To know more about it click here

Benefits and features

Add resources

Find relevant resources across AWS Regions using resource metadata, tags and simple keywords.

Act quickly

Quickly navigate to your resources and act on them in the appropriate service console.

Increase efficiency

Save time by finding and acting on resources without building a custom search infrastructure.

Search from anywhere

Search for resources from any service page using a unified search in the AWS Management Console.

Use cases

Better understand your application resources

Search for application resources across AWS Regions using application tags and resource metadata.

Locate resources to act on alerts promptly

Address alerts by finding and navigating to relevant resources directly from the unified search bar in the AWS Management Console.

Related AWS services

The following are the other AWS services whose primary purpose is to help you manage your AWS resources:

AWS Resource Access Manager (AWS RAM)

Share the resources in one AWS account with other AWS accounts. If your account is managed by AWS Organizations, you can use AWS RAM to share resources with the accounts in an organizational unit, or all of the accounts in the organization. The shared resources work for users in those accounts just like they would if they were created in the local account. For more Click here

AWS Resource Groups

Create groups for your AWS resources. Then, you can use and manage each group as a unit instead of having to reference every resource individually. Your groups can consist of resources that are part of the same AWS CloudFormation stack, or that are tagged with the same tags. Some resource types also support applying a configuration to a resource group to affect all relevant resources in that group. For more Click here

Tag editor and the AWS Resource Groups Tagging API

Tags are customer-defined metadata that you can attach to your resources. You can categorize your resources for purposes like cost allocation and attribute-based access control. For more Click here

Pricing

There are no charges to search for resources by using AWS Resource Explorer, including creating views, turning on Regions, or searching for resources. However, interacting with the resources that you find in your search results can result in usage charges that vary depending on the resource type and its AWS service. For more information about how AWS bills for the normal use of a specific resource type, refer to the documentation for that resource type’s owning service.

Setting up and configuring Resource Explorer

Let’s see how this works in practice.

Using AWS Resource Explorer
To start using Resource Explorer, we need to turn it on so that it creates and maintains the indexes that will provide fast responses to your search queries. Usually, the administrator of the account is the one taking these steps so that authorized users in that account can start searching.

To run a query, we need a view that gives access to an index. If the view is using an aggregator index, then the query can search across all indexed Regions.

If the view is using a local index, then the query has access only to the resources in that Region.

we can control the visibility of resources in your account by creating views that define what resource information is available for search and discovery. These controls are not based only on resources but also on the information that resources bring. For example, We can give access to the Amazon Resource Names (ARNs) of all resources but not to their tags which might contain information that we want to keep confidential.

let’s create a resource explorer

In the Resource Explorer console, choose Enable Resource Explorer. Then, select the Quick Setup option to have visibility for all supported resources within your account. This option creates local indexes in all Regions and an aggregator index in the selected Region. A default view with a filter that includes all supported resources in the account is also created in the same Region as the aggregator index.

Quick Setup

With the Quick setup option selected, You choose Go to Resource Explorer. A quick overview shows the progress of enabling Resource Explorer across Regions. After the indexes have been created, it can take up to 36 hours to index all supported resources, and search results might be incomplete until then. When resources are created or deleted, your indexes are automatically updated. These updates are asynchronous, so it can take some time (usually a few minutes) to see the changes.

Searching With AWS Resource Explorer

After resources have been indexed, You can choose to Proceed to resource search. In the Search criteria, choose which View to use. Currently, It has the default view selected. Then, you can start typing in the Query field to search through the resources in your AWS account across all Regions.

For example, I have an Ec2 instance where I used the convention to start resource names with EKS-Client. For the resources I created manually, I also added the Env tag with the value Dev.

To find the resource of this instance, start searching for it with the name EKS-client.

The results include resources from multiple services and Regions Optionally, you can filter results by Region or resource type. If you choose any of the listed resources, the link will bring me to the corresponding service console and Region with the resource selected.

To look for something in a specific Region, such as Mumbai, you can restrict the results by adding region:ap-south-1 to the query.

You can further restrict results to Amazon Ec2 resources by adding service:ec2 to the query. Now you only see the EC2 instances and Elastic IP in Mumbai. That’s what I was looking for!

You can also search using tags. For example, you can see the resources where we have added the Dev tag by including tag.value:Dev in a query. To specify the actual key-value pair of the tag, you can use tag:Env=Dev.

Creating a Custom View

Sometimes you need to control the visibility of the resources in your account. For example, all the EC2 instances used for the development of your account are in ap-south-1(Mumbai). You create a view for the development team by choosing a specific Region (ap-south-1) and filtering the results with service:ec2 in the query. Optionally, you could further filter results based on resource names or tags. For example, you can add tag:Env=Dev to only see resources that have been tagged to be in a development environment.

Now you allow access to this view to users and roles used by the development team. To do so, you can attach an identity-based policy to the users and roles of the development team. In this way, they can only explore and search resources using this view.

Unified Search in the AWS Management Console

After you turn Resource Explorer on, you can also search through your AWS resources in the search bar at the top of the Management Console. We call this capability unified search as it gives results that include AWS services, features, blogs, documentation, tutorials, events, and more.

To focus the search on AWS resources, add /Resources at the beginning of your search.

Note that unified search automatically inserts a wildcard character (*) at the end of the first keyword in the string. This means that unified search results include resources that match any string that starts with the specified keyword.

The search performed by the Query text box on the Resource search page in the Resource Explorer console does not automatically append a wildcard character but you can do it manually after any term in the search string to have similar results.

Unified search works when you have the default view in the same Region that contains the aggregator index. To check if a unified search works for you, look at the top of the Settings page.

Advanced Setup

With the Advanced setup option, you have access to more granular controls that are useful when there are specific governance requirements. For example, you can select in which Regions to create indexes. You can choose not to replicate resource information to any other Region so that resources from each AWS Region are searchable only from within the same Region. you can also control what information is available in the default view or avoid the creation of the default view.

Choose Go to Resource Explorer. A quick overview shows the progress of enabling Resource Explorer in a selected specific Region. When resources are created or deleted, your indexes are automatically updated. These updates are asynchronous, so it can take some time (usually a few minutes) to see the changes.

we can view the index which is created in ap-south-1, Resource Explorer -> indexes

Thanks for reading! I hope you found this helpful and informative.

I’m always happy to connect with fellow tech enthusiasts and answer any questions you may have. Don’t forget to follow me for more updates on tech, programming, and more.😄😄

Follow me on LinkedIn to see interesting posts like this : ) Linkedin

What is YAML?

subbaramireddy — Sun, 24 Sep 2023 16:03:39 +0000

YAML is a data serialization language for storing information in a human-readable form.
YAML stands for "YAML Ain't Markup Language."
YAML is not a Markup Language.
YAML is used to store complex data in a compact and readable format.
YAML was originally designed as a configuration language but is now used in various applications.
We can use YAML to define key and value pairs like variables, lists and objects
YAML is very similar to JSON (Javascript Object Notation) and XML (Extensible Markup Language)
YAML primarily focuses on readability and user-friendliness.
YAML is designed to be clean and easy to read.
We can define YAML files with two different extensions.
.yml
.yaml