Forem: bismillahkani

Architecting MLOps for Computer Vision using AWS Sagemaker

bismillahkani — Mon, 29 Jan 2024 08:48:59 +0000

In this post, we will dive into the world of Machine Learning Operations (MLOps) and explore how it can revolutionize the process of creating, deploying, and managing machine learning models. We will also walk you through a practical use case involving computer vision to demonstrate the real-world application of MLOps principles and how AWS SageMaker plays a pivotal role in streamlining this journey.

What is MLOps?

Let's start with the basics. MLOps, short for Machine Learning Operations, is all about making the process of creating, deploying, and managing machine learning models as smooth and efficient as possible. It's like taking the principles of DevOps, which helps streamline software development, and applying them to the world of machine learning. Imagine it as the smart engine that powers the lifecycle of machine learning, making it easier to build and deploy intelligent solutions.

https://neptune.ai/blog/mlops

Now, let's break down MLOps into four key principles that guide us through this journey. First, we have 'Version control,' which is like keeping track of changes in our machine learning assets, just like you would with documents. Next, there's 'Automation,' think of it as creating repeatable and consistent processes using ML pipelines to make your work smoother. Then, 'Continuous X' is all about continuous training and deployment of models, ensuring they stay up to date. Lastly, 'Model governance' is like having a structured process to review and validate our models, making sure they're reliable. As for the three levels of MLOps implementation, think of them as steps in our journey. Level 0 is the manual phase, where everything is done by hand. Level 1 brings in continuous model training through pipelines. And Level 2, that's the gold standard, where we cover everything from data collection, training, all the way to deployment, creating a seamless MLOps process.

Use case

Before we delve into MLOps, consider this practical scenario: improving quality control in a metal luggage tag manufacturing line using machine learning. Our goal is to detect defects like scratches in real-time, whether on the edge for immediate monitoring or in a cloud environment like AWS SageMaker.

https://aws.amazon.com/blogs/machine-learning/

MLOps pipeline

Now, let's outline the architecture of our MLOps pipeline. At this stage, we're focusing on the big picture, not specific technologies. Our goal is to automate the entire process, starting with capturing raw images of metal tags using an edge camera. We then label these images with bounding boxes, ensuring version control for traceability. Next, we train, fine-tune, and evaluate the model before deploying it, either at the edge for real-time inferences or in the cloud with AWS SageMaker.

MLOps Solution Architecture

Let's now take a closer look at how we can build our architecture using AWS services. We've designed a workflow that seamlessly integrates these services to optimize our MLOps pipeline. We start by storing raw image data in Amazon S3 for cost-efficiency. Next, we orchestrate the labeling process with AWS Step Functions and SageMaker Ground Truth, leveraging AWS Lambda for data preparation. For model training and evaluation, we rely on SageMaker Pipelines, SageMaker Training jobs, and SageMaker Processing jobs. The trained model is registered in Amazon SageMaker Model Registry for management and version control. For edge deployment, we automate the process with Step Functions and utilize AWS IoT Greengrass as our edge device runtime environment. Alternatively, we can deploy the model as SageMaker endpoints in the cloud for flexibility. This architecture minimizes operational effort by leveraging managed and serverless services while maintaining the integrity of our MLOps pipeline.

CI/CD pipeline

Now, let's explore how we integrate our data labeling, model training, and edge deployment steps into a fully automated CI/CD system using AWS services. We begin with AWS CodeCommit as our Git repository and use the AWS CDK to automate infrastructure deployment, allowing for independent deployments of each part. AWS CodePipeline is utilized to automate both code and infrastructure deployments, and we have separate pipelines for assets and workflows. This setup ensures that changes in our code or data trigger the orchestration process, making our end-to-end pipeline responsive to updates and data modifications.

Orchestrating Data Labelling

We employ AWS Step Functions to automate our labeling pipeline, which comprises four major steps. These steps include checking for new images, data preparation, initiating labeling and label verification jobs, and writing the final labels to the feature store.

Orchestrating Model Training

In this workflow, we have three key steps: data processing, model training, and model registration. First, data is loaded and split, then the model is trained and evaluated for mAP. If it meets the threshold, the model is registered in the SageMaker Model Registry.

Model deployment

After training and evaluation, deploying an ML model for real-world impact is crucial. Continuing our journey, we now turn our attention to automating the edge deployment phase in our end-to-end MLOps pipeline. There are two approaches for model deployment: one is IoT Greengrass for real-time edge inference, but today, due to time constraints, we'll focus on the second approach, AWS SageMaker endpoints, which offers a more straightforward deployment process. We've ensured portability and optimization by exporting the model in ONNX format and registering it in Amazon SageMaker Model Registry. Our deployment package consists of the trained model in ONNX format, and alongside it, we've implemented a private component for inference code. This component handles data preparation, communication with the model, and post-processing of results to ensure seamless and effective model deployment in production systems.

Summary

MLOps streamlines machine learning by applying DevOps principles, featuring version control, automation, continuous training, and model governance. We explored applying MLOps principles to an industrial computer vision use case, improving quality control in luggage tag manufacturing. Leveraging AWS services, we created an efficient MLOps pipeline, managing data, labeling, training, and deployment seamlessly. Through automated CI/CD using AWS tools, we ensure responsiveness to code and data changes throughout the end-to-end pipeline, ensuring efficiency and flexibility.

References

When Generative AI meets AWS Community Builders : Unveiling the Ultimate AWS Cost Advisor

bismillahkani — Fri, 30 Jun 2023 10:18:35 +0000

Building a Retrieval Augmented Generation (RAG) Pipeline with AWS SageMaker, Hugging Face Transformer and Langchain

Introduction

AWS Community Builders gave us an interesting opportunity to work on Integrating AI tools and solve AWS use-cases.

This unique collaboration sparked our collective imagination, leading us to brainstorm the (perfect) solution. After countless hours of innovation and collaboration, we are thrilled to unveil this groundbreaking tool that will redefine AWS Cost Savings and FinOps landscape: a Q&A Chatbot powered by Sagemaker, HuggingFace, LangChain, and supercharged with Retrieval Augmented Generation. Join us on this captivating journey as we present our revolutionary solution that harmonizes the worlds of generative AI and AWS Solution Architect skills

AWS Cost Advisor is an AI tool that is designed to help users optimize their cloud costs by providing cost recommendations, identifying cost anomalies, and offering cost-saving suggestions. The tool is built using Retrieval-Augmented Generation (RAG), a combination of retrieval-based and generation-based techniques. The knowledge database of the AWS Cost Advisor is built using AWS Cost Management FAQs as a vector database that is integrated into the RAG pipeline. In this blog post, we will demonstrate the Cost Advisor tool and its improved performance as compared to a generic text generation Q&A assistant.

Problem Statement

The AWS Cost Advisor tool was designed to help users optimize their cloud costs by providing cost recommendations, identifying cost anomalies, and offering cost-saving suggestions. However, the tool was facing a problem of accuracy and relevancy of the answers provided by the tool. The generic text generation Q&A assistant would often provide irrelevant answers or not answer the user's question at all. To solve this problem, we implemented the RAG pipeline.

No more Unnecessary Cash Burn

Available Solutions:

Before we delve into our groundbreaking solution, let's explore the existing options. Conventional cost optimization tools often lack the sophistication to comprehensively analyze complex billing data and provide granular insights. Additionally, chatbots typically struggle to handle the intricacies of AWS services and generate meaningful recommendations. These limitations hinder Solution Architect's progress, making it challenging to unlock the full potential of savings in the Cloud.

Existing solutions like Cost Explorer, AWS Budgets, can feel lot like DIY

Tools, tools, and lot of knobs and levers everywhere

Retrieval-Augmented Pipeline

The RAG pipeline uses a combination of retrieval-based and generation-based techniques to provide more relevant and accurate answers. The pipeline consists of three components: a retriever, a reader, and a generator. The retriever retrieves the most relevant documents from the knowledge database, the reader extracts the answer from the retrieved documents, and the generator generates a more concise and accurate answer.

Solution Architecture

High level solution overview

Huggingface Open-Source LLM Models

The Huggingface open-source large language models (LLMs) are used by AWS Cost Advisor for the generation-based component of the Retrieval-Augmented Generation (RAG) pipeline. These models are pre-trained on large amounts of text data and can generate high-quality responses to natural language questions. Huggingface LLM models are hosted as Sagemaker endpoints.

MODEL_CONFIG = {
    "huggingface-text2text-flan-t5-xxl": {
        "instance type": "ml.g4dn.xlarge",
        "env": {"SAGEMAKER_MODEL_SERVER_WORKERS": "1", "TS_DEFAULT_WORKERS_PER_MODEL": "1"},
        "parse_function": parse_response_model_flan_t5,
        "prompt": """Answer based on context:\n\n{context}\n\n{question}""",
    },
    "huggingface-textembedding-gpt-j-6b": {
        "instance type": "ml.g4dn.2xlarge",
        "env": {"SAGEMAKER_MODEL_SERVER_WORKERS": "1", "TS_DEFAULT_WORKERS_PER_MODEL": "1"},
    },
}

newline, bold, unbold = "\n", "\033[1m", "\033[0m"

for model_id in _MODEL_CONFIG_:
    endpoint_name = name_from_base(f"aws-costadvisor-rag-{model_id}")
    inference_instance_type = _MODEL_CONFIG_[model_id]["instance type"]

    # Retrieve the inference container uri. This is the base HuggingFace container image for the default model above.
    deploy_image_uri = image_uris.retrieve(
        region=None,
        framework=None,  # automatically inferred from model_id
        image_scope="inference",
        model_id=model_id,
        model_version=model_version,
        instance_type=inference_instance_type,
    )
    # Retrieve the model uri.
    model_uri = model_uris.retrieve(
        model_id=model_id, model_version=model_version, model_scope="inference"
    )
    model_inference = Model(
        image_uri=deploy_image_uri,
        model_data=model_uri,
        role=aws_role,
        predictor_cls=Predictor,
        name=endpoint_name,
        env=_MODEL_CONFIG_[model_id]["env"],
    )
    model_predictor_inference = model_inference.deploy(
        initial_instance_count=1,
        instance_type=inference_instance_type,
        predictor_cls=Predictor,
        endpoint_name=endpoint_name,
    )
    print(f"{bold}Model {model_id} has been deployed successfully.{unbold}{newline}")
    _MODEL_CONFIG_[model_id]["endpoint_name"] = endpoint_name

Langchain Retrieval QA

The Langchain Retrieval QA is used by AWS Cost Advisor as the retrieval-based component of the RAG pipeline. This tool is designed to retrieve the most relevant documents from the knowledge database based on the user's query. Langchain Retrieval QA uses a combination of semantic search and keyword matching to retrieve the most relevant documents, ensuring that the retrieved documents are highly relevant to the user's query.

AWS Cost Management FAQs

Question	Answer
Who should use the AWS Cost Management products?	We have yet to meet a customer who does not consider cost management a priority. AWS Cost Management tools are used by IT professionals, financial analysts, resource managers, and developers across all industries to access detailed information related to their AWS costs and usage, analyze their cost drivers and usage trends, and take action on their insights.
How do I get started with the AWS Cost Management tools?	The quickest way to get started with the AWS Cost Management tools is to access the Billing Dashboard. From there, you can access a number of products that can help you to better understand, analyze, and control your AWS costs, including, but not limited to, AWS Cost Explorer, AWS Budgets, and the AWS Cost & Usage Report.
What are the benefits of using AWS Cost Explorer?	AWS Cost Explorer lets you explore your AWS costs and usage at both a high level and at a detailed level of analysis, and empowering you to dive deeper using a number of filtering dimensions (e.g., AWS Service, Region, Member Account, etc.) AWS Cost Explorer also gives you access to a set of default reports to help you get started, while also allowing you to create custom reports from scratch.
What kinds of default reports are available?	AWS Cost Explorer provides a set of default reports to help you get familiar with the available filtering dimensions and types analyses that can be done using AWS Cost Explorer. These reports include a breakdown of your top 5 cost-accruing AWS services, and an analysis of your overall Amazon EC2 usage, an analysis of the total costs of your member accounts, and the Reserved Instance Utilization and Coverage reports.
Can I create and save custom AWS Cost Explorer reports?	Yes. You can currently save up to 50 custom AWS Cost Explorer reports.

FAISS Vector Database

The FAISS vector database is used by AWS Cost Advisor to store the vector representations of the documents in the knowledge database. These vector representations are used by the retriever component of the RAG pipeline to retrieve the most relevant documents based on the user's query. The FAISS vector database allows for fast and efficient retrieval of documents, ensuring that the user receives a response in real-time.

index_creator = VectorstoreIndexCreator(
    vectorstore_cls=FAISS,
    embedding=embeddings,
    text_splitter=CharacterTextSplitter(chunk_size=300, chunk_overlap=0),
)

AWS Cost Advisor leverages HuggingFace open-source large language models (LLMs) for the generation-based component of the RAG pipeline. The Langchain Retrieval QA is used for the retrieval-based component. The FAISS vector database is used to store the vector representations of the documents in the knowledge database, which are used by the retriever component to retrieve the most relevant documents.

Talk is cheap. Show me the code. ~ Linus Torvalds

Without any further ado, check out our Solution on Github:
https://github.com/bismillahkani/aws-cost-advisor/tree/main

Performance Comparison with Examples

To demonstrate the improved performance of the AWS Cost Advisor tool, we compared it to a generic text generation Q&A assistant. We asked both tools the same question: "How to get started with AWS cost and usage report?" The generic text generation Q&A assistant provided an irrelevant answer, while the AWS Cost Advisor tool provided a relevant answer with cost-saving suggestions.

Answer generated by LLM
Create an AWS account for a company or an individual. Navigate to the report section, then click on Create report. Select the Report template, and click Create Report.

Answer generated by RAG
The quickest way to get started with the AWS Cost Management tools is to access the Billing Dashboard.

How we're different?

Enter AMAC (Ask Me Anything/AWS Cost Advisor), a fusion of cutting-edge technologies poised to redefine the AWS FinOps experience. Powered by HuggingFace, the industry-leading natural language processing (NLP) platform, and LangChain, an innovative language understanding tool, our chatbot possesses unrivaled contextual comprehension and robust question-answering abilities. By integrating FinOps and AWS Solution Architect expertise, we empower the chatbot with the knowledge to provide insightful cost optimization recommendations specific to participants' AWS deployments.

But that's not all – our Q&A Chatbot goes beyond traditional approaches by incorporating Retrieval Augmented Generation. This groundbreaking technique seamlessly blends retrieval-based models and generative models to deliver the most accurate and contextually rich responses. The chatbot efficiently scans massive billing datasets, extracts pertinent information, and then generates coherent and concise answers, providing participants with actionable insights tailored to their specific cost optimization needs.

Real-life use-cases

The AWS Cost Advisor AI Hackathon Scenario: Imagine a team of Cloud/DevOps/FinOps/Solution Architects/Engineers eager to optimize costs in their AWS deployments. Empowered by our Q&A Chatbot, they effortlessly navigate through complex billing reports, analyzing costs and identifying areas for improvement. The chatbot comprehends their queries, retrieves relevant cost data, and provides instant, accurate recommendations, enabling the team to make informed decisions that result in substantial cost savings.

Conclusion (Why we should win)

Our solution is a game-changer for the AWS Cost Management landscape and We're just getting started as this is a side-project for an AI Hackathon (imagine what this will turn out into as a Full-blown project!!!), providing AWS customers with a powerful and efficient tool that significantly enhances their ability to optimize costs in their AWS deployments. By combining the cutting-edge technologies of Sagemaker, HuggingFace, LangChain, and Retrieval Augmented Generation with the expertise of AWS Solution Architects, our Q&A Chatbot offers unparalleled comprehension, lightning-fast retrieval, and human-like responses.

Meet the team behind this:

Bismillah Kani
- https://dev.to/bismillahkani
- https://www.linkedin.com/in/bismillah-kani-49a31115/
Manoj Partibhan
- https://www.linkedin.com/in/manojparthiban/
Md. Mostafa Al Mahmud
- https://www.linkedin.com/in/md-mostafa/
- https://dev.to/md__mostafa
Omkar Kadam

Introduction to AWS CDK

bismillahkani — Mon, 26 Jun 2023 04:24:41 +0000

What is AWS CDK?

AWS CDK is an opensource software development framework to create and deploy the cloud infrastructure using your familiar languages like Python. AWS CDK make the infrastructure management fun and easy 🕺

Manual infrastructure Deployment

Let’s have a look into the process of deploying infrastructure manually. Typically we use the management console to create and deploy resources but sometimes we use CLI tools. For instance, to create a S3 bucket you log into the AWS account, click Create Bucket and fill the form. In addition to creation of resources, you have to manually create the bucket policy, ACL and IAM policy in order to integrate to other components in the application. This is fine to begin with and when the infrastructure requirements are minimal. When it comes to enterprise level infrastructure management where you have to manage multiple applications with multiple stacks that has many resources the manual process of deployment is not reliable nor consistent. It is error prone and maintenance of application infrastructure becomes nightmare.

From this example, we can understand that deploying and managing infrastructure manually for a fairly complex application is not a viable solution.

Infrastructure as Code (IaC)

Infrastructure as Code is the process of provisioning and managing the resources programmatically through machine readable definition files applying the same rigor as the application code development. For AWS the Infrastructure as Code is implemented using AWS CloudFormation. With AWS CloudFormation the application resources can be provisioned by writing the definition of resources in a cloud formation template either in YAML or JSON. For example, the following YAML template describes AWS S3 bucket to create.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  S3Bucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
    DeletionPolicy: Retain
  BucketPolicy:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      PolicyDocument:
        Id: MyPolicy
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Join 
              - ''
              - - 'arn:aws:s3:::'
                - !Ref S3Bucket
                - /*
      Bucket: !Ref S3Bucket

Cloud formation YAML template

AWS CloudFormation takes this template and will create, update and delete resources on your AWS account according to the description in the template.

Benefits of IaC:

Visibility - The template act as singe source of reference for the resources. Web console is not required.
Stability - Version controlled and rollback to previous stable version incase new changes break the application.
Scalability - Reusable and scales horizontally seamlessly.

How CDK works?

CDK is built on top of the AWS CloudFormation service and uses it as the engine for provisioning resources. AWS CDK uses the familiarity and expressive power of favorite programming languages for modelling your applications. The AWS CDK supports TypeScript, JavaScript, Python, Java, C#/.Net, and (in developer preview) Go. Developers can use their favorite programming languages to define reusable components called Constructs and compose several Constructs into Stacks and Apps.

Image credit: https://docs.aws.amazon.com/cdk/v2/guide/home.html

CDK Constructs

Constructs are basic building blocks of AWS CDK apps. Constructs are high level components that preconfigure cloud resources with proven defaults to make the infrastructure provisioning easier. AWS CDK provides a library of constructs that cover many AWS features and services. Developers can define the application infrastructure at high level using these constructs. Constructs are adjustable and composable and one can easily change the parameters and build custom constructs.

AWS Constructs library

The CDK constructs library includes all the resources that are available in AWS. For example, s3.Bucket class represents an AWS S3 bucket and dynamodb.Table represents an AWS DynamoDB table.

There are three level of constructs in this library,

L1 Constructs are the low level constructs that directly represents the resources in AWS CloudFormation. It is also called CFN Resources.
L2 Constructs are the next higher level constructs that are similar in function to L1 constructs but provide defaults, boiler plate and glue logic.
L3 Constructs which are called as patterns are designed to accomplish common tasks in AWS For example, aws-apigateway.LambdaRestApi construct represents an API Gateway backed by AWS Lambda function.

from aws_cdk import App, Stack
import aws_cdk.aws_s3 as s3
from constructs import Construct

class HelloCdkStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        s3.Bucket(self, "MyFirstBucket", versioned=True)

app = App()
HelloCdkStack(app, "HelloCdkStack")

CDK Stacks and Apps

The smallest unit of deployment in AWS CDK is called a stack. All of the resources defined within the scope of a stack is deployed as a single unit. You can define any number of stacks inside your AWS CDK app.

As an example, here we declare a stack class MyFirstStack that includes a single Amazon S3 bucket. However, this only declares a stack and the stack have to defined in some scope in order to be deployed.

class MyFirstStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        s3.Bucket(self, "MyFirstBucket")

app = App()

MyFirstStack(app, 'stack1')
MySecondStack(app, 'stack2')

app.synth()

CDK Toolkit

The AWS CDK Toolkit known as cdk is the command line tool is the primary tool to provision and manage AWS CDK app. It executes the app, generates and deploys the AWS CloudFormation templates.

Setup

AWS CLI - Make sure you have the latest version of the AWS CLI installed on your system.
AWS Account and User - You’ll need access to an AWS account. Create IAM user with programmatic access and attach AdministratorAccess policy to the IAM user.
Node.js - The AWS CDK uses Node.js (>= 10.13.0, except for versions 13.0.0 - 13.6.0).
VSCode - IDE
Python - 3.6 or later.
AWS CDK Toolkit

npm install -g aws-cdk

cdk --version

CDK commands

Command	Function
cdk ls	Lists the stacks in the app
cdk synt	Synthesizes and prints the CloudFormation template for the specified stack(s)
cdk bootstrap	Deploys the CDK Toolkit staging stack; see Bootstrapping
cdk deploy	Deploys the specified stack(s)
cdk destroy	Destroys the specified stack(s)
cdk diff	Compares the specified stack with the deployed stack or a local CloudFormation template
cdk init	Creates a new CDK project in the current directory from a specified template

Your first CDK project

Create project directory 📁

mkdir cdk_workshop 
cd cdk_workshop

Use cdk init to create a new Python CDK projects

cdk init sample-app --language python

Activating the Virtualenv. The init script created bunch of code to get started and also created a virtual environment. Activate the virtual environment.

source .venv/bin/activate

Install the Python modules.

pip install -r requirements.txt

Project directory

The application entry point is app.py.

#!/usr/bin/env python3

import aws_cdk as cdk

from cdk_workshop.cdk_workshop_stack import CdkWorkshopStack

app = cdk.App()
CdkWorkshopStack(app, "cdk-workshop")

app.synth()

The constructs are defined in cdk_workshop/cdk_workshop_stack.py

from constructs import Construct
from aws_cdk import (
    Duration,
    Stack,
    aws_sqs as sqs,
    aws_sns as sns,
    aws_sns_subscriptions as subs,
)

class CdkWorkshopStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        queue = sqs.Queue(
            self, "CdkWorkshopQueue",
            visibility_timeout=Duration.seconds(300),
        )

        topic = sns.Topic(
            self, "CdkWorkshopTopic"
        )

        topic.add_subscription(subs.SqsSubscription(queue))

Synthesize cloud formation template from the CDK app

cdk synth

This will output the following template

Resources:
  CdkworkshopQueue18864164:
    Type: AWS::SQS::Queue
    Properties:
      VisibilityTimeout: 300
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/Resource
  CdkworkshopQueuePolicy78D5BF45:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
        Statement:
          - Action: sqs:SendMessage
            Condition:
              ArnEquals:
                aws:SourceArn:
                  Ref: CdkworkshopTopic58CFDD3D
            Effect: Allow
            Principal:
              Service: sns.amazonaws.com
            Resource:
              Fn::GetAtt:
                - CdkworkshopQueue18864164
                - Arn
        Version: "2012-10-17"
      Queues:
        - Ref: CdkworkshopQueue18864164
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/Policy/Resource
  CdkworkshopQueuecdkworkshopCdkworkshopTopic7642CC2FCF70B637:
    Type: AWS::SNS::Subscription
    Properties:
      Protocol: sqs
      TopicArn:
        Ref: CdkworkshopTopic58CFDD3D
      Endpoint:
        Fn::GetAtt:
          - CdkworkshopQueue18864164
          - Arn
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/cdkworkshopCdkworkshopTopic7642CC2F/Resource
  CdkworkshopTopic58CFDD3D:
    Type: AWS::SNS::Topic
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopTopic/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.18.0,jsii-runtime=Python/3.7.3

The template contains the following resources:

AWS::SQS::Queue - queue
AWS::SNS::Topic - topic
AWS::SNS::Subscription - the subscription between the queue and the topic
AWS::SQS::QueuePolicy - the IAM policy which allows this topic to send messages to the queue

Bootstrapping the environment 💼

For the first time you deploy CDK app into an environment or region, a bootstrap stack has to be installed which includes the needed resources for toolkit operations.

cdk bootstrap

Deploy 🚀

cdk deploy

In the CloudFormation console, you will see that the stack has been created successfully.

Clean up

cdk destroy

How CDK improve developers experience?

Infrastructure and code in one place
Easy to define IAM policy - minimal code and minimal permissions
Refactoring code
Snapshot test - Compare the changes to previously successful state
Fine grained assertion tests - Test specific aspects of cloud formation templates
Validation tests - Validate rules and checks

Conclusion

AWS CDK is a game changer in Infrastructure as Code. It is a complete developer friendly tool to provision and manage all your cloud infrastructure resources.

References

Author

Bismillah Kani is a passionate and multi-faceted AI Scientist and Cloud Architect with a passion for deep learning, machine learning, and artificial intelligence. As an AWS certified professional he is constantly striving for excellence and innovation in the field of AI and Cloud.

Generative AI using AWS SageMaker Jumpstart

bismillahkani — Sun, 16 Apr 2023 10:26:53 +0000

Introduction
Stable Diffusion
AWS SageMaker Jumpstart
- Deploy a pre-trained Stable Diffusion model
- Fine-tune a Stable Diffusion model
Amazon Bedrock
Conclusion
References

Introduction

Generative AI has become an increasingly popular topic in the field of machine learning, with applications in areas such as computer vision, natural language processing, and creative arts. Unlike traditional machine learning, which focuses on finding patterns in data, generative AI aims to create new data that mimics the patterns found in the training data. This can be useful for tasks such as image and text generation, as well as for creating realistic simulations of real-world scenarios.

One approach to Generative AI that has shown promising results is Stable Diffusion. Stable Diffusion is a type of generative model that uses a diffusion process to transform a noise vector into a sequence of image-like samples. This approach has several advantages over other generative models, including improved sample quality and stability.

In this blog post, we will explore how to use Stable Diffusion for Generative AI on AWS SageMaker Jumpstart, a cloud-based platform for building, training, and deploying machine learning models. We will start by introducing Stable Diffusion and its advantages for Generative AI, then move on to exploring how to deploy pre-trained models and fine-tune models on custom collections of images. Finally, we will discuss how Amazon Bedrock can be used to democratize Generative AI and scale projects. By the end of this post, you will have a practical understanding of how to use Stable Diffusion on AWS SageMaker Jumpstart for your own Generative AI projects.

Stable Diffusion

Stable Diffusion is a generative model that uses a diffusion process to transform a noise vector into a sequence of image-like samples. Unlike other generative models such as Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs), Stable Diffusion does not require a discriminator or an encoder, making it simpler and more stable to train.

In Stable Diffusion, the noise vector is iteratively transformed by a diffusion process that adds noise to the current sample. The diffusion process is controlled by a diffusion coefficient, which determines the amount of noise added at each iteration. The resulting sequence of samples is then transformed back into a single image using a reverse process.

Stable Diffusion has several advantages over other generative models. One key advantage is its stability during training, which can be a challenge for GANs and VAEs. Stable Diffusion does not require a discriminator to be trained against the generator, which can lead to instability and mode collapse. Instead, Stable Diffusion directly optimizes the likelihood of the data given the noise vector.

Another advantage of Stable Diffusion is its ability to generate high-quality samples. By iteratively adding noise to the samples, Stable Diffusion generates images that have fine details and realistic textures. This can be especially useful for tasks such as image generation or style transfer.

Stable Diffusion has been applied to a variety of tasks, including image generation, video prediction, and text generation. It has also been used in creative applications, such as generating realistic paintings or music.

In the next section, we will explore how to use Stable Diffusion on AWS SageMaker Jumpstart for Generative AI.

High level steps of text-to-image generationRamesh et al. 2022

AWS SageMaker Jumpstart

AWS SageMaker Jumpstart provides a variety of tools and resources for developers and data scientists, including pre-built machine learning models, notebooks, and data sets. Jumpstart also includes a range of features for deploying and scaling machine learning models, such as automatic model tuning and model hosting.

SageMaker Studio

One of the benefits of using AWS SageMaker Jumpstart for Generative AI is the ability to quickly deploy pre-trained models. AWS provides a number of pre-trained models for Stable Diffusion, which can be used to generate high-quality images with minimal training. These pre-trained models can be deployed and used in a variety of applications, such as image generation or style transfer.

Another advantage of using AWS SageMaker Jumpstart for Generative AI is the ability to fine-tune models on custom collections of images. Jumpstart provides a range of tools for fine-tuning models, including automatic hyperparameter tuning and data augmentation. This allows developers and data scientists to quickly and easily train models that are tailored to their specific use case.

Overall, AWS SageMaker Jumpstart provides a powerful platform for building, training, and deploying Generative AI models. With its range of pre-built models, notebooks, and tools, Jumpstart makes it easy for developers and data scientists to get started with Generative AI and to build models that are tailored to their specific use case.

SageMaker Jumpstart in Studio

Deploy a pre-trained Stable Diffusion model

To deploy a pre-trained Stable Diffusion model on AWS SageMaker Jumpstart, follow these steps:

Open the AWS SageMaker Studio console and select the "Stable Diffusion 2.1 base" model from the list of pre-built models in SageMaker Jumpstart. Once you click on the model card you will directed to the deploy and train page where you can deploy a pre-trained model easily.
Choose the pre-trained Stable Diffusion model that best matches your use case. AWS provides a range of pre-trained models for Stable Diffusion, each of which is optimized for a specific type of generative AI task.
Once you have selected the pre-trained model, choose the instance type and number of instances that you want to use for deployment. AWS SageMaker Jumpstart provides a range of instance types, from small instances suitable for testing and development to large instances suitable for production deployments.
After selecting the instance type, configure the deployment settings for the pre-trained model. This includes settings such as the endpoint name, and other configurations. Once you have configured the deployment settings, deploy the pre-trained model to the selected instance type. AWS SageMaker Jumpstart will automatically provision the necessary resources and deploy the model to the specified instance type.
After the model is deployed, you can use the AWS SageMaker Jumpstart console to test the model and generate images. AWS provides a range of tools for testing and evaluating generative AI models, including tools for visualizing the generated images and evaluating the quality of the generated output.

Overall, deploying a pre-trained Stable Diffusion model on AWS SageMaker Jumpstart is a straightforward process that can be completed quickly and easily. With a range of pre-built models and deployment options, AWS SageMaker Jumpstart provides a powerful platform for building and deploying generative AI models.

Fine-tune a Stable Diffusion model

In addition to deploying pre-trained models, AWS SageMaker Jumpstart also provides tools for fine-tuning Stable Diffusion models on custom collections of images. This can be useful for tasks such as generating images that are specific to a particular use case or domain. In this section, we will explore how to fine-tune a Stable Diffusion model on AWS SageMaker Jumpstart for a custom collection of images.
Before fine-tuning a Stable Diffusion model on a custom collection of images, you will need to prepare the data. This includes selecting a set of high-quality images that are representative of the domain you want to generate images in. For this example, we will fine-tune the model on a set of Koala bear images.

Collection of Koala images for fine-tuning

To fine-tune a Stable Diffusion model on a custom collection of images, you can use the example notebook provided by AWS SageMaker Jumpstart. This allows you to fine-tune pre-trained models on your custom collection of images. Clone the notebook, create a directory ‘training_images’, upload the Koala bear images to the directory, change the ‘use_local_images’ = True and run all cells. SageMaker spins up a training job for fine-tuning the model on the Koala bear images. After the training is completed, the model can be deployed for generating new images.

Once you have fine-tuned the Stable Diffusion model on your custom collection of images, you can use it to generate new images.

Overall, fine-tuning a Stable Diffusion model on AWS SageMaker Jumpstart is a powerful tool for generating high-quality images that are tailored to your specific use case. By selecting a custom collection of images and fine-tuning the model, you can generate images that are specific to your domain and that meet your specific requirements.

Amazon Bedrock

Amazon Bedrock is a new service that offers customers access to powerful foundation models (FMs) from various providers through an API. It democratizes access to FMs, making it easier for customers to build and scale generative AI-based applications. It allows customers to find the right model for their needs, customize FMs with their own data, and integrate and deploy them using AWS tools without managing any infrastructure. Bedrock is a scalable, reliable, and secure AWS managed service. It simplifies the development of generative AI applications without the need for managing physical infrastructure or high operational overhead.

Building with Generative AI on AWS

Conclusion

In this blog post, we explored Stable Diffusion, a type of generative model that uses a diffusion process to transform a noise vector into a sequence of image-like samples. We discussed its advantages over other generative models, including improved stability and sample quality. We then moved on to explore how to use AWS SageMaker Jumpstart, a cloud-based platform for building, training, and deploying machine learning models, for Generative AI. We discussed how to deploy pre-trained models and fine-tune models on custom collections of images. Finally, we discussed Amazon Bedrock, a new service that offers customers access to powerful foundation models (FMs) from various providers through an API, making it easier for customers to build and scale generative AI-based applications. With these tools, developers and data scientists can quickly and easily build and deploy Generative AI models that are tailored to their specific use case.

References

"Stable Diffusion 2.1." GitHub, https://github.com/ermongroup/stable-diffusion
"Amazon SageMaker Jumpstart." AWS, https://aws.amazon.com/sagemaker/jumpstart/
"Amazon Bedrock." AWS, https://aws.amazon.com/bedrock/
Text to image on AWS SageMaker Jumpstart, https://aws.amazon.com/blogs/machine-learning/generate-images-from-text-with-the-stable-diffusion-model-on-amazon-sagemaker-jumpstart/
Fine tune text to image on AWS SageMaker Jumpstart, https://aws.amazon.com/blogs/machine-learning/fine-tune-text-to-image-stable-diffusion-models-with-amazon-sagemaker-jumpstart/

Disclaimer: This blog post was generated with the assistance of AI tools. While the author has edited and reviewed the content, some portions of the article may have been generated or suggested by the AI tool. The opinions and views expressed in this post are solely those of the author.

Forem: bismillahkani

Architecting MLOps for Computer Vision using AWS Sagemaker

What is MLOps?

Use case

MLOps pipeline

MLOps Solution Architecture

CI/CD pipeline

Orchestrating Data Labelling

Orchestrating Model Training

Model deployment

Summary

References

When Generative AI meets AWS Community Builders : Unveiling the Ultimate AWS Cost Advisor

Introduction

Problem Statement

Available Solutions:

Retrieval-Augmented Pipeline

High level solution overview

Performance Comparison with Examples

How we're different?

Real-life use-cases

Conclusion (Why we should win)

Introduction to AWS CDK

What is AWS CDK?

Manual infrastructure Deployment

Infrastructure as Code (IaC)

How CDK works?

CDK Constructs

AWS Constructs library

CDK Stacks and Apps

CDK Toolkit

Setup

CDK commands

Your first CDK project

How CDK improve developers experience?

Conclusion

References

Author

Generative AI using AWS SageMaker Jumpstart

Table of Contents

Introduction

Stable Diffusion

AWS SageMaker Jumpstart

Deploy a pre-trained Stable Diffusion model

Fine-tune a Stable Diffusion model

Amazon Bedrock

Conclusion

References