Forem: NaDia

Building Enterprise-Ready AI Agents: Key Takeaways from AWS re:Invent 2025

NaDia — Thu, 04 Dec 2025 23:58:39 +0000

If you didn’t have a chance to attend AWS re:Invent this year, don’t worry. While key sessions will be available online, here is a concise summary of one of the standout sessions I attended at #reInvent2025.

All credit to AWS and the presenters of this session.

“Agents in Enterprise: Best Practices With Amazon Bedrock AgentCore”

Moving from POC to production with AI agents is rarely straightforward. Challenges arise around accuracy, scalability, latency, infrastructure costs, model inference expenses, security, observability, and memory retention. Many teams jump straight into building agents without planning where to start and how to operationalize an agentic platform at enterprise scale.

This session distilled nine core best practices for building robust, production-ready Agentic systems.

🔹 Top 9 Best Practices for Agentic Platform Success

1. Start Small & Work Backwards

Agent development is an interactive journey, you can adopt new models, add tools and improve prompts. Define what the agent should and shouldn't do, with clear and complete definitions and expected.

2. Implement Observability from Day One

Agents are OTEL compatible. Enable full trace-level visibility and observability dashboards early, not later.

3. Define Your Tooling Strategy Explicitly

Document tool requirements, input/output schemas, and error-handling logic.
Reducing ambiguity reduces tokens and costs. Leverage existing MCP servers and expose tools via the MCP server and show integration patterns with code samples.

4. Automate Evaluation

Define technical and business metrics early and include business users in the evaluation loop. Test across diverse user intents including misuse patterns to strengthen resilience.

5. Avoid the “One Agent With 100 Tools” Anti-Pattern

Use multi-agent architectures with clear roles, orchestrated workflows, and shared context.
Monitor how agents collaborate and escalate tasks.

6. Establish Proper Memory Boundaries

Plan for:
•short-term session memory
•long-term personalised memory

Isolate user context and enforce security policies at execution. Host agents and tools separately for compliance and performance.

7. Cost vs. Value: Be Pragmatic

If deterministic code works reliably, use it. Reserve agent reasoning for tasks that actually require reasoning rather than forcing agents into everything.

8. Test Relentlessly

Rerun evaluation after every update.
Use:
• A/B deployments
• drift monitoring
• automated rollback

Production monitoring is not optional, it’s mandatory.

9. Scale Through Platform Standardisation

Deploying agents to production is step one, not the finish line.
To scale safely:
•Build a central platform team for enablement
•Standardise governance, observability, and tooling
•Promote cross-team collaboration to avoid duplicated effort

The session showcased an excellent org model outlining split responsibilities between platform vs. use-case teams.

So Where Does AgentCore Fit In?

Amazon Bedrock AgentCore Operationalises these best practices out-of-the-box, enabling enterprise-grade agent development at scale.

Key Capabilities Overview:

Runtime: Supports any agent framework, prompt schema, tool routing & context injection.
MCP & A2A Compatible: Seamless interoperability between agents and MCP servers
Memory Layer: Persistent and session-based memory for personalisation.
Tooling: Catalog + governance + reuse capability. Define MCP servers, use AgentCore Browser Tooling for safe web navigation and data extraction. And Code Interpreter to execute code securely in isolation when needed.
Identity & Access Control: Ensures the right agent accesses the right tool securely.
Policy Enforcement: Applies organisational rules & compliance guardrails.
Evaluation Engine: Built-in testing and performance assessment with customisable metrics.

Final Takeaway

This session perfectly reinforced that building agents is not just about prompting, it’s about engineering:
• platform standardisation
• tooling governance
• secure orchestration
• memory boundaries
• rigorous evaluation
• enterprise scalability

AgentCore becomes the backbone that enables all of this, from experimentation to full-scale production with observability, governance, and operational safety built in.

Amazon Bedrock Blueprint: Architecting AI Projects with Amazon Bedrock

NaDia — Mon, 29 Apr 2024 12:57:29 +0000

Initial Words

If you're actively involved in the AI field and utilise AWS Cloud services, chances are you've explored Amazon Bedrock to enhance your applications with AI capabilities. Even if you haven't directly worked with it, you've likely heard about the advanced Foundational Models that Amazon Bedrock offers. In this blog post, I'll provide a comprehensive introduction to Amazon Bedrock components and delve into common workflows for integrating Amazon Bedrock into Generative AI projects.

Amazon Bedrock Components

Exploring various articles on Amazon Bedrock will give you enough information about its nature. As you may be aware, Amazon Bedrock offers a quick serverless experience, granting access to an extensive array of Foundational Models. Its unified API is especially noteworthy, as it streamlines the integration of these diverse models into your system.
However, the question remains: how does Amazon Bedrock achieve this? What components does it comprise that set it apart from other AI platforms or services? This is the exploration we aim to undertake in this section.

Foundational Models

Yes, Amazon Bedrock offers a wide range of Foundational Models like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon. But what's the advantage of using these models within Bedrock? You're not restricted to just one specific model. Bedrock's API makes it easy to integrate these models. If you decide to switch from Mistral to Cohere models, all you need to do is change the model ID within your "InvokeModel" API from Bedrock. Additionally, if your system needs to integrate with multiple models, Bedrock's API layer allows you to invoke as many models as you need in parallel but completely isolated from each other.

Knowledge Base

Using Foundational Models alone has limitations. However, there are effective approaches to overcome these limitations, which I'll discuss in the "AI Workflows With Bedrock" section. It's still important to understand these limitations such as:

Outdated information
Lack of knowledge on your data set.
Lack of transparency on how they arrived at specific answers.
Hallucination

I'm sure you're aware that Foundational Models are trained on vast amounts of data, but there's no guarantee they're always up to date with the latest information. These models provide answers without providing the source links for the context they used. The answers they give are very general, and if you want them to be based on your company's specific data, you'll need to retrain them using that data. However, if your data is constantly changing, continuously retraining the models can be computationally intensive and expensive. Additionally, by the time you finish retraining the model, your company may have already generated new data, making the model's information outdated.

To address issues such as providing source links or offering more specific domain-related answers, Amazon Bedrock offers the "Knowledge Base" component. This feature provides additional data access during runtime.
Using the Knowledge Base, you can create a RAG (Retrieval Augmented Generation) application that utilizes the "RetrieveAndGenerate API" to fetch information from your Knowledge Base (KB) and generate responses. Alternatively, you can build a basic RAG Application with the "Retrieve API", which retrieves information from the knowledge base and presents it to the user along with the source link.

Beyond answering user queries, a KB can augment prompts for Foundational Models by adding context to the prompt. This adds RAG capability to Agents for Amazon Bedrock.

Agents For Bedrock

The Amazon Bedrock Knowledge Base (KB) handles data ingestion, while agents manage the Retrieval Augmented Generation (RAG) workflow. Agents for Amazon Bedrock automates prompt engineering and the organization of user-requested tasks.

Agents can perform various tasks, including:

Take actions to fulfill the users request.

Agents have predefined Action groups, which are tasks they can autonomously perform. Each Action Group comprises some Lambda functions and API Schema. A crucial aspect of Schema Definition is the description of each Endpoint in your Schema. These descriptions can act as prompts to your Agent, helping it understand when to use which API Endpoint.
Break down complex user queries for Foundational Model.

Agents assist Foundation Models in comprehending user requests. In the upcoming workflow explanations, I will delve deeper into how Agents employ ReAct strategies to analyse user requests and determine the actions that Foundation Models should take to fulfill those queries.
Collect additional information

When you create an Agent for Amazon Bedrock, you can configure Agent to collect additional information from user through natural language conversation.

Common AI Workflows With Amazon Bedrock

Having explored all the components of Amazon Bedrock, let's now delve into the most common patterns for integrating Amazon Bedrock into our Generative API applications. Knowing the common blueprints will help us to identify when a specific service is a good addition to our architecture and which blueprint is a good candidate for our use-case.

Beginning with standard workflow to only use Amazon Bedrock API that invokes different models using "InnvokeModel" API.
This invocation can be initiated either by an event within you AWS account or through Application API.

In an event driven workflow, the Model Invocation can occurs by S3 notifications when a file is uploaded to a specific S3 Bucket. This might be necessary when new files are uploaded to the bucket, and you want to summarise the document using Amazon Bedrock Foundational Models.

You could also set up your Application API with an AWS Lambda Function that triggers a Foundational Model. For instance, it could generate text based on a user-provided topic or describe an image uploaded by the user.

This approach may appear simplistic, but that's the essence of utilising Amazon Bedrock as an API abstraction layer in your Generative AI application. Despite its simplicity, this method can yield effective responses and enhance answer quality using common techniques like Prompt engineering.

The next pattern I'd like to discuss involves creating RAG applications using Knowledge Base, which blends prompt engineering techniques with retrieving information from external sources.

To set up a RAG workflow, begin by creating a Knowledge Base in Amazon Bedrock. This involves specifying the S3 Bucket containing your external resources, determining the document chunk size, selecting the Embedding model to generate vectors for the dataset, and selecting a Vector Database to store the indexes, like Amazon OpenSearch. Setting the chunk size is crucial as it leads to finer embeddings, enhancing retrieval accuracy, and prevents overloading the model's context window with large source documents.

Similar to most of AI powered workflows, this one also starts with user input prompt. RAG uses the same embedding model to create a vector embedding representation of the input prompt. This embedding is then used to query the Knowledge Base for similar vector embeddings to return the most relevant text as the query result. The query result is then added to the prompt, and the augmented prompt is passed to the FM. The model uses the additional context in the prompt to generate the response to the user query.

Like many AI-powered workflows, this one begins with a user input prompt. RAG uses an embedding model to create a vector representation of the input prompt. This vector is used to search the Knowledge Base for similar vectors, returning the most relevant text as the query result. The query result is then combined with the prompt and passed to the FM. The model uses the augmented prompt to generate a response to the user's query.

Ever since I was young, I've saved the best for last. Let's talk about the Amazon Bedrock workflow with Agents. Here, you can surpass limitations by combining all Amazon Bedrock components, from the Knowledge Base to your company's APIs, to empower the Model to generate robust answers.

In an earlier section, I mentioned Agents extend FMs to understand user requests by breaking down complex tasks into multiple steps. This process occurs during the Pre-Processing phase. When an Agent receives a user request, it first analyses the request using ReAct Technique (Reason, Action, Observation).

During this phase, the Agent:

Reasons on the user query to understand the task at hand, determining whether it needs to call an API or access a Knowledge Base for information.
Takes action to fulfill the request by executing the necessary steps.
Returns the observation or results after completing the actions. It then incorporates this information into the input prompt, providing the model with additional context (Augmented Prompt).

Final Words

Understanding the structure of Amazon Bedrock and the typical architectures for integrating it into Gen AI applications can help us make more informed decisions about which components to use to achieve our goals. However, it can be challenging to determine the best workflow, especially for those new to the Gen AI field.

For simpler tasks that involve historical data, a standard approach with strong Prompt Engineering techniques is often effective. In contrast, for more complex tasks or when responses need to be specific to your dataset, leveraging Fine Tuning within Amazon Bedrock can be beneficial.

When the model requires external data resources to fulfill user requests, using a Knowledge Base or a combination of a Knowledge Base and an Agent can be helpful. A Knowledge Base workflow is suitable for relatively static data such as company documents or FAQs, while an Agent with a Knowledge Base is better for dynamic information like databases or APIs.

There is no one-size-fits-all solution, but the flexibility of Amazon Bedrock allows for various approaches to achieve the same result. The key is to choose the right approach for the task to achieve optimized results at minimal cost.

I hope you found this article useful. In the next part, I will demonstrate the most advanced workflow, where we will use an Agent with APIs and a Knowledge Base to create a Tourist and Travel Assistant using Amazon Bedrock providing all the code snippets and code repository for your reference.

CDK Stack Notification Options

NaDia — Thu, 07 Mar 2024 09:18:45 +0000

Today, I discovered yet again that there are countless ways to tackle a single task as a developer.

I was tasked with automating a workflow that involved an AWS Lambda Function triggered by an SNS event source. The goal was to publish a message to an SNS topic in a different AWS account when the status of a CloudFormation stack updated.
We use AWS CDK for infrastructure as code (IAC). While exploring the documentation and blog posts, I found that there is no direct equivalent of the Notification Policy in CloudFormation to publish notifications to an SNS topic on a CloudFormation stack status change. Instead, there are several common patterns to achieve this. Let's start with solution diagram. Here's a simplified version of the architecture diagram of what I implemented:
If this is what you are looking for, you can simply achieve it in 3 different ways:

Using AWS Event Bridge

Create SNS Topic as a Stack B CDK resource:

import { Topic } from "aws-cdk-lib/aws-sns";

   const SNSTopic = new Topic(this, "SNS_TOPIC_ID", {
      displayName: "YOUR DISPLAY NAME",
    });

_Note: you need to add an event source to your function or any other resources that is going to subscribe to this topic. in my case I needed to configure a lambda event source as _bellow:

MyFunction.addEventSource(new SnsEventSource(SNSTopic));

Add Event Rule

 new Rule(this, "Trigger", {
      eventPattern: {
        source: ["aws.cloudformation"],
        detailType: ["CloudFormation Stack Status Change"],
        detail: {
          eventName: ["CREATE_COMPLETE", "UPDATE_COMPLETE", "DELETE_COMPLETE"],
          requestParameters: {
            stackName: [this.stackName],
          },
        },
      },
      targets: [new SnsTopic(SNSTopic)],
    });

Using AWS Custom Resources

Second approach to achieve this is by using AWS Custom Resources.

Create an AWS Custom Resource within the Stack B CDK:

import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from "aws-cdk-lib/custom-resources";    

const Trigger = new AwsCustomResource(this, "TriggerOnSuccess", {
      onUpdate: {
        service: "SNS",
        action: "publish",
        parameters: {
          TopicArn: "YOUR_TOPIC_ARN",
          Message: "Stack updated successfully",
        },
        physicalResourceId: PhysicalResourceId.of("TriggerOnSuccess"),
      },
      onDelete: {
        service: "SNS",
        action: "publish",
        parameters: {
          TopicArn: "YOUR_TOPIC_ARN",
          Message: "Stack deleted successfully",
        },
        physicalResourceId: PhysicalResourceId.of("TriggerOnSuccess"),
      },
       policy: AwsCustomResourcePolicy.fromStatements( [new PolicyStatement({
        actions: ["sns:Publish"],
        effect: Effect.ALLOW,
        resources: [SNSTopic.topicArn],
      })]),
    });

Add Dependency order so that CDK doesn't return Dependency Cycle error

 Trigger.node.addDependency(YOUR_FUNCTION);
 Trigger.node.addDependency(SNSS_TOPIC);

Using AWS Custom Resource with Lambda Invoke Action

There is also a 3rd solution for this as well which I am not a big fan of it and that is. I personally prefer to use fan out approach, to populate events to Lambda via an "Event Service" such as SNS or EventBridge. If you look for a simplified Custom Resource, here is what you should update your AWS Custom Resource to:

import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from "aws-cdk-lib/custom-resources";    

const Trigger = new AwsCustomResource(this, "TriggerOnSuccess", {
      onUpdate: {
        service: "Lambda",
        action: "invoke",
        parameters: {
          FunctionName: "YOUR_FUNCTION_ARN",
          InvokationType: "Event"
        },
        physicalResourceId: PhysicalResourceId.of("TriggerOnSuccess"),
      },
      onDelete: {
        service: "Lambda",
        action: "invoke",
        parameters: {
          FunctionName: "YOUR_FUNCTION_ARN",
          InvokationType: "Event"
        },
        physicalResourceId: PhysicalResourceId.of("TriggerOnSuccess"),
      },
       policy: AwsCustomResourcePolicy.fromStatements( [new PolicyStatement({
        actions: ["lambda:InvokeFunction"],
        effect: Effect.ALLOW,
        resources: [YOUR_FUNCTION_ARN],
      })]),
    });

This concludes our brief discussion. While I'm still hopeful about discovering if CDK offers an API for configuring Stack Notification Options, I wanted to share these workarounds in the meantime.

Optimising Sentiment Analysis Workflows: AWS Zero-ETL and Amazon Redshift Synergy-Part 1

NaDia — Sun, 04 Feb 2024 05:01:39 +0000

Introduction

Being a passionate advocate for Machine Learning within Data Warehouses, I find the most intriguing aspect of this solution to be its ability to alleviate data fragmentation. By incorporating ML into your Data Warehouse, you centralize your data rather than dispersing fragments across various storage options such as Amazon S3 buckets or Azure Blob storage simply to enable accessibility for your machine learning tools. This represents just one of the numerous benefits that come with implementing ML in a Data Warehouse.

Data Warehouse Your New Data Lab!

Today, leading cloud providers have made it super simple to venture into Machine Learning right from your Data Warehouse. Amazon Redshift boasts the Amazon Redshift ML feature, Azure Synapse Analytics buddies up with Azure ML, and Google BigQuery is your go-to for ML adventures. Even Snowflake is in on the action, offering SQL-based ML magic with SnowPark ML.

At AWS:Reinvent, AWS made a big deal about Zero ETL Integration with Amazon Redshift and tossed in LLM models into Redshift ML. This seriously boosted the whole idea!

In this blog post series, we'll explore the details of Zero ETL Integration with Amazon Redshift. Part one kicks off with connecting Amazon Aurora to Amazon Redshift, facilitating almost real-time data interactions in the Data Warehouse. Subsequently, in Part 2, we will harness this data for insightful analyses using advanced LLM models now accessible in Redshift ML.

Simplify Data Movement With AWS Zero ETL Integration

The ETL process (Extract, Transform, Load) is super important to prepare the data for a central Data Warehouse. This means Gathering, Cleaning, Normalising, and Combining data from different sources to make sure it's all set for use in the downstream system.

Traditional ETL can be a bit of a hassle. It can cost a lot, be tricky to set up, and take a while to get the data ready.
What if your primary goal is to provide your Data Analytics team with immediate access to the data?
Zero-ETL integration is a fully managed solution. It has been designed to get your transactional or operational data into Amazon Redshift Data Warehouse almost in real time. As a fully managed solution it handles all the hard work on its own, making sure the data is secure and reducing the complexity of setting up the ETL Data Pipeline.

Aurora Zero ETL Integration With Amazon Redshift

Aurora is designed more for online transactional processing rather than analytics. When handling extensive analytics queries, its performance can drop noticeably. A common practice is to have a primary database cluster and a read replica for analytics to improve performance.

To tackle performance challenges, an advanced solution is also to set up an ETL data pipeline. You might opt for Amazon Data Migration Service (DMS) to move data to S3, utilise AWS Glue for ETL jobs, or employ Amazon EMR for distributed ETL and ML tasks. Afterwards, you can load the transformed data or model artifacts back to S3 and store the refined data in Redshift for analytics. With multiple steps involved, the concept of Zero ETL steps in, offering innovative solutions to simplify the process.

Set up Zero ETL Integration

Considerations

Before creating Zero ETL Integration in your Aurora database or any other databases that support this feature, it is essential to verify that the Aurora MySQL/PostgreSQL versions are compatible and indeed support Zero ETL Integration. To get the full list of prerequisites please check out here.

To Set up Zero ETL Integration for your Aurora Source Database simply follow these steps:

Create a Custom DB Parameter Group

First step to start with Aurora Zero ETL is to create a Custom DB Parameter Group that controls replication and associate it with your Aurora DB cluster.

Once the cluster parameter group is created successfully, select the custom group and modify the values for each parameter as per bellow and hit save changes. In addition, Make sure binlog_row_value_options parameter is unset.

  binlog_backup=0
  binlog_replication_globaldb=0
  binlog_format=ROW
  aurora_enhanced_binlog=1
  binlog_row_image=full
  binlog_row_metadata=full
  binlog_transaction_compression=OFF

Create Aurora Source Database

If your Amazon Aurora DB Cluster are not already set up, the next step is to create Aurora Source Database instance. You can simply follow the instruction. A few important notes to consider:

1- To make sure the Aurora MySQL version set to 3.05.0 or higher.

2- Change the default DB cluster parameter group to the custom parameter group that you created in the previous step.

3- To apply changes when associating the parameter group with the DB cluster after creating the cluster, you'll need to reboot the primary DB instance in the cluster before initiating a zero-ETL integration.

For demonstration purposes I have downloaded a public dataset called Consumer Reviews of Amazon Products from Kaggle and stored it into S3 Bucket. At the time of creating my Source DB Cluster I chose to restore data from S3. If you chose to restore data from S3 Bucket make sure you have given Aurora permissions to get objects from your S3 Bucket.

Set up Redshift Serverless

Next step is to set up Amazon Redshift Serverless Workgroup and NameSpace to use as our target data warehouse.

When Redshift NameSpace and WorkGroup are ready, for Zero ETL integration to be successful we must enable the enable_case_sensitive_identifier parameter. To enable case sensitivity on a Redshift Serverless workgroup run this AWS CLI command:

aws redshift-serverless update-workgroup \
  --workgroup-name <YOU_REDSHIFT_SERVERLESS_WORKGROUP> \
  --config-parameters parameterKey=enable_case_sensitive_identifier,parameterValue=true

One last step before creating a Zero ETL integration is to add our Aurora Source DB as an authorised integration source to the namespace. This allows the Aurora Source DB to update our Amazon Redshift data warehouse. For that go to the Resource Policy tab and Add the ARN of the Aurora source DB as authorised integration source. We also need to add our AWS Account ID as authorised principal for Amazon Redshift.

Create Zero ETL Integration

To make this showcase easy to follow, I am using clicks up approach instead of setting up the integration using AWS SDK.

To create an Aurora zero-ETL integration with Amazon Redshift simply follow these steps:

Review the configuration and select create.

It takes approximately 30 minutes for the integration to be active. When the integration is successfully created, the status of the integration and the target Amazon Redshift data warehouse both change to Active.

Create Destination DataBase

After successfully creating a zero-ETL integration, we must create a destination database within the target Amazon Redshift workgroup. I do it by using the query editor v2 by simply running the following SQL command:

CREATE DATABASE <DESTINATION_DB_NAME> FROM INTEGRATION '<INTEGRATION_ID>';

To get the integration ID, navigate to the integration list on the Amazon Redshift console.

As simple as that! Now it's time to test the Zero-ETL integration in action.

How Zero ETL Works

When Zero ETL integration is created, It first loads the existing data from source database to target Data Warehouse, then starts streaming transactional data into Amazon Redshift Destination Database.
Let's test the Zero ETL Integration by adding Data to our Aurora MySQL Source DB.

I use MySQL Workbench to connect to my Aurora instance and load new dataset into my table. As soon as new data is updated in the Aurora source database, we can query the destination table in Amazon Redshift and get the data back.

Note: If you have issues with connecting to your RDS instances I recommend to follow this page for troubleshooting. Also, If your RDS Clusters are inside a VPC make sure you have correct inbound policy attached to the security group.

Now that we have the product feedback data available in Amazon Redshift, we can leverage pre-trained publicly available LLMs from Amazon Sagemaker JumpStart in Amazon Redshift ML to summarise feedback, perform entity extraction, sentiment analysis and product feedback classification.

Final Words

To wrap it up I would like to review some advantages of employing Zero ETL integration include:

Data is seamlessly available in Redshift.
Enables us to run near real time analytics, visualisation and ML on the data without impacting the production workloads.
With Zero ETL we don’t need to build and maintain complex Data pipelines to perform ETL operations. Still there are lost of other use cases to create and maintain a data pipeline but if it is specifically to run analytics processing, it’s convenient to use zero ETL integration.
Zero ETL with Redshift is provided at no additional cost.
We can create integration from multiple source Databases into a single Redshift warehouse.
We can have end to end serverless solution with Aurora serverless and Redshift serverless.
There is consistent monitoring on Zero ETL Integration, it detects when data tables need to be reseeded. When integration need to be Fixed or Recovered, and it’s healed automatically.
Also Redshift sends integration related events to Amazon EventBridge.

Zero ETL is a one step easy and secure way to enable near real time analytics on transactional or operational Data. Also, The new Super data type has advanced the Amazon Redshift ML capabilities, allowing integration of large language models (LLM) from SageMaker JumpStart for remote inferences. Combining these two features will empower us to create an end to end robust ML/AI solution faster but cost effectively.
Keep an eye out for Part 2, where we leverage an LLM Model in Redshift for immediate sentiment analysis on our dataset.

Do you believe AI will replace your job?

NaDia — Thu, 14 Dec 2023 05:55:45 +0000

Introduction

A few weeks ago, I chatted with a lady at the bus station. We both work in tech and share an interest in it. When we talked about our passion for technology, she asked, "Do you think AI will replace your job?". This question often arises in discussions about AI and ML. In my view, AI and ML won't replace us; they'll boost our abilities, making medical diagnoses more accurate, strengthening security, and improving overall work efficiency.

Am I worried about AI taking over my job after AWS Re:Invent last week, where they introduced services like Amazon Q and Amazon CodeCatalyst?

Should I be concerned about my job as a Developer with these AI services? Let's explore together and learn more about them.

You probably know SST Framework and Svelte Kit – nothing groundbreaking. Despite not being an expert in Svelte and SST recently, I decided to experiment with these tools, combining SST and Svelte with Amazon Q and Amazon CodeWhisperer using the AWS Toolkit in the IDE.

In this blog post, I'll share my thoughts on this setup and how I found these services through the Amazon Toolkit. You'll also find out if we should be concerned about AI assistants taking over our jobs as Developers.

An Introduction to Svelte, SST, Amazon Q and Amazon CodeWhisperer

Before diving into writing code and utilising various services from the Amazon Toolkit in VSCode, it's worth taking a quick look at what we're about to use, especially if you're not familiar with them.

Svelte

Svelte is a JavaScript tool for constructing UI components, similar to other UI frameworks like React and Vue. However, what sets Svelte apart is that it functions as a compiler, transforming the code into a form compatible with native browser APIs.

SST Framework

SST is an open-source framework designed to facilitate the development and deployment of Serverless stacks on AWS. It operates under the hood by integrating with Amazon CDK. However, its primary benefit is in allowing us to concentrate on creating resources using familiar languages like TypeScript, treating them as Infrastructure as Code (IaC).

Amazon Q

Imagine OpenAI's ChatGPT, Microsoft's Copilots, and now, Amazon has introduced its own AI assistant called Amazon Q.

There are different ways we can use Amazon Q. You can connect it to your business data for a personalised touch to suit your specific nee
ds like a chatbot that is customised for your business.

It's also accessible within the AWS console as an expert who can give you suggestions on architecture and solution designs with best practices. Amazon Q seamlessly integrates with Amazon Insight. This integration assists with visualisations, data analysis, and answering any data-related questions you may have. More importantly, as a developer, you can leverage Amazon Q within your IDE for code improvements, debugging and troubleshooting. That's the use case I am going to focus on in this blog.

Amazon CodeWhisperer

CodeWhisperer is a GenAI-powered tool that assists with code recommendations based on Pseudocode or existing codes. It can be configured within your IDE or through the command line. Extending its capabilities, CodeWhisperer is compatible with certain AWS services, including Amazon SageMaker Studio and Amazon Glue Studio.

You can utilise Amazon CodeWhisperer at no cost by configuring it within your IDE, Authenticate and you're ready to start leveraging its features.

Experimenting with Amazon Q and Amazon CodeWhisperer

I built a full-stack application using React, NextJs, and AWS Serverless. Check out the blog post for a guide on incorporating ML features with Amazon Bedrock into your application. In this part, I'll explain how I used Amazon Q and CodeWhisperer to rebuild the story generator lambda function in my app. Since I wasn't familiar with the SST framework and Svelte initially, I did some reading and watched tutorials before using Amazon Q to assess responses or CodeWhisperer's code recommendations.

Amazon Q For Analysing And Explanation

I began by asking Amazon Q about the SST framework, hoping for a quick and helpful response.

Next, I am keen to know how SST integrates with Svelte.

This convinces me. Now, let's find out how to get started with Svelte. I asked Amazon Q for guidance.

I ran the suggested command, and the Svelte app was created successfully. Then, I asked Amazon Q to explain the structure of a Svelte project.

Before diving into this experiment, I studied the concept of using SST as a Serverless stack with a UI framework like Svelte. The key idea is to have a mono repo, a recommended structure that makes all resources accessible to the Frontend.

Understanding these fundamental concepts, even with an AI assistant, helps in asking accurate questions and receiving precise answers. This approach prevents blindly accepting the assistant's recommendations.

Next, I'll use Amazon Q to inquire about creating an SST Stack and initialising a Svelte app within an SST app.

I like how Amazon Q gives a source link for each part of the answer and suggests follow-up questions as you can see in the screenshots. In this case, I found the SST app creation command in the source. After a few minutes, the SST app was successfully created. To follow best practices, I'll move the Svelte app to the my-sst-app/packages folder, turning the project into a mono repository. Alternatively, you can create a new Svelte app within the SST App.

Amazon Q & CodeWhisperer For Coding

Now, with our SST and Svelte apps set up, let's try out Amazon CodeWhisperer and request an explanation of the code from Amazon Q. We'll use Amazon CodeWhisperer to write the initial lambda function to generate a story with Amazon Bedrock.

Here's the pseudo-code for the lambda function:

When using Amazon CodeWhisperer, wait for suggestions based on your pseudo code, then choose the most relevant one. Alternatively, start typing, and it will provide recommendations. Proceed line by line to complete the lambda function, and I will analyse the suggestions once the lambda implementation is finished.

To bring in the bedrock client in my SST App, I referred to Amazon Q for guidance.

Extract and validate the topic from the lambda event

Construct Prompt

Construct Payload

Initialise Bedrock Client and invoke the model

Amazon Q For Debugging

Close to completing the lambda implementation. After applying the suggested code for model invocation, I had to import InvokeModelCommand from Amazon Bedrock. Let's ask Amazon Q about importing the module.

If I follow the suggestions, I'll encounter an import resolution error. Now, I'm using Amazon Q for debugging.

I've used the Amazon Bedrock SDK before, and I know that InvokeModelCommand is in the client-bedrock-runtime package. Despite tweaking my question for specificity, the response continued to reference the client-bedrock package. Just to be thorough, I asked the same question to ChatGPT and received a similar response.

The answer seems a bit misleading. As I followed the AI assistant's suggestions for implementing the lambda function, more errors occurred. For example, I had to import the Bedrock Client from client-bedrock-runtime, and the suggested payload was incorrect. The AI couldn't guide me on how to destructure the generated text response from the Text Model. After fixing these issues, the lambda function for generating the story is now implemented properly.

So far, I used Amazon Q and CodeWhisperer for various tasks like general knowledge questions, coding, and debugging. This experiment continued, I explored using Amazon Q to integrate an AppSync API resource into an SST Stack, deploying the stack, and creating a simple form with a single input field in Svelte. To keep this post concise, I'll conclude my documentation of this experiment here. In the next part, I'll share my thoughts on how the experiment went.

How Did This Experiment Go?

My experiment with Amazon Q and CodeWhisperer wasn't limited to a lambda function; I used CodeWhisperer in Amazon Glue Studio and found it efficient.

Will I use or recommend Amazon Q? Absolutely. Setting it up in my VSCode is convenient, and it's not just for coding – Amazon Q works magic for data analytics and architecture solutions. Check out Wendy's blog post, an AWS Data Hero, explaining how Amazon Q integrates with Amazon Insights and unleashes its power. The provided source link is also helpful.

Are AI answers convincing? I'd say they're quite good. They work well not just for coding and debugging; if you provide an Infrastructure as Code (IaC) template, the AI can analyze it and suggest AWS best practices. When using AI assistants like ChatGPT or Amazon Q, the key is the user's input or "Prompt."

The assistant answers and recommends based on your input. For instance, when a lambda implementation error occurred, I used Amazon CodeWhisperer to import the Bedrock Client package. This time, modified the pseudo code from import Bedrock Client to import Bedrock runtime client, and it correctly imported the modules. Pseudo-code precision matters and CodeWhisperer also recommends based on existing code.

Will I use AI assistants for learning and software development?

It depends on whether they're designed for training and learning purposes. Otherwise, following their suggestions might be confusing. They're meant to assist, not do the job for you. I tested Amazon Q and Amazon CodeWhisperer for error fixing and code optimisation, and they performed well. While expecting AI bots like ChatGPT, Amazon Q, or Microsoft Copilot to build your app may be premature, Amazon Q provides solid AWS best practices recommendations.

Should we worry about AI taking over our jobs?
I believe that AI/ML is here to assist and improve our lives. AI extends beyond the technology, impacting various domains with daily use cases. However, will it replace our roles soon? I remain optimistic and say no. We build and train AI solutions, teaching them to save time, cost, and lives. the intention is not to replace roles but practicing responsible use of AI can lead to a brighter future.

In closing, I want to share one of my favourite paragraphs from Chip Huyen's book "Designing Machine Learning Systems." All credit to the author.

In early 2020, the Turning Award winner Professor Geoffrey Hinton proposed a heatedly debated question about the importance of interpretability in ML systems. Suppose you have cancer and you have to choose between a black box AI surgeon who can not explain how it works but has a 90% cure rate and a human surgeon with an 80% cure rate. Do you want the AI surgeon to be illegal? A couple of weeks later, when I asked this question to a group of 30 technology executives at public nontech companies, only half of them would want the highly effective but unable-to-explain AI surgeon to operate on them. The other half wanted the human surgeon. While most of us are comfortable with using a microwave without understanding how it works, many don't feel the same way about AI yet, especially if that AI makes important decisions about their lives.

Musical Concierge on AWS

NaDia — Mon, 04 Dec 2023 11:01:42 +0000

Initial Words

In the world of GenAI applications, we often overlook the amazing tools provided by cloud service providers like AWS to explore Machine Learning.

AWS, in particular, offers a comprehensive set of services, covering Data Analytics and Real-Time Data Streaming. When you combine these with AI and ML services like Amazon Rekognition, you can tap into Machine Learning without the complexity of setting up infrastructure or the costs and time required to train models from scratch.

To learn and understand these services, the best approach is to get hands-on. I usually start by thinking about real-world scenarios, explore existing solutions, and then build my idea using the tech and tools I'm comfortable with.

With the new year approaching, I wanted to welcome my guests uniquely. I remembered my security camera was getting dust in my storage. What if I could use it to capture my visitors' arrival and integrate it with a system that not only informs me when they arrive but also recommends a custom song for each visitor to keep them entertained while I get ready to welcome them to the party?

Great! We now have the idea, let's call it "Musical Concierge".

Mixing The Right Ingredients On AWS For A Musical Concierge

AWS Resources and Hardware to build your Musical Concierge:

Guided Tour Of Architecture Diagram

Understanding the essential cake ingredients is just the beginning; it's the art of blending them that gives each cake its unique flavour. Similarly, now that we have the ingredients for crafting our Concierge tool, let's dive into the designed architecture and follow the instructions to bring it to life.

In our tech setup, the main thing we need is an IP camera for the video source. There are many ways to do this project, but I went with what I have—a Tapo C310 IP camera because it supports RTSP (Real-Time Streaming Protocol). RTSP is important for smoothly streaming the captured video from the camera into a service like Amazon Kinesis Video Streams.

To playback the camera's video stream, we can use Amazon Kinesis Video Streams. This service works with Amazon Rekognition for computer vision and video analytics. So, when the video streams reach our AWS account, Amazon Rekognition can recognise familiar faces.

If you're wondering how Amazon Rekognition does this, it uses something called a "face collection". We can make different face collections and add faces to them. When the video stream data goes to Amazon Rekognition, it looks at the face collection and identifies faces based on the ones we added.

When it finds a match, Amazon Rekognition sends out the results. But we're not done yet—we need another way to deliver these results smoothly to a place like an AWS Lambda function. For this, we can use Amazon Kinesis Data Streams as a delivery service.

Now, you can get creative and use the analysis results however you want. But for our case, we need a Lambda function that sends the face recognition result to Amazon SNS (Simple Notification Service) as a messaging service. Amazon SNS lets us send the results to different subscribers, like a Lambda function that notifies us through SMS or apps like Telegram when our visitors arrive. We can even have a Lambda function subscribed to the same SNS topic that recommends a custom song for each visitor based on the music they like.

Now that we have a general idea of how the application works, it's time to set up the hardware and AWS resources.

Configure Resources

Camera

To set up the camera, first, you need to find its IP address. To do this, log in to your network router and look for the IP address of your device.

Create Camera Account

No matter what IP camera you have, it usually comes with a mobile app that lets you control the camera. For my 'Tapo C310' camera, I used the TP-Link App and made a camera account. This account info is needed by Amazon Kinesis Video Streams' Client to make sure it's authorised to get the video from the camera.

This camera account is separate from your TP-Link App login. If you don't give these details, Kinesis Video Streams won't be able to get the video from the camera. If you're using the same camera as me, you can check out the instructions here on how to create your camera account.

Now, your camera is all set to send the video to the cloud. The next step is getting things ready in the cloud.

Create AWS Resources With CloudFormation

Now that we know what services we need from the list, we can easily set them up in AWS. I'll use CloudFormation to create these resources in our AWS account.

Amazon Kinesis Resources

Let's start with Amazon Kinesis Family:

  # Amazon Data Stream
  MusicalConciergeDataStream:
    Type: "AWS::Kinesis::Stream"
    Properties: 
      Name: !Sub ${ApplicationName}-Data-Stream
      ShardCount: 1
  # Amazon Video Stream
  MusicalConciergeVideoStream:
    Type: AWS::KinesisVideo::Stream
    Properties:
      DataRetentionInHours: 24
      Name: !Sub ${ApplicationName}-Video-Stream

Please note that Amazon Kinesis Video Streams (KVS) availability is limited in certain regions. To optimize performance and ensure support for the KVS service, it's essential to deploy the stack in a region that is both geographically close to you and a supported region for KVS.

Amazon Rekognition Resources

Rekognition Stream Processor

Now that our camera's live video is flowing into Kinesis Video Streams in real-time, you might be curious about how it recognises your visitors and what services make it happen. Well, as I mentioned earlier the answer is simple: we just use "Amazon Rekognition Stream Producer" and a "Face Collection".

When the live video data gets to Amazon Rekognition, it looks through a collection of images from different people.

You can set up the face collection and Rekognition Stream Producer using this CloudFormation snippet:

  RekognitionFaceCollection:
    Type: AWS::Rekognition::Collection
    Properties:
      CollectionId: !Ref MusicalConciergeFaceCollectionId

  RekognitionStreamProcessor:
    Type: AWS::Rekognition::StreamProcessor
    Properties:
      Name: "MusicalConciergeStreamProcessor"
      RoleArn: !GetAtt RekognitionVideoIAMRole.Arn
      KinesisVideoStream: 
        Arn: !GetAtt MusicalConciergeVideoStream.Arn
      FaceSearchSettings:
        CollectionId: !Ref MusicalConciergeFaceCollectionId
        FaceMatchThreshold: 98
      KinesisDataStream: 
        Arn: !GetAtt MusicalConciergeDataStream.Arn

Rekognition Iam Role

We need an IAM role for Amazon Rekognition service that allows Rekognition to get the Video Data stream from Amazon Kinesis Video Streams service and put the Facial Match Records into Amazon Kinesis Data Stream, for that we need a policy like this:

  RekognitionVideoIAMRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          -
            Effect: Allow
            Principal:
              Service: rekognition.amazonaws.com
            Action: sts:AssumeRole
      Path: '/'
      Policies:
        -
          PolicyName: RekognitionVideoIAMRole-policy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              -
                Effect: Allow
                Action:
                    - 'kinesis:PutRecord'
                    - 'kinesis:PutRecords'
                Resource: !GetAtt MusicalConciergeDataStream.Arn
              -
                Effect: Allow
                Action:
                    - 'kinesisvideo:GetDataEndpoint'
                    - 'kinesisvideo:GetMedia'
                Resource: !GetAtt MusicalConciergeVideoStream.Arn
              -
                Effect: Allow
                Action:
                    - 'rekognition:*'
                Resource: '*'

Create Joy From Amazon Rekognition Analysis

Once the AWS Lambda gets the matching face analysis from Amazon Kinesis Data Streams, we can do lots of cool things with it. We can use it however we want—like sending a text to tell us our visitor is here with their name, or playing a nice song for them as they wait for our welcoming hello at the door.

To make our two Lambda functions—one for publishing a message to Amazon SNS topic containing the face recognition result and the other for suggesting music—we can use the cloudformation resources like this:

  GetVideoAnalysisLambda: 
    Type: "AWS::Lambda::Function"
    Properties: 
      Code: ./.build/GetVideoAnalysis.zip
      FunctionName: GetVideoAnalysisLambda
      Handler: src/GetVideoAnalysis.handler
      Role: !GetAtt GetVideoAnalysisLambdaRole.Arn
      Environment:
        Variables:
          SNS_TOPIC: !Ref SNSTopic
      Runtime: "nodejs18.x"
      MemorySize: 1024
      Timeout: "900"

  GetVideoAnalysisLambdaKinesisMapping:
    Type: "AWS::Lambda::EventSourceMapping"
    Properties: 
      BatchSize: 10
      Enabled: true
      EventSourceArn: !GetAtt MusicalConciergeDataStream.Arn
      FunctionName: !GetAtt  GetVideoAnalysisLambda.Arn
      StartingPosition: "TRIM_HORIZON"

  InformHostLambda: 
    Type: "AWS::Lambda::Function"
    Properties: 
      Code: .build/InformHost.zip
      FunctionName: InformHostLambda
      Handler: src/InformHost.handler
      Role: !GetAtt InformHostLambdaRole.Arn
      Environment:
        Variables:
          BUCKET_NAME: !Ref ConciergeAudioBucketName
          SECRET_NAME: !Ref TelegramBotSecretName
      Runtime: "nodejs18.x"
      MemorySize: 1024
      Timeout: "900"

  InformHostLambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !GetAtt InformHostLambda.Arn
      Principal: sns.amazonaws.com
      SourceArn: !Ref SNSTopic

If you want to know what each lambda function does, check out the examples in this code repository on GitHub. You can use them as a starting point and change them however you like.

Deploy Resources as Infrastructure as Code

We are nearly there, through cloudformation snippets from this blog post we almost created all the core resources required for the Musical Concierge. However, there are some other resources such as Amazon Secrets Manager or AWS S3 Buckets to store the face images or collection of musics, to access the full version of all the resources please check out the cloudformation file in this code repository.

Time to deploy all the resources in AWS account. Go ahead and continue with build, package and deploying the resources.

Build and Package

npm run build
npm run package

aws cloudformation package --template-file ./cloudformation.yaml --s3-bucket $ARTIFACT_BUCKET --output-template-file /<FOLDER>/cloudformation.yaml

aws cloudformation deploy \
  --template /path_to_template/my-template.yml \
  --stack-name <STACK_NAME> \
  --parameter-overrides Key1=Value1 Key2=Value2 \

In this project I have not set up CI/CD, if you are planing to productionise this project, make sure this step is part of your continues integration and continues deployment.

After successful deployment of cloudformation resources, it's time to link everything together and make the videos flow into Amazon resources. We'll do that in the next few steps.

Configure Face collection (AWS CLI)

To make Amazon Rekognition recognize faces in the live stream, we have to give it a collection of known faces. In earlier steps, we made this collection. Now, to add familiar faces to it, you can use AWS CLI. This part requires some manual work, though. You must have set up your AWS CLI and put in your credentials to run these commands and make it work.

Add images to the face collection:

I put some photos of faces in a folder on S3. When I run this command, it grabs the photo from S3 and adds it as a new face to my collection:

  aws rekognition index-faces \
    --image '{"S3Object":{"Bucket":"<BUCKET_NAME>","Name":"<FILE_NAME>.jpg"}}' \
    --collection-id "<COLLECTION_ID>" \
    --detection-attributes "ALL" \
    --external-image-id "<FACE-TAG>" \
    --region <AWS_REGION>

Tip: make sure the region of your bucket is same as face collection

Start Rekognition Stream Producer (AWS CLI)

Rekognition Stream Producer is the heart of the system. It pulls video from Kinesis video, analyzes it & pushes the results to Kinesis data. Earlier, We created the Amazon Rekognition Procession within the cloudformation lets get a list of existing processors:

To List Rekognition Stream Producer run this command in your terminal:

aws rekognition list-stream-processors

When the Rekognition Stream Producer is initially created, the default status is "STOPPED":

output:
{
    "StreamProcessors": [
        {
            "Name": "musical-concierge-rekognition-processor",
            "Status": "STOPPED"
        }
    ]
}

To Start Rekognition Stream Producer run this command in your terminal:

aws rekognition start-stream-processor \
    --name <PROCESSOR_NAME>

After starting the Stream Producer the status will change to "Running":

{
    "StreamProcessors": [
        {
            "Name": "musical-concierge-rekognition-processor",
            "Status": "RUNNING"
        }
    ]
}

Connect the Camera as a stream source (Producer) for Kinesis Video Stream

After setting up Amazon Kinesis, it's time to send data to it. We can use the SDK to create code for our application. This code grabs video data, called frames, from the video source and sends it to Kinesis Video Streams. These apps are also called producers.

The producer libraries usually have two parts:

Kinesis Video Streams Producer Client
Kinesis Video Streams Producer Library

Kinesis Video Streams doesn't have ready-made setups for devices like cameras. To get data from media devices, you need to write code to create your own custom media source. After that, you can register your custom media sources with 'KinesisVideoClient', and it will send the data to Kinesis Video Streams.

To implement the application to extract and upload the data to Kinesis Video stream from scratch I recommend to follow this document page on AWS.

That might seem to be very complex but thanks to Docker we can build the entire application as a Docker image and use one of the provided samples from AWS Amazon Kinesis Repository to start with uploading the data to Kinesis.

For Musical Concierge app I have used the docker image approach:

1- Set up docker if it's first time using Docker get Docker.
2- Copy provided Docker File in the source repo to the root of your project.
3- Build and Run the docker image:

        docker build -t <YOUR_IMAGE_NAME> .
        <!-- List docker images and find your image ID -->
        docker images   
        docker run -it <YOU_IMAGE_ID>

4- Run the gstreamer sample app with the requisite argument
In your running docker execute the following command:

AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID> \
AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY> \
./kvs_gstreamer_sample <STREAM_NAME> <RTSP_URL>

Tip1: Make sure that you are Authenticate with your AWS credentials. Set up aws config within your Docker.

Tip2: If you are using the same camera as mine, the RTSP url is typically something in this format rtsp://camera_username:camera_password@camera_ip:554/stream1

Set Up Telegram ChatBot

To be notified on my guests arrival, I have created a lambda function that sends me a message on Telegram application.

You can simply follow instruction from this web page to set up your Telegram chatbot. Once the chatbot is ready, you will be provided by an API Token. As it's a secret token you can store the API Token in Amazon Secrets Manager Service to get the secret in the lambda function. The Lambda Function will use this token to send the visitors names and the music file to the telegram bot.

Trouble shooting and Wrap up

After setting everything up, it's good to go! Try out the Musical Concierge with a friend, or the next time you have someone visiting, check your Telegram messages. Here's an example of a message I got when a friend visited me over the weekend ;)

Tip: Remember to start your Chatbot before proceeding with deployment process

Building your Amazon Kinesis Video Stream app gives you more control over your media stream frames. This helps prevent issues like getting the same stream records again. I recommend starting with the gstreamer sample app to save time and effort as you build on it.

In this article, I wanted to show how easily you can set up a budget-friendly custom concierge. This is just the start.You can add more, like displaying a welcome message on an LED board when guests arrive or sending a fun message to their phone. Get creative, maybe even tease them about forgetting to bring your favorite beverage!

I hope this guide has provided you with a solid foundation to get started with building your own Concierge. If you have any further questions or need assistance, please feel free to reach out on LinkedIn.

And if you are not willing to keep this experiment to avoid cost, don't forget to clean up the resources once you finished.

Clean up

After you've had fun with it, now it's time to delete everything!

Here's what you do:

Stop the Docker container.
Stop & delete the stream processor:

aws rekognition stop-stream-processor --name <REKOGNITION_STREAM_PRODUCER>

aws rekognition delete-stream-processor --name <REKOGNITION_STREAM_PRODUCER>

Delete cloudformation stack.
Delete the Rekognition faces collection:

aws rekognition delete-collection --collection-id <COLLECTION_ID

Story telling App with Amazon Bedrock

NaDia — Tue, 17 Oct 2023 03:36:41 +0000

Initial Words

Welcome to Part 2 of our introductory blog series! In Part 1, we delved into the basic concepts such as Foundation Models and discovered the amazing features and capabilities of Amazon Bedrock. Now, it's time for the fun part: building your very own storytelling application with Foundation Models from Amazon Bedrock.

By the time we're done here, you'll have deployed a Serverless setup with two APIs. First API will generate the story from a given topic, and the other one will illustrate each paragraph of the story. We'll be using AWS Appsync and GraphQL to make requests to these APIs and generate stories. If you're wondering how these APIs work with FM models to create stories and illustrations, that's the magic of Amazon Bedrock we'll uncover together. So, let's get started on this storytelling adventure!

Now that you've got a glimpse of what we're building, let's take a moment to unravel the complexity hidden behind this simple GenAI tool.

Imagine this: a user hops onto your web app and wants to create a story by giving it a topic prompt. From the user's standpoint, they expect the story to magically unfold, complete with illustrations, as quickly as they can think of it. They might want to edit and personalise the story, or even ensure that it suits the age group it's intended for. And what if they want to save and share their literary masterpiece with others?

All these amazing features and optimisations are like extra layers of icing on the cake, but for our project, we're keeping things simple and focused. So, while they're fascinating possibilities, we'll save them for another time!

Build An Storytelling App With Me

Take a look at the solution diagram below! It shows you exactly how our app works at every step:

The user starts by giving us a topic for their story.
When user clicks on "Generate Story," the web app sends a request to foundation model to create the story, and then it returns the generated story. The Frontend app does some cleaning on the API response and shows the story in separate paragraphs.
Now, here's where it gets interesting. They can add illustrations to the story! In this app, I've configured the FM model to generate an image for summary of each paragraph.
These generated images are stored in an S3 bucket, and the UI shows them to user once it gets back the S3 presigned URLs.

For all the nitty-gritty details, just check out the solution architecture diagram. It's like a map that guides you through the app's awesomeness.

Architecture

Web application

For UI facing of this GenAI tool, we won't be diving into the fancy design. I've laid out the basic structure of the application. You can grab the source code from this repository. Feel free to give it your own unique style or add more features if you'd like. Once you've got the code, just follow the simple steps in the ReadMe file to get your app running on your computer.

And if you're feeling adventurous and want to share your app with the world, you can host it on your AWS account. I won't get into the nitty-gritty details of that in this blog post, but all you really need is an Amazon S3 bucket to store your web app's resources. Then, set up Amazon CloudFront and use Route 53 to manage your domain's traffic and routing. It's not as complicated as it might sound, and it's a great way to take your project to the next level!

Amazon Bedrock Magician

To set up the necessary APIs for our app to function, we'll be creating a serverless stack. You can access the complete source code in this GitHub Repo. In this repository, you'll find the required Lambda functions as the API resolvers, IAM Roles, Amazon Appsync, S3 Bucket and all the managed Policies listed in the "serverless.yml" file.
To deploy the Backend resources, all you have to do is run the command specified in the "ReadMe.md" file.

However, I strongly recommend that before you deploy the serverless stack, you continue reading this article. I'll be sharing code snippets from various lambdas, explaining how to define your Input Prompt, how to access the API "Request" object, and different methods for invoking the Foundation Models for both Python and Nodejs project. It's like getting a sneak peek behind the scenes!

Configure the Bedrock runtime Client

Typically, when you need to issue commands to an AWS service, the first step involves initialising the service client. In this scenario, we'll initialise the Amazon Bedrock Runtime client.

Code snippets in Python and Nodejs:

# Implementation in Python
import boto3, json

bedrock_client = boto3.client(
    service_name="bedrock-runtime",
    region_name="us-east-1"
)

// Implementation in Nodejs

import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";

const bedrockRuntimeClient = new BedrockRuntimeClient({ region: "us-east-1" })

Model playground

Before Prompt engineering and constructing our request's payload, let's understand how to send requests for each Bedrock model. For that, you have two options. You can check out the Notebook examples in the Bedrock console, or you can use the model playground.

In the model playground, select the model you want, configure the inference options (the model parameters will impact the result), and then click "View API Request." This allows you to copy the request and modify the input as needed.

For our Generate Story API, we'll be using the "Jurassic-2 Ultra" model from the "AI21 Lab" category. Let's see how to get the api request example for this model. It's going to be a fun ride!

Within the Text playground, I select the category and model:

Next, type a random text, instead of invoking the model, select the "View API Request" from the screen and that will provide you with a request example to start with:

Copy the API request payload and continue with next step where I show you how to construct your Prompt and your invoke command input.

Construct your Request Payload

Now that we have the request payload, we can begin making it more versatile, allowing our model to generate stories for any given topic.

Here is an example of the Text generator model API request, where we configure the "Model Id", "Model Parameters" and the "Input Prompt".


kwargs = {
  "modelId": "ai21.j2-ultra-v1", <------ Text generator model
  "contentType": "application/json",
  "accept": "*/*",
  "body": "{\"prompt\":\" write a stroy up to 200 words about "+ storyTopic + "\",\"maxTokens\":300,\"temperature\":0.7,\"topP\":1,\"stopSequences\":[],\"countPenalty\":{\"scale\":0},\"presencePenalty\":{\"scale\":0},\"frequencyPenalty\":{\"scale\":0}}"  <-------- Body Object contains the Model Parameters & Input prompt
}

// Implementation in Nodejs
    private constructStoryRequestPayload = (prompt: string, maxToken: number) => {
        return {
            "modelId": this.textModelId,
            "contentType": "application/json",
            "accept": "*/*",
            "body": `{\"prompt\":\ ${prompt},\"maxTokens\": ${maxToken},\"temperature\":0.7,\"topP\":1,\"stopSequences\":[],\"countPenalty\":{\"scale\":0},\"presencePenalty\":{\"scale\":0},\"frequencyPenalty\":{\"scale\":0}}`
        }
    }

Invoke FM for inference

We're nearly there! It's as straightforward as this. The final step is to invoke the model (in this case, the text generator model "Jurassic-2 Ultra") and obtain inference. To get inference from models in Amazon Bedrock, we have two options. We can either use the "invoke_model" method or the "invoke_model_with_response_stream" method.

If you're wondering about the difference, here's the scoop:

With the "invoke_model" method, the model won't provide any response until it has fully generated the text or completed the requested task.
On the other hand, "invoke_model_with_response_stream" offers a smoother and more real-time experience for users. It sends stream response payloads back to clients as the model works its magic.

Code snippets for model inference:

# Implementation in Python

# invoke_model
story = bedrock_client.invoke_model(**kwargs)

#  invoke_model_with_response_stream
story = bedrock_client.invoke_model_with_response_stream(**kwargs)
stream = story.get('body)
if stream:
    for event in stream:
        chunk = event.get('chunk)
        if chunk:
            print(json.loads(chunk.get('bytes').get('completion'), end=""))

// Implementation in Nodejs

private invokeTextModel = async (prompt: string, maxToken: number): Promise<string> => {
        // construct model API payload
        const input = this.constructStoryRequestPayload(prompt, maxToken)
        const command = new InvokeModelCommand(input);
        // InvokeModelRequest
        const response = await this.bedrockRuntimeClient.send(command);
        const story = response.body.transformToString()
        // get the text body
        const parsedStory = JSON.parse(story)
        return parsedStory.completions[0].data.text
    }

With three simple steps we could generate the story from a topic! API response is returned in Json format and all we need to do is to destruct the generated text from response object.

Extract the generated text

Follow the steps to extract the story content from the API response:

story_stream = json.loads(story.get('body').read())
story_content = story_stream.get('completions')[0].get('data').get('text')

We still need another API to complete the Story Telling app. To create illustrations based on a generated story, repeat the 3 simple steps from previous API and simply utilise the "stable-diffusion-xl-v0" model from the "Stability AI" category to generate image based on the provided content. It's that easy!

Final Words

I've always been a fan of keeping things simple and staying grounded in the fundamentals. It's a great way to uncover new ideas, explore, and learn, all while having a good time building cool stuff.

In this two-part blog post, my goal was to introduce you to Amazon Bedrock, showcase its features, and demonstrate how you can easily integrate various FMs into your APIs to build amazing generative AI-powered applications.

I hope you've found it valuable. Now that you have a solid foundation of Amazon Bedrock and you know how to get inference from a basic model within Amazon Bedrock in 3 simple steps feel free to build upon it and explore even further! The possibilities are endless.

An Introduction to Amazon Bedrock

NaDia — Tue, 17 Oct 2023 03:36:07 +0000

Introduction

Did you catch the thrilling announcement? "Amazon Bedrock" is now officially accessible to all users. While a few incredible features of this service are still in the "Preview" stage, it is sufficient to empower teams with exciting capabilities for effortlessly creating and launching Generative AI applications.

This blog post comes in two exciting parts. In Part 1, I'll dive into the core concepts and terminology, taking apart the inner mechanisms of Amazon Bedrock. Then, brace yourself for Part 2, where I'll walk you through crafting a simple yet captivating storytelling bot with Amazon Bedrock. If you're already well-versed in the basics, feel free to jump ahead to Part 2 and embark on your creative journey!

Let's Begin With The Basics

If you share my approach of beginning with documentation, you probably have already visited the "Amazon Bedrock" homepage.

On this page, three significant phrases stand out: "Easiest way", "Scalable", and "Foundation Models". I am convinced that these attributes and characteristics are what set Amazon Bedrock apart from any other alternatives.

Ever wondered why AWS calls this the "easiest" and a "Scalable" option? Well, if you're an expert in generative AI (GenAI), you've likely tasted the complexity of setting up and managing the nuts and bolts needed for a generative AI app. It's like solving a puzzle with pieces like:

Picking the right computing power.
Network Configurations.
Ensuring model and data safety.
Monitoring and Keeping an eye on the infrastructure for a reliable app.
Data security

So, how does Amazon Bedrock make this complex stuff easy? Picture this: Bedrock often serves up pre-configured resources, custom-made for GenAI tasks. These ready-to-go setups come with all the software bits and pieces you need, like libraries, dependencies and tools, already installed. Plus, Bedrock integrates with AWS Managed services like Amazon S3, Amazon CloudWatch, and AWS Lambda, making tough tasks like configuring data storage, authentication, and monitoring a walk in the park for your GenAI apps.

Here's the cool part: Amazon Bedrock is "Serverless." That means it can automatically grow or shrink resources as your app's popularity ebbs and flows. So, when traffic goes up, Bedrock scales up your resources, ensuring peak performance without breaking the bank. All provided data to the Bedrock is encrypted at both rest and in transit. That would give your peace of mind if you want to adopt GenAI.

If I've got you excited about the ease and efficiency of GenAI apps with Amazon Bedrock, it's time to dive into the world of "Foundation Models".

Uncover The Magic Of Foundation Models (FMs)

FMs are like super-smart, giant neural networks trained on massive piles of data. Instead of reinventing the AI models every time from scratch, we use FMs as a launchpad to create our own customised models in a faster and cost-effective approach. These FMs are like all-in-one champs; they can execute multiple tasks with high accuracy, like generating image or text from a simple input prompt, answering tricky questions, and even solving math puzzles.

What makes FMs stand out is their versatility. Unlike regular ML models that are one-trick ponies, FMs are like jacks-of-all-trades, zipping through tasks quicker and cheaper. They're like the cool kids who make their own labels from data, thanks to something called self-supervised learning. This sets them apart from the old-school ML models, whether they were supervised or flying solo without supervision (unsupervised learning)!

Demystify Amazon Bedrock

Availability

As of writing this blog post, Amazon Bedrock is accessible in four regions, as listed below. But keep in mind that by the time you're reading this, AWS might have expanded its availability to additional regions. So, always stay tuned for the latest updates!

Bedrock Base Foundation Model Choices

If you're new to Amazon Bedrock and you're diving into the "Base Models" within your AWS console, you might notice a warning next to the listed models. By default, your Amazon Bedrock doesn't come with access to these base FMs. To use them, you'll need to request access first.

To make this happen, head to your AWS Amazon Bedrock console and navigate to "Model Access." There, you can pick the models you want to use and send in an access request for them. After a little while, maybe a few minutes or occasionally a few hours, you'll see those models go from "Pending" to "Access granted," just like in the screenshot below. Keep in mind that Model access is provided on a per-region basis. If you want models available in multiple regions, you'll need to request access for each region separately.

Feel free to explore the list of available Foundation Models (FMs) for Amazon Bedrock and discover their individual use cases.

Pricing for each model is determined by the pricing mode you've chosen, whether it's On-Demand or Provisioned. Additionally, it's influenced by factors like the length of the generated tokens and other considerations. For detailed pricing information, you can refer to this link.

Fine-Tune a Foundation Model

Isn't this exciting? The best part is that we're not restricted to just using Base Models. We have the flexibility to supply a labeled dataset, initiate a tuning job, and once we're satisfied with the model's performance and accuracy, we can seamlessly utilise the fine-tuned model for inference, just as easily as working with the Base Models.

Bedrock Agent (In Preview)

I must admit, this is my absolute favourite feature, and I'm eagerly awaiting the day when AWS announces it's available to everyone, perhaps at Re:Invent 2024!

If you're not sure what an agent means in the world of Generative AI, I've put together a brief blog post explaining Agents and Transformers. Agents have the incredible power to expand the capabilities of Foundation Models. They can grasp all sorts of user requests, tackle even the most complex ones by breaking them down into smaller tasks, and then take action to fulfill those requests. If you want to learn how to make your own Agent for Amazon Bedrock, you're in luck! Check out this fantastic article for all the details.

Knowledge Base (In Preview)

Like the Bedrock agent, this feature is still in "Preview". Creating agents for Amazon Bedrock offers a big advantage: it allows secure connections between FMs and your company's data sources. This means Bedrock can tap into additional datasets, resulting in more precise answers.

If you've got Preview access to Amazon Bedrock, don't hesitate any longer. Jump into your AWS console and follow this detailed blog post to learn how to kickstart the Knowledge Base for Amazon Bedrock.

Wrap Up

While Amazon Bedrock is still evolving, it's been a game-changer for sparking our creativity and making it easy and cost-effective to build advanced generative AI apps. Personally, I can't wait to try out Amazon Bedrock Agent and its Knowledge Base features; they promise even more exciting possibilities.

Now that you've got the basics of this service and its models, let's get hands-on. Follow along in Part 2 of this article, where I'll guide you through creating a storytelling app using Amazon Bedrock and some other cool Amazon services. It's time to bring your ideas to life!

Story telling App with Amazon Bedrock

NaDia — Sun, 15 Oct 2023 17:53:52 +0000

Initial Words

Now that you've got a glimpse of what we're building, let's take a moment to unravel the complexity hidden behind this simple GenAI tool.

Build An Storytelling App With Me

Take a look at the solution diagram below! It shows you exactly how our app works at every step:

The user starts by giving us a topic for their story.
When user clicks on "Generate Story," the web app sends a request to foundation model to create the story, and then it returns the generated story. The Frontend app does some cleaning on the API response and shows the story in separate paragraphs.
Now, here's where it gets interesting. They can add illustrations to the story! In this app, I've configured the FM model to generate an image for summary of each paragraph.
These generated images are stored in an S3 bucket, and the UI shows them to user once it gets back the S3 presigned URLs.

For all the nitty-gritty details, just check out the solution architecture diagram. It's like a map that guides you through the app's awesomeness.

Architecture

Web application

Amazon Bedrock Magician

Configure the Bedrock runtime Client

Typically, when you need to issue commands to an AWS service, the first step involves initialising the service client. In this scenario, we'll initialise the Amazon Bedrock Runtime client.

Code snippets in Python and Nodejs:

# Implementation in Python
import boto3, json

bedrock_client = boto3.client(
    service_name="bedrock-runtime",
    region_name="us-east-1"
)

// Implementation in Nodejs

import { BedrockRuntimeClient } from "@aws-sdk/client-bedrock-runtime";

const bedrockRuntimeClient = new BedrockRuntimeClient({ region: "us-east-1" })

Model playground

For our Generate Story API, we'll be using the "Jurassic-2 Ultra" model from the "AI21 Lab" category. Let's see how to get the api request example for this model. It's going to be a fun ride!

Within the Text playground, I select the category and model:

Next, type a random text, instead of invoking the model, select the "View API Request" from the screen and that will provide you with a request example to start with:

Copy the API request payload and continue with next step where I show you how to construct your Prompt and your invoke command input.

Construct your Request Payload

Now that we have the request payload, we can begin making it more versatile, allowing our model to generate stories for any given topic.

Here is an example of the Text generator model API request, where we configure the "Model Id", "Model Parameters" and the "Input Prompt".


kwargs = {
  "modelId": "ai21.j2-ultra-v1", <------ Text generator model
  "contentType": "application/json",
  "accept": "*/*",
  "body": "{\"prompt\":\" write a stroy up to 200 words about "+ storyTopic + "\",\"maxTokens\":300,\"temperature\":0.7,\"topP\":1,\"stopSequences\":[],\"countPenalty\":{\"scale\":0},\"presencePenalty\":{\"scale\":0},\"frequencyPenalty\":{\"scale\":0}}"  <-------- Body Object contains the Model Parameters & Input prompt
}

// Implementation in Nodejs
    private constructStoryRequestPayload = (prompt: string, maxToken: number) => {
        return {
            "modelId": this.textModelId,
            "contentType": "application/json",
            "accept": "*/*",
            "body": `{\"prompt\":\ ${prompt},\"maxTokens\": ${maxToken},\"temperature\":0.7,\"topP\":1,\"stopSequences\":[],\"countPenalty\":{\"scale\":0},\"presencePenalty\":{\"scale\":0},\"frequencyPenalty\":{\"scale\":0}}`
        }
    }

Invoke FM for inference

If you're wondering about the difference, here's the scoop:

With the "invoke_model" method, the model won't provide any response until it has fully generated the text or completed the requested task.
On the other hand, "invoke_model_with_response_stream" offers a smoother and more real-time experience for users. It sends stream response payloads back to clients as the model works its magic.

Code snippets for model inference:

# Implementation in Python

# invoke_model
story = bedrock_client.invoke_model(**kwargs)

#  invoke_model_with_response_stream
story = bedrock_client.invoke_model_with_response_stream(**kwargs)
stream = story.get('body)
if stream:
    for event in stream:
        chunk = event.get('chunk)
        if chunk:
            print(json.loads(chunk.get('bytes').get('completion'), end=""))

// Implementation in Nodejs

private invokeTextModel = async (prompt: string, maxToken: number): Promise<string> => {
        // construct model API payload
        const input = this.constructStoryRequestPayload(prompt, maxToken)
        const command = new InvokeModelCommand(input);
        // InvokeModelRequest
        const response = await this.bedrockRuntimeClient.send(command);
        const story = response.body.transformToString()
        // get the text body
        const parsedStory = JSON.parse(story)
        return parsedStory.completions[0].data.text
    }

With three simple steps we could generate the story from a topic! API response is returned in Json format and all we need to do is to destruct the generated text from response object.

Extract the generated text

Follow the steps to extract the story content from the API response:

story_stream = json.loads(story.get('body').read())
story_content = story_stream.get('completions')[0].get('data').get('text')

Final Words

I've always been a fan of keeping things simple and staying grounded in the fundamentals. It's a great way to uncover new ideas, explore, and learn, all while having a good time building cool stuff.

An Introduction to Amazon Bedrock

NaDia — Mon, 09 Oct 2023 04:38:39 +0000

Introduction

Let's Begin With The Basics

If you share my approach of beginning with documentation, you probably have already visited the "Amazon Bedrock" homepage.

Picking the right computing power.
Network Configurations.
Ensuring model and data safety.
Monitoring and Keeping an eye on the infrastructure for a reliable app.
Data security

If I've got you excited about the ease and efficiency of GenAI apps with Amazon Bedrock, it's time to dive into the world of "Foundation Models".

Uncover The Magic Of Foundation Models (FMs)

Demystify Amazon Bedrock

Availability

Bedrock Base Foundation Model Choices

Feel free to explore the list of available Foundation Models (FMs) for Amazon Bedrock and discover their individual use cases.

Fine-Tune a Foundation Model

Bedrock Agent (In Preview)

I must admit, this is my absolute favourite feature, and I'm eagerly awaiting the day when AWS announces it's available to everyone, perhaps at Re:Invent 2024!

Knowledge Base (In Preview)

Wrap Up

Self Service Learning Platform With Hugging Face Transformers!

NaDia — Fri, 30 Jun 2023 19:58:32 +0000

Introduction

About Quizify

The Self-Service Learning platform can be converted to an innovative app designed to revolutionise your learning experience. With this powerful tool, users can effortlessly generate custom quizzes based on any PDF file they download from the web. Gone are the days of tedious manual summarisation and translation! this potential application leverages the cutting-edge capabilities of Hugging Face transformers to simplify the entire process. Once you've obtained a PDF, simply import it into the app and watch as the magic unfolds.

As students, parents, or job seekers, we often encounter situations where we need to summarise a document promptly and prepare questions for ourselves or our children. This inspired the concept of a self-service learning platform.

This learning platform empowers you to retrieve files from the internet and summarise the document with ease. Not only that, but this app also offers built-in translation functionality, allowing you to understand the content in your preferred language.

But the true power of our app lies in its ability to transform your summarised and translated document into an interactive question-answer service. By analysing the text and extracting relevant information, the app generates thought-provoking questions that test your understanding of the material. Whether you're a student striving for academic excellence or a professional looking to enhance your knowledge, the Self Service Quiz Generator platform is your go-to tool for efficient and engaging learning.

Motivation

Words like ChatGPT, Generative AI, LLM, LangChain, Hugging Face, Transformers, and many others have become part of our daily vocabulary, frequently heard and mentioned throughout the day.

Regrettably, it is challenging to keep pace with the rapid advancements in technology and acquire knowledge about every new development introduced in the tech world.

Being a part of the AWS Community Builder offers numerous advantages, one of which is the opportunity to participate in a variety of challenges ranging from openMic sessions to Hackathons like the current one. Each challenge provides a valuable learning experience. In this instance, we were presented with the #AiHackathon challenge to develop a tool utilising the Transformer Tools framework.

My unwavering desire to learn about ML concepts and the constantly evolving landscape of ML and AI has made it impossible for me to let go of this incredible learning opportunity!

Process of shaping my idea

Prior to this challenge, I had no prior knowledge of Hugging Face Transformers and agents. To familiarise myself with the concept of transformers, I began by exploring tutorials and educational resources. As I delved deeper, I contemplated how to harness the immense potential of the Large Language Model technology.

With a more comprehensive understanding of the topic at hand, I embarked on a brainstorming session. After finalising the idea, I focused on outlining the features, which led me to develop this tool incorporating six custom Transformer Tools.

Downloader Service

To begin, You should provide a valid URL for your PDF resource.

If you choose to enter a URL, The Hugging Face Agent will seamlessly download the file for you. This process utilises a custom tool known as the download_file_tool working silently behind the scenes to retrieve the document.

Currently, This proof of concept exclusively supports PDF files at this stage.

As an alternative, you have the option to upload your PDF file. Our agent will utilise a specialised tool called read_file_tool to process and extract the content from the document. The extracted information will be saved for further use within the platform.

Summarisation Service

Get ready to dive into the exciting features of our app. How about downloading a summarisation of your uploaded document or web content? Let's embark on this thrilling journey together!"
By asking the agent to summarise the file content, the Hugging Face agent will generate a summary of your document.

Please note that the summarisation model used by the agent is the facebook/bart-large-cnn, the results may not be perfect. If your document is excessively large, there is a chance it may encounter difficulties or exhibit unexpected behaviour while processing.

Here is an example how I have leveraged Transformers pipeline to summarise a large document:

Translation Service

Perhaps English is your second language, just like mine! Or you have found an amazing document to read but not in the language that you are familiar with, don't worry, I've got your back 😉. With this tool, you can select between Italian, French, and Spanish languages. Not only that, but you can also have the summary translated into your preferred language for better understanding.

Text-To-Speech Service

In addition to all we mentioned so far, if you are keen to listen to the summary or the original uploaded content, in this platform you will be able to generate audio from the file.

Quiz Generator Service

Have you studied the summary carefully? Great! That means you're ready, right? Now, let's ask our Hugging Face agent to generate some engaging multiple-choice questions for you! Get ready to put your knowledge to the test!

You have the freedom to choose the language in which you want to be examined! Simply select your desired language option. Additionally, you can specify the number of questions you would like in your requested exam. Tailor the examination experience according to your preferences!
Once the agent has gathered this information, it will utilise another specialised tool called quiz_generator_tool to generate the quiz for you. This tool is specifically designed to create dynamic and engaging quizzes based on your selected preferences. Sit back and let the quiz generation process unfold!

Outcome and demonstration of Jupyter Notebook

The full Jupyter Notebook is available for reference at the provided Link. To begin, start by installing the necessary dependencies and logging in to the Hugging Face hub. Afterward, the remaining steps are straightforward and easy to follow.

Challenges I faced!

When I began my learning journey, I sought answers to all my "Why" and "How" questions. However, since it's a relatively new topic, the availability of learning materials was limited. Therefore, I had to delve into documentation and explore relevant forums to find the information I needed. For instance, I had to search for guidance on configuring a custom Prompt run_template, such as "How to configure a custom Prompt run_template."
Error message notebook_login! So I ended up using the login class instead of notebook_login

ValueError(“Invalid token passed.”)

By removing PreTools from the agent's toolBox, the agent's confusion can be reduced. However, it's important to note that the agent will still search for default tools, especially if they are within the same context, such as translation or summarisations.
Unsuccessful attempts on using Streamlit and Vercel as a fast and quick way to add a nice user interface in the such short time.

Managing dependencies for the tools proved to be a significant factor that discouraged me from considering combinations like AppSync/API Gateway and Lambda. Instead, I made a straightforward decision to implement the solution using a Jupyter Notebook, which allowed for easier dependency management.
LLM models are so sensitive with "Text" as their input! Prompt engineering is a "Thing".

What is next?

There are numerous ways to enhance this tool. First and foremost, it would be beneficial to transform the concept into an application by incorporating a user interface and providing proper API references to various tools. Additionally, the models can be fine-tuned and retrained to enhance the quality of each service, such as the Summarisation and Quiz Generator Tools.

Reflection

Throughout this entire journey, I have gained a wealth of knowledge about transformers and have witnessed the immense power of the Agent. I have successfully trained the most complex LLM Model and integrated it into the agent's toolbox. This has sparked my imagination, as I contemplate the vast possibilities of designing efficient systems with the agent at the centre, orchestrating decoupled tasks.

As the AiHackathon draws to a close, my enthusiasm remains undiminished, compelling me to revisit each challenge and seek their solutions one by one. I find myself continuously envisioning the end-to-end architecture of this solution in my mind. It is now time to bring this vision out of my head and construct it meticulously as a fully-fledged application.