Forem: Gianluigi Mucciolo

Sleigh-Ride Cloud Chronicles: A 25-Day Journey Through Cloud and Magic

Gianluigi Mucciolo — Sun, 30 Nov 2025 17:17:57 +0000

"The reindeer are in Ibiza. The elves are on strike. Christmas hangs in the balance. And you, you have 25 days to help Santa save it."

Introduction: The Night the North Pole Went Dark

It is December 1st, and something has gone terribly wrong at the North Pole.

The elves have gone on strike - something about dental coverage for candy-cane-related injuries. The reindeer? Last spotted in Ibiza, sipping piña coladas and posting photos that Santa probably shouldn't see. The toy workshop stands frozen and silent. And in Santa's office, letters from children around the world are piling up, unopened, unanswered, turning from hope into heartbreak with each passing hour.

For the first time in centuries, Santa is completely alone.

He stares at the infrastructure he has relied on for centuries, now collapsed overnight. Desperate, he calls the only organization that might possibly help: Amazon. But they don't send replacement elves or magical reindeer. They send him infrastructure. Cloud services. APIs. Foundation models.

Santa looks at this pile of raw technical materials and realizes: if he is going to save Christmas, he needs to become the world's first Generative AI Cloud Architect.

The Journey Begins

Santa takes a deep breath and opens the first letter. It's from a tech-savvy kid who uploaded it to an S3 bucket - full of emojis, memes, and AI-generated nonsense. Santa has no idea what half of it means.

But he has 25 days. And he has AWS Bedrock.

Day by day, challenge by challenge, Santa learns. He starts with the basics - teaching a foundation model to extract meaning from chaotic letters. Then he builds memory systems so he doesn't forget why Timmy painted the cat blue. He creates AI agents to help him: Rudy (anxious, brilliant, hyper-organized) and Elfie (enthusiastic, literal-minded, connected to the toy catalog). Together, they must learn to collaborate without causing disasters.

Like the incident with the uranium-contaminated chemistry kit. We don't talk about that.

The story isn't just decoration - it's the framework that makes complex AI concepts stick. When you're helping Santa search through ten years of "Naughty and Nice" records, you're learning RAG (Retrieval-Augmented Generation). When you're teaching his agents to work together, you're learning multi-agent orchestration. The narrative gives you something to anchor to.

The Technical Journey

The project is structured as 25 progressive challenges. You follow Santa as he rebuilds Christmas using generative AI. Underneath the narrative wrapping is a carefully architected progression from basic prompt engineering to multi-agent orchestration.

Phase 1 (Days 1-6): From Letters to Data

The Narrative: Santa learns to read his first digital letter - a tech-savvy kid uploaded it to an S3 bucket, full of emojis and AI-generated nonsense.

The Tech: Fundamentals of prompt engineering, entity extraction, text-to-image generation, validation logic, and chunking strategies. You teach Santa how to use a foundation model to make sense of the chaos.

Phase 2 (Days 7-12): Knowledge & Memory

The Narrative: The past matters. Santa needs to track conversations without forgetting who "Timmy" is or why he painted the cat blue.

The Tech: Vector stores, RAG patterns, multi-hop retrieval, session management, chain-of-thought reasoning, and qualitative ranking.

Phase 3 (Days 13-18): Agents & Tools

The Narrative: Santa realizes he can't do this alone. Enter Rudy (his first AI agent: anxious, brilliant, hyper-organized) and Elfie (enthusiastic, literal-minded, connected to the toy catalog).

The Tech: Defining AI agents with personas, function calling, multi-agent collaboration, API integration, guardrails, and error handling. You must teach them to collaborate without causing disasters (like the incident with the uranium-contaminated chemistry kit).

Phase 4 (Days 19-23): The Autonomous Workflow

The Narrative: Everything connects. The system must run on its own.

The Tech: Pipeline orchestration, context caching, multimodal consistency, human-in-the-loop patterns for high-stakes decisions, and automated reporting.

Phase 5 (Days 24-25): Production Readiness

The Narrative: A parent asks why their child got coal. You explain the reasoning. Then comes the final test: 5,000 letters, all at once.

The Tech: Explainability, observability, and a final comprehensive test that synthesizes everything.

How It Works

The repository is organized to mirror how you would actually work through this. You don't need to read ahead or understand the whole system before starting. You just need to show up for Day 1.

sleigh-ride-cloud-chronicles/
├── README.md
├── requirements.txt
├── datasets/          # Shared resources
├── resources/         # Constraints, personas, specs
├── day01/ through day25/
│   ├── task.md        # The narrative chapter & challenge
│   ├── input/         # Sample letters/data
│   └── output/        # Specification for success
└── utils/             # Your workspace

Time: One concept per day. 1-2 hours max.

Pace: Work through it at whatever pace your life permits. The repository is static and self-contained.

Prerequisites: You need an AWS account with Bedrock access, Python 3.9+, and a willingness to learn through story. No prior AI/ML experience is required.

What You'll Build

By the end of these 25 days, you will have built a complete pipeline:

Letter ingestion and entity extraction
Semantic search and behavior analysis
Multi-agent orchestration where AI personas collaborate
Validation layers that catch unsafe outputs
Explainability logging so every decision can be audited

More importantly, you'll understand how these pieces fit together. How embeddings connect to retrieval. How retrieval connects to generation. How generation connects to validation.

Your Mission

Santa's workshop is silent. The letters are piling up. And somewhere in the cloud, there is infrastructure waiting to be discovered.

Christmas hangs in the balance. And you have 25 days to help Santa save it.

🎁 Begin Your 25-Day Cloud + AI Journey
Start working through the challenges at your own pace in the Sleigh-Ride Cloud Chronicles GitHub repository.

PartyRock: Unleashing Creativity with SCAMPER

Gianluigi Mucciolo — Tue, 12 Mar 2024 18:05:22 +0000

I recently shared my fascination with PartyRock at an AWS User Group Rome meeting, discussing how I've ventured from RPG simulators to innovative chat applications. Highlighting the potential of generative AI with AWS, I showcased a new PartyRock app, emphasizing prompt engineering for interactive conversations and creative outputs. Catch the session on YouTube.

This journey inspired my latest project for the PartyRock Hackathon, utilizing the SCAMPER method to spur creativity and innovation among users, challenging conventional thinking.

Innovation from the Ground Up

My adventure began with a concept deeply rooted in the SCAMPER technique. This tool was imagined as a digital mentor, guiding users through the seven essential steps of creative problem-solving, pushing them to view challenges from perspectives they hadn't considered before.

Crafting with Cutting-Edge Technology

Amazon PartyRock became the foundation of my creation, bringing my ideas to life. Through immediate engineering and the use of the Claude model, I integrated the SCAMPER methodology into the core of my application. This combination was a deliberate choice, with the goal of positioning the tool at the intersection of technology and creativity.

A Step-by-Step Guide to Unleashing Creativity

Define the challenge: Start with clarity, introducing a section for users to articulate their challenge, laying the groundwork for targeted solutions.
Language Selection: Focusing on inclusivity, the tool includes a language selector, allowing users to explore solutions in the language they are most comfortable with.
Kickstart with Substitution: By activating the "Substitute Enabler," users can begin the substitution phase, examining alternatives and replacements for elements of their challenge, thereby broadening their horizons.
Explore Substitution in Depth: The tools includes a widget to help identify substitutable elements, offering guidance and summarizing proposed changes for a comprehensive exploration of potential solutions.
Navigate through the SCAMPER strategies: The tool includes a widget for each SCAMPER strategy: Replace, Combine, Adapt, Modify, Put to Another Use Items, Eliminate, Reverse, to Encourage a thorough investigation of all the possibilities for innovation.
Reveal the Final Solution: The "SCAMPER Result" widget presents an innovative solution or improvement at the end of the journey, demonstrating the effectiveness of structured creativity.
Address Contradictions: The "SCAMPER Contradictions" widget ensures the solution's coherence by addressing any contradictions that might emerge during the brainstorming process.
Expand with AI Insights: The "Critical AI SCAMPER" widget pushes the boundaries further, offering AI-driven suggestions to refine and enhance the idea.

Navigating Challenges and Learning Along the Way

A significant challenge was ensuring smooth performance with multiple widgets running at the same time. Introducing an "ON/OFF" feature dramatically improved the user experience. This journey wasn't just about creation; it deepened my understanding of prompt engineering and highlighted the importance of performance optimization.

The Road Ahead

Looking forward, I'm focused on enhancing the tool's performance and functionality. The community's engagement excites me, and I'm keen to integrate their feedback into the continuous development of our tool.

SCAMPER: A Powerful Tool for Creative Problem-Solving

Building Elastic and Fully Managed Cloud-Native VectorDB Milvus Infrastructure on AWS

Gianluigi Mucciolo — Tue, 05 Mar 2024 15:51:57 +0000

Introduction

Milvus is an advanced open-source vector database designed to revolutionize AI and analytics applications. With its ability to handle high-performance similarity search on massive-scale vector datasets, Milvus has become an essential tool for businesses in today's data-driven world. From recommendation systems to image recognition and natural language processing, Milvus enables organizations to unlock the full potential of generative AI algorithms and extract valuable insights from complex data.

To fully leverage the power of Milvus, deploying it within a robust, scalable, and managed infrastructure is crucial. In this blog post, I will explore how you can build an elastic and fully managed cloud-native Milvus infrastructure on AWS, taking advantage of its scalability, reliability, and ease of management. By harnessing the capabilities of Milvus in combination with AWS services, businesses can supercharge their generative AI initiatives and achieve remarkable results in fields such as content generation, recommendation engines, and personalized user experiences.

Overview of Milvus Architecture

At the core of Milvus lies its shared-storage architecture, consisting of four essential layers: the access layer, coordinator service, worker node, and storage. This architectural design allows for scalability, as well as the disaggregation of storage and computing resources, resulting in a cost-efficient and flexible infrastructure. The independent scalability of each layer further enhances the system's agility and resilience, ensuring seamless disaster recovery.

Milvus architecture distinctively separates compute resources from storage, incorporating dedicated storage nodes. A significant advantage of this infrastructure is the ability to scale the compute cluster down to zero while maintaining the data nodes active. This setup ensures that data remains accessible in Amazon S3, providing a flexible and efficient way to manage resources without compromising data availability.

The above Milvus architecture is from the official documentation.

Existing Infrastructure on Kubernetes

The current deployment of Milvus operates on Kubernetes, utilizing components such as etcd for distributed key-value storage, MinIO for object storage, and Pulsar for distributed messaging. While this setup is functional, Milvus architecture is designed to be portable, allowing it to run on various infrastructures. To leverage the benefits of an AWS-native solution and further enhance the deployment, you can introduce updates to leverage Serverless and fully-managed solutions available on AWS.

Serverless on AWS provides technologies for running code, managing data, and integrating applications without the need to manage servers. It offers automatic scaling, built-in high availability, and a pay-for-use billing model, increasing agility and optimizing costs. By leveraging serverless technologies, you can enhance the scalability and efficiency of the Milvus deployment on AWS.

AWS fully-managed services, on the other hand, are Amazon's cloud computing provisions where AWS handles the entire infrastructure and manages the required resources to deliver reliable services. This includes managing servers, storage, operating systems, databases, and other critical resources fundamental to the service infrastructure. By utilizing fully-managed services, you can ensure a robust and reliable Milvus deployment on AWS, reducing operational overhead and increasing the focus on utilizing Milvus's capabilities effectively.

By transitioning the existing Kubernetes deployment of Milvus to leverage serverless and fully-managed solutions on AWS, you can unlock the full potential of Milvus in terms of scalability, reliability, and ease of management. In the next sections, I will explore the proposed infrastructure using AWS services and its benefits in building an elastic and fully managed cloud-native Milvus infrastructure.

Proposed Infrastructure with AWS Services

To enhance the Milvus deployment on AWS, I propose replacing certain components with AWS services that offer scalability, reliability, and ease of management. These replacements include:

MSK (Managed Service for Apache Kafka): MSK replaces Pulsar for messaging and log management. It provides a fully managed Kafka service that ensures robust messaging and log processing, allowing for seamless integration into your Milvus deployment. For future exploration, it is worthwhile to consider utilizing AWS Kinesis, a fully managed streaming service that offers seamless integration with the AWS ecosystem.
Aurora Serverless: Aurora Serverless replaces etcd as the metadata storage and coordination system. It offers a serverless database service that automatically scales to match workload demands. With Aurora Serverless, you can ensure efficient and scalable management of metadata in your Milvus infrastructure. Currently Milvus only supports MySQL, but as an alternative metastore, it is also worth exploring the use of AWS DynamoDB, a highly scalable NoSQL database optimized for key-value workloads.
Application Load Balancer (ALB): ALB handles load balancing and routing of Milvus requests, ensuring high availability and efficient distribution of traffic to the various components. ALB's dynamic routing capabilities enable seamless traffic management within the Milvus infrastructure.
Amazon S3: Amazon S3 replaces MinIO for data persistence. It offers highly scalable, reliable, and cost-effective object storage. By leveraging Amazon S3, you can achieve seamless data persistence for your Milvus deployment, while benefiting from the scalability and durability of AWS's object storage service.
Amazon ECS: Milvus containers can be effortlessly deployed on AWS Fargate, a serverless compute engine specifically designed for containers. By utilizing ECS Fargate, you liberate yourself from the complexities of managing underlying infrastructure, enabling you to devote your attention to fine-tuning resource utilization and elevating the performance of your Milvus deployment. For future explorations, you can draw inspiration from the design considerations of Aurora Serverless for high throughput cloud-native vector databases. This involves separating storage and computation, ensuring that you only pay for computational power when it is actually needed, resulting in optimized cost efficiency and enhanced scalability.
AWS Cloud Map: Milvus distributed infrastructure requires effective service discovery mechanisms to enable efficient management and scaling of applications. With AWS Cloud Map, you can easily locate and communicate with the services you need, without the hassle of managing your own service registry.

By incorporating these AWS services and considering future possibilities, you can build an elastic and fully managed cloud-native Milvus infrastructure that maximizes scalability, reliability, and operational efficiency. In the next sections, I will delve into the architecture of this new infrastructure and explore its benefits in detail.

Architecture of the New Infrastructure

In the proposed infrastructure, AWS services seamlessly integrate into the Milvus deployment, enhancing scalability, manageability, and overall performance. MSK, Aurora Serverless, ALB, Amazon S3, and ECS Fargate play pivotal roles in ensuring a robust and elastic infrastructure.

Benefits of AWS Services for Milvus

The adoption of AWS services brings several key advantages to Milvus deployments:

Scalability: AWS services such as MSK, Aurora Serverless, and ECS Fargate enable effortless scaling of resources based on workload demands. This ensures efficient management of high-volume data, allowing your Milvus deployment to handle growing datasets with ease.
Managed Services: By leveraging managed services, you can significantly reduce operational overhead. AWS takes care of the underlying infrastructure, ensuring high availability and durability. This allows you to focus on leveraging Milvus's capabilities without the burden of managing the infrastructure yourself.
Reliability: AWS services provide a robust and reliable infrastructure, offering stability and performance for your Milvus deployment. With built-in redundancy and fault-tolerant designs, you can trust that your Milvus infrastructure will operate smoothly and reliably.
Cost Efficiency: AWS services offer cost-effective solutions for Milvus deployments. Services like Aurora Serverless and ECS Fargate enable you to pay only for the computational resources you actually use, optimizing cost efficiency. Additionally, Amazon S3 provides highly scalable and cost-effective object storage, eliminating the need for managing and provisioning your own storage infrastructure. By leveraging AWS services, you can achieve significant cost savings while maintaining the scalability and reliability required for your Milvus deployment.

By incorporating these benefits into your Milvus deployment, you can unleash the full potential of Milvus for high-performance similarity search on massive-scale vector datasets, while ensuring scalability, reliability, and cost efficiency.

Deployment Process on AWS and Challenges

I began by following the official instructions currently available in the documentation, which presents two options for deploying Milvus in Kubernetes: using Terraform and Ansible, or employing Docker Compose (not recommended for production environments). Initially, I opted for Docker Compose and attempted to deploy it on Amazon ECS using the ecs-cli. However, I encountered several incompatibilities and, after many hours of effort, decided to abandon Docker Compose. Despite this setback, the experience proved to be invaluable, as it greatly enhanced my understanding of both ecs-cli and Milvus' internal architecture.

Consequently, I decided to build the entire infrastructure from scratch. Given my previous experience, this approach seemed far simpler to manage. I began by deploying the Virtual Private Cloud (VPC), the ECS cluster, and then proceeded to install each of the Milvus components individually. During this process, Milvus introduced support for multiple coordinators in both active and standby modes, further complicating deployment, but in a more exciting way.

One of the most significant challenge I faced—and continue to face—is related to ETCD. As you may know, ETCD utilizes the Raft protocol, enabling a cluster of nodes to maintain a replicated state machine. I managed to deploy a single ETCD node in ECS, but to get Raft working, I had to implement several workarounds, such as assigning task names using tags. While not ideal, it was the only viable solution, particularly since ECS does not yet support StatefulSets.

Currently, I have a functioning cluster with ETCD that lacks high availability. If you have any suggestions on how to enhance the architecture, or if you're interested in collaborating on this project, your participation would be greatly appreciated.

Additionally, if you're willing, please consider helping to make the StatefulSets feature available on ECS by supporting this request: https://github.com/aws/containers-roadmap/issues/127 . 🙏

Conclusion

Deploying Milvus on AWS using managed services like MSK, Aurora Serverless, ALB, Amazon S3, and ECS Fargate offers significant benefits in terms of scalability, reliability, and ease of management. By adopting this infrastructure, businesses can unlock the full potential of Milvus for high-performance similarity search on massive-scale vector datasets. With AWS services, you can build an elastic and fully managed cloud-native Milvus infrastructure that can handle the most demanding AI and analytics workloads.

Serverless Apache Zeppelin on AWS

Gianluigi Mucciolo — Sun, 04 Feb 2024 12:20:04 +0000

What is Apache Zeppelin?

First of all, it is worth asking: what is a notebook interface? A notebook is an interface for interactively running code; it lets you explore and visualize data. You can mix narrative, rich media, and data in a unique space.

Now we can proceed with the definition of Apache Zeppelin. It is a web-based notebook that enables data-driven, interactive data analytics and collaborative documents with Python, Scala, SQL, Spark, and more. You can execute code and even schedule a job (via cron) to run at regular intervals.

It's easier to mix languages in the same notebook. You can write some code and then use markdown to document it all together. You can also easily convert your notebook into a presentation style - perhaps for presenting to management or using dashboards.

What does Serverless Means?

The idea behind serverless is that you as a developer shouldn't need to care about the server infrastructure. You pay to run the code without concerns about what type of physical infrastructure is running below.

There are quite a few advantages to serverless. Scalability essentially comes for free. Because you're just paying to run logic, the cloud provider can easily dedicate more hardware to run your code. Also, you pay by code execution rather than having a fixed rate. Even more, the cloud provider manages the server software and hardware. You shouldn't need to worry about that. Finally, serverless frees up developers to focus on what they're good at - coding.

Solution Requirements

Build a serverless infrastructure to run Apache Zeppelin and persist notebook files. The solution must be publicly available and provide login and logout capability. Also, the compute platform must automatically shut down after 30 minutes of inactivity.

High-level Architecture

The diagram below shows the high-level architecture. As you can see, it is a serverless infrastructure, and you can operate Apache Zeppelin using a public endpoint while Elastic File System stores the notebook files. Amazon CloudWatch custom metric counts the lines of logs and shuts down the Amazon Fargate container after 30 minutes of inactivity.

The only missing feature in this architecture is the login and logout capability. In this case, Apache Zeppelin provides Shiro for notebook authentication. Apache Shiro is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management. Here, you can find a step-by-step guide about how Shiro works. This example uses the default configuration.

Infrastructure as Code Description

The solution uses AWS SAM with the global configuration for Lambda functions and the public API you can use to access Apache Zeppelin. The stack deployment provides the URL as an output value.

Amazon API Gateway

Amazon API Gateway is used as the front door to interact with the application; it exposes the URL the user can use to trigger operations and use Serverless Apache Zeppelin.



AWSTemplateFormatVersion: '2010-09-09'
Globals:
  Function:
    Timeout: 60
    MemorySize: 128
    Architectures: 
      - arm64
Parameters:
  [...]

Resources:
  ZeppelinApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref ServiceName

Outputs:
   ZeppelinApi:
     Description: "API Gateway endpoint URL for Prod stage for Hello World function"
     Value: !Sub "https://${ZeppelinApi}.execute-api.${AWS::Region}.amazonaws.com/${ServiceName}/"

Elastic File System

When provisioned, each Amazon ECS task hosted on AWS Fargate receives ephemeral storage for bind mounts; everything on the disk is lost after container termination. To persist notebook files, the solution uses Amazon Elastic File System; all notebooks on EFS are preserved after container termination. The Access Point configuration allows Apache Zeppelin to have write permissions on Amazon Elastic File System.




AWSTemplateFormatVersion: '2010-09-09'
Globals:
  Function:
    Timeout: 60
    MemorySize: 128
    Architectures: 
      - arm64
Parameters:
  [...]

Resources:
  ZeppelinApi:
    [...]

  AccessPoint:
    Type: 'AWS::EFS::AccessPoint'
    Properties:
      FileSystemId: !Ref FileSystem
      PosixUser:
        Uid: "500"
        Gid: "500"
        SecondaryGids:
          - "2000"
      RootDirectory:
        CreationInfo:
          OwnerGid: "500"
          OwnerUid: "500"
          Permissions: "0777"
        Path: !Sub "/${ServiceName}"
  FileSystem:
    Type: AWS::EFS::FileSystem
    Properties:
      PerformanceMode: generalPurpose
      FileSystemTags:
      - Key: ServiceName
        Value: !Ref ServiceName
  MountTarget1:
    [Availability Zone A Configuration]
  MountTarget2:
    [Availability Zone B Configuration]
  MountTarget3:
    [Availability Zone C Configuration]

Amazon Cloud Watch Custom Metric

To provide an auto-shutdown feature, the Apache Serverless solution uses a custom metric. AWS Fargate saves logs into an Amazon CloudWatch Log Group, and the Amazon CloudWatch Custom Metric Filter counts the log lines. If the custom metric is zero for about 30 minutes, the alarm publishes a message to Amazon Simple Notification Service to terminate the Task.



AWSTemplateFormatVersion: '2010-09-09'
Globals:
  Function:
    Timeout: 60
    MemorySize: 128
    Architectures: 
      - arm64
Parameters:
  [...]

Resources:
  ZeppelinApi:
    [...]
  AccessPoint:
    [...]
  FileSystem:
    [...]
  ShutdownSnsTopic:
    [description later in this post]

  ZeppelinLogGroup:
    Type: AWS::Logs::LogGroup
    Properties: 
      LogGroupName: !Sub "/ecs/fargate-${ServiceName}"
      RetentionInDays: 1
  ActivityMetricFilter: 
    Type: AWS::Logs::MetricFilter
    Properties: 
      LogGroupName: !Ref ZeppelinLogGroup
      FilterPattern: "INFO"
      MetricTransformations: 
        - 
          MetricValue: "1"
          MetricNamespace: !Sub "${ServiceName}/Actions"
          MetricName: "ActionsCount"
  ZeppelinActionsCountAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: ZeppelinActionsCountAlarm
      MetricName: ActionsCount
      Namespace: !Sub "${ServiceName}/Actions"
      Statistic: SampleCount
      Period: '300'
      EvaluationPeriods: '6'
      TreatMissingData: breaching
      Threshold: '1'
      ComparisonOperator: LessThanOrEqualToThreshold
      AlarmActions:
      - !Ref ShutdownSnsTopic

AWS Fargate

Here is the AWS Fargate Cluster and Task Definition. The Apache Serverless solution uses Shiro to enable login and logout capability. As stated here, you can create a shiro.ini file by executing the cp command. You can find it in the EntryPoint property of the container definition.



AWSTemplateFormatVersion: '2010-09-09'

Globals:

  Function:

    Timeout: 60

    MemorySize: 128

    Architectures: 

      - arm64

Parameters:

  [...]

Resources:

  ZeppelinApi:

    [...]

  AccessPoint:

    [...]

  FileSystem:

    [...]

  ZeppelinLogGroup:

Cluster:

    Type: AWS::ECS::Cluster

    Properties:

      ClusterName: !Join ['', [!Ref ServiceName, Cluster]]

  ZeppelinTaskDefinition:

    Type: AWS::ECS::TaskDefinition

    Properties:

      RequiresCompatibilities:

        - "FARGATE"

      Cpu: !Ref ContainerCPU

      Memory: !Ref MemoryHardLimit

      NetworkMode: "awsvpc"

      TaskRoleArn: !GetAtt ZeppelinTaskRole.Arn

      ExecutionRoleArn: !GetAtt ZeppelinTaskRole.Arn

      ContainerDefinitions:

        - Name: !Ref ServiceName

          Image: "apache/zeppelin:0.10.0"

          EntryPoint: 

            - /bin/bash

            - -c

            - |

              cp conf/shiro.ini.template conf/shiro.ini 

              /usr/bin/tini -- bin/zeppelin.sh

          Command: ["echo", "done!"]

          MemoryReservation: !Ref MemorySoftLimit

          Memory: !Ref MemoryHardLimit

          PortMappings:

            - ContainerPort: !Ref ContainerPort

              Protocol: tcp

            - ContainerPort: 4040

              Protocol: tcp

          LogConfiguration:

            LogDriver: awslogs

            Options:

              awslogs-group: !Ref ZeppelinLogGroup

              awslogs-region: !Ref AWS::Region

              awslogs-stream-prefix: !Sub 'ecs-${ServiceName}-awsvpc'

          MountPoints:

            - ContainerPath: !Ref ZeppelinPersistNotebookPath

              SourceVolume: !Sub "${ServiceName}"

              ReadOnly: false

      Volumes:

        - Name: !Sub "${ServiceName}"

          EFSVolumeConfiguration:

            AuthorizationConfig: 

              IAM: ENABLED

              AccessPointId: !Ref AccessPoint

            FilesystemId: !Ref FileSystem

            TransitEncryption: ENABLED

AWS Lambda | Workflow

Below is the high-level workflow about how the implementation works, how the task is created, and shut down.

Start Apache Serverless

In the beginning, it checks if the Apache Zeppelin Container is running.

In case of a yes, AWS Lambda returns 302 to the Apache Zeppelin public IP. In case of a no, AWS Lambda executes the next step. Then, it checks if the Apache Zeppelin Container exists.

In case of a yes, AWS Lambda returns static web content. It is a loading page with an auto-refresh every 20 seconds. In case of a no, AWS Lambda starts a new Apache Zeppelin container and returns the loading page. Every 20 seconds, the client checks Apache Zeppelin provisioning and gets the notebook interface if the container is running; otherwise, it gets the loading page. When you have the notebook interface, to use Apache Zeppelin, you must provide your user credentials.

Shutdown Apache Serverless

If the custom metric is zero for about 30 minutes, the alarm publishes a message to Amazon Simple Notification Service, and an AWS Lambda Function terminates the cluster. The Amazon Simple Notification Service is the AWS Lambda Function trigger.

Usage Suggestions & Improvements

Apache Zeppelin supports Amazon S3 for persisting notebook files. As stated here, you can use ZEPPELIN_NOTEBOOK_STORAGE, ZEPPELIN_NOTEBOOK_S3_BUCKET, and ZEPPELIN_NOTEBOOK_S3_USER as environment variables.

On the other hand, Amazon Elastic File System offers a very generic solution that can be used for various purposes; the only limit is your imagination. Since Amazon EFS is a file system, you don't have to deal with Amazon S3 Object Storage. In this case, you can simply upload your application to a Docker container and run it on AWS Fargate, just by replacing Apache Zeppelin.

For example, you can run Serverless Visual Studio Code; check the container here.

Another improvement related to Serverless Apache Zeppelin on AWS is configuring Amazon DynamoDB as an external database for Shiro users.

What will be your next application to deploy as Serverless?

Innovative announcements from Amazon Bedrock at re:Invent: A turning point in the AI industry

Gianluigi Mucciolo — Sat, 02 Dec 2023 00:00:48 +0000

In 2023, there have been numerous developments in the field of artificial intelligence, characterized by the introduction of new libraries, tools, models and benchmark architectures. Among these, the recent announcements by Amazon re:Invent represent a significant turning point, bringing clarity and structure to a previously chaotic field. This development makes it possible to focus on the "important things": It's not just about knowing the tools and models, but also about creating a solid data foundation for further developments.

Amazon Bedrock and its development:

In the landscape of artificial intelligence, the recent developments of Amazon Bedrock represent a qualitative leap that brings clarity and organization to a field that was previously considered disorganized. At the heart of this innovation is the fully managed Retrieval Augmented Generation (RAG) solution with knowledge bases, which emphasizes the importance of data over algorithms and shifts the focus from simply knowing about tools and models to building robust data foundations. This approach puts an end to tinkering in the field of AI and allows a focus on more important aspects: Data!

The latest additions to Amazon Bedrock, such as personalization, agents and guardrails, extend this focus even further. Personalization allows AI models to be tailored to a company's unique style, a model evaluation that helps you choose the best model for your needs, while Agents simplify business tasks such as order management or customer support. Guardrails, on the other hand, standardize security controls and ensure a consistent and secure user experience. These features integrate and extend RAG, contributing to a robust and versatile AI ecosystem.

Amazon Q and integration with VSCode:

The launch of Amazon Q represents a significant advancement in the field of artificial intelligence, particularly in the context of the workplace. This generative AI assistant can be customized to specific business needs and, thanks to the ability to connect and external KB, can be effectively integrated into existing information systems and, in the future, into any AWS service. In line with industry trends where companies like Microsoft have already integrated similar tools into their operating systems and applications, Amazon Q represents another step towards effective synergies between data and humans.

Moreover, its integration with VSCode opens up new perspectives, especially when managing AWS-related tasks directly from the IDE. This integration is characterized by the fact that the length of the prompt is different between the console (1000 characters) and the IDE (4000 characters), which allows for greater flexibility. Currently, Amazon Q cannot access the text that is currently being worked on. I hope that this limitation will be overcome in the future to achieve greater synergy with Codewhisperer. The latter could significantly improve the generated code, especially if the possibility of a dialog with Amazon Q to refine the code generated by Codewhisperer is implemented, facilitating problem solving and code optimization.

Final thoughts:

Recent announcements in the field of artificial intelligence show that the use of these technologies is maturing. Although there are still pending expectations, such as the introduction of a vector database in Redshift, I remain optimistic about future innovations. A vector database in Redshift would be particularly useful for projects such as https://memgpt.ai/, which proposes to manage virtual contexts, taking inspiration from the hierarchical storage systems of traditional operating systems. This approach creates the illusion of large memory resources by moving data back and forth between fast and slow memory. This technology, in combination with Claude 2.1 200k, could be a big leap forward. I had hoped that OpenSearch, which is currently getting a lot of attention, combined with Amazon S3 (UltraWarm storage) would fill the gap left by the lack of a vector database in Redshift, but given the current limitations in approximate k-NN functions, this is not possible. I hope that these technological gaps will be closed in the future to further improve the capabilities and efficiency of AI-based systems.

Images by Titan Image Generator G1.