Forem: Amador Criado

RAG Chatbot with Amazon Bedrock & LangChain

Amador Criado — Mon, 04 Nov 2024 15:32:01 +0000

Introduction

Large Language Models (LLMs) are revolutionizing how we interact with information, but they face challenges with accuracy and access to up-to-date data. Retrieval Augmented Generation (RAG) addresses these limitations by grounding the LLM's responses in a dedicated knowledge base.

This article explores a project based on the implementation of a RAG chatbot using Amazon Bedrock and LangChain that enhances the chatbot's ability to provide more contextually relevant, and current information, making it a key approach for a wide range of applications based on generative AI. The full codebase of the project is available in my GitHub repository.

Design considerations

As can be seen in other articles of my Blog, I usually review the design considerations of any project, workload or cloud architecture showcase in order to provide more wide vision about the main purpose of it. For this case, the following design principles guided the development:

Serverless LLM Architecture : Leveraging Amazon Bedrock's serverless infrastructure eliminates the need for managing infrastructure, simplifying deployment, scaling, and experimentation. This allows developers to focus on building the core logic of the chatbot without worrying about server management.
Modularity: The codebase is designed with modularity in mind, facilitating easy extension, component swapping, and simplified maintenance as the project evolves. This approach promotes code reusability and makes it easier to integrate new features or adapt to changing requirements.
Streamlit for Rapid Prototyping: Streamlit's intuitive framework allows for rapid prototyping and development of user interfaces.
Dockerized Deployment: The project is containerized using Docker, ensuring consistent execution across different environments and simplifying deployment to various platforms.
Extensibility: The design allows for easy integration with different data sources and future enhancements like personalized responses or multi-modal search.
Cost-Effectiveness: While not fully optimized for production, cost-effective practices are considered, such as choosing appropriate model sizes and efficient data handling. This helps keep experimentation costs manageable and lays the groundwork for future optimization in a production environment.

Architecture

Key Components

Knowledge Base: Designed to be flexible, can ingest data from various sources, including CSV files, potentially expanding to databases and other formats in future iterations.
Vector Database (ChromaDB): Stores vector embeddings of the knowledge base generated using a suitable embedding model like amazon.titan-embed-text-v2:0 . This allows for efficient similarity searches when retrieving relevant information.
Streamlit: Provides the user interface for interacting with the chatbot. Streamlit's intuitive API and interactive components make it easy to build a user-friendly interface for querying the knowledge base and visualizing responses.
Amazon Bedrock: Provides access to the foundation models powering the chatbot. Specifically, amazon.titan-embed-text-v2:0 is used for creating text embeddings, while anthropic.claude-3-sonnet-20240229-v1:0 is employed for text generation and driving the conversational aspect of the application. Bedrock's serverless infrastructure simplifies deployment and experimentation with these powerful models.
LangChain: Orchestrates the interaction between the vector database, LLM, and user interface. It streamlines the development process and manages the retrieval and generation workflow.

Workflow

Data Preprocessing
The user enters a new question in the Streamlit Chat App.
The Chat history is retrieved from memory object and added prior the new message entered.
The question is converted to a vector using Amazon Titan Embeddings, then matched to the closests vector in the database to retireve context.
The combination of new Question received, Chat history, and Context from the Knowledge base are sent to the model.
The model's response is displayed to the user in the StreamLit App.

{
  "id": 1,
  "document": "What is Amazon Bedrock?\\nAmazon Bedrock is a fully managed service...",
  "metadata": {
    "topic": "bedrock"
  },
  "embedding": [-0.1018051, 0.01927839, 0.004059858, ...]
}

"id": A unique integer ID for each text chunk.
"document": The raw text content of the chunk, which includes questions like "What is Amazon Bedrock?" and their corresponding answers. Notice the \\n indicating newline characters within the text.
"metadata": Currently, metadata includes a "topic" field set to "bedrock", suggesting this data relates to information about Amazon Bedrock. This could be expanded to include other relevant metadata like source or date.
"embedding": A 768-dimensional vector representing the semantic meaning of the "document". These embeddings are pre-calculated using the amazon.titan-embed-text-v2:0 model, which supports a flexible embedding dimension size, and are crucial for efficient similarity search within the vector database. Storing them directly in the JSON avoids recalculation during query time, significantly improving performance.

This embedding strategy, coupled with the structured JSON format, optimizes retrieval efficiency and allows the chatbot to generate more relevant and contextually appropriate responses.

Conclusion

This RAG chatbot prototype provides a solid starting point for developers looking to explore and experiment with retrieval augmented generation. By combining Amazon Bedrock, Pinecone, and LangChain, we can build intelligent conversational AI systems that are more grounded and informative. The project demonstrates the potential of RAG for building a new generation of conversational applications.

Next Steps

The mentioned GitHub repository amazon-bedrock-rag-chatbot contains the complete code and instructions for running the project in local environment. Looking forward, next feature development could focus on:

Expanding data ingestion capabilities: Supporting more diverse data sources and formats.
Improving retrieval accuracy: Experimenting with different retrieval strategies and ranking algorithms.
Exploring advanced features: Adding personalization, multi-modal search, and more sophisticated user interfaces.

Ingesting F1 Telemetry UDP real-time data in AWS EKS

Amador Criado — Thu, 17 Oct 2024 10:28:13 +0000

Introduction

In this post I’ll continue with my series of Real Time data integration in Cloud. This time I’ll explain How I built a real time telemetry data ingestion pipeline using AWS EKS to process UDP data from the F1 2023 Playstation 4 game. The solution efficiently handles the ingestion, processing and visualization of live racing telemetry, all while ensuring scalability, reliability and cost-effectiveness.

One of the reasons because this use case was chosen because F1 car telemetry data is an excellent representation of high-frequency, combined with the complexity of UDP as a protocol make it an ideal scenario to showcase the solution.

As a Cloud Architect, I cannot proceed with the article without remarking that I review and apply the AWS Well Architected Framework in any project but also in my personal projects like this. The major part of the architectural decisions were made according to best practices and design principles of Well Aerchitected Framework. For example, using Spot instances in order to achieve cost efficiency, EKS/Kubernetes for scalability, and Grafana for maintain operational excellence via monitoring.

Design Considerations

As I’ve commented in the Introduction, the architecture was designed with several WAF key principles in mind:

Scalability: The system is capable of scaling to ingest and process real time data without performance degradation. AWS EKS and kubernetes provide a level of elasticity that allow to scale in and out in terms of Cluster and also more specifically horizontally the pods can leverage HPA (Horizontal Pod Austoscaling) in K8s (Kubernetes).
Reliability: A TCP HealthCheck sidecar ensures that the UDP listener service remains available, preventing data loss.
Cost Optimization: Spot instances and ARM-based instances (C6g) were chosen for their lower cost, while maintaining the necessary performance.
Performance efficiency: Low latency and high troughput is required. for that reason, the combination of Spot Instances Network load balancer and A*RM based architecture* is the best suited for high performance network operations.
Security: A Custom private network, with Load Balancers and security groups fine grained to allow only the specified access/permissions. In addition, AWS IAM policies and EKS Teams ensures to manage access securely.

Why UDP ?

I consider to slightly pause here, prior to go deep in the architecture details, and explain a bit the protocol that we are going to ingest in AWS for this architecture, that is UDP (User Datagram Protocol), to handle the telemetry data generated from the F1 2023 game.

UDP is a connectionless protocol, meaning it sends data without establishing a persistent connection between the sender and receiver, unlike TCP (Transmission Control Protocol), which maintains a connection and ensures reliable delivery through acknowledgments and retransmissions.

For the scope of this article is interesting to briefly explain and justify why UDP is being used, and not only in this case (F1 2023 game) but also as a standard to send telemetry or sensors/iot data. There are several reasons for that:

Low latency: In a fast-paced environment like racing, timing is crucial. UDP allows data to be sent without waiting for ACK from the receiver, unlike TCP, which requires confirmation that each packet has been received successfully. That makes UDP faster by reducing considerably latency.
Real-Time Data: Telemetry data becomes outdated very quickly, if a packet is lost, it’s often more important to receive the next one as soon as possible rather than retry sending old data.
High Throughput: UDP can handle a large volume of data without the overhead of managing the connection, errors, or delivering ordered.
Lightweight: The F1 telemetry is broadcast to multiple clients (such as applications or external systems) UDP is more efficient for multicast communication because don’t require setting up individual connections (as TCP would)

On the other hand, there are a few drawbacks that should be taken into account:

No guaranteed delivery: As UDP does not provide ACK of received packets, there’s no guarantee that the data will reach the destination.
No Error Connections: Corrupted data packets may be received without any mechanism to request a retransmission.
Out-of-Order delivery
Lack of congestion and Flow control: Unlike TCP, UDP doesn’t adjust retransmission rate based on network conditions. The quantity of data sent at once either is controlled, meaing that there’s a risk of overhelming the receiver if the data stream is too. fast and too frequent.

Let’s remark that for the purpose of our project, some of this drawbacks are not handled as we consider them ‘out of the scope’, but, I still consider interesting to take them into account and include them if this architecture is used as a blueprint for a new workload. It’s important to remark, that for the specific case of real-time F1 car telemetry, the drawbacks are mitigated by the fact that data is constantly being generated and not persisted. It’s only being monitored in real-time.

Architecture

High-level architecture diagram of the solution:

Key Components

PlayStation 4 with F1 2023: Sends real-time telemetry data via UDP to a specific IP and port. The game allows for the data to be broadcast to the entire network or to a targeted endpoint.
EKS Cluster: Deployed in the eu-central-1 region via AWS CDK (EKS Blueprints). The cluster is composed of spot instances, specifically ARM-based C6g instances.
Network Load Balancer (NLB): Exposes the UDP listener service to the public internet, mapping to a static Elastic IP.
Application Load Balancer (ALB): Provides secure access to the WebSocket server for external clients and to Grafana for real-time telemetry visualization.
UDP Listener Service: Receives telemetry data from the PlayStation and processes the UDP packets according to the F1 game’s telemetry specification. A sidecar service (Nginx) performs TCP health checks to ensure availability.
WebSocket Server: Publishes the telemetry data to connected clients. For this project, only speed data is transmitted.
Grafana OSS: Configured with a WebSocket plugin to allow real-time visualization of telemetry data, enabling users to monitor key metrics like vehicle speed.

Workflow

The diagram has been labeled with the main workflow process for telemetry ingestion. The different steps have been brifly explained next:

The PlayStation 4 with F1 2023 sends UDP telemetry data to the Elastic IP associated with the Network Load Balancer.
The NLB forwards this data to the UDP listener service running on AWS EKS.
A sidecar TCP health check ensures the UDP listener's availability by monitoring the service.
The UDP listener processes the telemetry data and forwards it to the WebSocket server.
The WebSocket server broadcasts the telemetry data to connected clients.
Grafana, connected via the WebSocket plugin, visualizes the data, providing real-time insights into the car's speed.

Demo / Showcase

After deploying the project in my AWS personal account I’ve recorded myself playing a short Race in the official Formula1 Circuit de Barcelona-Catalonia . Obviously is my favourite circuit, as I was born a few kilomentres away from it.

Next we can see the recorded game and, to monitor the telemetry generated, the Logs of Websocket server generated in real-time and the grafana dashboard with speed gauge configured:

Conclusion

This architecture demonstrates how AWS EKS, in combination with other services, can build a scalable, reliable, and cost effective solution for ingestion UDP data. In my opinion, this open source project serves well as a blueprint or template for integrating UDP-based real-time communication into the cloud. The workload is simple and cover only one use case (Car’s speed), but can be considered as a starting point, and can be adapted for other high frequency data streaming use cases, making it flexible and scalable template.

From an operational perspective, I want to mention that I’ve used Infrastructure as code (IaC) by defining the source code of the recources through CDK and the Kubernetes manifests via kustomize and ArgoCD for easy gitops. This ensures a repeatable ans maintanable process for scalins and evolving the olution over time.

For local testing, I’ve used Minikube, in order to emulate Kubernetes, that allowed me to test the deployments before to rolling them out to AWS, ensuring a smooth integration with the target environment (develop for this project)

Next Steps

The full code of this solution is available in my GitHub repository:

https://github.com/acriado-dev/aws-eks-udp-telemetry

Future improvements could include processing all Car’s telemetry or adding data analytics for the data ingested. I’m always openned to add some functionality in the project.

Connecting an Arduino MKR WiFi 1010 to AWS IoT Core

Amador Criado — Fri, 13 Sep 2024 08:56:44 +0000

Introduction

Internet of Things (IoT) has revolutionized how we interact with the world. It involves connecting everyday objects to the internet, enabling them to collect, send, and receive data. This connectivity allows for smarter decision-making, automation, and a new level of interactivity between the physical and digital worlds.

As a Cloud Architect, I’ve had the opportunity to work on several IoT cloud projects where the data from connected devices was processed centrally in AWS. These projects involved analyzing the data to drive critical business logic decisions, leading to improved efficiency, enhanced user experiences, and even the creation of new business models.

In this article, I’ll guide you through a step-by-step process of connecting an Arduino MKR1010 to AWS IoT Core. This tutorial is designed to help you understand how to leverage AWS’s powerful cloud services to manage and analyze IoT data, helping you to get started the way for your own innovative projects.

What is Arduino MKR Wifi 1010?

The Arduino MKR WiFi 1010 is the easiest point of entry to basic IoT and pico-network application design. Whether you are looking at building a sensor network connected to your office or home router, or if you want to create a Bluetooth® Low Energy device sending data to a cellphone, the MKR WiFi 1010 is your one-stop-solution for many of the basic IoT application scenarios.

The board is based on the SAMD21 microcontroller, which is a low-power ARM Cortex-M0+ processor, and includes an integrated u-blox NINA-W102 module for WiFi and Bluetooth connectivity.

Development Environment

Prerequisites

1- AWS IoT Account

Sign up for an AWS account if you don’t already have one: AWS Sign-Up
Select a predefined AWS region, ideally one closer to your current location, in our case will be Paris (eu-west-3)

Caution: not all regions support AWS IoT

2- Platform.io

I've selected Platform.io as development environment for Arduino projects because it offers enhanced features compared to the traditional Arduino IDE. It integrates seamlessly with Visual Studio Code, providing advanced code editing tools, debugging capabilities, and project management features

I recommend to follow the Platform.io Quick start guide

3- Arduino MKR1010 and USB cable

Arduino MKR WiFi 1010 board.
Micro-USB to USB cable to connect the MKR1010 to your computer.

Configuring the Board

In order to securely connect the Board to AWS there is a mandatory step that we have to follow. We have to create an sketch to generate a CSR for a private key generated in an ECC508/ECC608 crypto chip slot. Once the skecth is uploaded to the board, the user is prompted for the following information that is contained in the generated CSR:

country
state or province
locality
organization
organizational unit
common name

We can consider optional all fields except common name. After providing it, the user can also select a slot number to use for the private key.

Let's define the steps to generate the CSR:

Open Platform.io and ensure that the Board is available
Create new Project named ArduinoCSRConfig

Install ArduinoECCX08 library for this Project
In /src folder, modify main.cpp to include the sketch required, should be like:

/*
  ArduinoECCX08 - CSR (Certificate Signing Request
  The circuit:

  - Arduino MKR board equipped with ECC508 or ECC608 chip

  This example code is in the public domain.
*/
#include <Arduino.h>
#include <ArduinoECCX08.h>
#include <utility/ECCX08CSR.h>
#include <utility/ECCX08DefaultTLSConfig.h>

String readLine() {
  String line;

  while (1) {
    if (Serial.available()) {
      char c = Serial.read();

      if (c == '\r') {
        // ignore
        continue;
      } else if (c == '\n') {
        break;
      }

      line += c;
    }
  }

  return line;
}

String promptAndReadLine(const char* prompt, const char* defaultValue) {
  Serial.print(prompt);
  Serial.print(" [");
  Serial.print(defaultValue);
  Serial.print("]: ");

  String s = readLine();

  if (s.length() == 0) {
    s = defaultValue;
  }

  Serial.println(s);

  return s;
}

void setup() {
  Serial.begin(9600);
  while (!Serial);

  if (!ECCX08.begin()) {
    Serial.println("No ECCX08 present!");
    while (1);
  }

  String serialNumber = ECCX08.serialNumber();

  Serial.print("ECCX08 Serial Number = ");
  Serial.println(serialNumber);
  Serial.println();

  if (!ECCX08.locked()) {
    String lock = promptAndReadLine("The ECCX08 on your board is not locked, would you like to PERMANENTLY configure and lock it now? (y/N)", "N");
    lock.toLowerCase();

    if (!lock.startsWith("y")) {
      Serial.println("Unfortunately you can't proceed without locking it :(");
      while (1);
    }

    if (!ECCX08.writeConfiguration(ECCX08_DEFAULT_TLS_CONFIG)) {
      Serial.println("Writing ECCX08 configuration failed!");
      while (1);
    }

    if (!ECCX08.lock()) {
      Serial.println("Locking ECCX08 configuration failed!");
      while (1);
    }

    Serial.println("ECCX08 locked successfully");
    Serial.println();
  }

  Serial.println("Hi there, in order to generate a new CSR for your board, we'll need the following information ...");
  Serial.println();

  String country            = promptAndReadLine("Country Name (2 letter code)", "");
  String stateOrProvince    = promptAndReadLine("State or Province Name (full name)", "");
  String locality           = promptAndReadLine("Locality Name (eg, city)", "");
  String organization       = promptAndReadLine("Organization Name (eg, company)", "");
  String organizationalUnit = promptAndReadLine("Organizational Unit Name (eg, section)", "");
  String common             = promptAndReadLine("Common Name (e.g. server FQDN or YOUR name)", serialNumber.c_str());
  String slot               = promptAndReadLine("What slot would you like to use? (0 - 4)", "0");
  String generateNewKey     = promptAndReadLine("Would you like to generate a new private key? (Y/n)", "Y");

  Serial.println();

  generateNewKey.toLowerCase();

  if (!ECCX08CSR.begin(slot.toInt(), generateNewKey.startsWith("y"))) {
    Serial.println("Error starting CSR generation!");
    while (1);
  }

  ECCX08CSR.setCountryName(country);
  ECCX08CSR.setStateProvinceName(stateOrProvince);
  ECCX08CSR.setLocalityName(locality);
  ECCX08CSR.setOrganizationName(organization);
  ECCX08CSR.setOrganizationalUnitName(organizationalUnit);
  ECCX08CSR.setCommonName(common);

  String csr = ECCX08CSR.end();

  if (!csr) {
    Serial.println("Error generating CSR!");
    while (1);
  }

  Serial.println("Here's your CSR, enjoy!");
  Serial.println();
  Serial.println(csr);
}

void loop() {
  // do nothing
}

Click Upload and monitor and, in order to generate a new CSR, define only the common name with the value ArduinoMKR1010

Copy the generated CSR text including "-----BEGIN CERTIFICATE REQUEST-----" and "-----END CERTIFICATE REQUEST-----" and save it into a .csr file.

Finally we have a CSR to identify the board and be able to communicate with AWs IoT core securely.

NOTE: This locking process is permanent and cannot be undone, but it is necessary to use the crypto element. The configuration set by the sketch allows the use of 5 private key slots with any Cloud provider (or server). A CSR can be regenerated at any time for each of the other four slots.

Connecting to AWS IoT Core

Once the Board is properly initialized and the CSR configured, we can proceed with the tutorial and connect the device to IoT core using MQTT protocol. AWS requires to use X.509 certificates for authentication. Let's follow the steps:

1- Create Arduino Board as AWS IoT Thing

Access to your AWS account in the predefined region. For the purpose of our example, it is Paris(eu-west-3)
To create the Board in AWS IoT as a Thing we will use the same name that we used for the common name in the CSR configuration: ArduinoMKR1010
By the way, we are going to skip the CSR certificate Upload for the Board

2- Create a project in Platform.io environment

Create e new project in Platform.io, for our example we have named it Hello World - MKR wifi 101
Install the required libraries:
- WiFiNINA
- ArduinoECCX08
- ArduinoBearSSL
- ArduinoMqttClient

As we have two projects in our VsCode workspace, I recommend to set the new created project as a default:

3- Create the Certificate and attach it to the Thing

Move the .csr file generated in previous steps to a more appropiate folder inside this project, for example into a /resources folder.
Create the certificate in AWS IoT by updating the csr generated.
Download the generated .pem.crt certificate file generated in AWS and save it in the /resources folder. We will need this file in the Board in order to communicate with cloud.
Ensure that the certificate is activated and Attached to the Thing created:

4- Create a Policy

For the scope of this example, we are going ot create an open policy, that will allow all iot actions on AWS.
For real life projects, I strongly recommend to create a strict policy and follow the least principle privilege.
Once the policy is created, it should b attached to the Certificate created in the previous step.

5- Implement the Connecting sketch to AWS IoT Core

In the project Hello World - MKR wifi 101, modify the file src/main.cppand add the following code:

/*
      AWS IoT WiFi

      This sketch securely connects to an AWS IoT using MQTT over WiFi.
      It uses a private key stored in the ATECC508A and a public
      certificate for SSL/TLS authetication.

      It publishes a message every second to arduino/outgoing
      topic and subscribes to messages on the arduino/incoming
      topic.

      The circuit:
      - Arduino MKR WiFi 1010 or MKR1000

    */
    #include <Arduino.h>
    #include <ArduinoBearSSL.h>
    #include <ArduinoECCX08.h>
    #include <ArduinoMqttClient.h>
    #include <WiFiNINA.h> // change to #include <WiFi101.h> for MKR1000

    #include "arduino_secrets.h"

    /////// Enter your sensitive data in arduino_secrets.h
    const char ssid[]        = SECRET_SSID;
    const char pass[]        = SECRET_PASS;
    const char broker[]      = SECRET_BROKER;
    const char* certificate  = SECRET_CERTIFICATE;

    WiFiClient    wifiClient;            // Used for the TCP socket connection
    BearSSLClient sslClient(wifiClient); // Used for SSL/TLS connection, integrates with ECC508
    MqttClient    mqttClient(sslClient);

    unsigned long lastMillis = 0;

    unsigned long getTime() {
      // get the current time from the WiFi module  
      return WiFi.getTime();
    }



    void connectWiFi() {
      Serial.print("Attempting to connect to the SSID: ");
      Serial.print(ssid);
      Serial.print(" ");

    bool connected = false;
      int retryCount = 0;
      const int maxRetries = 10; 


    while (!connected && retryCount< maxRetries) {
      if (WiFi.begin(ssid, pass) == WL_CONNECTED) {
        connected = true;
      } else {
            // failed, retry
        Serial.print(".");
        delay(1000);
        retryCount++;
      }
    }

      if (connected) {
        Serial.println();
        Serial.println("You're connected to the network");
        Serial.println();
      } else {
        Serial.println();
        Serial.println("Failed to connect to the network after multiple attempts.");
      }


    }

    void connectMQTT() {
      Serial.print("Attempting to MQTT broker: ");
      Serial.print(broker);
      Serial.println(" ");

      while (!mqttClient.connect(broker, 8883)) {
        // failed, retry
        Serial.print(".");
        delay(3000);
      }
      Serial.println();

      Serial.println("You're connected to the MQTT broker");
      Serial.println();

      // subscribe to a topic
      mqttClient.subscribe("arduino/incoming");
    }

    void publishMessage() {
      Serial.println("Publishing message");

      // send message, the Print interface can be used to set the message contents
      mqttClient.beginMessage("arduino/outgoing");
      mqttClient.print("{\"deviceModel\": \"MKR 1010\" ");
      mqttClient.print(",\"temperature\": "  );
      mqttClient.print(random(15, 30));
      mqttClient.print(",\"humidity\": ");
      mqttClient.print(random(1, 500));
      mqttClient.print(",\"vehicleId\": \"MKR1010-1");
      mqttClient.print("\"}");
      mqttClient.endMessage();
    }



    void onMessageReceived(int messageSize) {
      // we received a message, print out the topic and contents
      Serial.print("Received a message with topic '");
      Serial.print(mqttClient.messageTopic());
      Serial.print("', length ");
      Serial.print(messageSize);
      Serial.println(" bytes:");

      // use the Stream interface to print the contents
      while (mqttClient.available()) {
        Serial.print((char)mqttClient.read());
      }
      Serial.println();

      Serial.println();
    }

    void setup() {
      Serial.begin(115200);
      while (!Serial);

      if (!ECCX08.begin()) {
        Serial.println("No ECCX08 present!");
        while (1);
      }

      // Set a callback to get the current time
      // used to validate the servers certificate
      ArduinoBearSSL.onGetTime(getTime);

      // Set the ECCX08 slot to use for the private key
      // and the accompanying public certificate for it
      sslClient.setEccSlot(0, certificate);

      // Optional, set the client id used for MQTT,
      // each device that is connected to the broker
      // must have a unique client id. The MQTTClient will generate
      // a client id for you based on the millis() value if not set
      //
      // mqttClient.setId("clientId");

      // Set the message callback, this function is
      // called when the MQTTClient receives a message
      mqttClient.onMessage(onMessageReceived);
    }

    void loop() {



      if (WiFi.status() != WL_CONNECTED) {
        int numNetworks = WiFi.scanNetworks();
        Serial.println("Discovered " + String(numNetworks) + " Networks");
        connectWiFi();
      }

      if (!mqttClient.connected()) {
        // MQTT client is disconnected, connect
        connectMQTT();
      }

      // poll for new MQTT messages and send keep alives
      mqttClient.poll();

      // publish a message roughly every 1 seconds.
      if (millis() - lastMillis > 1000) {
        lastMillis = millis();

        publishMessage();
      }
    }

Create in /include folder a file for the secrets named arduino_secrets.h that has already been included in the main sketch file.
arduino_secrets.h will contain the secrets and variables required to connect, first to the Wifi and the to the Cloud provider, for example:

#define SECRET_SSID "MIWIFI_SSID"
#define SECRET_PASS "XXXXXXXXXXXX"

#define SECRET_BROKER "a21nf7ui6d7k9-ats.iot.eu-central-1.amazonaws.com"

const char SECRET_CERTIFICATE[] = R"(
-----BEGIN CERTIFICATE-----
MIIClTCCAX2gAwIBAgIVAMtpt+I79Uj24rlM4MVB4Q2mqffdMA0GCSqGSIb3DQEB
CwUAME0xSzBJBgNVBAsMQkFtYXpvbiBXZWIgU2VydmljZXMgTz1BbWF6b24uY29t
IEluYy4gTD1TZWF0dGxlIFNUPVdhc2hpbmd0b24gQz1VUzAeFw0yNDA5MDIxNDA3
MDJaFw00OTEyMzEyMzU5NTlaMCQxCjAIBgNVBAYTASAxFjAUBgNVBAMTDU15TUtS
V2lGaTEwMTAwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARo/XaUyfCffmCSIyl1
x1Tc4pbi4d1gu+IzOa98Fl52HaxxroFkDrKCiC6cyaK653BFXkRpWcQ7ivmNouO7
Fz06o2AwXjAfBgNVHSMEGDAWgBTwrTWex0zOILY7T5K6neYroFcHYDAdBgNVHQ4E
FgQUiqwBn2n3ixwwL4E8YI5d4b4eWdMwDAYDVR0TAQH/BAIwADAOBgNVHQ8BAf8E
BAMCB4AwDQYJKoZIhvcNAQELBQADggEBAK8iH5HpmGaxw9XQh/zyPsKxGG/dXXWr
U/mbZH6HGnf/Ux3ULnyZElBOB99KgqCutu8+k8lgpR8gLtsfC3XpL+TwIG4UHVak
fqP//CFl+n0DRWZvqw2wyvKBVyP1d9WUcTAW7x1k5ZN3AeXDT1Iy/KIkxhQXHIQi
wA1nw+YPDafHNFAviLKrUYd+Sx9hpKLDLI17609HbeY/9dsZh8MvxUmgjod7jvIy
0EGOgf8EhqyDvTVMb18ZUT0LKCwDWpMug0fZ+XK0kHA98Th0ndkE3997MqJSuY2u
tPND16KqhQPWim/akbJb1yDmP0M5xzF9Vtoh9Tv8JLxynmgc0KDq5/g=
-----END CERTIFICATE-----
)";

Replace the secrets for the correct ones, specially the SECRET_CERTIFICATE, that should be the one that is contained in the certificate file that was downloaded from AWS in a previous step in resources folder.
To replace the SECRET_BROKER, the endpoint should be obtained from AWS:

6- Connecting to AWS IoT

Once the project code is properly added to Hello World - MKR wifi 101 , Ensure that the Board is connected through USB and that the device is available in PIO HOME Devices:

There are several Project Tasks availables. We need to Build the sketch code and upload it to the Board connected.
I recommend to use the following tasks:
- Full Clean
- Build
- Upload and Monitor
Finally, if all steps have worked successfully, a Serial monitor terminal is openned and the Board is going to Connect to the Wifi and send messages to AWS IoT

Testing and Monitoring Data

The board is continously sending a MQTT message to IoT core with random generated data every second with this composition, see example:

{
  "deviceModel": "MKR 1010",
  "temperature": 22,
  "humidity": 317,
  "vehicleId": "MKR1010-1"
}

1- ** Monitor Outgoing Data from the Board**

We have to use the MQTT test client to Subscribe to the topic arduino/outgoing

2- Send a Message to the Board

We should subscribe to the topic arduino/incoming and publis a message from the MQTT Test client in AWS IoT
The message sent should appear in the Serial console that we have openned in Platform.io

Conclusion

In this guide, we’ve walked through the process of connecting the Arduino MKR WiFi 1010 to AWS IoT Core using Platform.io framework in Visual Studio Code. You’ve learned how to set up your development environment, configure AWS IoT Core, and establish communication between your Arduino board and the cloud using the MQTT protocol.

Although this is a specific project, it can be considered the first step toward building scalable IoT solutions. In my opinion, Arduino is one of the best platforms for growing in the IoT space and for enjoying the process of building personal projects. It’s also a viable option for production systems due to its flexibility and ecosystem. Additionally, using Platform.io enhances local development capabilities and simplifies the integration of new tasks, making it an excellent tool for both hobbyists and professionals.

As a next step, I plan to extend this project by creating an AWS IoT Rule that processes the data coming from the Arduino MKR1010 and integrates it with a real-time platform. This will involve uploading the data from the incoming MQTT topic directly to DynamoDB.

I’ll be covering this in my upcoming articles, where I’ll explain the integration in more detail. You can also check out my previous article on building a real-time platform using AWS services here, which will serve as a foundation for the next phase of this project.

Reference

1- Github Project:

https://github.com/acriado-dev/arduino-mkr-1010-aws-iot-core

2- Arduino MKR 1010 Official Docs:

https://store.arduino.cc/en-es/products/arduino-mkr-wifi-1010?gad_source=1&gclid=CjwKCAjwreW2BhBhEiwAavLwfFKUmllSUvjuY6mNsFugNXUCh-aoR3kiqFvGyX0dToSSzvty08dyBhoCE28QAvD_BwE

3- Platform.io

https://docs.platformio.org/en/latest/what-is-platformio.html

4- AWS IoT core Developer guide

https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html

Building a "Real-Time" Data Integration Platform on AWS

Amador Criado — Fri, 13 Sep 2024 07:54:34 +0000

Introduction

During my work on several projects as a Cloud Architect, I noticed a recurring challenge:

Integrating diverse data sources and delivering real-time updates to multiple clients

Whether it's monitoring environmental conditions in smart agriculture, tracking assets in logistics, or managing connected devices in smart cities, the ability to process and act on data instantly can make a significant difference.

In this post, I'll share my journey of building a serverless Near Real-Time Data Integration Platform using the AWS Serverless Application Model (SAM).

All the infrastructure code is available in a GitHub repository:

https://github.com/acriado-dev/sam-real-time-websockets

Overview

Understanding Real-Time vs Near Real Time

Before diving into the details and architectural components, it's important to clarify a core concept for the platform that sometimes causes confusion. In the context of data processing, what is the difference between "real-time" and "near real-time"?

Real-Time: Data is processed and delivered as soon as it is generated, with virtually no delay. This is critical in scenarios where immediate action is required. Examples include industrial automation, connected vehicles, and healthcare monitoring.
Near Real-Time: This refers to a slight delay in data processing, typically ranging from milliseconds to seconds. Examples include data analytics, CDNs, and social media monitoring.

About this Project

To answer the question,

is this platform Real-time or Near Real-Time?

We have to take into account that TRUE Real-Time implies that data is processed and delivered with minimal latency, often microsecons or milliseconds, with strict timing constraints. This level of immediacy is not accomplished with the Cloud resources used for thi project implementation, let's analyze the most significant ones:

AWS Lambda: Introduces some latency due to cold starts and the execution time of the function. While this latency is generally small (typically within milliseconds to a couple of seconds), it does mean that it cannot be considered "true" Real-Time but rather operates in near Real-Time.
WebSocket API: This component allow low-latency communication, which is closer to Real-Time, but the overall latency is influenced by the procesisng times of the connected Services.
DynamoDB and Streams: Provides very fast data retrieval and storage capabilities, but Lambda triggered by streams still introduces additional processing time.

Given these factors, the most accurate description for the platform should be 'near Real-Time'. However, there are some benefits of this approach, from cost efficiency to scalability and simplicity, but remember that depending on the use case you're facing of for your application, Near Real-time could be the best decision.

Architecture

I will explore the project's architecture, detailing various components and explaining how the serverless approach offered an appropiate and efficient solution. In addition, a workflow example will be described in order to understand better How the project works:

Components:

Client Integration: Frontend application (Mobile & WebApp) that is able to connect to the WebSocket API and receive the real time data updates.
WebSocket API Gateway: Entry point of the application. It's responsible for handling the WebSocket connections and the messages that are sent to the clients.
OnConnect Lambda Function: Is triggered when a client connects to the WebSocket API Gateway. Stores the connection information in the DynamoDB table.
OnDisconnect Lambda Function: Is triggered when a client disconnects from the WebSocket API Gateway. Removes the connection information from the DynamoDB table.
OnReceiveRealTimeItem Lambda Function: Is triggered when a new item is added to the DynamoDB table. Sends the real time data updates to the clients that are interested in the item.
RealTimeData DynamoDB Table: Used to store the real time data items. Each item has a unique key and a value that represents the data that is sent to the clients.
WebSocketConnectionManager DynamoDB Table: Stores the connection information of the clients that are connected to the WebSocket API Gateway. Each item has a unique connectionId and a realTimeItemKey that represents the item that the client is interested in.
Integration sources: Data sources responsible for sending the real time data updates to the platform. For this project, the following sources has been considered:
- SDK: AWS SDK and CLI for any language compatible.
- Data transfer: Any data transfer mechanism that is able to send the data to the platform. For example, data replication from other database.
- Device Location & Sensors: IoT devices mainly sending sensor data and telemetry. For example, GPS location of a vehicle.
- REST API: For example, a weather API that sends the weather data to the platform.
- Async (SNS, SQS): Asynchronous mechanism that is able to send the data to the platform. For example, an SNS topic that sends the data to the platform.
- S3 Events: Event triggered by an S3 bucket. For example, when a new file is uploaded to the bucket.

Workflow:

Next, we are going to describe the different numbered steps marked in the diagram:

The client integration connects to the WebSocket API Gateway.
The WebSocket API Gateway triggers the OnConnect Lambda function.
The OnConnect and OnDisconnect Lambda functions store and remove the connection information from the WebSocketConnectionManager DynamoDB table.
RealTimeData items are added to the RealTimeData DynamoDB table from the integration sources.
The OnReceiveRealTimeItem lambda function is triggered through DynamoDB Streams.
The OnReceiveRealTimeItem lambda function sends the real time data updates to the clients that are interested in the item.
The client integration receives the real time data updates from the WebSocket API Gateway.

Setup and Test the Platform

In order to test the Platform accordingly we are going to use wscat utility to simulate an API client and monitor the websocket API. This way we can allow each tes client to receive the data instantly.

npm install wscat

We can retrieve the WebSocket API endpoint in AWS Console although is useful to have it as an output for the SAM project:

The Endpoint should have the following composition:

wscat -c wss://[api-id].execute-api.[aws-region-id].amazonaws.com/[environment]?realTimeItemKey=[item-key]

Notice that the realTimeItemKey query parameter is required to connect to the WebSocket API. This parameter is used to filter the messages that are sent to the client. The value of this parameter must be the same as the realTimeItemKey attribute of the item that you want to receive updates for.

According to the project specification, the realTimeItemKey is a dynamic value and can be adapted depending on the client/integration needs.

In our example, we are going to use vehicleId as real time item to receive updates from multiple clients, then, the realTimeItemKey should be vehicleId, and the command to connect should be:

wscat -c  wss://svcz00plil.execute-api.eu-central-1.amazonaws.com/develop?vehicleId=3

Let's see some Examples of the three main events of the Platform:

OnConnect:

%[https://youtu.be/lUFf14zZsxg]

OnDisconnect

%[https://youtu.be/x8ZYvpD_Q1k]

OnReceiveRealTimeItem

%[https://youtu.be/8YyZ7hJ0Ft4]

Conclusion

This project is a fantastic example of a simple near real-time integration in a Serverless architecture. It can be used as a starting point for implementing a more complex integration or added to another project to handle real-time processing.

In my opinion, it's important to optimize these kinds of data integrations and make them as efficient as possible. Many projects overuse polling, which can be considered an anti-pattern for proper real-time implementation.

Additionally, note that a special part of the project focuses on IaC, as it's crucial to implement workloads that can be reused and reproduced in any Cloud environment (in this case, AWS).

[Lab] AWS Lambda LLRT vs Node.js

Amador Criado — Wed, 15 May 2024 19:18:59 +0000

Introduction

AWS Lambda is a serverless compute service that runs code in response to events and automatically manages the underlying resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own backend services that operate at AWS scale, security and the main feature considered for this lab, performance.

One of the biggest challenges that is common among AWS Lambda and serverless computing is the Performance. Unlike traditional server-based environments where infrastructure resources can be provisioned and fine-tuned to meet specific performance requirements, serverless platforms like Lambda abstract away much of the underlying infrastructure management, leaving less control over performance optimization in the hands of developers. Additionally, as serverless architectures rely heavily on event-driven processing and pay-per-execution pricing models, optimizing performance becomes crucial for minimizing costs, ensuring responsive user experiences, and meeting stringent latency requirements.

What is Lambda LLRT?

Warning

LLRT is an experimental package. It is subject to change and intended only for evaluation purposes.

AWS has open-sourced its JavaScript runtime, called LLRT (Low Latency Runtime), an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.

LLRT is designed to address the growing demand for fast and efficient Serverless applications. LLRT offers up to over 10x faster startup and up to 2x overall lower cost compared to other JavaScript runtimes running on AWS Lambda. It's built in Rust, utilizing QuickJS as JavaScript engine, ensuring efficient memory usage and swift startup.

Real-Time processing or data transformation are the main use cases for adopting LLRT for future projects. LLRT Enables a focus on critical tasks like event-driven processing or streaming data, while seamless integration with other AWS services ensures rapid response times.

Key features of LLRT

Faster Starup Times: Over 10x faster startup compared to other JavaScript runtimes running on AWS Lambda.
Cost optimization: Up to 2x overall lower cost compared to other runtimes. By optimizing memory usage and reducing startup time, Lambda LLRT helps minimize the cost of running serverless workloads.
Built on Rust: Improves performance, reduced cold start times, memory efficiency, enhanced concurrency and safety.
QuickJS Engine: Lightweight and efficient JavaScript engine written in C. Ideal for fast execution, efficient memory usage, and seamless integration for embedding JavaScript in AWS Lambda.
No JIT compiler: unlike NodeJs, Bun & Deno, LLRT not incorporates JIT compiler. That contributes to reduce system complexity and runtime size. Without the JIT overhead, CPU and memory resources can be more efficiently allocated.

Overview

The goal of this lab is to conduct a comparative analysis between AWS Lambda's Low Latency Runtime (LLRT) and traditional Node.js runtime, focusing on their performance and efficiency in real-world serverless applications. By deploying identical functions with LLRT and Node.js, with the aim to evaluate their respective startup times, execution speeds, and resource consumption.

The previous results of performance profiling by AWS Labs, available on GitHub, offer valuable benchmarks and insights:

LLRT - DynamoDB Put, ARM, 128MB:

Node.js 20 - DynamoDB Put, ARM, 128MB:

Lab

The lab is composed by 2 Lambda functions "llrt-put-item" and "nodejs-put-item", both designed to put any received event into a DynamoDB table named "llrt-sam-table".

Implemented using AWS SAM, the project leverages AWS Lambda Power Tuning for performance profiling and benchmarking. While resembling the default benchmark by AWS Labs, all code developed is accessible on my personal GitHub repository, facilitating easy replication and further exploration of the lab's results:

https://github.com/acriado-otc/example-aws-lambda-llrt

Benchmarking: LLRT vs Node.js

To benchmark the lab, AWS Lambda Power Tuning has been chosen as a open-source tool developed by AWS Labs. AWS Lambda Power Tuning is a state machine powered by AWS Step Functions that will helps us to get results for both lab's Lambda functions and get results for cost and/or performance in a data-driven way.

While you can manually run tests on functions by selecting different memory allocations and measuring the time taken to complete, the AWS Lambda Power Tuning tool allows us to automate the process.

The state machine is designed to be easy to deploy and fast to execute. Also, it's language agnostic.

Deployment

To deploy AWS Lambda Power Tuning, clone the official repository and choose the preferred deployment method. In my case I used SAM cli for simplicity, but I highly recommend to use AWS CDK for IaC deployments:

https://github.com/alexcasalboni/aws-lambda-power-tuning

Executing the State Machine

Independently of how you've deployed the state machine, you can execute it in a few different ways. Programmatically, using the AWS CLI, or AWS SDK. In the case of our lab, we are going to execute it manually, using the AWS Step Functions web console.

Once properly deployed, You will find the new state machine in the Step Functions Console in the AWS account and region defined for the lab:

Find it and click "Start execution".
A descriptive name and input should be provided.
We are going to execute llrt-put-item first, that's the input used**:**

{
  "lambdaARN": "arn:aws:lambda:eu-west-1:XXXXXXXXXXX:function:llrt-sam-LlrtPutItemFunction-42aAa2of5wqn",
  "powerValues": [
    128,
    256,
    512,
    1024
  ],
  "num": 500,
  "payload": {"message": "hello llrt lambda"},
  "parallelInvocation": true,
  "strategy": "speed"
}

All other fields should remain by default. Finally, find it and click "Start execution".
After some seconds/minutes the Execution should have the status "Succeded"
Repeat the same process for the nodejs-put-item lambda function. The unique field to change from the input should be the function's ARN and optionally the payload.
While running each execution, we can see the status/progress of the step function execution with the graph view:

Results and Analysis

In order to get accurate results several executions have been made for each Lambda function, with 'speed' strategy. Next we are presenting the results for each Lambda separately, and a final comparison:

LLRT Lambda function:

{
  "power": 1024,
  "cost": 9.519999999999999e-8,
  "duration": 6.25,
  "stateMachine": {
    "executionCost": 0.00025,
    "lambdaCost": 0.0001901263,
    "visualization": "https://lambda-power-tuning.show/#gAAAAQACAAQ=;REREQWbm5kIREdFAAADIQA==;ata9Muy90zTBcEwzwXDMMw=="
  }
}

Nodejs

{
  "power": 1024,
  "cost": 1.512e-7,
  "duration": 8.666666666666666,
  "stateMachine": {
    "executionCost": 0.00025,
    "lambdaCost": 0.00025384590000000003,
    "visualization": "https://lambda-power-tuning.show/#gAAAAQACAAQ=;d7fdQ7y7iEKamatBq6oKQQ==;Ckp6Nc+VmzRwbUY0ilkiNA=="
  }
}

Comparison

After several executions of the AWS Power Tuning step function. It's clear that both lambda functions has a similar performance for 1024MB and 512MB of memory allocation. The huge difference appears in the interval from 128MB to 256MB, where LLRT is more suitable option and should be considered for small functions.

In terms of cost, for LLRT 128MB is the cheaper average execution, and 256MB the wrost, meanwhile for NodeJs function the wrost cost 128MB, just the opposite of LLRT. However, it's important to remark that for this laboratoy, the criteria applied for the benchmark has been 'speed'. To get more accurate cost results should be executed with 'cost' strategy accordingly.

Conclusion

LLRT clearly can add advantages for some projects, such as faster startup times and potential cost savings, but it currently lacks the stability, support, and real-world testing required to be considered for production use.

For developers working with smaller serverless functions that prioritize rapid response times and efficient resource utilization, LLRT is an alternative to traditional Lambda runtimes. However, it's essential to evaluate carefully LLRT and consider the specific requirements of your application before.

As LLRT continues to evolve and mature, it's adoption will increase for sure, and progressively a refinement of this runtime for latency-sensitive use cases will be provided by AWS. In the meantime, let's monitor LLRT's development and wait for news, specially for production environments.

Forem: Amador Criado

RAG Chatbot with Amazon Bedrock & LangChain

Introduction

Design considerations

Architecture

Key Components

Workflow

Conclusion

Next Steps

Ingesting F1 Telemetry UDP real-time data in AWS EKS

Introduction

Design Considerations

Why UDP ?

Architecture

Key Components

Workflow

Demo / Showcase

Next Steps

Connecting an Arduino MKR WiFi 1010 to AWS IoT Core

Introduction

What is Arduino MKR Wifi 1010?

Development Environment

Prerequisites

Configuring the Board

Connecting to AWS IoT Core

Testing and Monitoring Data

Conclusion

Reference

Building a "Real-Time" Data Integration Platform on AWS

Introduction

Overview

Understanding Real-Time vs Near Real Time

About this Project

Architecture

Components:

Workflow:

Setup and Test the Platform

Conclusion

[Lab] AWS Lambda LLRT vs Node.js

Introduction

What is Lambda LLRT?

Key features of LLRT

Overview

Lab

Benchmarking: LLRT vs Node.js

Deployment

Executing the State Machine

Results and Analysis

Conclusion

References and resources