Forem: Sulaiman Olubiyi

Building a RAG-Based AWS VPC Flow Log Analyzer

Sulaiman Olubiyi — Sat, 28 Feb 2026 15:06:19 +0000

Understanding network traffic inside a Virtual Private Cloud (VPC) directly impacts your security posture, performance visibility, and compliance readiness. Yet most teams still sift through raw flow logs manually, reacting to incidents instead of proactively investigating them.

Rather than grepping through thousands of log lines or exporting data to spreadsheets, we can turn VPC Flow Logs into an interactive layer.

What if you could simply ask your logs questions like this?

Was that SSH connection rejected?
Which IP keeps hitting port 443?
Is this traffic normal or a problem?

In this article, we’ll build a Retrieval-Augmented Generation (RAG) powered VPC Flow Log Analyzer that turns static network telemetry into an interactive security assistant

The Challenge of Manual Log Analysis

AWS VPC Flow Logs capture essential information about network traffic. Yet, analysing these raw logs to detect threats like SQL injection attempts or unauthorised access presents significant challenges:

Information Overload: The sheer volume of logs is overwhelming. Finding specific patterns or anomalies is like searching for a needle in a haystack.
Context Fragmentation: Raw logs lack context. Identifying related packets across different components and time frames is labour-intensive and error-prone.

The RAG-based VPC Flow Log Analyser uses:

Streamlit (interactive UI)
LangChain (RAG orchestration)
Chroma (vector database)
OpenAI GPT-4o (reasoning engine)

At the end, you'll have a conversational security assistant capable of answering questions like:

“Which IPs were rejected?”
“Was there unusual traffic to port 22?”
“Which destinations received the most packets?”

Functional Components

Data Ingestion & Transformation ("Translator")
Raw VPC Flow Logs are just strings of numbers and IPs (e.g., 2 123... 443 6 ACCEPT).
The Component: A custom Python parser.
It "hydrates" the logs, turning them into human-readable sentences like "Source 10.0.1.5 sent 1000 bytes to Port 443 and was ACCEPTED." This makes it much easier for the AI to "understand" the relationship between data points.
Embedding Model ("Encoder")
We can't search text mathematically, so we have to turn it into numbers (vectors).
Component: OpenAI text-embedding-3-small.
It creates a numerical "fingerprint" for every log line. Similar events (like multiple SSH brute-force attempts) will have similar numerical fingerprints, allowing for "fuzzy" or semantic searching.
Vector Database ("Memory")
Standard databases search for exact words; a vector DB searches for meaning.
Component: ChromaDB.
It stores thousands of these "fingerprints" locally. When you ask a question, it instantly finds the top 10 or 15 log entries that are most relevant to your specific query.
RAG Orchestration & LLM ("Brain")
This is where the actual "chatting" happens.
Component: LangChain + GPT-4o.

LangChain takes the question, grabs the relevant logs from ChromaDB, and hands them both to GPT-4o with a set of instructions: "You are a security engineer; tell me what happened here."

Streamlit Frontend ("Cockpit") Component: Streamlit Web Framework. It provides the UI for uploading .txt files, managing your API keys via .env, and providing the chat interface so you don't have to touch a terminal to investigate your network.

Steps involved in the implementation:

Check out the codebase on GitHub

Step 1: Creating Virtual Environment and Installing Dependencies

git clone https://github.com/Damdev-95/rag_aws_flow_logs

python -m venv venv

source venv/bin/activate

cd rag_aws_flow_logs

pip install -r requirements.txt

Step 2: Configuration handling

The environment variables include handling sensitive data, such as openai keys.

ENV_API_KEY = os.getenv("OPENAI_API_KEY")

Step 3: Running the streamlit App

streamlit run app.py

Once you click on 'Browse files', you will be able to upload log files on the application; ensure the log file format is in txt.

Select "Build Knowledge Base" to store the raw log data in the vector database after it has been converted into vectors.

Successfully created index events after the embedding process
Yes, we are live, I asked the below
What is the summary of the flow logs based on traffic accept and reject

Additional examples of queries with interaction

Stay tuned for additional RAG and GenerativeAI projects in cloud networking by reading my articles.

I look forward to your comments.

The "Zero Latency" AI Battle: RAG vs CAG

Sulaiman Olubiyi — Tue, 24 Feb 2026 14:14:19 +0000

We’ve all been there. You’re building a cool internal tool, maybe a bot that helps your team interact with your company internal documents. You ask it a question, and then... you wait.

The "searching..." spinner dances for 3, 4, maybe 5 seconds. By the time the AI answers, you could have just searched the docs by yourself. This is the RAG Tax, and if you're aiming for a seamless dev experience, it’s a high price to pay.

But there’s a new architecture called CAG (Cache-Augmented Generation) that promises to kill that latency.
Let’s break down why the AI is lagging and how "Context Caching" changes the game.

Understanding the RAG Pipeline

To understand why it's slow, we have to look at the three actors in a standard RAG (Retrieval-Augmented Generation) setup. Think of it like a courtroom trial:

Retrieval (The Researcher)

TWhen you ask a question, the Researcher doesn't know the answer. They have to run to the archives and find the relevant folders.

Primary Actor: The Vector Database. It performs a mathematical similarity search to find text chunks that "look" like your question.
Latency Source: This is where the clock starts. Embedding the query and searching a database takes time.

Augmentation (The Legal Assistant)

This actor takes those raw folders from the archives and pins them to a clipboard for the Judge to see. They "stuff" the context into the prompt.

Primary Actor: The Orchestrator (LangChain, LlamaIndex, or your own Python script). It formats the data so the AI can read it.
Latency Source: Minimal, but adding thousands of words to a prompt increases the "processing" time for the next step.

Generation (The Judge)

The Judge looks at the evidence and provides a final verdict (the answer).

Primary Actor: The LLM (Gemini, GPT, etc.).
Latency Source: The "Time to First Token". The Judge has to read everything on the clipboard before speaking.

Enter CAG: The "Photographic Memory" Upgrade

If RAG is a Researcher with a library card, CAG (Cache-Augmented Generation) is an expert with a photographic memory.

Instead of searching for data after you ask a question, CAG pre-loads your entire documentation or codebase into the LLM’s "active memory"—specifically the KV (Key-Value) Cache.

Primary Actor in CAG: The Model Context Window. Because the information is already "warmed up" inside the model, there is no Researcher and no archival search. The "Judge" already has the entire case memorised.

Why is it "Zero Latency"?

In CAG, the bottleneck (the Vector DB search) is deleted. You aren't doing "Just-in-Time" retrieval; you're doing "Ahead-of-Time" caching.

When you hit enter, the LLM doesn't have to wait for a database to return results. It starts streaming the answer instantly. For those of us in SRE, engineering or network automation, this is the difference between fixing a downed router in 10 seconds vs. 60 seconds.

Which one should you build?

Stick with RAG if you have millions of documents that change every hour (like a live news feed). It’s the "Big Data" solution.

Move to CAG if you have a specific, high-value knowledge set (like your company's API docs or a specific project's source code). It’s the "High Performance" solution.

We’re moving toward a world where "context is king", and caching that context is the fastest way to make your AI feel like a natural extension of your brain, rather than a slow search engine.

I will be writing more about RAG development in my further articles; stay tuned

Reference

Google Gemini
Cisco AI Technical Series

Master AWS Transit Gateway Management with Terraform: A Step-by-Step Guide

Sulaiman Olubiyi — Sun, 12 Jan 2025 17:55:49 +0000

The Solace Fashion App is ready to redefine how customers shop and engage with style. And at the heart of this exciting journey lies a critical mission: designing and deploying a robust Virtual Private Cloud (VPC) architecture to power it all.

As the cloud network engineer, you are entrusted with the responsibility of turning this vision into a seamless, scalable, and secure reality. From planning subnets and routing tables to implementing security policies that safeguard sensitive user data, your role is to lay the foundation that ensures the app operates flawlessly under any circumstance. This is more than just building infrastructure; it’s about delivering the reliability, performance, and agility needed to match the bold ambitions of the Solace Fashion App.

Let’s dive into how we’ll bring this cloud architecture to life and ensure saclability.
The architecture consists of the following virtual private clouds (VPCs):

Frontend App VPC: 10.10.100.0/16
Backend VPC: 172.30.100.0/16
Database VPC: 192.168.100.0/16

The objectives of the architecture are as follows:

Restrict Communication: Only allow the App VPC (10.10.100.0/24) to connect to the Backend VPC (172.30.100.0/24).
Controlled Access: Permit the Backend VPC (172.30.100.0/24) to communicate with the Database VPC (192.168.100.0/24).
Enforce Isolation: Ensure that direct communication between the App VPC and the Database VPC is strictly prohibited.

We are utilizing Terraform to efficiently manage the development and deployment of the architecture.

Develop Terraform code in a modular structure to enhance maintainability and efficiency by isolating components like VPCs, subnets, and transit gateway into reusable modules.

Using a transit gateway to efficiently manage and centralize the connections between VPCs, ensuring scalability, simplified routing, and streamlined network management.

Using the following commands to install terraform

wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install terraform

Applying aws credentials on the terminal, either using aws configure or environment variables on your working terminal
The Transit Gateway comprises three key components:
1. Attachment: Connects VPCs, on-premises networks, or other resources to the Transit Gateway.
2. Route: Defines the traffic flow between attachments through routing tables.
3. Propagation: Automatically shares routes from attached resources to the Transit Gateway routing tables, enabling dynamic updates.
Creating each vpc using terraform

Creating the transit gateway and each vpc attachment and route-table

for testing purpose, I will be using the public subnet only

Creating EC2 instances for reachability tests to verify network connectivity between VPCs.

Initializing the terraform code using terrfarom init

Formatting the terraform code using terraform fmt

Planning the terraform code using terraform plan

Applying the terraform code after the plan was successful using terraform apply --auto-approve

Testing connectivity from Frontend VPC

PING TO BACKEND VPC REACHABLE

PING TO DATABASE VPC NOT REACHABLE

Testing connectivity from Database VPC

PING TO BACKEND VPC REACHABLE

PING TO FRONTEND APP VPC NOT REACHABLE

Whaooooo!!!
Been a long ride, I hope you follow through and practice at your pace
Check the Project_CodeBase

Demistfying AWS VPC Lattice

Sulaiman Olubiyi — Fri, 10 Jan 2025 07:27:09 +0000

Did you know that AWS VPC Lattice could be the missing piece in achieving seamless cloud deployments? It's like the perfect convergence of service networking—neutral, efficient, and capable of bridging diverse environments with ease.

AWS VPC Lattice is a fully managed application networking service that simplifies connecting, securing, and monitoring communications between services. It's specifically designed to streamline service-to-service communication in distributed applications.

AWS VPC Lattice Reference
Source: https://aws.amazon.com/blogs/aws/introducing-vpc-lattice-simplify-networking-for-service-to-service-communication-preview/

Why VPC Lattice Stands Out

Let’s start by understanding what makes AWS VPC Lattice the missing piece in your cloud architecture:

Service Network: Centralizes service-to-service communication for seamless interaction.
Service Directory: Keeps everything organized with a centralized registry for services.
Authentication and Authorization: Secures communication using AWS IAM for access control.
Traffic Management: Provides smart routing and resilience to optimize service performance.

Roles and Layers

Networking layer: provides connectivity between applications through the deployments. This is managed by the admin team.
Application layer: applications deployed across multiple VPCs and accounts. This is managed by the Dev team
Security layer: this is applied across all depths of both networking and deployments; the responsibility is shared among the admin and dev teams.

Developers love speed—spinning up instances and hardcoding credentials to get things moving fast, often ignoring risks like IP conflicts or security gaps. Admins, on the other hand, focus on governance and security, slowing things down with strict controls. The real challenge? Striking a balance between innovation and control, so teams can build fast without compromising on safety.

Components

Service: Think of a service as a standalone unit of software that performs a specific task. It can live in any VPC or account and run on virtual machines, containers, or serverless functions. A service configuration includes:
- Target Group: The backend where your application runs—this could be EC2 instances, IP addresses, Lambda functions, or Kubernetes Pods.
- Listener: Defines the port and protocol your service uses to receive traffic. Supported protocols include HTTP/1.1, HTTP/2, gRPC, and HTTPS.
- Rule: Determines how traffic is routed, forwarding requests to target groups based on conditions and priorities.
Service Network: Picture this as a logical boundary that ties your services together. It simplifies service discovery, enforces common access policies, and ensures connectivity between services.
Service Directory: A one-stop registry for all your services within VPC Lattice. Whether they’re yours or shared with you via AWS Resource Access Manager (RAM), you can find them here.
Auth Policies: These IAM resource policies let you enforce authentication and context-specific authorization. Apply them at the service or network level to enhance security and control.

Practical Hands-on
I will be creating a web application that has two backend services:

EC2 instance (python application), Below include the app.py on the ec2 instance

from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
  return 'Howdy, response from the EC2 instance'

app.run(host='0.0.0.0', port=8080)

Lamdda function

exports.handler = async (event) => {
    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello Lambda!'),
    };
    return response;
};

Create a target group for each service on VPC Lattice

Create Service for the AWS VPC lattice

Create service network and associate with service and VPC

Testing the service from another VPC (Test-VPC) The test-VPC has been associated with the service network; this will ensure the connectivity test across the VPCs

CloudWatch Logs for observability and logging

[Everything about AWS VPC Lattice](https://repost.aws/articles/ARRz07hcqrQ2qcO5s5aYMiAw/get-started-with-amazon-vpc-lattice-resources-content

A Step-by-Step Guide to Easily Deploying EKS Infrastructure and Applications Using Terraform

Sulaiman Olubiyi — Sun, 04 Feb 2024 07:26:22 +0000

Terraform is like the wizard of deployment tools. It's an open-source Infrastructure as Code (IaC) tool that lets you define and provision infrastructure using a declarative configuration language. Instead of manually setting up infrastructure such as servers, databases, and other resources, you can easily describe your desired infrastructure in code using HashiCorp Configuration Language (HCL).

This article focus on project using terraform for EKS , deploying applications using respective manifest files, and an application load balancer ingress controller using Helm.

The Github repository for this project EKS Terraform with application deployment

Below contains the detailed steps in this project, ensure you have an active AWS account before getting started;

List the contents of the terraform files

Initialize terraform on the directory to download required providers

terraform init

Validate the terraform file using :

terraform plan

Apply the terraform configuraton file

terraform apply --auto-approve

Copy the output of the terraform configuration to the ~/.kube/config
Installing aws-iam-authenticator
This enables using AWS IAM credentials to authenticate to a Kubernetes cluster

curl -Lo aws-iam-authenticator https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v0.5.9/aws-iam-authenticator_0.5.9_linux_amd64
chmod +x ./aws-iam-authenticator
mkdir -p $HOME/bin && cp ./aws-iam-authenticator $HOME/bin/aws-iam-authenticator && export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc

Communicate with Kuberbetes cluster using kubectl get all

Get the pods in the cluster using 'kubectl get pods --all-namespaces'

Deploy the manifest for the pods deployment

kubectl apply -f manifests/deployment.yaml

Validate the deployment using kubectl get pods

Testing the deployment using the port forward

kubectl port-forward hello-kubernetes-6bf86759db-7jf7j 8080:8080

ALB Ingress Controller can be installed with Helm
Install Helm package

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh

Install the collection of YAML files necessary to run the ALB Ingress Controller. Add the following repository

helm repo add incubator https://charts.helm.sh/incubator

Install the ALB Ingress Controller in my cluster

helm install ingress incubator/aws-alb-ingress-controller \
  --set autoDiscoverAwsRegion=true \
  --set autoDiscoverAwsVpcID=true \
  --set clusterName=terraform-eks

Deploy the service loadbalancer on the cluster kubectl apply -f manifest/loadbalancer.yaml

Deploy the ingress.yaml for the service

wget https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.9/docs/examples/alb-ingress-controller.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.9/docs/examples/rbac-role.yaml

the cluster name and vpc id is change in the alb-ingress-controller.yaml

Log API Calls using AWS CloudTrail

Sulaiman Olubiyi — Sat, 20 Jan 2024 21:28:38 +0000

Enhanced governance, compliance, operational and risk auditing of your AWS account can be achieved with the aid of AWS CloudTrail.
A user, role, or AWS service's actions are referred to as events in CloudTrail. AWS Management Console, AWS CLI, and AWS SDKs and APIs are sources where events can occur.

CloudTrail stores API calls and activities on the accounts, which include;

Management events: include activities on the control plane such as creating IAM, EC2 instance, and interacting with AWS services on the management level
Data events: include data events such as Lambda invocation, SNS and SQS, and interaction between AWS services.

Detailed steps in creating CloudTrail for your AWS account:

Hover to the search bar on the AWS Console, type CloudTrail then click on the **create a trail **as shown below;

Input the relevant parameters, including the trail name and storage bucket

The JSON Policy for IAMRole for the CloudTrail to access CloudWatch logs
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AWSCloudTrailCreateLogStream2014110", "Effect": "Allow", "Action": [ "logs:CreateLogStream" ], "Resource": [ "arn:aws:logs:us-east-1:014285054687:log-group:CloudTrailRoleForCloudWatchLogs-Douxtech:log-stream:014285054687_CloudTrail_us-east-1*", "arn:aws:logs:us-east-1:014285054687:log-group:CloudTrailRoleForCloudWatchLogs-Douxtech:log-stream:o-je4worq6xn_*" ] }, { "Sid": "AWSCloudTrailPutLogEvents20141101", "Effect": "Allow", "Action": [ "logs:PutLogEvents" ], "Resource": [ "arn:aws:logs:us-east-1:014285054687:log-group:CloudTrailRoleForCloudWatchLogs-Douxtech:log-stream:014285054687_CloudTrail_us-east-1*", "arn:aws:logs:us-east-1:014285054687:log-group:CloudTrailRoleForCloudWatchLogs-Douxtech:log-stream:o-je4worq6xn_*" ] } ] }
Choose the respective events desired for the cloud trail, either management or data events with the corresponding aws service.

The Cloudtrail has been successfully deployed, and the relevant logs streams are shown below.

Voilaaa !!!, I hope you find it insightful and am waiting for your feedback.

Comprehensive Guide to Passing the AWS Advanced Networking Specialty Exam

Sulaiman Olubiyi — Sun, 18 Jun 2023 18:47:40 +0000

Becoming an AWS Certified Networking Specialty professional demonstrates your expertise in designing and implementing advanced networking architectures on the AWS platform. This exam reinforces your knowledge skills in hybrid connectivity, DNS resolution and in-depth of AWS cloud networking services.
This certification opens doors to exciting career opportunities and establishes you as a skilled professional in cloud networking. In this article, I will explore valuable resources and links to help you prepare for and pass the AWS Certified Advanced Networking Specialty exam. We'll also dive into the importance of grit and perseverance, showcasing a real-life example of determination and success.

The Power of Grit and Perseverance:
Success rarely comes without challenges and setbacks. One's journey towards certification often involves hurdles that test their commitment and determination. The path to becoming an AWS Certified Networking Specialty professional is no exception. Let me share my personal experience to illustrate the significance of grit and perseverance.

My Story:
When I initially attempted the AWS Certified Networking Specialty exam, I poured countless hours into studying, meticulously reviewing resources, and practicing with sample questions. However, despite my efforts, I received a disappointing score of 722, falling short of the passing mark. It would have been easy to get discouraged and abandon my pursuit of the certification. But instead, I chose to embrace the principles of grit and perseverance.

Undeterred by my initial setback, I took a step back to reflect on my preparation approach. I analyzed my weaknesses, identified knowledge gaps, and sought out additional resources to strengthen my understanding of complex networking concepts. Determined to succeed, I decided to give the exam another try.

Armed with renewed determination and an updated study plan, I dove back into my preparations. I enrolled in the AWS Skill Builder Class led by Julie Elkins, which provided invaluable insights and practical guidance specific to the exam. Additionally, I used the following resources in this pursuit, which further enhanced my understanding of advanced networking concepts.

Official AWS Certification Exam Guide:
The first step in your exam preparation journey should be to review the official AWS Certified Networking Specialty Exam Guide. This guide provides an overview of the exam domains, knowledge areas, and the specific topics you need to focus on. Understanding the exam blueprint is crucial for effective study planning.
Link to AWS Certified Networking Specialty Exam Guide
AWS Exam Readiness Training:
The AWS Certified Networking Specialty Exam Readiness Training is a valuable resource offered by AWS. It is a self-paced online course designed to help you prepare for the exam. The course covers various networking concepts and technologies relevant to the AWS platform, including Amazon VPC, Direct Connect, VPN, Route 53, and more.. Julie Elkins did a detailed work in the readiness training, explain each of the course curriculum topic and what is expected of you to understand for the exam.
Udemy Course
Stephen Maarek and Chettan Aggarwal offers an in-depth Udemy course titled "Ultimate AWS Certified Advanced Networking - Specialty." This course covers all the exam objectives, provides hands-on labs. The course helps me to understand the fundamentals of those services and solidify my knowledge
Adrian Cantril Course
This is detailed for hands-on practice and configuration, also this course explains the technologies including Direct Connect, VPN connection, Route53 and more.
He also explained basics of networking and security including BGP,DNS,OSI Layers, encryption method and more.
AWS Twitch videos
This seems like a real life situation for AWS Networking services, AWS Networking experts explains and analyze the services staring with fundamentals, deployment and configuration with best practices.
I really felt more confident after going through those sessions.
Link for the twitch sessions

Whitepapers and Architectural Documentation:
AWS offers a collection of whitepapers and architectural documentation that covers advanced networking topics and best practices. These resources delve into network design considerations, hybrid architectures, security, and more. Reading these documents will not only enhance your knowledge but also provide you with insights into real-world scenarios and solutions.
Link for AWS Whitepapers
Practice Tests
Do as much you can, analyse scenarios and case studies, make inference the most cost efficient, most operational efficient and security perspective.
I used Jon Bonso practice test during my preparation which assisted in some scenario based questions.

Conclusion
Remember, success is not always immediate, but with the right mindset and continuous effort, you can conquer any challenge. Embrace the power of grit and perseverance as you embark on your journey towards becoming an AWS Certified Networking Specialty professional.Conclusion:
My personal journey highlights the significance of grit and perseverance when pursuing the AWS Certified Networking Specialty certification. It is natural to face setbacks and obstacles along the way, but it is through perseverance that we overcome them. By leveraging the resources and links provided in this guide, combined with unwavering determination, you too can achieve success in the AWS Certified Networking Specialty exam.

Remember, success is not always immediate, but with the right mindset and continuous effort, you can conquer any challenge. Embrace the power of grit and perseverance as you embark on your journey towards becoming an AWS Certified Networking Specialty professional.

Grow like an AWS Transit Gateway.

Sulaiman Olubiyi — Sun, 15 Jan 2023 21:14:25 +0000

Earlier before the invention of AWS Transit Gateway, solution architect designed connectivity with VPCs using the following methods;

VPC peering: full private ip connectivity between VPCs, it is non-transitive. For instance, A company has 10 VPCs, there is a need to enable connectivity across all the VPCs. Using the full mesh formula,

No of VPC peering = n(n-1)/2

where n is the number of VPCS
No of VPC peering = 10(10-1)/2
No of VPC peering = 45

This infers that the network architect needs to setup 45 VPC peering connections to connectivity across 10 VPCs.

According to AWS;

125 Amazon VPC Peering connection per Amazon VPC
50 static routes per Amazon VPC route table(default)

VPN connection: a secured tunnel connection over the public using a between on-premise network to AWS VPC. The Virtual private network is achieved between the customer gateway CGW and virtual gateway VGW on the AWS VPC.
Also, this is an independent connection to each VPC from the customer network. using the above example, the company would need to setup 10 VPN connections.
AWS Direct connect: Dedicated connection between the customer network and the AWS network with high bandwidth and availability. Independent connection is built from the direct connect gateway to each VPC.

With the architecture above has several limitations;

Time consuming
Prone to errors
Complicated with many routes on the route table
Non-Scalable

Let's discuss about the growth...

Amazon Web Services (AWS) Transit Gateway is a fully managed service that enables customers to connect their Amazon Virtual Private Clouds (VPCs) and on-premises data centers to a single network. This service simplifies the network architecture and management, making it easier for customers to connect their distributed networks.

One of the main benefits of Transit Gateway is that it allows customers to connect multiple VPCs and on-premises networks to a single transit network, reducing the need for complex peering connections. This simplifies the network architecture and reduces the number of devices required to connect the various VPCs and on-premises networks.

In addition, Transit Gateway enables customers to scale their network connections easily and quickly. customers can add or remove connections to the transit network without any downtime, and the service automatically scales to handle the increased traffic.

TGW Route Table
VPC A: Attachment 1
VPC B: Attachment 2
VPC C: Attachment 3
VPC D: Attachment 4
On-prem: VPN

Attachment: A VPN and Amazon VPC connection to the Transit Gateway
Association: The packets from the attachment are routed using a route table.
Propagation: Route tables where the attachment's routes are installed.propagation

According to AWS;

5 TGWs per account/ TGW attachment per Amazon VPC
10,000 routes per TGW
5,000 TGW Attachment per region per account
Support up to 5,000 VPCs
50Gbps maximum burstable bandwidth per attachment
1.25Gbps maximum bandwidth per VPN connection

Overall, AWS Transit Gateway has greatly increase the scalability and availability of hybrid connectivity of Amazon VPCs to corporate data centers.
Next article will discuss on the implementation of AWS Transit gateway across VPCs in single and multiple AWS accounts.

Fundamentals of Container Networking

Sulaiman Olubiyi — Mon, 09 Jan 2023 07:37:25 +0000

Containers are successor to virtual machines, they are abstracted within the operating system level by the container engine. Container networking is a crucial aspect of modern software development and deployment, as it enables communication between containers, microservices, and applications in a containerized environment.In this article, we will explore the basics of container networking and how it works, types, as well as its benefits and challenges.

What is container networking?
Containers are a lightweight and portable way to package and deploy applications, as they include all the necessary libraries, dependencies, and runtime environment in a single package.
Container runtimes use network drivers to define how containers connect with each other by controlling the host iptables, as well as with external networks and services

Follow the guidelines to setup a docker engine: Docker Installation

Verify the Docker engine Installation

Types of Container networking
There are several container networking models, each with its own benefits and limitations.

None: With this type of networking for containers, a network stack is provided to the container. Although it only gets a loopback interface, the container lacks an external network interface. When no networking is utilized. This model can be used to setup containers for future network connections, allocate containers without external contact, and test containers.
Bridge: This is the default network type for docker, in which a network bridge is created on the host system to connect the containers to the host network. To enable single-host networking, bridge networking makes use of iptables for NAT and port mapping

This shows the default docker bridge network is 172.17.0.0/16, all containers belong to this network by default

This shows ubuntu container was assigned an ip address within the docker bridge network(172.17.0.2/16)
Host: A newly built container can interact with the host to share a network namespace. There is no longer a need for NAT because it offers better performance that is practically as fast as bare metal networking. Ports conflict may result from this type of networking model.

The container has access to the host's network interfaces.

Overlay: Another popular type is the overlay model, which creates an additional layer of abstraction on top of the host network to connect the containers. This model allows containers to communicate with each other across multiple hosts using protocols VXLAN, Calico, making it suitable for distributed systems such as kubernetes. The overlay model is more complex to set up and manage than the bridge model, but it offers better isolation and security

Benefits of container networking
There are several benefits to using container networking:

Portability: Containers are designed to be portable, and container networking allows them to communicate and interact with each other and with the host system regardless of the underlying infrastructure. This makes it easier to deploy and scale applications across different environments and platforms.
Isolation: Container networking enables the isolation of different applications and services within the same host system, reducing the risk of interference and conflicts. This is especially useful in multi-tenant environments where multiple applications and services are running on the same host.
Resource efficiency: Containers are lightweight and use fewer resources than traditional virtual machines, and container networking allows multiple containers to share resources and infrastructure. This makes it easier to optimize resource usage and reduce costs.

Challenges of container networking

Despite the benefits of container networking, there are also several challenges to consider:

Complexity: Container networking can be complex to set up and manage, especially in large and distributed systems. It requires a good understanding of networking concepts and technologies, as well as the underlying infrastructure and environment.
Security: Container networking introduces additional security risks, as it allows containers to communicate with each other and with external networks. It is important to implement appropriate security measures and controls to prevent unauthorized access and attacks.
Performance: Container networking can impact the performance of applications and services, especially in high-traffic environments. It is important to monitor and optimize the container networking configuration and performance to ensure the best possible user experience.

Your comments and feedbacks will be appreciated, wactchout for the next article on Container Network Management

Practical steps using AWS Organization

Sulaiman Olubiyi — Wed, 29 Jun 2022 16:01:10 +0000

Scenario where you have to manage several accounts in your company being the Cloud Administrator.

AWS Organization offers solution to the pain-point with several advantages;

AWS Organizations helps you centrally manage and govern your environment as you grow and scale your AWS resources.
Using AWS Organizations, you can create accounts and allocate resources, group accounts to organize your workflows.
Apply policies for governance using SCP(Service Control Policy)
Simplify billing by using a single payment method for all of your accounts(consolidated billing)

It consists of two entities:

Management/Master account: The management account creates the organization.
Member account: these are accounts which are invited by the management account.

Steps in creating an AWS Organization

Click to the AWS console
Login to you accounts with your IAM credentials
Search for the aws organization on the search bar
Click on the AWS Organization title indicate above
Click the Create Organization on the home page
After creating an organization, invitation can be send to member account either existing account or a new account

An existing account can be invited using account ID or email
A new account can be invited using email only
when you create a new account the role is automatically given, BUT adding an existing account requires to manually create the role
Grant OrganizationAccountAccessRole to the master account from the member account IAM for trusted entity and permission.
Go to the IAM service, Click on role, then create the trusted entity
The account ID of the master account is given as a trusted entity for this role.
Also, assign the AWS Managed permission AdministratorAccess to the role
Ensure that the role name is OrganizationAccountAccessRole , add description based on your preference
OrganizationAccountAccessRole and click the create icon.
For an existing AWS account, the admin needs to accept the invitation sent by the master account to join the organization.
Proceed to Master account and switch role
Login into the member accounts with the credentials created earlier.

After putting the correct login credentials, you will be successfully login into the member account as a Federated user*

HIERACCCHY STRUCTURE

This enable to create the OU(Organizational Unit) within the root container of the organization. Members account can be grouped into the OU.

In the AWS Organization Dashboard (Master Account), tick the root as specified , then click actions
Create the Organizational unit to structure the member accounts
Click the member account to add to OU, then click on actions to move.

USING SERVICE CONTROL POLICY(SCP).

SCP is used to perform permission across member accounts, the permission given should also be allowed in the IAM of the member accounts.
SCP basically restricts the access to AWS services prior before the IAM permission takes over.

By default, the SCP is disabled in the organization.

Thanks for reading!!!

AWS VPC: What you need to understand

Sulaiman Olubiyi — Sun, 23 Jan 2022 18:47:01 +0000

Have you been wondering how AWS defines the backbone of its networking service?

Well, this is all thanks to VPC Virtual Private Cloud, a virtual network within the AWS cloud, and it comprises the following: security groups, network access control list (NACl), subnet, route tables, internet gateway and NAT gateway.
Using an analogy, AWS cloud can be described as an estate which contains several houses.
A VPC is an individual house in that estate and you can place your properties in different positions in your house. Some properties may be placed in the living room while others in the bedroom. When expecting a visitor, that is, traffic from the internet. This can refer to the isolated logical network in the AWS cloud where you provision your resources such as application and database servers. The concept of VPC components are explained thus:

Internet Gateway: This is likened to the telephone in your house and it is the only way for anyone that wants to visit you to reach you. If your telephone line is off, nobody can reach your house and only the people within your house can talk to each other. In AWS, this refers to the default route to the internet which enables your resources in the VPC to communicate with the internet.

Subnets: These are logical segmentation of your resources, they can be likened to the properties in your house. The properties placed in the living room are public subnets, all your visitors can see them such as television, stereo system etc.
In AWS, Web/Application servers are deployed in the public subnets, external users can have access to them and are reachable on the internet.
The properties placed in your bedroom are private subnets, they are accessible within your house, that is, only your family members have access to them.
In AWS, database servers are mostly placed in the private subnets in VPC because they are only accessible within your VPC network.
NAT Gateway: By default, only people staying in the living room can meet the visitors, perhaps you are in the bedroom and you want to meet the visitor, you can use your mobile phone to talk to them, but you would be the one to make a call request.
In AWS, this allows resources deployed in the private subnets to have access to the internet, and is especially used for upgrade and software patches for database servers or to enhance the security level of the system.

Network Access Control List: These are the security guards guarding your home, they will check the visitor,to either grant access to the building or not.
In AWS, this serves as a security measure at the subnet level for your VPC network to deny or allow inbound and outbound traffic. At default, it allows both inbound and outbound traffic.
Inbound traffic: User's request entering the VPC
Outbound traffic: User's response leaving the VPC
Security Group: You have an electronic door which checks the visitors before they can come in. If they have an appointment or invite, they will be granted access to your living room and welcomed to your home, and when the visitor is departing, he wouldn’t be subjected to another check (stateful).
In AWS, this is a security measure at the instance level, it only allows traffic and it is stateful which means once the traffic is allowed in, automatically the traffic will be permitted out.

I hope you understand the basics of VPC, your comments are welcome
Cheers 😊