Forem: oteri

How to Scrape Any Website Using Bright Data MCP Server and AI Agents

oteri — Fri, 09 Jan 2026 09:37:27 +0000

I came across the Bright Data MCP (model context protocol) server from Noah Kalson and decided to try my hands on this incredible tool.

Bright Data MCP server is a unified, AI-ready access layer that provides seamless, scalable, and unblockable access to any public websites enabling LLMs, agents and apps to access, discover and extract web data in real-time.

The common problem I found worthy of solving was to scrape a public website like Nike.com particularly sneakers for men and have it readily available for users to view showcasing details like the product item, price, availability, and product link.

So I built this all in one tool that allows for the extraction of its data without any restriction because with Bright Data MCP, your applications can effortlessly retrieve both static and dynamic content from across the web, eliminating the need to build or maintain complex data scraping and unlocking infrastructure.

For technologies used in the project, I used:

LangChain: allows for the connection to multiple servers or LLM providers at the same time.
LangGraph: connects to all the request points or nodes together.
next.js: A React framework for building scalable and user friendly UI interface
Python: for the backend using FastAPI.
Claude (Anthropic API): to orchestrate multiple operation.

Here's a walkthrough of the final product in this demo.

The code is public for viewing, check this repo.

Let me know what you think by giving this app a try and searching for your favorite Nike sneaker.

Build in Public (Data Engineering): Day Zero

oteri — Mon, 10 Nov 2025 16:18:17 +0000

It is great to come back to sharing my thoughts on the next line of action I want for my career. At this moment, I want to go all in with data engineering with focus on mastery.

Data is critical in this modern time as it has always been. Some social media platforms like LinkedIn and Instagram have made it explicity known that they will be using your data for training if you are comfortable with it.

In this data challenge series, I will be going from the basics to the complex to grasp the full concept of what it is to be a data engineer right from ingesting, storing, cleaning, and processing the data.

Who are Data Engineers?

From the brief intro above, data engineers are integral part of the company for maintaining systems that ingest data from both internal or external sources like databases and APIs, store this data for further processing, clean, and process them after going through a series of transformation steps.

One of the most popular data processing engine is Apache Spark which allows you to efficiently process large amounts of data.

How Spark works

To use Spark, you first need to have a Spark cluster. A cluster is a collection of computers running Spark software.

For a Spark application, the cluster consists of two components:

driver: this orchestrates the data processing
executors: this process the data itself

Installing Spark

brew install apache-spark

# after installation, run:
pyspark

# install jupyter notebook in the virtual env
pip install jupyter

# configure pyspark to use the jupyter notebook when we start it
# define these two environment variables
export PYSPARK_DRIVER_PYTHON='jupyter'
export PYSPARK_DRIVER_PYTHON_OPTS='lab'

# run or start the development server in the venv
pyspark

Practical demo

For you to become better, you must practice. This demo will use the data processing framework, Apache Spark to start the development server in Jupyter Notebook in a virtual environment.

This documentation is a good starting guide to writing efficient Spark applications and its functionality.

Now, let's create a SparkSession by reading an actual data from a CSV. To follow along, download a sample data from this website.

from pyspark.sql import SparkSession

spark = SparkSession \
        .builder \
        .appName('Read inside airbnb data') \
        .getOrCreate()

listings = spark.read.csv('data/listings.csv.gz',
                          header=True,
                          inferSchema=True,
                          sep=',',
                          quote='"',
                          escape='"',
                          multiLine=True,
                          mode='PERMISSIVE'
                         )

listings.printSchema()

for list in listings.schema:
    print(list)

description = listings.select(listings.description)

description.show(20, truncate=False)

When not to work with Apache Spark

when working on small amount of data
processing data in real-time (it is better to use data streaming)

Where to find out more about this journey

The public repo is available here:

https://github.com/Terieyenike/data-engineering

If you want to connect and reach out, I am active on LinkedIn:

https://linkedin.com/in/terieyenike

Build a Serverless Distance Calculator on AWS (Amplify, Lambda, API Gateway & DynamoDB)

oteri — Tue, 12 Aug 2025 11:00:00 +0000

AWS is the world's leading cloud provider, and its cloud computing infrastructure has an 85% adoption rate in the enterprise segment. With its multitude of services, you can combine all the pieces to create an application that you could use in the real world.

This project will show you how to design and build a web application from scratch with five AWS services, Amplify, Lambda, IAM, API Gateway, and DynamoDB. The concept behind this app is to calculate the distance in kilometers between two points using the latitude and longitude of a location.

Let’s get started!

Prerequisites

Basic AWS knowledge
Python
HTML, CSS, and JavaScript

What is AWS Amplify?

AWS Amplify allows for the hosting and deployment of frontend applications which is crucial in the software development lifecycle of products. In this scenario, we will have to deploy our project to Amplify, similar to other deployment providers like Vercel, Netlify, Render, and so on.

Setting up of Web App

To deploy our web app to Amplify, create a new folder called TwoPointOh and within the directory, add an index.html file.

Copy and paste this code:

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Two Point Oh!</title>
</head>

<body>
    Two Point Oh!
</body>
</html>

Now, let’s compress (zip) this file to deploy this app without git. Head over to your Amplify console and click on the create new app button.

Select the option “Deploy without Git” and click the Next button.

Next, give your app a name and drag and drop the zipped index.html file.

At the bottom of the page, click on the “Save and deploy” button to start the deployment process. After the process is complete, Amplify will generate a public URL.

What is AWS Lambda?

The Lambda function will handle the math calculation for the distance effortlessly.

AWS Lambda function is a bit of code that runs serverlessly in response to some trigger. For example, if you upload a picture to an S3 Bucket, it could trigger a Lambda function to process a thumbnail of that picture.

To put it in layman's terms, AWS Lambda is code that runs when you need it, meaning you don’t have to set up servers to run the code, it just happens automatically behind the scenes.

Let’s write the Python code that uses the math library to do the calculations we need between our two points. In your AWS console, select the Lambda service and create a function.

To create the function, we need to give it a name and select Python 3.13 for the runtime which is the language the function will be written. After that, make sure to click on the Create function button as you can leave the other default option.

In the code tab, copy and paste this code into the text editor:

import json
import math

import boto3
from time import gmtime, strftime

# define the handler function that the Lambda service will use an entry point
def lambda_handler(event, context):

# get the latitude and longitude of the two points from the event
    lat1 = float(event['lat1'])
    lon1 = float(event['lon1'])
    lat2 = float(event['lat2'])
    lon2 = float(event['lon2'])

# convert the latitude and longitude to radians
    lat1_rad = math.radians(lat1)
    lon1_rad = math.radians(lon1)
    lat2_rad = math.radians(lat2)
    lon2_rad = math.radians(lon2)

# calculate the distance in kilometers using the Haversine formula
    dlon = lon2_rad - lon1_rad
    dlat = lat2_rad - lat1_rad
    a = math.sin(dlat / 2)**2 + math.cos(lat1_rad) * math.cos(lat2_rad) * math.sin(dlon / 2)**2
    c = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
    distance = 6371 * c  # Earth's radius in km

    return {
        'statusCode': 200,
        'body': json.dumps('Your result is ' + str(distance))
    }

The code snippet showcases a function called def lambda_handler(event, context) which the lambda service uses as an entry point to pull out the different parameters that get sent to the function when it is invoked.

With the code in place, deploy the code by clicking the Deploy button. After this is complete, switch to the Test tab where the user will pass in four numbers of the location both their latitude and longitude values.

Paste this JSON code below in the Event JSON section to represent the four different values that normally will come in from the user interface. Or better still use your values.

{
  "lat1": "47.5975024",
  "lon1": "-122.3492781",
  "lat2": "40.7579787",
  "lon2": "-73.9900326"
}

Now click on the Save button and after that the Test button to configure a test. Run this test to ensure things are calculated properly.

The executed code should look like this with the response of the status code and the body, the result of the distance.

To invoke the math functionality, we need a public endpoint or URL that can be called to trigger that function to run which will make use of API Gateway. The core service in AWS that you can use to build, deploy, and manage APIs.

API Gateway

On your API Gateway console, create a new API and select the REST API option. Click on Build.

Next, create a method.

With the new page opened after clicking the Create method, select the method type from the dropdown and choose POST (as we will be passing data from the user). The integration type should be Lambda function and beside your region, type in the name or alias of the lambda function you created earlier as shown:

Scroll and click the Create method button.

Before deploying the API, we need to enable CORS. Click on the / path and the Enable CORS button.

The function of enabling CORS (cross-origin resource sharing) is to give the web application on one origin or domain the right to access resources on another origin or domain. Since Amplify will be running on one domain and our Lambda function on another to access the API Gateway, the resource is required for this step. So in essence, we need to work across those domains or origins.

Under the Access-Control-Allow-Methods section, select the POST method and save it.

Let’s deploy the API. When you click on the Deploy API button, a pop-up appears, and on the Stage option dropdown, select New Stage and give it a Stage name. Make sure to click on Deploy so it generates the invoke URL.

Copy the URL as we will be needing it later in our HTML page. The invoke URL should look something like this:

https://410ddoituf.execute-api.eu-west-1.amazonaws.com/dev

The last step of this setup is to validate the API by testing it. On the resources tab, select the POST method and Test tab while entering your four values of the latitude and longitude.

Scroll the page to click on the Test button and you will see the result of your response.

AWS DynamoDB, its Identity and Access Management (IAM)

Most real-world applications these days have databases and incorporating a database to persist and store the math result and return it to the user is crucial for dynamic apps. Also, we will learn how to permit Lambda to write to the database.

DynamoDB is a key-value NoSQL database that is generally light-weight than relational databases where you have to set up the schema ahead of time and the relationships.

Back in your console, navigate to DynamoDB and create a new table as shown with the table name and the partition key, as ID by selecting the Tables tab or clicking the Create table button.

Within the Additional Info section copy the ARN (amazon resource name) which identifies the table within AWS.

Next, navigate back to the Lambda function to update the code to be able to save values into DynamoDB. Before doing that, let’s configure the permission by clicking on the Role name.

It opens in a new tab. Click Add permissions and from the dropdown select Create inline policy.

Copy and paste this permission statements code in the JSON editor:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "VisualEditor0",
        "Effect": "Allow",
        "Action": [
            "dynamodb:PutItem",
            "dynamodb:DeleteItem",
            "dynamodb:GetItem",
            "dynamodb:Scan",
            "dynamodb:Query",
            "dynamodb:UpdateItem"
        ],
        "Resource": "YOUR-TABLE-ARN" # replace with the ARN of the table you created earlier
    }
    ]
}

Now, paste your copied ARN you copied previously. Click on Next to give it a policy name and create the policy.

To check the data is saved to DynamoDB, under the code tab of the Lambda function, paste this updated code:

import json
import math

import boto3
from time import gmtime, strftime

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(‘YOUR DYNAMODB DATABASE NAME’)
now = strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())

def lambda_handler(event, context):

    lat1 = float(event['lat1'])
    lon1 = float(event['lon1'])
    lat2 = float(event['lat2'])
    lon2 = float(event['lon2'])

    lat1_rad = math.radians(lat1)
    lon1_rad = math.radians(lon1)
    lat2_rad = math.radians(lat2)
    lon2_rad = math.radians(lon2)

    dlon = lon2_rad - lon1_rad
    dlat = lat2_rad - lat1_rad
    a = math.sin(dlat / 2)**2 + math.cos(lat1_rad) * math.cos(lat2_rad) * math.sin(dlon / 2)**2
    c = 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a))
    distance = 6371 * c  # Earth's radius in km

    response = table.put_item(
        Item={
            'ID': str(distance),
            'LatestGreetingTime':now
            })

    return {
        'statusCode': 200,
        'body': json.dumps('Your result is ' + str(distance))
    }

This code block will write the distance and the time of the calculation in DynamoDB. Replace the value of the name of your database within the parenthesis:

table = dynamodb.Table('TwoPointOhDB')

The same process of deploying and testing is necessary to ensure it sends the correct data to DynamoDB.

In your console, go back to DynamoDB to check if the distance and time were saved in the database by clicking on the Explore table items.

It worked! The data saved successfully in the DynamoDB table.

Connect Amplify and the API Gateway

The only thing left to have a proper working app is a connector between Amplify and API Gateway. In your index.html, copy-paste this new updated piece of code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Two Point Oh!</title>
    <style>
        body {
            margin: 0;
            font-family: 'Arial', sans-serif;
            background-color: #FFFFFF;
            color: #000000;
            display: flex;
            flex-direction: column;
            min-height: 100vh;
        }

        header {
            text-align: center;
            padding: 20px;
            background-color: #0056D2;
            color: #FFFFFF;
        }

        header h1 {
            margin: 0;
            font-size: 2rem;
        }

        header p {
            margin-top: 10px;
            font-size: 1rem;
        }

        main {
            flex: 1;
            display: flex;
            justify-content: center;
            align-items: center;
            padding: 20px;
        }

        .form-container {
            max-width: 500px;
            width: 100%;
            background-color: #F8F9FA;
            padding: 20px;
            border-radius: 8px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
            text-align: center;
        }

        .form-container label {
            display: block;
            margin-top: 10px;
            font-size: 1rem;
            color: #333333;
        }

        .form-container input {
            width: calc(100% - 20px);
            padding: 10px;
            margin-top: 5px;
            font-size: 1rem;
            border: 1px solid #CCCCCC;
            border-radius: 4px;
            color: #333333;
            background-color: #FFFFFF;
        }

        .form-container button {
            width: 100%;
            margin-top: 20px;
            padding: 10px;
            font-size: 1rem;
            background-color: #0056D2;
            color: #FFFFFF;
            border: none;
            border-radius: 4px;
            cursor: pointer;
            transition: background-color 0.3s ease;
        }

        .form-container button:hover {
            background-color: #003C99;
        }

        #result {
            margin-top: 20px;
            font-size: 1.2rem;
            color: #0056D2;
        }

        footer {
            text-align: center;
            padding: 10px;
            background-color: #0056D2;
            color: #FFFFFF;
            font-size: 0.9rem;
        }

        @media (max-width: 768px) {
            header h1 {
                font-size: 1.5rem;
            }

            header p {
                font-size: 0.9rem;
            }

            .form-container {
                padding: 15px;
            }

            .form-container button {
                font-size: 0.9rem;
            }

            footer {
                font-size: 0.8rem;
            }
        }
    </style>
    <script>
        const callAPI = (lat1, lon1, lat2, lon2) => {
            if (!lat1 || !lon1 || !lat2 || !lon2) {
                document.getElementById('result').innerText = "Please fill in all fields.";
                return;
            }

            const headers = new Headers({ "Content-Type": "application/json" });
            const body = JSON.stringify({ lat1, lon1, lat2, lon2 });

            fetch("YOUR API GATEWAY ENDPOINT", {
                method: 'POST',
                headers,
                body,
            })
                .then(response => response.text())
                .then(result => {
                    const parsedResult = JSON.parse(result).body;
                    document.getElementById('result').innerText = `Distance: ${parsedResult} km`;
                })
                .catch(error => {
                    document.getElementById('result').innerText = "Error calculating distance. Please try again.";
                    console.error(error);
                });
        };
    </script>
</head>
<body>
    <header>
        <h1>Two Point Oh!</h1>
        <p>Calculate the distance between two points using their latitude and longitude.</p>
    </header>
    <main>
        <div class="form-container">
            <label for="lat1">Latitude 1:</label>
            <input type="text" id="lat1" placeholder="Enter Latitude 1">

            <label for="lon1">Longitude 1:</label>
            <input type="text" id="lon1" placeholder="Enter Longitude 1">

            <label for="lat2">Latitude 2:</label>
            <input type="text" id="lat2" placeholder="Enter Latitude 2">

            <label for="lon2">Longitude 2:</label>
            <input type="text" id="lon2" placeholder="Enter Longitude 2">

            <button type="button" onclick="callAPI(
                document.getElementById('lat1').value,
                document.getElementById('lon1').value,
                document.getElementById('lat2').value,
                document.getElementById('lon2').value
            )">Calculate</button>
            <div id="result"></div>
        </div>
    </main>
    <footer>
        &copy; 2024 Two Point Oh! All rights reserved.
    </footer>
</body>
</html>

The following occurs in this block of code:

There is a form to accept the user input and a button calculate that when the user clicks it, the four values are interpreted with the callAPI method
The callAPI method will take the latitude and longitude numbers as parameters and send your request using your API Gateway endpoint which should be replaced in the parenthesis of the fetch method

With this complete, zip the index.html again and head to your Amplify console to drag and drop the updated file to redeploy the existing app by clicking on the Deploy updates button.

For the complete source code, check out the GitHub repository.

Wrapping Up

In conclusion, AWS is a complete suite for delivering outstanding applications that make a difference in how you approach development. Using the combination of all these AWS services in one project, you are guaranteed continuous integration and continuous delivery or deployment without downtime when it matters most because of its infrastructure and cloud computing capabilities.

Whatever you do as a developer or an enterprise solution, AWS scales and manages your work efficiently from initialization to deployment without the hassle of managing your infrastructure.

Build a Content-Aware Media Cropper Using AI, Cloudinary, and Streamlit

oteri — Mon, 07 Jul 2025 13:43:05 +0000

Do you want to be productive as a developer working with a tool that automatically crops your images and videos to the right dimension? If yes, you do not need to use sophisticated tools or editing software to make this happen.

With Cloudinary, you are in for an efficient and effective cropping using content-aware which is an AI powered tool which deliver images and videos to perfectly fit your graphic design and layout, on any device.

In this tutorial, you'll use Cloudinary and Streamlit to build an engaging and performant web experience for interaction saving you time and effort in delivering visuals to users.

What You'll Need

To get started with this application, the following is what you need to know or have:

Create a free Cloudinary account
Understanding of Python and Streamlit

Check out the Streamlit documentation if you are not conversant with this open-source app framework.

Setting Up the Project

To setup the environment, you need to create a virtual environment and install the dependencies.

mkdir content_aware
cd content_aware

.
├── app.py
├── README.md
├── .env
├── .gitignore
└── requirements.txt

In the terminal, run the following command to create a virtual environment in the root of the project.

python -m venv .venv

Activate the virtual environment.

source .venv/bin/activate

Install the dependencies.

pip install streamlit cloudinary python-dotenv

It is important to include the packages and its versions in the requirements.txt file as shown below:

cloudinary==1.41.0
python-dotenv==1.0.1
streamlit==1.39.0

Environment Variables

Create a file called .env which ensures that your environment variables (API keys) are not exposed to the public.

CLOUDINARY_CLOUD_NAME=your_cloud_name
CLOUDINARY_API_KEY=your_api_key
CLOUDINARY_API_SECRET=your_api_secret

Go to your Cloudinary dashboard and get your credentials to replace the placeholders with your own.

After doing this, create a file called .gitignore and include the .env file in it like this:

.gitignore

.env

Creating the Streamlit App

Before you begin building the app, create a file, app.py and add the following code:

app.py

import streamlit as st
import cloudinary
from cloudinary import CloudinaryImage, CloudinaryVideo
from dotenv import load_dotenv
import os

# Load environment variables from .env file
load_dotenv()

# Initialize Cloudinary configuration
cloudinary.config(
    cloud_name=os.getenv('CLOUDINARY_CLOUD_NAME'),
    api_key=os.getenv('CLOUDINARY_API_KEY'),
    api_secret=os.getenv('CLOUDINARY_API_SECRET'),
    secure=True
)

The above code initializes the Cloudinary configuration and loads the environment variables from the .env file.

Next, create a function that generates the image URL using the CloudinaryImage class with this code:

app.py

... # existing code

def generate_image_url(public_id: str, aspect_ratio: float, width: int) -> str:
    image = CloudinaryImage(public_id)
    transformation = [
        {'aspect_ratio': aspect_ratio, 'gravity': "auto", 'width': width, 'crop': "fill"},
        {'quality': "auto"},
        {'fetch_format': "auto"}
    ]
    return image.build_url(transformation=transformation)

The function, generate_image_url takes three parameters with its data type as follows:

public_id: The public ID of the image in Cloudinary.
aspect_ratio: The desired aspect ratio for cropping.
width: The width for the output image.

The function returns the URL of the processed image with the transformation applied.

Let's do the same for the video URL by defining the generate_video_url function with this code:

app.py

... # 

def generate_video_url(public_id: str, aspect_ratio: float, width: int) -> str:
    video = CloudinaryVideo(public_id)
    transformation = [
        {'aspect_ratio': aspect_ratio, 'gravity': "auto", 'width': width, 'crop': "fill"},
        {'quality': "auto"},
        {'fetch_format': "auto"}
    ]
    return video.build_url(transformation=transformation)

Building the UI for the Image

So far, we have created the functions that generate the image and video URLs without the actual UI. In this section, copy and paste the following code:

... # existing code

def main():
    st.title("AI-Powered Content-Aware Cropping for Images and Videos with Cloudinary")
    st.write("This app demonstrates content-aware cropping using Cloudinary's AI capabilities.")

    # User input for Cloudinary image public ID
    public_id = st.text_input("Enter the Cloudinary image public ID:", "profile_uzviqu")

    # Aspect ratio options with corresponding widths
    aspect_ratio_options = {
        "1:2": (0.5, 433),
        "5:2": (2.5, 1300),
        "1:1": (1.0, 867)
    }

    st.subheader("Aspect Ratio:")
    selected_option = st.radio("Select Aspect Ratio", list(aspect_ratio_options.keys()), index=1, key="image_radio")

    # Get aspect ratio and width based on selection
    aspect_ratio, width = aspect_ratio_options[selected_option]

    # Generate and display image URL
    if public_id:
        image_url = generate_image_url(public_id, aspect_ratio, width)
        st.image(image_url, caption="Content-Aware Cropped Image")

The code above depicts the following:

Displays the title, description, and a text input for the Cloudinary image public ID.
Shows the aspect ratio options as radio buttons with its corresponding widths.
In Streamlit, functions can accept arguments such that index=1 in the selected_option variable indicates the selected option of the radio button is on the aspect ratio of 5:2 when the page loads.
To generate and display the image URL, the public ID provided in the text input must be valid. Afterwards, the image URL is displayed using the st.image function with a caption below the image.

Displaying the Video

To display the video, follow the same process as the image above. The only difference is that the st.video function is used instead of the st.image function to call the video URL.

Include the following code in the app.py file:

... # existing code

def video():
    public_id = st.text_input("Enter the Cloudinary Video public ID: ", "video-player/gushing-waterfall")

    video_aspect_ratio_options = {
        "1:2": (0.5, 364),
        "5:2": (2.5, 1300),
        "1:1": (1.0, 728)
    }

    st.subheader("Aspect Ratio:")
    selected_option = st.radio("Select Aspect Ratio", list(video_aspect_ratio_options.keys()), index=1, key="video_radio")

    # Get aspect ratio and width based on selection
    aspect_ratio, width = video_aspect_ratio_options[selected_option]

    if public_id:
        video_url = generate_video_url(public_id, aspect_ratio, width)
        st.video(video_url)

Calling the Functions

Add the following code to the app.py file:

... # existing code

if __name__ == "__main__":
    main()
    video()

Running the App

Now that everything is in place, run the app with this command:

streamlit run app.py

The app should look something like this:

Try it out for yourself.

If you want to make this app yours, then you have to clone it from this repo and include your Cloudinary credentials in the .env file.

Deploying your App

Streamlit offers a free tier for hosting your app using the Streamlit Community Cloud.

Follow the instructions in this guide to deploy your app in production. Remember to include your Cloudinary credentials stored in your .env file in the app settings of the Streamlit Community Cloud.

Wrapping Up

Cloudinary offers a great service for resizing images and videos to desired width which eliminate the need to manually do it, but rather depend on the AI for its smart content-aware cropping.

In this article, we also discovered how to build and deploy this app with Streamlit making you showcase your work without frontend experience.

Resources

Cloud 101: A No-Fluff Guide to AWS Concepts That Matter

oteri — Mon, 14 Apr 2025 08:00:00 +0000

Investing in the cloud with AWS instead of managing your on-premise infrastructure is worth every penny for you either as an individual or business. In essence, just move to the cloud.

With AWS, you don't play the guessing game with website traffic or worry about serving users across the globe. Any of its services you use, scales with you whether you have 10 visitors or 10 million.

You essentially use the AWS services when you need it, and pay for what you use. Kind of like electricity bill, but smarter.

In this article, we will explore the core concepts of AWS cloud, including the key benefits of the cloud, some of the foundational services you need to know, and so on.

Let's begin this exploration!

What is the Cloud?

The cloud is not mystical but a collection of distributed servers in a warehouse and other infrastructure like networking hardware, database storage, and hard drives accessible over the internet.

What are the AWS Foundational Services?

AWS has hundreds of services available but there are just a few of these tools that are foundation to everything else which are the following:

Elastic Compute Cloud (EC2)
Relational Database Service (RDS) and DynamoDB
Simple Storage Service (S3)
Virtual Private Cloud (VPC)
Identity and Access Management (IAM)

To learn more about each of these services, the AWS official documentation covers in-depth details about them.

Benefits of the Cloud

As a business, I believe you should leverage some of these benefits that makes AWS the leading cloud provider.

Elasticity: Elasticity deals with the ability to adapt to workload changes. A typical example will be Amazon.com handling a huge spike in traffic on prime day and scales back those resources the next day to a normal load.
Scalability: The ability to handle increased workload by adding resources like scaling up a database because the size has grown over time.
High availability: In the event of an environment disaster, deploy your application to multiple regions. Therefore, high availability allows a system to continue functioning, even if some components fail. When a data center is down, traffic is routed to another.
Reliability: Reliability means performing accurately and consistently whenever it is needed.
Agility: Develop, test, and launch applications to deliver business value. With AWS, it is possible to launch a new business application in days rather than months.

Other benefits of the cloud include:

Global reach
Pay-as-you-go pricing
Economies of scale

Economics of AWS

When considering to use AWS, these are the things you need to look into:

Total cost of ownership: Just like a car, apart from the cost, there are other cost that needs your attention like insurance, taxes and registration, repairs/maintenance, and so on which can be divided into capital (up-front) and operating (ongoing) expenses.
Capital expenses: The same concepts can be applied to hardware and software. Your up-front cost goes into purchasing your servers, racks, cables, software licenses, and if the servers gets old, you may need to purchase new ones over time. The beauty of AWS is that they handle all these things for you, so you don't need to worry about the capital expenses.
Operational expenses: On the other hand, the operating expenses demand that you need people to maintain the servers, electricity, air conditioning, training your staff to have the latest skills, and so on. Also, some of the requirements here can be removed as well such as people required to setup and maintain the servers, electricity, air conditioning is covered by AWS in the data centers.

P.S.: The rent and subscription you pay AWS to use their service is considered as operating expenses.

Now, let's discuss on ways to reduce costs in AWS.

Ways to Reduce Costs in AWS

Right-sized infrastructure: Only use the infrastructure you need like the right number of machines, don't just spin up a bunch of services. This is important to note before you gets charge uncontrollably 😭.
Automation: Schedule start and stop time for servers that you use for development. You can also automate your billing and send notification on spend limit to save money .
Compliance scope: Adhere to regulations based on your industry.
Managed services: AWS provisions the underlying infrastructure, handles patching, updates, and so on allowing you to use the service without worrying about creating servers, installing software and licensing costs.

Conclusion

Understanding the concepts and fundamentals are important in guiding how you approach AWS to build smarter and scalable apps. AWS is not a service that is overwhelming to use because it gives you access to only the tools you need at any given time with a structure to monitor and pay for what you use.

Resources

SQL fundamentals and databases

oteri — Mon, 17 Feb 2025 09:47:39 +0000

No company today exists without data.

From the big companies like Netflix, Google, Instagram, and so on have a repository where their data are stored in databases and it powers every app we use daily.

What is SQL?

Let's simply put it this way, structured query language (SQL) is a programming language used to talk to databases which is a declarative language using SQL statement.

A typical example of a statement is as follows:

SELECT NAME
FROM USERS
WHERE ROLE = 'MANAGER';

Want to try out some SQL? Check out this exercise on DB Fiddle, an online SQL playground to practice.

p.s.: SQL is a standardized language meaning it has its set of rules

How to use SELECT

As previously stated, if you want to get the data from a database, the SELECT keyword is your go to.

To get the rows and columns, use the wildcard (*):

SELECT * FROM employees;

 emp_no | birth_date |   first_name   |    last_name     | gender | hire_date  
--------+------------+----------------+------------------+--------+------------
  10001 | 1953-09-02 | Georgi         | Facello          | M      | 1986-06-26
  10002 | 1964-06-02 | Bezalel        | Simmel           | F      | 1985-11-21
  10003 | 1959-12-03 | Parto          | Bamford          | M      | 1986-08-28
  10004 | 1954-05-01 | Chirstian      | Koblick          | M      | 1986-12-01
  10005 | 1955-01-21 | Kyoichi        | Maliniak         | M      | 1989-09-12

Get only two columns from the table, employees:

SELECT emp_no, firstname
FROM employees
LIMIT 5;

This statement returns the list of only 5 employees from the table with the LIMIT keyword.

 emp_no |   first_name   
--------+----------------
  10001 | Georgi
  10002 | Bezalel
  10003 | Parto
  10004 | Chirstian
  10005 | Kyoichi

What is a database?

Before databases, there were file processing systems (FPS). One disadvantage of using this system is you can't keep the data updated across multiple files.

Databases on the other hand with its pictorial representation as drums, is a collection of data, a method of accessing and manipulating that data. And to manage the database, a software or program (DBMS) is required like PostgreSQL making it a fun experience to produce statements to view the data as tables.

To discover more of the SQL fundamentals, I shared my thoughts in this post below like database models, tables and components, and the differences between OLTP vs OLAP:

'teri eyenike on LinkedIn: 🎯 Discover the language, SQL that powers databases

Is SQL still popular today? Lately, I have been revisiting and improving on my SQL knowledge which is an interesting query language to use. I will walk you…

linkedin.com

Boost Your Productivity with Momentum Builder: A Web App to Overcome Procrastination and Track Progress

oteri — Wed, 15 Jan 2025 21:34:31 +0000

This is a submission for the GitHub Copilot Challenge : New Beginnings

What I Built

The momentum builder app's concept uses the Pomodoro technique. This technique lets you engage in deep work and remain focused. At the end of your work, it alerts you to take a break. In addition, the application keeps track of your tasks, allows you to overcome procrastination, and provides a motivational quote.

Demo

The application is hosted on Vercel which is available via this public URL. Here is a screenshot of the demo app which uses local storage to persist the stored data.

Repo

Want to see the code? The project repository is in GitHub which contains the HTML, CSS, and JavaScript scripts. Try it out yourself.

Terieyenike / momentum_builder

Momentum Builder is a web application designed to help users overcome procrastination and provide motivational nudges. The app encourages consistency by tracking streaks and displaying progress reminders.

Momentum Builder

Momentum Builder is a web application designed to help users overcome procrastination and provide motivational nudges. The app encourages consistency by tracking streaks and displaying progress reminders.

Features

Add tasks
Set timers for tasks
Receive motivational quotes and progress reminders
Track and display streaks to encourage consistency
Display error messages when a timer is already running for another task
Automatically clear error messages and input fields after a few seconds

Getting Started

Prerequisites

To run this application, you need a web browser that supports HTML, CSS, and JavaScript.

Installation

Clone the repository or download the source code.
Open the index.html file in your web browser.

Usage

Enter a task in the input field and click the "Add Task" button.
For each task, you can add micro-tasks by entering a description and clicking the "Add Micro-Task" button.
Set a timer for each task or micro-task by entering the number…

View on GitHub

Copilot Experience

Since I have some experience using prompts to query the LLM capability to produce resulting code and developed a Tic-Tac-Toe game with Python in the past, using the embedded model was not strange to me.

Prompts: My prompts centered around tweaking the responses for the code and I ensured it aligned with the end goal while checking the functionality of the app on the live server so it meets my expectation
Edits: GitHub Copilot did an awesome job suggesting possible solutions and making meaningful edits appropriately and timely. The model intelligence remembering previous input and generating code was a breeze. There is no doubt of its efficiency.
Chat: I found the chat feature to be a lifesaver for improving the codebase. Deleting the previous result and starting on a clean slate made it superb
Model switcher: I tried and switched between the models to experiment with different results.
Voice: This feature helped me to ask for the possible article title idea for this blog

GitHub Models

For context, the models gave me a quicker way to speed up my development process in creating the workflow from the header to the footer. The model handled the responsiveness, functionality, and the app's building blocks well enough.

Conclusion

The brainstorming capacity of GitHub Copilot helps you debug, explain parts of your code that you find tricky, and offer solutions to fix which makes it a great buddy assistant to rely on. I found this task of building the momentum builder app exciting as Copilot intelligently suggested and pointed out several ways to improve the codebase, and also not without the additional fixes by a human to make sure it meets the standard of a fully developed and deployable app.

Convert Emojis to Text in SMS with Infobip: A Step-by-Step Guide

oteri — Tue, 17 Dec 2024 16:30:07 +0000

Have you found it difficult to understand many emojis used today in tweets, LinkedIn posts, and others?

In this guide, you’ll learn how to decode and convert these emojis to text in defined strings, and with the help of Infobip SMS API and Python SDK, you can send all the information as a text message to your phone.

Let's get started!

Prerequisites

An Infobip account (a free trial works fine)
Knowledge of Python
Python on your local machine

Setup your Environment

Since this tutorial uses the Python CLI to run the script, install the Infobip SDK with the command:

pip install infobip-api-python-sdk

Now, create a folder, emoji and add the following files:

.env: Your environment variables from your admin dashboard
app.py: All the scripts will be here

Environment variables

The .env file helps hide all sensitive information in public repos on GitHub, preventing you from sharing your secret keys and variables with the public.

Copy-paste these variables:

IB_BASE_URL="<your API Base URL>"
IB_API_KEY="<your API Key>"
DESTINATION_NUMBER="<your phone number>"

Replace the variables for the keys with your actual values. Log in to your dashboard to access your details.

Creating the Emoji Decoder App

The demoji module or package finds emojis within a string and converts these emoticons or emojis into text. Run this command to install the Python module.

pip install demoji

In the app.py file, import the necessary module and the Infobip SMS channel.

import demoji
from infobip_channels.sms.channel import SMSChannel
import os

Next, below the import, include your desired text with as many or one emoji that the demoji module will help convert to text.

text = """
I am launching this exciting project 🚀🔥
which I have been building in public for 
some months 💻💪🇳🇬. 
"""

The text variable is wrapped around in docstrings that allow multiline characters denoted with the triple quote mark.

Prepare the SMS Payload

The DESTINATION_NUMBER is the number you want to send SMS messages. If you’re using an Infobip trial account, the number should have been verified with your account.

The Python function send_sms_from_app invokes the SMS response creating an instance of the SMSChannel class, containing all the methods. The instance SMSChannel.from_env() will read the set variables in the .env file.

def send_sms_from_app(text):
   channel = SMSChannel.from_env()
   sms_response = channel.send_sms_message({
       'messages': [{
           'from': 'Know meanings of emojis',
           'text': text,
           'destinations': [{
               'to': os.environ['DESTINATION_NUMBER']
           }],
       }]
   })
   print(sms_response)

Send the SMS and Test the Script

Before sending the SMS, we need to convert the resulting output from a dictionary to a formatted string with the emoji and their descriptions to make it meaningful in SMS format.

{'🚀': 'rocket', '🔥': 'fire', '🇳🇬': 'flag: Nigeria', '💻': 'laptop', '💪': 'flexed biceps'}

In addition, the findall method with the text passed as an argument finds the emojis in the text.

emoji_dict = demoji.findall(text)
result_text = '\n'.join([f'{emoji}: {description}' for emoji, description in emoji_dict.items()])
send_sms_from_app(result_text)

The result_text variable loops through the key-value pair in the emoji_dict output and joins the formatted string with a newline character using the escape character \n.

Finally, the send_sms_from_app function sends the processed text via SMS.

To put this script to test, in your terminal, run the command:

python app.py

Learn More

Explore this guide to learn how to use the Infobip SMS Channel to receive important updates such as reports directly on your phone.

Check out this project you can use your voice to transcribe and translate into text.

oteri — Fri, 29 Nov 2024 10:39:06 +0000

Building a Voice Transcription and Translation App with OpenAI Whisper and Streamlit

oteri ・ Nov 29 '24

#python #streamlit #openai #ai

Building an AI Voice Application in Streamlit using OpenAI

oteri — Fri, 29 Nov 2024 10:37:43 +0000

Every digital device like the smartphones, computers, tablets, and more come with an in-built default voice recording application which makes it useful for many use cases. This technology have evolved over the years with its integration that powers virtual assistants like Siri, Alexa, and Google Assistant because of its convenience and hands-free accessibility.

As we know it, the instructions for effective communication require voice commands only and a playback technology to create outstanding results with accurate automated transcription.

In this guide, we will showcase the power of AI in turning speech to text and at the same time, translate another audio recording from a different language into English. All using the OpenAI technology and its model called Whisper. Also, you would be able to download the transcribed content as a text file in the .txt format.

Want to try it live? Check out the Your Voice app and see the magic from speech to text.

Prerequisites

Basic Python knowledge
Streamlit
OpenAI API key. Sign up for an account

What is Whisper

Whisper is a trained open-source neural network that approaches human-level robustness and accuracy in English speech recognition.

The OpenAI API provides two endpoints:

Transcriptions
Translations

What is Streamlit

From the official website, Streamlit is a faster way to build and share data apps. It is an open-source Python library that helps you build web applications for sharing analytical results, building complex interactive experiences, and iterating on top of new machine-learning models.

Streamlit is a top choice for Python developers because it has built-in and convenient methods, from taking in user inputs like text, numbers, and dates to showing interactive graphs using the most popular and powerful Python graphing libraries.

Installing Streamlit

To run any Streamlit apps, you must first install Streamlit using the command:

pip install streamlit

Installing Other Libraries

Since we are working with transforming audio to text, we need to store our environment variables securely.

pip install openai python-dotenv

Creating the Environment Variable

Create a new file in the root project directory and name it .env.

Paste in your OpenAI API key:

.env

OPENAI_API_KEY="sk-..."

Creating the App

In your directory, create this file, streamlit_app.py which will contain all the Python code to transcribe and translate our audio and output the resulting text.

To initialize an instance of the OpenAI client, copy-paste this code:

streamlit_app.py

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

api_key = os.getenv('OPENAI_API_KEY')

client = OpenAI()

The code block connects and reads our secret key in the .env file, making sure we are authenticated as users.

PS: Using the OpenAI API is not free as you have to buy some credits to use the service.

Transcription with Whisper

Let's update the streamlit_app.py with the following:

streamlit_app.py

...
import streamlit as st

st.logo(
  "logo.png",
  size="medium",
  link="https://platform.openai.com/docs",
)

st.title("Transcription with Whisper")

audio_value = st.audio_input("record a voice message to transcribe")

if audio_value:
  transcript = client.audio.transcriptions.create(
    model="whisper-1",
    file = audio_value
  )

  transcript_text = transcript.text
  st.write(transcript_text)

The transcriptions API will convert our audio using the st.audio_input widget to record our voice. If the recording exists, the model, Whisper is used to create the desired file format for the transcription of the audio and output the text using the st.write() function which takes a string and writes it directly into our web app.

To use the exact logo at the top left of the app, download this and save it in your project directory.

Now, let's run this app with this command in the terminal:

streamlit run streamlit_app.py

Downloading the Transcribed Text

The ability to download the transcribed message for later would serve for record-keeping when you need it.

Streamlit offers an input widget that allows for the display of a download button. Back to the streamlit_app.py file, update the codebase with the following:

streamlit_app.py

...

  txt_file = "transcription.txt"

  # Initialize session state for download confirmation
  if "downloaded" not in st.session_state:
        st.session_state.downloaded = False

    # Download button
  if st.download_button(
       label="Download Transcription",
       file_name="transcription.txt",data=transcript_text,
  ):
       st.session_state.downloaded = True

    # Show success message after download
  if st.session_state.downloaded:
        st.success("Transcription file downloaded successfully!")

The following occurs in the lines of code above:

st.session_state in Streamlit allows you to share variables between reruns, for each user session
The transcript_text variable will contain the content of the transcribed text
The txt_file variable with the assigned value, transcription.txt is the file name of the transcribed text when the file is downloaded.
Within the function of the st.download_button(), the label describes to the user what the button is for.

The st.success status element displays a success message when the file is saved as shown:

Translation with Whisper

The process of creating the translation is similar to creating the transcription. The translation endpoint will translate a foreign language into written text in English from the input of the audio file.

Copy and paste this code.

streamlit_app.py

...

st.header("Translation with Whisper", divider="gray")

audio_translate = st.audio_input("record a voice message to translate")

if audio_translate:
  translate = client.audio.translations.create(
    model="whisper-1",
    file=audio_translate
  )

  st.write(translate.text)

If you want to create a file to save your translated audio file as text, you can do the same as with the Download transcription button.

The complete source code is in this repository.

Good luck!

Conclusion

Instead of using pre-recorded audio from the internet as seen in the OpenAI docs, Streamlit offers you the opportunity to use your voice and pair it with the transcription and translation endpoints provided by OpenAI to create this outstanding project.

The microphone in your device can do so much as technology has made it possible to go beyond using it for communication during meetings and calls.

Building a Console-Based Calculator in C#

oteri — Mon, 11 Nov 2024 10:58:23 +0000

Learning something new like a programming language comes with a project to solidify your understanding of the underlying concepts and fundamentals. To know something deeply, do not rush through the fundamentals as it leaves gaps in your knowledge.

This guide shows you how to build a console-based calculator in C# and interact with the interface in your terminal.

Prerequisites

As a first-timer to C#, I expect you to know how to declare variables and use the various data types. As you know, C# supports strong type checking and it is a simple, modern, general-purpose, object-oriented programming language.

Some of the important types include:

bool
int
long
float
double
char

For the console-based calculator, these are some of the features that will make a working app:

Ability to ask the user for two numbers and a calculation method
Valid calculation methods like addition, exponent, modulo, multiplication, division, and subtraction
Calculate the result and print it to the console after input from the user
After the result is printed, ask the user for another calculation. The program ends when the user decides by typing "no"

Let's begin.

Creating the Calculator App

In your Program.cs file within the directory, copy-paste this code snippet replacing the Hello World script.

Console.WriteLine("Console bassed calculator\n");
Console.WriteLine("========================");

bool continueCalculating = true;

while (continueCalculating)
{
  Console.WriteLine("Enter your first number: ");
  string? firstInput = Console.ReadLine();
  if (firstInput == null)
  {
    Console.WriteLine("First input cannot be null.");
    return;
  }
  double number1 = double.Parse(firstInput);

  Console.WriteLine("Enter the second number: ");
  string? secondInput = Console.ReadLine();
  if (secondInput == null)
  {
    Console.WriteLine("Second input cannot be null.");
    return;
  }
  double number2 = double.Parse(secondInput);

  Console.WriteLine("Choose an operation (multiply, divide, subtract, add, exponent, modulo) ");
  string? operation = Console.ReadLine().ToLower();

  switch (operation)
  {
    case "multiply":
      Console.WriteLine($"Result: {Math.Round(number1 * number2, 2)}");
      break;
    case "divide":
      if (number2 != 0)
        Console.WriteLine($"Result: {Math.Round(number1 / number2, 2)}");
      else
        Console.WriteLine("Error: Division by zero is not allowed.");
      break;
    case "subtract":
      Console.WriteLine($"Result: {Math.Round(number1 - number2, 2)}");
      break;
    case "add":
      Console.WriteLine($"Result: {Math.Round(number1 + number2, 2)}");
      break;
    case "exponent":
      Console.WriteLine($"Result: {Math.Round(Math.Pow(number1, number2), 2)}");
      break;
    case "modulo":
      if (number2 != 0)
      {
        Console.WriteLine($"Result: {number1 % number2}");
      }
      else
      {
        Console.WriteLine("Error: Modulo by zero is not allowed.");
      }
      break;
    default:
      Console.WriteLine("Error: Unsupported operator. Please use 'multiply', 'divide', 'subtract', or 'add'.");
      break;
  }
  Console.WriteLine("Would you like to perform another calculation? (yes to continue, no to exit): ");
  string? response = Console.ReadLine()?.Trim().ToLower();

  if (response != "yes")
  {
    Console.WriteLine("Thank you for using the calculator. Goodbye!");
    break;
  }
}

The block of code above ensures the result from this app gives the accurate result of any calculation from the user and it also makes use of the Switch statements.

Running the program

Use the command, dotnet run in your terminal or command prompt.

You can find the complete source code in this GitHub Gist.

Share your thoughts on what you think about this project.

Getting Started with C# and .NET Development: A Beginner's Guide

oteri — Sat, 09 Nov 2024 15:04:23 +0000

It is interesting to begin learning C# and .NET after never considering coming close to it as a software engineer. From my standpoint, I will share my thoughts from a series of tutorials.

I am delighted to embark on this journey from zero to hero to master the concepts and become proficient in using this tool to build outstanding applications.

What is C#?

C# is a statically typed programming language that provides type safety, meaning the compiler helps you prevent bugs before you even run your program or app.

According to research on the top 10 most in-demand programming languages in 2024, C# ranks 4 behind Python, JavaScript, and Java.

What is .NET?

.NET is an open-source developer platform created by Microsoft for building many different types of cross-platform applications. One major driving force of .NET is that it supports three programming languages, C#, F#, and Visual Basic (VB) .NET.

Some use cases and why you should learn C# and .NET development include the following:

Desktop applications
Mobile apps
Web development
Game development
Internet of Things (IoT)
Machine Learning
Cloud-First development
Console applications

Setup and Installation

You only need a text editor like VS Code, Cursor, or any other you are comfortable using for setup. Some of the extensions that are worthy to use are:

After the necessary installation, run the following command in your terminal to check for the presence of the SDKs and the runtimes:

dotnet --list-sdks

dotnet --list-runtimes

Creating your first C# Console Application

Create a new directory to contain the boilerplate code and run this command within the directory:

dotnet new console

You should have something similar to this:

.
├── Program.cs
├── obj
│   ├── project.assets.json
│   ├── project.nuget.cache
│   ├── try-dotnet.csproj.nuget.dgspec.json
│   ├── try-dotnet.csproj.nuget.g.props
│   └── try-dotnet.csproj.nuget.g.targets
└── try-dotnet.csproj

To test and run the program, use this command in your terminal:

dotnet run