Forem: Ethan

Creating an AI Discord Bot with Ollama

Ethan — Sun, 14 Dec 2025 08:09:46 +0000

Hello! 😎
In this tutorial I will teach you how to create your own AI-powered Discord bot from scratch. By the end, you'll have a bot that can:

Respond to messages using AI
Have conversations with users

No prior experience is needed but it does help to have some basic knowledge about Python.

What you will learn

How to create a Discord bot application
How to install and use Ollama (Fully local)
How to write Python code to connect them
How to make your bot talk like a real person
How to run the bot 24/7

What you will need

Python 3.8+ - Programming language
Discord Account - To create the bot
Ollama - Free local AI
Text Editor - And IDE you like

Part 1: Setting up your Discord bot

First we need to create the Discord bot at the Discord developers page:

https://discord.com/developers/applications

Next navigate to the Bot tab and scroll down to the privileged gateway intents and enable the following, this step is important.

Presence Intent
Server Members Intent
Message Content Intent

When enabled save all the changes, this will allow your bot to see messages and user info.

Next we need the token scroll back up and click on Reset Token and copy it to your clipboard and save it somewhere where only you can see.

Next we need to invite the bot to the server, this can be done via the sidebar click on OAuth2.

We need to allow scopes and permissions for scopes check the following two:

bot
applications.commands

For permissions select the following:

Send Messages
Send Messages in Threads
Read Message History
View Channels
Use Slash Commands
Add Reactions

Once done copy and access the generated url, select the server you want the bot to run in and accept. With this the bot should now be in the Discord server as offline.

Next lets install Ollama!

Part 2: Installing Ollama

Ollama is free software that runs AI models on your own computer. The benefits to this are:

Free - No API costs/tokens
Private - Runs locally, no data send to servers
Fast - Responses in seconds
Offline - Works without internet (after downloading models)

Download Ollama based on your machine.

Mac:

brew install ollama

Linux:

curl -fsSL https://ollama.ai/install.sh | sh

Windows:
Download the exe from https://ollama.ai/download

Next start Ollama:

ollama serve

The terminal should show that ollama is running. Keep this terminal open, the bot will need this to work.

Next we need to download an AI Model, for this example im using a simple model but feel free to use anything you want.

ollama pull llama3.1

This downloads the LLaMA 3.1 model it's:

Free and open source
Great for conversations
Fast responses
Good at following instructions

Wait for the download to complete, which takes around a couple of minutes.

Lastly lets see if the model is installed correctly.

ollama run llama3.1

You should see output, feel free to try conversing with it, next we will finally be getting into the coding part.

Part 3: Installing Python Dependencies

First create a virtual environment and activate it.

python3 -m venv env
source env/bin/activate

Next install the following dependencies:

pip3 install py-cord
pip3 install requests
pip3 install python-dotenv

py-cord - Discord bot library
requests - Makes HTTP requests to talk to Ollama
python-dotenv - Loads environment variables, ie the Discord token

Part 4: Writing Your Bot Code

Setup the directories for the project and create a new .env file to store the token:

vim .env

Add this line replace the token with your actual token.

DISCORD_TOKEN="token"

Done now we can finally start coding the bot.

Part 5: The Complete Bot Code

Create a new file called bot.py and fill it with the following content, it will be explained later.

# ============================================
# IMPORTS - Load the libraries we need
# ============================================
import discord
from discord.ext import commands
import requests
import json
import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# ============================================
# CONFIGURATION
# ============================================

# Get bot token from .env file
TOKEN = os.getenv("DISCORD_TOKEN")

# Ollama API endpoint (where AI is running)
OLLAMA_URL = "http://localhost:11434/api/generate"

# Which AI model to use
OLLAMA_MODEL = "llama3.1"

# Bot command prefix (e.g., !chat hello)
PREFIX = "!"

# ============================================
# BOT SETUP
# ============================================

# Create bot with all permissions
intents = discord.Intents.all()
bot = commands.Bot(command_prefix=PREFIX, intents=intents)

# ============================================
# HELPER FUNCTION: Talk to Ollama AI
# ============================================

def ask_ai(prompt, temperature=0.9):
    """
    Send a prompt to Ollama and get AI response.

    Args:
        prompt (str): What to ask the AI
        temperature (float): Creativity level (0.0-1.0)
            - 0.0 = Very predictable, factual
            - 0.5 = Balanced
            - 1.0 = Very creative, random

    Returns:
        str: AI's response
    """
    try:
        # Prepare the request
        payload = {
            "model": OLLAMA_MODEL,
            "prompt": prompt,
            "stream": False,  # Get complete response at once
            "options": {
                "temperature": temperature,
                "num_predict": 200  # Max response length in tokens
            }
        }

        # Send request to Ollama
        response = requests.post(OLLAMA_URL, json=payload, timeout=30)

        # Check if successful
        if response.status_code == 200:
            data = response.json()
            return data.get("response", "").strip()
        else:
            return "Sorry, I had trouble thinking of a response!"

    except requests.exceptions.ConnectionError:
        return "Error: Ollama is not running! Start it with 'ollama serve'"
    except Exception as e:
        return f"Error: {str(e)}"

# ============================================
# EVENT: Bot is Ready
# ============================================

@bot.event
async def on_ready():
    """Called when bot successfully connects to Discord"""
    print(f"✅ Bot is online as {bot.user.name}")
    print(f"✅ Connected to {len(bot.guilds)} server(s)")
    print(f"✅ Prefix: {PREFIX}")
    print("Ready to chat!")

# ============================================
# EVENT: Message Received
# ============================================

@bot.event
async def on_message(message):
    """Called whenever a message is sent in a channel the bot can see"""

    # Don't respond to yourself (prevents infinite loops!)
    if message.author == bot.user:
        return

    # Don't respond to other bots
    if message.author.bot:
        return

    # Check if bot was mentioned
    if bot.user.mentioned_in(message):
        # Remove the mention from the message
        user_message = message.content.replace(f'<@{bot.user.id}>', '').strip()

        # If they just mentioned with no text
        if not user_message:
            user_message = "Hello!"

        # Show typing indicator (looks more natural)
        async with message.channel.typing():
            # Ask AI for response
            ai_response = ask_ai(user_message)

            # Send response
            await message.channel.send(ai_response)

    # Process commands (important for !commands to work)
    await bot.process_commands(message)

# ============================================
# COMMAND: !chat
# ============================================

@bot.command(name="chat")
async def chat_command(ctx, *, message: str = None):
    """
    Chat with the AI bot.
    Usage: !chat How are you today?
    """
    if not message:
        await ctx.send("Please provide a message! Example: `!chat Hello!`")
        return

    # Show typing indicator
    async with ctx.typing():
        # Get AI response
        response = ask_ai(message)

        # Send it
        await ctx.send(response)

# ============================================
# COMMAND: !help
# ============================================

@bot.command(name="help")
async def help_command(ctx):
    """Show available commands"""

    help_text = """
**🤖 AI Bot Commands**

`!chat <message>` - Chat with the AI
Example: `!chat What's your favorite color?`

`@Bot <message>` - Mention the bot to talk
Example: `@MyBot Tell me a joke!`

`!help` - Show this help message

**How it works:**
This bot uses Ollama (local AI) to generate responses. The AI runs on the computer hosting the bot, so all conversations are private!
    """

    await ctx.send(help_text)

# ============================================
# START THE BOT
# ============================================

if __name__ == "__main__":
    print("Starting bot...")
    print(f"Ollama URL: {OLLAMA_URL}")
    print(f"Model: {OLLAMA_MODEL}")

    # Run the bot (this blocks until bot stops)
    bot.run(TOKEN)

Lets break down what each part does.

import discord
from discord.ext import commands
import requests

The imports import the libraries that we need.

TOKEN = os.getenv("DISCORD_TOKEN")
OLLAMA_URL = "http://localhost:11434/api/generate"

Configs the Discord token and the Ollama endpoint url.

def ask_ai(prompt, temperature=0.9):
    payload = {
        "model": OLLAMA_MODEL,
        "prompt": prompt,
        "stream": False,
        "options": {
            "temperature": temperature,
            "num_predict": 200
        }
    }

    response = requests.post(OLLAMA_URL, json=payload, timeout=30)

    if response.status_code == 200:
        data = response.json()
        return data.get("response", "").strip()
    else:
        return "Sorry, I had trouble thinking of a response!"

This ask_ai function takes your message (prompt), sends it to Ollama, waits for AI to generate a response and then returns the response as text.

Now for the parameters:

temperature - How creative the AI is (0.0 = boring, 1.0 = wild)
num_predict - Maximum length of response (in tokens, roughly words)
stream - Set to false to wait for the complete response (not word-by-word)

Next i will explain what each event does.

@bot.event
async def on_ready():
    print(f"✅ Bot is online as {bot.user.name}")

Nothing too hard, this gets fired when the bot successfully connects to Discord and prints a confirmation message.

@bot.event
async def on_message(message):
    if message.author == bot.user:
        return  # Don't respond to yourself!

    if bot.user.mentioned_in(message):
        user_message = message.content
        ai_response = ask_ai(user_message)
        await message.channel.send(ai_response)

What this does is:

Triggers every time someone sends a message
Checks if bot was mentioned
If yes, extracts the message
Asks AI for response
Sends AI's response back

The bot should not reply to itself unless you want an infinite loop of spam.

Next I will explain the command.

@bot.command(name="chat")
async def chat_command(ctx, *, message: str = None):
    async with ctx.typing():  # Show "Bot is typing..."
        response = ask_ai(message)
        await ctx.send(response)

What this does it:

Creates a command: !chat your message here
Shows typing indicator (makes it feel more human)
Gets AI response
Sends it back

bot.run(TOKEN)

Finally the above just starts the bot, connects to Discord using the token and keeps the bot running until you turn it off.

Part 7: Running Your Bot

First make sure Ollama is running with the previous command and then start the bot.

python3 bot.py

You should see the following console:

Starting bot...
Ollama URL: http://localhost:11434/api/generate
Model: llama3.1
✅ Bot is online as My AI Bot
✅ Connected to 1 server(s)
✅ Prefix: !
Ready to chat!

Feel free to test the bot in Discord. Try mentioning it, it should respond. Feel free to try the !chat command also.

Done you have now created a new Discord bot!

Part 8: Customizing Your Bot

You can customize various things with your bot such as its personality.
You can add a personality to the bot via a system prompt to make more personal.

If you want to increase the response length just change num_predict to a higher number.

You can also change the temperature to increase the creativity of the bot.

Other feature you can also add are things like:

Respond to DMs
Remember Context
Random Reactions

If you would like I can create tutorials for all the above.

Part 10: Running Your Bot 24/7

You could keep your local computer running which is free and gives you full control. This will however need your machine to stay on 24/7 and may strain your electric bill.

If you aren't able to run your machine 24/7 I recommend you run it on a cloud server using Digital Ocean, AWS, or Oracle Cloud which has a free tier.

Feel free to pick the best option that works for you.

Part 11: Next Steps

Next you can make the bot even more advanced, via the following implementations. I am also thinking of doing more tutorials expanding on the following topics.

Beginner Projects

Add more commands
Make it respond to specific words
Add personality

Intermediate Projects

Database integration - Remember user preferences
Multiple AI Models - Switch between models for different tasks
Image Generation - Use Stable Diffusion with Ollama
Voice Channel Support - Make the bot speak with people in voice channels

Advanced Projects

Multi-server bot - Deploy to many servers
Web Dashboard - See stats and control the bot from the browser
Custom AI Model - Train your own model with Ollama
Bot Network - Multiple bots that interact with each other

Have fun coding.

Conclusion

Well done you've built your very own AI powered Discord Bot.

You now know how to:

Create Discord bot applications
Use Ollama for local AI
Connect Python code to both services
Handle commands and messages
Deploy your bot

I am planning to expand this series with more features such as mixing in Text to Speech to allow the bot to converse with users in the voice channel.

Side note: for production usage it's best to limit the bots intents to only the intents it needs.

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

Flask Video Streaming App Tutorial

Ethan — Mon, 03 Mar 2025 06:38:12 +0000

Introduction

Hello! 😎
In this tutorial I will show you how to build a video streaming application using Flask and FFmpeg.

Overview

The application is designed as a Flask web server that manages video uploads, processes them using FFmpeg, and server adaptive streaming content to the end user. Adaptive streaming formats such as HLS (HTTP Live Streaming) and DASH (Dynamic Adaptive Streaming over HTTP) are essential for providing a smooth playback experience across varying network conditions.

File Uploading: Securely handling video file uploads, verifying file types and enforcing size limits.
Video Conversion: Converting uploaded videos to HLS and DASH formats using FFmpeg.
Streaming Endpoints: Providing endpoints that serve video streams to clients.
Modern UI: A responsive front-end that enables drag-and-drop file uploads, video listing and a dynamic video player.

Now that we have an overview of the application, let's start building the backend.🤓

Building The Back End With Flask

Flask is chosen for its lightweight nature and simplicity. The code leverages several libraries including os, uuid, subprocess, logging and extensions such as Flask-Cors for handling cross origin.

First, we need to create a Python virtual environment. This can be done and activated with the following command:

python3 -m venv env && source env/bin/activate

Next create a new file called main.py and populate it with the required imports:

import os
import uuid
import subprocess
import logging
from flask import Flask, request, jsonify, send_from_directory, render_template
from werkzeug.utils import secure_filename
from flask_cors import CORS

Next we need to configure some variables for the directories and logging etc. Next add the following Python code:

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

app = Flask(__name__)
CORS(app)  # Enable Cross-Origin Resource Sharing

# Configuration
UPLOAD_FOLDER = 'uploads'
OUTPUT_FOLDER = 'streams'
ALLOWED_EXTENSIONS = {'mp4', 'avi', 'mov', 'mkv', 'wmv', 'flv', 'webm'}
MAX_CONTENT_LENGTH = 100 * 1024 * 1024  # 100MB max upload size

# Create necessary directories
os.makedirs(UPLOAD_FOLDER, exist_ok=True)
os.makedirs(OUTPUT_FOLDER, exist_ok=True)

app.config['UPLOAD_FOLDER'] = UPLOAD_FOLDER
app.config['OUTPUT_FOLDER'] = OUTPUT_FOLDER
app.config['MAX_CONTENT_LENGTH'] = MAX_CONTENT_LENGTH

Here we initialize the app and logging etc.

Next we need a helper function to ensure the uploaded file does not exceed 100MB and is a valid video format:

def allowed_file(filename):
    return '.' in filename and filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

After that we will write a function that handles the conversion to HLS:

def convert_to_hls(input_path, output_dir):
    """Convert video to HLS format using FFmpeg"""
    os.makedirs(output_dir, exist_ok=True)

    hls_playlist = os.path.join(output_dir, 'playlist.m3u8')

    # HLS conversion command
    hls_cmd = [
        'ffmpeg', '-i', input_path,
        '-profile:v', 'baseline',
        '-level', '3.0',
        '-start_number', '0',
        '-hls_time', '10',
        '-hls_list_size', '0',
        '-f', 'hls',
        hls_playlist
    ]

    try:
        subprocess.run(hls_cmd, check=True)
        logger.info(f"HLS conversion completed for {input_path}")
        return True
    except subprocess.CalledProcessError as e:
        logger.error(f"HLS conversion failed: {e}")
        return False

Now that we have a function to handle HLS we next need to handle the DASH side, below the above function add the following function:

def convert_to_dash(input_path, output_dir):
    """Convert video to DASH format using FFmpeg"""
    os.makedirs(output_dir, exist_ok=True)

    dash_playlist = os.path.join(output_dir, 'manifest.mpd')

    # DASH conversion command
    dash_cmd = [
        'ffmpeg', '-i', input_path,
        '-map', '0:v', '-map', '0:a',
        '-c:v', 'libx264', '-x264-params', 'keyint=60:min-keyint=60:no-scenecut=1',
        '-b:v:0', '1500k',
        '-c:a', 'aac', '-b:a', '128k',
        '-bf', '1', '-keyint_min', '60',
        '-g', '60', '-sc_threshold', '0',
        '-f', 'dash',
        '-use_template', '1', '-use_timeline', '1',
        '-init_seg_name', 'init-$RepresentationID$.m4s',
        '-media_seg_name', 'chunk-$RepresentationID$-$Number%05d$.m4s',
        '-adaptation_sets', 'id=0,streams=v id=1,streams=a',
        dash_playlist
    ]

    try:
        subprocess.run(dash_cmd, check=True)
        logger.info(f"DASH conversion completed for {input_path}")
        return True
    except subprocess.CalledProcessError as e:
        logger.error(f"DASH conversion failed: {e}")
        return False

Feel free to change any of the variables. Now that we've handled both HLS and DASH we can now start programming the API routes.

The first route is an index route that returns the index page:

@app.route('/')
def index():
    return render_template('index.html')

After that we also need to handle a route for the user to upload a video file:

@app.route('/upload', methods=['POST'])
def upload_file():
    # Check if the post request has the file part
    if 'file' not in request.files:
        return jsonify({'error': 'No file part'}), 400

    file = request.files['file']

    # If user does not select file, browser also
    # submit an empty part without filename
    if file.filename == '':
        return jsonify({'error': 'No selected file'}), 400

    if file and allowed_file(file.filename):
        # Generate a unique ID for this video
        video_id = str(uuid.uuid4())

        # Create directories for this video
        video_upload_dir = os.path.join(app.config['UPLOAD_FOLDER'], video_id)
        os.makedirs(video_upload_dir, exist_ok=True)

        stream_output_dir = os.path.join(app.config['OUTPUT_FOLDER'], video_id)
        os.makedirs(stream_output_dir, exist_ok=True)

        # Save the original file
        filename = secure_filename(file.filename)
        file_path = os.path.join(video_upload_dir, filename)
        file.save(file_path)

        logger.info(f"File uploaded: {file_path}")

        # Create directories for each format
        hls_output_dir = os.path.join(stream_output_dir, 'hls')
        dash_output_dir = os.path.join(stream_output_dir, 'dash')

        # Process video asynchronously
        def process_video():
            # Convert to HLS
            hls_result = convert_to_hls(file_path, hls_output_dir)

            # Convert to DASH
            dash_result = convert_to_dash(file_path, dash_output_dir)

            return hls_result and dash_result

        success = process_video()

        if success:
            return jsonify({
                'id': video_id,
                'status': 'success',
                'hls_url': f'/stream/{video_id}/hls/playlist.m3u8',
                'dash_url': f'/stream/{video_id}/dash/manifest.mpd',
                'player_url': f'/player/{video_id}'
            })
        else:
            return jsonify({'error': 'Conversion failed'}), 500

    return jsonify({'error': 'File type not allowed'}), 400

The above route c # For simplicity, we'll process synchronously in this example
# In production, use a task queue like Celeryhecks that the file is valid and if so converts it to both HLS and DASH, if ok it returns both the HLS and DASH stream information.

Next we will create an endpoint for the video stream:

@app.route('/stream/<video_id>/<format_type>/<path:filename>')
def stream_file(video_id, format_type, filename):
    """Serve the video stream files"""
    directory = os.path.join(app.config['OUTPUT_FOLDER'], video_id, format_type)
    return send_from_directory(directory, filename)

After that we will next create an endpoint that allows for the video to be played:

@app.route('/player/<video_id>')
def player(video_id):
    """Render the video player page"""
    # Make sure we're explicitly passing video_id to the template
    hls_url = f'/stream/{video_id}/hls/playlist.m3u8'
    dash_url = f'/stream/{video_id}/dash/manifest.mpd'
    return render_template('player.html', video_id=video_id, hls_url=hls_url, dash_url=dash_url)

Lastly we will create an endpoint that allows the users to view all video files available:

@app.route('/videos')
def video_list():
    """List all available videos"""
    videos = []

    # Get all subdirectories in the streams folder
    for video_id in os.listdir(app.config['OUTPUT_FOLDER']):
        video_dir = os.path.join(app.config['OUTPUT_FOLDER'], video_id)

        if os.path.isdir(video_dir):
            hls_path = os.path.join(video_dir, 'hls', 'playlist.m3u8')
            dash_path = os.path.join(video_dir, 'dash', 'manifest.mpd')

            if os.path.exists(hls_path) or os.path.exists(dash_path):
                videos.append({
                    'id': video_id,
                    'hls_url': f'/stream/{video_id}/hls/playlist.m3u8' if os.path.exists(hls_path) else None,
                    'dash_url': f'/stream/{video_id}/dash/manifest.mpd' if os.path.exists(dash_path) else None,
                    'player_url': f'/player/{video_id}'
                })

    return jsonify(videos)

At the end we need a main function to run the application:

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=5000)

Phew! Now that the backend is finally done we can now work on the frontend. 😆

Building The Front End

First create a new folder called "templates" and inside that directory create a new file called "index.html" and populate it with the following:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Video Streaming App</title>
    <link href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.2.19/tailwind.min.css" rel="stylesheet">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/alpinejs/3.10.2/cdn.min.js" defer></script>
</head>
<body class="bg-gray-100 min-h-screen">
    <div class="container mx-auto px-4 py-8" x-data="{ 
        dragging: false,
        file: null,
        fileName: '',
        uploading: false,
        progress: 0,
        response: null,
        error: null,
        videos: []
    }" x-init="fetch('/videos')
        .then(response => response.json())
        .then(data => { videos = data })
        .catch(err => { error = 'Failed to load videos' })">

        <h1 class="text-3xl font-bold text-center mb-8">Video Streaming App</h1>

        <!-- Upload Section -->
        <div class="bg-white rounded-lg shadow-md p-6 mb-8">
            <h2 class="text-xl font-semibold mb-4">Upload Video</h2>

            <!-- Drag & Drop Area -->
            <div 
                class="border-2 border-dashed rounded-lg p-8 text-center transition-colors"
                :class="{ 'border-blue-500 bg-blue-50': dragging, 'border-gray-300': !dragging }"
                @dragover.prevent="dragging = true"
                @dragleave.prevent="dragging = false"
                @drop.prevent="
                    dragging = false;
                    const droppedFile = $event.dataTransfer.files[0];
                    if (droppedFile) {
                        file = droppedFile;
                        fileName = file.name;
                    }
                "
            >
                <template x-if="!file">
                    <div>
                        <svg class="w-12 h-12 mx-auto text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
                            <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 16a4 4 0 01-.88-7.903A5 5 0 1115.9 6L16 6a5 5 0 011 9.9M15 13l-3-3m0 0l-3 3m3-3v12"></path>
                        </svg>
                        <p class="mt-2 text-sm text-gray-600">Drag & drop your video file or</p>
                        <label class="mt-2 inline-block px-4 py-2 bg-blue-500 text-white text-sm font-medium rounded-md hover:bg-blue-600 cursor-pointer">
                            Browse Files
                            <input type="file" class="hidden" accept="video/*" @change="
                                file = $event.target.files[0];
                                if (file) fileName = file.name;
                            ">
                        </label>
                    </div>
                </template>

                <template x-if="file">
                    <div>
                        <p class="text-sm font-medium" x-text="fileName"></p>
                        <button 
                            class="mt-2 px-3 py-1 bg-red-500 text-white text-xs rounded hover:bg-red-600"
                            @click.prevent="file = null; fileName = ''"
                        >
                            Remove
                        </button>
                    </div>
                </template>
            </div>

            <!-- Upload Button -->
            <div class="mt-4">
                <button 
                    class="w-full py-2 px-4 bg-blue-500 text-white font-medium rounded-md hover:bg-blue-600 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2 disabled:opacity-50"
                    :disabled="!file || uploading"
                    @click="
                        uploading = true;
                        error = null;
                        response = null;

                        const formData = new FormData();
                        formData.append('file', file);

                        const xhr = new XMLHttpRequest();
                        xhr.open('POST', '/upload');

                        xhr.upload.addEventListener('progress', (e) => {
                            if (e.lengthComputable) {
                                progress = Math.round((e.loaded * 100) / e.total);
                            }
                        });

                        xhr.addEventListener('load', () => {
                            if (xhr.status >= 200 && xhr.status < 300) {
                                response = JSON.parse(xhr.responseText);
                                videos.push(response);
                            } else {
                                try {
                                    error = JSON.parse(xhr.responseText).error;
                                } catch (e) {
                                    error = 'Upload failed';
                                }
                            }
                            uploading = false;
                            file = null;
                            fileName = '';
                            progress = 0;
                        });

                        xhr.addEventListener('error', () => {
                            error = 'Network error';
                            uploading = false;
                            progress = 0;
                        });

                        xhr.send(formData);
                    "
                >
                    <span x-show="!uploading">Upload Video</span>
                    <span x-show="uploading">
                        <span x-text="`Uploading ${progress}%`"></span>
                    </span>
                </button>
            </div>

            <!-- Success/Error Messages -->
            <div class="mt-4">
                <div x-show="error" class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded relative">
                    <span x-text="error"></span>
                </div>

                <div x-show="response" class="bg-green-100 border border-green-400 text-green-700 px-4 py-3 rounded relative">
                    <p>Video uploaded successfully!</p>
                    <p class="text-sm">
                        <a :href="response.player_url" class="underline" target="_blank">Click here to view</a>
                    </p>
                </div>
            </div>
        </div>

        <!-- Video List -->
        <div class="bg-white rounded-lg shadow-md p-6">
            <h2 class="text-xl font-semibold mb-4">Your Videos</h2>

            <div x-show="videos.length === 0" class="text-center text-gray-500 py-8">
                No videos uploaded yet
            </div>

            <div x-show="videos.length > 0" class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-4">
                <template x-for="video in videos" :key="video.id">
                    <div class="border rounded-lg overflow-hidden">
                        <div class="bg-gray-200 h-40 flex items-center justify-center">
                            <svg class="w-12 h-12 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
                                <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M14.752 11.168l-3.197-2.132A1 1 0 0010 9.87v4.263a1 1 0 001.555.832l3.197-2.132a1 1 0 000-1.664z"></path>
                                <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M21 12a9 9 0 11-18 0 9 9 0 0118 0z"></path>
                            </svg>
                        </div>
                        <div class="p-4">
                            <h3 class="font-medium mb-2" x-text="'Video ' + video.id.substring(0, 8)"></h3>
                            <div class="flex space-x-2">
                                <a :href="video.player_url" class="px-3 py-1 bg-blue-500 text-white text-sm rounded hover:bg-blue-600" target="_blank">
                                    Play
                                </a>
                                <a x-show="video.hls_url" :href="video.hls_url" class="px-3 py-1 bg-gray-500 text-white text-sm rounded hover:bg-gray-600" target="_blank">
                                    HLS
                                </a>
                                <a x-show="video.dash_url" :href="video.dash_url" class="px-3 py-1 bg-gray-500 text-white text-sm rounded hover:bg-gray-600" target="_blank">
                                    DASH
                                </a>
                            </div>
                        </div>
                    </div>
                </template>
            </div>
        </div>
    </div>
</body>
</html>

This page is the entry point of the application, it is built using:

Tailwind CSS: For fast and responsive styling
Alpine.js: For handling reactive state without the overhead of a full fledged framework
Drag and Drop File Upload: The drag-and-drop interface makes it easy for users to upload videos. When a file is dropped, the Alpine.js component updates the file state and displays the files name.
XHR Upload Process: The file is uploaded via an XMLHttpRequest. During the file upload, the progress is tracked and dynamically updated, providing real-time feedback to the user.
Video Listing: After a successful upload, the video list is updated with new entries. Each video entry includes buttons to play the video and view different streaming formats.

The implementation demonstrates careful attention to user feedback and error handling. For instance, if the upload fails, an error message is displayed to guide the user.

Now thats taken care of we can now code the player page, under templates create a new file called player.html and populate it with the following:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Video Player</title>
    <link href="https://cdnjs.cloudflare.com/ajax/libs/tailwindcss/2.2.19/tailwind.min.css" rel="stylesheet">
    <link href="https://cdnjs.cloudflare.com/ajax/libs/video.js/7.20.3/video-js.min.css" rel="stylesheet">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/video.js/7.20.3/video.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/videojs-contrib-hls/5.15.0/videojs-contrib-hls.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/dashjs/4.0.1/dash.all.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/videojs-contrib-dash/4.2.1/videojs-dash.min.js"></script>

    <style>
        .video-container {
            position: relative;
            padding-top: 56.25%; /* 16:9 Aspect Ratio */
        }
        .video-container .video-js {
            position: absolute;
            top: 0;
            left: 0;
            width: 100%;
            height: 100%;
        }
    </style>
</head>
<body class="bg-gray-100 min-h-screen">
    <div class="container mx-auto px-4 py-8">
        <div class="mb-4">
            <a href="/" class="text-blue-500 hover:underline">&larr; Back to all videos</a>
        </div>

        <div class="bg-white rounded-lg shadow-md p-6">
            <h1 class="text-2xl font-bold mb-4">Video Player</h1>

            <div class="video-container">
                <video
                    id="my-video"
                    class="video-js vjs-big-play-centered"
                    controls
                    preload="auto"
                    width="640"
                    height="360"
                    data-setup='{"html5": {"hls": {"withCredentials": true}}}'
                >
                    <source src="{{ hls_url }}" type="application/x-mpegURL">
                    <source src="{{ dash_url }}" type="application/dash+xml">
                    <p class="vjs-no-js">
                        To view this video please enable JavaScript, and consider upgrading to a
                        web browser that supports HTML5 video
                    </p>
                </video>
            </div>

            <div class="mt-4">
                <h3 class="font-medium mb-2">Stream URLs:</h3>
                <ul class="space-y-1">
                    <li>
                        <a href="{{ hls_url }}" class="text-blue-500 hover:underline" target="_blank">HLS Stream</a>
                    </li>
                    <li>
                        <a href="{{ dash_url }}" class="text-blue-500 hover:underline" target="_blank">DASH Stream</a>
                    </li>
                </ul>
            </div>
        </div>
    </div>
    <script>
        document.addEventListener('DOMContentLoaded', function() {
            var player = videojs('my-video', {
                techOrder: ["html5"],
                sources: [
                    {
                        src: "{{ hls_url }}",
                        type: "application/x-mpegURL"
                    },
                    {
                        src: "{{ dash_url }}",
                        type: "application/dash+xml"
                    }
                ]
            });

            player.on('error', function() {
                console.log('Video player error:', player.error());
            });
        });
    </script>
</body>
</html>

This page is designed to offer a robust video playback experience, it uses the following:

Video.js: A popular HTML5 video player that simplifies the integration of HLS and DASH streams.
HLS and DASH Sources: Both streaming formats are provided as source elements, allowing the player to choose the best method supported by the browser
Responsive Design: The video player is contained within a responsive container, ensuring compatibility with various screen sizes.
Error Handling: The player listens for error events, and any issues are logged to the console for debugging purposes.

This setup highlights how combining modern libraries and frameworks can yield a powerful, production-ready video streaming solution.

Running The Application

Now that the application is built, we can actually run it, first create a new requirements.txt file and populate it with the following:

flask
flask-cors
Werkzeug

Then run the following command to install the modules:

pip install -r requirements.txt

Then run the server with the following:

python main.py

If you direct your browser to http://localhost:5000 you should see the index page, try uploading a video file and viewing it. 👀

Main Page:

Player Page:

Deployment Considerations

If you plan to deploy this application to a production environment, you will need to consider the following:

Async Processing

This code provides video processing synchronously, production systems should offload intensive tasks to background workers like Celery. This decouples the user experience from backend processing, ensuring the uploads are fast and responsive.

Security

File Validation: Always ensure that uploaded files are validated both by type and by content.
Directory Permissions: The upload and output directories should have strict permissions to prevent unauthorized access.
CORS and CSRF Protection: Although Flask-CORS is used, further measures (like CSRF tokens) might be needed for securing the API endpoints

Scalability

For high traffic, consider using a WSGI server such as Gunicorn or uWSGI behind a reverse proxy like Nginx. Additionally using cloud storage solutions for storing video files can offload file system storage from the application server.

Monitoring and Logging

The application uses Python's build in logging, but in production, integrate with centralized logging and monitoring systems. This ensures that errors, performance bottlenecks, and security issues are promptly identified and addressed.

Conclusion

In this tutorial I have shown you how to build a simple video streaming app using Python, Flask and FFmpeg.

I hope this tutorial has been of use to you and as always you can find the source code on my github:
https://github.com/ethand91/flask-streamer

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

WebCodec - Sending and Receiving

Ethan — Wed, 09 Oct 2024 06:45:39 +0000

Introduction

Hello! 😎

In this tutorial I will show you how to use the WebCodec API to both send and receive video.

First lets get coding the server.

Setting Up The Server

In order to send and receive packets between peers we will need a websocket server.

For this we will create a very basic server using nodejs. First initialize the project:

npm init -y

Then install the required modules:

npm i ws express

Next create a new file called "index.js" and populate it with the following code:

// server.js
const WebSocket = require('ws');
const express = require('express');

const app = express();
const port = 3000;
const connectedClients = new Set();

app.use(express.static(__dirname + '/public'));

const wss = new WebSocket.Server({ noServer: true });

wss.on('connection', ws => {
  console.log('new connection');
  connectedClients.add(ws);

  ws.on('message', message => {
    connectedClients.forEach(client => {
      if (client !== ws && client.readyState === WebSocket.OPEN) {
        client.send(message);
      }
    });
  });

  ws.once('close', () => {
    connectedClients.delete(ws);
    console.log('connection closed');
  });
});

const server = app.listen(port, () => {
  console.log(`server running on port ${port}`);
});

server.on('upgrade', (request, socket, head) => {
  wss.handleUpgrade(request, socket, head, (ws) => {
    wss.emit('connection', ws, request);
  });
});

Nothing too complicated the above code serves the public directory and handles the websocket connection sending packets to all connected peers. 😸

Next we will handle the sender part, but first create a new directory called "public"

mkdir public

Creating The Sender

The first front end file we will create is the one that is broadcasting, create a new file called "sender.html" under public and populate it with the following HTML:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
    <title>Sender</title>
    <style>
      video, canvas {
        width: 640px;
        height: 480px;
        border: 2px solid black;
        margin: 10px;
      }
    </style>
  </head>
  <body>
    <video id="video" autoplay playsinline></video>
    <canvas id="canvas" width="640" height="480"></canvas>

    <script>
      const videoElement = document.getElementById('video');
      const canvas = document.getElementById('canvas');
      const ctx = canvas.getContext('2d');
      let videoEncoder;
      let socket;

      const initWebSocket = () => {
        socket = new WebSocket('ws://localhost:3000');

        socket.onopen = () => console.log('WebSocket connected');
        socket.onerror = error => console.error('WebSocket error:', error);
      };

      const initEncoder = () => {
        videoEncoder = new VideoEncoder({
          output: (encodedChunk) => {
            const chunkData = new Uint8Array(encodedChunk.byteLength);
            encodedChunk.copyTo(chunkData);

            if (socket.readyState === WebSocket.OPEN) {
              socket.send(chunkData.buffer);
            }
          },
          error: (error) => console.error('Encoding error:', error)
        });

        videoEncoder.configure({
          codec: 'vp8',
          width: 640,
          height: 480,
          bitrate: 1_000_000,
          framerate: 30
        });
      };

      navigator.mediaDevices.getUserMedia({ video: true })
        .then((stream) => {
          videoElement.srcObject = stream;
          const videoTrack = stream.getVideoTracks()[0];
          const processor = new MediaStreamTrackProcessor(videoTrack);
          const reader = processor.readable.getReader();

          const processFrames = async () => {
            while (true) {
              const { value: videoFrame, done } = await reader.read();
              if (done) break;

              ctx.drawImage(videoFrame, 0, 0, canvas.width, canvas.height);
              videoEncoder.encode(videoFrame, { keyFrame: true });
              videoFrame.close();
            }
          };

          processFrames();
        })
        .catch((error) => console.error('Failed to get camera', error));

        initEncoder();
        initWebSocket();
    </script>
  </body>
</html>

To breakdown and explain what the code does.

HTML Structure
- The video element displays live video from the user's camera
- The canvas element is used to display individual frames of the video feed. This provides a visual preview of each encoded frame
JavaScript code
- The initWebSocket function connects to a WebSocket server. This connection is essential for streaming the encoded frames to a receiver
- The initEncoder function creates a VideoEncoder object. It defines an output callback that runs every time the encoder produces a new chunk
- The videoEncoder.configure() sets the codec to VP8 with a 1Mbps bitrate and 30 FPS, ensuring a smooth and high quality encoding
- getUserMedia call is used to request access to the camera. The video feed is assigned to the video element, and the VideoTrackProcessor allows processing each frame in real time
- The processFrames function reads frames from the video, displays them on the canvas element, and encodes each frame using videoEncoder.encode(). Each frame is then sent to the server as an encoded chunk.

Phew! Hopefully that was understandable to you. Next we will create the file that will receive the stream. 👀

Creating The Receiver

This file receives the encoded video chunks via WebSocket, decodes them, and displays them on a canvas element.

Create a new file under the public directory called "receiver.html" and populate it with the following:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
    <title>Receiver</title>
    <style>
      canvas {
        width: 640px;
        height: 480px;
        border: 2px solid black;
        margin: 10px;
      }
    </style>
  </head>
  <body>
    <canvas id="canvas" width="640" height="480"></canvas>

    <script>
      const canvas = document.getElementById('canvas');
      const ctx = canvas.getContext('2d');
      let videoDecoder;

      const initWebSocket = () => {
        const socket = new WebSocket('ws://localhost:3000');
        socket.binaryType = 'arraybuffer';

        socket.onmessage = event => {
          decodeFrame(event.data);
        };
        socket.onerror = error => console.error('WebSocket error:', error);
      };

      const initDecoder = () => {
        videoDecoder = new VideoDecoder({
          output: (videoFrame) => {
            ctx.drawImage(videoFrame, 0, 0, canvas.width, canvas.height);
            videoFrame.close();
          },
          error: (error) => console.error('Decoding error:', error)
        });

        videoDecoder.configure({
          codec: 'vp8',
          width: 640,
          height: 480
        });
      };

      const decodeFrame = (encodedData) => {
        const chunk = new EncodedVideoChunk({
          type: 'key',
          timestamp: performance.now(),
          data: new Uint8Array(encodedData)
        });

        videoDecoder.decode(chunk);
      };

      initDecoder();
      initWebSocket();
    </script>
  </body>
</html>

To break down the above file:

HTML
- The canvas element is the primary display area for the decoded video frames. It has a fixed width, height, and border, same as the sender page.
JavaScript
- The initWebSocket function creates a new WebSocket connection, receiving encoded frames from the sender and passing them to decodeFrame() for decoding.
- initDecoder initializes a VideoDecoder object configured for the VP8 codec. The decoder outputs each frame to the canvas.
- decodeFrame takes the encoded data, wraps it in an EncodedVideoChunk (as a key frame with the current timestamp), and decodes it via videoDecoder.decode(). Each frame is displayed on the canvas in real-time

Phew! Now that we have all the pieces needed, lets actually run it! 🤓

Running The Code

To run the code simply run the following command:

node index.js

Then point your browser to http://localhost:3000/sender.html
Allow access to your camera and then open another tab to
http://localhost:3000/receiver.html

Like the below you should see the stream being sent from the sender.

Conclusion

In this tutorial I have shown how to get access to the camera, encode it, send the chunks over WebSocket and decode and display them on the receiver side. I hope this tutorial was of use to you. 🥳

As always you can get the code via my github:
https://github.com/ethand91/webcodec-stream

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Introduction to the WebCodec API - Real Time Video Encoding and Display

Ethan — Mon, 16 Sep 2024 05:38:15 +0000

Introduction

Hello! 😎

In this tutorial I will be introducing the WebCodec API and how to use it to display a video track in a canvas element.

The WebCodec API is an exciting advancement for developers looking to access low-level control over encoding and decoding of media streams in the browser. This API allows for efficient, high-performance encoding and decoding of media files such as video and audio without the need for complex external libraries. With WebCodec, developers can handle tasks like real-time video encoding, transcoding, and more, enabling user cases such as video conferencing, live streaming and video editing directly in the browser.

In this post, we'll break down a practical example of how to use WebCodec API to capture video from a user's camera, encode it in real time using VP8 encoding, and display both the raw video feed and the encoded frames on the screen. This will help you understand not only the fundamentals of the WebCodec API but also how to apply it to real- world scenarios. 😃

Why Use the WebCodec API?

Before diving into the code, let's discuss why the WebCodec API is such a valuable tool for developers:

Performance: WebCodec provides hardware-accelerated encoding and decoding, making it faster and more efficient than traditional JavaScript methods. This is especially important for real-time applications like video conferencing and live streaming.
Low-level Control: Unlike high-level APIs, WebCodec allows developers to handle raw media frames directly, providing fine-grained control over encoding parameters such as bit rate, resolution and codec.
Real-Time Processing: WebCodec is optimized for real-time video and audio processing. By allowing direct access to the codec, developers can reduce latency, a critical requirement for applications like gaming, video streaming and conferencing.

Now that we understand the importance of WebCodec, let's dive into the example code and break it down in detail. 👀

The Complete Example

Here is the full HTML file that demonstrates how to use WebCodec to capture video, encode it and display both the original feed and the encoded frames.

I'll be explaining the code after this.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1.0"/>
    <title>WebCodec Example</title>
    <style>
      body {
        display: flex;
        justify-content: center;
        align-items: center;
        height: 100vh;
        margin: 0;
      }

      video, canvas {
        width: 640px;
        height: 480px;
        border: 2px solid black;
        margin: 0 10px;
      }
    </style>
  </head>
  <body>
    <video id="video" autoplay playsinline></video>
    <canvas id="canvas" width="640" height="480"></canvas>

    <script>
      const videoElement = document.getElementById('video');
      const canvas = document.getElementById('canvas');
      const ctx = canvas.getContext('2d');
      let videoEncoder;

      const initEncoder = () => {
        videoEncoder = new VideoEncoder({
          output: (frame) => {
            // console.log('new frame', frame);
          },
          error: (error) => {
            console.error('encode error', error);
          }
        });

        videoEncoder.configure({
          codec: 'vp8',
          width: 640,
          height: 480,
          bitrate: 1_000_000,
          framerate: 30
        });
      };

      const handleStream = (stream) => {
        videoElement.srcObject = stream;

        const videoTrack = stream.getVideoTracks()[0];
        const processor = new MediaStreamTrackProcessor(videoTrack);
        const reader = processor.readable.getReader();

        const processFrames = async () => {
          while (true) {
            const { value: videoFrame, done } = await reader.read();

            if (done) {
              console.log('stream ended');
              break;
            }

            ctx.drawImage(videoFrame, 0, 0, canvas.width, canvas.height);

            const insertKeyFrame = true;
            videoEncoder.encode(videoFrame, { keyframe: insertKeyFrame });

            videoFrame.close();
          }
        };

        processFrames();
      };

      initEncoder();

      navigator.mediaDevices.getUserMedia({ video: true })
        .then((stream) => handleStream(stream))
        .catch((error) => console.error('failed to get camera', error));
    </script>
  </body>
</html>

Detailed Breakdown of the Code

HTML Structure

The HTML is quite simple.We have two main elements: a video element and a canvas element.

The video element is used to display the raw video stream directly from the user's camera.
The canvas element is used to display the frames that have been processed by WebCodec

Both elements are given the same dimensions of 640x480 pixels, they are aligned side by side using Flexbox, for easy comparison.

CSS for Styling

The CSS part defines a simple layout:

The body element uses Flexbox to center the video and canvas elements both horizontally and vertically on the screen.
Both the video and canvas elements are styled to be 640 pixels wide and 480 pixels high with a solid black border.

This styling ensures that both elements are displayed clearly and have the same size.

Javascript Initialization and Camera Access

The following code, enables us to get access to the camera:

navigator.mediaDevices.getUserMedia({ video: true })
  .then((stream) => handleStream(stream))
  .catch((error) => console.error('failed to get camera', error));

The method requests access to the user's camera. If the request for the camera access fails an error message is logged to the console.

Next we set the video source with the following:

videoElement.srcObject = stream;

Inside the handleStream function, the srcObject is set to the camera stream, which allows the browser to display the live video feed directly in the video element.

const videoTrack = stream.getVideoTracks()[0];
const processor = new MediaStreamTrackProcessor(videoTrack);
const reader = processor.readable.getReader();

With the above, we access the video track from the stream. The MediaStreamTrackProcessor API is then used to process individual frames from the video track. It converts the video track into a stream of individual frames that be read asynchronously using a ReadableStream reader.

The readable.getReader() method creates a ReadableStreamDefaultReader that allows us to read individual video frames.

Next we initialize the VideoEncoder:

const initEncoder = () => {
  videoEncoder = new VideoEncoder({
    output: (frame) => {
      // Process encoded frame (e.g., save or transmit)
    },
    error: (error) => {
      console.error('encode error', error);
    }
  });

  videoEncoder.configure({
    codec: 'vp8',
    width: 640,
    height: 480,
    bitrate: 1_000_000,
    framerate: 30
  });
};

The initEncoder function sets up the VideoEncoder. The VideoEncoder is initialized with two functions:

output: This function is called every time a frame is successfully encoded. You could use this function to handle the encoded frames (e.g saving them to a file or streaming them to a server).
error: This function logs any encoding errors.

In the configure method, we specify that the encoder will use the VP8 codec, with a resolution of 640x480 pixels, a bit rate of 1Mbps and a frame rate of 30 frames per second.

Next we will read and encode the frames:

const processFrames = async () => {
  while (true) {
    const { value: videoFrame, done } = await reader.read();

    if (done) {
      console.log('stream ended');
      break;
    }

    ctx.drawImage(videoFrame, 0, 0, canvas.width, canvas.height);

    const insertKeyFrame = true;
    videoEncoder.encode(videoFrame, { keyframe: insertKeyFrame });

    videoFrame.close();
  }
};

The processFrames function reads each frame from the video stream using the reader.read() method. This method returns a promise that resolves with the next video frame.

For each frame:

We use the ctx.drawImage method to draw the frame onto the canvas element.
We then pass the frame to the videoEncoder.encode() method to encode it. Here, we specify that a keyframe should be inserted. Keyframes are crucial in video encoding because they serve as reference points for decoding the rest of the frames.
Finally, we call videoFrame.close() to release the memory used by the frame.

If the video stream ends, the done property of the reader.read() call will be set to true, at which point we break out of the while loop and stop processing frames. 😁

Conclusion

Here I have shown the power and flexibility of the WebCodec API for handling real-time media streams in the browser. By using WebCodec, you can gain low-level control over video encoding, which is essential for applications requiring high performance and low latency, such as video conferencing or live streaming.

Key points to remember are:

MediaStreamTrackProcessor is used to break down a video stream into individual frames that can be processed.
VideoEncoder from the WebCodec API encodes raw video frames into a compressed format like VP8.
You can configure the encoder's parameters, such as resolution, bit rate, and frame rate, to suit your specific use case.

This combination of APIs opens up new possibilities for high-performance, real time video processing directly in the browser, paving a way for advanced web-based video applications.

I hope this tutorial helped you, I had a lot of fun trying this API out and I may expand on this and turn it into a series.

As always you can find the code on my Github:
https://github.com/ethand91/webcodec-example

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Streaming Camera with C++ WebRTC GStreamer

Ethan — Wed, 26 Jun 2024 04:33:33 +0000

Introduction

Hello! 😎

In this advanced WebRTC tutorial I will show you how to stream your camera to a HTML page using WebRTC, GStreamer and C++. We will be using boost to handle the signaling. By the end of this tutorial you should have a simple understanding on WebRTC GStreamer. 👀

Requirements

GStreamer and its development libraries
Boost libraries
CMake for building the project
A C++ compiler
Basic C++ Knowledge

Creating the Project

First we need a place to house our projects files, create a new directory like so:



mkdir webrtc-stream && cd webrtc-stream

First we need to create a build file in order to build the completed project, create a new file called "CMakeLists.txt" and populate it with the following:



cmake_minimum_required(VERSION 3.10)

# Set the project name and version
project(webrtc_server VERSION 1.0)

# Specify the C++ standard
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED True)

# Find required packages
find_package(PkgConfig REQUIRED)
pkg_check_modules(GST REQUIRED gstreamer-1.0 gstreamer-webrtc-1.0 gstreamer-sdp-1.0)

find_package(Boost 1.65 REQUIRED COMPONENTS system filesystem json)

# Include directories
include_directories(${GST_INCLUDE_DIRS} ${Boost_INCLUDE_DIRS})

# Add the executable
add_executable(webrtc_server main.cpp)

# Link libraries
target_link_libraries(webrtc_server ${GST_LIBRARIES} Boost::system Boost::filesystem Boost::json)

# Set properties
set_target_properties(webrtc_server PROPERTIES
    CXX_STANDARD 14
    CXX_STANDARD_REQUIRED ON
)

# Specify additional directories for the linker
link_directories(${GST_LIBRARY_DIRS})

# Print project info
message(STATUS "Project: ${PROJECT_NAME}")
message(STATUS "Version: ${PROJECT_VERSION}")
message(STATUS "C++ Standard: ${CMAKE_CXX_STANDARD}")
message(STATUS "Boost Libraries: ${Boost_LIBRARIES}")
message(STATUS "GStreamer Libraries: ${GST_LIBRARIES}")

The above links all the required libraries together in order to build the code into an executable that can be executed.

Now we can get on to coding the project. 🥸

Coding the Project

Now we can start coding the source code for the project, create a new file called "main.cpp", we will start by importing the necessary headers for GStreamer, WebRTC, Boost and standard libraries:



#define GST_USE_UNSTABLE_API
#include <gst/gst.h>
#include <gst/webrtc/webrtc.h>
#include <boost/beast.hpp>
#include <boost/asio.hpp>
#include <boost/json.hpp>
#include <iostream>
#include <thread>

namespace beast = boost::beast;
namespace http = beast::http;
namespace websocket = beast::websocket;
namespace net = boost::asio;
using tcp = net::ip::tcp;
using namespace boost::json;

Next we will be define constants that will be used later, mainly the STUN server and port that the server will listen on:



#define STUN_SERVER "stun://stun.l.google.com:19302"
#define SERVER_PORT 8000

Now we will declare global variables for the GStreamer main loop and pipeline elements:



GMainLoop *loop;
GstElement *pipeline, *webrtcbin;

Next we will create the functions to handle each of the events. First one being a function that sends ICE candidates to the WebSocket client:



void send_ice_candidate_message(websocket::stream<tcp::socket>& ws, guint mlineindex, gchar *candidate)
{
  std::cout << "Sending ICE candidate: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;

  object ice_json;
  ice_json["candidate"] = candidate;
  ice_json["sdpMLineIndex"] = mlineindex;

  object msg_json;
  msg_json["type"] = "candidate";
  msg_json["ice"] = ice_json;

  std::string text = serialize(msg_json);
  ws.write(net::buffer(text));

  std::cout << "ICE candidate sent" << std::endl;
}

The next "on_answer_created" function handles the creation of a WebRTC answer and sends it back to the client:



void on_answer_created(GstPromise *promise, gpointer user_data)
{
  std::cout << "Answer created" << std::endl;

  websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
  GstWebRTCSessionDescription *answer = NULL;
  const GstStructure *reply = gst_promise_get_reply(promise);
  gst_structure_get(reply, "answer", GST_TYPE_WEBRTC_SESSION_DESCRIPTION, &answer, NULL);
  GstPromise *local_promise = gst_promise_new();
  g_signal_emit_by_name(webrtcbin, "set-local-description", answer, local_promise);

  object sdp_json;
  sdp_json["type"] = "answer";
  sdp_json["sdp"] = gst_sdp_message_as_text(answer->sdp);
  std::string text = serialize(sdp_json);
  ws->write(net::buffer(text));

  std::cout << "Local description set and answer sent: " << text << std::endl;

  gst_webrtc_session_description_free(answer);
}

The next function is just a placeholder for handling negotiation events, this event is not needed in this example:



void on_negotiation_needed(GstElement *webrtc, gpointer user_data)
{
  std::cout << "Negotiation needed" << std::endl;
}

The "on_set_remote_description" function sets the remote description and creates an answer:



void on_set_remote_description(GstPromise *promise, gpointer user_data)
{
  std::cout << "Remote description set, creating answer" << std::endl;

  websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
  GstPromise *answer_promise = gst_promise_new_with_change_func(on_answer_created, ws, NULL);

  g_signal_emit_by_name(webrtcbin, "create-answer", NULL, answer_promise);
}

The "on_ice_candidate" function handles ICE candidate events and sends them to the WebSocket client:



void on_ice_candidate(GstElement *webrtc, guint mlineindex, gchar *candidate, gpointer user_data)
{
  std::cout << "ICE candidate generated: mlineindex=" << mlineindex << ", candidate=" << candidate << std::endl;

  websocket::stream<tcp::socket>* ws = static_cast<websocket::stream<tcp::socket>*>(user_data);
  send_ice_candidate_message(*ws, mlineindex, candidate);
}

The "handle_websocket_session" function manages the WebSocket connection, setting up the GStreamer pipeline and handling both SDP and ICE messages:



void handle_websocket_session(tcp::socket socket)
{
  try
  {
    websocket::stream<tcp::socket> ws{std::move(socket)};
    ws.accept();

    std::cout << "WebSocket connection accepted" << std::endl;

    GstStateChangeReturn ret;
    GError *error = NULL;

    pipeline = gst_pipeline_new("pipeline");
    GstElement *v4l2src = gst_element_factory_make("v4l2src", "source");
    GstElement *videoconvert = gst_element_factory_make("videoconvert", "convert");
    GstElement *queue = gst_element_factory_make("queue", "queue");
    GstElement *vp8enc = gst_element_factory_make("vp8enc", "encoder");
    GstElement *rtpvp8pay = gst_element_factory_make("rtpvp8pay", "pay");
    webrtcbin = gst_element_factory_make("webrtcbin", "sendrecv");

    if (!pipeline || !v4l2src || !videoconvert || !queue || !vp8enc || !rtpvp8pay || !webrtcbin)
    {
      g_printerr("Not all elements could be created.\n");
      return;
    }

    g_object_set(v4l2src, "device", "/dev/video0", NULL);
    g_object_set(vp8enc, "deadline", 1, NULL);

    gst_bin_add_many(GST_BIN(pipeline), v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, webrtcbin, NULL);

    if (!gst_element_link_many(v4l2src, videoconvert, queue, vp8enc, rtpvp8pay, NULL))
    {
      g_printerr("Elements could not be linked.\n");
      gst_object_unref(pipeline);
      return;
    }

    GstPad *rtp_src_pad = gst_element_get_static_pad(rtpvp8pay, "src");
    GstPad *webrtc_sink_pad = gst_element_get_request_pad(webrtcbin, "sink_%u");
    gst_pad_link(rtp_src_pad, webrtc_sink_pad);
    gst_object_unref(rtp_src_pad);
    gst_object_unref(webrtc_sink_pad);

    g_signal_connect(webrtcbin, "on-negotiation-needed", G_CALLBACK(on_negotiation_needed), &ws);
    g_signal_connect(webrtcbin, "on-ice-candidate", G_CALLBACK(on_ice_candidate), &ws);

    ret = gst_element_set_state(pipeline, GST_STATE_PLAYING);

    if (ret == GST_STATE_CHANGE_FAILURE)
    {
      g_printerr("Unable to set the pipeline to the playing state.\n");
      gst_object_unref(pipeline);
      return;
    }

    std::cout << "GStreamer pipeline set to playing" << std::endl;

    for (;;)
    {
      beast::flat_buffer buffer;
      ws.read(buffer);

      auto text = beast::buffers_to_string(buffer.data());
      value jv = parse(text);
      object obj = jv.as_object();
      std::string type = obj["type"].as_string().c_str();

      if (type == "offer")
      {
        std::cout << "Received offer: " << text << std::endl;

        std::string sdp = obj["sdp"].as_string().c_str();

        GstSDPMessage *sdp_message;
        gst_sdp_message_new_from_text(sdp.c_str(), &sdp_message);
        GstWebRTCSessionDescription *offer = gst_webrtc_session_description_new(GST_WEBRTC_SDP_TYPE_OFFER, sdp_message);
        GstPromise *promise = gst_promise_new_with_change_func(on_set_remote_description, &ws, NULL);
        g_signal_emit_by_name(webrtcbin, "set-remote-description", offer, promise);
        gst_webrtc_session_description_free(offer);

        std::cout << "Setting remote description" << std::endl;
      }
      else if (type == "candidate")
      {
        std::cout << "Received ICE candidate: " << text << std::endl;

        object ice = obj["ice"].as_object();
        std::string candidate = ice["candidate"].as_string().c_str();
        guint sdpMLineIndex = ice["sdpMLineIndex"].as_int64();
        g_signal_emit_by_name(webrtcbin, "add-ice-candidate", sdpMLineIndex, candidate.c_str());

        std::cout << "Added ICE candidate" << std::endl;
      }
    }
  }
  catch (beast::system_error const& se)
  {
    if (se.code() != websocket::error::closed)
    {
      std::cerr << "Error: " << se.code().message() << std::endl;
    }
  }
  catch (std::exception const& e)
  {
    std::cerr << "Exception: " << e.what() << std::endl;
  }
}

The next "start_server" function initializes the server, sccepting TCP connections and spawning new threads to handle each connection:



void start_server()
{
  try
  {
    net::io_context ioc{1};
    tcp::acceptor acceptor{ioc, tcp::endpoint{tcp::v4(), SERVER_PORT}};

    for (;;)
    {
      tcp::socket socket{ioc};
      acceptor.accept(socket);
      std::cout << "Accepted new TCP connection" << std::endl;
      std::thread{handle_websocket_session, std::move(socket)}.detach();
    }
  }
  catch (std::exception const& e)
  {
    std::cerr << "Exception: " << e.what() << std::endl;
  }
}

Finally we just need to create the final main function to initialize GStreamer, start the server and run the main loop:



int main(int argc, char *argv[])
{
  gst_init(&argc, &argv);
  loop = g_main_loop_new(NULL, FALSE);

  std::cout << "Starting WebRTC server" << std::endl;

  std::thread server_thread(start_server);
  g_main_loop_run(loop);

  server_thread.join();

  gst_element_set_state(pipeline, GST_STATE_NULL);
  gst_object_unref(pipeline);
  g_main_loop_unref(loop);

  std::cout << "WebRTC server stopped" << std::endl;

  return 0;
}

Done, now we can finally build the project! 😄

Building the Project

To build the above source code into an executable first create a new directory called build:



mkdir build && cd build

Build the project:



cmake ..
make

If all goes well the project should be built successfully and you should have an executable.

Next we need to create a page to view the stream. 😸

Creating the Frontend

Create a new directory called "public" and in it create a new html file called "index.html" and populate it with the following code:



<!DOCTYPE html>
<html>
<head>
  <title>WebRTC Stream</title>
</head>
<body>
  <video id="video" autoplay playsinline muted></video>
  <script>
    const video = document.getElementById('video');
    const signaling = new WebSocket('ws://localhost:8000/ws');
    let pc = new RTCPeerConnection({
      iceServers: [{urls: 'stun:stun.l.google.com:19302'}]
    });

    signaling.onmessage = async (event) => {
      const data = JSON.parse(event.data);
      console.log('Received signaling message:', data);

      if (data.type === 'answer') {
        console.log('Setting remote description with answer');
        await pc.setRemoteDescription(new RTCSessionDescription(data));
      } else if (data.type === 'candidate') {
        console.log('Adding ICE candidate:', data.ice);
        await pc.addIceCandidate(new RTCIceCandidate(data.ice));
      }
    };

    pc.onicecandidate = (event) => {
      if (event.candidate) {
        console.log('Sending ICE candidate:', event.candidate);
        signaling.send(JSON.stringify({
          type: 'candidate',
          ice: event.candidate
        }));
      }
    };

    pc.ontrack = (event) => {
      console.log('Received track:', event);
      if (event.track.kind === 'video') {
        console.log('Attaching video track to video element');
        video.srcObject = event.streams[0];
        video.play().catch(error => {
          console.error('Error playing video:', error);
        });

        video.load();
      }
    };

    pc.oniceconnectionstatechange = () => {
      console.log('ICE connection state:', pc.iceConnectionState);
    };

    pc.onicegatheringstatechange = () => {
      console.log('ICE gathering state:', pc.iceGatheringState);
    };

    pc.onsignalingstatechange = () => {
      console.log('Signaling state:', pc.signalingState);
    };

    async function start() {
      pc.addTransceiver('video', {direction: 'recvonly'});
      const offer = await pc.createOffer();
      console.log('Created offer:', offer);
      await pc.setLocalDescription(offer);
      console.log('Set local description with offer');
      signaling.send(JSON.stringify({type: 'offer', sdp: pc.localDescription.sdp}));
    }

    start();
  </script>
</body>
</html>

The above can be explained in my other WebRTC tutorials, but it simple communicates with the signalling server and when a remote stream is received plays the video in the video HTML element.

Done now we can actually run the project! 👍

Running the Project

To run the project simple execute the following command:



./webrtc_server

To run the html page we will use a python module:



python3 -m http.server 9999

Navigate your browser to http://localhost:9999 and on load you should see your camera showing in the video element like so:

Done! 😁

Considerations

In order to improve the above, I would like to implement the following:

Handle multiple viewers
Handle receiving a stream from HTML
Creating an SFU
Recording

Conclusion

In this tutorial I have shown you how to stream your camera using native C++, GStreamer and view the stream in a HTML page. I hope this tutorial has taught you something, I certainly had a lot of fun creating it.

As always you can find the source code for the project on my Github:
https://github.com/ethand91/webrtc-gstreamer

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Creating a CRUD App With Go

Ethan — Mon, 24 Jun 2024 06:51:51 +0000

Introduction

Hello! 😎

In this tutorial I will show you how to build a CRUD app using the Go programming language. By the end, you'll have a fully functioning CRUD (Create, Read, Update, Delete) app running locally. Let's get started.

Requirements

Go installed
Basic understanding of Go

Setting up the project

Create a new directory for the project and navigate into it:



mkdir crud-app && cd crud-app

Next we will initialize the go module and install the dependencies:



go mod init go-crud-app
go get -u github.com/gorilla/mux

Now that the project and required packages are installed we can start working on the application. 😸

Coding the Application

Create a new file called "main.go" and import the following:



package main

import (
    "encoding/json"
    "log"
    "net/http"
    "strconv"
    "github.com/gorilla/mux"
)

encoding/json: For encoding and decoding JSON data
log: For logging errors
net/http: For building the HTTP server
strconv: For converting strings to integers
github.com/gorilla/mux: A popular package for routing HTTP requests

Next we define our global variables like so:



var memos []Memo
var idCounter int

memos: A slice that will store all our memos
idCounter: A counter to keep track of memo IDs

Now we need to create a struct to define a memo:



type Memo struct {
    ID      int    `json:"id"`
    Title   string `json:"title"`
    Content string `json:"content"`
}

Next we will create handlers for each of the CRUD operations, the first one will handle create a new memo:



func createMemo(w http.ResponseWriter, r *http.Request) {
    var memo Memo
    json.NewDecoder(r.Body).Decode(&memo)
    idCounter++
    memo.ID = idCounter
    memos = append(memos, memo)
    json.NewEncoder(w).Encode(memo)
}

Decode the JSON request body into a memo struct
Increment the idCounter and assign a new memo
Append the new memo to the memos slice
Encode and return the created memo as a JSON response

The next function will handle getting all the memos:



func getMemos(w http.ResponseWriter, r *http.Request) {
    json.NewEncoder(w).Encode(memos)
}

Encode and return the memos slice as a JSON response

Below is the handler to handle getting a single memo:



func getMemo(w http.ResponseWriter, r *http.Request) {
    params := mux.Vars(r)
    id, _ := strconv.Atoi(params["id"])

    for _, memo := range memos {
        if memo.ID == id {
            json.NewEncoder(w).Encode(memo)
            return
        }
    }

    http.Error(w, "Memo not found", http.StatusNotFound)
}

Retrieve the id from the URL parameters
Search for the memo with the matching ID in the memos slice
Return the memo if found, otherwise returns a 404 error

The next function handles updating a memo:



func updateMemo(w http.ResponseWriter, r *http.Request) {
    params := mux.Vars(r)
    id, _ := strconv.Atoi(params["id"])

    for i, memo := range memos {
        if memo.ID == id {
            json.NewDecoder(r.Body).Decode(&memo)
            memo.ID = id
            memos[i] = memo
            json.NewEncoder(w).Encode(memo)
            return
        }
    }

    http.Error(w, "Memo not found", http.StatusNotFound)
}

Retrieve the id from the URL parameters
Search for the memo with the matching ID in the memos slice
Decode the JSON request body into the found memo and update it
Return the updated memo as a JSON response or return a 404 error if not found

Lastly we need a function to handle the delete operation:



func deleteMemo(w http.ResponseWriter, r *http.Request) {
    params := mux.Vars(r)
    id, _ := strconv.Atoi(params["id"])

    for i, memo := range memos {
        if memo.ID == id {
            memos = append(memos[:i], memos[i+1:]...)
            json.NewEncoder(w).Encode("The memo was deleted successfully")
            return
        }
    }

    http.Error(w, "Memo not found", http.StatusNotFound)
}

Retrieve the id from the URL parameters
Search for the memo with the matching ID in the memos slice
Delete the memo by removing it from the slice
Return a success message or a 404 error if the memo is not found

Next we need to set up the routes to handle each of the CRUD requests:



func initializeRouter() {
    router := mux.NewRouter()

    router.HandleFunc("/memos", createMemo).Methods("POST")
    router.HandleFunc("/memos", getMemos).Methods("GET")
    router.HandleFunc("/memos/{id}", getMemo).Methods("GET")
    router.HandleFunc("/memos/{id}", updateMemo).Methods("PUT")
    router.HandleFunc("/memos/{id}", deleteMemo).Methods("DELETE")

    log.Fatal(http.ListenAndServe(":8000", router))
}

Initialize a new router using mux.NewRouter
Define routes for each CRUD operation and map them to the respective handlers
Start the HTTP server on port 8000

Finally we need to implement the main function, as so:



func main() {
    memos = append(memos, Memo{ID: 1, Title: "First memo", Content: "Hello World"})
    idCounter = 1
    initializeRouter()
}

Initialize the memos slice with a sample memo
Set the idCounter to 1
Call the previous initializeRouter function to start the server

Done! Now we can move onto testing the application. 😆

Testing the Application

First we need to start the server before we can make requests to it, this is done via the following command:



go run main.go

Now we can use the following CURL commands to test each of the endpoints.

Create a memo:



curl -X POST -d '{"title":"New Memo","content":"This is a new memo."}' -H "Content-Type: application/json" http://localhost:8000/memos

Get all memos:



curl http://localhost:8000/memos

Get a memo by it's ID:



curl http://localhost:8000/memos/1

Update a memo:



curl -X PUT -d '{"title":"Updated Memo","content":"This is an updated memo."}' -H "Content-Type: application/json" http://localhost:8000/memos/1

Delete a memo:



curl -X DELETE http://localhost:8000/memos/1

Feel free to change the contents and have a play around with it. 👀

Conclusion

In this tutorial I have shown how to implement a CRUD application using the Go programming language, I'm having a lot of fun learning Go and I hope this tutorial has helped you.

As always you can find the sample code on my Github:
https://github.com/ethand91/go-crud

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Creating a Face Swapping Application with Python and OpenCV

Ethan — Thu, 13 Jun 2024 07:46:51 +0000

Introduction

Hello! 😎

In this tutorial I will teach you how to create a face-swapping application using Python, OpenCV and dlib.
Face swapping involves taking the face from one image and seamlessly blending it onto another face in a different image.
This tutorial is beginner-friendly and will guide you through the entire process. By the end you'll have a working face-swapping application and a good understanding of some essential image processing techniques.

Requirements

For this tutorial you will need to have the following installed:

Python
Pip (Python package installer)

Setting Up the Environment

First we will need to create a virtual environment for the project.
Create a new directory that will house our project via the following command:



mkdir face_swap && cd face_swap

Next create the virtual environment and activate it:



python3 -m venv env
source env/bin/activate

Now we need to install the packages required by this project, create a new file called "requirements.txt" and populate it with the following:



opencv-python
dlib
imutils
numpy

To install the required packages run the following command:



pip install -r requirements.txt

Additionally, you need to download the pre-trained shape predictor model for facial landmarks from dlib. Download the file from the following link and extract it into your project directory.

http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2

Done! Now we are ready to write the code! 😆

Writing the Face Swapping Code

First create a new file called "main.py", we will start by importing the necessary libraries and defining a function to apply an affine transform:



import cv2
import dlib
import numpy as np
import imutils
from imutils import face_utils
import argparse

def apply_affine_transform(src, src_tri, dst_tri, size):
    warp_mat = cv2.getAffineTransform(np.float32(src_tri), np.float32(dst_tri))
    dst = cv2.warpAffine(src, warp_mat, (size[0], size[1]), None, flags=cv2.INTER_LINEAR, borderMode=cv2.BORDER_REFLECT_101)
    return dst

Next, we will define a function to warp the triangles from the source image to the destination image like so:



def warp_triangle(img1, img2, t1, t2):
    r1 = cv2.boundingRect(np.float32([t1]))
    r2 = cv2.boundingRect(np.float32([t2]))

    t1_rect = []
    t2_rect = []
    t2_rect_int = []

    for i in range(0, 3):
        t1_rect.append(((t1[i][0] - r1[0]), (t1[i][1] - r1[1])))
        t2_rect.append(((t2[i][0] - r2[0]), (t2[i][1] - r2[1])))
        t2_rect_int.append(((t2[i][0] - r2[0]), (t2[i][1] - r2[1])))

    img1_rect = img1[r1[1]:r1[1] + r1[3], r1[0]:r1[0] + r1[2]]

    size = (r2[2], r2[3])
    img2_rect = apply_affine_transform(img1_rect, t1_rect, t2_rect, size)

    mask = np.zeros((r2[3], r2[2], 3), dtype=np.float32)
    cv2.fillConvexPoly(mask, np.int32(t2_rect_int), (1.0, 1.0, 1.0), 16, 0)

    img2[r2[1]:r2[1] + r2[3], r2[0]:r2[0] + r2[2]] = img2[r2[1]:r2[1] + r2[3], r2[0]:r2[0] + r2[2]] * (1 - mask) + img2_rect * mask

Now, we will write the main face_swap function that will handle the face detection, landmark extraction and face swapping.



def face_swap(image1_path, image2_path):
    detector = dlib.get_frontal_face_detector()
    predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

    image1 = cv2.imread(image1_path)
    image2 = cv2.imread(image2_path)

    gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
    gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)

    rects1 = detector(gray1, 1)
    rects2 = detector(gray2, 1)

    if len(rects1) == 0 or len(rects2) == 0:
        print("Error: Could not detect faces in one or both images.")
        return

    shape1 = predictor(gray1, rects1[0])
    shape2 = predictor(gray2, rects2[0])

    points1 = face_utils.shape_to_np(shape1)
    points2 = face_utils.shape_to_np(shape2)

    hullIndex = cv2.convexHull(points2, returnPoints=False)
    hull1 = points1[hullIndex[:, 0]]
    hull2 = points2[hullIndex[:, 0]]

    rect = (0, 0, gray2.shape[1], gray2.shape[0])
    subdiv = cv2.Subdiv2D(rect)
    subdiv.insert(hull2.tolist())
    triangles = subdiv.getTriangleList()
    triangles = np.array(triangles, dtype=np.int32)

    indexes_triangles = []
    for t in triangles:
        pts = [(t[0], t[1]), (t[2], t[3]), (t[4], t[5])]
        indices = []
        for pt in pts:
            ind = np.where((hull2 == pt).all(axis=1))
            if len(ind[0]) == 0:
                continue
            indices.append(ind[0][0])
        indexes_triangles.append(indices)

    img2_new_face = np.zeros_like(image2)

    for indices in indexes_triangles:
        t1 = [hull1[indices[0]], hull1[indices[1]], hull1[indices[2]]]
        t2 = [hull2[indices[0]], hull2[indices[1]], hull2[indices[2]]]

        warp_triangle(image1, img2_new_face, t1, t2)

    mask = np.zeros_like(gray2)
    cv2.fillConvexPoly(mask, np.int32(hull2), (255, 255, 255))

    r = cv2.boundingRect(np.float32([hull2]))
    center = (r[0] + int(r[2] / 2), r[1] + int(r[3] / 2))
    output = cv2.seamlessClone(img2_new_face, image2, mask, center, cv2.NORMAL_CLONE)

    cv2.imwrite("output.jpg", output)

Next to wrap up the code for the application, we will add the command line argument parsing and the main function in order to run our script:



def main():
    parser = argparse.ArgumentParser(description="Face Swapping Application")
    parser.add_argument("image1", type=str, help="Path to the first image (source face)")
    parser.add_argument("image2", type=str, help="Path to the second image (destination face)")

    args = parser.parse_args()

    face_swap(args.image1, args.image2)

if __name__ == "__main__":
    main()

That's the end of the code, now we can actually run our application! 👀

Running the Application

In order to run our application, you will need to images in order to perform the face swap. Once you have two images run the above script with the following command:



python main.py [image1] [image2]

Replace image1 and image2 with the paths to your images. The script will detect faces in the images, swap them, and then save the new image as "output.jpg".

Once the command has run checkout the new "output.jpg". 🥸

Original:

Output:

Conclusion

In this tutorial I have shown how to use Python, OpenCV and dlib to swap faces. This example is pretty simple, so it may not work right with multiple faces. I hope this tutorial has taught you something as I certainly had fun making it.

If you know of any ways to further refine the face swapping, please tell me.

As always the code for this example can be found on my Github:
https://github.com/ethand91/face-swap

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Creating a CRUD Application With Express and HTMX

Ethan — Fri, 07 Jun 2024 04:45:15 +0000

Introduction

Hello! 😎

In this tutorial I will show you how to create a simple Todo CRUD application using Express for the backend and HTMX for the frontend.

Creating a CRUD(Create, Read, Update, Delete) application is a great way to understand the basics of web development. By the end of this tutorial, you'll have a working application that allows you to add, view, edit and delete tasks. Let's get coding! 😸

Requirements

nodeJS installed (https://nodejs.org/)

Setting Up the Backend With Express

First we need an API server, so to keep things simple I will be using Express.

First create a new directory for the project and initialize it:



mkdir htmx-crud && cd htmx-crud
yarn init -y

Next install the packages required for this project:



yarn add express body-parser cors

Now we need to create a src folder to store the source code files:



mkdir src

Create a new file in the newly created src directory called "server.js", first we will import the required modules:



const express = require('express');
const bodyParser = require('body-parser');
const cors = require('cors');
const path = require('path');

Next we need to initialize express and load the required middleware, this can be done via the following:



const app = express();
const PORT = 3000;

app.use(cors());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static(path.join(__dirname, '../public/')));

The above initializes express and loads the required middleware, we will be handling JSON and we have enabled cors for all origins.

Next we will define the todo list array with mock data:



let todos = [
  { id: 1, task: 'Learn HTMX' },
  { id: 2, task: 'Feed Cat' }
];

Now to define the routes that will be needed by the front end, here is the routes for all CRUD operations:



app.get('/api/todos', (req, res) => {
  try {
    res.status(200).json(todos);
  } catch (error) {
    console.error('Failed to get todos', error);
  }
});

app.post('/api/todos', (req, res) => {
  try {
    const newTodo = { id: todos.length + 1, task: req.body.task };
    todos.push(newTodo);

    res.status(201).json(newTodo);
  } catch (error) {
    console.error('Failed to create todo', error);
  }
});

app.put('/api/todos/:id', (req, res) => {
  try {
    const id = parseInt(req.params.id);
    const todo = todos.find(t => t.id === id);

    if (!todo) {
      res.status(404).send('Todo not found');

      return;
    }

    todo.task = req.body.task;

    res.status(200).json(todo);
  } catch (error) {
    console.error('failed to edit todo', error);
  }
});

app.delete('/api/todos/:id', (req, res) => {
  try {
    const id = parseInt(req.params.id);
    todos = todos.filter(t => t.id !== id);

    res.status(204).send();
  } catch (error) {
    console.error('failed to delete todo', error);
  }
});

The above defines four routes:

GET /api/todos: This route returns the list of todos
POST /api/todos: This route adds a new todo to the list
PUT /api/todos/🆔 Updates an existing todo based on the provided ID
DELETE /api/todos/🆔 Deletes a todo based on the provided ID

Finally we will end the server side by providing an index route to server the HTML file, this is done via the following code:



app.get('/', (req, res) => {
  res.sendFile(path.join(__dirname, 'index.html'));
});

app.listen(PORT, () => {
  console.log(`server is running on port ${PORT}`);
});

Phew! Thats the server finished, now we can start coding the frontend! 🥸

Setting Up the Frontend with HTMX

First create a directory called "public":



mkdir public

Create a new file in the public directory called "index.html" and add the following head tag:



<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>HTMX CRUD</title>
    <link href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" rel="stylesheet">
    <script src="https://unpkg.com/htmx.org@1.6.1"></script>
    <script src="https://unpkg.com/htmx.org@1.9.12/dist/ext/client-side-templates.js"></script>
  </head>

We will be using HTMX and the styling will be done via Bootstrap. Make sure to add the closing tags for each tag.

First we will create a container and create the modal and form that will be used to create a new todo item:



  <body>
    <div class="container">
      <h1 class="mt-5">Sample HTMX CRUD Application</h1>
      <div id="todo-list" hx-get="/api/todos" hx-trigger="load" hx-target="#todo-list" hx-swap="innerHTML" class="mt-3"></div>
      <button class="btn btn-primary mt-3" data-toggle="modal" data-target="#addTodoModal">Add Todo</button>
    </div>

    <!-- Add Todo Modal -->
    <div class="modal fade" id="addTodoModal" tabindex="-1" role="dialog" aria-labelledby=addTodoModalLabel" aria-hidden="true">
      <div class="modal-dialog" role="document">
        <div class="modal-content">
          <div class="modal-header">
            <h5 class="modal-title" id="addTodoModalLabel">Add Todo</h5>
            <button type="button" class="close" data-dismiss="modal" aria-label="Close">
              <span aria-hidden="true">&times;</span>
            </button>
          </div>

          <div class="modal-body">
            <form hx-post="/api/todos" hx-target="#new-todo-container" hx-swap="beforeend">
              <div class="form-group">
                <label for="task">Task</label>
                <input type="text" class="form-control" id="task" name="task" required />
              </div>
              <button type="submit" class="btn btn-primary">Add</button>
            </form>
          </div>
        </div>
      </div>
    </div>

    <div id="new-todo-container" style="display: none;"></div>

In the above we define a modal that contains a form for adding new todo items to the list.

The form uses HTMX attributes "hx-post" to specify the URL for adding todos, "hx-target" to specify where to inset the new todo, and "hx-swap" to determine how the response is handled.

Next we will add the modal for editing todos:



    <!-- Edit Todo Modal -->
    <div class="modal fade" id="editTodoModal" tabindex="-1" role="dialog" aria-labelledby="editTodoModalLabel" aria-hidden="true">
      <div class="modal-dialog" role="document">
        <div class="modal-content">
          <div class="modal-header">
            <h5 class="modal-title" id="editTodoModalLabel">Edit Todo</h5>
            <button type="button" class="close" data-dismiss="modal" aria-label="Close">
              <span aria-hidden="true">&times;</span>
            </button>
          </div>
          <div class="modal-body">
            <form id="editTodoForm">
              <div class="form-group">
                <label for="editTask">Task</label>
                <input type="text" class="form-control" id="editTask" name="task" required />
              </div>
              <button type="submit" class="btn btn-primary">Save</button>
            </form>
          </div>
        </div>
      </div>
    </div>

The above modal is similiar to the add modal but will be used for editing existing todos.
Note this time it does not contain HTMX attributes because we will handle the form submission with JavaScript.

Next we will use a HTMX template to display the todos in a Bootstrap card:



    <!-- Todo Template -->
    <script type="text/template" id="todo-template">
      <div class="card mb-2" id="todo-{{id}}">
        <div class="card-body">
          <h5 class="card-title item-task">{{task}}</h5>
          <button class="btn btn-warning" onclick="openEditModal('{{id}}', '{{task}}')">Edit</button>
          <button class="btn btn-danger" onclick="deleteTodo('{{id}}')">Delete</button>
        </div>
      </div>
    </script>

In the above script we define a HTML template for displaying each todo item. The template uses placeholders that are in braces, this will be replaced with actual data.

Finally add the JavaScript to handle various functions:



    <script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.5.3/dist/umd/popper.min.js"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>

    <script>
      function renderTodoItem(todo) {
        const template = document.getElementById('todo-template').innerHTML;

        return template.replace(/{{id}}/g, todo.id).replace(/{{task}}/g, todo.task);
      }

      function openEditModal(id, task) {
        const editForm = document.getElementById('editTodoForm');
        editForm.setAttribute('data-id', id);
        document.getElementById('editTask').value = task;
        $('#editTodoModal').modal('show');
      }

      function deleteTodo(id) {
        fetch(`/api/todos/${id}`, {
          method: 'DELETE'
        })
        .then(() => {
          document.querySelector(`#todo-${id}`).remove();
        });
      }

      document.addEventListener('htmx:afterRequest', (event) => {
        if (event.detail.requestConfig.verb === 'post') {
          document.querySelector('#addTodoModal form').reset();
          $('#addTodoModal').modal('hide');

          const newTodo = JSON.parse(event.detail.xhr.responseText);
          const todoHtml = renderTodoItem(newTodo);

          document.getElementById('todo-list').insertAdjacentHTML('beforeend', todoHtml);

          event.preventDefault();
        } else if (event.detail.requestConfig.verb === 'put') {
          $('#editTodoModal').modal('hide');
        }
      });

      document.addEventListener('htmx:afterSwap', (event) => {
        if (event.target.id === 'todo-list') {
          const todos = JSON.parse(event.detail.xhr.responseText);

          if (Array.isArray(todos)) {
            let html = '';

            todos.forEach(todo => {
              html += renderTodoItem(todo);
            });

            event.target.innerHTML = html;
          } else {
            const todoHtml = renderTodoItem(todos);
            event.target.insertAdjacentHTML('beforeend', todoHtml);
          }
        }
      });

      document.getElementById('editTodoForm').addEventListener('submit', function (event) {
        event.preventDefault();

        const id = event.target.getAttribute('data-id');
        const task = document.getElementById('editTask').value;

        fetch(`/api/todos/${id}`, {
          method: 'PUT',
          headers: {
            'Content-Type': 'application/json'
          },
          body: JSON.stringify({ task })
        })
        .then(response => response.json())
        .then(data => {
          const todoHtml = renderTodoItem(data);
          document.querySelector(`#todo-${id}`).outerHTML = todoHtml;
          $('#editTodoModal').modal('hide');
        })
        .catch(error => console.error(error));
      });
    </script>
  </body>
</html>

In the above:

renderTodoItem(todo): Renders a todo item using the previously defined template
openEditModal(id, task): Opens the modal to edit the todo
deleteTodo(id): Deletes a todo item
Event listeners handle after-request and after-swap events for HTMX to manage the modal states and update the DOM.

Done! Next we can finally run the server! 😆

Running the Application

To run the application, open your terminal and navigate to the project directory. Start the server with the following command:



node src/server.js

Open your browser and navigate to "http://localhost:3000". You should see your CRUD application running. You can add, edit and delete tasks, and the changes will be reflected without reloading the page. 👀

Conclusion

In this tutorial I have shown you how to build a simple CRUD application using Express and HTMX. This application allows you to add, view, edit and delete tasks without the need for any page reloading. We've used Bootstrap for styling and HTMX for handling AJAX requests. By following this tutorial, you should now have a good understanding of how to build a CRUD application with Express and HTMX.

Feel free to try implement a database to store the todos and improve on this example!

As always you can find the code on my Github:
https://github.com/ethand91/htmx-crud

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Creating A GRPC Endpoint With Rust

Ethan — Thu, 04 Apr 2024 10:22:39 +0000

Introduction

Hello! 😎

In this tutorial, I will show you how to use Rust and Proto to create a simple hello world endpoint using gRPC. 👀

Requirements

Rust installed

Creating The Project

First, we need to create the project! Assuming you already have Rust installed, just run the following command to create a new rust application:

cargo new rustgrpc && cd rustgrpc

Feel free to change the app name; the above command will create a new rust application and change directories to the application directory.

Next, we need to declare the dependencies that will be used for this project. Open up the "Cargo.toml" file and replace the contents with the following:

[package]
name = "rust"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
tonic = "0.7"
tokio = { version = "1", features = ["full"] }
prost = "0.10"

[build-dependencies]
tonic-build = "0.7"

This example makes use of Tonic, which is a rust implementation of gRPC. It is fast, high performance, open source, general RPC framework that puts mobile and HTTP/2 first! Now that the dependencies have been defined, we can now move on to actually coding the application!

Creating The Proto File

First, we will create the proto file to handle the request and response messages. A proto file helps the Client and the Server understand the messages that are sent and received.

Create a new directory called "protos" in your current directory and create a new file called "hello.proto". Fill the file with the following contents:

syntax = "proto3";

package hello;

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

Nothing too complicated; the syntax declares the version of proto we want to use, in this case, the latest version which is proto3. The package declares the namespace, in this case, hello. Next, we define the service which takes a HelloRequest request (name) and returns HelloReply (message), to keep things simple only one variable is declared for each.

Next, we need to create a build file to help build the project.

Creating The Build File

Now that the proto file has been defined next we need to create a build file to build the project. Create a new file in the current directory called "build.rs" and populate it with the following contents:

fn main() -> Result<(), Box<dyn std::error::Error>> {
  tonic_build::compile_protos("protos/hello.proto")?;
  Ok(())
}

Again nothing too complicated here; we tell tonic where to find the above proto files so that it can be built.

Next, we will write the final code for the example project.

Writing The Main Code

Finally, we will write the code for the main function of this application. Open up "src/main.rs" and delete the boilerplate code.

First, we will declare the imports and modules like so:

use tonic::{transport::Server, Request, Response, Status};

pub mod hello {
  tonic::include_proto!("hello");
}

use hello::{greeter_server::{Greeter, GreeterServer}, HelloReply, HelloRequest};

The first line imports things from the tonic module, after that we declare the hello module which includes the hello.proto that we wrote earlier, finally, we import the needed things from the hello proto file.

Next, we need to define a struct that will implement our service, add the following code below the above:

#[derive(Default)]
pub struct MyGreeter {}

Next, we need to create an implementation in order to handle the hello request and response, add the following to the above:

#[tonic::async_trait]
impl Greeter for MyGreeter {
  async fn say_hello(
    &self,
    request: Request<HelloRequest>
  ) -> Result<Response<HelloReply>, Status> {
    let reply = hello::HelloReply {
      message: format!("Hello {}!", request.into_inner().name)
    };
    Ok(Response::new(reply))
  }
}

The function takes a HelloRequest and returns a HelloReply, in simple terms "Hello (Request name)".

Finally, we need to define a main function that will start the server and listen for requests add the following to the above:

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
  let addr = "[::1]:50051".parse()?;
  let greeter = MyGreeter::default();

  Server::builder()
    .add_service(GreeterServer::new(greeter))
    .serve(addr)
    .await?;

  Ok(())
}

Done!

Now all we need to do now is build and run the project! 🥳

Running The Application

Finally, all we need to do is build and run the project; the project can be built via the following command:

cargo build

To run the application use the following command: (you might need grpcurl if you don't have it already)

cargo run

Now that the server is running, you can test the request with the following command (make sure to run the command in the same directory as the application):

grpcurl -plaintext -proto protos/hello.proto -d '{"name": "World"}' [::]:50051 hello.Greeter/SayHello

If all has gone right, you should see the response along the lines of "Hello World!" in the terminal! Like the following image:

Conclusion

Here I have shown a basic example of using Rust and Proto to create a simple hello world endpoint. I hope this tutorial has helped you the same I had fun writing it!

As always, you can find the example code for this tutorial at my Github:
https://github.com/ethand91/rust-grpc

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Simple Introduction To HTMX

Ethan — Sun, 17 Mar 2024 06:51:25 +0000

Introduction

Hello! 😎

In the ever-evolving landscape of web development, where the lines between server and client-side scripting blur, a tool named HTMX has emerged.

It offers simplicity for those weary of the complexity required by heavy JavaScript frameworks. HTMX allows you to harness the power of AJAX, WebSockets, and server-sent events with nothing but HTML, enabling you to create web pages with minimal effort.

What is HTMX?

HTMX is a small, yet powerful JavaScript library that extends HTML's capabilities enabling dynamic updates to the web page without needing to write JavaScript. Inspired by the principles of progressive enhancement and unobtrusive JavaScript. HTMX allows you to create modern, user-friendly web applications. It's designed to be an ally that empowers your HTML to perform tasks you never thought possible, while keeping the complexity at bay.

Imagine a world where you can update parts of your webpage, submit forms, and react to server-side events with straightforward HTML attributes. That's the realm HTMX offers to transport you to, a place where the web is fast, responsive and elegant, all without a single line of JavaScript. 😆

Advantages of HTMX

HTMX has a variety of advantages including but not limited to the following:

Ease of Use: It's like HTML on steroids, but easier to manage than traditional JavaScript frameworks
Lightweight: No large downloads, allowing your site to be speedy
Progressive Enhancement: Works right on top of your existing HTML, making progressively enhanced

Creating a Simple Login Form

First we will create a simple backend PHP file to handle the request. Create a new file called "login_handler.php" and populate it with the following:



<?php
$username = $_POST['username'];

echo '<p>Welcome, ' . htmlspecialchars($username) . '</p>';
?>

Since the focus is on HTMX the above is nothing to complicated. All it does is display what the user entered into the username field. Normally you would want authentication etc. But I will leave that challenge to you.

To demonstrate HTMX lets create a simple login form that will be styled using bootstrap. Create a new file called "index.html" and populate it with the following, don't worry I will explain what the code does below.



<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <title>Login Form</title>
        <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">
    </head>

    <body>
        <div class="container mt-5">
            <div class="row justify-content-center">
                    <div class="col-md-6">
                        <form hx-post="login_handler.php" hx-target="#login-response" hx-swap="outerHTML" class="card p-4">
                        <div class="mb-3">
                            <label for="username" class="form-label">Username:</label>
                            <input type="text" id="username" name="username" required class="form-control">
                        </div>

                        <div class="mb-3">
                            <label for="password" class="form-label">Password:</label>
                            <input type="password" id="password" name="password" required class="form-control">
                        </div>

                        <button type="submit" class="btn btn-dark">Log In</button>
                    </form>
                    <div id="login-response"></div>
                </div>
            </div>
        </div>
    </body>

    <script src="https://unpkg.com/htmx.org"></script>
</html>

In the header we import the bootstrap stylesheet.
After that we create the login form hx-post tells the form to submit its data to "login_handler.php" which is the back end via POST request.

The hx-target and hx-swap attributes work together to update the content of the page with the server's response. When the user clicks on the login button, HTMX leaps into action sending the form data to the server and displaying the response within the "login-response" div, all without refreshing the page. 😁

Running The Example

To display the above page you can use the following php command:



php -S localhost:8000

This will fire up a simple HTTP server, next access the page via:
http://localhost:8000

Try entering a username and password and click the login button. The response will instantly appear.

Conclusion

Here I have given a very simple introduction to HTMX. I hope you have learned something from this. I am also very new to HTMX but I do enjoy learning it.

If you have any good resources please feel free to share them.

As always you can find the example code on my Github:
https://github.com/ethand91/simple-htmx-form

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Implementing Passport With Google OAuth2

Ethan — Sun, 03 Mar 2024 09:09:08 +0000

Introduction

Hello! 😎

In this tutorial I will continue on with the previous tutorial and add authentication with Google OAuth2 with passport. ☺️

Requirements

Code from the previous tutorial
Ngrok (Or a global domain)
Google Developer Console OAuth2 client ID and secret (https://console.cloud.google.com/)

Creating The OAuth2 Client Id And Secret

First if you don't already have a client id and secret head on over to the Google Developer Console (https://console.cloud.google.com)

Create a new project naming it anything you want (auth-test) for example and then in the search bar search for OAuth and then select credentials like so:

Next click on Create Credentials and select OAuth client id.

Then select Web Application, give it a random name or just leave the default name.

Next we need to configure the redirect URIs. I will show you how to do this using Ngrok but if you have your own domain feel free to use that.

If you haven't already install a tool called Ngrok, this tool is useful as it allows us to server local addresses as a global address. (Note since it allows access from outside always stop the command after use).

With ngrok installed just run the below command to get a global domain:

ngrok http 3000

Now we can set the redirect URI, add the ngrok domain you got from the above command and add the following to the end of the URI:

/auth/google/callback

Don't worry about the link just yet as we will support it later.

Once all needed fields are filled out click on Save.

You should be presented with the client id and secret. Make sure to make a note of them.
Now that thats done we can now add Google OAuth2 to our local passport application. 🙂

Setting Up The Database

First what we need to do is alter the users database to allow Google. Since google does not use a passport we will need to allow null passwords, we also need to add a google id field to store the id.

Create a new file called "sql/google.sql" and populate it with the following:

ALTER TABLE users
ADD COLUMN googleId VARCHAR(255);

ALTER TABLE users
ALTER COLUMN password DROP NOT NULL;

The above file adds a new googleId column and alters the password column to allow null passwords.

To execute the above sql log in to postgreSQL database and run the following command:

\i sql/google.sql

This should modify the database. Now we can move on to the TypeScript code.

Adding Google Auth

First we will need to add the Google OAuth2 client id and secret to the .env file.

GOOGLE_CLIENT_ID=google-client-id
GOOGLE_CLIENT_SECRET=google-client-secret

Replace the above with your real id and secret.

Next we will add the needed modules with the following command:

yarn add passport-google-oauth20
yarn add -D @types/passport-google-oauth20

First we will edit the database code to allow searching and inserting users into the database when they are authed by google, open up "src/db/psql.ts" and add the following function. (Make sure to not forget exporting it)

const fetchOrCreateByGoogleId = async (googleId: string, email: string): Promise<any | Error> => {
    const res = await pool.query('SELECT * FROM users WHERE googleId = $1', [googleId]);

    if (res.rows.length) {
        return res.rows[0];
    }

    const newUser = await pool.query('INSERT INTO users (googleId, email) VALUES ($1, $2) RETURNING *', [googleId, email]);

    return newUser;
};

The above returns the user if they are found, if not it creates a new user and returns the new user. Don't forget to export it from the index file as well.

Next we will create a new passport strategy for google. Create a new file called "src/routes/auth/passport/google.ts" and populate it with the following code:

import passport from 'passport';
import { Strategy as GoogleStrategy, Profile } from 'passport-google-oauth20';
import { fetchOrCreateByGoogleId, deserializeUserById } from './../../../db';
require('dotenv').config();

passport.use(new GoogleStrategy({
    clientID: (process.env.GOOGLE_CLIENT_ID as string),
    clientSecret: (process.env.GOOGLE_CLIENT_SECRET as string),
    callbackURL: '/auth/google/callback'
}, async (accessToken: string, refreshToken: string, profile: Profile, done: (error: any, user?: any) => void) => {
    try {
        const user = await fetchOrCreateByGoogleId(profile.id, profile.emails![0].value);

        return done(null, user);
    } catch (error) {
        console.error(error);

        return done(error);
    }
}));

passport.serializeUser((user: any, done) => {
    done(null, user.rows[0].id);
});

passport.deserializeUser(async (id: number, done) => {
    try {
        const user = await deserializeUserById(id);

        done(null, (user as Express.User));
    } catch (error) {
        done(error, null);
    }
});

export { passport };

The above pretty much does the same as the local strategy but this time with google profile. (you may need to allow access to profile in the developer console)

Next we need to add new express routes to handle google auth. Open the file "src/routes/auth/auth.ts" and add the following two new routes:

router.get('/auth/google', googlePassport.authenticate('google', { scope: ['profile', 'email'] }));

router.get('/auth/google/callback', googlePassport.authenticate('google'), (req: Request, res: Response) => {
    res.send('Google auth login successful'); 
});

One is for the authentication and one for when the authentication is successful.

Finally we need to initialize the google passport, open up "src/app.ts" and add the following:

app.use(googlePassport.initialize());
app.use(googlePassport.session());

After that you will need to export both passports from some index files like so:

export { passport as localPassport } from './passport';
export { passport as googlePassport } from './google';

Make sure to change the variables to the above for each passport and then run the following command to build:

yarn build

The build should pass, now we can actually test the application. 😀

Running The Application

To run the application simply run the following command:

node dist/app.js

Now access the ngrok url with the path:

/auth/google

Well done you have now added Google auth to the previous application! 😆

Conclusion

In this tutorial I have shown you how to add google OAuth2 to the previous application.
I wasn't sure about seperating the passport files but it allows me to copy the same file into another application.

I hope this tutorial has taught you something new.

As always you can find the source code for the above on my Github:
https://github.com/ethand91/passport-local-demo

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch

Implementing Passport Local With Yarn, Typescript, Express and PostgreSQL

Ethan — Sun, 25 Feb 2024 09:25:03 +0000

Introduction

Hello! 😎

In this tutorial I will show you how to implement passport local into a new Nodejs project. This tutorial will use PostgreSQL for the database but feel free to change it to a database of your choice. ☺️

Requirements

Basic knowledge of Nodejs
PostgreSQL installed

Initialize The Project

First we need to initialize the project, change the current directory to somewhere of your choice and run the following command to initialize the project:



yarn init -y

This will create a simple package.json file in the current directory. Next we will need to install the dependencies that will be used for this project, to install them run the following command:



yarn add express bcrypt body-parser connect-pg-simple dotenv express-session passport passport-local pg

The above command will install the dependencies needed to run the project, since this project will be using TypeScript, we will need to install the type files etc. To ensure they are only installed for a development environment we will use yarn with the "-D" flag, as follows:



yarn add --dev @types/connect-pg-simple @types/pg @types/bcrypt @types/express @types/express-session @types/node @types/passport @types/passport-local typescript

Now that the dependencies have been taken care of we need to create a "tsconfig" file to handle converting TypeScript into JavaScript, to initialize this file simply run the following command:



npx tsc --init

Once the file is created, open it up and populate it with the following config:



{
    "compilerOptions": {
        "target": "es6",
        "module": "commonjs",
        "esModuleInterop": true,
        "strict": true,
        "outDir": "./dist"
    },
    "include": ["src/**/*"],
    "exclude": ["node_modules"]
}

The above is nothing too complicated, it initializes strict mode and converts all files under the src directory into JavaScript files outputted to the dist directory. Pretty basic configuration but feel free to customize it to your needs.

Now that the project has been initialized we can now create the sql files that will be used in our PostgreSQL database. 👀

Configuring The Database

Next up we need to create the tables that will be used for the database, if you have not done so yet create a new database in PostgreSQL, this can be done in the psql shell with the following command:



create database app;

This will create a new database called "app".

The first table we will create will handle the express sessions information, create a new directory in the current directory called "sql" and create a new sql file called "sql/session.sql" and populate it with the following:



CREATE TABLE session(
    sid VARCHAR(255) NOT NULL COLLATE "default",
    sess json NOT NULL,
    expire TIMESTAMP(6) NOT NULL
)

WITH (OIDS=FALSE);
ALTER TABLE session ADD CONSTRAINT session_pkey PRIMARY KEY ("sid") NOT DEFERRABLE INITIALLY IMMEDIATE;

This file will create a new table to handle the express sessions we will be implementing later.

We also need another table to handle users that will authenticate using our local passport strategy, create a new file called "sql/user.sql" and populate it with the following:



CREATE SEQUENCE users_id_seq;

CREATE TABLE users (
    id INTEGER NOT NULL DEFAULT nextval('users_id_seq'),
    email VARCHAR(255) NOT NULL,
    password VARCHAR(255) NOT NULL,
    PRIMARY KEY (id),
    UNIQUE (email)
);

ALTER SEQUENCE users_id_seq OWNED BY users.id;

CREATE INDEX ON users (email);

The above will create a table to handle users, we also want the email to be unique so another user cannot register with the same email address.

To create the tables enter the psql shell with the following command and then run \i with the location to the above sql files like so:



psql -d app

\i sql/session.sql
\i sql/user.sql

This should create both of the tables without any errors.

Now that the database is taken care of we can now start writing the code for our application.

Writing The Database Code

First off we will write the code for connecting and interacting with the PostgreSQL database. Create a new directory called "src/db". Next create a file called "src/db/psql.ts" and add the following imports:



import { Pool } from 'pg';
import session from 'express-session';
import pgSession from 'connect-pg-simple';
import bcrypt from 'bcrypt';

Next to initialize the PostgreSQL pool and the session store add the following below the above imports:



const pgSessionStore = pgSession(session);

const pool = new Pool({
    connectionString: process.env.DATABASE_URL
});

The DATABASE_URL is a reference to the psql link that will be used to connect to our database, don't worry about it too much now.

Next we will create a function to test the connection to the database because if we can't connect, there isn't much point for the application to run:



const testConnection = (): Promise<void> => {
    return new Promise((resolve, reject) => {
        pool.connect((error, client, release) => {
            if (error || !client) {
                reject(new Error('Failed to connect to the database'));
            }

            release();
            resolve();
        })
    });   
};

The above code tests if we can connect to the PostgreSQL database, if the connection is ok it resolves, if the connection is not ok it will reject with a connection failure error.

Next we need a function to insert a new user into the database, this will be done when the user has registered with the application. The code for this is simply as follows:



const insertUser = async (email: string, password: string): Promise<void> => {
        const hashedPassword = await bcrypt.hash(password, 10);

        await pool.query('INSERT INTO users (email, password) VALUES ($1, $2)', [email, hashedPassword]);
};

Always make sure to hash the passwords before inserting the data into the database, pretty small but powerful function.

Next we will need a function to authenticate the user, in this case via email:



const authenticateUserByEmail = async (email: string, password: string): Promise<Express.User | Error> => {
    const res = await pool.query('SELECT * FROM users WHERE email = $1', [email]);

    if (res.rows.length) {
        const user = res.rows[0];
        const match = await bcrypt.compare(password, user.password);

        if (match) {
            return user;
        } else {
            throw new Error('Incorrect email and/or password');
        }
    } else {
        throw new Error('User not found');
    }
};

The above checks if the user actually exists in the database and if so makes sure the passwords match up, if everything is ok the user is returned, if something went wrong an error is thrown. Such as is the user is not found or the passwords don't match up.

The final function we need is one that will help deserialize the user, this function is also pretty straight forward:



const deserializeUserById = async (id: number): Promise<Express.User | Error> => {
    const res = await pool.query('SELECT * FROM users WHERE id = $1', [id]);

    if (res.rows.length) {
        return res.rows[0];
    } else {
        throw new Error('User was not found');
    }
};

If the user exists return the user, if they don't exist throw an error. This will be explained in more detail later on.

Finally we need to export the needed variables and functions:



export {
    pgSessionStore,
    pool,
    testConnection,
    insertUser,
    authenticateUserByEmail,
    deserializeUserById
};

If you're like me and like to have an "index.ts" file in each directory you can create a new file called "src/db/index.ts" and have it export what is needed:



export {
    pgSessionStore,
    pool,
    testConnection,
    insertUser,
    authenticateUserByEmail,
    deserializeUserById
} from './psql';

Exporting what is needed from an index file also means you only need to import one file to include everything which is neat.

Thats the code to handle the database completed, next up we will implement the code needed for passport. 😄

Writing The Passport Code

Next we will write the code for the passport module that will alway us to authenticate the user.

Create a new directory to handle the routes and the authentication side of things using the following command:



mkdir -p src/routes/auth/passport

Create a new file called "src/routes/auth/passport/passport.ts" and add the following imports:



import passport from 'passport';
import { Strategy as LocalStrategy } from 'passport-local';
import { authenticateUserByEmail, deserializeUserById } from './../../../db';

The above imports the needed modules for this file including some functions from the file we created in the previous section.

The first thing we will implement is the local strategy that will be used to authenticate the users:



passport.use(new LocalStrategy({
        usernameField: 'email',
        passwordField: 'password'
    },
    async (email, password, done) => {
        console.log('local');
        try {
            const user = await authenticateUserByEmail(email, password);

            return done(null, (user as Express.User));
        } catch (error) {
            return done(error);
        }
    }
));

-
Since we will using email and not username we specify email in the usernameField. After that we simple authenticate the user, if they exist and the passwords match. If ok the user will be logged in, if not ok an error will be returned.

Finally we need two functions, one to serialize the user and one to deserialize the user, nothing too complicated the functions are simply the following:



passport.serializeUser((user: Express.User, done) => {
    done(null, user.id);
});

passport.deserializeUser(async (id: number, done) => {
    try {
        const user = await deserializeUserById(id);

        done(null, (user as Express.User));
    } catch (error) {
        done(error, null);
    }
});

export { passport }

As you can see the last thing we do is export the passport. Also create a new file called "src/routes/auth/passport/index.ts" and export the passport from there as well:



export { passport } from './passport';

Next we will create the express routes to handle registration and login.

Writing The Express Code

Now that we have our passport code completed we need to actually use it, this is what we will be implemented in this section.

First we need to create two routes, one to register a new user and one to login the user. Create a new file called "src/routes/auth/auth.ts" and add the following imports:



import { Request, Response, NextFunction, Router } from 'express';
import { passport } from './passport';
import { insertUser } from './../../db';

We will then create a Router object and implement the route for the user to use to register a new account:



const router = Router();

router.post('/register', async (req: Request, res: Response) => {
    try {
        if (req.body.password !== req.body.passwordConfirmation) {
            res.status(401).send('Passwords do not match');

            return;
        }

        await insertUser(req.body.email, req.body.password);

        res.status(201).send('User was created');
    } catch (error) {
        res.status(500).send('Error registering user');
    }
});

The above checks if the passwords match up, if ok it will try to insert the user into the PostgreSQL database, if this is successful the user has their data stored in the database and is able to use the next login route to login.

Next we need a route so that the user can login:



router.post('/login', passport.authenticate('local'), (req: Request, res: Response) => {
    console.log('login request');
    res.send('Logged in successfully');
});

export { router };

The above route uses our local passport strategy to authenticate the user, if successful the user is logged in and can access the protected routes that we will create later. We also export the router object.

We also create a helper middleware function that will be used to ensure that the user is logged in and is allowed to access certain routes. Create new directory called "src/routes/auth/helpers" and create a new file called "src/routes/auth/helpers/helpers.ts" and populate it with the following:



import { Request, Response, NextFunction } from 'express';

const ensureAuthenticated = (req: Request, res: Response, next: NextFunction) => {
    if (req.isAuthenticated()) {
        return next();
    }

    res.status(401).send('Unauthorized');
};

export { ensureAuthenticated }

The above makes sure that the user is authenticated and is allowed to access the route that uses this middleware.

Create a new file called "src/routes/auth/helpers/index.ts" and also export the function there:



export { ensureAuthenticated } from './helpers';

Finally create the "src/routes/auth/index.ts" for the auth module:



export { router } from './auth';
export { passport } from './passport';
export { ensureAuthenticated } from './helpers';

In order to test out authentication we will need a route that the user needs to be logged in to see.

Create a new routes directory called "src/routes/protected" and create a new file called "src/routes/protected/protected.ts" and add the following:



import { Request, Response, NextFunction, Router } from 'express';
import { ensureAuthenticated } from './../auth';

const router = Router();

router.get('/protected', ensureAuthenticated, (req: Request, res: Response) => {
    res.send('You are logged in, so you can see this');
});

export { router }

The above route will only be accessible to users that have logged in successfully, next create a new file called "src/routes/protected/index.ts" and export the above:



export { router } from './protected';

Now all we need to do to finish this section is export our express routes from "src/routes/index.ts":



export { router as authRoutes, passport } from './auth';
export { router as protectedRoutes } from './protected';

Here we export both our auth routes and our protected routes.

Now all we need to do is put everything together. 😉

Writing The Main Code

The final part, all we have to do now it put everything together. Create a new file called "src/app.ts" and add the following imports:



import express from 'express';
import bodyParser from 'body-parser';
import session from 'express-session';
import { authRoutes, passport, protectedRoutes } from './routes';
import { pool, pgSessionStore, testConnection } from './db';
require('dotenv').config();

The above imports all the stuff we need to code the main application file. Next we decalre a new interface for the user like so:



declare global {
    namespace Express {
        interface User {
            id: number;
            email: string;
            password: string
        }
    }
}

Next we will initialize the express application via the following:



const app = express();

app.use(bodyParser.json());

The following will initialize express-session, using the session table we created at the start to store the session information:



app.use(session({
    store: new pgSessionStore({
        pool,
        tableName: 'session'
    }),
    secret: (process.env.SESSION_SECRET as string),
    resave: false,
    saveUninitialized: false,
    cookie: {
        maxAge: 30 * 24 * 60 * 60 * 1000,
        httpOnly: true,
        secure: process.env.NODE_ENV === 'production',
        sameSite: 'lax'
    }
}));

Next we will initialize passport and the routes that we created in the previous section:



app.use(passport.initialize());
app.use(passport.session());

app.use(authRoutes);
app.use(protectedRoutes);

We also want to make sure that we can connect to the PostgreSQL database, this can be done via the following:



testConnection()
    .then(() => {
        console.log('Connection to database success');
    }).catch(() => {
        console.error('Failed to connect to database');
        process.exit(1);
    });

Finally all we need to do is start the express server:



app.listen(3000, () => {
    console.log('Server is running on port 3000');
});

Phew! Now we have coded the application.

Last but not least create a new file called ".env" and populate it with the following making sure to replace the information with your own information:



DATABASE_URL=postgresql://ethan:ethan99@localhost:5432/users
SESSION_SECRET=ethanrules

To build the TypeScript project run the following command:



yarn build

To start the express server run the following command:



node dist/app.js

The server should not be running.

Testing The Application

Next we can actually call the routes to register and login.

You can use something like PostMan, or you can be like me and just use curl.

To register a user run the following command:



curl -X POST http://localhost:3000/register \
-H "Content-Type: application/json" \
-d '{"email": "user2@example.com", "password": "password", "passwordConfirmation": "password"}'

The above command should register a new user, also if you try running the above command a second time it should fail due to the email being unique.

To login as the above user run the following command:



curl -X POST http://localhost:3000/login \ost:3000/login \
-H "Content-Type: application/json" \
-d '{"email": "user2@example.com", "password": "password"}' \
-c cookies.txt

The above should give you a "Logged in successfully" response, also try using a non existant email/password the request should fail.

Finally if you try to access the protected url with the cookies file created with the above you should get an ok response:



curl -X GET http://localhost:3000/protected \
-b cookies.txt

Try the above route without being authenticated and the request should fail.

Done! Well done for following the tutorial to the end!

Conclusion

Here I have shown how you can implement the passport local strategy into a expressjs nodejs application using Yarn, TypeScript and PostgreSQL.

I hope this helps you as much as I had fun writing it. Also feel free to try and implement logout!

As always you can find the code for this porject on my Github:
https://github.com/ethand91/passport-local-demo

Happy Coding! 😎

Like my work? I post about a variety of topics, if you would like to see more please like and follow me.
Also I love coffee.

If you are looking to learn Algorithm Patterns to ace the coding interview I recommend the [following course](https://algolab.so/p/algorithms-and-data-structure-video-course?affcode=1413380_bzrepgch