Forem: Alin Climente

How to add Docling Serve API in your Docker Compose Stack

Alin Climente — Sun, 15 Mar 2026 07:03:46 +0000

Docling is good at taking a file or url and converting it to markdown. You have the option to use it as an API, which is pretty nice.

Here is how you can add it to your docker compose stack:

  yourappname-docling-serve:
    image: quay.io/docling-project/docling-serve-cpu
    container_name: yourappname-docling-serve
    restart: unless-stopped
    ports:
      - 5001:5001
    env_file:
      - .env

In the env file or environment you can choose to enable the UI for tests: DOCLING_SERVE_ENABLE_UI=1

If you have nvidia available you can use a different docker image than quay.io/docling-project/docling-serve-cpu.

In production make sure to remove this part which exposes the port to outside traffic.

    ...
    ports:
      - 5001:5001
    ...

Here is an example on how you can make a request to docling serve api:


def get_markdown_from_facsimil_image_url(url: str):

    url = "http://localhost:5001/v1/convert/source"

    payload = {
        "options": {
            "from_formats": [
                "docx",
                "pptx",
                "html",
                "image",
                "pdf",
                "asciidoc",
                "md",
                "xlsx",
            ],
            "to_formats": ["md", "json", "html", "text", "doctags"],
            "image_export_mode": "placeholder",
            "ocr": True,
            "force_ocr": True,
            "ocr_engine": "easyocr",
            "ocr_lang": ["ro"],
            "pdf_backend": "dlparse_v2",
            "table_mode": "accurate",
            "abort_on_error": False,
        },
        "sources": [{"kind": "http", "url": url}],
    }

    response = requests.post(
        url, json=payload, headers={"X-Api-Key": DOCLING_SERVE_API_KEY}
    )

    data = response.json()

    return data

More endpoints are documented at: http://localhost:5001/docs

Docling serve is a FastAPI api so you can view the code and see what else is possible. For example: you can set RQ for processing jobs by setting env: DOCLING_SERVE_ENG_RQ_REDIS_URL=redis://:password@docker-compose-redis:6379/1 and setting DOCLING_SERVE_ENG_KIND=rq (by default is local).

More envs here: docling settings

Meilisearch Python setup

Alin Climente — Mon, 09 Mar 2026 15:18:51 +0000

First create a docker compose service:

Name the service however you want. Checkout the latest version of meilisearch (1.16 may be old by now).

  chatcodfiscal-meilisearch:
    image: getmeili/meilisearch:v1.16
    container_name: chatcodfiscal-meilisearch
    restart: unless-stopped
    expose:
       - "7700"
    env_file:
      - ./webapp/.env.prod
    environment:
      - MEILI_HTTP_ADDR=0.0.0.0:7700
      # uncomment this for prod if new index is created
      # - MEILI_IMPORT_DUMP=/meili_data/dumps/backup.dump
    volumes:
      # uncomment this for prod if new index is created
      # - ./webapp/data/meili_data/dumps/backup.dump:/meili_data/dumps/backup.dump
      - ./webapp/data/meili_data:/meili_data
    networks:
      - web

In your env file add MEILI_MASTER_KEY and MEILI_HTTP_ADDR (use meilisearch service name from docker compose if the api is called within docker network).

MEILI_MASTER_KEY=super-secret-key
# uncomment this for prod
# MEILI_HTTP_ADDR=http://chatcodfiscal-meilisearch:7700
# uncomment this for local
MEILI_HTTP_ADDR=http://0.0.0.0:7700

Index some documents:

import meilisearch
from django.conf import settings
from webapp.logger import log

from ingestor.models import ChunksModel

LEGAL_SYNONYMS = {
    # -------------------------------------------------------------
    # I. IMPOZITE, CONTRIBUȚII ȘI DECLARAȚII FISCALE
    # -------------------------------------------------------------
    "tva": [
        "taxa pe valoarea adaugata",
        "platitor de taxa",
        "inregistrare tva",
        "cod de inregistrare in scopuri de tva",
    ],
    "impozit": ["taxa", "bir", "obligatii fiscale", "de dat la stat"],
    "cif": ["cod de identificare fiscala", "cod fiscal"],
    "cas": [
        "contributia de asigurari sociale",
        "pensii",
        "contributie pensii",
        "cota cas",
    ],
    "cass": [
        "contributia de asigurari sociale de sanatate",
        "sanatate",
        "asigurari sociale de sanatate",
        "cota cass",
    ],
    # etc - You can add some usual synonyms useful for similar search.
}


def index_chunks():
    client = meilisearch.Client(settings.MEILI_HTTP_ADDR, settings.MEILI_MASTER_KEY)

    index_uid = "chunks"
    log.info(f"Deleting existing index '{index_uid}'...")
    try:
        client.delete_index(index_uid)
    except Exception:
        pass

    index = client.index(index_uid)

    log.debug(f"Configuring index settings for {index_uid}...")

    index.update_filterable_attributes(["nume_fisier"])

    index.update_searchable_attributes(["nume_sursa", "text_summary", "text_markdown"])

    log.debug("Updating synonyms dictionary...")
    synonyms_task = index.update_synonyms(LEGAL_SYNONYMS)

    client.wait_for_task(synonyms_task.task_uid)

    log.debug("Fetching data from Django DB...")
    chunks = ChunksModel.objects.all().iterator()

    documents = []
    for chunk in chunks:
        doc = {
            "id": str(chunk.pk),
            "nume_sursa": chunk.nume_sursa,
            "nume_fisier": chunk.nume_fisier,
            "text_markdown": chunk.text_markdown,
            "text_summary": chunk.text_summary,
        }
        documents.append(doc)

    log.debug(
        f"Prepared {len(documents)} documents. Sending to Meilisearch index '{index_uid}'..."
    )

    response = index.add_documents(documents)

    task_uid = response.task_uid

    log.info(f"Upload started. Task UID: {task_uid}")

    client.wait_for_task(task_uid)

    log.debug("Creating dump Meilisearch...")

    response = client.create_dump()

    client.wait_for_task(response.task_uid)

    log.success("DONE")

    return

Here ChunksModel is a django model source, but you can add whatever source you want.

Let's start searching!

Here we initialize meilisearch to make some searches. The chunksfts is what we will import where we need to do some searches. You can notice that we've made some pydantic models to parse the result from meilisearch.

import meilisearch
from pydantic import BaseModel
from webapp.settings import MEILI_HTTP_ADDR, MEILI_MASTER_KEY

meiliclient: meilisearch.Client = meilisearch.Client(MEILI_HTTP_ADDR, MEILI_MASTER_KEY)

chunksfts = meiliclient.index("chunks")


class FTSHit(BaseModel):
    id: int
    nume_sursa: str
    nume_fisier: str
    text_markdown: str
    text_summary: str


class FTSHits(BaseModel):
    hits: list[FTSHit]
    query: str

To search you'll just need to do this:

from .fts import chunksfts, FTSHits

meili_query = "Some query"
meili_options = {"limit": 20} # See more options on meilisearch docs

result = chunksfts.search(meili_query, meili_options)
fts_results = FTSHits(**result)

That's it! Now you've got full text search with Meilisearch.

Simple BOT blocker with Caddy and Django

Alin Climente — Wed, 17 Dec 2025 15:18:05 +0000

Tired of seeing this in your Django logs?

They do no harm because Django handles them properly based on allowed hosts env, but those bot requests pass the reverse-proxy and end up in the Python application - which is not as fast as a reverse proxy built in Go, C, C++, Zig or something else.

Here is how to stop those bot requests using just a django command which generates the Caddyfile template.

The generated Caddyfile will just block by default all routes that are not registered by Django and will block any hosts other that the one specified (ex: bot trying to curl on VPS IP).

The Caddyfile template

We'll use Django Templates to pass existing paths to Caddyfile template.

# Catch any request not matching your domain (like IP scans)
:80, :443 {
    abort
}

{{ app_url }} {

    # Create a named matcher for the host
    @strict_host {
        not host {{ app_url }}
    }
    # Abort if the host isn't exactly yours
    handle @strict_host {
        abort
    }

    # Enable high-performance compression
    encode zstd gzip

    # 1. Static Files (Update with path to staticfiles)
    # Handled first to ensure they are always accessible. 
    handle_path /static/* {
        root * /webapp/staticfiles
        file_server
    }

    # 2. Define Allowed Routes
    # This matcher includes all Django URLs plus your explicit SSE route.
    @allowed {

        path /favicon.ico

        path /robots.txt

        path /sitemap.xml

        path /sse-events*
        {% for route in routes %}
        path {{ route }}
        {% endfor %}
    }

    # 3. Handle Allowed Traffic
    handle @allowed {
        # SSE Sidecar (Specific route handling inside the allowed block)
        reverse_proxy /sse-events* docker-service-name-sse-sidecar:5687

        # Main Django App (Default for other allowed routes)
        reverse_proxy docker-service-name-app:8000
    }

    # 4. Fallback: Block everything else
    handle {
        respond "Access Denied" 403
    }
}

The Django Command

In any app create a new caddyfile_gen.py in your_app/management/commands folder.

Paste this code there in caddyfile_gen.py

from django.core.management.base import BaseCommand
from django.template.loader import render_to_string
from django.urls import get_resolver
from django.urls.resolvers import URLPattern, URLResolver



class Command(BaseCommand):
    help = "Generates a Caddyfile based on registered Django URLs"

    def handle(self, *args, **options):
        # You can set your domain name in env
        app_url = "chatcodfiscal.ro"

        resolver = get_resolver()
        raw_routes = self.get_urls(resolver.url_patterns)

        final_routes = set()
        for r in raw_routes:
            final_routes.add(r)
            # ex: "/chat/" -> adds "/chat"
            if r.endswith("/") and len(r) > 1:
                final_routes.add(r.rstrip("/"))

        # Remove duplicates
        clean_routes = sorted(list(set(final_routes)))

        context = {"app_url": app_url, "routes": clean_routes}

        # Assumes caddyfile.tpl is in your templates/components folder
        caddyfile_content = render_to_string("components/caddyfile.tpl", context)

        with open("Caddyfile", "w") as f:
            f.write(caddyfile_content)

        print("Caddyfile was created!")

    def get_urls(self, patterns, prefix=""):
        urls = []
        for pattern in patterns:
            if isinstance(pattern, URLPattern):
                path_str = str(pattern.pattern)
                full_path = prefix + path_str

                # 1. Remove Regex anchors
                full_path = full_path.replace("^", "").replace("$", "")

                # 2. Ensure it starts with /
                if not full_path.startswith("/"):
                    full_path = "/" + full_path

                if full_path.startswith("/admin"):
                    urls.append("/admin")
                    urls.append("/admin/*")
                    continue

                if "<" in full_path:
                    full_path = full_path.split("<")[0] + "*"

                urls.append(full_path)

            elif isinstance(pattern, URLResolver):
                # Recursively handle included URLconfs
                new_prefix = prefix + str(pattern.pattern)
                urls.extend(self.get_urls(pattern.url_patterns, new_prefix))

        return urls

A Caddyfile file will be generated and it will look similar to this one:


# Catch any request not matching your domain (like IP scans)
:80, :443 {
    abort
}

your-domain.com {

    # Create a named matcher for the host
    @strict_host {
        not host your-domain.com
    }
    # Abort if the host isn't exactly yours
    handle @strict_host {
        abort
    }

    # Enable high-performance compression
    encode zstd gzip

    # 1. Static Files
    # Handled first to ensure they are always accessible.
    handle_path /static/* {
        root * /webapp/staticfiles
        file_server
    }

    # 2. Define Allowed Routes
    # This matcher includes all Django URLs plus your explicit SSE route.
    @allowed {

        path /favicon.ico

        path /robots.txt

        path /sitemap.xml

        path /sse-events*

        path /

        path /admin

        path /admin/*

        path /some-path

        path /some-path/

        path /legal

        path /legal/

        path /login/*

        path /logout

        path /logout/

        path /meniu

        path /meniu/

        path /robots.txt

        path /sitemap.xml

        path /sse-token

        path /sse-token/

    }

    # 3. Handle Allowed Traffic
    handle @allowed {
        # SSE Sidecar (Specific route handling inside the allowed block)
        reverse_proxy /sse-events* docker-service-name-sse-sidecar:5687

        # Main Django App (Default for other allowed routes)
        reverse_proxy docker-service-name-app:8000
    }

    # 4. Fallback: Block everything else
    handle {
        respond "Access Denied" 403
    }
}

The Docker Compose file

In your docker-compose.yml file make sure you don't expose ports to external network.

# DEV
# ports:
#   - "8000:8000"
# PROD - expose port only inside docker network
expose:
    - "8000"

And now all those BOT requests will just stop at Caddy level!

How to implement SSE in Django with WSGI

Alin Climente — Sun, 26 Oct 2025 19:08:18 +0000

What are Server Sent Events (SSE)?

Server Sent Events (SSE) are a super lightweight method of sending messages to frontend. Let's say you have to wait for some long processing of files, chit-chat between your AI Agents, the user will stare at a spinning wheel, get bored then cancel his subscription.

Django is great, but not at async stuff, speed and DX. SSE needs an async web server to run - something like daphne or uvicorn.

Current solutions for implementing SSE in Django

I tried django-eventstream + daphne but hot-reload for server and browser broke. Then I tried a custom implementation of SSE + uvicorn - same hot-reload didn't worked (I used --reload flag).

I wasted a few hours saving 5 seconds of restarting server and refreshing browser on each change to create a solution.

New way of implementing SSE in Django with WSGI

The solution is a Golang service which listens to messages published by Django service and sends those messages to frontend. Bassically, a middle man which can handle async well.

It's built in Go to have a small footprint on server resources. It uses 3MB RAM and it has a size of 8MB.

How to use go-sse-wsgi-sidecar

Here is how you can use it.

Add this in your .env file (both services need to have access to it use env_file in docker compose):

GO_SSE_SIDECAR_HOST=localhost
GO_SSE_SIDECAR_PORT=5687
GO_SSE_SIDECAR_REDIS_URL=redis://:your-password@localhost-or-docker-container-name:6379/0
GO_SSE_SIDECAR_TOKEN=secret-token-here

Add this to your docker compose file. Or, use the Dockerfile in this repo. You also have the option to download the binary available on releases.


services:
  sse_sidecar:
    image: climentea/go-sse-wsgi-sidecar:latest
    container_name: sse-sidecar
    restart: unless-stopped
    ports:
      - "5687:5687"
    env_file:
      - .env

On the Python/Django app:

Install pyjwt package - this will be used to make sure only the authentificated user can have access to the server sent events.
Install redis package - we'll use here redis pub/sub functionality to have the Django app sending events and this app to send those recived events to frontend. If you already have celery/django-rq or other similar packages you could use the same connection as I did.

Add this view to your main urls.py file. This view will be used by frontend to get an authorization token.


import jwt
import datetime
from django.conf import settings
from django.http import JsonResponse


@login_required
def get_sse_token_view(request):
    exp_dt = datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(
        minutes=15
    )

    payload = {
        "user_id": request.social_user_id,
        "exp": exp_dt,
    }

    try:
        token = jwt.encode(payload, settings.GO_SSE_SIDECAR_TOKEN, algorithm="HS256")
    except Exception as e:
        return JsonResponse({"error": str(e)}, status=500)

    return JsonResponse(
        {
            "token": token,
            "expires_in": 900,
        }
    )


urlpatterns = [
    # etc
    path("admin/", admin.site.urls),
    path("sse-token/", get_sse_token_view, name="sse_token"),
]

I've used the connection of django_rq because it was already in my setup, but you can create a new redis connection if you want.
It must be the same connection for both services so they can write to the same pub/sub server.

Add this somewhere in your utils package:

import json

import django_rq


def publish(event_name: str, data: dict):
    r = django_rq.get_connection()
    r.publish("events", json.dumps({"event": event_name, "data": data}))

In your main scripts.js file or in base.html file add this EventSource listener.
You can change the urls based on what ports you've exposed.
When you'll run the app entirely in docker compose change localhost with the name of the service (Django service and Go service).
Add your handlers on alert(JSON.stringify(e.data)); and make it do react on a new event however you want.

let evtSource = null;

async function startSSE() {
  const res = await fetch("http://localhost:8000/sse-token");
  const { token } = await res.json();

  evtSource = new EventSource(`http://localhost:5687/sse-events?ssetoken=${token}`);

  evtSource.onmessage = (e) => {
    console.log("Received:", e.data);
    alert(JSON.stringify(e.data));
  };

  evtSource.onerror = (e) => {
    console.error("SSE error", e);
    evtSource.close();
    setTimeout(startSSE, 2000);
  };

}

startSSE();

Cool, now just import publish function where you need and start sending how many events you want to frontend.

You can see the code here.

Fun, simple, NOT scalable background worker based on Django Cache

Alin Climente — Sat, 25 Oct 2025 20:56:16 +0000

I felt like procrastinating a bit. So I created an simple background worker based on Django Cache Framework.

Here is how it works:
In a django management command start CacheWorker. Provide a list of functions which you need to use as background tasks:

CacheWorker.start_cache_worker(worker_functions)

Anywhere in your app you can send to background those long running functions which you need to collect in an a list (worker_functions) and pass it to CacheWorker.

Example of such mock functions:

import time

from webapp.logger import log


def func1(a: int, b: int):
    time.sleep(5)
    log.debug(f"func1 result a + b = {a + b}")


def func2(a: int, b: dict):
    time.sleep(15)
    log.debug(f"func2 a:{a}, b: {b}")

    raise Exception("some error")

Here is how you can send to CacheWorker those functions for execution:


t1 = CacheWorker.set_task("func1", 1, 2)
t2 = CacheWorker.set_task("func2", 1, {"some": 1, "dict": [1, "heloo"]})

log.debug(f"t1: {t1}, t2: {t2}")

What you need to do (if you want to try it). Add this to your settings.py file.

CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.filebased.FileBasedCache",
        "LOCATION": CACHE_DIR,
        "OPTIONS": {
            "MAX_ENTRIES": 10_000,
            "CULL_FREQUENCY": 5000,
        },
        "TIMEOUT": 3600,
    }
}

Create a new file next to settings.py file and paste this code (add your own logger).

import json
import threading
import time
import uuid
from typing import Callable

from django.core.cache import cache
from webapp.logger import log


class CacheWorker:
    PREFIX_TASK = "cw:task"
    PREFIX_STATUS = "cw:status"
    PREFIX_MESSAGE = "cw:message"

    STATUS_PENDING = "PENDING"
    STATUS_RUNNING = "RUNNING"
    STATUS_SUCCESS = "SUCCESS"
    STATUS_FAILED = "FAILED"

    @staticmethod
    def set_task(name: str, *args, **kwargs):
        task_id = uuid.uuid4().hex
        cache.set(
            f"{CacheWorker.PREFIX_TASK}:{task_id}",
            json.dumps({"name": name, "args": args, "kwargs": kwargs}),
        )
        cache.set(f"{CacheWorker.PREFIX_STATUS}:{task_id}", CacheWorker.STATUS_PENDING)

        task_ids = cache.get("cw:tasks") or []
        task_ids.append(task_id)
        cache.set("cw:tasks", task_ids)

        return task_id

    @staticmethod
    def get_task(task_id: str) -> dict:
        cached = cache.get(f"{CacheWorker.PREFIX_TASK}:{task_id}")
        if not cached:
            return
        return json.loads(cached)

    @staticmethod
    def remove_task(task_id: str):
        task_keys = [
            f"{CacheWorker.PREFIX_TASK}:{task_id}",
            f"{CacheWorker.PREFIX_STATUS}:{task_id}",
            f"{CacheWorker.PREFIX_MESSAGE}:{task_id}",
        ]
        cache.delete_many(task_keys)

    @staticmethod
    def get_task_status(task_id: str):
        status = cache.get(f"{CacheWorker.PREFIX_STATUS}:{task_id}")
        return status

    @staticmethod
    def get_task_message(task_id: str):
        message = json.loads(
            cache.get(f"{CacheWorker.PREFIX_MESSAGE}:{task_id}")
            or json.dumps({"message": None})
        )
        return message

    @staticmethod
    def set_task_status(task_id: str, status: str):
        return cache.set(f"{CacheWorker.PREFIX_STATUS}:{task_id}", status)

    @staticmethod
    def set_task_message(task_id: str, message: dict):
        return cache.set(
            f"{CacheWorker.PREFIX_MESSAGE}:{task_id}", json.dumps({"message": message})
        )

    @staticmethod
    def start_cache_worker(worker_functions: list[Callable]):
        cache.clear()

        worker_functions = {func.__name__: func for func in worker_functions}
        log.info(
            f"Cache worker started with {len(worker_functions)} functions. Registered functions: {', '.join(list(worker_functions.keys()))}"
        )

        def execute_task(task_id: str, func: Callable, args: tuple, kwargs: dict):
            try:
                log.info(f"Triggered task_id: {func.__name__}_{task_id}")
                CacheWorker.set_task_status(task_id, CacheWorker.STATUS_RUNNING)
                func(*args, **kwargs)
                CacheWorker.set_task_status(task_id, CacheWorker.STATUS_SUCCESS)
                log.success(
                    f"Finished task_id: {func.__name__}_{task_id} successfully!"
                )
            except Exception as ex:
                log.exception(ex)
                CacheWorker.set_task_status(task_id, CacheWorker.STATUS_FAILED)
                log.error(
                    f"Failed to execute task_id: {func.__name__}_{task_id} because of error: {ex}"
                )

        running_threads = set()

        while True:
            task_ids = cache.get("cw:tasks") or []
            for task_id in task_ids:
                if CacheWorker.get_task_status(task_id) != CacheWorker.STATUS_PENDING:
                    continue

                task = CacheWorker.get_task(task_id)
                if not task:
                    continue

                if task["name"] not in worker_functions:
                    log.warning(
                        f"Worker function {task['name']} was not found in provided worker functions."
                    )
                    continue

                thread = threading.Thread(
                    target=execute_task,
                    args=(
                        task_id,
                        worker_functions[task["name"]],
                        task["args"],
                        task["kwargs"],
                    ),
                    daemon=True,
                )
                thread.start()
                running_threads.add(thread)

                running_threads = {t for t in running_threads if t.is_alive()}

            time.sleep(2)

Some proof that it works:

Use it at your own risk. It's not scalable.

A simple way to encrypt sensitive data using python

Alin Climente — Sat, 04 Oct 2025 15:07:34 +0000

If you have some sensitive data you want to store safely pycryptodome is a super useful package that can help you with that.

Install pycryptodome:

pip install pycryptodome

Then add this class to your utils package:

import uuid
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
from hashlib import sha256


class AESCBC:
    def __init__(self, secret_key: str):
        self.key = sha256(uuid.uuid5(uuid.NAMESPACE_DNS, secret_key).bytes).digest()

    def encrypt(self, text: str):
        cipher = AES.new(self.key, AES.MODE_CBC)
        ciphertext = cipher.encrypt(pad(text.encode(), AES.block_size))
        return f"{cipher.iv.hex()}:{ciphertext.hex()}"

    def decrypt(self, encrypted_text: str):
        iv_hex = encrypted_text.split(":")[0]
        encrypted_text_hex = encrypted_text.split(":")[1]
        iv = bytes.fromhex(iv_hex)
        encrypted_text = bytes.fromhex(encrypted_text_hex)
        cipher = AES.new(self.key, AES.MODE_CBC, iv)
        plaintext = unpad(cipher.decrypt(encrypted_text), AES.block_size)
        return plaintext.decode()

When it's time to use it:

aes_cbc = AESCBC("secret key")

encrypted_message_hex = aes_cbc.encrypt("sensitive data")
print("encrypted_message_hex", encrypted_message_hex)

decrypted_message = aes_cbc.decrypt(encrypted_message_hex)
print("decrypted_message", decrypted_message)

Static Site Generator - 11ty v3 tutorial

Alin Climente — Sun, 23 Mar 2025 14:50:55 +0000

I was looking for a faster way to create static sites. I've looked into Nuxt and the other js frameworks - they all pack lots of javascript in the export and they don't look at all SEO friendly nor they are simple to use. Hugo looks too complicated and I didn't tried Jekyll.

Eleventy is such an intuitive package to use for creating static sites.

Installation

Make a folder or create a repo then git clone it locally. Initialize a node project with:

npm init -y

Install elevently:

npm install @11ty/eleventy

Create 2 folders next to package.json: site and public.

In the package.json update type to module ("type": "module") and add this command in the scripts field:

"scripts": {
    "dev": "npx @11ty/eleventy --input=./site --output=./public --serve --port=3000 --incremental"
}

With npm run dev 11tly will look into site directory and export the static site in the public directory.

First page

In the site directory create an index.md file in which you can add this markdown:

# This will be parsed into an h1 tag

Now go to localhost:3000 and you'll see the rendered html page.

What's crazy is that you can have html in the .md file with css style and script tags:

# This will be parsed into an h1 tag

<h1 id="header">You could have html as well</h1>

<!-- add css styles like this -->

<style>
    h1 {
        color: red;
    }
</style>

<script>

    let header = document.getElementById('header');

    header.addEventListener('click', function () {
        alert("You clicked the header!");
    });

    document.addEventListener('DOMContentLoaded', function () {
        alert("Hello, World!");
    });
</script>

vscode intelisense will not be great you'll need to change extension name to .html then back to .md. You are not forced to use only .md files you can mix .md with .html files.

Multiple pages and routes

A static site will have multiple pages. To add a new page, next to index.md file create another .md file let's say about.md. Add some header in it like # this is the about page. Now go to browser at localhost:3000/about and see the changes (11tly has hot reload as well).

Yeah, but we don't want to go manually between pages - we need to create a link. Super easy and intuitive:

In index.md file:

# the home page
[go to about](/about)

In about.md file:

# this is the about page
[go home](/)

Creating more nested routes is super simple. You just need to create directories and files. The index.* file will act as a root path for the url.

site
├── about.md            : /about 
├── index.md            : /
└── posts               
    ├── authors
    │   ├── author1.md   : /posts/authors/author1
    │   ├── author2.md   : /posts/authors/author2
    │   ├── author3.md   : /posts/authors/author3
    │   └── index.md     : /posts/authors
    ├── index.md         : /posts
    ├── post1.md         : /posts/post1
    └── post2.md         : /posts/post2

One thing I noticed is that the public folder (the folder which contains the exported html files) does not delete the exported files which you delete in the source directory (the site dir with .md files). So when you do the final export make sure to cleanup the public dir.

Base layouts

Let's say you have a blog - the layout (html+css+js) of that page will be the same for all blogs. The only thing what will change will be the content (images, text, links etc.).

To create a layout in site dir create a folder named _includes and a file named base.njk we'll use nunjucks as our templating language (it will help us with placing variables, making for loops, if else, adding some basic template logic more specifically).

VScode doesn't know .njk and I didn't found a proper extension which keeps the intelisense of the .html file - so as a "hack" you can go in the right lower corner and configure file association language to html.

In the site/_includes/base.njk create the layout to want for your pages. You can have multiple layouts and you can also use "inheritance" to build layouts.

Paste this in the base.njk file created:

---
title: this is the default title unless is overwritten in another file
---

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css">
    <title>{{ title }}</title>
</head>

<body>

    {{ content | safe }}

</body>

</html>

What you see there different from the usual html is nunjucks template syntax and some Frontmatter syntax which helps us deal with handling data. You can see that we added an external css library so you could add Bootstrap, Bulma or other classic css library without any fancy setup.

Data here will go in the head of the html file and some data will be used by 11tly to figure out which layout to use and other stuff.

---
title: this is the default title unless is overwritten in another file
---

Here 11lty using nunjucks will insert in the curly braces the content of index.md for example.

{{ content | safe }}

Here for layout: base.njk 11tly knows which template to use for this file. The title: overwritten title will overwrite existing title in the head for base.njk.

---
layout: base.njk
title: overwritten title
---

# the home page

[go to about](/about)
[go to posts](/posts)
[go to authors](/posts/authors)

If you need to reuse a layout in another layout you can use the frontmatter meta tags to specify from which template to inherit.

Create another layout named dashboard.njk in site/_includes folder and add the following content:

---
layout: base.njk
---

<aside>
    <ul>
        <li><a href="/">home</a> </li>
        <li><a href="/posts">posts</a> </li>
        <li><a href="/posts/authors">authors</a> </li>
    </ul>
</aside>

<main>
    {{ content | safe }}
</main>

As you can see we inherited from base.njk so we removed some duplicated code this way. Now to see it in action go to site/index.md and switch existing layout to layout: dashboard.njk.

Note that we could create that aside element taking use of 11tly collections. The special field tags will be gathered under a special variable called collections so we can iterate over collections.all links collected or by tag (ex: iterate over all posts)

---
tags:
  - post
---

This is useful for generating sitemaps, navigation etc.

<ul>
   {%- for post in collections.all -%}
        <li><a href="{{ post.url }}">{{ post.url }}</a></li>
   {%- endfor -%}
</ul>

Checkout pagination is kinda similar to collections.

Adding static assets like css, js, images, videos tailwind, daisy ui

First create a folder which will hold these static files. Next to package.json create a folder named assets and add there your .css, .js, .png, .jpg etc. static files. Now next to the same package.json file (in the root folder) create a file named eleventy.config.js and add the following code:

export default function (eleventyConfig) {
    eleventyConfig.addPassthroughCopy("./assets");
};

This will tell 11tly to copy the assets folder in the public folder on build.

Now to use those files in the head tag of the layout add them like this:


<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@picocss/pico@2/css/pico.min.css">

    <link rel="stylesheet" href="/assets/styles.css">
    <link rel="stylesheet" href="/assets/some_other_styles.css">
    <script src="/assets/scripts.js"></script>
    <script src="/assets/some_other_scripts.js"></script>

    <title>{{ title }}</title>
</head>

If you have images or other videos you can add them like this in the html files like this:

![the image](/assets/elevently.jpg)

or this:

<img src="/assets/elevently.jpg" alt="">

To add Tailwind v4 and Daisy UI 5 to elevently you need to do the following:

Install tailwind, daisy ui, and eleventy-plugin-vite

npm install @11ty/eleventy-plugin-vite@^6.0.0 @tailwindcss/vite@^4.0.15 daisyui@^5.0.9 tailwindcss@^4.0.15

Now in eleventy.config.js we need to configure tailwindcss with vite:

import EleventyVitePlugin from "@11ty/eleventy-plugin-vite";
import tailwindcss from '@tailwindcss/vite';


export default function (eleventyConfig) {
    eleventyConfig.addPassthroughCopy({
        "./assets/tailwind.css": "/assets/styles.css",
        // other static files you may have
        "./assets/custom.css": "/assets/custom.css",
        "./assets/custom.js": "/assets/custom.js",
    });

    eleventyConfig.addPlugin(EleventyVitePlugin, {
        viteOptions: { plugins: [tailwindcss()] }
    });
};

How to deploy one or more static sites on a VPS?

You can deploy multiple 11tly sites on a VPS using docker and Caddy.

Create a folder named websites and inside it make another folder with the domain name of the static site you want to serve. Copy the contents generated by 11tly into websites/my-static-site

Now create a file named Caddyfile and add the following contents:

mystaticsite.localhost {
    tls your-email-used-to-buy-domain@gmail.com
    root * /srv/websites/my-static-site
    push
    file_server browse
}

The .localhost will allow you to view the website on localhost. On the VPS change it to the domain you bought (ex: mystaticsite.com).

Create a docker-compose.yml file:


services:

  proxy:
    container_name: multisite-apps-caddy-proxy
    image: caddy:2-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./websites:/srv/websites # here we mount the static sites
      - $PWD/Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    networks:
      - web

volumes:
  caddy_data:
  caddy_config:

networks:
  web:
    driver: bridge

Cool now do a docker compose up -d wait for it to build and you can see your site(s) at mystaticsite.localhost

You can do even more with 11tly: create pages from json data, support multiple languages, handle dates based on timezone, add plugins to optimize images and many more.

Check more docs at 11ty.dev

How to create a simple scheduler in Django

Alin Climente — Sun, 20 Oct 2024 08:01:45 +0000

If you need to run a function every X minutes/seconds etc. to do some cleanups, trigger some operations you can do a simple scheduler with the help of threading module and django custom cli commands.

Let's say I want to invoke a function every 5 seconds to post something on an external API.

In your django app create a folder/package named management inside that folder create another folder named commands. In the commands folder create a module named runposter.py. In the end you'll have something like this structure yourapp/management/commands/runposter.py.

In this code we use a thread which runs a while loop as long as it's not stopped every 5 seconds. Replace print("posting") with the function/logic you want to run.

# runposter.py
import time
from threading import Thread, Event
from django.conf import settings
from django.core.management.base import BaseCommand


stop_event = Event()


def my_job():
    while not stop_event.is_set():
        try:
            print("posting")
            time.sleep(5)
        except KeyboardInterrupt:
            break
        except Exception as err:
            print(err)
            continue



class Command(BaseCommand):
    help = "Run Poster."

    def handle(self, *args, **options):

        poster = Thread(target=my_job)

        try:
            print("Starting poster...")
            poster.start()
            while poster.is_alive():
                poster.join(timeout=1)
        except KeyboardInterrupt:
            print("Stopping poster...")
            stop_event.set()
            poster.join()
            print("Poster shut down successfully!")

Nice, now open another terminal window and run python manage.py runposter. Command runposter as you can see was created from the module name we've given.

Of course, for something more complex I recommend using rq-scheduler or celery periodic tasks or django-q.

But, for simple cases this should be good enough.

How to Scale a Web Application Without Kubernetes

Alin Climente — Thu, 19 Sep 2024 06:30:05 +0000

Many folks add Kubernetes into their stack from the start so they can scale instantly when users flood their web application. Trust me, if the app is a paid app, you will be able to predict the number of users and scale with ease. You just need to know how to set up a virtual machine from any reputable hosting provider, which is super easy:

Choose a VM with the resources you need (RAM, CPU, Storage).
Choose Ubuntu as the OS.
Configure SSH keys.
Configure GitHub SSH on the VM (so you can copy the code).

It's the same setup as on your laptop.

Single VM

When starting out, if data is not critical, you can just use a simple virtual machine with your web app dockerized and Caddy as a proxy to handle incoming requests. You can scale vertically by adding more resources to the VM. You can go up to 448 GB RAM, 32 vCPUs, and 6 TB SSD. You can even order a bare metal instance with insane resources.

Single VM with Separate VM for Database

If data is important, you can add another VM which will hold just the database.

Single VM with Managed DB and Storage

If data is critical, I would suggest unburdening yourself of the responsibility of keeping the data safe, doing backups, and so on. Using a managed database and separate storage will allow you to focus just on your app. This is my favorite!

Multiple VMs with Managed DB and Storage

If you get to this point, congratulations! Now you can buy a private jet and go to dinner with Elon (if not, add a paywall to your app). You can add multiple VMs to handle the incoming requests and keep the managed DB/Storage for peace of mind (if data is super important).

Multiple VMs with Self-Managed DB and Storage

Probably the managed DB/Storage got way too expensive and you want to cut some costs. You can spin up some VMs for the database. Keep unrelated data on separate DBs.

Now You Probably Need to Learn Kubernetes

Manually deploying doesn't cut it anymore. It is time to add Kubernetes to your stack and properly set up CI/CDs.

Why Not Cloud Functions/Lambda?

I would rather have a few hours of downtime than go homeless from a mistake or a random bot DDoS. Yeah, but you can prevent it! Sure, but is it worth the risk though?

Keep it simple, folks.

Why is everything in software development so f*king complicated?

Alin Climente — Tue, 20 Aug 2024 12:58:40 +0000

I feel that this subject has been discussed multiple times, but I want to provide my take on this matter.

Too many tools

There are a ton of programming languages, packages, frameworks, and libraries that accomplish the same thing.

I'm sure you know TypeScript, but I saw in some job posts ReScript popping up. The chances of TypeScript dying are pretty low now since it's included in most JS runtimes. That's an example of a new thing to learn if you get hired on that project.

"As a software developer, you need to learn constantly" - I'm starting to dislike this statement very much. Some tools/technologies will die in 5 to 10 years or you won't find any job with it (Svelte). Why waste time (life is short) on learning shiny new technologies?

Always use old popular technologies and try to have a minimal stack that you could run on your laptop.

Useless induced complexity

Some projects are indeed complex because they need to solve complex problems. Creating ML models from scratch, scaling to millions of users while keeping costs down, operating systems, kernels, software for nuclear power plants, airplanes, CAD software, you get the point.

But... let's be honest, more than 90% of the projects out there are just glorified CRUD apps.

"Let's do this and that maybe we will need it in the future", "Let's try this new fancy framework/tool, fu%k Django is old and slow", "This thing is used in 2 places let's abstract it", "Applying random design patterns and principles with disregard to business requirements" > welcome to legacy sh!tty code that no one understands.

There are cases where developers force some technologies into the project just to spice up their CV.

Don't make it harder than it needs to be. Grow your app organically. Do some tests when a feature is very well defined. Document business requirements (make sure to keep them up to date). Keep it simple.

Capitalism and VC money

I think this is the main reason why everything is so complicated (not only in software development). It's crazy the amount of work we need to do in order to sustain a decent living in this day and age where everything is/can be automated.

Each company is a bet that it may make money by providing value. Once a company becomes profitable, other companies will be created and will compete on value/price.

Let's compare that with a gold mine. You start digging, others follow, they start digging and slowly, but surely the gold runs out and it gets harder and harder to find another gold nugget (C# reference 😄).

Same with the companies: One company starts delivering value in one domain, other companies follow, and the pool of available users starts to run out making it harder and harder to make a profit.

All this pressure is put on devs/employees to perform faster, better, and cheaper on increasingly useless complex projects.

Utopia

In some other alternative universe, we agreed on one programming language, package, and framework for each task.

We all just magically agree on PRs and it all works just fine.

When a company activates in a domain, that company will provide the best imaginable value and price on the market, so other companies won't dare to compete.

But, that won't happen anytime soon. That's the system we live in and can't escape it easily.

How to hide pycache folders in VSCode

Alin Climente — Mon, 29 Jul 2024 08:20:27 +0000

Want to hide pycache folders from vscode file explorer?

Here is how you can do that:

Open settings (File > Preferences > Settings or ctrl + ,);
Search for 'exclude' in the Settings search bar;
On Files: Exclude click on Add Pattern button and add **/__pycache__ pattern;

Nice! Now your file explorer is clean. You can do the same for other files you don't want to see in the vscode file explorer.

Push notifications from server with Telegram Bot API

Alin Climente — Sun, 16 Jun 2024 12:54:29 +0000

I needed a way to keep myself updated with what happens on the server (new user signup, daily stats with visitors, new user send a message to me thru contact form etc.).

You can do that with just email, but you can also do it with the help of Telegram Bot API.

Install telegram on your phone. Once installed, search for BotFather - click start and then type /newbot (follow the steps there, save safely the private bot token for the api). After the new bot is created search it and start a conversation with it (this will help you get chat_id which we will need later).

After your first conversation with the bot get the chat_id:

curl -X GET 'https://api.telegram.org/botTOKEN:FROM-BOTFATHER/getUpdates'

The response will look similar to this:

{
  "ok": true,
  "result": [
    {
      "update_id": 123123123123,
      "message": {
        "message_id": 124,
        "from": {
          "id": CHAT_ID,
          "is_bot": false,
          "first_name": "NAME",
          "last_name": "NAME",
          "language_code": "en"
        },
        "chat": {
          "id": "HERE-IS-CHAT_ID-YOU-NEED",
          "first_name": "NAME",
          "last_name": "NAME",
          "type": "private"
        },
        "date": 1718539209,
        "text": "/start",
        "entities": [
          {
            "offset": 0,
            "length": 6,
            "type": "bot_command"
          }
        ]
      }
    }
  ]
}

Take chat_id value from: HERE-IS-CHAT_ID-YOU-NEED (see up).

Nice, now save CHAT_ID and TELEGRAM_API_TOKEN in .env file (or somewhere else just don't push it to a public repo).

Here is the code to send push notifications (messages from the server to your phone) at each 10 seconds.


import time
import requests
from config import cfg


def server_push_notifications():
    try:
        while True:
            url = f"https://api.telegram.org/bot{cfg.TELEGRAM_API_TOKEN}/sendMessage"
            response = requests.post(
                url=url,
                params={'chat_id': cfg.CHAT_ID, 'text': 'All system are ok.', 'parse_mode': 'Markdown'}
            )
            print(url, response.json())
            time.sleep(10)
    except KeyboardInterrupt:
        return 


if __name__ == "__main__":
    server_push_notifications()

As you can see the base is just a POST request to telegram api with chat_id and text as body. What you send as a message is up to you.

For something more serious I would recommend using scheduler, but I don't see why a barebones while loop with a time.sleep won't do the same thing.