Forem: Naimul Karim

Building a Multi-Channel Notification Service in .NET 8

Naimul Karim — Fri, 08 May 2026 20:56:20 +0000

TL;DR — This post walks through building a complete notification service in .NET 8 that delivers messages across Email, SMS, and Push channels using a message queue, Scriban template engine, per-channel rate limiting, and parallel fan-out dispatch. Full source code on GitHub at the end.

Why Build a Notification Service?

Every production application eventually needs to notify its users — a welcome email, a password reset SMS, a push alert when their order ships. The naive approach is to call SendGrid inline in your controller. That works until:

Your SMTP provider is slow and your API response time balloons
You hit a Twilio rate limit and lose messages silently
You need to add a new channel and it requires touching 12 files
A downstream outage takes your whole API down with it

A proper notification service decouples delivery from your application logic. Requests are queued instantly, workers deliver asynchronously, and channels are isolated — one failing provider never blocks another.

Architecture Overview

Producers (API / Scheduled Jobs / Webhooks)
                    │
                    ▼
         ┌─────────────────┐
         │  Message Queue  │   RabbitMQ (topic exchange)
         └────────┬────────┘
                  │
    ┌─────────────▼──────────────────┐
    │          Dispatcher            │
    │  ┌──────────────────────────┐  │
    │  │    Template Service      │  │   Scriban engine + cache
    │  └──────────────────────────┘  │
    │  ┌──────────────────────────┐  │
    │  │      Rate Limiter        │  │   Sliding window per recipient
    │  └──────────────────────────┘  │
    └──────┬──────────┬──────────┬───┘
           │          │          │
    ┌──────▼──┐  ┌────▼───┐  ┌──▼──────┐
    │  Email  │  │  SMS   │  │  Push   │   Channel Workers
    │ Worker  │  │ Worker │  │ Worker  │   (IHostedService)
    └──────┬──┘  └────┬───┘  └──┬──────┘
           │          │          │
    ┌──────▼──┐  ┌────▼───┐  ┌──▼──────┐
    │  SMTP   │  │Twilio  │  │Firebase │   External Providers
    │ MailKit │  │  API   │  │  FCM    │
    └─────────┘  └────────┘  └─────────┘
                    │
         ┌──────────▼──────────┐
         │    Observability    │   OpenTelemetry + Serilog + Prometheus
         └─────────────────────┘

Data flow in plain English

Your API receives POST /api/notifications and immediately publishes to RabbitMQ — the HTTP response is instant
RabbitMqConsumer (a BackgroundService) picks up the message
The Dispatcher resolves the recipient, checks the rate limiter, renders the template, and fans out to all requested channels in parallel
Each channel worker calls its external provider (MailKit, Twilio, FCM)
Failures nack the message — RabbitMQ retries up to 3 times, then routes to a dead-letter queue

Project Structure

NotificationService/
├── src/
│   ├── NotificationService.Api/            # ASP.NET Core Web API
│   ├── NotificationService.Core/           # Domain models & interfaces
│   ├── NotificationService.Templates/      # Scriban template engine
│   ├── NotificationService.Dispatcher/     # Rate limiter + orchestration
│   ├── NotificationService.Channels/       # Email / SMS / Push workers
│   └── NotificationService.Infrastructure/ # RabbitMQ, persistence, DI
├── tests/
│   ├── NotificationService.UnitTests/
│   └── NotificationService.IntegrationTests/
├── docker-compose.yml
└── NotificationService.sln

The key design decision: every layer depends only on Core interfaces. Swapping RabbitMQ for Azure Service Bus, or Twilio for Vonage, touches exactly one file.

Part 1 — Core Domain Models

Everything starts with the models. Keeping them in a separate project enforces that no business logic leaks into infrastructure.

// Core/Models/NotificationModels.cs

public record NotificationRequest
{
    public Guid Id { get; init; } = Guid.NewGuid();
    public required string RecipientId { get; init; }
    public required string[] Channels { get; init; }      // "email" | "sms" | "push"
    public required string TemplateName { get; init; }
    public Dictionary<string, object> Data { get; init; } = new();
    public NotificationPriority Priority { get; init; } = NotificationPriority.Normal;
    public DateTimeOffset CreatedAt { get; init; } = DateTimeOffset.UtcNow;
}

public record RenderedNotification
{
    public required string Recipient { get; init; }
    public required string Subject { get; init; }
    public required string Body { get; init; }
    public string? HtmlBody { get; init; }
    public Dictionary<string, string> Metadata { get; init; } = new();
}

public record DeliveryResult
{
    public bool IsSuccess { get; init; }
    public DeliveryStatus Status { get; init; }
    public string? MessageId { get; init; }
    public string? ErrorMessage { get; init; }

    public static DeliveryResult Success(string? messageId = null) => new()
        { IsSuccess = true, Status = DeliveryStatus.Delivered, MessageId = messageId };

    public static DeliveryResult Failure(string error) => new()
        { IsSuccess = false, Status = DeliveryStatus.Failed, ErrorMessage = error };
}

The core interfaces define the contracts every implementation must honour:

// Core/Interfaces/INotificationInterfaces.cs

public interface INotificationChannel
{
    string ChannelName { get; }
    Task<DeliveryResult> SendAsync(RenderedNotification notification, CancellationToken ct = default);
}

public interface ITemplateService
{
    Task<RenderedNotification> RenderAsync(
        string templateName,
        Dictionary<string, object> data,
        string channel,
        string recipient);
}

public interface IDispatcher
{
    Task DispatchAsync(NotificationRequest request, CancellationToken ct = default);
}

public interface IRateLimiter
{
    bool TryAcquire(string recipientId, string channel);
    Task<bool> TryAcquireAsync(string recipientId, string channel, CancellationToken ct = default);
}

Part 2 — Message Queue with RabbitMQ

Publisher

The publisher converts a NotificationRequest to bytes and publishes to a topic exchange. Priority is encoded as the AMQP message priority so high-priority messages jump the queue.

// Infrastructure/Queue/RabbitMqPublisher.cs

public class RabbitMqPublisher : INotificationPublisher, IDisposable
{
    public Task PublishAsync(NotificationRequest request, CancellationToken ct = default)
    {
        EnsureConnected();

        var body  = JsonSerializer.SerializeToUtf8Bytes(request);
        var props = _channel!.CreateBasicProperties();
        props.Persistent = true;
        props.Priority   = (byte)request.Priority;
        props.MessageId  = request.Id.ToString();

        var routingKey = $"notification.{request.Priority.ToString().ToLower()}";

        _channel.BasicPublish(
            exchange:        _opts.ExchangeName,
            routingKey:      routingKey,
            basicProperties: props,
            body:            body);

        return Task.CompletedTask;
    }
}

Consumer (BackgroundService)

The consumer runs as a hosted service. Failed messages are nacked — RabbitMQ routes them to the dead-letter queue after max retries.

// Infrastructure/Queue/RabbitMqConsumer.cs

public class RabbitMqConsumer : BackgroundService
{
    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        Connect();

        var consumer = new AsyncEventingBasicConsumer(_channel!);

        consumer.Received += async (_, ea) =>
        {
            try
            {
                var request = JsonSerializer.Deserialize<NotificationRequest>(ea.Body.Span)!;

                using var scope  = _scopeFactory.CreateScope();
                var dispatcher   = scope.ServiceProvider.GetRequiredService<IDispatcher>();

                await dispatcher.DispatchAsync(request, stoppingToken);
                _channel!.BasicAck(ea.DeliveryTag, multiple: false);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error processing notification");
                _channel!.BasicNack(ea.DeliveryTag, multiple: false, requeue: false);
            }
        };

        _channel!.BasicConsume(_opts.QueueName, autoAck: false, consumer);
        return Task.CompletedTask;
    }
}

Queue topology

Exchange: notifications (topic)
    │
    ├── notification.critical ──► Queue: notifications.main
    ├── notification.high     ──► Queue: notifications.main
    ├── notification.normal   ──► Queue: notifications.main
    └── notification.low      ──► Queue: notifications.main
                                           │
                                     (nack, no requeue)
                                           │
                                           ▼
                                  Queue: notifications.dlq

Part 3 — Template Service (Scriban)

Scriban is a fast, sandboxed templating engine for .NET. Templates are stored per (name, channel, part) — so welcome:email:body and welcome:sms:body can have completely different content. Compiled templates are cached in a ConcurrentDictionary — parsing happens once per unique key, never again.

// Templates/Services/ScribanTemplateService.cs

public class ScribanTemplateService(
    ITemplateRepository repository,
    ILogger<ScribanTemplateService> logger) : ITemplateService
{
    private static readonly ConcurrentDictionary<string, Template> _cache = new();

    public async Task<RenderedNotification> RenderAsync(
        string templateName, Dictionary<string, object> data,
        string channel, string recipient)
    {
        var subject = await RenderPartAsync(templateName, channel, "subject", data);
        var body    = await RenderPartAsync(templateName, channel, "body",    data);

        return new RenderedNotification
        {
            Recipient = recipient,
            Subject   = subject,
            Body      = body
        };
    }

    private async Task<string> RenderPartAsync(
        string name, string channel, string part,
        Dictionary<string, object> data)
    {
        var key = $"{name}:{channel}:{part}";

        var compiled = _cache.GetOrAdd(key, _ =>
        {
            var source = repository.GetTemplateAsync(name, channel, part)
                .GetAwaiter().GetResult()
                ?? throw new InvalidOperationException($"Template not found: {key}");
            return Template.Parse(source);
        });

        var ctx   = new TemplateContext { StrictVariables = false };
        var model = new ScriptObject();
        foreach (var (k, v) in data) model.Add(k, v);
        ctx.PushGlobal(model);

        return await compiled.RenderAsync(ctx);
    }
}

Seeded templates (ready out of the box)

Template	Channel	What it sends
`welcome`	email	Full welcome with verification link
`welcome`	sms	Short text with verification link
`welcome`	push	Tap-to-verify push notification
`password-reset`	email / sms / push	Reset link + expiry notice
`order-confirmation`	all	Order ID, total, delivery date, tracking link

Part 4 — Rate Limiter (Sliding Window)

The rate limiter sits inside the Dispatcher — not at the API layer — so limits apply regardless of how notifications enter the system.

// Dispatcher/RateLimiting/SlidingWindowRateLimiter.cs

public class SlidingWindowRateLimiter(
    IMemoryCache cache,
    IOptions<RateLimitOptions> options) : IRateLimiter
{
    public bool TryAcquire(string recipientId, string channel)
    {
        var key    = $"rl:{channel}:{recipientId}";
        var window = cache.GetOrCreate(key, entry =>
        {
            entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(_opts.WindowSeconds);
            return new WindowCounter(_opts.MaxPerWindow);
        })!;

        return window.TryConsume();
    }

    private sealed class WindowCounter(int max)
    {
        private int _count;
        public bool TryConsume() => Interlocked.Increment(ref _count) <= max;
    }
}

Each (recipientId, channel) pair has its own independent window. Configure via appsettings.json:

"RateLimit": {
  "MaxPerWindow": 10,
  "WindowSeconds": 3600
}

For distributed deployments: replace IMemoryCache with a Redis-backed implementation using INCR + EXPIRE per key.

Part 5 — Dispatcher (Orchestration)

The Dispatcher wires together template rendering, rate limiting, and channel delivery — all in parallel.

// Dispatcher/Services/NotificationDispatcher.cs

public class NotificationDispatcher(
    ITemplateService templateService,
    IEnumerable<INotificationChannel> channels,
    IRateLimiter rateLimiter,
    IRecipientResolver recipientResolver,
    ILogger<NotificationDispatcher> logger) : IDispatcher
{
    public async Task DispatchAsync(NotificationRequest request, CancellationToken ct = default)
    {
        var recipient = await recipientResolver.ResolveAsync(request.RecipientId, ct);
        if (recipient is null)
        {
            logger.LogWarning("Recipient {Id} not found", request.RecipientId);
            return;
        }

        var targets = channels
            .Where(c => request.Channels.Contains(c.ChannelName, StringComparer.OrdinalIgnoreCase))
            .ToList();

        // Fan-out to all channels IN PARALLEL
        var tasks = targets.Select(ch => SendToChannelAsync(ch, request, recipient, ct));
        await Task.WhenAll(tasks);
    }

    private async Task SendToChannelAsync(
        INotificationChannel channel, NotificationRequest request,
        RecipientInfo recipient, CancellationToken ct)
    {
        if (!await rateLimiter.TryAcquireAsync(request.RecipientId, channel.ChannelName, ct))
        {
            logger.LogWarning("Rate limited: {Recipient} on {Channel}",
                request.RecipientId, channel.ChannelName);
            return;
        }

        var address = channel.ChannelName.ToLower() switch
        {
            "email" => recipient.Email,
            "sms"   => recipient.PhoneNumber,
            "push"  => recipient.DevicePushToken,
            _       => null
        };

        if (address is null) return;

        var rendered = await templateService.RenderAsync(
            request.TemplateName, request.Data, channel.ChannelName, address);

        var result = await channel.SendAsync(rendered, ct);

        if (result.IsSuccess)
            logger.LogInformation("✓ {Channel} → {Recipient}", channel.ChannelName, request.RecipientId);
        else
            logger.LogError("✗ {Channel} failed → {Error}", channel.ChannelName, result.ErrorMessage);
    }
}

Task.WhenAll means an ["email", "sms", "push"] request sends all three concurrently. One channel throwing never blocks the others.

Part 6 — Channel Workers

Email — MailKit

public class EmailChannel(IOptions<SmtpOptions> opts, ILogger<EmailChannel> logger) : INotificationChannel
{
    public string ChannelName => "email";

    public async Task<DeliveryResult> SendAsync(RenderedNotification n, CancellationToken ct = default)
    {
        try
        {
            var message = new MimeMessage();
            message.From.Add(new MailboxAddress(opts.Value.FromName, opts.Value.FromAddress));
            message.To.Add(MailboxAddress.Parse(n.Recipient));
            message.Subject = n.Subject;

            var builder = new BodyBuilder();
            if (n.HtmlBody is not null) { builder.HtmlBody = n.HtmlBody; builder.TextBody = n.Body; }
            else builder.TextBody = n.Body;
            message.Body = builder.ToMessageBody();

            using var client = new SmtpClient();
            await client.ConnectAsync(opts.Value.Host, opts.Value.Port,
                SecureSocketOptions.StartTlsWhenAvailable, ct);
            await client.AuthenticateAsync(opts.Value.User, opts.Value.Password, ct);
            await client.SendAsync(message, ct);
            await client.DisconnectAsync(quit: true, ct);

            return DeliveryResult.Success(message.MessageId);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, "Email failed to {Recipient}", n.Recipient);
            return DeliveryResult.Failure(ex.Message);
        }
    }
}

SMS — Twilio

public class SmsChannel(IOptions<TwilioOptions> opts, ILogger<SmsChannel> logger) : INotificationChannel
{
    public string ChannelName => "sms";

    public async Task<DeliveryResult> SendAsync(RenderedNotification n, CancellationToken ct = default)
    {
        EnsureInitialized();

        try
        {
            var message = await MessageResource.CreateAsync(
                to:   new PhoneNumber(n.Recipient),
                from: new PhoneNumber(opts.Value.FromNumber),
                body: n.Body);

            return message.ErrorCode is null
                ? DeliveryResult.Success(message.Sid)
                : DeliveryResult.Failure($"[{message.ErrorCode}] {message.ErrorMessage}");
        }
        catch (Exception ex)
        {
            logger.LogError(ex, "SMS failed to {Recipient}", n.Recipient);
            return DeliveryResult.Failure(ex.Message);
        }
    }
}

Push — Firebase FCM

public class PushChannel(IOptions<FirebaseOptions> opts, ILogger<PushChannel> logger) : INotificationChannel
{
    public string ChannelName => "push";

    public async Task<DeliveryResult> SendAsync(RenderedNotification n, CancellationToken ct = default)
    {
        await EnsureInitializedAsync();

        try
        {
            var message = new Message
            {
                Token        = n.Recipient,
                Notification = new FirebaseAdmin.Messaging.Notification
                    { Title = n.Subject, Body = n.Body },
                Android = new AndroidConfig { Priority = Priority.High },
                Apns    = new ApnsConfig { Aps = new Aps { Sound = "default" } },
                Data    = n.Metadata
            };

            var messageId = await FirebaseMessaging.GetMessaging(_app!).SendAsync(message, ct);
            return DeliveryResult.Success(messageId);
        }
        catch (FirebaseMessagingException ex)
            when (ex.MessagingErrorCode == MessagingErrorCode.Unregistered)
        {
            return DeliveryResult.Failure("FCM token is no longer valid");
        }
        catch (Exception ex)
        {
            logger.LogError(ex, "Push failed to {Recipient}", n.Recipient);
            return DeliveryResult.Failure(ex.Message);
        }
    }
}

Part 7 — API Layer

The controller is intentionally thin — validate, publish, return 202 Accepted. No business logic.

[ApiController]
[Route("api/[controller]")]
public class NotificationsController(
    INotificationPublisher publisher,
    ILogger<NotificationsController> logger) : ControllerBase
{
    [HttpPost]
    [ProducesResponseType(typeof(NotificationResponse), StatusCodes.Status202Accepted)]
    public async Task<IActionResult> Send(
        [FromBody] SendNotificationRequest request, CancellationToken ct)
    {
        if (request.Channels.Length == 0)
            return BadRequest(new { error = "At least one channel is required" });

        var notification = new NotificationRequest
        {
            RecipientId  = request.RecipientId,
            Channels     = request.Channels,
            TemplateName = request.TemplateName,
            Data         = request.Data,
            Priority     = request.Priority
        };

        await publisher.PublishAsync(notification, ct);
        return Accepted(new NotificationResponse(notification.Id, "Queued", DateTimeOffset.UtcNow));
    }
}

Example request

POST /api/notifications
Content-Type: application/json

{
  "recipientId": "user-123",
  "channels": ["email", "sms", "push"],
  "templateName": "welcome",
  "data": {
    "firstName": "Alice",
    "verificationLink": "https://app.example.com/verify?token=abc123"
  },
  "priority": "High"
}

Response

{
  "notificationId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "status": "Queued",
  "queuedAt": "2024-01-15T10:30:00Z"
}

Part 8 — DI Wiring (Program.cs)

builder.Services.AddSingleton<ITemplateRepository, InMemoryTemplateRepository>();
builder.Services.AddScoped<ITemplateService, ScribanTemplateService>();

builder.Services.AddMemoryCache();
builder.Services.Configure<RateLimitOptions>(builder.Configuration.GetSection("RateLimit"));
builder.Services.AddSingleton<IRateLimiter, SlidingWindowRateLimiter>();

builder.Services.Configure<SmtpOptions>(builder.Configuration.GetSection("Smtp"));
builder.Services.Configure<TwilioOptions>(builder.Configuration.GetSection("Twilio"));
builder.Services.Configure<FirebaseOptions>(builder.Configuration.GetSection("Firebase"));
builder.Services.AddScoped<INotificationChannel, EmailChannel>();
builder.Services.AddScoped<INotificationChannel, SmsChannel>();
builder.Services.AddScoped<INotificationChannel, PushChannel>();

builder.Services.AddScoped<IDispatcher, NotificationDispatcher>();

builder.Services.AddRabbitMq(builder.Configuration);
builder.Services.AddInfrastructure(builder.Configuration);

builder.Services.AddOpenTelemetry()
    .WithTracing(t => t.AddAspNetCoreInstrumentation().AddConsoleExporter());

builder.Services.AddHealthChecks()
    .AddRabbitMQ(rabbitConnectionString: "amqp://guest:guest@localhost/");

app.MapHealthChecks("/health");

Part 9 — Unit Tests

[Fact]
public void TryAcquire_AtLimit_ReturnsFalse()
{
    var limiter = CreateLimiter(max: 2);
    limiter.TryAcquire("user-1", "sms").Should().BeTrue();
    limiter.TryAcquire("user-1", "sms").Should().BeTrue();
    limiter.TryAcquire("user-1", "sms").Should().BeFalse(); // blocked
}

[Fact]
public void TryAcquire_DifferentChannels_TrackSeparately()
{
    var limiter = CreateLimiter(max: 1);
    limiter.TryAcquire("user-2", "email").Should().BeTrue();
    limiter.TryAcquire("user-2", "email").Should().BeFalse();
    limiter.TryAcquire("user-2", "sms").Should().BeTrue();  // sms is independent
}

[Fact]
public async Task DispatchAsync_WhenRateLimited_SkipsChannel()
{
    _rateLimiter.Setup(r => r.TryAcquireAsync("u1", "email", default))
        .ReturnsAsync(false);

    await dispatcher.DispatchAsync(request);

    _emailChannel.Verify(
        c => c.SendAsync(It.IsAny<RenderedNotification>(), default), Times.Never);
}

[Fact]
public async Task RenderAsync_InterpolatesVariables()
{
    var service = CreateService("Hello, {{ name }}!");
    var result  = await service.RenderAsync("tpl", new() { ["name"] = "Alice" }, "email", "a@b.com");
    result.Body.Should().Be("Hello, Alice!");
}

Run the full suite:

dotnet test

Part 10 — Docker Compose

services:
  api:
    build:
      context: .
      dockerfile: src/NotificationService.Api/Dockerfile
    ports:
      - "5000:8080"
    environment:
      - RabbitMq__Host=rabbitmq
    depends_on:
      rabbitmq:
        condition: service_healthy

  rabbitmq:
    image: rabbitmq:3.13-management-alpine
    ports:
      - "5672:5672"
      - "15672:15672"
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "ping"]
      interval: 10s
      retries: 5

  redis:
    image: redis:7.4-alpine
    ports:
      - "6379:6379"

Start everything:

docker-compose up -d

API → http://localhost:5000 · Swagger → http://localhost:5000/swagger · RabbitMQ UI → http://localhost:15672

Part 11 — GitHub Actions CI

name: CI
on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    services:
      rabbitmq:
        image: rabbitmq:3.13-alpine
        ports: [ 5672:5672 ]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: 8.0.x
      - run: dotnet restore
      - run: dotnet build --no-restore -c Release
      - run: dotnet test tests/NotificationService.UnitTests -c Release --logger trx

  docker:
    runs-on: ubuntu-latest
    needs: build-and-test
    steps:
      - uses: actions/checkout@v4
      - run: docker build -f src/NotificationService.Api/Dockerfile -t notification-service:${{ github.sha }} .

Extending the Service

Add a new channel (e.g. Slack) — 3 steps

Step 1 — Implement INotificationChannel:

public class SlackChannel(IOptions<SlackOptions> opts) : INotificationChannel
{
    public string ChannelName => "slack";

    public async Task<DeliveryResult> SendAsync(RenderedNotification n, CancellationToken ct)
    {
        var payload  = new { text = $"*{n.Subject}*\n{n.Body}" };
        var response = await _httpClient.PostAsJsonAsync(opts.Value.WebhookUrl, payload, ct);

        return response.IsSuccessStatusCode
            ? DeliveryResult.Success()
            : DeliveryResult.Failure(response.ReasonPhrase!);
    }
}

Step 2 — Register it:

builder.Services.AddScoped<INotificationChannel, SlackChannel>();

Step 3 — Add templates for slack:subject and slack:body. The Dispatcher discovers it automatically. Zero changes to existing code.

Distributed rate limiting (Redis)

public async Task<bool> TryAcquireAsync(string recipientId, string channel, CancellationToken ct)
{
    var key   = $"rl:{channel}:{recipientId}";
    var count = await _db.StringIncrementAsync(key);
    if (count == 1)
        await _db.KeyExpireAsync(key, TimeSpan.FromSeconds(_opts.WindowSeconds));
    return count <= _opts.MaxPerWindow;
}

Key Design Decisions

Message queue over direct delivery — The API never waits on SMTP, Twilio, or FCM. Response time is constant regardless of provider latency. Dead-letter queuing means no silent message loss during outages.

Scriban over Razor — Scriban compiles to a sandboxed AST. It cannot execute arbitrary C# code — important when templates might be user-editable. It also benchmarks significantly faster for pure string interpolation at scale.

Rate limiting in the Dispatcher, not the API — Notifications can enter from multiple sources. Placing rate limiting in the Dispatcher guarantees it applies everywhere, not just the HTTP path.

Task.WhenAll fan-out — One slow provider doesn't delay others. Email taking 800ms doesn't block push from delivering in 50ms. Failures are isolated — one channel exception never affects the others.

IEnumerable<INotificationChannel> in the Dispatcher — ASP.NET Core's DI resolves all registered implementations automatically. Adding a channel is one AddScoped call with zero changes to the Dispatcher.

What's Next

Delivery receipts — webhook callbacks from Twilio/FCM to track delivery status in your DB
User preferences — let recipients opt out of channels or specific notification types
Scheduled notifications — Hangfire or Quartz.NET for delayed delivery
Template management API — CRUD endpoints to edit templates without redeploying
Redis rate limiter — for multi-instance deployments
End-to-end OpenTelemetry traces — trace a notification from API call through queue through dispatcher through provider

Source Code

The complete, production-ready source code is on GitHub — includes all the code shown in this post, Docker Compose setup, GitHub Actions CI pipeline, full test suite (xUnit + Moq + FluentAssertions), and architecture documentation.

🔗 github.com/naimulkarim/NotificationService

If this was helpful, drop a ⭐ on the repo and feel free to open issues or PRs for improvements.

Built with .NET 8 · RabbitMQ · MailKit · Twilio · Firebase FCM · Scriban · OpenTelemetry

Building a Cloud-Native Event Ticketing System with DDD, Event-Driven Architecture & CQRS in .NET 8

Naimul Karim — Thu, 07 May 2026 05:35:39 +0000

Full source code: github.com/naimulkarim/TicketingSystem

Stack: .NET 8 · Angular 17 · SQL Server · RabbitMQ · Redis · MassTransit · MediatR · YARP · Docker · GitHub Actions

When you buy a concert ticket online, a deceptively large amount happens in the few seconds between clicking "Purchase" and receiving your confirmation email. Seats must be reserved atomically before payment is taken. Payment can fail — in which case the seat must be released so another customer can buy it. A confirmation email must fire. An admin dashboard must stay consistent. None of this can be lost, even if a service crashes mid-workflow.

This article walks through how I designed and built a production-grade online ticketing platform for concerts, sports, and live shows — from domain modelling through to Docker Compose and CI. Every code snippet comes from the real repository at github.com/naimulkarim/TicketingSystem with direct file links.

The three patterns at the heart of the design:

Domain-Driven Design (DDD) — keeps business logic in the domain, not scattered across controllers or services
Event-Driven Architecture (EDA) — decouples services through asynchronous messages and a saga for distributed workflows
CQRS — separates the write model (commands) from the read model (queries) for clarity and independent scalability

The Problem Domain & Why These Patterns
Architecture Overview — The 30,000 ft View
Solution Structure & Full File Map
The Shared Kernel — DDD Building Blocks
Bounded Context 1: Identity Service
Bounded Context 2: Event Catalog Service
Bounded Context 3: Booking Service — DDD + CQRS + Saga
Bounded Context 4: Payment Service
Bounded Context 5: Notification Service
The API Gateway — YARP
Event-Driven Architecture Deep Dive
The Booking Saga — Distributed Workflow
The Outbox Pattern — Guaranteed Delivery
Redis Seat Locking — Preventing Overselling
The Angular Frontend — NgRx Architecture
All Features & Use Cases — End to End
Infrastructure: Docker Compose & SQL Server
CI/CD: GitHub Actions Pipeline
Unit Tests
What to Build Next
Conclusion & GitHub Link

1. The Problem Domain & Why These Patterns

An event ticketing system exposes several genuinely hard problems:

Problem	Why it's hard	Pattern that solves it
Two users booking the last seat simultaneously	Race condition — DB locks don't scale	Redis distributed lock (SET NX)
Payment fails after seat is reserved	Must release seat as compensation	Saga with compensating transactions
Service crashes between saving booking and publishing event	Message lost — booking stuck	Outbox pattern (atomic write + publish)
High read traffic for event search vs. low-frequency writes	Write complexity pollutes read performance	CQRS — separate read/write paths
Notification service needs to react to bookings	Tight coupling if called directly	Integration events on RabbitMQ
Business rules scattered across controllers	Impossible to test, easy to break	Aggregates with invariant enforcement

These are not hypothetical edge cases. They are the daily reality of any system that handles money and inventory under concurrent load.

2. Architecture Overview

┌────────────────────────────────────────────────────────────────────┐
│                         CLIENT TIER                                │
│              Angular 17 SPA (NgRx · OIDC PKCE · Signals)          │
└─────────────────────────────┬──────────────────────────────────────┘
                              │ HTTPS / JWT
                              ▼
┌────────────────────────────────────────────────────────────────────┐
│                      API GATEWAY (YARP)                            │
│           Auth · Rate Limiting (300 req/min) · Routing             │
│                     localhost:5000                                  │
└──────┬──────────────┬────────────────┬────────────────┬────────────┘
       │              │                │                │
       ▼              ▼                ▼                ▼
┌──────────┐  ┌──────────────┐  ┌──────────┐  ┌──────────────┐
│ Identity │  │ EventCatalog │  │ Booking  │  │   Payment    │
│ :5001    │  │ :5002        │  │ :5003    │  │   :5004      │
│          │  │              │  │ ◄── SAGA │  │              │
│ OAuth2   │  │ Events       │  │ Aggreg.  │  │ Stripe PSP   │
│ OIDC     │  │ Venues       │  │ CQRS     │  │ Refunds      │
│ JWT      │  │ Seat Maps    │  │ Outbox   │  │              │
└──────────┘  └──────────────┘  └────┬─────┘  └──────┬───────┘
                                     │               │
                    ┌────────────────▼───────────────▼──────────┐
                    │         RabbitMQ Message Bus               │
                    │   Domain Events · Integration Events       │
                    │   Dead-Letter Queues · Saga State          │
                    └───────────┬────────────────┬──────────────┘
                                │                │
                    ┌───────────▼──┐  ┌──────────▼──────────┐
                    │ Notification │  │  Read Model          │
                    │ :5005        │  │  Projectors          │
                    │ Email/SMS    │  │  (future)            │
                    └──────────────┘  └─────────────────────┘

DATA TIER
┌──────────────┐  ┌────────────────────────────────────────────┐
│    Redis     │  │  SQL Server (one database per service)      │
│  Seat locks  │  │  Booking_DB · EventCatalog_DB · Payment_DB  │
│  TTL: 10min  │  │  Identity_DB                                │
└──────────────┘  └────────────────────────────────────────────┘

OBSERVABILITY
┌────────────────────────────────────────────────────────────────┐
│  OpenTelemetry → Jaeger (tracing) · Prometheus → Grafana       │
│  Serilog (structured logs) · /health endpoints per service     │
└────────────────────────────────────────────────────────────────┘

Key architectural decisions:

One database per service — no shared schema, no cross-service joins. Queries that need data from multiple services use the read model or a BFF.
RabbitMQ as the only cross-service channel — services never call each other's HTTP APIs directly. All integration happens through published events and commands on the bus.
Saga in Booking — the Booking service owns the checkout workflow and coordinates Payment via commands on RabbitMQ.
Outbox in every write service — domain events are atomically persisted to an outbox table in the same DB transaction as aggregate state, guaranteeing no message is lost.

3. Solution Structure & Full File Map

TicketingSystem/                              ← github.com/naimulkarim/TicketingSystem
│
├── TicketingSystem.sln
├── docker-compose.yml
├── .env.example
├── setup-github.sh
├── GITHUB_SETUP.md
│
├── src/
│   ├── SharedKernel/
│   │   ├── SharedKernel.csproj
│   │   ├── Domain/
│   │   │   ├── AggregateRoot.cs
│   │   │   ├── Primitives.cs               ← DomainEvent, ValueObject, Entity<T>
│   │   │   └── Result.cs                   ← Result<T> error pattern
│   │   ├── Application/Behaviours/
│   │   │   └── PipelineBehaviours.cs       ← ValidationBehaviour, LoggingBehaviour
│   │   └── Infrastructure/
│   │       └── OutboxMessage.cs
│   │
│   ├── Services/
│   │   ├── Identity/
│   │   │   ├── Api/Program.cs
│   │   │   ├── Api/IdentityConfig.cs       ← OAuth2 clients, scopes, resources
│   │   │   ├── Identity.csproj
│   │   │   └── Dockerfile
│   │   │
│   │   ├── EventCatalog/
│   │   │   ├── Domain/Aggregates/Event.cs  ← Event + SeatMap aggregates
│   │   │   ├── Application/EventHandlers.cs← Commands, Queries, Validators
│   │   │   ├── Infrastructure/Persistence/EventCatalogDbContext.cs
│   │   │   ├── Api/Controllers/EventsController.cs
│   │   │   ├── Api/Program.cs
│   │   │   ├── EventCatalog.csproj
│   │   │   └── Dockerfile
│   │   │
│   │   ├── Booking/
│   │   │   ├── Domain/
│   │   │   │   ├── Aggregates/Booking.cs   ← Booking aggregate root
│   │   │   │   ├── Events/BookingEvents.cs ← Domain + integration events
│   │   │   │   └── Repositories/IBookingRepository.cs
│   │   │   ├── Application/
│   │   │   │   ├── Commands/CreateBookingCommand.cs
│   │   │   │   ├── Queries/GetBookingQuery.cs
│   │   │   │   └── Sagas/BookingSagaStateMachine.cs
│   │   │   ├── Infrastructure/
│   │   │   │   ├── Persistence/BookingDbContext.cs
│   │   │   │   ├── Persistence/BookingRepository.cs
│   │   │   │   └── RedisSeatLockService.cs
│   │   │   ├── Api/Controllers/BookingsController.cs
│   │   │   ├── Api/Program.cs
│   │   │   ├── Booking.csproj
│   │   │   └── Dockerfile
│   │   │
│   │   ├── Payment/
│   │   │   ├── Domain/Aggregates/Payment.cs
│   │   │   ├── Infrastructure/Messaging/ProcessPaymentConsumer.cs
│   │   │   ├── Infrastructure/Persistence/PaymentDbContext.cs
│   │   │   ├── Api/Program.cs
│   │   │   ├── Payment.csproj
│   │   │   └── Dockerfile
│   │   │
│   │   └── Notification/
│   │       ├── Application/Consumers/BookingConsumers.cs
│   │       ├── Api/Program.cs
│   │       ├── Notification.csproj
│   │       └── Dockerfile
│   │
│   └── ApiGateway/
│       ├── Program.cs
│       ├── appsettings.json                ← YARP routes + clusters
│       ├── ApiGateway.csproj
│       └── Dockerfile
│
├── frontend/
│   ├── src/app/
│   │   ├── app.component.ts               ← root shell + nav
│   │   ├── app.config.ts                  ← NgRx, router, OAuth providers
│   │   ├── app.routes.ts                  ← lazy-loaded feature modules
│   │   ├── core/
│   │   │   ├── guards/auth.guard.ts
│   │   │   ├── interceptors/auth.interceptor.ts
│   │   │   └── services/auth.service.ts
│   │   ├── features/
│   │   │   ├── events/event-list.component.ts
│   │   │   ├── events/event-detail.component.ts
│   │   │   ├── account/my-bookings.component.ts
│   │   │   └── admin/admin-events.component.ts
│   │   └── store/
│   │       ├── auth/auth.reducer.ts
│   │       ├── booking/booking.reducer.ts
│   │       └── events/events.reducer.ts
│   ├── Dockerfile
│   └── nginx.conf
│
├── tests/
│   └── Unit/
│       ├── Booking/BookingAggregateTests.cs
│       ├── Booking/CreateBookingCommandHandlerTests.cs
│       └── SharedKernel/ValueObjectTests.cs
│
└── .github/workflows/ci.yml

4. The Shared Kernel — DDD Building Blocks

📁 src/SharedKernel/

The Shared Kernel has no business logic and no external dependencies beyond FluentValidation and MediatR abstractions. It defines the DDD primitives that every service inherits.

AggregateRoot

// src/SharedKernel/Domain/AggregateRoot.cs

public abstract class AggregateRoot<TId> where TId : notnull
{
    public TId Id { get; protected set; } = default!;

    private readonly List<DomainEvent> _domainEvents = new();
    public IReadOnlyList<DomainEvent> DomainEvents => _domainEvents.AsReadOnly();

    protected void RaiseDomainEvent(DomainEvent domainEvent) =>
        _domainEvents.Add(domainEvent);

    public void ClearDomainEvents() => _domainEvents.Clear();
}

The aggregate collects domain events in memory during a business operation. The infrastructure layer (DbContext's SaveChangesAsync override) reads them, writes them to the outbox table, and clears them — all in one atomic transaction. This is the core of reliable event publishing.

DomainEvent, ValueObject, Entity

// src/SharedKernel/Domain/Primitives.cs

public abstract record DomainEvent
{
    public Guid Id { get; init; } = Guid.NewGuid();
    public DateTime OccurredOnUtc { get; init; } = DateTime.UtcNow;
}

public abstract class ValueObject
{
    protected abstract IEnumerable<object?> GetEqualityComponents();

    public override bool Equals(object? obj)
    {
        if (obj is null || obj.GetType() != GetType()) return false;
        return ((ValueObject)obj).GetEqualityComponents()
            .SequenceEqual(GetEqualityComponents());
    }

    public override int GetHashCode() =>
        GetEqualityComponents()
            .Select(x => x?.GetHashCode() ?? 0)
            .Aggregate((x, y) => x ^ y);

    public static bool operator ==(ValueObject? left, ValueObject? right) =>
        left?.Equals(right) ?? right is null;

    public static bool operator !=(ValueObject? left, ValueObject? right) =>
        !(left == right);
}

Value objects are immutable and identity-free. Two Money instances with the same amount and currency are equal regardless of reference — the equality is structural, not referential. This models real-world concepts like prices, seat numbers, and currency codes correctly.

Result Pattern

// src/SharedKernel/Domain/Result.cs

public class Result<T>
{
    public T? Value { get; }
    public string? Error { get; }
    public bool IsSuccess { get; }
    public bool IsFailure => !IsSuccess;

    private Result(T value)      { Value = value; IsSuccess = true; }
    private Result(string error) { Error = error; IsSuccess = false; }

    public static Result<T> Success(T value) => new(value);
    public static Result<T> Failure(string error) => new(error);
}

Commands return Result<T> rather than throwing exceptions for expected failures. This forces the caller to handle both paths explicitly.

MediatR Pipeline Behaviours

// src/SharedKernel/Application/Behaviours/PipelineBehaviours.cs

// Runs before every Command and Query — validates the request first
public class ValidationBehaviour<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse> where TRequest : notnull
{
    private readonly IEnumerable<IValidator<TRequest>> _validators;
    public ValidationBehaviour(IEnumerable<IValidator<TRequest>> validators) =>
        _validators = validators;

    public async Task<TResponse> Handle(
        TRequest request, RequestHandlerDelegate<TResponse> next, CancellationToken ct)
    {
        if (!_validators.Any()) return await next();

        var context = new ValidationContext<TRequest>(request);
        var failures = _validators
            .Select(v => v.Validate(context))
            .SelectMany(r => r.Errors)
            .Where(f => f != null)
            .ToList();

        if (failures.Count != 0)
            throw new ValidationException(failures);

        return await next();
    }
}

// Structured logging for every request — zero boilerplate in handlers
public class LoggingBehaviour<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse> where TRequest : notnull
{
    private readonly ILogger<LoggingBehaviour<TRequest, TResponse>> _logger;
    public LoggingBehaviour(ILogger<LoggingBehaviour<TRequest, TResponse>> logger) =>
        _logger = logger;

    public async Task<TResponse> Handle(
        TRequest request, RequestHandlerDelegate<TResponse> next, CancellationToken ct)
    {
        var name = typeof(TRequest).Name;
        _logger.LogInformation("Handling {RequestName}", name);
        var response = await next();
        _logger.LogInformation("Handled {RequestName}", name);
        return response;
    }
}

Both behaviours are registered as open generics — they automatically apply to every Command and Query in every service without any per-handler wiring:

builder.Services.AddMediatR(cfg =>
{
    cfg.RegisterServicesFromAssemblyContaining<CreateBookingCommand>();
    cfg.AddOpenBehavior(typeof(ValidationBehaviour<,>));
    cfg.AddOpenBehavior(typeof(LoggingBehaviour<,>));
});

5. Bounded Context 1: Identity Service

📁 src/Services/Identity/

The Identity service is the single source of truth for authentication. It runs Duende IdentityServer and issues JWTs. All other services validate tokens against its discovery document — they never call it at runtime beyond token validation.

OAuth2 Client Configuration

// src/Services/Identity/Api/IdentityConfig.cs

public static class IdentityConfig
{
    public static IEnumerable<ApiScope> ApiScopes => new[]
    {
        new ApiScope("booking-api",      "Booking Service"),
        new ApiScope("eventcatalog-api", "Event Catalog Service"),
        new ApiScope("payment-api",      "Payment Service"),
    };

    public static IEnumerable<Client> Clients => new[]
    {
        // Angular SPA — Authorization Code + PKCE (no client secret in the browser)
        new Client
        {
            ClientId = "angular-spa",
            ClientName = "TicketingSystem Angular App",
            AllowedGrantTypes = GrantTypes.Code,
            RequirePkce = true,
            RequireClientSecret = false,                   // public client
            RedirectUris           = { "http://localhost:4200/auth/callback" },
            PostLogoutRedirectUris = { "http://localhost:4200" },
            AllowedCorsOrigins     = { "http://localhost:4200" },
            AllowedScopes = {
                "openid", "profile", "email",
                "booking-api", "eventcatalog-api"
            },
            AccessTokenLifetime = 3600,                    // 1 hour
        },

        // Machine-to-machine: services calling each other
        new Client
        {
            ClientId = "m2m-client",
            ClientSecrets = { new Secret("m2m-secret".Sha256()) },
            AllowedGrantTypes = GrantTypes.ClientCredentials,
            AllowedScopes = { "booking-api", "payment-api", "eventcatalog-api" }
        }
    };
}

Program.cs

// src/Services/Identity/Api/Program.cs

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddIdentityServer(opts =>
{
    opts.Events.RaiseErrorEvents   = true;
    opts.Events.RaiseSuccessEvents = true;
})
.AddInMemoryApiScopes(IdentityConfig.ApiScopes)
.AddInMemoryClients(IdentityConfig.Clients)
.AddInMemoryApiResources(IdentityConfig.ApiResources)
.AddDeveloperSigningCredential(); // ← swap for X.509 cert in production

var app = builder.Build();
app.UseIdentityServer();
app.MapHealthChecks("/health");
app.Run();

Feature delivered: UC-00 — all authenticated flows. The Angular frontend connects via PKCE; backend services use client credentials for M2M calls.

6. Bounded Context 2: Event Catalog Service

📁 src/Services/EventCatalog/

The Event Catalog owns everything about live events: their details, venues, seat maps, and availability. It is the source of truth for what can be booked.

The Event Aggregate

// src/Services/EventCatalog/Domain/Aggregates/Event.cs

public class Event : AggregateRoot<Guid>
{
    public string Title { get; private set; } = null!;
    public string Description { get; private set; } = null!;
    public string Category { get; private set; } = null!;  // Concert | Sports | Theater
    public string Venue { get; private set; } = null!;
    public string City { get; private set; } = null!;
    public DateTime StartsAtUtc { get; private set; }
    public decimal BasePrice { get; private set; }
    public string Currency { get; private set; } = null!;
    public int TotalSeats { get; private set; }
    public int AvailableSeats { get; private set; }
    public bool IsPublished { get; private set; }           // draft until published

    private readonly List<SeatMap> _seats = new();
    public IReadOnlyList<SeatMap> Seats => _seats.AsReadOnly();

    private Event() { }  // EF Core

    public static Event Create(string title, string description, string category,
        string venue, string city, DateTime startsAt,
        decimal basePrice, string currency, int totalSeats)
    {
        var ev = new Event
        {
            Id = Guid.NewGuid(),
            Title = title, Description = description,
            Category = category, Venue = venue, City = city,
            StartsAtUtc = startsAt, BasePrice = basePrice,
            Currency = currency, TotalSeats = totalSeats,
            AvailableSeats = totalSeats,  // starts fully available
            IsPublished = false           // always created as draft
        };

        ev.RaiseDomainEvent(new EventCreatedDomainEvent(ev.Id, title, startsAt));
        return ev;
    }

    public void Publish()
    {
        IsPublished = true;
        RaiseDomainEvent(new EventPublishedDomainEvent(Id, Title));
    }

    // Called when the Booking saga confirms a reservation
    public void ReserveSeat(string seatNumber)
    {
        if (AvailableSeats <= 0)
            throw new InvalidOperationException("No seats available");
        AvailableSeats--;  // invariant: cannot go below 0
    }

    // Called when the Booking saga compensates on payment failure
    public void ReleaseSeat(string seatNumber)
    {
        AvailableSeats++;
    }
}

CQRS — Commands and Queries

// src/Services/EventCatalog/Application/EventHandlers.cs

// ── Create Event Command ──────────────────────────────────────────────────────
public record CreateEventCommand(
    string Title, string Description, string Category,
    string Venue, string City, DateTime StartsAtUtc,
    decimal BasePrice, string Currency, int TotalSeats) : IRequest<Guid>;

public class CreateEventValidator : AbstractValidator<CreateEventCommand>
{
    public CreateEventValidator()
    {
        RuleFor(x => x.Title).NotEmpty().MaximumLength(200);
        RuleFor(x => x.Category).NotEmpty();
        RuleFor(x => x.StartsAtUtc).GreaterThan(DateTime.UtcNow);
        RuleFor(x => x.BasePrice).GreaterThanOrEqualTo(0);
        RuleFor(x => x.TotalSeats).GreaterThan(0);
    }
}

public class CreateEventHandler : IRequestHandler<CreateEventCommand, Guid>
{
    private readonly EventCatalogDbContext _ctx;
    public CreateEventHandler(EventCatalogDbContext ctx) => _ctx = ctx;

    public async Task<Guid> Handle(CreateEventCommand cmd, CancellationToken ct)
    {
        var ev = Event.Create(cmd.Title, cmd.Description, cmd.Category,
            cmd.Venue, cmd.City, cmd.StartsAtUtc,
            cmd.BasePrice, cmd.Currency, cmd.TotalSeats);
        _ctx.Events.Add(ev);
        await _ctx.SaveChangesAsync(ct);
        return ev.Id;
    }
}

// ── Search Events Query (read side — AsNoTracking for performance) ───────────
public record SearchEventsQuery(
    string? City, string? Category, DateTime? From,
    int Page = 1, int PageSize = 20) : IRequest<List<EventDto>>;

public class SearchEventsHandler : IRequestHandler<SearchEventsQuery, List<EventDto>>
{
    private readonly EventCatalogDbContext _ctx;
    public SearchEventsHandler(EventCatalogDbContext ctx) => _ctx = ctx;

    public async Task<List<EventDto>> Handle(SearchEventsQuery q, CancellationToken ct)
    {
        var query = _ctx.Events.AsNoTracking()  // read-only: no change tracking overhead
            .Where(e => e.IsPublished);

        if (!string.IsNullOrEmpty(q.City))     query = query.Where(e => e.City == q.City);
        if (!string.IsNullOrEmpty(q.Category)) query = query.Where(e => e.Category == q.Category);
        if (q.From.HasValue)                   query = query.Where(e => e.StartsAtUtc >= q.From);

        return await query
            .OrderBy(e => e.StartsAtUtc)
            .Skip((q.Page - 1) * q.PageSize)
            .Take(q.PageSize)
            .Select(e => new EventDto(e.Id, e.Title, e.Description, e.Category,
                e.Venue, e.City, e.StartsAtUtc, e.BasePrice, e.Currency,
                e.TotalSeats, e.AvailableSeats, e.IsPublished))
            .ToListAsync(ct);
    }
}

REST Controller

// src/Services/EventCatalog/Api/Controllers/EventsController.cs

[ApiController]
[Route("api/[controller]")]
public class EventsController : ControllerBase
{
    private readonly IMediator _mediator;
    public EventsController(IMediator mediator) => _mediator = mediator;

    // Public — no auth required for browsing
    [HttpGet]
    public async Task<IActionResult> Search(
        [FromQuery] string? city, [FromQuery] string? category,
        [FromQuery] DateTime? from, [FromQuery] int page = 1,
        CancellationToken ct = default)
    {
        var results = await _mediator.Send(new SearchEventsQuery(city, category, from, page), ct);
        return Ok(results);
    }

    [HttpGet("{id:guid}")]
    public async Task<IActionResult> GetById(Guid id, CancellationToken ct)
    {
        var ev = await _mediator.Send(new GetEventByIdQuery(id), ct);
        return ev is null ? NotFound() : Ok(ev);
    }

    // Admin only — requires JWT with Admin role
    [HttpPost]
    [Authorize(Roles = "Admin")]
    public async Task<IActionResult> Create([FromBody] CreateEventCommand cmd, CancellationToken ct)
    {
        var id = await _mediator.Send(cmd, ct);
        return CreatedAtAction(nameof(GetById), new { id }, new { id });
    }

    [HttpPost("{id:guid}/publish")]
    [Authorize(Roles = "Admin")]
    public async Task<IActionResult> Publish(Guid id, CancellationToken ct)
    {
        await _mediator.Send(new PublishEventCommand(id), ct);
        return NoContent();
    }
}

Features delivered: UC-01 Browse Events, UC-02 View Event Details, UC-08 Admin Creates Event, UC-09 Admin Publishes Event.

7. Bounded Context 3: Booking Service — DDD + CQRS + Saga

📁 src/Services/Booking/

The Booking service is the most complex. It owns the checkout workflow, acts as the saga orchestrator, enforces seat locking, and publishes all booking-related events.

The Booking Aggregate

// src/Services/Booking/Domain/Aggregates/Booking.cs

public enum BookingStatus { Pending, Confirmed, Cancelled, Refunded }

public class Booking : AggregateRoot<Guid>
{
    public Guid EventId { get; private set; }
    public Guid UserId { get; private set; }
    public string SeatNumber { get; private set; } = null!;
    public decimal TotalAmount { get; private set; }
    public string Currency { get; private set; } = null!;
    public BookingStatus Status { get; private set; }
    public DateTime CreatedAtUtc { get; private set; }
    public DateTime? ConfirmedAtUtc { get; private set; }
    public string? PaymentReference { get; private set; }

    private Booking() { }  // EF Core

    public static Booking Create(Guid eventId, Guid userId, string seatNumber,
        decimal totalAmount, string currency = "USD")
    {
        var booking = new Booking
        {
            Id = Guid.NewGuid(),
            EventId = eventId,
            UserId = userId,
            SeatNumber = seatNumber,
            TotalAmount = totalAmount,
            Currency = currency,
            Status = BookingStatus.Pending,
            CreatedAtUtc = DateTime.UtcNow
        };

        // Raises event — captured by DbContext and written to Outbox
        booking.RaiseDomainEvent(new BookingCreatedEvent(
            booking.Id, eventId, userId, seatNumber, totalAmount, currency));

        return booking;
    }

    public void Confirm(string paymentReference)
    {
        // Guard the invariant — cannot confirm a non-pending booking
        if (Status != BookingStatus.Pending)
            throw new InvalidOperationException(
                $"Cannot confirm booking in status {Status}");

        Status = BookingStatus.Confirmed;
        PaymentReference = paymentReference;
        ConfirmedAtUtc = DateTime.UtcNow;

        RaiseDomainEvent(new BookingConfirmedEvent(Id, EventId, UserId, paymentReference));
    }

    public void Cancel(string reason)
    {
        if (Status == BookingStatus.Cancelled)
            throw new InvalidOperationException("Booking already cancelled");

        Status = BookingStatus.Cancelled;
        RaiseDomainEvent(new BookingCancelledEvent(Id, EventId, UserId, SeatNumber, reason));
    }
}

Domain Events vs Integration Events

// src/Services/Booking/Domain/Events/BookingEvents.cs

// ── Domain Events (in-process, raised by the aggregate) ──────────────────────
public record BookingCreatedEvent(
    Guid BookingId, Guid EventId, Guid UserId,
    string SeatNumber, decimal TotalAmount, string Currency) : DomainEvent;

public record BookingConfirmedEvent(
    Guid BookingId, Guid EventId, Guid UserId,
    string PaymentReference) : DomainEvent;

public record BookingCancelledEvent(
    Guid BookingId, Guid EventId, Guid UserId,
    string SeatNumber, string Reason) : DomainEvent;

// ── Integration Events (cross-service contracts on RabbitMQ) ─────────────────
// These are the PUBLIC API of the Booking bounded context.
// Other services depend on these types — never on domain events directly.

public record BookingCreatedIntegrationEvent(
    Guid BookingId, Guid EventId, Guid UserId,
    string SeatNumber, decimal TotalAmount, string Currency);

public record BookingConfirmedIntegrationEvent(
    Guid BookingId, Guid UserId, string PaymentReference);

public record BookingCancelledIntegrationEvent(
    Guid BookingId, Guid EventId, string SeatNumber, string Reason);

// ── Commands sent to other services ──────────────────────────────────────────
public record ProcessPaymentCommand(
    Guid BookingId, Guid UserId, decimal Amount, string Currency);

public record ReleaseSeatsCommand(Guid EventId, string SeatNumber);
public record IssueTicketCommand(
    Guid BookingId, Guid EventId, Guid UserId, string SeatNumber);

// ── Result event received from Payment service ────────────────────────────────
public record PaymentProcessedEvent(
    Guid BookingId, bool Success,
    string? PaymentReference, string? FailureReason);

CreateBooking Command + Handler

// src/Services/Booking/Application/Commands/CreateBookingCommand.cs

public record CreateBookingCommand(
    Guid EventId, Guid UserId, string SeatNumber,
    decimal TotalAmount, string Currency = "USD") : IRequest<Guid>;

public class CreateBookingCommandValidator : AbstractValidator<CreateBookingCommand>
{
    public CreateBookingCommandValidator()
    {
        RuleFor(x => x.EventId).NotEmpty();
        RuleFor(x => x.UserId).NotEmpty();
        RuleFor(x => x.SeatNumber).NotEmpty().MaximumLength(20);
        RuleFor(x => x.TotalAmount).GreaterThan(0);
        RuleFor(x => x.Currency).NotEmpty().Length(3);
    }
}

public class CreateBookingCommandHandler : IRequestHandler<CreateBookingCommand, Guid>
{
    private readonly IBookingRepository _repo;
    private readonly IUnitOfWork _uow;
    private readonly ISeatLockService _seatLock;

    public CreateBookingCommandHandler(
        IBookingRepository repo, IUnitOfWork uow, ISeatLockService seatLock)
    {
        _repo = repo;
        _uow = uow;
        _seatLock = seatLock;
    }

    public async Task<Guid> Handle(CreateBookingCommand cmd, CancellationToken ct)
    {
        // Step 1 — Acquire Redis distributed lock for this seat
        var locked = await _seatLock.TryLockAsync(
            cmd.EventId, cmd.SeatNumber, TimeSpan.FromMinutes(10), ct);

        if (!locked)
            throw new InvalidOperationException(
                $"Seat {cmd.SeatNumber} is currently held by another user.");

        // Step 2 — Create the aggregate (raises BookingCreatedEvent internally)
        var booking = Booking.Create(
            cmd.EventId, cmd.UserId, cmd.SeatNumber, cmd.TotalAmount, cmd.Currency);

        // Step 3 — Persist aggregate + outbox message in ONE transaction
        _repo.Add(booking);
        await _uow.SaveChangesAsync(ct);
        // ↑ BookingCreatedEvent → OutboxMessage written atomically here

        return booking.Id;
    }
}

Queries — Read Side

// src/Services/Booking/Application/Queries/GetBookingQuery.cs

public record BookingDto(
    Guid Id, Guid EventId, Guid UserId,
    string SeatNumber, decimal TotalAmount, string Currency,
    string Status, DateTime CreatedAtUtc, DateTime? ConfirmedAtUtc);

public record GetBookingByIdQuery(Guid BookingId) : IRequest<BookingDto?>;
public record GetBookingsByUserQuery(Guid UserId) : IRequest<List<BookingDto>>;

public class GetBookingByIdHandler : IRequestHandler<GetBookingByIdQuery, BookingDto?>
{
    private readonly IBookingRepository _repo;
    public GetBookingByIdHandler(IBookingRepository repo) => _repo = repo;

    public async Task<BookingDto?> Handle(GetBookingByIdQuery query, CancellationToken ct)
    {
        var b = await _repo.GetByIdAsync(query.BookingId, ct);
        return b is null ? null : new BookingDto(
            b.Id, b.EventId, b.UserId, b.SeatNumber,
            b.TotalAmount, b.Currency, b.Status.ToString(),
            b.CreatedAtUtc, b.ConfirmedAtUtc);
    }
}

Repository and Unit of Work

// src/Services/Booking/Infrastructure/Persistence/BookingRepository.cs

public class BookingRepository : IBookingRepository
{
    private readonly BookingDbContext _ctx;
    public BookingRepository(BookingDbContext ctx) => _ctx = ctx;

    public async Task<Booking?> GetByIdAsync(Guid id, CancellationToken ct = default) =>
        await _ctx.Bookings.FirstOrDefaultAsync(b => b.Id == id, ct);

    public async Task<List<Booking>> GetByUserIdAsync(Guid userId, CancellationToken ct = default) =>
        await _ctx.Bookings.Where(b => b.UserId == userId).ToListAsync(ct);

    public void Add(Booking booking)    => _ctx.Bookings.Add(booking);
    public void Update(Booking booking) => _ctx.Bookings.Update(booking);
}

public class UnitOfWork : IUnitOfWork
{
    private readonly BookingDbContext _ctx;
    public UnitOfWork(BookingDbContext ctx) => _ctx = ctx;

    public async Task<int> SaveChangesAsync(CancellationToken ct = default) =>
        await _ctx.SaveChangesAsync(ct);
}

DbContext with Outbox Support

// src/Services/Booking/Infrastructure/Persistence/BookingDbContext.cs

public class BookingDbContext : DbContext
{
    public BookingDbContext(DbContextOptions<BookingDbContext> options) : base(options) { }

    public DbSet<Booking> Bookings => Set<Booking>();
    public DbSet<OutboxMessage> OutboxMessages => Set<OutboxMessage>();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Booking>(b =>
        {
            b.HasKey(x => x.Id);
            b.Property(x => x.SeatNumber).HasMaxLength(20).IsRequired();
            b.Property(x => x.Currency).HasMaxLength(3).IsRequired();
            b.Property(x => x.TotalAmount).HasColumnType("decimal(18,2)");
            b.Property(x => x.Status).HasConversion<string>();
        });

        modelBuilder.Entity<OutboxMessage>(b =>
        {
            b.HasKey(x => x.Id);
            b.Property(x => x.EventType).HasMaxLength(300);
            b.Property(x => x.RowVersion).IsRowVersion(); // SQL Server ROWVERSION
        });

        // MassTransit saga persistence tables (auto-created by EF migrations)
        modelBuilder.AddInboxStateEntity();
        modelBuilder.AddOutboxMessageEntity();
        modelBuilder.AddOutboxStateEntity();
    }
}

Full DI Wiring in Program.cs

// src/Services/Booking/Api/Program.cs

var builder = WebApplication.CreateBuilder(args);

// ── SQL Server ────────────────────────────────────────────────────────────────
builder.Services.AddDbContext<BookingDbContext>(opts =>
    opts.UseSqlServer(builder.Configuration.GetConnectionString("SqlServer"),
        sql => sql
            .MigrationsAssembly("Booking")
            .EnableRetryOnFailure(5)));          // retries on transient failures

// ── Repositories ──────────────────────────────────────────────────────────────
builder.Services.AddScoped<IBookingRepository, BookingRepository>();
builder.Services.AddScoped<IUnitOfWork, UnitOfWork>();

// ── Redis ─────────────────────────────────────────────────────────────────────
builder.Services.AddSingleton<IConnectionMultiplexer>(
    ConnectionMultiplexer.Connect(builder.Configuration.GetConnectionString("Redis")!));
builder.Services.AddScoped<ISeatLockService, RedisSeatLockService>();

// ── MediatR + pipeline behaviours ─────────────────────────────────────────────
builder.Services.AddMediatR(cfg =>
{
    cfg.RegisterServicesFromAssemblyContaining<CreateBookingCommand>();
    cfg.AddOpenBehavior(typeof(ValidationBehaviour<,>));
    cfg.AddOpenBehavior(typeof(LoggingBehaviour<,>));
});
builder.Services.AddValidatorsFromAssemblyContaining<CreateBookingCommandValidator>();

// ── MassTransit + RabbitMQ + Saga ─────────────────────────────────────────────
builder.Services.AddMassTransit(x =>
{
    x.AddSagaStateMachine<BookingSagaStateMachine, BookingSagaState>()
        .EntityFrameworkRepository(r =>
        {
            r.ExistingDbContext<BookingDbContext>();
            r.UseSqlServer();  // saga state persisted in SQL Server
        });

    x.SetKebabCaseEndpointNameFormatter();

    x.UsingRabbitMq((ctx, cfg) =>
    {
        cfg.Host(builder.Configuration["RabbitMQ:Host"], "/", h =>
        {
            h.Username(builder.Configuration["RabbitMQ:Username"]!);
            h.Password(builder.Configuration["RabbitMQ:Password"]!);
        });
        cfg.ConfigureEndpoints(ctx);
    });
});

// ── JWT Auth ──────────────────────────────────────────────────────────────────
builder.Services.AddAuthentication("Bearer")
    .AddJwtBearer("Bearer", opts =>
    {
        opts.Authority = builder.Configuration["Identity:Authority"];
        opts.Audience  = "booking-api";
    });

builder.Services.AddControllers();
builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("SqlServer")!)
    .AddRedis(builder.Configuration.GetConnectionString("Redis")!);

var app = builder.Build();

// Auto-migrate on startup — idempotent
using (var scope = app.Services.CreateScope())
    await scope.ServiceProvider
        .GetRequiredService<BookingDbContext>().Database.MigrateAsync();

app.UseAuthentication();
app.UseAuthorization();
app.MapControllers();
app.MapHealthChecks("/health");
app.Run();

Features delivered: UC-03 Reserve Seat, UC-05 Confirm Booking, UC-06 Cancel on Failure, UC-07 View My Bookings, UC-10 Session Abandonment.

8. Bounded Context 4: Payment Service

📁 src/Services/Payment/

The Payment service has no HTTP endpoints exposed to users. It is a pure event consumer — it listens for ProcessPaymentCommand from the Booking saga, charges the customer, and publishes PaymentProcessedEvent back.

The Payment Aggregate

// src/Services/Payment/Domain/Aggregates/Payment.cs

public enum PaymentStatus { Pending, Succeeded, Failed, Refunded }

public class Payment : AggregateRoot<Guid>
{
    public Guid BookingId { get; private set; }
    public decimal Amount { get; private set; }
    public string Currency { get; private set; } = null!;
    public PaymentStatus Status { get; private set; }
    public string? PspReference { get; private set; }   // Stripe PaymentIntent ID
    public string? FailureReason { get; private set; }

    private Payment() { }

    public static Payment Create(Guid bookingId, Guid userId, decimal amount, string currency)
    {
        var p = new Payment
        {
            Id = Guid.NewGuid(), BookingId = bookingId,
            Amount = amount, Currency = currency,
            Status = PaymentStatus.Pending, CreatedAtUtc = DateTime.UtcNow
        };
        p.RaiseDomainEvent(new PaymentInitiatedEvent(p.Id, bookingId, amount));
        return p;
    }

    public void Succeed(string pspReference)
    {
        Status = PaymentStatus.Succeeded;
        PspReference = pspReference;
        RaiseDomainEvent(new PaymentSucceededEvent(Id, BookingId, pspReference));
    }

    public void Fail(string reason)
    {
        Status = PaymentStatus.Failed;
        FailureReason = reason;
        RaiseDomainEvent(new PaymentFailedEvent(Id, BookingId, reason));
    }
}

RabbitMQ Consumer

// src/Services/Payment/Infrastructure/Messaging/ProcessPaymentConsumer.cs

public class ProcessPaymentConsumer : IConsumer<ProcessPaymentCommand>
{
    private readonly PaymentDbContext _ctx;
    private readonly IStripeService _stripe;

    public ProcessPaymentConsumer(PaymentDbContext ctx, IStripeService stripe)
    {
        _ctx = ctx;
        _stripe = stripe;
    }

    public async Task Consume(ConsumeContext<ProcessPaymentCommand> context)
    {
        var cmd = context.Message;

        // Create the payment aggregate
        var payment = Payment.Create(cmd.BookingId, cmd.UserId, cmd.Amount, cmd.Currency);
        _ctx.Payments.Add(payment);

        try
        {
            // Call PSP — swap IStripeService for real Stripe SDK in production
            var pspRef = await _stripe.ChargeAsync(cmd.Amount, cmd.Currency);
            payment.Succeed(pspRef);
        }
        catch (Exception ex)
        {
            payment.Fail(ex.Message);
        }

        await _ctx.SaveChangesAsync(context.CancellationToken);

        // Publish result back — the Booking saga is waiting for this
        await context.Publish(new PaymentProcessedEvent(
            cmd.BookingId,
            payment.Status == PaymentStatus.Succeeded,
            payment.PspReference,
            payment.FailureReason));
    }
}

// Stripe adapter — swap for real Stripe.net SDK in production
public class StripeService : IStripeService
{
    public async Task<string> ChargeAsync(decimal amount, string currency)
    {
        await Task.Delay(100); // simulate PSP round-trip
        return $"pi_{Guid.NewGuid():N}";
    }
}

Features delivered: UC-04 Process Payment, UC-06 Payment Failure Compensation.

9. Bounded Context 5: Notification Service

📁 src/Services/Notification/

The Notification service is the simplest — a pure event consumer with no domain logic and no database. It reacts to integration events and dispatches messages to users.

// src/Services/Notification/Application/Consumers/BookingConsumers.cs

public class BookingConfirmedConsumer : IConsumer<BookingConfirmedIntegrationEvent>
{
    private readonly IEmailService _email;
    private readonly ILogger<BookingConfirmedConsumer> _logger;

    public BookingConfirmedConsumer(IEmailService email,
        ILogger<BookingConfirmedConsumer> logger)
    { _email = email; _logger = logger; }

    public async Task Consume(ConsumeContext<BookingConfirmedIntegrationEvent> ctx)
    {
        _logger.LogInformation("Sending confirmation for booking {BookingId}",
            ctx.Message.BookingId);

        await _email.SendAsync(
            to:      $"user-{ctx.Message.UserId}@example.com",
            subject: "Your ticket is confirmed! 🎟️",
            body:    $"Booking {ctx.Message.BookingId} confirmed. " +
                     $"Payment reference: {ctx.Message.PaymentReference}");
    }
}

public class BookingCancelledConsumer : IConsumer<BookingCancelledIntegrationEvent>
{
    private readonly IEmailService _email;
    public BookingCancelledConsumer(IEmailService email) => _email = email;

    public async Task Consume(ConsumeContext<BookingCancelledIntegrationEvent> ctx)
    {
        await _email.SendAsync(
            to:      $"user-{ctx.Message.BookingId}@example.com",
            subject: "Booking cancelled",
            body:    $"Your booking {ctx.Message.BookingId} was cancelled. " +
                     $"Reason: {ctx.Message.Reason}. " +
                     $"Your seat {ctx.Message.SeatNumber} has been released.");
    }
}

// Swap for SendGrid / SMTP / AWS SES in production
public class SmtpEmailService : IEmailService
{
    private readonly ILogger<SmtpEmailService> _logger;
    public SmtpEmailService(ILogger<SmtpEmailService> logger) => _logger = logger;

    public Task SendAsync(string to, string subject, string body)
    {
        _logger.LogInformation("EMAIL → {To} | {Subject}", to, subject);
        return Task.CompletedTask;
    }
}

Features delivered: UC-05 Confirmation Email, UC-06 Cancellation Email.

10. The API Gateway — YARP

📁 src/ApiGateway/

YARP (Yet Another Reverse Proxy) acts as the single entry point for all client traffic. It validates JWTs, applies rate limiting, and routes to the correct downstream service — with zero business logic.

// src/ApiGateway/Program.cs

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

// Validate JWTs at the gateway edge — downstream services trust forwarded tokens
builder.Services.AddAuthentication("Bearer")
    .AddJwtBearer("Bearer", opts =>
    {
        opts.Authority = builder.Configuration["Identity:Authority"];
        opts.RequireHttpsMetadata = false;
    });

builder.Services.AddAuthorization(opts =>
    opts.AddPolicy("default", p => p.RequireAuthenticatedUser()));

// 300 requests/minute per client — protects downstream services
builder.Services.AddRateLimiter(opts =>
    opts.AddFixedWindowLimiter("api", cfg =>
    {
        cfg.Window      = TimeSpan.FromMinutes(1);
        cfg.PermitLimit = 300;
        cfg.QueueLimit  = 0;
    }));

var app = builder.Build();
app.UseAuthentication();
app.UseAuthorization();
app.UseRateLimiter();
app.MapReverseProxy();
app.MapHealthChecks("/health");
app.Run();

// src/ApiGateway/appsettings.json — declarative routing, no code changes needed

{
  "ReverseProxy": {
    "Routes": {
      "events-route": {
        "ClusterId": "eventcatalog",
        "Match": { "Path": "/api/events/{**catch-all}" }
      },
      "bookings-route": {
        "ClusterId": "booking",
        "Match": { "Path": "/api/bookings/{**catch-all}" },
        "AuthorizationPolicy": "default"
      },
      "payments-route": {
        "ClusterId": "payment",
        "Match": { "Path": "/api/payments/{**catch-all}" },
        "AuthorizationPolicy": "default"
      }
    },
    "Clusters": {
      "eventcatalog": {
        "Destinations": { "d1": { "Address": "http://eventcatalog:8080/" } }
      },
      "booking": {
        "Destinations": { "d1": { "Address": "http://booking:8080/" } }
      },
      "payment": {
        "Destinations": { "d1": { "Address": "http://payment:8080/" } }
      }
    }
  }
}

11. Event-Driven Architecture Deep Dive

Why Two Kinds of Events?

The system uses two distinct event types with very different concerns:

Domain Event                          Integration Event
─────────────────────────────────     ──────────────────────────────────
In-process only                       Cross-service, via RabbitMQ
Synchronous, raised during tx         Asynchronous, published after tx
Rich domain type                      Minimal DTO (only what others need)
Cleared after SaveChanges             Persisted in outbox until delivered
Never leaves the aggregate's context  Public API of the bounded context

Event Flow Diagram

┌────────────────────────────────────────────────────────────────────┐
│  Booking Service                                                   │
│                                                                    │
│  booking.Create() ──raises──▶ BookingCreatedEvent (domain)        │
│                                        │                           │
│  DbContext.SaveChangesAsync()          │                           │
│    ├── writes Booking row              │                           │
│    ├── converts domain event ──────────┘                           │
│    └── writes OutboxMessage row                                    │
│                    │ (same transaction — atomic)                   │
│                    │                                               │
│  OutboxProcessor   │                                               │
│    polls every 5s  │                                               │
│    reads unprocessed OutboxMessages                                │
│    publishes ──────▶ RabbitMQ ──▶ BookingCreatedIntegrationEvent  │
│    marks ProcessedOnUtc                                            │
└────────────────────────────────────────────────────────────────────┘

RabbitMQ
  Exchange: booking-created-integration-event
  Queue:    booking-saga (BookingSagaStateMachine)
  Queue:    notification (BookingConfirmedConsumer)

MassTransit Consumer Registration

All consumers and sagas are auto-registered by endpoint name formatter — no manual queue configuration:

x.SetKebabCaseEndpointNameFormatter();
// BookingConfirmedConsumer → queue: booking-confirmed-consumer
// BookingSagaStateMachine  → queue: booking-saga-state-machine

x.UsingRabbitMq((ctx, cfg) =>
{
    cfg.Host("rabbitmq", "/", h =>
    {
        h.Username("ticketing");
        h.Password("ticketing_pass");
    });
    cfg.ConfigureEndpoints(ctx); // ← auto-creates all queues + bindings
});

12. The Booking Saga — Distributed Workflow

📁 src/Services/Booking/Application/Sagas/BookingSagaStateMachine.cs

The saga orchestrates the multi-service checkout workflow. Its state is persisted to SQL Server so it survives process restarts.

State Machine Diagram

                    ┌─────────────────────────────────┐
                    │         BookingSagaState         │
                    │                                  │
  BookingCreated ──▶│  Initial ──▶ AwaitingPayment    │
                    │                 │                │
  PaymentProcessed  │        Success ─┤                │
  (Success = true) ▶│                 │──▶ Completed  │
                    │                 │                │
  PaymentProcessed  │        Failure ─┤                │
  (Success = false)▶│                 │──▶ Failed     │
                    └─────────────────────────────────┘

  On Completed: Publish IssueTicketCommand
                Publish BookingConfirmedIntegrationEvent

  On Failed:    Publish ReleaseSeatsCommand (compensate)
                Publish BookingCancelledIntegrationEvent

// src/Services/Booking/Application/Sagas/BookingSagaStateMachine.cs

public class BookingSagaState : SagaStateMachineInstance
{
    public Guid CorrelationId { get; set; }   // = BookingId — ties all messages together
    public string CurrentState { get; set; } = null!;
    public Guid EventId { get; set; }
    public Guid UserId { get; set; }
    public string SeatNumber { get; set; } = null!;
    public decimal TotalAmount { get; set; }
    public string Currency { get; set; } = null!;
    public string? PaymentReference { get; set; }
    public string? FailureReason { get; set; }
    public DateTime CreatedAt { get; set; }
}

public class BookingSagaStateMachine : MassTransitStateMachine<BookingSagaState>
{
    public State AwaitingPayment { get; private set; } = null!;
    public State Completed { get; private set; } = null!;
    public State Failed { get; private set; } = null!;

    public Event<BookingCreatedIntegrationEvent> BookingCreated { get; private set; } = null!;
    public Event<PaymentProcessedEvent> PaymentCompleted { get; private set; } = null!;

    public BookingSagaStateMachine()
    {
        InstanceState(x => x.CurrentState);

        // Correlation: all events for one booking share the same BookingId
        Event(() => BookingCreated,
            x => x.CorrelateById(ctx => ctx.Message.BookingId));
        Event(() => PaymentCompleted,
            x => x.CorrelateById(ctx => ctx.Message.BookingId));

        Initially(
            When(BookingCreated)
                .Then(ctx =>
                {
                    // Store all data we'll need later in the saga state
                    ctx.Saga.EventId     = ctx.Message.EventId;
                    ctx.Saga.UserId      = ctx.Message.UserId;
                    ctx.Saga.SeatNumber  = ctx.Message.SeatNumber;
                    ctx.Saga.TotalAmount = ctx.Message.TotalAmount;
                    ctx.Saga.Currency    = ctx.Message.Currency;
                    ctx.Saga.CreatedAt   = DateTime.UtcNow;
                })
                // Command the Payment service to charge the customer
                .Publish(ctx => new ProcessPaymentCommand(
                    ctx.Saga.CorrelationId,
                    ctx.Saga.UserId,
                    ctx.Saga.TotalAmount,
                    ctx.Saga.Currency))
                .TransitionTo(AwaitingPayment));

        During(AwaitingPayment,

            // ── Happy path ─────────────────────────────────────────────────────
            When(PaymentCompleted, ctx => ctx.Message.Success)
                .Then(ctx => ctx.Saga.PaymentReference = ctx.Message.PaymentReference)
                // Tell EventCatalog to decrement available seats permanently
                .Publish(ctx => new IssueTicketCommand(
                    ctx.Saga.CorrelationId,
                    ctx.Saga.EventId,
                    ctx.Saga.UserId,
                    ctx.Saga.SeatNumber))
                // Notify all subscribers (Notification service sends email)
                .Publish(ctx => new BookingConfirmedIntegrationEvent(
                    ctx.Saga.CorrelationId,
                    ctx.Saga.UserId,
                    ctx.Saga.PaymentReference!))
                .TransitionTo(Completed)
                .Finalize(),

            // ── Compensating path — payment failed ─────────────────────────────
            When(PaymentCompleted, ctx => !ctx.Message.Success)
                .Then(ctx => ctx.Saga.FailureReason = ctx.Message.FailureReason)
                // Release the Redis seat lock so others can book
                .Publish(ctx => new ReleaseSeatsCommand(
                    ctx.Saga.EventId, ctx.Saga.SeatNumber))
                // Notify all subscribers (Notification sends failure email)
                .Publish(ctx => new BookingCancelledIntegrationEvent(
                    ctx.Saga.CorrelationId,
                    ctx.Saga.EventId,
                    ctx.Saga.SeatNumber,
                    ctx.Saga.FailureReason ?? "Payment failed"))
                .TransitionTo(Failed)
                .Finalize());

        SetCompletedWhenFinalized();
    }
}

13. The Outbox Pattern — Guaranteed Delivery

Without the outbox, there is a window between saving the booking to the database and publishing the event to RabbitMQ where a crash loses the message permanently. The outbox closes that window.

// src/SharedKernel/Infrastructure/OutboxMessage.cs

public class OutboxMessage
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string EventType { get; set; } = null!;     // full CLR type name
    public string Payload { get; set; } = null!;       // JSON-serialised event
    public DateTime OccurredOnUtc { get; set; } = DateTime.UtcNow;
    public DateTime? ProcessedOnUtc { get; set; }      // null = not yet published
    public string? Error { get; set; }

    [Timestamp]   // SQL Server ROWVERSION — optimistic concurrency for processor
    public byte[] RowVersion { get; set; } = null!;
}

The flow in sequence:

1. Command handler calls _uow.SaveChangesAsync()
        │
        ├── EF writes Booking row          ─┐
        └── EF writes OutboxMessage row    ─┘  one ACID transaction

2. OutboxProcessor (IHostedService) polls every 5 seconds:
        SELECT TOP 10 * FROM OutboxMessages
        WHERE ProcessedOnUtc IS NULL
        ORDER BY OccurredOnUtc

3. For each message:
        a. Deserialize Payload to the event type
        b. Publish to RabbitMQ via IPublishEndpoint
        c. UPDATE OutboxMessages SET ProcessedOnUtc = GETUTCDATE()
               WHERE Id = @id AND RowVersion = @rowVersion
           (optimistic concurrency — prevents double-publish if two instances race)

4. If RabbitMQ is down:
        OutboxProcessor retries on next poll cycle
        Booking data is safe — it was committed in step 1

This pattern guarantees at-least-once delivery. Consumers must be idempotent (or use an inbox table to deduplicate).

14. Redis Seat Locking — Preventing Overselling

// src/Services/Booking/Infrastructure/RedisSeatLockService.cs

public class RedisSeatLockService : ISeatLockService
{
    private readonly IDatabase _redis;
    public RedisSeatLockService(IConnectionMultiplexer redis) =>
        _redis = redis.GetDatabase();

    public async Task<bool> TryLockAsync(
        Guid eventId, string seat, TimeSpan ttl, CancellationToken ct = default)
    {
        // Redis SET key value NX EX seconds
        // NX = only set if key does NOT exist (atomic test-and-set)
        // Returns true  = lock acquired (key was not present)
        // Returns false = lock NOT acquired (another user holds it)
        var key = $"seat-lock:{eventId}:{seat}";
        return await _redis.StringSetAsync(key, "locked", ttl, When.NotExists);
    }

    public async Task ReleaseAsync(Guid eventId, string seat, CancellationToken ct = default)
    {
        var key = $"seat-lock:{eventId}:{seat}";
        await _redis.KeyDeleteAsync(key);
    }
}

Why Redis instead of a database lock?

Approach	Throughput	Scope	TTL support
SQL `SELECT FOR UPDATE`	Low (row lock, connection held)	Single DB	No
Optimistic concurrency	Medium (retries on conflict)	Single DB	No
Redis SET NX	Very high (microsecond op, connection pool)	Distributed	✅ Yes

The TTL (10 minutes) acts as a safety valve — if a user abandons checkout or their browser crashes, the seat is automatically available again after 10 minutes. No scheduled cleanup job needed.

15. The Angular Frontend — NgRx Architecture

📁 frontend/src/app/

The frontend follows the Redux pattern via NgRx: all state is in the store, all side effects (HTTP calls, navigation) are in Effects, all views are driven by Selectors.

Application Bootstrap

// frontend/src/app/app.config.ts

export const appConfig: ApplicationConfig = {
  providers: [
    provideRouter(routes),
    provideHttpClient(withInterceptors([authInterceptor, errorInterceptor])),
    // NgRx store slices
    provideStore({
      events:  eventsReducer,
      booking: bookingReducer,
      auth:    authReducer
    }),
    provideEffects([EventsEffects, BookingEffects, AuthEffects]),
    provideRouterStore(),
    provideStoreDevtools({ maxAge: 25 }),
    importProvidersFrom(OAuthModule.forRoot()),
  ]
};

Lazy-Loaded Feature Routes

// frontend/src/app/app.routes.ts

export const routes: Routes = [
  { path: '',       redirectTo: 'events', pathMatch: 'full' },
  {
    path: 'events',
    loadChildren: () => import('./features/events/events.routes')
      .then(m => m.EVENTS_ROUTES)
    // No auth required — browse without login
  },
  {
    path: 'booking',
    loadChildren: () => import('./features/booking/booking.routes')
      .then(m => m.BOOKING_ROUTES),
    canActivate: [authGuard]  // JWT required
  },
  {
    path: 'account',
    loadChildren: () => import('./features/account/account.routes')
      .then(m => m.ACCOUNT_ROUTES),
    canActivate: [authGuard]
  },
  {
    path: 'admin',
    loadChildren: () => import('./features/admin/admin.routes')
      .then(m => m.ADMIN_ROUTES),
    canActivate: [authGuard],
    data: { roles: ['Admin'] }
  },
];

NgRx Booking Store

// frontend/src/app/store/booking/booking.reducer.ts

export interface BookingState {
  currentBookingId: string | null;
  myBookings: Booking[];
  loading: boolean;
  error: string | null;
}

export const BookingActions = {
  createBooking: createAction('[Booking] Create',
    props<{ eventId: string; seatNumber: string; totalAmount: number }>()),
  createBookingSuccess: createAction('[Booking] Create Success',
    props<{ bookingId: string }>()),
  createBookingFailure: createAction('[Booking] Create Failure',
    props<{ error: string }>()),
  loadMyBookings: createAction('[Booking] Load My Bookings',
    props<{ userId: string }>()),
  loadMyBookingsSuccess: createAction('[Booking] Load My Bookings Success',
    props<{ bookings: Booking[] }>()),
};

// Pure reducer — no side effects, easy to test
export const bookingReducer = createReducer(initialState,
  on(BookingActions.createBooking,
    s => ({ ...s, loading: true, error: null })),
  on(BookingActions.createBookingSuccess,
    (s, { bookingId }) => ({ ...s, currentBookingId: bookingId, loading: false })),
  on(BookingActions.createBookingFailure,
    (s, { error }) => ({ ...s, loading: false, error })),
  on(BookingActions.loadMyBookingsSuccess,
    (s, { bookings }) => ({ ...s, myBookings: bookings })),
);

Effects — HTTP Side Effects

@Injectable()
export class BookingEffects {

  createBooking$ = createEffect(() => this.actions$.pipe(
    ofType(BookingActions.createBooking),
    switchMap(({ eventId, seatNumber, totalAmount }) =>
      this.http.post<{ id: string }>(`${environment.apiUrl}/api/bookings`, {
        eventId, seatNumber, totalAmount, currency: 'USD'
      }).pipe(
        map(res  => BookingActions.createBookingSuccess({ bookingId: res.id })),
        catchError(err => of(BookingActions.createBookingFailure({ error: err.message })))
      )
    )
  ));

  loadMyBookings$ = createEffect(() => this.actions$.pipe(
    ofType(BookingActions.loadMyBookings),
    switchMap(({ userId }) =>
      this.http.get<Booking[]>(`${environment.apiUrl}/api/bookings/user/${userId}`).pipe(
        map(bookings => BookingActions.loadMyBookingsSuccess({ bookings })),
        catchError(() => of(BookingActions.loadMyBookingsSuccess({ bookings: [] })))
      )
    )
  ));

  constructor(private actions$: Actions, private http: HttpClient) {}
}

Auth Guard & Interceptor

// frontend/src/app/core/guards/auth.guard.ts

export const authGuard: CanActivateFn = () => {
  const auth   = inject(AuthService);
  const router = inject(Router);
  if (auth.isLoggedIn) return true;
  auth.login();   // redirect to IdentityServer PKCE flow
  return false;
};

// frontend/src/app/core/interceptors/auth.interceptor.ts

export const authInterceptor: HttpInterceptorFn = (req, next) => {
  const auth = inject(AuthService);
  if (auth.accessToken) {
    req = req.clone({
      setHeaders: { Authorization: `Bearer ${auth.accessToken}` }
    });
  }
  return next(req);
};

export const errorInterceptor: HttpInterceptorFn = (req, next) =>
  next(req).pipe(
    catchError(err => {
      if (err.status === 401) inject(AuthService).login();
      return throwError(() => err);
    })
  );

Event List Component — Search & Browse

// frontend/src/app/features/events/event-list.component.ts

@Component({
  selector: 'app-event-list',
  standalone: true,
  imports: [CommonModule, ReactiveFormsModule, RouterModule],
  template: `
    <div class="events-container">
      <h1>Upcoming Events</h1>

      <form [formGroup]="filterForm" (ngSubmit)="search()" class="filters">
        <input formControlName="city"     placeholder="City" />
        <select formControlName="category">
          <option value="">All categories</option>
          <option value="Concert">Concert</option>
          <option value="Sports">Sports</option>
          <option value="Theater">Theater</option>
        </select>
        <input formControlName="from" type="date" />
        <button type="submit">Search</button>
      </form>

      <div *ngIf="loading$ | async" class="loading">Loading events...</div>

      <div class="event-grid">
        <div *ngFor="let event of events$ | async" class="event-card">
          <div class="event-category">{{ event.category }}</div>
          <h2>{{ event.title }}</h2>
          <p class="event-meta">
              {{ event.venue }}, {{ event.city }}<br/>
              {{ event.startsAtUtc | date:'medium' }}
          </p>
          <div class="event-footer">
            <span class="price">
              From {{ event.basePrice | currency:event.currency }}
            </span>
            <span class="seats">{{ event.availableSeats }} seats left</span>
            <a [routerLink]="['/events', event.id]" class="btn-primary">
              View Tickets
            </a>
          </div>
        </div>
      </div>
    </div>
  `
})
export class EventListComponent implements OnInit {
  events$  = this.store.select(selectAllEvents);
  loading$ = this.store.select(selectEventsLoading);
  filterForm = this.fb.group({ city: [''], category: [''], from: [''] });

  constructor(private store: Store, private fb: FormBuilder) {}

  ngOnInit() { this.search(); }

  search() {
    const { city, category, from } = this.filterForm.value;
    this.store.dispatch(EventActions.search({
      city:     city     || undefined,
      category: category || undefined,
      from:     from     || undefined,
    }));
  }
}

Admin Event Management Component

// frontend/src/app/features/admin/admin-events.component.ts (excerpt)

export class AdminEventsComponent implements OnInit {
  createForm = this.fb.group({
    title:       ['', [Validators.required, Validators.maxLength(200)]],
    description: ['', Validators.required],
    category:    ['Concert', Validators.required],
    venue:       ['', Validators.required],
    city:        ['', Validators.required],
    startsAtUtc: ['', Validators.required],
    basePrice:   [0,  [Validators.required, Validators.min(0)]],
    currency:    ['USD', [Validators.required, Validators.minLength(3)]],
    totalSeats:  [100, [Validators.required, Validators.min(1)]],
  });

  createEvent() {
    if (this.createForm.invalid) return;
    this.saving = true;
    this.http.post(`${environment.apiUrl}/api/events`, this.createForm.value)
      .subscribe({
        next: () => {
          this.successMessage = 'Event created!';
          this.loadEvents();
          this.saving = false;
        },
        error: err => {
          this.errorMessage = err.error?.message ?? 'Failed to create event';
          this.saving = false;
        }
      });
  }

  publishEvent(id: string) {
    this.http.post(`${environment.apiUrl}/api/events/${id}/publish`, {})
      .subscribe({ next: () => this.loadEvents() });
  }
}

16. All Features & Use Cases — End to End

Feature Matrix

#	Feature	Actor	Services involved	Pattern
UC-01	Browse & search events	Anonymous	EventCatalog	Query
UC-02	View event details & seat map	Anonymous	EventCatalog	Query
UC-03	Reserve a seat	User	Booking, Redis	Command + Redis Lock
UC-04	Process payment	System	Payment → Booking	Event Consumer
UC-05	Confirm booking & issue ticket	System	Booking, Notification	Saga + Event
UC-06	Cancel on payment failure	System	Booking, EventCatalog, Notification	Saga + Compensation
UC-07	View my ticket history	User	Booking	Query
UC-08	Admin create event (draft)	Admin	EventCatalog	Command
UC-09	Admin publish event	Admin	EventCatalog	Command
UC-10	Session abandonment / timeout	System	Redis	TTL auto-release
UC-11	Rate limiting	System	ApiGateway	Fixed window 300/min
UC-12	Token refresh	User	Identity, Angular	Silent refresh OIDC

UC-01 & UC-02: Browse & View Events

GET /api/events?city=London&category=Concert&page=1
          │
API Gateway (no auth required for this route)
          │
EventCatalog: SearchEventsQuery dispatched via MediatR
          │
ValidationBehaviour → LoggingBehaviour → SearchEventsHandler
          │
EF Core: SELECT * FROM Events WHERE IsPublished=1
         AND City='London' AND Category='Concert'
         ORDER BY StartsAtUtc
         OFFSET 0 ROWS FETCH NEXT 20 ROWS ONLY
          │
Returns List<EventDto> → 200 OK

UC-03: Reserve a Seat (Full Journey)

POST /api/bookings
{ "eventId": "...", "seatNumber": "A12", "totalAmount": 99.99 }
          │
API Gateway: JWT validated → route to Booking:8080
          │
BookingsController.Create()
          │
MediatR: CreateBookingCommand dispatched
          │
ValidationBehaviour: all fields validated
LoggingBehaviour: "Handling CreateBookingCommand" logged
          │
CreateBookingCommandHandler:
  1. Redis: SETNX seat-lock:{eventId}:A12 "locked" EX 600
       → true: lock acquired, continue
       → false: throw "Seat held by another user" → 409 Conflict
  2. Booking.Create() → Status=Pending, raises BookingCreatedEvent
  3. _repo.Add(booking)
  4. _uow.SaveChangesAsync():
       ├── INSERT INTO Bookings ...
       └── INSERT INTO OutboxMessages (EventType, Payload, OccurredOnUtc)
           (same transaction)
  5. Return booking.Id → 201 Created

Background (OutboxProcessor polls every 5 seconds):
  SELECT TOP 10 FROM OutboxMessages WHERE ProcessedOnUtc IS NULL
  → Deserialize BookingCreatedEvent
  → Publish BookingCreatedIntegrationEvent to RabbitMQ
  → UPDATE OutboxMessages SET ProcessedOnUtc = NOW()

RabbitMQ → BookingSagaStateMachine:
  Initially: receives BookingCreated
  → stores saga state to SQL Server
  → publishes ProcessPaymentCommand
  → transitions to AwaitingPayment

UC-04 & UC-05: Payment Processing & Confirmation

RabbitMQ → ProcessPaymentConsumer (Payment Service):
  Payment.Create(bookingId, userId, amount, currency)
  → StripeService.ChargeAsync(99.99, "USD")
  → payment.Succeed("pi_3Nxxxx")
  → INSERT INTO Payments ...
  → Publish PaymentProcessedEvent(Success=true, PspReference="pi_3Nxxxx")

RabbitMQ → BookingSagaStateMachine:
  During AwaitingPayment, PaymentCompleted(Success=true):
  → saga.PaymentReference = "pi_3Nxxxx"
  → Publish IssueTicketCommand (EventCatalog decrements AvailableSeats)
  → Publish BookingConfirmedIntegrationEvent
  → Transition to Completed → Finalize

RabbitMQ → BookingConfirmedConsumer (Notification Service):
  Receive BookingConfirmedIntegrationEvent
  → IEmailService.SendAsync(
       to: "user@example.com",
       subject: "Your ticket is confirmed! 🎟️",
       body: "Booking abc123 confirmed. Payment: pi_3Nxxxx")

UC-06: Payment Failure & Compensation

RabbitMQ → ProcessPaymentConsumer (Payment Service):
  payment.Fail("Insufficient funds")
  → INSERT INTO Payments (Status='Failed', FailureReason='...')
  → Publish PaymentProcessedEvent(Success=false, FailureReason="Insufficient funds")

RabbitMQ → BookingSagaStateMachine:
  During AwaitingPayment, PaymentCompleted(Success=false):
  → Publish ReleaseSeatsCommand:
       Redis DEL seat-lock:{eventId}:A12  ← seat available again
       EventCatalog: event.ReleaseSeat()   ← AvailableSeats++
  → Publish BookingCancelledIntegrationEvent
  → Transition to Failed → Finalize

RabbitMQ → BookingCancelledConsumer (Notification Service):
  Receive BookingCancelledIntegrationEvent
  → IEmailService.SendAsync(
       subject: "Booking cancelled",
       body: "Your booking was cancelled. Reason: Insufficient funds.
              Seat A12 has been released.")

UC-07: View My Ticket History

GET /api/bookings/user/{userId}
          │
API Gateway: JWT validated
          │
BookingsController.GetByUser()
          │
MediatR: GetBookingsByUserQuery
          │
GetBookingsByUserHandler:
  EF Core: SELECT * FROM Bookings WHERE UserId = @userId
          │
Returns List<BookingDto> with status, seat, amount, timestamps

UC-08 & UC-09: Admin Creates & Publishes Event

POST /api/events (requires Admin role in JWT)
{ "title": "Coldplay Live", "category": "Concert", "totalSeats": 500 ... }
          │
EventsController: [Authorize(Roles = "Admin")]
          │
MediatR: CreateEventCommand
          │
CreateEventValidator: all fields validated
          │
CreateEventHandler:
  Event.Create(...)  → IsPublished=false (always draft)
  → INSERT INTO Events ...
          │
Returns 201 Created with event.Id

POST /api/events/{id}/publish (Admin only)
          │
PublishEventCommand
          │
event.Publish() → IsPublished=true
          │
Event now appears in search results (WHERE IsPublished=1)

UC-10: Session Abandonment / Timeout

User starts checkout → seat locked in Redis (TTL: 10 minutes)
User closes browser / session expires
          │
Redis TTL expires automatically after 10 minutes:
  DEL seat-lock:{eventId}:{seat}
          │
No cleanup job needed.
Seat becomes available again for the next customer.

If payment was already initiated (saga in AwaitingPayment):
  → Saga timeout (configurable) triggers compensation
  → ReleaseSeatsCommand published
  → BookingCancelledIntegrationEvent published

UC-11: Rate Limiting (Gateway)

Client sends >300 requests/minute to any /api/** route
          │
YARP RateLimiter middleware:
  AddFixedWindowLimiter("api", window=1min, permitLimit=300)
          │
→ 429 Too Many Requests returned immediately
  Downstream services are protected — they never see the request

UC-12: Token Refresh (Silent OIDC)

Angular SPA: access token approaching expiry
          │
angular-oauth2-oidc: setupAutomaticSilentRefresh()
  → silent iframe to Identity service
  → Identity issues new access token via refresh flow
  → OAuthService.accessToken updated
          │
Next HTTP request automatically carries fresh token via authInterceptor
No user interaction required

17. Infrastructure: Docker Compose & SQL Server

📁 docker-compose.yml

name: ticketing-system

services:
  sqlserver:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      ACCEPT_EULA: "Y"
      MSSQL_SA_PASSWORD: "${SA_PASSWORD}"
    ports: ["1433:1433"]
    volumes: [sqlserver_data:/var/opt/mssql]
    healthcheck:
      test: ["CMD", "/opt/mssql-tools18/bin/sqlcmd",
             "-S", "localhost", "-U", "sa", "-P", "${SA_PASSWORD}",
             "-Q", "SELECT 1", "-No"]
      interval: 15s
      retries: 10
      start_period: 30s    # MSSQL needs time to initialise

  rabbitmq:
    image: rabbitmq:3.13-management-alpine
    environment:
      RABBITMQ_DEFAULT_USER: "${RABBITMQ_USER}"
      RABBITMQ_DEFAULT_PASS: "${RABBITMQ_PASS}"
    ports:
      - "5672:5672"
      - "15672:15672"      # Management UI → http://localhost:15672
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "ping"]
      interval: 15s
      retries: 10

  redis:
    image: redis:7-alpine
    command: redis-server --requirepass ${REDIS_PASSWORD}
    ports: ["6379:6379"]

  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"      # Jaeger UI → http://localhost:16686
      - "4317:4317"        # OTLP gRPC

  booking:
    build:
      context: .
      dockerfile: src/Services/Booking/Dockerfile
    environment:
      ASPNETCORE_URLS: http://+:8080
      ConnectionStrings__SqlServer: >
        Server=sqlserver,1433;Database=TicketingSystem_Booking;
        User Id=sa;Password=${SA_PASSWORD};TrustServerCertificate=true
      ConnectionStrings__Redis: "redis:6379,password=${REDIS_PASSWORD}"
      RabbitMQ__Host: rabbitmq
      RabbitMQ__Username: "${RABBITMQ_USER}"
      RabbitMQ__Password: "${RABBITMQ_PASS}"
    ports: ["5003:8080"]
    depends_on:
      sqlserver: { condition: service_healthy }
      rabbitmq:  { condition: service_healthy }
      redis:     { condition: service_healthy }

  # ... identity, eventcatalog, payment, notification, gateway, frontend
  # Full file: https://github.com/naimulkarim/TicketingSystem/blob/main/docker-compose.yml

volumes:
  sqlserver_data:
  rabbitmq_data:
  redis_data:

Local endpoint reference:

Service	URL	Notes
Frontend SPA	http://localhost:4200	Angular
API Gateway	http://localhost:5000	All API calls go here
Identity	http://localhost:5001	OIDC discovery at /.well-known/openid-configuration
Event Catalog	http://localhost:5002	Swagger at /swagger
Booking	http://localhost:5003	Swagger at /swagger
Payment	http://localhost:5004	Internal only
Notification	http://localhost:5005	Internal only
RabbitMQ UI	http://localhost:15672	user: ticketing / ticketing_pass
Jaeger UI	http://localhost:16686	Distributed traces

18. CI/CD: GitHub Actions Pipeline

📁 .github/workflows/ci.yml

name: CI

on:
  push:         { branches: [main, develop] }
  pull_request: { branches: [main] }

jobs:
  dotnet:
    name: .NET Build & Test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with: { dotnet-version: "8.0.x" }
      - run: dotnet restore TicketingSystem.sln
      - run: dotnet build TicketingSystem.sln --no-restore -c Release
      - run: |
          dotnet test tests/Unit/ -c Release \
            --logger "trx;LogFileName=unit-results.trx" \
            --collect:"XPlat Code Coverage"
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: unit-test-results, path: "**/*.trx" }

  angular:
    name: Angular Build
    runs-on: ubuntu-latest
    defaults: { run: { working-directory: frontend } }
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20.x", cache: npm,
                cache-dependency-path: frontend/package-lock.json }
      - run: npm ci
      - run: npm run lint || true
      - run: npm run build      # production build: tree-shaking + strict types

  docker:
    name: Docker Image Build
    needs: [dotnet, angular]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        service:
          - { name: identity,     dockerfile: src/Services/Identity/Dockerfile }
          - { name: eventcatalog, dockerfile: src/Services/EventCatalog/Dockerfile }
          - { name: booking,      dockerfile: src/Services/Booking/Dockerfile }
          - { name: payment,      dockerfile: src/Services/Payment/Dockerfile }
          - { name: notification, dockerfile: src/Services/Notification/Dockerfile }
          - { name: gateway,      dockerfile: src/ApiGateway/Dockerfile }
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v5
        with:
          context: .
          file: ${{ matrix.service.dockerfile }}
          push: false
          tags: ticketing/${{ matrix.service.name }}:ci
          cache-from: type=gha    # GitHub Actions layer cache
          cache-to: type=gha,mode=max

The pipeline runs 3 parallel jobs. docker waits for both dotnet and angular to pass. All 6 service images are built simultaneously in the matrix — if any Dockerfile is broken, the pipeline fails before anything reaches production.

19. Unit Tests

📁 tests/Unit/

Stack: xunit · FluentAssertions · NSubstitute

Aggregate Invariant Tests

// tests/Unit/Booking/BookingAggregateTests.cs

public class BookingAggregateTests
{
    [Fact]
    public void Create_ShouldSetPendingStatus()
    {
        var booking = MakeBooking();
        booking.Status.Should().Be(BookingStatus.Pending);
    }

    [Fact]
    public void Create_ShouldRaiseBookingCreatedEvent()
    {
        var booking = MakeBooking();
        booking.DomainEvents.Should().ContainSingle()
            .Which.Should().BeOfType<BookingCreatedEvent>();
    }

    [Fact]
    public void Confirm_WhenPending_ShouldSetConfirmedStatus()
    {
        var booking = MakeBooking();
        booking.Confirm("pi_123");
        booking.Status.Should().Be(BookingStatus.Confirmed);
    }

    [Fact]
    public void Confirm_WhenAlreadyConfirmed_ShouldThrow()
    {
        var booking = MakeBooking();
        booking.Confirm("pi_123");
        var act = () => booking.Confirm("pi_456");
        act.Should().Throw<InvalidOperationException>();
    }

    [Fact]
    public void Cancel_WhenPending_ShouldRaiseBookingCancelledEvent()
    {
        var booking = MakeBooking();
        booking.Cancel("Payment failed");
        booking.DomainEvents.Should()
            .Contain(e => e is BookingCancelledEvent);
    }

    [Fact]
    public void Cancel_WhenAlreadyCancelled_ShouldThrow()
    {
        var booking = MakeBooking();
        booking.Cancel("first");
        var act = () => booking.Cancel("second");
        act.Should().Throw<InvalidOperationException>();
    }

    [Fact]
    public void ClearDomainEvents_ShouldRemoveAll()
    {
        var booking = MakeBooking();
        booking.Confirm("pi_x");
        booking.ClearDomainEvents();
        booking.DomainEvents.Should().BeEmpty();
    }

    private static Booking MakeBooking() =>
        Booking.Create(Guid.NewGuid(), Guid.NewGuid(), "A1", 99.99m, "USD");
}

Command Handler Tests with Mocks

// tests/Unit/Booking/CreateBookingCommandHandlerTests.cs

public class CreateBookingCommandHandlerTests
{
    private readonly IBookingRepository _repo    = Substitute.For<IBookingRepository>();
    private readonly IUnitOfWork        _uow     = Substitute.For<IUnitOfWork>();
    private readonly ISeatLockService   _lock    = Substitute.For<ISeatLockService>();

    [Fact]
    public async Task Handle_WhenSeatAvailable_ShouldAddBookingAndReturnId()
    {
        _lock.TryLockAsync(Arg.Any<Guid>(), Arg.Any<string>(),
            Arg.Any<TimeSpan>(), Arg.Any<CancellationToken>())
            .Returns(true);

        var handler = new CreateBookingCommandHandler(_repo, _uow, _lock);
        var cmd = new CreateBookingCommand(Guid.NewGuid(), Guid.NewGuid(), "B5", 50m);

        var id = await handler.Handle(cmd, CancellationToken.None);

        id.Should().NotBeEmpty();
        _repo.Received(1).Add(Arg.Any<Booking>());
        await _uow.Received(1).SaveChangesAsync(Arg.Any<CancellationToken>());
    }

    [Fact]
    public async Task Handle_WhenSeatLocked_ShouldThrowAndNotPersist()
    {
        _lock.TryLockAsync(Arg.Any<Guid>(), Arg.Any<string>(),
            Arg.Any<TimeSpan>(), Arg.Any<CancellationToken>())
            .Returns(false);

        var handler = new CreateBookingCommandHandler(_repo, _uow, _lock);
        var cmd = new CreateBookingCommand(Guid.NewGuid(), Guid.NewGuid(), "C3", 75m);

        var act = async () => await handler.Handle(cmd, CancellationToken.None);

        await act.Should().ThrowAsync<InvalidOperationException>()
            .WithMessage("*held by another user*");

        _repo.DidNotReceive().Add(Arg.Any<Booking>());
        await _uow.DidNotReceive().SaveChangesAsync(Arg.Any<CancellationToken>());
    }
}

Value Object Tests

// tests/Unit/SharedKernel/ValueObjectTests.cs

public class ValueObjectTests
{
    [Fact]
    public void TwoMoneyObjects_WithSameValues_ShouldBeEqual()
    {
        var a = new Money(99.99m, "USD");
        var b = new Money(99.99m, "USD");
        a.Should().Be(b);
        (a == b).Should().BeTrue();
    }

    [Fact]
    public void TwoMoneyObjects_WithDifferentAmounts_ShouldNotBeEqual()
    {
        var a = new Money(10m, "USD");
        var b = new Money(20m, "USD");
        a.Should().NotBe(b);
    }

    [Fact]
    public void Money_WithNegativeAmount_ShouldThrow()
    {
        var act = () => new Money(-1m, "USD");
        act.Should().Throw<ArgumentException>();
    }
}

20. What to Build Next

Event Sourcing on the Booking aggregate — instead of storing current state, store every domain event that led to it (BookingCreated, BookingConfirmed, etc.). This provides a complete immutable audit log and the ability to replay history to any point in time.

Dedicated read model — currently queries hit the same SQL Server write database. For high read traffic (event search, seat availability), a separate read store updated by RabbitMQ projectors would scale reads independently of writes.

Kubernetes Helm charts — the Docker Compose configuration translates directly to Helm. Each service gets a Deployment, HorizontalPodAutoscaler, Service, and Ingress. Cert-manager handles TLS automatically.

Pact contract tests — integration events are the public API between services. Pact consumer-driven contract tests verify that producer events always satisfy what consumers expect, catching breaking changes before they reach the main branch.

Inbox deduplication — the outbox guarantees at-least-once delivery. The natural complement is an Inbox table per consumer that records received message IDs and skips duplicates, achieving exactly-once processing semantics.

Real Stripe integration — replace StripeService stub with Stripe.net SDK, add webhook endpoint for async payment confirmation, handle 3DS authentication flows.

21. Conclusion & GitHub Link

DDD, Event-Driven Architecture, and CQRS answer three distinct problems that every non-trivial distributed system eventually faces:

DDD answers: where does business logic live, and how do we stop it from rotting? — in aggregates, expressed in the language of the domain, protected by invariants that can never be violated regardless of which service or layer calls into them.

CQRS answers: how do we decouple the complexity of writes from the performance demands of reads? — commands own the write path with full domain model validation; queries own the read path with optimised projections, neither touching the other.

Event-Driven Architecture answers: how do loosely-coupled services coordinate without becoming entangled? — through immutable events on a message bus, with the booking saga orchestrating multi-service workflows, the outbox pattern ensuring zero message loss, and Redis seat locking preventing the fundamental overselling race condition.

Together these patterns produce a system where:

Each service can be deployed, scaled, and tested independently
Business invariants are impossible to violate by accident
Failures are handled by design, not by hope
The codebase reads like the domain it models

🚀 Get the Full Source Code

GitHub Repository: github.com/naimulkarim/TicketingSystem

# Clone and run the full stack locally
git clone https://github.com/naimulkarim/TicketingSystem.git
cd TicketingSystem
cp .env.example .env       # add your secrets
docker compose up --build  # starts everything
# Open http://localhost:4200

Direct file links referenced in this article:

File	Link
SharedKernel — AggregateRoot	`src/SharedKernel/Domain/AggregateRoot.cs`
SharedKernel — Primitives	`src/SharedKernel/Domain/Primitives.cs`
SharedKernel — Pipeline Behaviours	`src/SharedKernel/Application/Behaviours/PipelineBehaviours.cs`
Identity Config	`src/Services/Identity/Api/IdentityConfig.cs`
Event Aggregate	`src/Services/EventCatalog/Domain/Aggregates/Event.cs`
Booking Aggregate	`src/Services/Booking/Domain/Aggregates/Booking.cs`
Booking Events	`src/Services/Booking/Domain/Events/BookingEvents.cs`
CreateBooking Command	`src/Services/Booking/Application/Commands/CreateBookingCommand.cs`
Booking Queries	`src/Services/Booking/Application/Queries/GetBookingQuery.cs`
Booking Saga	`src/Services/Booking/Application/Sagas/BookingSagaStateMachine.cs`
Booking DbContext	`src/Services/Booking/Infrastructure/Persistence/BookingDbContext.cs`
Redis Seat Lock	`src/Services/Booking/Infrastructure/RedisSeatLockService.cs`
Booking Controller	`src/Services/Booking/Api/Controllers/BookingsController.cs`
Payment Aggregate	`src/Services/Payment/Domain/Aggregates/Payment.cs`
Payment Consumer	`src/Services/Payment/Infrastructure/Messaging/ProcessPaymentConsumer.cs`
Notification Consumers	`src/Services/Notification/Application/Consumers/BookingConsumers.cs`
API Gateway Program	`src/ApiGateway/Program.cs`
Outbox Message	`src/SharedKernel/Infrastructure/OutboxMessage.cs`
Auth Service (Angular)	`frontend/src/app/core/services/auth.service.ts`
NgRx Booking Store	`frontend/src/app/store/booking/booking.reducer.ts`
Docker Compose	`docker-compose.yml`
GitHub Actions CI	`.github/workflows/ci.yml`
Booking Aggregate Tests	`tests/Unit/Booking/BookingAggregateTests.cs`

If this article helped you, leave a comment — I'm happy to go deeper on any section.

Tags: dotnet csharp architecture ddd eventdriven cqrs angular microservices rabbitmq masstransit designpatterns cleanarchitecture

Running Local AI Models for Free: A Step-by-Step Guide with Python

Naimul Karim — Fri, 01 May 2026 03:45:24 +0000

Large language models are no longer locked behind expensive APIs. Today, you can run powerful AI models locally on your own machine, often for free, while keeping full control over data, latency, and cost.

In this guide, we’ll walk through how to run local models step by step using:

Ollama (CLI + API approach)
LM Studio (GUI approach)
Python integration for automation

Why Run AI Models Locally?

Running LLMs locally is becoming increasingly practical. Here’s why developers are doing it:

Privacy: Data stays on your machine
Zero API costs
Low latency for repeated tasks
Offline usage
Full control over models and prompts

Popular Free Local Models

Some of the best open models you can run locally today:

Llama 3 (Meta)
Mistral / Mixtral
Qwen2 / Qwen2.5 (Alibaba)
Gemma (Google)

These models come in different sizes (7B, 8B, 13B, etc.), allowing you to choose based on your hardware.

Method 1: Running Models with Ollama (Recommended)

Ollama is the easiest way to run local LLMs.

Step 1: Install Ollama

Download it from:

https://ollama.com

Install it like any normal application.

Supported platforms:

Windows
macOS
Linux

Step 2: Run Your First Model

Open your terminal and run:

bash ollama run llama3

Step 3: Chat with the Model

Just type:
Explain machine learning in simple terms

METHOD 2: Ollama with Python API

Ollama runs a local API server at:

http://localhost:11434

Basic Python Example

import requests
url = "http://localhost:11434/api/generate"

payload = {
    "model": "llama3",
    "prompt": "Explain recursion in simple terms",
    "stream": False
}
response = requests.post(url, json=payload)
print(response.json()["response"])

SecretVault: One Interface to Rule All Your Cloud Secrets in .NET 9

Naimul Karim — Wed, 29 Apr 2026 01:40:39 +0000

Every .NET project that touches the cloud eventually runs into the same problem: secret sprawl.

You start with AWS Secrets Manager. Then a new service needs Azure Key Vault. Someone spins up a HashiCorp Vault. Before long, your codebase is littered with provider-specific SDKs, inconsistent error handling, and zero ability to swap providers without rewriting half your infrastructure code.

I built SecretVault to fix this. One interface. Four providers. Zero lock-in.

The Problem in Plain Code

Here's what most codebases look like today:

// Somewhere in your AWS service
var awsClient = new AmazonSecretsManagerClient();
var response = await awsClient.GetSecretValueAsync(new GetSecretValueRequest
{
    SecretId = "prod/db-password"
});
var secret = response.SecretString;

// Somewhere else, for Azure
var kvClient = new SecretClient(vaultUri, new DefaultAzureCredential());
var azureSecret = await kvClient.GetSecretAsync("db-password");
var value = azureSecret.Value.Value;

Different SDKs. Different error types. Different mental models. And when you want to add caching or rotate secrets? You wire it up manually — for each provider — every time.

The SecretVault Way

// That's it. Same line. Any provider.
var password = await secrets.GetSecretAsync("db-password");

ISecretManager is the only interface your application code ever sees. Swap AWS for Azure in one line of DI configuration — your business logic never changes.

What's in the Box

SecretVault ships as 6 focused NuGet packages — install only what you need:

Package	What it does
`SecretVault.Core`	The `ISecretManager` interface + caching + chaining
`SecretVault.Aws`	AWS Secrets Manager provider
`SecretVault.Azure`	Azure Key Vault provider
`SecretVault.HashiCorp`	HashiCorp Vault KV v2 provider
`SecretVault.Gcp`	Google Cloud Secret Manager provider
`SecretVault.Extensions.Hosting`	ASP.NET Core DI + `IConfiguration` + health checks

dotnet add package SecretVault.Core
dotnet add package SecretVault.Aws
dotnet add package SecretVault.Extensions.Hosting

Quick Start: AWS

using Amazon.SecretsManager;
using SecretVault.Aws;

var client = new AmazonSecretsManagerClient();
ISecretManager secrets = new AwsSecretManager(client, logger);

// Fetch a raw string
var apiKey = await secrets.GetSecretAsync("prod/api-key");

// Fetch and deserialize JSON directly into a typed object
var dbConfig = await secrets.GetSecretAsync<DatabaseConfig>("prod/db-config");

Switching to Azure? One line changes:

// Before
ISecretManager secrets = new AwsSecretManager(client, logger);

// After
ISecretManager secrets = new AzureKeyVaultSecretManager(kvClient, logger);

Your application code above stays identical.

Built-in Caching

Cloud secret APIs have rate limits and latency. Fetching the same secret on every request is wasteful. Wrap any provider with CachedSecretManager:

var cached = new CachedSecretManager(
    inner: secrets,
    cache: memoryCache,
    logger: logger,
    options: new CachedSecretManagerOptions
    {
        AbsoluteExpiration = TimeSpan.FromMinutes(10)
    });

The cache is automatically invalidated on writes and deletes — so you never serve stale secrets after a rotation.

Fallback Chaining

Running a hybrid cloud? Or want a local fallback for development? ChainedSecretManager tries providers in order and returns the first successful result:

var chained = new ChainedSecretManager(
    [awsSecrets, localSecrets],
    logger);

// Tries AWS first, falls back to local dev secrets if not found
var value = await chained.GetSecretAsync("api-key");

This is great for local development too — point your fallback at environment variables or a local JSON file without touching your production code path.

Secret Rotation

All provider implementations also implement ISecretRotationManager, giving you first-class rotation support:

ISecretRotationManager rotatable = new AwsSecretManager(client, logger);

var result = await rotatable.RotateSecretAsync(
    secretName: "prod/db-password",
    newValue: GenerateSecurePassword(),
    gracePeriod: TimeSpan.FromHours(1)  // old version stays accessible during rollover
);

Console.WriteLine($"Rotated to version {result.NewVersion} at {result.RotatedAt}");
// Optionally: notify dependent services, invalidate connection pools, etc.

You can also list all versions and retrieve a specific one:

var versions = await rotatable.ListVersionsAsync("prod/db-password");
var oldValue = await rotatable.GetSecretVersionAsync("prod/db-password", versions[1].Version);

ASP.NET Core Integration

Load secrets into `IConfiguration` at startup

Pull secrets directly into the config pipeline so they're accessible via IConfiguration just like appsettings.json:

// Program.cs
builder.Configuration.AddSecretVault(secretManager, [
    "db-connection-string",
    "stripe-api-key",
    "sendgrid-api-key"
]);

// Then use them as normal config
var connStr = builder.Configuration["db-connection-string"];

Register via Dependency Injection

builder.Services.AddSecretVault(secretManager, options =>
{
    options.AbsoluteExpiration = TimeSpan.FromMinutes(5);
});

Inject at runtime

public class PaymentService(ISecretManager secrets)
{
    public async Task ProcessPayment(decimal amount)
    {
        var stripeKey = await secrets.GetSecretAsync("stripe-api-key");
        // use it
    }
}

Health checks

builder.Services.AddHealthChecks()
    .AddSecretVaultHealthCheck("__probe__", name: "secrets");

This adds a /health endpoint entry that pings your secret provider on every health check — giving you early warning if credentials expire or network access is lost.

Error Handling

SecretVault normalizes provider-specific exceptions into a clean hierarchy so you never have to catch Amazon.Runtime.AmazonServiceException or Azure.RequestFailedException in your business logic:

try
{
    var secret = await secrets.GetSecretAsync("my-secret");
}
catch (SecretNotFoundException ex)
{
    // Secret doesn't exist in the provider
    logger.LogWarning("Secret {Name} not found", ex.SecretName);
}
catch (SecretAccessDeniedException)
{
    // IAM / RBAC permissions issue
    logger.LogError("Access denied — check your IAM role");
}
catch (SecretProviderUnavailableException)
{
    // Network or service outage
    logger.LogCritical("Secret provider is down");
}

CI/CD — Automated NuGet Publishing

The repo ships with two GitHub Actions workflows out of the box.

ci.yml — runs on every push and PR:

Restores dependencies
Builds in Release mode
Runs all tests with code coverage

publish.yml — triggered by a version tag:

git tag v1.0.0
git push origin v1.0.0

This automatically:

Runs the full test suite
Packs all 6 NuGet packages with the correct version
Pushes them to nuget.org
Creates a GitHub Release with auto-generated changelog

Zero manual steps after tagging.

Design Decisions Worth Knowing

Why separate packages instead of one big one?
If you only use AWS, you shouldn't have the Azure and GCP SDKs in your dependency tree. Each provider package is independent — install exactly what you need.

Why not just use IConfiguration directly?
IConfiguration is great for startup binding, but it doesn't support on-demand fetching, rotation, versioning, or cache invalidation. SecretVault gives you both — startup binding and a runtime API through the same abstraction.

Why .NET 9 only?
To keep the codebase lean and take advantage of modern C# features (primary constructors, collection expressions, nullable reference types). A netstandard2.0 target may come in a future release if there's demand.

Get Started

# Clone
git clone https://github.com/YourUsername/SecretVault.git
cd SecretVault

# Build
dotnet build SecretVault.sln

# Test
dotnet test SecretVault.sln

Or grab it straight from NuGet:

dotnet add package SecretVault.Core
dotnet add package SecretVault.Aws        # or .Azure / .HashiCorp / .Gcp
dotnet add package SecretVault.Extensions.Hosting

What's Next

A few things on the roadmap:

SecretVault.Local — a file/environment variable provider for local development
Automatic rotation scheduling — background IHostedService that rotates on a cron
Encryption at rest layer — wrap any provider with AES encryption before storing
.NET Standard 2.0 target — for broader framework compatibility

PRs and issues welcome. If you're using a provider not listed here, opening an issue with your use case is the fastest path to getting it supported.

Built with .NET 9, xUnit, FluentAssertions, and NSubstitute. MIT licensed.

⭐ If SecretVault saves you some boilerplate, a star on GitHub goes a long way.

View on GitHub · NuGet Packages

A Unified View of AI Evolution: From Machine Learning to LLMs, RAG, and Fine-Tuning

Naimul Karim — Tue, 28 Apr 2026 03:34:24 +0000

Artificial Intelligence has progressed far beyond its early rule-based origins. What once depended on predefined logic has evolved into systems that can learn from data, reason through problems, and even generate entirely new content. This transformation is largely driven by Machine Learning (ML), where algorithms improve their performance by identifying patterns in large datasets rather than following hard-coded instructions.

Building on ML, Deep Learning introduces neural networks—layered computational structures inspired by the human brain. These networks excel at processing complex data such as images, speech, and text. The next leap in this evolution is Generative AI, which shifts the focus from analyzing data to creating it. Whether producing text, images, or audio, generative systems mimic human creativity in increasingly sophisticated ways.

Large Language Models: The Core of Modern GenAI

At the center of today’s generative revolution are Large Language Models (LLMs). These models are designed to interpret and produce human-like language, enabling natural conversations, content generation, and problem-solving.

Most modern LLMs are built on the Transformer architecture, introduced in the landmark concept “Attention Is All You Need.” This architecture uses attention mechanisms to understand how words relate to each other in context, making it far more effective than earlier sequence models.

Some prominent LLM families include:

OpenAI models such as GPT-4.5, GPT-4o, and smaller optimized variants
Anthropic’s Claude series (e.g., Claude 3.5 Sonnet, Claude 3 Opus)
Meta’s Llama models (e.g., Llama 3.x series)
Google’s Gemini models

These models power a wide range of applications, from chatbots and virtual assistants to marketing content generation, document summarization, and even software development tasks like debugging and code generation.

How LLMs Work in Practice

Accessing LLMs

Users can interact with LLMs through intuitive interfaces (chat-based systems) or integrate them into applications using APIs.

Prompting and Instructions

To guide an LLM toward the desired output, users provide structured inputs—this process is known as prompt engineering. The clarity and design of prompts significantly influence the quality of responses.

Understanding Language via Embeddings

LLMs convert text into numerical representations called embeddings. These vectors capture semantic meaning, enabling the model to understand relationships between words, phrases, and broader contexts.

Controlling Output with Temperature

LLMs do not always produce the same answer. A parameter called temperature controls how deterministic or creative the output is. Lower values lead to predictable responses, while higher values increase variability and creativity.

Grounding LLMs with Real-World Knowledge

Despite their capabilities, LLMs are inherently general-purpose. They do not automatically know company-specific or real-time information. To make them useful in practical settings, additional context must be provided. Two key approaches enable this: Retrieval-Augmented Generation (RAG) and Fine-Tuning.

Tokenization: How Models Read Language

Computers don’t interpret language the way humans do. Instead of understanding full words or sentences directly, Large Language Models break text into smaller units called tokens. These tokens are then converted into numbers so the model can process them mathematically.

A token isn’t always a whole word—it can be:

A complete word (“river”)
A fragment of a word (“run” + “ning”)
Symbols or punctuation (like “?” or “.”)

Different AI models use different methods to split text into tokens. Some rely on frequently occurring patterns, while others use statistical approaches to segment text.

On average, in English:

1 token is roughly equal to 4 characters
Or about ¾ of a word

Why this matters:

Efficiency: Breaking text into tokens allows models to process language in a structured way.
Flexibility: Even unfamiliar words can be understood by splitting them into smaller parts.
Cost impact: More tokens directly increase usage costs.

Example:
Sentence: “Learning AI is fun.”
Possible tokens: “Learn”, “ing”, “ AI”, “ is”, “ fun”, “.”
Each of these is internally mapped to a numeric ID for computation.

Context Windows: The Model’s Working Memory

LLMs don’t have unlimited memory. Instead, they operate within a fixed limit called a context window, which defines how many tokens the model can consider at one time.

Think of it as short-term memory:

Smaller models handle a few thousand tokens
Advanced models can process very large inputs, even entire books

If the input exceeds this limit, older parts are removed from consideration. The model then loses access to that earlier information.

Why this matters:

Conversations: Important details from earlier messages can disappear in long chats.
Large files: Long reports or documents must often be split into sections.
Cost vs capability: Larger windows provide more context but require more resources.

Example:
Imagine summarizing a long report. If the report is longer than the model’s context window, only the most recent sections might be considered unless special techniques are used.

Token-Based Pricing: Why Usage Adds Up

Most AI platforms charge based on token usage. This includes both:

Input tokens: The text you send
Output tokens: The text generated in response

The total cost depends on the combined number of tokens processed.

Simple breakdown:

Total tokens = input + output
Pricing is typically calculated per 1,000 tokens

Why this matters:

Efficiency saves money: Shorter prompts reduce cost
Control output: Limiting response length prevents unnecessary usage
Planning: Developers often estimate token usage before sending requests

Example:

Input: 300 tokens (a short question with context)
Output: 450 tokens (a detailed answer)
Total: 750 tokens

This total determines the cost of that interaction.

Retrieval-Augmented Generation (RAG)

RAG enhances LLMs by connecting them to external knowledge sources such as databases, documents, or APIs.

How It Works

A user submits a query
Relevant information is retrieved from a knowledge source
This information is added to the model’s input
The LLM generates a response grounded in both its training and the retrieved data

Benefits

Produces more accurate, fact-based responses
Keeps information up to date without retraining
Cost-efficient since the base model remains unchanged

Trade-offs

Slightly slower due to the retrieval step

Fine-Tuning: Customizing Intelligence

Fine-tuning takes a pre-trained LLM and further trains it on domain-specific data. This process embeds specialized knowledge directly into the model.

How It Works

A base model is trained further on curated datasets
It learns domain terminology, patterns, and workflows

Benefits

Faster responses (no external lookup required)
Highly tailored outputs aligned with specific use cases

Trade-offs

Expensive in terms of compute and maintenance
Requires retraining to incorporate new information

RAG vs Fine-Tuning

RAG and fine-tuning serve different but complementary purposes:

RAG is ideal for dynamic, frequently changing knowledge
Fine-tuning is best for consistent, domain-specific expertise

In real-world systems, combining both often yields the best results—fine-tuning for behavior and tone, and RAG for factual accuracy and freshness.

Understanding LLM Performance

The effectiveness of an LLM is closely tied to its size, typically measured by the number of parameters (often in billions). Larger models tend to perform better on complex tasks but require significant computational resources to train and deploy.

This trade-off has led to the rise of smaller, efficient models—sometimes called “mini-giants”—which aim to deliver strong performance with lower cost and latency.

I Built a Natural Language to SQL Generator with LangChain, Groq, and Streamlit

Naimul Karim — Tue, 28 Apr 2026 03:32:57 +0000

TL;DR: I built an app where you type "Show me the top 5 customers by spending" and it writes the SQL, runs it against a real database, shows you the results in a table, and explains what they mean — all powered by a free LLM. This is the full tutorial.

The Problem This Solves

Most people who need data don't know SQL. They know the question they want answered — "which products are selling the most?", "which customers haven't ordered in 3 months?" — but they can't translate it into a query.

Text-to-SQL bridges that gap. You ask in plain English, the AI writes the SQL, the database runs it, and you get your answer.

This is one of the hottest enterprise AI use cases right now. Companies like Salesforce, Notion, and linear are all building natural language database interfaces. In this tutorial, you'll build one from scratch in about an hour — using completely free tools.

What We're Building

A Streamlit web app with:

A text input where you type your question
An AI that generates the correct SQL query
A live SQLite database that executes the query
Results displayed as a sortable, downloadable table
A plain English explanation of what the results mean
A sidebar with 10 example questions to try
A query history panel tracking your last 10 queries
A safety filter that blocks any non-SELECT queries

Here's what the full pipeline looks like:

User question
     ↓
LangChain + Groq LLM (reads schema + generates SQL)
     ↓
Safety validator (blocks DROP, DELETE, UPDATE)
     ↓
SQLite execution
     ↓
Results table + CSV download + plain English explanation

The Tech Stack

Component	Tool	Why
LLM	Llama 3.1 8B via Groq	Free, fast (< 2s), great SQL generation
Database	SQLite	Zero setup, file-based, perfect for demos
ORM	SQLAlchemy	Schema inspection for LLM context
Framework	LangChain	Prompt management and pipeline structure
Frontend	Streamlit	Full web UI in pure Python

Total cost: $0. Groq gives you 14,400 free requests per day.

Setup

Install dependencies

pip install streamlit langchain langchain-community langchain-core \
  groq python-dotenv sqlalchemy tabulate pandas

Get your free Groq API key

Go to console.groq.com
Sign up with Google — one click, no credit card
Click API Keys → Create API Key
Copy the key

Create your `.env` file

GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxxxxx

Step 1: Create a Sample Database

First we need something to query. Here's a script that creates a realistic company database with customers, products, orders, and order items:

# sample_db.py
import sqlite3

conn = sqlite3.connect("company.db")
cur  = conn.cursor()

cur.executescript("""
CREATE TABLE IF NOT EXISTS customers (
    id      INTEGER PRIMARY KEY,
    name    TEXT NOT NULL,
    email   TEXT UNIQUE,
    city    TEXT,
    country TEXT,
    joined  DATE
);

CREATE TABLE IF NOT EXISTS products (
    id       INTEGER PRIMARY KEY,
    name     TEXT NOT NULL,
    category TEXT,
    price    REAL,
    stock    INTEGER
);

CREATE TABLE IF NOT EXISTS orders (
    id          INTEGER PRIMARY KEY,
    customer_id INTEGER REFERENCES customers(id),
    order_date  DATE,
    status      TEXT
);

CREATE TABLE IF NOT EXISTS order_items (
    id         INTEGER PRIMARY KEY,
    order_id   INTEGER REFERENCES orders(id),
    product_id INTEGER REFERENCES products(id),
    quantity   INTEGER,
    unit_price REAL
);
""")

cur.executemany("INSERT OR IGNORE INTO customers VALUES (?,?,?,?,?,?)", [
    (1, "Alice Johnson", "alice@email.com", "New York",  "USA",       "2022-03-15"),
    (2, "Bob Smith",     "bob@email.com",   "London",    "UK",        "2021-07-20"),
    (3, "Carla Rossi",   "carla@email.com", "Rome",      "Italy",     "2023-01-10"),
    (4, "David Lee",     "david@email.com", "Seoul",     "Korea",     "2022-11-05"),
    (5, "Emma Wilson",   "emma@email.com",  "Sydney",    "Australia", "2021-05-30"),
])

cur.executemany("INSERT OR IGNORE INTO products VALUES (?,?,?,?,?)", [
    (1, "Laptop Pro 15",  "Electronics", 1299.99, 45),
    (2, "Wireless Mouse", "Electronics",   29.99, 200),
    (3, "Standing Desk",  "Furniture",    499.99,  30),
    (4, "Office Chair",   "Furniture",    299.99,  50),
    (5, "Python Handbook","Books",         39.99, 120),
])

cur.executemany("INSERT OR IGNORE INTO orders VALUES (?,?,?,?)", [
    (1, 1, "2024-01-15", "completed"),
    (2, 2, "2024-01-18", "completed"),
    (3, 3, "2024-02-02", "shipped"),
    (4, 1, "2024-02-10", "completed"),
    (5, 4, "2024-03-01", "pending"),
])

cur.executemany("INSERT OR IGNORE INTO order_items VALUES (?,?,?,?,?)", [
    (1, 1, 1, 1, 1299.99),
    (2, 1, 2, 2,   29.99),
    (3, 2, 3, 1,  499.99),
    (4, 3, 4, 1,  299.99),
    (5, 4, 1, 1, 1299.99),
    (6, 5, 2, 3,   29.99),
])

conn.commit()
conn.close()
print("✅ company.db created!")

Run it:

python sample_db.py

You now have 4 tables with realistic data. The interesting queries will involve JOINs across multiple tables — exactly what tests an LLM's SQL abilities.

Step 2: The SQL Pipeline

This is the core ML logic. It handles schema reading, SQL generation, safety validation, execution, and explanation.

# sql_pipeline.py
from langchain_community.utilities import SQLDatabase
from groq import Groq
from dotenv import load_dotenv
import sqlite3, os, re

load_dotenv()

GROQ_API_KEY = os.getenv("GROQ_API_KEY")
GROQ_MODEL   = "llama-3.1-8b-instant"
DB_PATH      = "company.db"

client = Groq(api_key=GROQ_API_KEY)

Reading the schema

The key insight in text-to-SQL is that the LLM needs to know your database structure to write correct queries. We pass the schema as part of every prompt:

def get_schema() -> str:
    db = SQLDatabase.from_uri(f"sqlite:///{DB_PATH}")
    return db.get_table_info()

get_table_info() returns something like:

CREATE TABLE customers (
    id INTEGER,
    name TEXT NOT NULL,
    email TEXT,
    ...
)
/*
3 rows from customers table:
id  name          email
1   Alice Johnson alice@email.com
...
*/

This gives the LLM both the structure AND sample data — dramatically improving query quality.

Safety filter

Before executing anything, we block destructive queries:

BLOCKED = ["drop", "delete", "insert", "update", "alter", "truncate", "create"]

def is_safe_query(sql: str) -> bool:
    return not any(kw in sql.lower() for kw in BLOCKED)

This is simple but effective. For production you'd want parameterised queries and a proper SQL parser, but for a demo this covers the important cases.

Generating SQL

The prompt is the most important part. A poorly written prompt produces wrong SQL every time:

def generate_sql(question: str, schema: str) -> str:
    prompt = f"""You are an expert SQL developer working with a SQLite database.

Given the database schema below and a user question, write a correct SQLite SELECT query.

Rules:
- Only write SELECT queries — never DROP, DELETE, UPDATE, INSERT
- Use proper SQLite syntax
- Use table aliases for clarity
- Limit results to 20 rows unless the user asks for more
- Return ONLY the SQL query — no explanation, no markdown, no backticks

Schema:
{schema}

Question: {question}

SQL Query:"""

    response = client.chat.completions.create(
        model=GROQ_MODEL,
        messages=[{"role": "user", "content": prompt}],
        temperature=0,   # deterministic — we want consistent SQL
        max_tokens=512,
    )
    sql = response.choices[0].message.content.strip()

    # Strip markdown code blocks if LLM wraps them anyway
    sql = re.sub(r"```

sql|

```", "", sql).strip()
    return sql

Three things to notice:

temperature=0 — SQL generation should be deterministic. We don't want creative variations, we want the correct query every time.

"Return ONLY the SQL query" — without this instruction, the LLM tends to explain itself before writing the query. That explanation then gets executed as SQL and crashes.

Strip markdown — even with instructions, LLMs sometimes wrap code in backticks. The re.sub handles that defensively.

Executing the query

def execute_sql(sql: str) -> tuple:
    if not is_safe_query(sql):
        raise ValueError("Only SELECT queries are allowed.")

    conn = sqlite3.connect(DB_PATH)
    cur  = conn.cursor()
    cur.execute(sql)
    columns = [desc[0] for desc in cur.description] if cur.description else []
    rows    = cur.fetchall()
    conn.close()
    return columns, rows

Explaining results in plain English

This is what makes the app genuinely useful rather than just a query tool:

def explain_results(question: str, sql: str, columns: list, rows: list) -> str:
    if not rows:
        return "The query returned no results."

    preview = "\n".join(
        str(dict(zip(columns, row))) for row in rows[:5]
    )

    prompt = f"""A user asked: "{question}"

The SQL query returned {len(rows)} rows. Here are the first few:
{preview}

Write a clear, friendly 2-3 sentence explanation of what these results mean.
Highlight the most interesting finding. Do not repeat the raw data."""

    response = client.chat.completions.create(
        model=GROQ_MODEL,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.3,
        max_tokens=256,
    )
    return response.choices[0].message.content.strip()

The full pipeline function

def run_pipeline(question: str) -> dict:
    try:
        schema              = get_schema()
        sql                 = generate_sql(question, schema)

        if not is_safe_query(sql):
            return {"error": "Only SELECT queries are allowed.", "sql": sql}

        columns, rows       = execute_sql(sql)
        explanation         = explain_results(question, sql, columns, rows)

        return {
            "sql":         sql,
            "columns":     columns,
            "rows":        rows,
            "explanation": explanation,
            "error":       None,
        }
    except Exception as e:
        return {
            "sql":         sql if "sql" in locals() else "",
            "columns":     [],
            "rows":        [],
            "explanation": "",
            "error":       str(e),
        }

Step 3: The Streamlit Frontend

# app.py
import streamlit as st
import pandas as pd
from sql_pipeline import run_pipeline
from dotenv import load_dotenv

load_dotenv()

st.set_page_config(page_title="Text to SQL", page_icon="🗄️", layout="wide")
st.title("🗄️ Natural Language to SQL")
st.caption("Ask questions about the database in plain English — get SQL queries and results instantly.")

Sidebar with example questions

Clickable example questions make the app immediately usable without needing to think of a question:

with st.sidebar:
    st.header("💡 Try these questions")

    examples = [
        "Show me all customers from the USA",
        "What are the top 5 most expensive products?",
        "How many orders are completed vs pending?",
        "Which customer has spent the most money?",
        "What is the total revenue per product category?",
        "Show me customers who have placed more than one order",
    ]

    for ex in examples:
        if st.button(ex, use_container_width=True, key=ex):
            st.session_state.question = ex

Main input and results

question = st.text_input(
    "Ask anything about the database:",
    value=st.session_state.get("question", ""),
    placeholder="e.g. Show me top 3 customers by total spending",
)

if st.button("🔍 Run", type="primary") and question.strip():
    with st.spinner("Generating SQL and querying database..."):
        result = run_pipeline(question)

    if result["error"]:
        st.error(f"❌ {result['error']}")
        if result["sql"]:
            st.code(result["sql"], language="sql")
    else:
        tab1, tab2, tab3 = st.tabs(["📊 Results", "🔧 SQL Query", "💬 Explanation"])

        with tab1:
            df = pd.DataFrame(result["rows"], columns=result["columns"])
            st.success(f"✅ {len(result['rows'])} row(s) returned")
            st.dataframe(df, use_container_width=True)
            st.download_button("⬇️ Download CSV", df.to_csv(index=False),
                               "results.csv", "text/csv")

        with tab2:
            st.code(result["sql"], language="sql")

        with tab3:
            st.markdown(result["explanation"])

Run it

python sample_db.py     # create database (once)
python -m streamlit run app.py

Open http://localhost:8501 and try asking:

"Which customer has spent the most money?"
"What is the total revenue per product category?"
"Show me all Electronics products under $100"
"How many orders are completed vs pending?"

How the LLM Generates Correct SQL

The schema injection is doing most of the heavy lifting. When you ask "Which customer has spent the most money?", the LLM sees:

Schema:
CREATE TABLE customers (id INTEGER, name TEXT, ...)
CREATE TABLE orders (id INTEGER, customer_id INTEGER, ...)
CREATE TABLE order_items (id INTEGER, order_id INTEGER, unit_price REAL, quantity INTEGER, ...)

Question: Which customer has spent the most money?

And generates:

SELECT c.name, SUM(oi.unit_price * oi.quantity) AS total_spent
FROM customers c
JOIN orders o ON c.id = o.customer_id
JOIN order_items oi ON o.id = oi.order_id
GROUP BY c.id, c.name
ORDER BY total_spent DESC
LIMIT 1

That's a 3-table JOIN with aggregation — written correctly on the first attempt. This is why schema context matters so much. Without it, the LLM would guess at table names and column names and get them wrong.

Things I Learned Building This

1. Temperature 0 is critical for SQL

I initially used temperature=0.3 for SQL generation and got inconsistent results — sometimes valid SQL, sometimes slightly wrong syntax. Dropping to temperature=0 made the output perfectly consistent. SQL is deterministic by nature; the generation should be too.

2. "Return ONLY the SQL" needs to be explicit

Without that instruction, Llama 3.1 often writes something like:

"Here's the SQL query to answer your question:
SELECT * FROM customers
"

That entire string then gets passed to cursor.execute() and crashes. Being explicit in the prompt — "no explanation, no markdown, no backticks" — eliminates this.

3. Sample rows in schema context matter

SQLDatabase.get_table_info() includes sample rows from each table. This is crucial. If the LLM can see that the status column contains values like "completed", "shipped", "pending", it writes much better WHERE clauses than if it only sees status TEXT.

4. Schema injection scales linearly

The schema string grows with your number of tables. For large databases with 50+ tables, you'd need to be selective about which tables you include in the prompt — otherwise you hit context limits. Techniques like table selection (ask the LLM which tables are relevant first, then pass only those schemas) are used in production text-to-SQL systems.

5. The explanation step adds real value

I almost skipped the explanation feature — it felt unnecessary when the data is right there. But in testing, it genuinely helped. When someone asks "which customers haven't ordered recently?" and gets back a table with dates, the explanation highlights the most important finding rather than making them interpret the data themselves. It's the difference between a report and an analyst.

What I'd Add Next

Table selection for large databases — for databases with 20+ tables, first ask the LLM which tables are relevant to the question, then pass only those schemas. This reduces prompt size and improves accuracy.

Query caching — identical questions should return cached results rather than making a new API call every time.

Error correction loop — if the generated SQL throws an error, pass the error message back to the LLM and ask it to fix the query. Most SQL errors are self-correcting with one retry.

Chart generation — for aggregation queries, automatically detect when a bar chart or line chart would be more useful than a table and render it with Plotly.

Multi-database support — swap SQLite for PostgreSQL or MySQL by changing the connection string. The pipeline is database-agnostic.

Full Project Structure

text-to-sql/
├── app.py            # Streamlit frontend
├── sql_pipeline.py   # SQL generation + execution pipeline
├── sample_db.py      # Creates demo SQLite database
├── requirements.txt  # Dependencies
├── .env              # API keys (never commit this)
└── README.md         # Setup instructions

Full source code: github.com/naimulkarim/text-to-sql

Key Takeaways

Text-to-SQL is one of the most practically useful LLM applications because the output is verifiable. You can always check whether the generated SQL is correct by running it and inspecting the results. That makes it safer to deploy than open-ended generation tasks.

The three things that make it work well:

Schema context — the LLM must see your table structures and sample data
Temperature 0 — deterministic generation for consistent SQL
Explicit output format — tell the LLM exactly what to return and what not to include

If you build something with this, I'd love to see it in the comments. And if you connect it to a real database — PostgreSQL, MySQL, whatever — let me know how it performs on complex queries.

Drop a ❤️ if this was useful, and follow for more ML project walkthroughs.

I Built a PDF Q&A App with RAG, FAISS, and Llama 3.1 — Here's Everything I Learned

Naimul Karim — Mon, 27 Apr 2026 04:20:37 +0000

TL;DR: I built an end-to-end RAG application that lets you upload any PDF and chat with it. It uses FAISS for vector search, sentence-transformers for embeddings, and Llama 3.1 via Groq for free LLM inference. This article walks through the full architecture, every bug I hit, and how to build it yourself.

Why I Built This

I was working through a Machine Learning course final project and kept running into the same problem: I had dozens of research papers and lecture notes as PDFs, and finding specific information meant scrolling through hundreds of pages manually.

So I built a solution. An AI assistant that reads your PDFs, understands them semantically, and answers your questions in plain English — with explanations, not just copied text.

The result is PDF Q&A Pro: a multi-tab Streamlit app powered by a RAG pipeline.

Here's what it looks like in action:

Upload one or more PDFs
Ask any question in natural language
Get a detailed, explained answer with the exact source page cited
Generate summaries, key insights, and topic analyses with one click

And the best part — it runs entirely on free APIs.

What is RAG and Why Should You Care?

Before we dive into code, let me explain the core concept: Retrieval-Augmented Generation (RAG).

The naive approach to PDF Q&A is to paste the entire document into an LLM. That breaks immediately for anything longer than a few pages — LLMs have context limits, and sending 200 pages is expensive and slow.

RAG solves this elegantly:

Instead of: [Entire PDF] + Question → LLM → Answer

RAG does:   Question → Find relevant chunks → [Top 4 chunks] + Question → LLM → Answer

You only send the relevant parts of the document to the LLM. This makes it fast, cheap, and accurate.

The pipeline has two phases:

Indexing Phase (run once per upload):

PDF → Extract Text → Split into Chunks → Embed Chunks → Store in FAISS

Query Phase (run per question):

Question → Embed Question → FAISS Search → Top-K Chunks → LLM → Answer

The Tech Stack

Here's what I used and why:

Component	Choice	Why
PDF Loading	LangChain PyPDFLoader	Handles multi-page extraction with metadata
Text Splitting	RecursiveCharacterTextSplitter	Preserves sentence boundaries
Embeddings	all-MiniLM-L6-v2	90MB, runs on CPU, excellent quality
Vector Store	FAISS	In-memory, millisecond search, no server needed
LLM	Llama 3.1 8B via Groq	Free, fast (< 2s), genuinely good quality
Frontend	Streamlit	Fast to build, easy to deploy
Orchestration	LangChain LCEL	Clean pipeline composition

Building It — Step by Step

Step 1: Install Dependencies

pip install streamlit langchain langchain-community langchain-core \
  langchain-text-splitters faiss-cpu pypdf sentence-transformers \
  python-dotenv groq torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/cpu

Use the CPU-only PyTorch build — it's 10x smaller and perfectly sufficient since we're not training anything.

Step 2: Set Up Your Free Groq API Key

Go to console.groq.com
Sign up with Google — takes 30 seconds
Click API Keys → Create API Key
Create a .env file:

GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxxxxx

Groq's free tier gives you 14,400 requests/day. More than enough.

Step 3: The RAG Pipeline (`rag_pipeline.py`)

Let's build the core ML logic:

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.runnables import RunnableLambda
from groq import Groq
from dotenv import load_dotenv
import tempfile, os

load_dotenv()

EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
GROQ_MODEL      = "llama-3.1-8b-instant"
GROQ_API_KEY    = os.getenv("GROQ_API_KEY")

PDF Loading and Chunking:

def load_and_index(uploaded_files):
    all_chunks = []

    for uploaded_file in uploaded_files:
        # Save to temp file so PyPDFLoader can read it
        with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
            tmp.write(uploaded_file.read())
            tmp_path = tmp.name

        loader    = PyPDFLoader(tmp_path)
        documents = loader.load()
        os.unlink(tmp_path)  # clean up

        # Tag each page with its source filename
        for doc in documents:
            doc.metadata["source_file"] = uploaded_file.name

        # Split into overlapping chunks
        splitter = RecursiveCharacterTextSplitter(
            chunk_size=500,
            chunk_overlap=50,  # overlap prevents losing info at boundaries
        )
        chunks = splitter.split_documents(documents)
        all_chunks.extend(chunks)

    # Embed and index
    embeddings = HuggingFaceEmbeddings(
        model_name=EMBEDDING_MODEL,
        model_kwargs={"device": "cpu"},
    )
    vectorstore = FAISS.from_documents(all_chunks, embeddings)
    return vectorstore

Why chunk_overlap=50? Imagine a sentence that spans two chunks. Without overlap, the context at the boundary gets lost. With 50-token overlap, both chunks contain the boundary text — retrieval stays accurate.

The LLM Call:

def answer_question(question: str, context: str, history: list) -> str:
    client = Groq(api_key=GROQ_API_KEY)

    messages = [
        {
            "role": "system",
            "content": """You are an intelligent assistant and expert tutor.
Given relevant excerpts from a PDF document and a question:
1. Explain the concept thoroughly in simple, clear language
2. Use the document as your primary source but add helpful context
3. Structure answers with bullet points when helpful
4. Always cite which file and page the information came from
5. End with a key takeaway in one sentence"""
        }
    ]

    # Inject last 6 messages as conversational memory
    for h in history[-6:]:
        messages.append({"role": h["role"], "content": h["content"]})

    messages.append({
        "role": "user",
        "content": f"Context:\n{context}\n\nQuestion: {question}"
    })

    response = client.chat.completions.create(
        model=GROQ_MODEL,
        messages=messages,
        temperature=0.3,
        max_tokens=1024,
    )
    return response.choices[0].message.content.strip()

Building the Chain:

def build_qa_chain(vectorstore):
    retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

    def run_chain(inputs: dict):
        docs    = retriever.invoke(inputs["question"])
        context = "\n\n".join(
            f"[{doc.metadata.get('source_file')} | Page {doc.metadata.get('page', 0)+1}]\n{doc.page_content}"
            for doc in docs
        )
        answer = answer_question(
            inputs["question"],
            context,
            inputs.get("history", [])
        )
        return answer, docs

    return run_chain, retriever

Step 4: The Streamlit Frontend (`app.py`)

import streamlit as st
from rag_pipeline import load_and_index, build_qa_chain
from dotenv import load_dotenv

load_dotenv()

st.set_page_config(page_title="PDF Q&A Pro", page_icon="📄", layout="wide")
st.title("📄 PDF Q&A Pro")

# Sidebar: upload
with st.sidebar:
    uploaded_files = st.file_uploader(
        "Upload PDFs", type="pdf", accept_multiple_files=True
    )
    if uploaded_files:
        file_key = "_".join(sorted(f.name for f in uploaded_files))
        if st.session_state.get("file_key") != file_key:
            with st.spinner("Indexing PDFs..."):
                vs = load_and_index(uploaded_files)
                st.session_state.qa_fn, st.session_state.retriever = build_qa_chain(vs)
                st.session_state.file_key = file_key
                st.session_state.messages = []
            st.success("Ready!")

# Chat interface
for msg in st.session_state.get("messages", []):
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

if question := st.chat_input("Ask anything about your PDFs..."):
    st.session_state.messages.append({"role": "user", "content": question})
    with st.chat_message("user"):
        st.markdown(question)

    with st.chat_message("assistant"):
        with st.spinner("Thinking..."):
            answer, sources = st.session_state.qa_fn({
                "question": question,
                "history": st.session_state.messages[:-1],
            })
        st.markdown(answer)

        with st.expander("📚 Sources"):
            for i, doc in enumerate(sources, 1):
                st.markdown(
                    f"**Chunk {i}** — `{doc.metadata.get('source_file')}` "
                    f"| Page {doc.metadata.get('page', 0)+1}"
                )
                st.caption(doc.page_content[:300] + "...")

    st.session_state.messages.append({"role": "assistant", "content": answer})

Step 5: Run It

python -m streamlit run app.py

Open http://localhost:8501, upload a PDF, and ask your first question.

The Bugs I Hit (So You Don't Have To)

This is the part most tutorials skip. Here's every error I encountered and how I fixed it:

1. LangChain Import Errors

LangChain v0.2+ split into multiple packages. If you see ModuleNotFoundError, use these correct imports:

# OLD (broken in v0.2+)
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.prompts import PromptTemplate

# NEW (correct)
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.prompts import PromptTemplate

Lesson: Always check which sub-package a LangChain class lives in. The migration guide at python.langchain.com is your friend.

2. Hugging Face Free Tier Model Availability

Not all models work on the free inference API. I discovered this the hard way after getting 400 and 404 errors from flan-t5 and Mistral.

Test your models before building around them:

import requests

models = ["google/flan-t5-large", "facebook/bart-large-cnn", "deepset/roberta-base-squad2"]
token  = "hf_yourtoken"
headers = {"Authorization": f"Bearer {token}"}

for model in models:
    r = requests.post(
        f"https://router.huggingface.co/hf-inference/models/{model}",
        headers=headers,
        json={"inputs": "test"}
    )
    print(f"{model}: {r.status_code}")

Lesson: Test API endpoints programmatically. Don't assume availability from documentation.

3. Extractive vs Generative Models

I initially used deepset/roberta-base-squad2 thinking it would answer questions. It only copies exact spans of text from the document — it can't explain or elaborate.

Extractive (roberta-squad2):
Q: "What is machine learning?"
A: "a subset of artificial intelligence"  ← just copied from PDF

Generative (Llama 3.1):
Q: "What is machine learning?"
A: "Machine learning is a branch of AI that enables computers to learn 
   patterns from data without being explicitly programmed. According to 
   page 3 of your document..."  ← actual explanation

Lesson: For explanation-heavy use cases, you need a generative model. Extractive models are only useful for simple fact extraction.

4. The `InferenceClient` Version Conflict

The huggingface_hub library's InferenceClient had breaking changes across versions, causing:

'InferenceClient' object has no attribute 'post'

The fix: bypass the wrapper entirely and call the REST API directly.

import requests

def call_hf_api(prompt, token, model_url):
    response = requests.post(
        model_url,
        headers={"Authorization": f"Bearer {token}"},
        json={"inputs": prompt, "options": {"wait_for_model": True}},
        timeout=60
    )
    return response.json()[0]["generated_text"]

Lesson: When library wrappers cause version conflicts, the REST API is always a reliable fallback.

5. Prompt Engineering Makes or Breaks Everything

Early system prompt:

"Use only the context to answer. Do not make up information."

Result: Short, unhelpful answers that just copied document sentences.

Improved system prompt:

"You are an expert tutor. Explain thoroughly in simple language. 
Add examples. Structure with bullets. Cite sources. End with a key takeaway."

Result: Rich, explanatory answers that actually help the user understand.

Lesson: The system prompt is the most impactful variable in your entire pipeline. Spend real time on it.

Key Architecture Decisions

Why FAISS over Chroma/Pinecone?
FAISS runs entirely in-memory with no external server. For a student project or MVP, the simplicity is unbeatable. Chroma is great when you need persistence; Pinecone when you need scale.

Why chunk_size=500?
Too small: chunks lose context. Too large: you hit the LLM's context window and retrieval becomes less precise. 500 tokens with 50-token overlap is a well-tested sweet spot for most documents.

Why k=4 chunks?
Sending 4 × 500 = ~2000 tokens of context to the LLM gives enough information without overwhelming it or blowing the budget.

Why Groq over OpenAI?
Groq is genuinely free (not just a trial), returns responses in under 2 seconds thanks to custom LPU hardware, and Llama 3.1 8B is good enough for document Q&A. For a course project, there's no reason to pay.

What I'd Add Next

If I were to extend this project:

Persistent FAISS index — Save the index to disk so re-uploading the same document doesn't re-embed everything
Streaming responses — Stream LLM tokens to the UI for a ChatGPT-like feel
Hybrid search — Combine FAISS semantic search with BM25 keyword search for better recall
Document comparison — "How does Document A's approach differ from Document B's?"
Export to Anki — Auto-generate flashcards from document content

The Full Feature List

The complete app (link at the bottom) includes:

✅ Multi-PDF chat — cross-document search with source attribution
✅ Per-document chat — isolated history per file
✅ Conversational memory — last 6 turns injected as context
✅ Document dashboard — type, complexity, tone, top topics
✅ Auto-summary and key insights generation
✅ Topic analysis — most discussed themes with frequency
✅ Source highlighting — file name + page number on every answer
✅ Download chat history, summaries, and insights
✅ Token and cost tracking per query

What This Project Taught Me

Building this was genuinely humbling. The ML concepts — RAG, embeddings, vector search — took an afternoon to understand. The debugging took two days.

And that's actually the point. The error messages were where the real learning happened:

A 404 from a HuggingFace endpoint taught me to always test APIs before building around them
An import error taught me that library versioning is a real engineering concern, not just housekeeping
A bad answer from an extractive model taught me the difference between finding text and understanding text
A flat, unhelpful LLM response taught me that prompt engineering is a skill worth investing in

If you're learning ML, my advice is simple: pick a problem, build something end-to-end, and let the bugs teach you.

Resources

-GitHub Repository: github.com/naimulkarim/pdf-qa-app
-RAG Paper (Lewis et al., 2020): arxiv.org/abs/2005.11401
-Embedding Model: huggingface.co/sentence-transformers/all-MiniLM-L6-v2
-Groq (Free LLM API): console.groq.com

FAISS: github.com/facebookresearch/faiss
LangChain Docs: python.langchain.com

If this helped you, drop a ❤️ and share it with someone learning ML. And if you build something on top of this — I'd love to see it in the comments.

Basic Features of Calude AI

Naimul Karim — Sun, 19 Apr 2026 19:15:20 +0000

As of 2026, Claude AI has evolved from a simple chatbot into a highly capable "agentic" assistant. Through features like Claude Code, Claude Cowork, and the Model Context Protocol (MCP), its day-to-day use cases have expanded significantly beyond just writing and answering questions.

1. Conversation Import

Import chats or data from external sources (e.g., ChatGPT, documents)

Examples:

Import a ChatGPT conversation and summarize key points

2. Structured Prompting

Guided Chat / Structured Prompting (AIM) in Claude AI is a simple way to interact with AI using a clear structure instead of random prompts. It helps users interact more effectively with the AI.

Action → What you want the AI to do
Intent → Why you need it (context or goal)
Method → How the output should be structured or delivered

Examples:

Action: Plan a weekend family trip. Intent: Relaxing, budget-friendly 2-day getaway. Method: Provide a structured itinerary with activities, food options, time schedule, and estimated costs.

3. Skills

Instead of asking random questions, you shape Claude into a role like:
tutor, planner, coach, assistant, or writer

Examples:

Claude as Fitness Coach

Skill customization:
“Act as a beginner fitness coach. Create safe, simple home workout plans with no equipment and focus on consistency.”

Daily workout routines
Weight loss or general fitness plans
Adjusting intensity over time

4. Connectors

Integrate Claude to external tools, APIs, or data sources.

Examples :

Connect to Gmail to summarize inbox, draft replies or send emails
Connect to Google Drive to read and summarize documents
Pull data from spreadsheets and generate insights or summaries
Integrate with calendars to help plan schedules

5. Projects

Projects let you create separate workspaces for different tasks, each with its own chats and information. Inside a project, you can upload files, add context, and have focused conversations with Claude AI.

Examples :

Organize tasks, files, and conversations into projects
Personal Finance Workspace

Budget planning notes
Expense summaries
Investment research
Tax-related questions

6. Artifacts (Generated Outputs)

Artifacts turn a conversation into a finished output you can directly use, not just an explanation. It can generate structured outputs like code, documents, UI components.

Examples :

Build a simple website or landing page (HTML/CSS or React structure)
Generate a project report or business document

7. Collaboration Mode (Co-work)

Claude Cowork is a desktop tool. It connects Claude directly to your computer, files, apps, workflows. It can handle files, folders, and desktop automation tasks.

Examples :

Working together on documents, plans, and structured outputs
Managing files, folders, and project-related content

LLM vs RAG

Naimul Karim — Thu, 16 Apr 2026 02:38:57 +0000

LLM (Large Language Model)

An LLM like GPT-4 or Claude is:

A pretrained model on massive text data
Generates answers based on what it has learned during training
Doesn’t know your private or real-time data unless provided in the prompt

Limitation:

Can hallucinate
Knowledge is static (cutoff-based)

RAG (Retrieval-Augmented Generation)

RAG is a system design pattern, not a model.

It works like this:

User asks a question
System retrieves relevant data (docs, DB, APIs, vector search)
That data is injected into the prompt
LLM generates an answer using that context

LLM can be seen as a generator
RAG is a combination of retriever and LLM

Core Differences

Aspect	LLM	RAG
Type	Model	Architecture / Pattern
Knowledge Source	Training data	External + Real-time data
Accuracy	Can hallucinate	More grounded
Updates	Requires retraining	Just update data source
Use Case	General tasks	Domain-specific, factual Q&A

Without RAG:

User: “What’s the latest interest rate?”
LLM: Might guess or give outdated info

With RAG:

System fetches latest rates from DB/API
LLM answers using that data
Accurate and up-to-date

Usage

Use LLM alone when:

Creative writing
General coding help
Brainstorming

Use RAG when:

You need company data / internal docs
Accuracy matters (finance, legal, healthcare)
Data changes frequently

AI for Developers

Naimul Karim — Wed, 15 Apr 2026 04:47:10 +0000

AI has become a core part of software development, reshaping how developers write code, build systems, and deliver applications through increasingly advanced tools and automation.

Agentic Workflows and Orchestration

n8n helps automate workflows by connecting APIs, services, and AI models into end-to-end processes.

Chat and Assistant Tools

Claude helps with reasoning, long-context understanding, and code-related discussions.
ChatGPT supports developers with coding, debugging, and general problem-solving.
Gemini provides AI assistance across research and multimodal tasks within Google’s ecosystem.

IDE

GitHub Copilot offers real-time code suggestions inside the editor.
Cursor is an AI-native IDE that enables coding through natural language.

Coding Agents

Cursor Agents execute multi-step development tasks autonomously.
GitHub Copilot Agent performs coding tasks beyond simple suggestions.
Codex powers AI-based code generation models.
Claude Code supports advanced coding and problem-solving tasks.
OpenCode runs AI coding agents directly in the terminal.

Full-App Builders

Replit lets users build and deploy apps quickly in the cloud with AI help.
V0 generates frontend components from text prompts.

AI Developer Libraries

LangChain helps build applications powered by large language models.
LangGraph structures AI workflows using graph-based design.
LlamaIndex connects AI models to external data sources.
Haystack builds search and QA systems using LLMs.

Documentation & Knowledge Tools

Problem solving with ML : Domains That Actually Matter

Naimul Karim — Tue, 06 Jan 2026 14:41:58 +0000

When people start learning machine learning, they often focus on different things. But in practice, the most important question is much simpler:

What kind of work do you want to do?

What problems do you want to solve using ML?

When you look at machine learning from this perspective, the entire field can be cleanly divided into five main work domains.

1. Predictive Modeling

This domain works primarily with structured, tabular, or numerical data. The objective is to predict an outcome based on historical patterns.

Examples:

Helping businesses detect and prevent fraud and avoid financial losses
Analyzing effectiveness of past promotional activity
Forecasting patient admissions and readmissions
Predicting energy consumption
Regression, classification, and time-series forecasting all belong here. This is the oldest and most battle-tested area of machine learning, and where most practitioners build their foundation.

2. Perception AI (Vision / Audio)

In this domain, machines learn to interpret visual and auditory signals from the real world.

Examples:

Counting people in public spaces using camera feeds
Detecting cracks or defects in infrastructure from drone images
Recognizing spoken commands in voice-controlled systems
Identifying equipment malfunctions from vibration or sound patterns
Identifying pedestrians and traffic signs in a self-driven car
Models such as CNNs and Vision Transformers (ViT) are heavily used. Perception AI enables machines to convert raw sensory data into meaningful signals.

3. Language Intelligence (NLP)

This domain focuses on understanding and processing written or spoken language.

Examples:

Extracting key clauses from contracts and legal documents
Automatically generating meeting notes from transcripts
Detecting abusive content in online communities
Grouping news articles by topic at scale
Speech recognition, Spell check, autocomplete etc
Transformer-based models like BERT and GPT-style architectures power this domain. The aim is to help machines understand nuance, context, and intent in language.

4. Generative AI (Multimodal)

This domain is about creation, not just prediction. Models here can generate entirely new outputs across multiple data types.

Examples:

Creating a short story based on the style of a particular author,
Generate a realistic image of a person who doesn't exist
Composing a symphony in the style of a famous composer
Generating test cases or documentation from source code
Large Language Models (LLMs), diffusion models, and multimodal systems define this space, where language, vision, and reasoning intersect.

5. Decision & Control Systems (Reinforcement Learning)

What kind of work this is: Learning what action to take in an environment to maximize long-term reward.

Examples:

Dynamic pricing and bidding systems
Recommendation systems with long-term user engagement
Robotics control and motion planning
Resource allocation and scheduling

Agentic AI: The Extension

Agentic AI is an extension, not a standalone ML domain. It builds on Generative AI to enable systems that can reason, plan, and act.

Examples:

An AI assistant that schedules meetings by checking calendars and availability
A system that monitors metrics, detects anomalies, and triggers alerts
An AI agent that runs experiments, compares results, and selects the best approach
Multiple agents collaborating to complete a complex workflow
In short, Agentic AI represents AI systems that don’t just respond but take action.

Unit testing : How to Mock Public Methods in C# with NSubstitute

Naimul Karim — Tue, 06 Jan 2026 02:30:31 +0000

With NSubstitute, a public method can only be mocked if it is virtual (or abstract).
If the method is public but not virtual, NSubstitute cannot mock it.

The options to enables to mock it are :

1. Make the method virtual

This is the Minimal change and fully supported by NSubstitute

public class ProductService
{
    public virtual int GetPrice()
    {
        return 1;
    }
}

var service = Substitute.For<MyService>();
service.GetValue().Returns(42);

2. Extract an interface (cleanest design)

Refactor for Dependency Injection. Refactor your code so that the logic in the method is moved to a dependency (e.g., a helper/service class) that can be mocked. This is the best option.

public interface IProductService
{
    int GetPrice();
}

Inject the interface and mock it instead.

var service = Substitute.For<IProductService>();
service.GetPrice().Returns(42);

3. Use a wrapper / adapter (for third-party or legacy code)

public interface IProductServiceWrapper
{
    int GetPrice();
}

public class ProductServiceWrapper : IProductServiceWrapper
{
    private readonly ProductService _service;
    public int GetPrice() => _service.GetValue();
}

Mock ProductServiceWrapper

Forem: Naimul Karim

Building a Multi-Channel Notification Service in .NET 8

Why Build a Notification Service?

Architecture Overview

Data flow in plain English

Project Structure

Part 1 — Core Domain Models

Part 2 — Message Queue with RabbitMQ

Publisher

Consumer (BackgroundService)

Queue topology

Part 3 — Template Service (Scriban)

Seeded templates (ready out of the box)

Part 4 — Rate Limiter (Sliding Window)

Part 5 — Dispatcher (Orchestration)

Part 6 — Channel Workers

Email — MailKit

SMS — Twilio

Push — Firebase FCM

Part 7 — API Layer

Example request

Response

Part 8 — DI Wiring (Program.cs)

Part 9 — Unit Tests

Part 10 — Docker Compose

Part 11 — GitHub Actions CI

Extending the Service

Add a new channel (e.g. Slack) — 3 steps

Distributed rate limiting (Redis)

Key Design Decisions

What's Next

Source Code

🔗 github.com/naimulkarim/NotificationService

Building a Cloud-Native Event Ticketing System with DDD, Event-Driven Architecture & CQRS in .NET 8

Table of Contents

1. The Problem Domain & Why These Patterns

2. Architecture Overview

3. Solution Structure & Full File Map

4. The Shared Kernel — DDD Building Blocks

AggregateRoot

DomainEvent, ValueObject, Entity

Result Pattern

MediatR Pipeline Behaviours

5. Bounded Context 1: Identity Service

OAuth2 Client Configuration

Program.cs

6. Bounded Context 2: Event Catalog Service

The Event Aggregate

CQRS — Commands and Queries

REST Controller

7. Bounded Context 3: Booking Service — DDD + CQRS + Saga

The Booking Aggregate

Domain Events vs Integration Events

CreateBooking Command + Handler

Queries — Read Side

Repository and Unit of Work

DbContext with Outbox Support

Full DI Wiring in Program.cs

8. Bounded Context 4: Payment Service

The Payment Aggregate

RabbitMQ Consumer

9. Bounded Context 5: Notification Service

10. The API Gateway — YARP

11. Event-Driven Architecture Deep Dive

Why Two Kinds of Events?

Event Flow Diagram

MassTransit Consumer Registration

12. The Booking Saga — Distributed Workflow

State Machine Diagram

13. The Outbox Pattern — Guaranteed Delivery

14. Redis Seat Locking — Preventing Overselling

15. The Angular Frontend — NgRx Architecture

Application Bootstrap

Lazy-Loaded Feature Routes

NgRx Booking Store

Effects — HTTP Side Effects

Auth Guard & Interceptor

Event List Component — Search & Browse

Admin Event Management Component

16. All Features & Use Cases — End to End

Load secrets into `IConfiguration` at startup

Create your `.env` file