Forem: Elsayed Kamal

Mastering Laravel Queues: A Complete Guide to Asynchronous Task Processing

Elsayed Kamal — Fri, 13 Jun 2025 13:31:03 +0000

Mastering Laravel Queues: A Complete Guide to Asynchronous Task Processing

Laravel's queue system is one of its most powerful features for building scalable web applications. By handling time-consuming tasks asynchronously, queues dramatically improve your application's response time and user experience. Let's dive deep into everything you need to know about Laravel Queues.

What are Laravel Queues?

Laravel Queues allow you to defer time-consuming tasks (like sending emails, processing images, generating reports) to be processed in the background. Instead of making users wait for these operations to complete, your application can respond immediately while the heavy lifting happens behind the scenes.

Key Concepts

1. Queue Drivers

Laravel supports multiple queue drivers to fit different infrastructure needs:

Database - Stores jobs in your database (great for getting started)
Redis - Fast, in-memory data structure store (recommended for production)
Amazon SQS - AWS Simple Queue Service (perfect for cloud deployments)
Beanstalkd - Simple, fast work queue
Sync - Processes jobs immediately (useful for testing)

2. Jobs

Jobs are classes that contain the logic for tasks you want to queue. Here's a typical job structure:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Mail;

class SendEmailJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $user;

    public function __construct($user)
    {
        $this->user = $user;
    }

    public function handle()
    {
        // Send email logic here
        Mail::to($this->user->email)->send(new WelcomeEmail($this->user));
    }
}

3. Dispatching Jobs

You can dispatch jobs in several flexible ways:

// Basic dispatch
SendEmailJob::dispatch($user);

// Dispatch with delay
SendEmailJob::dispatch($user)->delay(now()->addMinutes(10));

// Dispatch to specific queue
SendEmailJob::dispatch($user)->onQueue('emails');

// Dispatch with custom connection
SendEmailJob::dispatch($user)->onConnection('redis');

Configuration

Queue Configuration (`config/queue.php`)

'default' => env('QUEUE_CONNECTION', 'sync'),

'connections' => [
    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
    ],

    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
    ],
],

Environment Variables

QUEUE_CONNECTION=redis
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

Queue Workers

Running Workers

Queue workers are the processes that actually execute your queued jobs:

# Start a queue worker
php artisan queue:work

# Work specific queue
php artisan queue:work --queue=high,default

# Work with timeout
php artisan queue:work --timeout=60

# Work with memory limit
php artisan queue:work --memory=512

Supervisor Configuration

For production environments, use Supervisor to manage queue workers and ensure they restart automatically:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/var/log/laravel-worker.log
stopwaitsecs=3600

Advanced Features

1. Job Batching

Process multiple jobs as a group and handle their collective completion:

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessPodcast($podcast),
    new ProcessPodcast($podcast),
    new ProcessPodcast($podcast),
])->then(function (Batch $batch) {
    // All jobs completed successfully
})->catch(function (Batch $batch, Throwable $e) {
    // First batch job failure detected
})->finally(function (Batch $batch) {
    // The batch has finished executing
})->dispatch();

2. Job Chaining

Execute jobs in sequence, where each job runs only after the previous one completes:

ProcessPodcast::withChain([
    new OptimizePodcast,
    new ReleasePodcast
])->dispatch();

3. Failed Jobs

Laravel automatically handles failed jobs with built-in retry mechanisms:

# View failed jobs
php artisan queue:failed

# Retry failed job
php artisan queue:retry 5

# Retry all failed jobs
php artisan queue:retry all

# Flush all failed jobs
php artisan queue:flush

4. Job Events

Monitor and react to job lifecycle events:

use Illuminate\Queue\Events\JobProcessed;
use Illuminate\Queue\Events\JobFailed;

// In AppServiceProvider
Queue::after(function (JobProcessed $event) {
    // Job processed successfully
});

Queue::failing(function (JobFailed $event) {
    // Job failed
});

Best Practices

1. Job Design

Keep jobs small and focused - Each job should have a single responsibility
Make jobs idempotent - Safe to run multiple times without side effects
Handle exceptions gracefully - Always include proper error handling
Use proper serialization - Be careful with model serialization

2. Performance Optimization

Configure timeouts, retries, and backoff strategies:

class ProcessPodcast implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $timeout = 120;
    public $tries = 3;
    public $maxExceptions = 2;

    public function retryUntil()
    {
        return now()->addMinutes(10);
    }

    public function backoff()
    {
        return [1, 5, 10]; // Exponential backoff
    }
}

3. Queue Prioritization

Use multiple queues to prioritize different types of jobs:

// High priority queue
ProcessPayment::dispatch($payment)->onQueue('high');

// Low priority queue
SendNewsletter::dispatch($newsletter)->onQueue('low');

4. Monitoring

Use Laravel Horizon for Redis queues to get beautiful dashboards and monitoring:

composer require laravel/horizon
php artisan horizon:install
php artisan horizon

Common Use Cases

Laravel Queues are perfect for:

Email Sending - Newsletter campaigns, notifications
Image/Video Processing - Resizing, compression, format conversion
Report Generation - Large CSV exports, analytics reports
API Integrations - Third-party service calls
Database Cleanup - Bulk operations, data archiving
File Uploads - Processing uploaded files
Notification Processing - Push notifications, SMS

Database Setup

If you're using the database driver, don't forget to run the migrations:

php artisan queue:table
php artisan queue:failed-table
php artisan migrate

Conclusion

Laravel Queues are essential for building scalable, responsive applications. They provide a clean, powerful way to handle background processing while maintaining excellent user experience. Whether you're sending emails, processing images, or integrating with external APIs, queues will help you build more robust and performant Laravel applications.

Start with the database driver for development, then move to Redis for production. Remember to monitor your queues, handle failures gracefully, and always test your job logic thoroughly.

Happy queueing! 🚀

Laravel 12.17.0: What's New in 2025 and Why You Should Care

Elsayed Kamal — Mon, 09 Jun 2025 20:04:28 +0000

Laravel 12.17.0: What's New in 2025 and Why You Should Care

Laravel continues to dominate the PHP ecosystem with its elegant syntax, powerful features, and thriving community. With the recent release of Laravel 12.17.0 on June 3, 2025, and the exciting ecosystem developments, there's never been a better time to be a Laravel developer.

🚀 What's New in Laravel 12.17.0

AsUri Model Cast

One of the standout features in Laravel 12.17.0 is the new AsUri model cast. This makes handling URIs in your Eloquent models incredibly straightforward:

use Illuminate\Database\Eloquent\Casts\AsUri;

class Website extends Model
{
    protected $casts = [
        'homepage' => AsUri::class,
    ];
}

// Usage
$website = Website::create(['homepage' => 'https://example.com']);
$uri = $website->homepage; // Returns a proper URI object

Contextual Binding with PHP 8 Attributes

Laravel 12.17.0 introduces contextual implementation and interface binding using PHP 8 attributes, making dependency injection even more powerful:

use Illuminate\Contracts\Container\ContextualAttribute;

#[ContextualAttribute]
class PaymentProcessor
{
    public function __construct(
        #[For(StripeService::class)] PaymentGateway $gateway
    ) {
        // Automatically injects StripeService when needed
    }
}

Enhanced Query Builder

The new reorderDesc() method makes ordering queries more intuitive:

// Before
$users = User::orderBy('created_at', 'desc')->get();

// Now - more expressive
$users = User::reorderDesc('created_at')->get();

Cursor Paginator Improvements

Resource helpers have been added to cursor paginators, making API responses cleaner:

return UserResource::collection(
    User::cursorPaginate(15)
)->additional([
    'meta' => ['total_pages' => $users->hasPages()]
]);

🌟 Laravel Ecosystem Highlights

Laravel Cloud is Live!

The most exciting news is that Laravel Cloud is now officially available! This game-changing platform allows you to:

Deploy Laravel applications in under 1 minute
Enjoy SOC 2 Type 1 compliance out of the box
Scale automatically based on traffic
Integrate seamlessly with the Laravel ecosystem

Laravel Nova 5.0

Laravel Nova received a major update with version 5.0, bringing:

Enhanced performance
Better user experience
New field types and filters
Improved customization options

Inertia 2.0: The Frontend Revolution

Inertia.js 2.0 has been announced, completely redefining how we build SPAs with Laravel:

// Cleaner syntax and better TypeScript support
import { router } from '@inertiajs/react'

router.visit('/users', {
  data: { search: query },
  preserveState: true,
  only: ['users']
})

💰 The $57M Investment Impact

Laravel recently secured a massive $57M investment from Accel, which means:

Faster development of new features
Better long-term stability for the framework
Enhanced ecosystem tools and services
Stronger community support

🔧 Developer Experience Improvements

Laravel Herd 1.11

The latest version of Laravel Herd brings:

Faster local development setup
Better PHP version management
Enhanced debugging tools
Seamless integration with Laravel Cloud

PHP 8.4 Support

Laravel Vapor now fully supports PHP 8.4, giving you access to:

Performance improvements
New language features
Better type system enhancements

🎯 Why Choose Laravel in 2025?

1. Mature Ecosystem

Laravel isn't just a framework; it's a complete ecosystem:

Eloquent ORM for database interactions
Blade templating for beautiful UIs
Artisan CLI for productivity
Laravel Mix for asset compilation
Laravel Sanctum for API authentication

2. Enterprise Ready

With Laravel Cloud achieving SOC 2 compliance and major enterprise adoptions, Laravel is proven at scale.

3. Active Community

Over 70k GitHub stars
Thriving Discord and forums
Regular conferences (Laracon)
Extensive package ecosystem

4. Learning Resources

Laravel Bootcamp for beginners
Laracasts for video tutorials
Laravel Daily for tips and tricks
Official documentation that's second to none

🚀 Getting Started with Laravel 12

# Install Laravel via Composer
composer create-project laravel/laravel my-app

# Or use the Laravel installer
composer global require laravel/installer
laravel new my-app

# Start developing
cd my-app
php artisan serve

🔮 What's Next?

The Laravel team continues to innovate:

Laravel Octane improvements for better performance
Laravel Livewire 3.x enhancements
More AI integrations and tooling
Enhanced testing capabilities

Conclusion

Laravel 12.17.0 represents another step forward in Laravel's evolution. With new features like AsUri casting, contextual binding, and the revolutionary Laravel Cloud platform, there's never been a better time to start or continue your Laravel journey.

The recent $57M investment ensures Laravel's future, while the active community and comprehensive ecosystem make it the perfect choice for developers and businesses alike.

Whether you're building your first web application or scaling an enterprise solution, Laravel provides the tools, community, and stability you need to succeed.

Ready to dive into Laravel? Check out the official documentation and join the community on Discord.

What features are you most excited about in Laravel 12? Share your thoughts in the comments below!

Mastering Laravel Queues: A Complete Guide to Background Job Processing

Elsayed Kamal — Sun, 08 Jun 2025 17:14:18 +0000

Mastering Laravel Queues: A Complete Guide to Background Job Processing

Laravel Queues are one of the most powerful features of the Laravel framework, allowing you to defer time-consuming tasks and process them in the background. This comprehensive guide will walk you through everything you need to know about Laravel Queues, from basic setup to advanced patterns.

What are Laravel Queues?

Laravel Queues provide a unified API for handling background job processing. Instead of making users wait for expensive operations like sending emails, processing images, or generating reports, you can push these tasks to a queue and process them asynchronously.

Key Benefits

Improved User Experience: Users don't have to wait for slow operations
Better Performance: Your application remains responsive
Scalability: Process jobs across multiple workers and servers
Reliability: Built-in retry mechanisms and failure handling

Setting Up Laravel Queues

1. Configuration

First, configure your queue driver in .env:

QUEUE_CONNECTION=database
# Or use Redis for better performance
# QUEUE_CONNECTION=redis

2. Database Setup

If using the database driver, create the jobs table:

php artisan queue:table
php artisan migrate

3. Creating a Job

Generate a new job class:

php artisan make:job ProcessUserRegistration

This creates a job class in app/Jobs/ProcessUserRegistration.php:

<?php

namespace App\Jobs;

use App\Models\User;
use App\Mail\WelcomeEmail;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Support\Facades\Mail;

class ProcessUserRegistration implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $user;

    public function __construct(User $user)
    {
        $this->user = $user;
    }

    public function handle()
    {
        // Send welcome email
        Mail::to($this->user->email)->send(new WelcomeEmail($this->user));

        // Create user profile
        $this->user->profile()->create([
            'bio' => 'Welcome to our platform!',
            'avatar' => 'default-avatar.png'
        ]);

        // Log the registration
        logger("User {$this->user->id} registration processed");
    }
}

Dispatching Jobs

Basic Dispatching

use App\Jobs\ProcessUserRegistration;

// In your controller
public function register(Request $request)
{
    $user = User::create($request->validated());

    // Dispatch the job
    ProcessUserRegistration::dispatch($user);

    return response()->json(['message' => 'Registration successful']);
}

Delayed Jobs

// Dispatch after 5 minutes
ProcessUserRegistration::dispatch($user)->delay(now()->addMinutes(5));

// Dispatch at specific time
ProcessUserRegistration::dispatch($user)->delay(now()->addHour());

Job Chaining

use App\Jobs\ProcessUserRegistration;
use App\Jobs\SendWelcomeEmail;
use App\Jobs\CreateUserProfile;

// Chain jobs to run in sequence
ProcessUserRegistration::withChain([
    new SendWelcomeEmail($user),
    new CreateUserProfile($user),
])->dispatch($user);

Advanced Queue Features

Job Batching

Laravel 8+ supports job batching for processing related jobs together:

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessOrder($order1),
    new ProcessOrder($order2),
    new ProcessOrder($order3),
])->then(function (Batch $batch) {
    // All jobs completed successfully
    logger('All orders processed successfully');
})->catch(function (Batch $batch, Throwable $e) {
    // First batch job failure detected
    logger('Order processing failed: ' . $e->getMessage());
})->finally(function (Batch $batch) {
    // The batch has finished executing
    logger('Order batch processing completed');
})->dispatch();

Rate Limiting

Control how many jobs are processed per minute:

use Illuminate\Queue\Middleware\RateLimited;

class ProcessApiRequest implements ShouldQueue
{
    public function middleware()
    {
        return [new RateLimited('api-requests')];
    }
}

Define the rate limit in your AppServiceProvider:

use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;

public function boot()
{
    RateLimiter::for('api-requests', function ($job) {
        return Limit::perMinute(100);
    });
}

Unique Jobs

Prevent duplicate jobs from being queued:

use Illuminate\Contracts\Queue\ShouldBeUnique;

class ProcessPayment implements ShouldQueue, ShouldBeUnique
{
    public $uniqueFor = 3600; // 1 hour

    public function uniqueId()
    {
        return $this->order->id;
    }
}

Error Handling and Retries

Automatic Retries

Configure retry attempts and backoff:

class ProcessUserRegistration implements ShouldQueue
{
    public $tries = 3;
    public $maxExceptions = 2;
    public $backoff = [1, 5, 10]; // Wait 1, 5, then 10 seconds

    public function retryUntil()
    {
        return now()->addMinutes(10);
    }
}

Failed Job Handling

public function failed(Throwable $exception)
{
    // Log the failure
    logger("Job failed for user {$this->user->id}: " . $exception->getMessage());

    // Notify administrators
    Mail::to('admin@example.com')->send(new JobFailedNotification($this->user, $exception));
}

Running Queue Workers

Basic Worker

php artisan queue:work

Worker with Options

# Process specific queue
php artisan queue:work --queue=high,default

# Set memory limit
php artisan queue:work --memory=512

# Set timeout
php artisan queue:work --timeout=60

# Auto-restart when code changes
php artisan queue:work --daemon

Supervisor Configuration

For production, use Supervisor to manage queue workers:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/path/to/your/app/worker.log
stopwaitsecs=3600

Monitoring and Debugging

Queue Monitoring

Use Laravel Horizon for Redis queues:

composer require laravel/horizon
php artisan horizon:install
php artisan horizon

Debugging Failed Jobs

# List failed jobs
php artisan queue:failed

# Retry specific failed job
php artisan queue:retry 1

# Retry all failed jobs
php artisan queue:retry all

# Clear failed jobs
php artisan queue:flush

Best Practices

1. Keep Jobs Small and Focused

// Good: Single responsibility
class SendWelcomeEmail implements ShouldQueue
{
    public function handle()
    {
        Mail::to($this->user->email)->send(new WelcomeEmail($this->user));
    }
}

// Better: Break into smaller jobs
class ProcessUserRegistration implements ShouldQueue
{
    public function handle()
    {
        SendWelcomeEmail::dispatch($this->user);
        CreateUserProfile::dispatch($this->user);
        UpdateUserStats::dispatch($this->user);
    }
}

2. Use Job Middleware

class ProcessSensitiveData implements ShouldQueue
{
    public function middleware()
    {
        return [
            new WithoutOverlapping($this->user->id),
            new RateLimited('sensitive-data'),
            (new ThrottlesExceptions(3, 5))->backoff(5),
        ];
    }
}

3. Handle Job Dependencies

class ProcessOrder implements ShouldQueue
{
    public function handle()
    {
        // Check if dependencies still exist
        if (!$this->order->exists) {
            $this->delete();
            return;
        }

        // Process the order
    }
}

Testing Queue Jobs

Testing Job Dispatch

use Illuminate\Support\Facades\Queue;

public function test_user_registration_dispatches_job()
{
    Queue::fake();

    $user = User::factory()->create();

    // Trigger the action that dispatches the job
    $this->post('/register', $userData);

    // Assert the job was dispatched
    Queue::assertPushed(ProcessUserRegistration::class, function ($job) use ($user) {
        return $job->user->id === $user->id;
    });
}

Testing Job Execution

public function test_process_user_registration_job()
{
    $user = User::factory()->create();

    $job = new ProcessUserRegistration($user);
    $job->handle();

    // Assert job side effects
    $this->assertTrue($user->profile()->exists());
    Mail::assertSent(WelcomeEmail::class);
}

Conclusion

Laravel Queues are essential for building scalable web applications. They help you:

Improve user experience by deferring time-consuming tasks
Scale your application horizontally with multiple workers
Handle failures gracefully with retry mechanisms
Monitor and debug background job processing

Start with simple jobs and gradually adopt advanced features like batching, rate limiting, and unique jobs as your application grows. Remember to monitor your queues in production and have proper error handling in place.

Have you implemented Laravel Queues in your projects? Share your experiences and tips in the comments below!

Understanding JavaScript: The Language of the Web

Elsayed Kamal — Tue, 03 Jun 2025 19:25:29 +0000

Introduction

JavaScript is a versatile and powerful programming language that is essential for web development. It allows developers to create dynamic and interactive web applications. In this article, we'll explore the fundamentals of JavaScript, its features, and how to get started.

What is JavaScript?

JavaScript is a high-level, interpreted programming language that was initially developed to create interactive web pages. It has since evolved into a multi-paradigm language, supporting object-oriented, imperative, and functional programming styles.

Key Features

Dynamic Typing: Variables in JavaScript can hold values of any data type, and the type can change at runtime.
First-Class Functions: Functions in JavaScript are first-class objects, meaning they can be assigned to variables, passed as arguments, and returned from other functions.
Event-Driven: JavaScript is often used to handle events, such as user interactions, which makes it ideal for dynamic applications.

Getting Started

To start using JavaScript, you can include it in your HTML files using the <script> tag:

<script>
    console.log('Hello, World!');
</script>

You can also write JavaScript in external files with a .js extension and link to them in your HTML:

<script src="script.js"></script>

Conclusion

JavaScript is an essential language for web developers, enabling them to create rich, interactive experiences on the web. Whether you are a beginner or an experienced developer, mastering JavaScript will open doors to endless possibilities in web development.

Getting Started with Laravel: A Beginner's Guide

Elsayed Kamal — Tue, 03 Jun 2025 19:22:16 +0000

Introduction

Laravel is a powerful PHP framework designed for building web applications. In this article, we'll cover the basics of getting started with Laravel, including installation, routing, and views.

Installation

To start using Laravel, you first need to install Composer. Once Composer is set up, you can create a new Laravel project by running:

composer create-project --prefer-dist laravel/laravel my-project

Routing

Laravel makes it easy to define routes for your application. You can define a route in the routes/web.php file:

Route::get('/', function () {
    return view('welcome');
});

Views

Views in Laravel are stored in the resources/views directory. You can create a new view file and return it from your controller methods or routes.

Conclusion

Laravel is an excellent choice for building modern web applications. With its elegant syntax and robust features, you'll find it easy to develop maintainable code.
Your journey with Laravel starts here!

Getting Started with Ubuntu: A Beginner's Guide to Linux

Elsayed Kamal — Sun, 01 Jun 2025 09:06:09 +0000

Getting Started with Ubuntu: A Beginner's Guide to Linux

Introduction

Ubuntu is one of the most popular Linux distributions due to its ease of use and community support. This guide aims to help beginners download, install, and use Ubuntu effectively.

What is Ubuntu?

Ubuntu is a Linux-based operating system that offers a free and open-source alternative to Windows and macOS. Offering user-friendly interface and vast repository of apps, Ubuntu is perfect for those looking to explore Linux.

How to Install Ubuntu

Download the Ubuntu ISO: Visit the Ubuntu official site and download the latest version of Ubuntu.
Create a Bootable USB: Use tools like Rufus or Etcher to create a bootable USB drive with the downloaded ISO.
Boot from USB: Start your computer, access the boot menu, and select the bootable USB drive.
Install Ubuntu: Follow on-screen instructions to install Ubuntu.

Basic Commands to Know

ls: List files and directories.
cd: Change directories.
sudo apt update: Update the list of available packages.
sudo apt upgrade: Upgrade installed packages.

Getting Help and Community Support

Join forums, online communities, and explore the vast collection of documentation available on sites like Ask Ubuntu.

Conclusion

Ubuntu provides a smooth transition for beginners venturing into Linux territories, thanks to its friendly user interface and strong community support.

Explore Ubuntu and unlock the potential of Linux today!

Introduction to Apache Kafka: Essential Guide for Beginners

Elsayed Kamal — Sun, 01 Jun 2025 08:25:03 +0000

Introduction to Apache Kafka: Essential Guide for Beginners

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Why Use Kafka?

Kafka is chosen for its ability to handle real-time data feeds. Kafka is fast, scalable, durable, and fault-tolerant, making it a cornerstone in the modern data ecosystem.

Core Concepts

Producer: An application that sends messages to Kafka.
Consumer: An application that reads messages from Kafka.
Broker: A Kafka server that stores messages submitted by producers.
Topic: A category or feed name to which records are published.
Partition: A horizontal scaling mechanism that allows Kafka to split topics across multiple servers.

Getting Started with Kafka

Installation

Download the Kafka distribution from the Apache Kafka website.
Extract the files and navigate to the Kafka directory.
Start the ZooKeeper service and Kafka server using the appropriate scripts provided.

Basic Operations

Create a Topic: Learn how to create and manage topics.
Write a Producer: Develop a sample producer application that sends messages to Kafka.
Create a Consumer: Develop a simple consumer application that logs received messages.

Key Benefits

Real-time processing and analytics
High throughput and low latency
Durability and reliability
Scalability to meet growing data demands

Conclusion

Apache Kafka is an indispensable tool in the toolkit of any data engineer or developer working with modern, real-time data streams. By understanding its core concepts and getting hands-on with a simple Kafka setup, you can start leveraging Kafka's powerful capabilities for your own projects.

Laravel Queues Explained: Building Scalable Background Job Processing Systems

Elsayed Kamal — Sat, 31 May 2025 15:19:44 +0000

Laravel Queues Explained: Building Scalable Background Job Processing Systems

Laravel's queue system is a game-changer for building responsive web applications. Instead of forcing users to wait for time-consuming operations like sending emails, processing images, or generating reports, Laravel queues allow you to defer these tasks to background workers. This comprehensive guide will walk you through everything you need to know about implementing and optimizing Laravel queues.

Why Laravel Queues Matter

The Problem with Synchronous Processing

Imagine a user uploading a large image to your application. Without queues, the user would have to wait while your server:

Validates the image
Resizes it to multiple formats
Optimizes file size
Uploads to cloud storage
Updates the database
Sends notification emails

This could take 30+ seconds, creating a poor user experience.

The Queue Solution

With Laravel queues, you can:

Accept the upload instantly
Return immediate feedback to the user
Process everything in the background
Notify the user when complete

Setting Up Laravel Queues

1. Choose Your Queue Driver

Laravel supports multiple queue backends:

# Database (good for development/small apps)
QUEUE_CONNECTION=database

# Redis (recommended for production)
QUEUE_CONNECTION=redis

# Amazon SQS (for AWS environments)
QUEUE_CONNECTION=sqs

# Sync (for testing - processes immediately)
QUEUE_CONNECTION=sync

2. Database Queue Setup

For database queues, create the necessary tables:

php artisan queue:table
php artisan queue:failed-table
php artisan migrate

3. Redis Queue Setup

Install Redis and configure the connection:

# Install Redis
sudo apt-get install redis-server

# Install PHP Redis extension
composer require predis/predis

Update your .env:

REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

Creating and Dispatching Jobs

Basic Job Creation

Generate a new job class:

php artisan make:job ProcessImageUpload

This creates a job class with the following structure:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Models\Image;
use Intervention\Image\Facades\Image as ImageProcessor;

class ProcessImageUpload implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $image;

    // Job configuration
    public $timeout = 120; // 2 minutes
    public $tries = 3; // Retry up to 3 times
    public $maxExceptions = 1; // Stop after 1 exception

    public function __construct(Image $image)
    {
        $this->image = $image;
    }

    public function handle()
    {
        // Create thumbnails
        $this->createThumbnails();

        // Optimize original image
        $this->optimizeImage();

        // Upload to cloud storage
        $this->uploadToCloud();

        // Update database
        $this->updateDatabase();

        // Send notification
        $this->notifyUser();
    }

    private function createThumbnails()
    {
        $sizes = [150, 300, 600];

        foreach ($sizes as $size) {
            $thumbnail = ImageProcessor::make($this->image->path)
                ->resize($size, $size, function ($constraint) {
                    $constraint->aspectRatio();
                    $constraint->upsize();
                });

            $thumbnailPath = storage_path("thumbnails/{$size}_{$this->image->filename}");
            $thumbnail->save($thumbnailPath, 85);
        }
    }

    private function optimizeImage()
    {
        $optimized = ImageProcessor::make($this->image->path)
            ->encode('jpg', 90);

        $optimized->save($this->image->path);
    }

    // Additional methods...
}

Dispatching Jobs

There are several ways to dispatch jobs:

// Basic dispatch
ProcessImageUpload::dispatch($image);

// Dispatch with delay
ProcessImageUpload::dispatch($image)
    ->delay(now()->addMinutes(5));

// Dispatch to specific queue
ProcessImageUpload::dispatch($image)
    ->onQueue('image-processing');

// Conditional dispatch
ProcessImageUpload::dispatchIf($image->needs_processing, $image);
ProcessImageUpload::dispatchUnless($image->is_processed, $image);

// Chain jobs
ProcessImageUpload::withChain([
    new OptimizeImage($image),
    new GenerateMetadata($image),
    new SendNotification($image->user)
])->dispatch($image);

Advanced Queue Features

Job Batching

Process multiple related jobs together:

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$images = Image::where('user_id', $userId)->get();

$batch = Bus::batch(
    $images->map(fn($image) => new ProcessImageUpload($image))
)->then(function (Batch $batch) {
    // All images processed successfully
    Log::info("Batch {$batch->id} completed successfully");
})->catch(function (Batch $batch, Throwable $e) {
    // Handle batch failure
    Log::error("Batch {$batch->id} failed: " . $e->getMessage());
})->finally(function (Batch $batch) {
    // Cleanup regardless of success/failure
    Log::info("Batch {$batch->id} finished processing");
})->name('Image Processing Batch')
->allowFailures()
->dispatch();

// Check batch progress
$progress = $batch->processedPercentage();

Job Middleware

Create reusable logic for jobs:

<?php

namespace App\Jobs\Middleware;

class RateLimited
{
    protected $key;
    protected $maxAttempts;
    protected $decayInSeconds;

    public function __construct($key, $maxAttempts = 10, $decayInSeconds = 60)
    {
        $this->key = $key;
        $this->maxAttempts = $maxAttempts;
        $this->decayInSeconds = $decayInSeconds;
    }

    public function handle($job, $next)
    {
        if (RateLimiter::tooManyAttempts($this->key, $this->maxAttempts)) {
            // Release job back to queue with delay
            $job->release(RateLimiter::availableIn($this->key));
            return;
        }

        RateLimiter::hit($this->key, $this->decayInSeconds);

        $next($job);
    }
}

Apply middleware to jobs:

public function middleware()
{
    return [
        new RateLimited("image-processing:{$this->image->user_id}", 5, 60)
    ];
}

Handling Failed Jobs

Implement proper error handling:

public function failed(Exception $exception)
{
    // Log the failure
    Log::error('Image processing failed', [
        'image_id' => $this->image->id,
        'error' => $exception->getMessage(),
        'trace' => $exception->getTraceAsString()
    ]);

    // Update image status
    $this->image->update(['status' => 'failed']);

    // Notify user
    $this->image->user->notify(new ImageProcessingFailed($this->image));

    // Send admin notification for critical failures
    if ($exception instanceof CriticalException) {
        Notification::route('slack', config('slack.admin_webhook'))
            ->notify(new CriticalJobFailure($this));
    }
}

Running Queue Workers

Development

For development, use the simple worker:

# Process default queue
php artisan queue:work

# Process specific queues with priority
php artisan queue:work --queue=high,default,low

# Stop after processing 100 jobs (prevents memory leaks)
php artisan queue:work --max-jobs=100

# Stop after 1 hour
php artisan queue:work --max-time=3600

Production with Supervisor

Create a Supervisor configuration:

[program:laravel-queue-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/your-app/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600 --max-jobs=1000
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=4
redirect_stderr=true
stdout_logfile=/var/www/your-app/storage/logs/queue-worker.log
stdout_logfile_maxbytes=100MB
stdout_logfile_backups=5

Start the workers:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-queue-worker:*

Queue Monitoring and Optimization

Laravel Horizon

For Redis queues, use Laravel Horizon for monitoring:

composer require laravel/horizon
php artisan horizon:install
php artisan migrate

Configure queues in config/horizon.php:

'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['high', 'default', 'low'],
            'balance' => 'auto',
            'maxProcesses' => 10,
            'maxTime' => 0,
            'maxJobs' => 0,
            'memory' => 512,
            'tries' => 3,
            'timeout' => 60,
            'nice' => 0,
        ],
    ],
],

Start Horizon:

php artisan horizon

Performance Monitoring

Track queue performance with custom metrics:

// In your job's handle method
$startTime = microtime(true);

// Your job logic here...

$duration = microtime(true) - $startTime;

// Log performance metrics
Log::info('Job performance', [
    'job' => self::class,
    'duration' => $duration,
    'memory_peak' => memory_get_peak_usage(true),
    'queue' => $this->queue,
]);

// Report to monitoring service
app('metrics')->timing('queue.job.duration', $duration, [
    'job' => self::class,
    'queue' => $this->queue,
]);

Real-World Use Cases

Email Campaign Processing

class SendEmailCampaign implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $campaign;
    public $timeout = 300; // 5 minutes
    public $tries = 2;

    public function __construct(EmailCampaign $campaign)
    {
        $this->campaign = $campaign;
    }

    public function handle()
    {
        $recipients = $this->campaign->getRecipients();

        // Batch recipients to avoid overwhelming the mail service
        $batches = $recipients->chunk(100);

        foreach ($batches as $batch) {
            dispatch(new SendEmailBatch($this->campaign, $batch))
                ->onQueue('email-sending')
                ->delay(now()->addSeconds(10)); // Rate limiting
        }

        $this->campaign->update(['status' => 'sending']);
    }
}

Data Export Processing

class GenerateUserReport implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $reportRequest;
    public $timeout = 1800; // 30 minutes for large reports

    public function handle()
    {
        $filename = "user_report_{$this->reportRequest->id}.xlsx";
        $filePath = storage_path("reports/{$filename}");

        // Generate Excel file
        Excel::store(
            new UsersExport($this->reportRequest->filters),
            $filename,
            'reports'
        );

        // Upload to S3
        $s3Path = Storage::disk('s3')->putFile('reports', $filePath);

        // Update request with download link
        $this->reportRequest->update([
            'status' => 'completed',
            'download_url' => Storage::disk('s3')->url($s3Path),
            'expires_at' => now()->addDays(7)
        ]);

        // Notify user
        $this->reportRequest->user->notify(
            new ReportReady($this->reportRequest)
        );

        // Cleanup local file
        unlink($filePath);
    }
}

Best Practices and Tips

1. Job Design

Keep jobs focused on a single responsibility
Make jobs idempotent (safe to run multiple times)
Use type hints for better error reporting
Implement proper timeout and retry logic

2. Queue Management

Use different queues for different priorities
Monitor queue sizes and processing times
Implement proper logging and alerting
Use Redis for high-throughput applications

3. Error Handling

Always implement the failed() method
Log detailed error information
Implement graceful degradation
Set up alerts for critical failures

4. Performance Optimization

Use job batching for related tasks
Implement rate limiting for external APIs
Monitor memory usage and restart workers
Use appropriate timeout values

5. Security Considerations

Validate job data thoroughly
Use queue encryption for sensitive data
Implement proper access controls
Monitor for queue injection attacks

Testing Queue Jobs

// In your test file
use Illuminate\Support\Facades\Queue;

class ImageUploadTest extends TestCase
{
    public function test_image_upload_dispatches_processing_job()
    {
        Queue::fake();

        $user = User::factory()->create();
        $this->actingAs($user);

        $file = UploadedFile::fake()->image('test.jpg');

        $response = $this->post('/upload', ['image' => $file]);

        Queue::assertPushed(ProcessImageUpload::class, function ($job) use ($file) {
            return $job->image->filename === $file->getClientOriginalName();
        });
    }

    public function test_job_processes_image_correctly()
    {
        $image = Image::factory()->create();

        $job = new ProcessImageUpload($image);
        $job->handle();

        $this->assertTrue($image->fresh()->is_processed);
        $this->assertFileExists(storage_path("thumbnails/150_{$image->filename}"));
    }
}

Conclusion

Laravel queues are essential for building scalable, responsive applications. They allow you to:

Improve user experience by deferring slow operations
Scale your application horizontally
Handle failures gracefully
Process large volumes of background tasks

Start with database queues for development, then move to Redis for production. Always implement proper monitoring, error handling, and testing to ensure your queue system is robust and reliable.

Remember: A well-designed queue system is like a good foundation – invisible when working properly, but essential for supporting everything built on top of it.

Have you implemented Laravel queues in your projects? Share your experiences and challenges in the comments below! What queue drivers do you prefer and why?

Building Model Context Protocol (MCP) Servers with stdio: A Complete Guide

Elsayed Kamal — Sat, 31 May 2025 14:30:25 +0000

Building Model Context Protocol (MCP) Servers with stdio: A Complete Guide

The Model Context Protocol (MCP) is revolutionizing how AI applications interact with external tools and data sources. In this comprehensive guide, we'll walk through building MCP servers using the stdio (standard input/output) transport method, which is perfect for local development and testing.

What is MCP?

The Model Context Protocol is an open standard that enables AI assistants to securely connect to external data sources and tools. Think of it as a universal adapter that allows AI models to interact with databases, APIs, file systems, and custom tools in a standardized way.

Why Choose stdio Transport?

The stdio transport method offers several advantages:

Simplicity: Easy to implement and debug
Local Development: Perfect for testing and development
Security: Runs in the same process space
Performance: Low latency for local operations

Setting Up Your Environment

First, let's set up our development environment. We'll use Python with the FastMCP library for rapid development.

1. Project Structure

Create a new directory for your MCP server:

mkdir stdio_mcp_server
cd stdio_mcp_server

2. Dependencies

Create a pyproject.toml file:

[project]
name = "stdio-mcp-server"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "httpx>=0.28.1",
    "mcp[cli]>=1.9.2",
]

Install dependencies using uv (or pip):

uv sync

Building Your First MCP Server

Let's start with a simple calculator server that demonstrates the core concepts.

Simple Calculator Server

Create calc_two_numbers_mcp_server.py:

from mcp.server.fastmcp import FastMCP
import asyncio
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# 1. Create a FastMCP server instance
mcp = FastMCP(
    name="sum_two_numbers_mcp_server",
)

# 2. Define a tool that sums two numbers
@mcp.tool(
    name="sum_two_numbers",
    description="A tool that sums two numbers.",
)
async def sum_two_numbers_tool(a: int, b: int) -> int:
    """A tool that sums two numbers."""
    return f"The sum of {a} and {b} is {a + b} (Calculated by MCP server)."

if __name__ == "__main__":
    # 3. Run the FastMCP server
    asyncio.run(mcp.run())

Running Your Server

uv run python calc_two_numbers_mcp_server.py

That's it! You now have a working MCP server that can perform calculations.

Building a More Complex Server

Let's create a more sophisticated server that interacts with external APIs. We'll build a dev.to article search server.

Dev.to Search Server

Create dev_to_search_tool_mcp_server.py:

from mcp.server.fastmcp import FastMCP
from typing import List, Dict, Any, Optional
import asyncio
import logging
import httpx
import click

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

async def serve(auth_token: str):
    # Create FastMCP server instance
    mcp = FastMCP(name="DevTo_Search_MCP_Server")

    # Base URL for dev.to API
    DEV_TO_BASE_URL = "https://dev.to/api"

    @mcp.tool(
        name="search_dev_to_articles",
        description="""
        Search for articles on dev.to by topic/keyword.

        Args:
            query (str): The search query/topic to search for
            per_page (int, optional): Number of articles to return (default: 10)
            page (int, optional): Page number for pagination (default: 1)
            tag (str, optional): Filter by specific tag
            username (str, optional): Filter by specific username

        Returns:
            List of article objects with title, description, URL, tags, etc.
        """,
    )
    async def search_dev_to_articles(
        query: str,
        per_page: Optional[int] = 10,
        page: Optional[int] = 1,
        tag: Optional[str] = None,
        username: Optional[str] = None,
    ) -> List[Dict[str, Any]]:
        """Search for articles on dev.to using the public API."""
        try:
            # Build query parameters
            params = {
                "per_page": min(per_page, 1000),  # API limit
                "page": page,
            }

            if query:
                formatted_query = query.replace(" ", "+")
                params["q"] = formatted_query

            # Add optional filters
            if tag:
                params["tag"] = tag
            if username:
                params["username"] = username

            # Make API request
            async with httpx.AsyncClient() as client:
                response = await client.get(
                    f"{DEV_TO_BASE_URL}/articles", params=params
                )
                response.raise_for_status()
                return response.json()

        except httpx.HTTPError as e:
            logger.error(f"HTTP error occurred: {e}")
            return {"error": f"Failed to fetch articles: {str(e)}"}
        except Exception as e:
            logger.error(f"Unexpected error occurred: {e}")
            return {"error": f"An unexpected error occurred: {str(e)}"}

    # Run the server
    await mcp.run()

if __name__ == "__main__":
    asyncio.run(serve("your-api-token"))

Key MCP Concepts

1. Tools

Tools are the core functionality your MCP server provides. They're defined using the @mcp.tool decorator:

@mcp.tool(
    name="tool_name",
    description="Clear description of what the tool does",
)
async def tool_function(param1: type, param2: type) -> return_type:
    """Implementation of your tool"""
    return result

2. Type Hints

Always use proper type hints for parameters and return values. This helps the AI understand how to use your tools correctly.

3. Error Handling

Implement robust error handling to provide meaningful feedback when things go wrong.

4. Logging

Use logging to help debug issues and monitor your server's performance.

Best Practices

1. Clear Descriptions

Write detailed descriptions for your tools. The AI uses these to understand when and how to use each tool.

2. Parameter Validation

Validate input parameters to prevent errors and provide helpful feedback.

3. Async/Await

Use async/await for I/O operations to maintain server responsiveness.

4. Resource Management

Properly manage resources like HTTP connections using context managers.

5. Configuration

Use environment variables or configuration files for sensitive data like API keys.

Testing Your MCP Server

1. Local Testing

Test your server locally using the MCP CLI tools or by integrating with compatible AI applications.

2. Integration Testing

Test with actual AI assistants that support MCP to ensure compatibility.

3. Error Scenarios

Test error conditions to ensure your server handles failures gracefully.

Deployment Considerations

1. Process Management

For production use, consider using process managers like systemd or supervisord.

2. Monitoring

Implement health checks and monitoring to ensure your server remains available.

3. Security

Validate all inputs
Use environment variables for secrets
Implement rate limiting if needed
Consider authentication for sensitive operations

Conclusion

MCP servers with stdio transport provide a powerful yet simple way to extend AI capabilities. Start with simple tools and gradually build more complex functionality as needed. The key is to focus on clear interfaces, robust error handling, and comprehensive documentation.

The examples we've built demonstrate both basic arithmetic operations and complex API interactions. Use these patterns as starting points for your own MCP servers, whether you're integrating with databases, file systems, or external services.

Happy building! 🚀

Resources

Have you built an MCP server? Share your experience in the comments below!

Mastering Laravel Queues: A Complete Guide to Background Job Processing

Elsayed Kamal — Sat, 31 May 2025 14:27:59 +0000

Mastering Laravel Queues: A Complete Guide to Background Job Processing

Laravel's queue system is one of its most powerful features, allowing developers to defer time-consuming tasks and improve application performance. Whether you're sending emails, processing images, or generating reports, queues can help you build more responsive applications.

What Are Laravel Queues?

Laravel queues provide a unified API for various queue backends, allowing you to push time-consuming tasks to the background. This means your web requests can respond quickly while heavy lifting happens behind the scenes.

Why Use Queues?

1. Improved User Experience

Instead of making users wait for slow operations, you can queue them and provide immediate feedback.

2. Better Performance

Offload CPU-intensive tasks to background workers, keeping your web interface responsive.

3. Fault Tolerance

Failed jobs can be retried automatically, ensuring reliability.

4. Scalability

Run multiple queue workers across different servers to handle increased load.

Setting Up Laravel Queues

Configuration

First, configure your queue driver in .env:

QUEUE_CONNECTION=database
# or redis, sqs, etc.

For database queues, run the migration:

php artisan queue:table
php artisan migrate

Creating Jobs

Generate a new job class:

php artisan make:job ProcessPayment

This creates a job class in app/Jobs:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class ProcessPayment implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $payment;

    public function __construct($payment)
    {
        $this->payment = $payment;
    }

    public function handle()
    {
        // Process the payment
        $paymentGateway = new PaymentGateway();
        $paymentGateway->charge($this->payment);

        // Send confirmation email
        Mail::to($this->payment->user)->send(new PaymentConfirmation($this->payment));
    }
}

Dispatching Jobs

Basic Dispatching

// Dispatch immediately
ProcessPayment::dispatch($payment);

// Dispatch with delay
ProcessPayment::dispatch($payment)->delay(now()->addMinutes(5));

// Dispatch to specific queue
ProcessPayment::dispatch($payment)->onQueue('payments');

Conditional Dispatching

// Only dispatch if condition is met
ProcessPayment::dispatchIf($payment->requires_processing, $payment);

// Dispatch unless condition is met
ProcessPayment::dispatchUnless($payment->is_processed, $payment);

Queue Workers and Management

Running Queue Workers

Start processing jobs:

# Process jobs from default queue
php artisan queue:work

# Process specific queue
php artisan queue:work --queue=payments,emails

# Process jobs with timeout
php artisan queue:work --timeout=60

Supervisor Configuration

For production, use Supervisor to manage queue workers:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=8
redirect_stderr=true
stdout_logfile=/path/to/your/app/storage/logs/worker.log

Advanced Queue Features

Job Batching

Process related jobs together:

use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;

$batch = Bus::batch([
    new ProcessPayment($payment1),
    new ProcessPayment($payment2),
    new ProcessPayment($payment3),
])->then(function (Batch $batch) {
    // All jobs completed successfully
})->catch(function (Batch $batch, Throwable $e) {
    // First batch job failure detected
})->finally(function (Batch $batch) {
    // The batch has finished executing
})->dispatch();

Job Middleware

Create reusable job logic:

<?php

namespace App\Jobs\Middleware;

class RateLimited
{
    public function handle($job, $next)
    {
        Redis::throttle('key')
                ->block(0)->allow(1)->every(5)
                ->then(function () use ($job, $next) {
                    $next($job);
                }, function () use ($job) {
                    $job->release(5);
                });
    }
}

Apply middleware to jobs:

public function middleware()
{
    return [new RateLimited];
}

Failed Jobs

Handle job failures gracefully:

public function failed(Exception $exception)
{
    // Send user notification of failure, etc..
    Log::error('Payment processing failed', [
        'payment_id' => $this->payment->id,
        'error' => $exception->getMessage()
    ]);
}

Retry failed jobs:

# Retry all failed jobs
php artisan queue:retry all

# Retry specific job
php artisan queue:retry 5

Monitoring and Optimization

Queue Monitoring

Use Laravel Horizon for Redis queues:

composer require laravel/horizon
php artisan horizon:install
php artisan horizon

Performance Tips

Use appropriate queue drivers - Redis for high-throughput, database for simplicity
Set proper timeouts - Prevent jobs from running indefinitely
Monitor memory usage - Restart workers periodically
Use job batching - Process related jobs efficiently
Implement proper error handling - Log failures and notify administrators

Queue Configuration Best Practices

// config/queue.php
'connections' => [
    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
    ],
],

Real-World Examples

Email Notifications

class SendWelcomeEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $user;

    public function __construct(User $user)
    {
        $this->user = $user;
    }

    public function handle()
    {
        Mail::to($this->user)->send(new WelcomeEmail($this->user));
    }
}

// Usage
SendWelcomeEmail::dispatch($user);

Image Processing

class ProcessUploadedImage implements ShouldQueue
{
    public $image;

    public function handle()
    {
        $image = Image::make($this->image->path);

        // Create thumbnails
        $image->resize(300, 200)->save(storage_path('thumbnails/' . $this->image->name));

        // Optimize original
        $image->encode('jpg', 85)->save();
    }
}

Conclusion

Laravel queues are essential for building scalable, responsive applications. By moving time-consuming tasks to the background, you can provide better user experiences while maintaining system performance.

Start with simple database queues for development, then move to Redis or SQS for production. Remember to monitor your queues, handle failures gracefully, and scale your workers based on demand.

The investment in setting up a proper queue system will pay dividends as your application grows and handles more complex background tasks.

What's your experience with Laravel queues? Share your tips and challenges in the comments below!

Forem: Elsayed Kamal

Mastering Laravel Queues: A Complete Guide to Asynchronous Task Processing

Mastering Laravel Queues: A Complete Guide to Asynchronous Task Processing

What are Laravel Queues?

Key Concepts

1. Queue Drivers

2. Jobs

3. Dispatching Jobs

Configuration

Queue Configuration (config/queue.php)

Environment Variables

Queue Workers

Running Workers

Supervisor Configuration

Advanced Features

1. Job Batching

2. Job Chaining

3. Failed Jobs

4. Job Events

Best Practices

1. Job Design

2. Performance Optimization

3. Queue Prioritization

4. Monitoring

Common Use Cases

Database Setup

Conclusion

Laravel 12.17.0: What's New in 2025 and Why You Should Care

Laravel 12.17.0: What's New in 2025 and Why You Should Care

🚀 What's New in Laravel 12.17.0

AsUri Model Cast

Contextual Binding with PHP 8 Attributes

Enhanced Query Builder

Cursor Paginator Improvements

🌟 Laravel Ecosystem Highlights

Laravel Cloud is Live!

Laravel Nova 5.0

Inertia 2.0: The Frontend Revolution

💰 The $57M Investment Impact

🔧 Developer Experience Improvements

Laravel Herd 1.11

PHP 8.4 Support

🎯 Why Choose Laravel in 2025?

1. Mature Ecosystem

2. Enterprise Ready

3. Active Community

4. Learning Resources

🚀 Getting Started with Laravel 12

🔮 What's Next?

Conclusion

Mastering Laravel Queues: A Complete Guide to Background Job Processing

Mastering Laravel Queues: A Complete Guide to Background Job Processing

What are Laravel Queues?

Key Benefits

Setting Up Laravel Queues

1. Configuration

2. Database Setup

3. Creating a Job

Dispatching Jobs

Basic Dispatching

Delayed Jobs

Job Chaining

Advanced Queue Features

Job Batching

Rate Limiting

Unique Jobs

Error Handling and Retries

Automatic Retries

Failed Job Handling

Running Queue Workers

Basic Worker

Worker with Options

Supervisor Configuration

Monitoring and Debugging

Queue Monitoring

Debugging Failed Jobs

Best Practices

1. Keep Jobs Small and Focused

2. Use Job Middleware

3. Handle Job Dependencies

Queue Configuration (`config/queue.php`)