Forem: Mateusz Cholewka

Beyond the Hype: Why We Chose Redis Streams Over Kafka for Microservices Communication

Mateusz Cholewka — Tue, 19 Aug 2025 23:10:01 +0000

I'm writing this article based on my personal experience implementing a production-ready solution using Redis Streams with my team. This is a result of a few months working with Redis Streams, development and maintenance. If you're not sure, which solution to choose, I hope this article helps you choose the best solution for your needs.

Introduction

The problem we were facing was quite common. The question was how to communicate between two microservices in a simple, fast, and reliable way. Our first thought was to choose between two of the most popular solutions: RabbitMQ or Kafka. We wanted something that could handle higher throughput in the future, so it seemed, that Kafka would be better choice. It met all of the above requirements, and it's also a resume-friendly solution.

However, there were two problems: we didn't have it in our stack yet, and nobody on the team had experience with Kafka or operating a Kafka server. Also, since the project runs on PHP, integration is not as straightforward as it is with Java for example. Additionally, a PHP extension to integrate with Kafka was not well supported these days, and we didn't feel comfortable using it. Nowadays, of course, the situation is different, and there are some stable solutions for PHP Kafka.

When we were focused on learning Kafka, and all the features that we don't actually need, someone suggested that Redis has a messaging system called Redis Streams. So we began researching it.

How Redis can be an alternative to Kafka?

When I first heard about Streams, I thought, "Okay, I'm not sure if it will help, but let's see." I started researching, and step by step, I became more and more convinced that it actually meets all of our expectations, and it's much easier from operational perspective. Also we already had it in our stack.

So, we had to compare Kafka and Redis. The best option is probably to write down some architectural drivers and compare them. Let's take a few:

Persistence and replayability: We don't need too much of this.
Operational complexity: It's an MVP, so it needs to be as simple as possible.
Performance: We might need to handle high throughput in the future.
Learning curve: Smaller is better.

Driver	Kafka	Redis Streams
persistence and replayability	persisting in cheaper disk space, and can replay from the very beginning	persisting data in more expensive RAM, long streams are possible but more expensive
operational complexity	High	Low or Very Low if you have Redis already in stack
performance	Highest throughput Medium latency	High throughput Low latency
learning curve	Steep, needs to understand how the system works and how to integrate it	Gentle, the same well known API as used in many different cases. Easy to integrate, every language have it applied

Kafka often achieves higher throughput at the expense of higher latency, while Redis Streams has lower latency but its maximum throughput may be lower.

Following those few most important drivers, it seems that Redis Streams could be a good solution. What still leaves me unconvinced is the lack of information about the real problems we could face. Most articles are theoretical. That's also why I decided to write this post.

Thankfully, I finally found a document that totally convinced me. It was official document, prepared by Redis, and it dives deeper into the details of the comparison between Kafka and Redis Streams. There are many examples with visual presentations. You can download the document from the Redis website: https://redis.io/resources/understanding-streams-in-redis-and-kafka-a-visual-guide/. This document also shows some great ideas, how to achieve similar features as in Kafka, like partitioning.

How Redis Streams works?

The above document explains everything very well, but I will describe the basics here. I encourage you to test these commands using Redis Insight (https://redis.io/insight/). This will demonstrate how simple it is to start using streams.

Simple messaging

The easiest way to start working with streams is to set up a stream and start consuming messages. In fact, the only way to create a stream is to simply start sending messages to it. If you try:

XADD mymessages * payload "This is the payload of a message"

This will automatically create the stream named mymessages and add a message with a payload field containing the message in quotes.

Okay, so now we should read this message somehow. We can do that by running:

XREAD COUNT 1 STREAMS mymessages 0-0

Output:
1) 1) "mymessages"
   2) 1) 1) 1526984818136-0
         2) 1) "payload"
            2) This is the payload of a message"

Where 1 you can specify how many messages you want to read from the stream at once. You can increase this number if you're processing more messages at once. After reading and processing it properly, you can remove the message by running the following command:

XDEL mymessages 1526984818136-0

This will clean up already processed messages from our stream.

That's how it works for the simplest case, with one consumer.

Consumer groups

Okay, we know how to use streams for one consumer. But the real world is not that simple. You may want to read streams from multiple instances of a single microservice or even multiple microservices. How can we manage this case while ensuring deliverability and preventing one message from being processed multiple times? This is where consumer groups shine.

To create a consumer group, simply run:

XGROUP CREATE mymessages mygroup 0

You can use any name to describe the group of services that will consume those messages. Next, when you want to assign a specific instance of a microservice, for example, what do you do? As always, there is an upsert approach, so you just need to start listening from a specific group.

XREADGROUP GROUP mygroup consumerid STREAMS mymessages 0

What is consumerid? It can be any string that helps you identify the specific consumer in case of any problems with consumption. For example, if you're using Kubernetes, you can use the container ID. This will help you identify the correct container in your infrastructure.

Since Redis promises to guarantee message delivery within a group, we need to inform it when a message has been properly processed. Otherwise, Redis won't know if it can take another entry from the stream, and it won't be able to properly share entries between all consumers. That's why we need to acknowledge it.

XACK mymessages mygroup 1526984818136-0

Now the message is marked as acknowledged and will not be delivered again to the same consumer group.

Control stream length

As with multiple consumer groups you need to keep longer streams. You can't just remove message after processing, because they have to be delivered to all groups. Since messages are stored in memory, you should monitor, and control its size to avoid overflow. You can control the size manually by running:

XTRIM mymessages MAXLEN 1000

This will reduce the length of the stream to 1,000 by removing the oldest messages. However, this method may be inefficient.

There is a better solution. You can configure Redis to automatically adjust the stream length, by modifying XADD command following way:

XADD mymessages MAXLEN ~ 1000 * payload "This is the payload of a message"

You've probably noticed the ~ after MAXLEN; it stands for "approximate trimming." This makes Redis less strict about the stream's length, allowing it to trim messages asynchronously in the background. This approach is far more efficient as it doesn't block write operations. In my case, with the stream length oscillating around just a few messages, so this shouldn't be a problem in most cases.

Real world problems

Of course, the real world is not as beautiful as documentation will describe to you. There are always known problems that you'll only face when you start using Redis. It's also good to mention that every specific use case can cause different problems and expose different limitations. I've encountered some of these problems and will describe them here, along with possible solutions that you can use when you encounter them.

What if Redis is out?

The first thing you should consider if you want to use this solution in production is ensuring high availability. There are two main ways to do so: Redis Sentinel or Redis Cluster. I won't show you how to configure either of them in your infrastructure because that would require an entire article, and this one is already quite long. However, there is an alternative if you're building an MVP and have less time for configuration.

If your project is written in Node.js, there is one more option. There's a library called ioredis that has a very useful feature. It's an offline queue. What does this mean? When you have a single instance of Redis without any high availability configuration and Redis goes down, either on purpose or accidentally, ioredis will keep all the redis commands you've sent, in memory. Once Redis is back, all the queued commands will be sent to Redis, and all your subsequent commands will be sent normally. In this case, you won't lose any messages, even if Redis stops working.

I don’t know if any library in other languages has a similar feature, but I can describe what it looks like in PHP. In PHP, there are probably two main Redis libraries: Predis and PHPRedis (as a C extension). Unfortunately, neither of them provides this functionality. If you need this functionality and cannot ensure an HA configuration, you can try implementing it yourself. It's probably possible by using APCu storage, which works between all PHP-FPM workers and is a shared storage. However, a better option is to implement a cluster or a sentinel.

Memory size and stream length

Redis stores data in memory, which makes it really fast, that's well-known. However, this can cause problems, especially with streams. Messages can have different-sized payloads, and sometimes you know the size of the message you can get, but sometimes you have no control over that. For example, consider processing data received from an external system via webhooks.

What is the problem here? Redis Streams actually allows you to set the size of a stream, but only for its length. There is no option to set the maximum memory consumption of a stream. This small detail can sometimes cause problems. How can this problem be handled? There are a few possible solutions.

Compress the payload - This solution is simple and effective. Compression is widely adopted by many programming languages, and you can choose from many different compression algorithms. The easiest way to implement this is to use gzip, which is probably available everywhere. You can take the metadata and send it in one non-compressed field. You can add data, such as an identifier or information that helps you decide whether the message should be processed and whether it's worth uncompressing. In our case, compressing a JSON payload gives us a compression ratio of around 3. It's also good to mention that you can send compressed binary data directly into the Redis Stream entry field. If you use Redis Insight, you can set the compression method in the connection settings and see the uncompressed data in the stream preview.

Hybrid Solution - If you know you'll have to process huge message payloads, you should consider a hybrid solution. For example, you could send only the metadata of a message using a stream and store the rest in S3. This solution is complex, but it will save you if you encounter such a situation. Metadata is more predictable, that can be passed through a stream. By setting the proper stream length, you can easily control memory usage.

Use XACKDEL or XDEL - if you don't need to store messages for re-reading after processing; you can just delete them. You can use XDEL, but be careful — XDEL removes an entry regardless of whether it was acknowledged by all consumer groups. Since Redis 8.2, you can use XACKDEL, which is useful if you have multiple consumer groups and want to maintain delivery to all groups. You can choose a strategy for removing the entry. For example, you can use ACKED, which removes the entry after processing only if all consumers have acknowledged the message. This option was added to Redis 8.2, which was released a week ago, so it was not available when I started using Redis Streams.

Data eviction policy and streams

This point relates to the previous one. Redis has several strategies for when it runs out of memory. You can remove the latest keys, random keys, etc. Intuitively, you might think that setting maxmemory to remove the oldest keys would remove the oldest messages and everything would be fine. However, that's not the case: https://github.com/redis/redis/issues/4543. The problem is that "key" is a keyword. In Redis streams, a stream references a key in terms of Redis. So, if you set this policy to LRU, LFU, or Random, and stream is like any other key, it can be removed entirely.

Replaying long streams and persistence

Redis Streams allows you to replay entire stream from the very beginning. Of course only as far as long is your stream, and if you're not using XDEL after ACK or XACKDEL. But if you really need this possibility to exist, you also need to remember that all this data is stored in the memory, so it's not persisted. Even if you don't need to keep very long streams, you should care about the persistence in terms of any outage.

Redis Streams are working same way as all other keys in Redis, and they are also included into persistence settings. So if you already have Redis in your infrastructure or you're creating a new one, you should take care about this configuration. If you're not familiar with redis persistence, here you can find all the possible options

Conclusion

Redis Streams is a great alternative to a messaging service like Kafka, especially if you're looking for simplicity and ease of use. After using Redis Streams in production for a year, I can confidently say that it was the right choice for our use case.

Choose Redis Streams if you:

Already have Redis in your stack
Need low latency and moderate throughput
Want minimal operational complexity

Stick with Kafka if you:

Need to process millions of messages per second (for sure)
Need long-term persistence and replay capabilities
Have a dedicated infrastructure team with knowledge about Kafka

The YAGNI principle applies perfectly here. If you've started working with a communication system that uses messaging, you probably don't need all the fancy features that other solutions has. For many real-world scenarios, Redis Streams might be the sweet spot between functionality and complexity. Redis Streams are also inexpensive to implement, so if you decide to switch to Kafka in the future, you won't have wasted too much work.

References

Understanding Streams in Redis and Kafka – A Visual Guide - if you're considering to use streams IMO it's obligatory doc to read, even if you're not choosing between Kafka and Redis, the mechanisms of streams are very well visually explained.
Kafka, RabbitMQ, and Redis Streams Benchmark - this article may help you to choose better solution from performance perspective
Redis Streams 8.2 Release changelog - since 8.2, you can use XACKDEL command to safely remove processed messages, and still delivery them to all consumer groups.
Note about offline queue in ioredis - useful when you have single non HA Redis instance and want to ensure deliverability
Redis persistence configuration - you should take care about that if you want your stream to be resistant to any Redis restart

Originaly posted on https://mateuszcholewka.com/post/why-we-chose-redis-streams-over-kafka/

A messy local environment can slow you down more than you think. In this post, I walk through 5 frequent setup issues and simple ways to clean them up

Mateusz Cholewka — Sun, 23 Mar 2025 19:42:45 +0000

5 Local Environment Mistakes I See Everywhere, and How to Fix Them Properly

Mateusz Cholewka ・ Mar 21

#devops #programming #beginners #backend

5 Local Environment Mistakes I See Everywhere, and How to Fix Them Properly

Mateusz Cholewka — Fri, 21 Mar 2025 20:18:46 +0000

Every new project brings fresh challenges, but the very first one is always the same: getting it to run locally.

Every time I start working on a new project, I end up asking other developers how to launch it. Documentation is usually long, outdated, and full of manual steps, like adjusting configs, copying envs, secrets, and so on.

Luckily, it doesn’t have to be this way. A few small practices can make your local environment clean, repeatable, and easy to work with, not just for you, but for the entire team.

1. Too Many Steps Instead of One Clean Command

Most common thing I see in readme.md in projects I’m starting working on, is a list of steps to make your local env up and running. And it’s always the same story:

Alright, let’s go:
Install Docker — check.
Install PHP — check.
Install Composer — check...
…
Okay, looks good. Time to run docker compose up — and... oh, yea it’s failing.
Now I’m messaging another developer: “Hey, I did everything in the README and it’s not working.”
We debug it together.
“Oh, you just have to copy veryimportantfile.txt to very_veryimportantfile.txt, then it’ll work.”
Cool. Let’s add the 2137th step to the setup instructions, then next developer won’t make same mistake…

In my opinion, every project should be bootstrapped with a single command. One. Not seven, not five, just one. It doesn’t matter if you use Docker, install local dependencies manually, or mix both. Starting the environment should be easy and predictable.

So instead of this:

    ## Requirements
    - docker
    - php >= 8.0
    ...

You can simply make this:

    # ./launch.sh:
    #!/bin/bash

    if ! command -v brew &> /dev/null; then
      echo "❌ Homebrew is not installed. Install it from https://brew.sh/"
      exit 1
    fi

    if ! command -v docker &> /dev/null; then
      echo "⚙️ Installing Docker..."
      brew install --cask docker
    fi

    if ! command -v php &> /dev/null; then
      echo "🐘 Installing PHP..."
      brew install php@8.3
    fi

Isn’t that simpler? You don’t need to update the docs, just update script here and that’s it.

Next, you have to bootstrap your project. Do you need to copy .env.dist ? Let’s copy it here:

    [ -f .env ] || cp .env.local .env

You need to run yarn? That’s your next bash script line:

    yarn

You need to run docker compo…, and so on, until your project will be up and running.

And one more thing — make your bootstrap script idempotent.
That means every operation should give the same result no matter how many times it runs. Installing dependencies? Only if they’re not already installed. Copying files? Only if they don’t exist.
It makes your script faster, safer, and a lot less likely to break because "something was already there".

That rule, allows you to keep your readme very simple and clean, with only the most important things.

2. Make CLI, but don’t reinvent the wheel, use existing tools

All applications are different, and all of them require different actions to do, during development, like make migrations, run tests, reset database etc. Different frameworks already have some CLI commands for such actions, but there are also many of others. To make it easy for developers, you can build CLI specifically for your project, which collect all useful actions in one place. But how to create that?

There are plenty of solid, battle-tested tools out there like Makefile, Taskfile, or even plain shell scripts. They work cross-platform, they’re easy to read, and they don’t require extra effort to set up or maintain. In most cases, they’re more than enough.

Avoid building custom CLIs that need to be compiled or rely on specific runtimes. I met with the projects that has some tools like that, compiled for linux, and it was hard to run them on mac or windows. I had to start a virtual machine with linux to run them. I can’t imagine what will I do if they will stop working some day, and I don’t have any source code of them, because it was not shared when transfering application between software houses.

3. Make developers experience friendly

If you want to let developers choose which services to run or ask them for input during setup, keep it simple. Don’t reinvent CLI menus in raw bash. It’s messy, unmaintainable, and hard to debug.

There are tools built exactly for that. One of the best is Gum - a small utility that helps build terminal user interfaces with checkboxes, file pickers, prompts, and more:

It’s lightweight, works well across platforms, and integrates easily with your scripts. You only need to install it at the very beginning of your bash script.

Use tools like this to keep your setup flexible and clean. Good DX doesn’t have to be complicated.

4. Let developers choose what services to run

Not every developer needs every service running all the time. A frontend dev might not care about your backend profiler or a local database admin tool. Forcing everyone to run the full stack wastes resources and slows things down.

If you are using docker compose, you can use functionality called profiles. Let’s take this example:

    version: "3.9"

    services:
      app:
        image: my-app
        build: .
        ports:
          - "8080:80"

      db:
        image: mysql:8
        environment:
          MYSQL_ROOT_PASSWORD: root
        ports:
          - "3306:3306"

      mailhog:
        image: mailhog/mailhog
        profiles:
          - optional
        ports:
          - "8025:8025"

      adminer:
        image: adminer
        profiles:
          - optional
        ports:
          - "8081:8080"

      grafana:
        image: grafana/grafana-oss:latest
        container_name: grafana
        ports:
          - "3000:3000"
        profiles:
          - monitoring

Now you can add a COMPOSE_PROFILES= variable to your project’s .env file. Docker Compose will read this value automatically and run only the services that match the specified profiles.
For example:

    COMPOSE_PROFILES=optional,monitoring

This way, developers can easily control what parts of the stack get started

You can even take it one step further and write a small bash script using gum to give developers a clean, interactive UI to select which profiles they want to launch. It makes setup feel more like an app than a checklist.

5. Don’t make developers import the database data manually

If your project doesn’t use any stack specific tool to setup database, and relies on a database dump to set things up, don’t expect developers to import it by hand. That should be part of the automated bootstrap process, just like anything else.

You can easily set this up in Docker. For example, if you're using MySQL or MariaDB, just mount a .sql dump into the container. MySQL will automatically import it on first run:

    services:
      db:
        image: mysql:8
        environment:
          MYSQL_DATABASE: app
          MYSQL_ROOT_PASSWORD: root
        volumes:
          - ./dump.sql:/docker-entrypoint-initdb.d/dump.sql

That’s it. First time the container starts, it’ll load the dump. No need to run mysql -u root by hand. No need to write it in the README. And no chance for someone to skip the step or break things.

Bonus: MacOS and Docker Desktop limitations

On macOS, Docker runs inside a VM, most developers use Docker Desktop, which works but comes with two main issues:

Licensing - it's free only for small teams. Larger companies need a paid plan (check Docker's terms).
Manual configuration - Docker Desktop’s default memory settings are too low for many real-world projects, and increasing them requires opening the UI. You can’t configure it from the terminal, which makes automation impossible and leads to confusing, silent errors.

A better option is Colima. It replaces Docker Desktop, uses the exact same CLI (docker), and gives you full control via the terminal, so you can set it up in your launch script.

To start Colima with more RAM, just run:

    colima start --memory 16

Now you're running Docker with 16GB of RAM, no licensing issues, and full automation support. One command and you're ready to go, clean and repeatable.

Linux users don’t have this problem, as Docker runs natively there.

Summary

Local environments don’t have to be painful. With a few simple rules you can make setup smooth for everyone on the team.
Use existing tools, avoid overengineering, and document only what you can't automate. It’ll save hours on onboarding, reduce frustration, and make your project feel a lot more professional.

You can find more articles like this at mateuszcholewka.com. Got a question? Drop a comment below or reach out on LinkedIn.

Introducing LaraTUI: A Terminal UI for Laravel Environments

Mateusz Cholewka — Mon, 24 Jun 2024 23:02:13 +0000

I'm excited to introduce LaraTUI, a new open-source project I've been working on. LaraTUI is a terminal user interface designed to help you manage your Laravel local environment using PHP. It's still a work in progress!

Current Features:

Sail Service Info: See your sail services status
Composer Versions check: Check for new updates to your Composer packages
Migrations info: Get information about new migrations pending to run

Planned Features:

Artisan Commands: Run Artisan commands directly from the interface.
Migration Management: View and run pending migrations.
Sail Service Management: Start/Stop and mange your local sail environment.
Log Viewing: Access and filter your application logs.
Environment Variables: Verify your .env settings within the TUI.

Why PHP for a TUI?

Using PHP for a terminal UI is not very common, but it offers a great way to leverage the existing Laravel ecosystem and my PHP knowledge for a new kind of tool. It's also possible thanks to awesome php-tui library: https://github.com/php-tui/php-tui

Get Involved

LaraTUI is open source, and I'd love to get feedback and contributions from the community.
Check it out on GitHub: https://github.com/mtk3d/LaraTUI

Feel free to share your thoughts and suggestions!

Happy coding! 🚀

Restore missing Command Bus in new Laravel versions

Mateusz Cholewka — Tue, 22 Mar 2022 00:21:55 +0000

Command Bus is a very powerful pattern, which can help you in decoupling your application modules. You can in an easy way separate small actions, and trigger them using a single dependency called CommandBus.

The cause why this pattern is able to decouple your modules is that actually, the module that is dispatching the command doesn’t have to care about the actual implementation of the action. No matter if that’s a simple DB query, complicated calculations, or calling another remote service. Using this pattern will help you also if you want to implement one of CQS and CQRS patterns.

The CommandBus pattern

This pattern requires three elements:

Command - is a small class that name should describe the action that you want to call. I should store all data required to execute actual action.

class ReserveResource
{
    public function __construct(
        private ResourceId $resourceId,
        private Period $period
    ) {}

    public function getId(): ResourceId
    {
        return $this->resourceId;
    }

    public function getPeriod(): Period
    {
        return $this->period;
    }
}

Handler - it's the actual implementation of the action. It should implement handle or __invoke method that is handling a Command.

class ReserveResourceHandler
{
    public function __construct(
        private ResourceRepository $resourceRepository
    ) {}

    public function handle(ReserveResource $reserveResource): void
    {
        $resource = $this->resourceRepository->find($reserveResource->getId());
        $resource->reserve($reserveResource->getPeriod());
        $this->resourceRepository->save($resource);
    }
}

Bus - it’s the last element of this puzzle. This is the element that keeps all definitions of relations between Commands and Handlers, and it’s also able to run right Handler. That’s the part that is generic and you can use ready implementations, so implementing that on your own doesn’t make sense.

Example usage of CommandBus:

$bus = new CommandBus(); // Create new CommandBus instance
$bus->map([ReserveResource::class, ReserveResourceHandler::class]); // Map command and handler

// Create command
$reserveResource = new ReserveResource(
    'affa2136-5b90-4f7c-933e-73cffc39f1d9',
    new Period('2020-12-06 15:30', '2020-12-06 16:30')
);

$bus->dispatch($reserveResource); // Dispatch command

Bus implementation

There are some PHP, ready implementations of CommandBus. One of them is the component of the popular Symfony framework, it’s called Messenger. You can also use one of The PHP League packages called Tactician. But if you’re programming in Laravel and you need simple as possible CommandBus implementation, there is an option to use the existed part of the framework.

Older versions of Laravel had an implementation of CommandBus out of the box. Today the name of this feature was changed, and it’s split to Events and Jobs / Queues, but the event can be queued too…??? I do not fully understand this separation, but I know it may be helpful when you need to implement something quickly.

Ok, never mind let's take a look at Laravel 5 documentation:

Bus::dispatch(
    new PurchasePodcast(Auth::user(), Podcast::findOrFail($podcastId))
);

Ok, this is a usage of Facade called Bus. Let's try to find out if this class is still available in Laravel code. Here it is still available.
Ok, but I don't like facades, I think that the name of them is badly used, and Laravel facades are like hidden dependencies of your classes. It's always better to inject the dependency from the DI container. So I was investigating the code, and I got to the Illuminate\Bus\Dispatcher class. And that's the part of Laravel framework that is implemented like Command Bus functionality.

I’m not recommending using this class directly. This class is not specified in documentation and it could be removed or replaced one day in Laravel (it probably won't). Also, you may want to improve the functionalities of your command bus, so using this one directly, will cause the problem in replacing it in all the places of usage.

Let's create the Adapter that will be also our safe proxy. But also do not include our Adapter directly. Let's use an interface that will allow us to juggle with implementations.

interface CommandBus
{
    public function dispatch($command): void;
    public function map(array $map): void;
}

And next the implementation:

use Illuminate\Bus\Dispatcher;

class IlluminateCommandBus implements CommandBus
{
    public function __construct(private Dispatcher $bus) {}

    public function dispatch($command): void
    {
        $this->bus->dispatch($command);
    }

    public function map(array $map): void
    {
        $this->bus->map($map);
    }
}

Next, we can declare it in the DI container in the service provider. Remember to register it as a Singleton. It's important to get the same instance of the bus in every place of the application.

public function register()
{
    $this->app->singleton(CommandBus::class, IlluminateCommandBus::class);
}

Now you can define the Commands and Handlers map, and the Command Bus it's ready to use.

So first register

public function register()
{
    /** @var CommandBus $bus */
    $bus = $this->app->make(CommandBus::class);
    $bus->map([
        ReserveResource::class => ReserveResourceHandler::class,
        TurnOnResource::class => TurnOnResourceHandler::class,
        WithdrawResource::class => WithdrawResourceHandler::class,
        CreateResource::class => CreateResourceHandler::class,
    ]);
}

and use

class ReservationController
{
    public function __construct(private CommandBus $bus) {}

    public function reserve(int $id, ReserveRequest $request): void
    {
        $command = ReserveResource::fromRaw(
            $id,
            $request->get('from'),
            $request->get('to')
        );

        $this->bus->dispatch($command);
    }
}

Alpine.js modal transition in Livewire the missing manual part

Mateusz Cholewka — Sun, 13 Feb 2022 19:51:45 +0000

For the last few days, I'm learning how to use the Laravel Livewire framework. I'm really impressed with how well it works, and how fast we can develop working applications using Livewire along with Tailwind CSS. It's a great option for someone who wants to start his own startup as a one-man army. Only you need to know is PHP or more precisely Laravel and some HTML / CSS.

Not everything is possible over the wire

HTML over the wire (which is the main concept behind Livewire) is very powerful, but not everything is possible to do using only on the backend. Some UX features require to use of the frontend API.

One of the things that are not possible using only Livewire is transition animations of HTML elements. There was functionality wire:transition before the first release of Livewire, but it has been removed in the final release for some reason.

When you try to look for a Livewire transition in livewire/livewire GitHub issues, almost all of them contain the information, that there are no attributes like wire:transition and that Livewire supports Alpine.js transitions. That's fine, but there are no working examples, and some GitHub users were a bit confused about that.

The solution

tl:dr If you want just to copy and paste the working solution, please go ahead 😉
I prepared a working modal component in a gist: https://gist.github.com/mtk3d/699502a70ee9af1cd412ddcb805e20da

First let's create a basic modal in Livewire, just using blade @if statement.

class Modal extends Component
{
    public string $content;
    public bool $show;

    public function mount(): void
    {
        $this->content = 'Modal content';
    }

    public function render()
    {
        return view('livewire.modal');
    }
}

And the blade template:

<div>
    <button wire:click="$set('show', true)">
        Open
    </button>
    @if($show)
    <div>
        <div>{{ $content }}</div>
        <button wire:click="$set('show', false)">
            Close
        </button>
     </div>
    @endif
</div>

Of course instead of using $set() you can set up PHP open/close methods in the component and call them using wire:click.

After refreshing the page, it should works like this one:

Ok, and now, let's modify this component, to work with Alpine.js transitions. You need only to change the template part:

<div x-data>
    <button wire:click="$set('show', true)">
        Open
    </button>
    <div x-cloak
        x-show="$wire.show"
        x-transition.opacity.duration.500ms
    >
        <div>{{ $content }}</div>
        <button wire:click="$set('show', false)">
            Close
        </button>
    </div>
</div>

First, in the main component element (div in this example) add the x-data attribute. That will inform Alpine.js that this component contains some attributes to process.

Next remove @if statement and add x-show and x-transition attributes to the modal element. As a parameter for x-show you can use the value of $show property from component, using JS $wire proxy object.

Do not forget about the x-cloak attribute, and the CSS definition for it. x-cloak is the attribute that prevents hidden elements from blinking before the Alpine.js is loaded. By default, it hides the element by CSS, and then it's removed by Alpine.js on load.

Add to your CSS file:

[x-cloak] { display: none !important; }

The rest of the code is the same as before. Now our modal should work like this:

And that's it, we have working Alpine.js transitions in Livewire 😁

Read this before you start using the multistage builds for your docker images

Mateusz Cholewka — Mon, 22 Nov 2021 00:20:37 +0000

In my last post, I showed how to refactor the Dockerfile in your project. I didn't want to make the previous post too long, so I didn't touch the multistage builds. Here is the post where I will show you how to improve the Dockerfile even more, using multistage builds. Using stages in your builds is not hard to use, but there are some things where the below knowledge may help you 😃

Let's take the result of the last article:

FROM php:7.4.25-fpm
WORKDIR /app

COPY --from=composer:2.1.11 /usr/bin/composer /usr/bin/composer

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 && \
    apt purge -y --auto-remove

RUN docker-php-ext-install \
    exif \
    gd \
    intl \
    mbstring \
    mysqli \
    opcache \
    pdo_mysql \
    sockets

ENV COMPOSER_ALLOW_SUPERUSER 1

COPY composer.json .
COPY composer.lock .
RUN composer install --no-dev --no-scripts

COPY . .
RUN composer dumpautoload --optimize

Don't keep not necessary dependencies

Every dependency in your image is potentially the source of vulnerabilities, and you should keep them up to date. So, it's good practice to keep only a minimum of really required dependencies.

So let's try to get rid of the composer from our final image. Composer is required to install our backend dependencies, but it's not required in the runtime of our app. Actually, you shouldn't keep it in your final image, because every change, like for example composer update is removed after the container restart, so it's presence may be even confusing.

You can get rid of this dependency using multistage builds:

FROM composer:2.1.11 AS build
WORKDIR /app

COPY composer.json .
COPY composer.lock .
RUN composer install --no-dev --no-scripts --ignore-platform-reqs

COPY . .
RUN composer dumpautoload --optimize

FROM php:7.4.25-fpm
WORKDIR /app

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 && \
    apt purge -y --auto-remove

RUN docker-php-ext-install \
    exif \
    gd \
    intl \
    mbstring \
    mysqli \
    opcache \
    pdo_mysql \
    sockets

COPY --from=build /app /app

So, what happened here? As you can see, I've added a new FROM instruction before the PHP stage. In this way, we can add multiple stages to our docker image. The AS word is an alias for our stage, that we can use to refer to it. Referring to the stage is helpful if we need to copy something between stages, or use the previous stage as a base image for another stage.

In this case, I've added a build stage using a composer image, that will install all my dependencies and generate autoload files of my project. Now, when you start building this image, docker will create the build container first. When the build stage is finished, docker will start building the next stage. For the docker build process, those stages are recognized as different images, so the final image will keep only layers from the latest stage of build.

Also as you can see, I had added the --ignore-platform-reqs flag. This flag will allow you to install your dependencies even if you don't have the required PHP extensions installed. Otherwise, compose may stop installing process if some packages require extensions that do not exist in the base composer image.

Cache

As I've explained before, for docker build our stages are like different images. That makes a small problem for the building process that had confused me when I've tried the first time to build this image using cache from the previously built image. When I tried to build this image a few times in CI/CD adding --cache-from flag, the docker didn't use the cache. On my first time trying to get it working with using cache, I've spent a lot of time debugging the pipeline, I've even looked for problems in my runner. The problem wasn't in my runner, and what's more that's behavior is expected.

Because the final image keeps only the layers that belong to the target stage (default the latest target), the final image doesn't have layers of different stages than the target. That's obvious when you think about it because that helps you to make your images smaller. It may be confusing when you create the CI/CD pipeline when you probably don't have this thinking in your mind.

So how to resolve this problem? The goal is to keep all stage layers to use later as a cache. So we need to keep them during the build and then push them to our registry, to be able to get them on the next builds.

To achieve that, we can use the --target flag:

docker pull myimage:latest-build | true
docker pull myimage:latest | true

docker build . --target=build --cache-from=myimage:latest-build -t myimage:latest-build
docker build . --cache-from=myimage:latest-build --cache-from=myimage:latest -t myimage:latest

docker push myimage:latest-build
docker push myimage:latest

In this way, our application will always use the cache when building your images if it's available of course.

The first two lines will pull your previous builds images. The ... | true makes that commands always returning the 0 code for shell, even if the image doesn't exist that happens during the very first build. Another return code may stop your pipeline.

Image for local environment

The next thing that I had a problem with, was creating a single Dockerfile for local and running environments. The answer to my need was just to add another stage in Dockerfile.

I'm using the docker-compose to create the local development environment to work on my applications. In my case I had two Dockerfiles in my source code. The first of the files was the PHP with all extensions required to run the app. The second one was the runtime image that contains configured PHP same as the previous one, but with my application code installed in it. The PHP with extensions was the common part of both of those images. It's stupid to make the same changes in two separated Dockerfiles like I had to do it.

There is a way to solve this problem using stages:

FROM php:7.4.25-fpm AS base
WORKDIR /app

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 && \
    apt purge -y --auto-remove

RUN docker-php-ext-install \
    exif \
    gd \
    intl \
    mbstring \
    mysqli \
    opcache \
    pdo_mysql \
    sockets

FROM composer:2.1.11 AS build
WORKDIR /app

COPY composer.json .
COPY composer.lock .
RUN composer install --no-dev --no-scripts --ignore-platform-reqs

COPY . .
RUN composer dumpautoload --optimize

FROM base AS final

COPY --from=build /app /app

The first change is to move the part that prepares my base PHP image and name this target as base. The next part is the same as before. The last part is the target created from the base stage result, named final, that also copies our application from the build stage.

Now to build this image for your running environment like production, you need to build the final stage, which automatically builds all previous stages. For using that Dockerfile in your docker-compose you need to specify which stage you need to run your local environment:

services:
  web:
    #...
    port:
      - 8080:80

  php:
    build:
      context: .
      dockerfile: Dockerfile
      target: base
    ports:
      - "9000"
    volumes:
      - .:/app

And since now, we have had the same Dockerfile for the local and running environments.

So that's all what I want to show you in this article.

Have a nice day, and keep taking care of your Dockerfiles 😃

Originally posted on mateuszcholewka.com

Here are the Dockerfile tips you can apply to get your builds faster and safer

Mateusz Cholewka — Sat, 13 Nov 2021 02:58:15 +0000

Nowadays we are using docker a lot in web development. It's easy to use, great in scaling, and gives us an immutable environment for running your application from local development to deploy on production.
To get the best experience with docker you should apply some practices to get fast and light builds of your docker images.

In this article, I want to show you some of those practices based on this example:

FROM php:7-fpm
WORKDIR /app

COPY . .

ADD https://deb.nodesource.com/setup_12.x .
RUN bash setup_12.x

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin/ --filename=composer

RUN apt update && \
    apt install -y \
    curl \
    git \
    htop \
    libicu-dev \
    libgd-dev \
    mariadb-client \
    libonig-dev \
    vim \
    unzip \
    nodejs

RUN apt purge -y --auto-remove
RUN npm install -g yarn

RUN docker-php-ext-install \
    exif \
    gd \
    intl \
    mbstring \
    mysqli \
    opcache \
    pdo_mysql \
    sockets

ENV COMPOSER_ALLOW_SUPERUSER 1
RUN composer install

RUN yarn install

RUN yarn run build

Base your builds on specific image version

The first thing to change is the base image tag. As you can see in this Dockerfile the PHP7 is used, but the tag name is not precise enough. Here is the first improvement that we can make.

When you are using dependencies managers like yarn / composer, you probably use the lock files. Using them will keep exactly the same version of dependencies on every install. So why don't do it with all dependencies?

So the first dependency is the image tag we base our image on.

FROM php:7-fpm
...

We can change it to:

FROM php:7.4.25-fpm
...

That should save you for situations where your image doesn't work after a few months because of differences in newer PHP versions.

COPY your code last

Docker images are built from layers. Every layer can be cached, and this cache can be reused for the next builds if nothing has been changed. Docker can use cache only if all of the previous layers are loaded from cache too.

...
COPY . /app/
...

You should order your build steps by frequency of changes. Your application code is probably the thing that is changing most often, so you should put it as late as possible.

FROM php:7.4.25-fpm
WORKDIR /app
## remove COPY from here
...
## rest of commands
...
COPY . .
## final commands

Do not use ADD for remote dependencies

ADD instruction in Dockerfile allows you to copy files from remote locations by URLs. This feature also can unpack the zip archives which is great, but it has one problem. It doesn't cache your files.

ADD https://deb.nodesource.com/setup_12.x ./node_setup.bash
RUN bash node_setup.bash && \
    rm node_setup.bash

Ok, that's better.

The setup script file is undesirable, so it can be removed after the installation. But the problem is that the layers in Dockerfiles works like commits in git. When you put something to the repository using commit you can delete it with the next one, but because git works incrementally, both versions are kept in history, and the repository size increases.
To avoid this in docker images, you should create and remove undesirable files in the same instruction.

RUN curl -sS https://deb.nodesource.com/setup_12.x ./node_setup.bash && \
    bash node_setup.bash && \
    rm node_setup.bash

Better, but still not the best.

RUN curl -sS https://deb.nodesource.com/setup_12.x ./node_setup.bash | bash -

You can do all that things in one-line command using a pipe. In this example, the file content will be fetched and pushed directly to the bash that will execute it.

Using composer in Dockerfile

Here we have the composer installed in our container. It will be kept for all environments. It's not the best idea to keep it in final image, because it's not necessery and may add some vulnerabilities. There is a better option to use composer with multistage build that I want to describe in the next article.

...
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin/ --filename=composer
...

This line is ok it will be cached, and do not leave any garbage.
Maybe we should use the hash checking script that you can find in the official install script.
You can also use this trick:

...
COPY --from=composer:2.1.11 /usr/bin/composer /usr/bin/composer
...

That will copy the composer bin from the external official composer image.

Installing apt packages

Next, we have some packages installed using apt manager. Let's check if all of them are needed.

The git may be required for pulling packages or building some binaries from source. I can't see any reason to keep it. Let's remove it for now.

The htop may be useful for debugging, but not for the final image, we can install it when we really need it. Vim is useless too because you shouldn't make any changes in the working container. It's stateless, so your changes will disappear on a restart. Also mariadb-client is probably required only for development.

The rest of the packages may be required, but there is one more problem. The docker is using layers for caching. Every layer is built from dingle instruction. The cache is invalidated if the instruction or previous instruction had changed. So in this case if you do not change this instruction, the newer packages could be never installed, and they may vary depends on build environment.

If you add a specific version of every package, you will be sure that every image built from this Dockerfile has the same versions of packages, and the cache will be invalidated correctly.

You can do this by specifying the version after the = sign. To check which version you need to install, go to your current working container, or to the container that you build your image from, and check it with a list command:

$ apt list libonig-dev
Listing... Done
libonig-dev/stable,now 6.9.6-1.1 amd64 [installed]

In this example the currently working version is 5.5.9999+default, so let's check the rest and specify them.

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 \
    nodejs=12.22.7-deb-1nodesource1

RUN apt purge -y --auto-remove

Of course, you need to keep them up to date manually. It's good to check them frequently.

There is one more thing to do. After the install command, there is a commend that's cleaning your system after installing instruction. It's very good that that is here, but this is done in the separated instruction. As we remember, if we remove something on another layer, that will still exist in the previous layers of our final image. So let's do the cleaning in the same command. That should decrease your final image size.

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 \
    nodejs=12.22.7-deb-1nodesource1 && \
    apt purge -y --auto-remove

Composer dependencies

Let's get to the next lines. There is another one RUN instruction, that will install all of our composer dependencies. The first thing that is missed here is that we install all dependencies also with dev dependencies, that's are not necessary for the running environment. So let's put some flags here.

RUN composer install --optimize-autoloader --no-dev

Those flags will install all dependencies exclude dev, with autoloader optimization.

As you remember, we have to move the COPY instruction of our code from the beginning of this file as much as possible at the end. Here is the line where we need our project files. But do we need our entire codebase? How often do you change the dependencies in your project? For sure less often than your application code. So do we need to pull our dependencies every time when we change something in our code? Probably no 😃

So the only files that we need are the composer files there.

COPY composer.json .
COPY composer.lock .
RUN composer install --no-dev --no-scripts

Now the cache will work for our composer depenencies.

The code

Ok, it's time when we need our code because there are the build steps. Let's paste our COPY instruction from the beginning here.

COPY . .

And now, we need to generate the autoloader file with all our project files

RUN composer dumpautoload --optimize

Node dependencies

For a node there is the same situation as in composer. So first copy packages files and next install all dependencies.

RUN yarn install

RUN yarn run build

Do we need all dependencies or only non-dev dependencies? Maybe we don't need any node dependencies in the container because we use it only to build our frontend. So why not install everything and remove it after the build?

RUN yarn install && \
    yarn run build && \
    rm -rf node_modules && \
    yarn cache clean

And right now, we have no node dependencies that are not necessary. The problem here is that we cannot cache those dependencies. There are two ways to resolve this problem. The first one is the multistage build, but it's the topic for another article, which will be available soon. The second option will be to move entire frontend building to the nginx Dockerfile.

Values for now

Applying all those changes, let's check how much build process time we get.

Old image build 4m28s* 901MB

New image build 3m57s* 711MB

So we safe almost 200MB for final image. Our build time is not much better than before, but let's check how our cache is working now:

Old image with cache 4m35s*

New image with cache 25.1s*

So yea, the cache is working better for our new image.

Do you really need node for running PHP application?

In our example Dockerfile we are building our frontend app in the backend container, and then copy it to our frontend container:

FROM nginx:latest

COPY --from=backend /app/public /app/public

COPY docker/nginx/default.conf /etc/nginx/default.conf

Why to not build our app directly in the frontend image.

FROM nginx:1.21.4
WORKDIR /app

COPY docker/nginx/default.conf /etc/nginx/default.conf

RUN curl -sS https://deb.nodesource.com/setup_12.x ./node_setup.bash | bash -

RUN apt install nodejs=12.22.7-deb-1nodesource1 && \
    apt purge -y --auto-remove

COPY . .

RUN npm install -g yarn

RUN yarn install && \
    yarn run build && \
    rm -rf node_modules && \
    yarn cache clean

And our backend Dockerfile

FROM php:7.4.25-fpm
WORKDIR /app

COPY --from=composer:2.1.11 /usr/bin/composer /usr/bin/composer

RUN apt update && \
    apt install -y \
    libicu-dev=67.1-7 \
    libgd-dev=2.3.0-2 \
    libonig-dev=6.9.6-1.1 \
    unzip=6.0-26 && \
    apt purge -y --auto-remove

RUN docker-php-ext-install \
    exif \
    gd \
    intl \
    mbstring \
    mysqli \
    opcache \
    pdo_mysql \
    sockets

ENV COMPOSER_ALLOW_SUPERUSER 1

COPY composer.json .
COPY composer.lock .
RUN composer install --no-dev --no-scripts

COPY . .
RUN composer dumpautoload --optimize

So right now our backend image without cache is building in 3m8s* and with cache in 6s*, and it weight is 597MB.

The frontend image is building 57s* and it weight is 310MB.

You can build them in parallel, so the final time could be the maximum time for one of the images.

Multistage builds

All of those changes may be even better with using feature called multistage builds.
This topic should be available soon in the next article on my blog 😃

Edit: It's now available

*All the times that appear in this article, I got on my Mac with intel i5 and 16GB RAM environment.

Please remember about using non root user in your docker images.

Originally posted on mateuszcholewka.com

How to configure PHP logs for Docker

Mateusz Cholewka — Sun, 07 Nov 2021 14:06:06 +0000

If you are using docker and cloud services to run your application live, you should manage your logs.

The most common method to store them is to put them in the text file. It's the default configuration for most backend frameworks. This option is ok if you run your application locally or on the VPS server for test.

When you run your application in a production environment, you should choose a better option to manage your logs. Almost every cloud has a tool for rotating logs or if not, you can use for example Grafana Loki or ELK stack. Those solutions are better because give you interfaces to rotate and search your logs. Also, you have easy access to them, you no need to connect to your server to review them.

If you are using Docker containers, and you running your application in cloud services, often they will be automatically writing the logs of your containers to tools like AWS CloudWatch or GCloud Stackdriver.

But first, you need to redirect your log streams to the output of the Docker container to be able to use them.

Linux streams

Docker containers are running the Linux processes. In linux every running process has 3 streams, STDIN, STDOUT, STDERR. STDIN it's command input stream, that you can provide for ex. by your keyboard. STDOUT is the stream where the running command may print the output. STDERR is the standard error stream, but the name I think is a bit confusing, because it is basically intended for diagnostic output.

When you run the docker logs [container] command in your terminal, you will see the output of STDOUT and STDERR streams. So our goal is to redirect our logs to one of those streams.

Official docker documentation page

PHP-FPM

In PHP we are often running our application using the PHP-FPM (Process Manager). If you run your docker with FPM inside a docker container, and you run the docker logs command, you should see the output with processed requests, or errors.

So the PHP-FPM is already writing its output to STDOUT.

The PHP-FPM allow us to catch workers output and forward them to the STDOUT. To do that we need to make sure that the FPM is configured properly. You can create new config file, and push it for example to the /usr/local/etc/php-fpm.d/logging.conf file:

[global]
error_log = /proc/self/fd/2

[www]
access.log = /proc/self/fd/2

catch_workers_output = yes
decorate_workers_output = no

The error_log and access.log parameters are configuration of streams of logs output.

The catch_workers_output option is turning on the worker's output caching. The decorate_workers_output is the option that turns off the output decoration. If you leave this option turned on, FPM will decorate your application output like this:

[21-Mar-2016 14:10:02] WARNING: [pool www] child 12 said into stdout: "[your log line]"

Remember that decorate_workers_output option is available only for PHP 7.3.0 and higher.

If you are using official docker php-fpm image, this configuration is already set in the /usr/local/etc/php-fpm.d/docker.conf file, so you no need to do anything more 😎

PHP application configuration

Right now everything that will be put to the stdout from PHP workers will be shown in our docker logs. But when logs are forwarded to that stream in PHP?

To write something to STDIN on PHP level, we need to just write to the php://stdout stream.

In the simplest way you can do this like that:

<?php

file_put_contents('php://stdout', 'Hello world');

When you execute this code in php cli, you will get the Hello world text on the output.

But it's not the optimal way to push your logs to the STDOUT. Every modern framework should have a PSR-3 Logger. I think that the most popular now is the monolog, so I will show you how to configure it in Symfony, Laravel, and in pure usage.

Monolog

Monolog is great library to handle logs in your application. It's easy and elastic in configuration.

Basic monolog configuration

If you are using monolog in your project with manual configuration, you need to configure handler in this way:

(Modified documentation example)

<?php

use Monolog\Logger;
use Monolog\Handler\StreamHandler;

$log = new Logger('stdout');
$log->pushHandler(new StreamHandler('php://stdout', Logger::DEBUG));

$log->debug('Foo');

You just need to configure StreamHandler, to write to the php://stdout file.

Symfony

Symfony Kernel since the Flex was provided, is using minimalist PSR-3 logger, that logs everything to php://stderr by default.

In Symfony, monolog as other components is configured in YAML files. So the same configuration will look like this:

# config/packages/monolog.yaml
monolog:
    handlers:
        stdout:
            type: stream
            path: "php://stdout"
            level: debug

Laravel

Laravel use the arrays for configuration so the same thing will look like this:

# config/logging.php
<?php

use Monolog\Handler\StreamHandler;

return [
    'channels' =>
        'stdout' => [
            'driver' => 'monolog',
            'handler' => StreamHandler::class,
            'level' => env('LOG_LEVEL', 'debug'),
            'with' => [
                'stream' => 'php://stdout',
            ],
        ],
];

STDERR or STDOUT

In some articles on the internet, you can read that someone uses stderr, and someone uses stdout streams to write logs there. Right now I cannot fin any reasons to choose one of them which is better.

The only information that I found on this topic is that post.

I think that stderr is more popular in some examples, also Fabien Potencier set it as default in his minimalistic logger, so I think we can assume that this one is better.

Personally, I always used the stdout, so that's the reason why I use it in this post's examples. If I will find a great reason for using one of them over another I will update this post.

Originally posted on https://mateuszcholewka.com