Forem: run_as_root GmbH

Good-bye Docker, Hello Nix: Configuring a Magento 2 Development Environment with Rooter

Cristiano Pacheco — Tue, 20 Feb 2024 10:50:29 +0000

Introduction
Requirements
Creating a Magento 2 dev environment
- 1. Creating the project directory
- 2. Entering in the created directory
- 3. Creating the environment with rooter
- 4. Allowing direnv to execute
- 5. Cloning the Magento 2 repository
- 6. Starting the environment stack
- 7. Setting up the database
Rooter Appendix
- The .devenv directory
- Versioning
- Diving deeper into the most important files
  - .env
  - .envrc
  - devenv.nix
  - devenv.yaml
- Accessing the traefik dashboard
- Accessing the mailpit UI
- Rooter commands
- Xdebug

Introduction:

In 2023, I switched from Linux to macOS. I used Linux for years, mainly because it supported Docker natively. However, during a system update, I encountered a problem with the video driver that cost me an entire Saturday searching for solutions, tweaking files I didn't even know existed until finally, at the end of the day, I found a workaround to get the darn Nvidia driver working.

Come Monday morning, I shared my tech nightmare with a friend at work, who promptly asked:

"Why don't you switch to macOS?"

I replied: "Because it doesn't have native Docker support, and I don't want to use a virtual machine."

With a virtual machine, you have to allocate a good chunk of RAM, not to mention the slower disk processing. While disk speed might not matter much for many applications, it makes a huge difference for a Magento 2 environment.

"Why do you even need Docker?" he asked.

That sparked a quick conversation where he introduced me to Nix and devenv.

With these tools, it's possible to create a development environment that's entirely reproducible and native to macOS!

It was love at first sight! When I saw it in action, my mind was blown!

During that conversation, I first contacted one of the early versions of rooter.

Rooter is a powerful command-line tool to automate repetitive tasks in creating and maintaining development environments.

Shortly after, I bought a MacBook and started using rooter to manage my projects. Nowadays, I don't have Docker installed on my machine, and I couldn't be happier!

Requirements

You need macOS operating system and the softwares below installed:

Just follow the instructions on the rooter documentation.

Just giving a brief introduction about the these tools:

rooter is a CLI tool created By Matthias Walter. This tool will be your best friend during your development journey. It is similar to Warden or DDEV. It takes care of things like automatic DNS resolution using dnsmasq, managing project routes with Traefik, creating SSL certificates, and even includes handy commands for importing and exporting databases, plus a whole lot more.
nix has been around since 2003 and it's super reliable. It's like a toolbox with lots of useful tools inside, such as a programming language, an operating system, and a package manager. It helps you make software systems that are consistent and easy to recreate.
devenv is kind of like docker-compose, but way stronger. It's like being able to use a programming language to set up your environment, giving you even more control and flexibility.
direnv is a tool that figures out and loads all the stuff you need for your current project whenever you step into its folder in your terminal. We'll get to see how it works soon.

Creating a Magento 2 dev environment

So getting straight to the point, let's set up a development environment to run Magento 2.

At this point, I'm assuming you've got rooter and all the necessary tools it relies on installed and ready to roll. Let's dive in!

1. Creating the project directory

mkdir -p ~/Sites/magento2

2. Entering in the created directory

cd ~/Sites/magento2

3. Creating the environment with `rooter`

rooter env:create

Rooter will ask what type of environment:

Type 2 and enter for Magento 2

It generated 4 files:

.env
.envrc
devenv.nix
devenv.yaml

4. Allowing direnv to execute

You probably saw this message in your terminal:

You need to allow direnv to execute.

Run the command below:

direnv allow .

Now you'll see something like this in your terminal:

In this moment, your dev stack declared in the devenv.nix file will be downloaded and configured. This is the longest step. Wait until the installation process finishes.

You will see something like this when the process finishes:

Now, please take a look into this image:

If you've noticed, when I'm not in the project folder, the PHP version in the terminal is different, not 8.2. But as soon as I step into the project folder, direnv works its magic and loads the entire setup into the terminal. Pretty neat, isn't it?

5. Cloning the Magento 2 repository

composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition:2.4.6-p4 magento2-codebase

It'll clone the Magento 2 repository into a directory named magento2-codebase. Once it's done, just copy everything from the magento2-codebase folder to the current directory, and then you can delete the magento2-codebase directory with the command below:

cp -R magento2-codebase/* . && rm -Rf magento2-codebase

6. Starting the environment stack

I like to start the environment in the debug mode in the first attempt, because this shows detailed output about the initialization and in case there is an issue, it is visible right away:

rooter env:start --debug

Basically devenv will download all services upon first start. That could also be nginx, opensearch, rabbitmq etc.

After the first start I like to stop the environment (CMD + c) and start it again without the debug flag:

rooter env:start

The reason being that now it is running in the background rather than the foreground which it was with --debug flag passed.

To see what is the status of the environment, run the command below:

rooter env:status

It will show something like this:

As mentioned earlier, it uses process-compose in the background. There is a command to attach to the process-compose and get a control panel to check the various processed that got launched.

rooter env:process-compose

As you can see in the image below, you can easily switch between the different services and watch their logs update live.

7. Setting up the database

Run the command below to setup the database:

rooter magento2:db-install

Rooter will drop and recreate the database and by default run the commands below:

bin/magento setup:install
bin/magento setup:upgrade
bin/magento indexer:reindex

An admin user will be created with the username admin and password admin123.

If you pass the option --config-data-import it will run the command bin/magento config:data:import config/store dev/rooter

Note: The config:data:import is provided by the Magento2-ConfigImportExport module. You need to install it to use the --config-data-import option flag.

You can skip the reindex by passing the option --skip-reindex

Run the commands below to set some extra configs:

bin/magento config:set web/unsecure/base_url https://magento2.rooter.test/ && \
bin/magento config:set web/secure/base_url https://magento2.rooter.test/ && \
bin/magento config:set web/secure/offloader_header X-Forwarded-Proto && \
bin/magento config:set web/secure/use_in_frontend 1 && \
bin/magento config:set web/secure/use_in_adminhtml 1 && \
bin/magento config:set web/seo/use_rewrites 1 && \
bin/magento config:set catalog/search/enable_eav_indexer 1 && \
bin/magento config:set dev/static/sign 0 && \
bin/magento deploy:mode:set -s developer && \
bin/magento cache:clean

These are the Store front and admin URLs:

Tcharaaam!

What really blows me away is the speed. Running Magento natively on Mac makes everything super fast and smooth.

Rooter Appendix

The `.devenv` directory

The .devenv directory contains the data about the profile, nix garbage collector, the stack state, etc.

The .devenv/state directory is where some of the stack code lives, as for example, elasticsearch, mysql, php-fpm, nginx, rabbitmq, etc.

You don't need to touch these folders or the files on it.

Versioning

Which files and directories should you version or remove from versioning:

You don't need to version the files and folders below, so add them to your .gitignore file.

.devenv
.direnv
.env

You need to version the files below:

.devenv.flake.nix
.envrc
devenv.lock
devenv.nix
devenv.yaml

Note: None of this files is required by rooter. They are required from devenv and direnv.

Diving deeper into the most important files

`.env` file:

Contains all variable environments that is used in your project stack

ROOTER_ENV_TYPE defines the type of the environment, it can be:

magento1
magento2
php-plan
shopware6
symfony

DEVENV_HTTP_PORT defines the http port used from the Nginx webserver

DEVENV_HTTPS_PORT defines the https (SSL) port used from the Nginx webserver

DEVENV_DB_PORT defines the Mysql/MariaDB/Percona database port

Note: In the current version of rooter, only Mysql* database is supported.

Mysql* I mean any of the versions or forks of it, for instance MariaDB, Percona, etc.

DEVENV_MAIL_SMTP_PORT defines the mailpit SMTP port

DEVENV_MAIL_UI_PORT defines the mailpit user interface port

DEVENV_REDIS_PORT the name already explains it :)

DEVENV_AMQP_PORT defines the RabbitMQ port that will be used in your application

DEVENV_AMQP_MANAGEMENT_PORT defines the RabbitMQ admin panel

DEVENV_ELASTICSEARCH_PORT defines the ElasticSearch port that will be used from your application

DEVENV_ELASTICSEARCH_TCP_PORT the name already explains it :)

DEVENV_PROCESS_COMPOSE_PORT the name already explains it :)

We'll see more about the process compose later.

In case you noticed and are wondering, where the heck did the values of the ports defined in these variables come from?

Rooter has a mechanism that will get the available ports in your environment, taking into account the existing environments that were created by rooter.

It will assign ports using certain ranges, you can see how it works in this file, please look:

`.envrc` file:

This file is used by the direnv tool.

It will load all the declared stack to the current terminal context.

`devenv.nix` file:

This file is used by devenv, it uses the Nix language to declare the environment.

You can find more information about it here.

Let's get more details of the most important parts of this file:

{ pkgs, inputs, lib, config, ... }:

In Nix, when you see this line, it's like setting up a template for a function. It says that the function will at least get these four things: pkgs, inputs, lib, and config. But it's cool because it can also get more stuff using the ... thingy.

let
    rooterBin = if builtins.getEnv "ROOTER_BIN" != "" then builtins.getEnv "ROOTER_BIN" else "rooter";
    composerPhar = builtins.fetchurl{
        url = "https://github.com/composer/composer/releases/download/2.2.22/composer.phar";
        sha256 = "1lmibmdlk2rsrf4zr7xk4yi5rhlmmi8f2g8h2izb8x4sik600dbx";
    };
    magerun2Phar = builtins.fetchurl{
        url = "https://github.com/netz98/n98-magerun2/releases/download/7.2.0/n98-magerun2.phar";
        sha256 = "0z1dkxz69r9r9gf8xm458zysa51f1592iymcp478wjx87i6prvn3";
    };
in {

The let...in block in the provided devenv.nix file defines a scope for local definitions that can be used later in the expression. This construct allows for declaring variables and their values before using them in the main body of the Nix expression.

Here's a breakdown of the let block and its components:

rooterBin: Defined based on the ROOTER_BIN environment variable. If set, rooterBin uses its value; if not, it defaults to "rooter". Enables flexible configuration.
composerPhar/magerun2Phar: Downloads the Composer PHAR and n98-magerun2 PHAR file from GitHub using a URL and a sha256 hash for security.

    dotenv.enable = true;
    env = {
        PROJECT_NAME = "magento2";
        PROJECT_HOST = "magento2.rooter.test";

        NGINX_PKG_ROOT = pkgs.nginx;
        DEVENV_STATE_NGINX = "${config.env.DEVENV_STATE}/nginx";

        DEVENV_PHPFPM_SOCKET = "${config.env.DEVENV_STATE}/php-fpm.sock";

        DEVENV_DB_NAME = "app";
        DEVENV_DB_USER = "app";
        DEVENV_DB_PASS = "app";

        DEVENV_AMQP_USER = "guest";
        DEVENV_AMQP_PASS = "guest";
    };

The env block declares the environment variables.

The project's domain is determined by the PROJECT_HOST variable. So, you can access the project using the address: https://magento2.rooter.test.

If you need to change the PROJECT_HOST variable value, follow the instructions here.

It is possible to have custom subdomains, you can see more info about it here.

Just a heads up, the database will use as name, user, and password, all set to app by default. Feel free to tweak these settings whenever you need.

Similarly, for RabbitMQ, the default user and password are both guest. You're welcome to adjust these as necessary.

This is where we define and set up PHP. If you ever need to switch the PHP version, simply make the change in the highlighted section shown in the image.

The same for MariaDb version. If you need a different version just change it.

To know what are the supported versions, you can search in directly in the nix packages page, for example:

Note: please note that we're using the unstable channel, it is the default channel for devenv, because it has more packages and with newer versions on it. So when you'll search the package, don't forget to select the unstable channel.

Last we have RabbitMQ declaration, if your project does not use it, just remove it from the devenv.nix file. We'll maintain it :)

`devenv.yaml` file:

This file is used by devenv. It will serve to declare what is the source of the packages. You can have multiple inputs, using different channels, for example.

Accessing the `traefik` dashboard

To access the traefik dashboard, just type rooter traefik:dashboard

Accessing the `mailpit` UI

Mailpit is a fork of mailhog, It is newer and constantly receives updates.

It is per environment it will have always the URL with this pattern:

https://PROJECT_NAME-mail.rooter.test

In our case it will be:

http://magento2-mail.rooter.test

Note: The PROJECT_NAME variable is defined in the devenv.nix file

Rooter commands

Just a heads up, to explore the existing rooter commands, you can simply run the command rooter commands. This will give you a list of all available commands. Happy exploring!

Xdebug

The only thing action you need to configure Xdebug is to set this environment env to your terminal:

export XDEBUG_CONFIG="idekey=PHPSTORM-XDEBUG"

You can append it with the command below:

echo 'export XDEBUG_CONFIG="idekey=PHPSTORM-XDEBUG"' >> ~/.zshrc

Now you're able to debug your Magento 2 store on CLI or throughout web without any configuration.

On PHPStorm for instance, you just have to click on the "Bug" button to start listening to Xdebug connections:

If you have any specific scenarios you'd like guidance on, like setting up Magento with multiple websites or store views, feel free to leave your question in the comments below. I'll do my best to assist you!

I hope you found this helpful! 👋

Frontend Performance Optimization tips | Magento 2 | Adobe Commerce

Denys Sliepnov — Mon, 20 Nov 2023 17:41:57 +0000

Some backstory

Everybody knows that the performance is important. Many of us have already done many things to improve our performance.
Performance optimization is a complex process that requires a lot of time and effort.
It's a never-ending process, and you should always keep in mind that you can do better.
I'm also kept the performance optimization challenge and spent a lot of time improving the performance of the Magento 2 store.
You know what is a headache to speed up a big project.

The beginning of the journey

I started with the Google Lighthouse tool.
It's a great tool that helps you to find the most common problems with the performance.
I read a lot of articles about performance optimization and tried to apply the best practices.
I would suggest this article, combining most practices.
I don't want to repeat the same things, you are already familiar with them, I guess.
Most of them are well-known and described in the official documentation.
They are really helpful, but I still had many problems with the performance.

The next step - the deep dive into the performance by myself

After I applied all the best practices, I still had a lot of problems with the performance:

The DOM size was too big
The JS main-thread work was too long
The Largest Contentful Paint (LCP) score was too high

I can't remove required blocks from the page, I can't remove required JS code, I can't remove required images.
I decided to go deeper and started to investigate the performance by myself.
There are no resources how to decrease a DOM size, how to decrease a JS main-thread work, how to decrease a LCP score. Or I didn't find them, ha-ha.
So, let me introduce the insights that I found during my investigation.

Decrease the DOM size whenever it's possible

Don't create unnecessary DOM elements

Each DOM element has a cost. The more elements you have, the more memory is used.
The more memory is used, the slower the page is. So, don't create unnecessary DOM elements.

Try to avoid rendering elements that are not visible

If you have some visible component in the viewport that has some part of visible elements, like a button or input, and a tree of elements that are not visible, try to avoid rendering them. It makes sense to use some Visual Fasade. It's a component that renders only visible elements and hides the rest, rendering of the hidden elements will be started after the user interaction.

Example:

Your store uses the Search Autocomplete component. It renders a list of suggestions. The list is hidden by default.
When the user starts typing, the list is shown. The list is rendered only when the user starts typing.
It has a visible input element and hidden templates for suggestions.
Visible elements are rendered immediately, and the rest of the list is rendered after the user interaction.

If you have elements that are not visible, but they are rendered, try to prevent this. It's possible to render the specific part of the content after the user interaction. To do this, you can use the following steps:
- assign a hidden content to the JS variable:

// We use the $escaper variable to make sure that the JS content inside the child block is escaped properly.
<script>
    const hiddenContent = `<?= $escaper->escapeJs($block->getChildHtml('block_identifier')) ?>`;
</script>

use wrapper element to render the content:

<div id="wrapper"></div>

inject a few helper js functions:

// The 'init-external-scripts event' is triggered when any of the standard 
// interaction events (touchstart, mouseover, wheel, scroll, keydown) are fired, 
// and it also handles the cleanup, so the event is only fired once.
(events => {
    const dispatchUserInteractionEvent = () => {
        events.forEach(type => window.removeEventListener(type, dispatchUserInteractionEvent))
        window.dispatchEvent(new Event('init-external-scripts'))
    };
    events.forEach(type => window.addEventListener(type, dispatchUserInteractionEvent, {once: true, passive: true}))
})(['touchstart', 'mouseover', 'wheel', 'scroll', 'keydown'])

// This function renders the content inside the wrapper element. 
// And it also handles the script execution when the content is rendered inside the wrapper element.
function setInnerHTML(elm, html) {
    elm.innerHTML = html;

    Array.from(elm.getElementsByTagName("script"))
        .forEach(oldScriptEl => {
            const newScriptEl = document.createElement("script");

                Array.from(oldScriptEl.attributes).forEach(attr => {
                newScriptEl.setAttribute(attr.name, attr.value)
            });

            const scriptText = document.createTextNode(oldScriptEl.innerHTML);
            newScriptEl.appendChild(scriptText);

            oldScriptEl.parentNode.replaceChild(newScriptEl, oldScriptEl);
        });
    }

render the content after the user interaction:

window.addEventListener('init-external-scripts', () => {
    const wrapper = document.getElementById('wrapper');
    setInnerHTML(wrapper, hiddenContent);
});

Full example:


<div id="wrapper"></div>

<script>
    (events => {
        const dispatchUserInteractionEvent = () => {
            events.forEach(type => window.removeEventListener(type, dispatchUserInteractionEvent))
            window.dispatchEvent(new Event('init-external-scripts'))
        };
        events.forEach(type => window.addEventListener(type, dispatchUserInteractionEvent, {once: true, passive: true}))
    })(['touchstart', 'mouseover', 'wheel', 'scroll', 'keydown'])

    function setInnerHTML(elm, html) {
        elm.innerHTML = html;

        Array.from(elm.getElementsByTagName("script"))
            .forEach(oldScriptEl => {
                const newScriptEl = document.createElement("script");

                Array.from(oldScriptEl.attributes).forEach(attr => {
                    newScriptEl.setAttribute(attr.name, attr.value)
                });

                const scriptText = document.createTextNode(oldScriptEl.innerHTML);
                newScriptEl.appendChild(scriptText);

                oldScriptEl.parentNode.replaceChild(newScriptEl, oldScriptEl);
            });
    }

    const hiddenContent = `<?= $escaper->escapeJs($block->getChildHtml('block_identifier')) ?>`;

    window.addEventListener('init-external-scripts', () => {
        const wrapper = document.getElementById('wrapper');
        setInnerHTML(wrapper, hiddenContent);
    });
</script>

This approach minimizes the DOM size and JS main-thread work (script execution).
Also, if hidden content contains some images, the loading of the images will be started after the user interaction.

Defer non-critical JS initialization and execution

If you have some custom JS code that is not required for the initial page load, try to defer its execution.
Imagine that you have some custom JS code that pushes page data to external system.

Example:

function pushPageDataToExternalSystem() {
    // Get page data
    const pageData = {
        url: window.location.href,
        title: document.title,
        // ...
    };

    // Push page data to external system
    externalSystem.push(pageData);

    // Some other code
}

// Some other code
pushPageDataToExternalSystem();

In this case, you can defer the initialization and execution of the pushPageDataToExternalSystem() function:

insert your custom JS code into PHP variable
assign the variable to the JS variable using the $escaper->escapeJs() function
use the init-external-scripts event to initialize and execute the pushPageDataToExternalSystem() function

<?php $scriptString = <<<script
    <script>
        function pushPageDataToExternalSystem() {
            // Get page data
            const pageData = {
                url: window.location.href,
                title: document.title,
                // ...
            };

            // Push page data to external system
            externalSystem.push(pageData);

            // Some other code
        }

        pushPageDataToExternalSystem();
    </script>
script;
?>

<div id="uniq_id">
    <script>
        (events => {
            const dispatchUserInteractionEvent = () => {
                events.forEach(type => window.removeEventListener(type, dispatchUserInteractionEvent))
                window.dispatchEvent(new Event('init-external-scripts'))
            };
            events.forEach(type => window.addEventListener(type, dispatchUserInteractionEvent, {once: true, passive: true}))
        })(['touchstart', 'mouseover', 'wheel', 'scroll', 'keydown'])

        window.addEventListener('init-external-scripts', () => {
            const html = `<?= $escaper->escapeJs($scriptString) ?>`;
            const el = document.getElementById('uniq_id');
            setInnerHTML(el, html);
        }, { once: true, passive: true });
    </script>
</div>

The function pushPageDataToExternalSystem() will be executed after the user interaction, which contributes to minimizing the DOM size and JS main-thread work.
Obviously, this function is not critical for the performance, but when you apply this approach for tons of JS code, the profit will surprise you.

Largest Contentful Paint (LCP) optimization

The common problem with LCP is the loading of the images. In most cases, it's some Banner image on the Home page or Product image on the Product page.
You can find a lot of posts about LCP optimization:

etc... , and it's really helpful.

What to do if you have already done all of this and still have a problem with LCP?

Here I would like to share with you one tip that is not so common, but beneficial.
You should load the image ASAP. It means that you should load the image with the highest priority.

LCP scores

LCP phase	Description
Time to first byte (TTFB)	The time from when the user initiates loading the page until when the browser receives the first byte of the HTML document response.
Load delay	The delta between TTFB and when the browser starts loading the LCP resource.
Load time	The time it takes to load the LCP resource itself.
Render delay	The delta between when the LCP resource finishes loading until the LCP element is fully rendered.

In most cases, the LCP score is high because of the long load delay and load time.

These two metrics are more important than TTFB and render delay.
So, you should focus on them. To determine the LCP resource, you can use the Chrome DevTools. Open the Network tab and check the order of the LCP resource loading.
The LCP resource should be loaded as soon as possible. If you have some other resources loaded before the LCP resource, you should change the loading order.
As faster LCP resources will be loaded, as better load delay and load time metrics will be.

I can't provide you with the exact solution because it depends on your project, but I hope this tip will help you improve the LCP score.
It's better than just a solution, I give you an idea. Do your magic!

Conclusion

There are a lot of things that you can do to improve your performance. I hope this article will help you improve your store's performance.
I will be glad to hear your feedback and ideas. Feel free to contact me if you have any questions.

Should you upgrade your Magento store at all? Developer view

Vladyslav Podorozhnyi 🇺🇦 🌻 — Fri, 27 Oct 2023 15:45:13 +0000

Hello everyone. 👋

This article will be a bit less technical and more about the value of the upgrade for store owners. However, all of my thoughts would be described from the view of the developer that design, implement, and maintain such stores - so expect at least some level of technical expertise and judgments here.

Obviously, we would speak in contexts of Magento 2 / Adobe Commerce / Mage-OS - but I believe that findings and ideas described here would be useful for Shopware too, as well as for any other e-commerce system.

So - lets move on and have a small walkthrough of this topic. 🚶‍♀️🚶‍♂️

What does any upgrade bring us:

new features
new possibilities for developers
bugfixes
security fixes
agility - possibility to upgrade faster to all future New versions

Let’s check each point and ask next questions:

What value does it bring to the project?
Price?
Do we need this value?

New features

What value does it bring to the project?

New business possibilities for store owners. Or improved UX in case it is an upgrade of Hyvä.

Or improved code execution performance ( yes - I think it is a feature 😄 ).

Price?

Is it free? - YES. Then we need it! [ actually, it is not free - see my explanation below ]

That is what the client says 😄. For him, it looks like a FREE fancy addition to all the hard work that is going to be done.

Not all features from the upgrade can be easily adapted if we take into account all customization project already has - the minimum you should consider is styling, maximum - is rebuilding business logic from top to bottom.

Do we need this value?

If the store owner needs a new feature - the store owner would ask for it explicitly. 😄

Answer: more NO than yes. At least in scope of the upgrade.

New Possibilities for Developers

What value does it bring to the project?

We have some variations here, but here is a compilation of the benefits it gives us:

build code faster
Make code less shaky & buggy

Price?

Is it free? - YES. Then we need it!

That is what the developer says 😄 (déjà vu). And again - it looks free, but there could be a clown that eats you alive. 🤡

Do we need this value?

Answer: It depends - but more NO than yes. Unless it is a painless upgrade from php 5 to php 7 - then totally yes, yep? 😄
Don't get me wrong - I love new technologies and new versions of old ones, and I wish to work on the edge of tech! However, is it worth it here and now? 🤷

I would plan tech stack upgrade not as an additional point to the system/extension upgrade - but as quite a big chunk of work. So plan carefully and do not think it will be magically transferred without any effort.

Bugfixes

What value does it bring to the project?

less bugs 😅

Price?

Is it a pain in the ass? - sure.

Is it more expensive than just fixing it by yourself? - usually, no.

Do we need this value?

Answer: Yes in 99% of the cases. Other 1% - Developer decides.

Upgrade is a tool to achieve a goal - in this case to fix the bug/bugs. But monkeypatch is also a tool 🙉.

Choose wisely!

Security Fixes

What value does it bring to the project?

Shop won’t be spammed / hacked / Arbitrary code execution won't happen.

e.g. - no disappointed buyers attack you 😸

Price?

Well - security is priceless 😎

However, sometimes it is cheaper to just transfer security fix changes - if you know what and where to transfer, which is not usually the case.
Let’s say it this way - I would not risk the security of my client to save another couple of hours on an upgrade.

Do we need this value?

Yes - we need it. Always.

Answer: Yes.

Agility - you can upgrade faster to all future NEW versions

What value does it bring to the project?

Constant upgrades -> minimise time spent on one particular upgrade.

e.g. - clean your home once a week and it takes you half an hour, do it once a year - you would cry for help and won’t finish in a days 🙉

Price?

Do upgrades constantly - usually, once a quarter you have a patch upgrade, and some small p1, p2, p… versions during the quarter. Also, once 1,5 years, you do the bigger upgrade (Minor by semantic ver and somewhat Major based on Magento lifecycle ).
In return, you will get a healthy webshop with a high level of agility to get new versions. Recent security patches once again reminded us how important it is to be on the edge (see APSB23-50 ).

Do we need this value?

I afraid so.

Answer: Yes.

Sum Up

We need upgrades. As a minimum for security and agility needs. But also to keep the shop healthy and without technical debt.

It also gets in use for other points from our list and can be useful or harmful for different points.

Upgrade, as an ability, needs to be used mindfully but not blindly 🙈 Build correct expectations from your upgrade, estimate, and plan your work accordingly.
Last but not least - explain why upgrades are needed and what is going to be delivered with this specific release.

Hope my a bit structured thought flow will gonna help you to understand upgrades, what we need to take out of them, and how to communicate it right.

Chao Kako ☕

Cron Jobs in Magento 2: How to Adjust Schedules and Cron Groups Seamlessly

Vladyslav Podorozhnyi 🇺🇦 🌻 — Fri, 22 Sep 2023 18:53:56 +0000

Introduction

Hello!

In today's article, I'll cover a topic that's not too tricky, but can be a bit annoying without some useful tips. Specifically, we'll dive into Magento 2 ( Mage-OS / Adobe Commerce ) to discuss cron jobs, their configurations, and the process of rescheduling them and altering their execution group.

You might wonder, why bother changing the schedule or group of a cron job? Let's find out.

Why Adjust the Cron Schedule?

Occasionally, a cron job might execute too frequently, leading to server load issues or even deadlocks. On the other hand, if it doesn't run often enough, you could be left waiting for emails that take an hour to send. This scenario becomes particularly tricky when these cron jobs are associated with third-party extensions.

Let's imagine we are using 3rd party Feed and Mailing extensions, and we are not OK with their cron jobs schedule.

How can you modify the execution patterns of these cron jobs without tampering with the third-party code directly? By implementing changes in your own codebase – your module. Let's explore this next.

Rescheduling Cron Jobs

Let's have a look at the cron job of the Feed extension feed_generation.

Here is its definition:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Cron:etc/crontab.xsd">
    <group id="default">
      ...
        <job name="feed_generation" instance="\Feed\Cron\FeedGenerator" method="execute">
            <schedule>* * * * *</schedule>
        </job>
      ...
    </group>
</config>

Executing every minute, this job can impose a significant server load. We certainly don't need a new feed every 60 seconds!

To modify its frequency, we'll craft a module, Devto_ChangeFeedSchedule, and incorporate an etc/config.xml file within.

So, the structure would look like this:

Devto_ChangeFeedSchedule/
│
├── etc/
│   ├── config.xml      # our config file
│   └── module.xml      
│
├── registration.php    
│
└── composer.json

The etc/config.xml should contain:

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Store:etc/config.xsd">
    <default>
        <crontab>
            <default> <!-- cron group -->
                <jobs>
                    <feed_generation> <!-- job name -->
                        <schedule>
                            <cron_expr>0 * * * *</cron_expr> <!-- new cron execution schedule -->
                        </schedule>
                    </feed_generation>
                </jobs>
            </default>
        </crontab>
    </default>
</config>

And last but not least - add sequence into etc/module.xml to declare dependency over Feed module and configs apply order:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="Devto_ChangeFeedSchedule" >
        <sequence>
            <!-- makes Devto_ChangeFeedSchedule configs be applied after Feed extension configs -->
            <module name="Vendor_FeedExtesnion"/>
        </sequence>
    </module>
</config>

Once the module Devto_ChangeFeedSchedule is activated and the cache cleared, your new job schedule will take effect - run every hour but not a minute.

But what about cron groups? Let's delve into that next.

The Need to Change Cron Group

Imagine we're still working with the Feed and Mailing third-party extensions. These extensions utilize distinct cron groups, each operating as an isolated process. Thanks to this, cron groups can run concurrently/parallely, using individual processes.

Now, suppose these two cron jobs conflict, resulting in data inconsistencies in the Feed and Emails. Worse yet, deadlocks can crop up. You might ask, "Why the deadlocks?"

Well, imagine these extensions read & write the same table, say sales_order_item. When they run simultaneously, the stage is set for potential deadlocks. 🤷‍♂️

If only these jobs were executed sequentially! By relocating them to a shared group, you can ensure they no longer run in parallel.

Altering the Cron Job's Group

A straightforward (though not perfect) solution would be to transfer the Mailing extension's cron job to the default group, where the Feed extension's job resides. Here's the Mailing cron job definition:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Cron:etc/crontab.xsd">
    <group id="mailing_extension"> <!-- mailing cron group -->
      ...
        <job name="send_mails_from_mailing_extension" instance="\Mailing\Cron\SendMailsCron" method="execute">
            <schedule>*/20 * * * *</schedule>
        </job>
      ...
    </group>
</config>

However, a small, non-intuitive trick is essential here: Simply changing the group by redefining crontab.xml won't work. Instead, you need to craft a new cron job while deactivating the old one. This step is necessary since crontab.xml configurations (especially groups) can't be overridden, only augmented.

For instance, merely defining the send_mails_from_mailing_extension cron job in the default group would make it run in both the default and mailing_extension groups.

To disable send_mails_from_mailing_extension, an easy trick is to schedule it for February 30th—a date that doesn't exist!

nerd fact 🤓: actually February 30 happened in Sweden in 1712

Following our earlier approach, we'd add the following content to the etc/config.xml in our module, Devto_ChangeMailingJobGroup:

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Store:etc/config.xsd">
    <default>
        <crontab>
            <default>
                <jobs>
                    <!-- Disable cron execution by scheduling it to Feb 30th. -->
                    <send_mails_from_mailing_extension>
                        <schedule>
                            <cron_expr>0 0 30 2 *</cron_expr>
                        </schedule>
                    </send_mails_from_mailing_extension>
                </jobs>
            </default>
        </crontab>
    </default>
</config>

Next, define a NEW cron job in the default group:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Cron:etc/crontab.xsd">
    <group id="default"> <!-- default cron group -->
      ...
        <!-- added devto prefix to cron name -->
        <job name="devto_send_mails_from_mailing_extension" instance="\Mailing\Cron\SendMailsCron" method="execute">
            <schedule>*/20 * * * *</schedule>
        </job>
      ...
    </group>
</config>

And do not forget to add Mailing module sequence into etc/module.xml:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="Devto_ChangeMailingJobGroup" >
        <sequence>
            <!-- makes Devto_ChangeMailingJobGroup configs be applied after Mailing extension configs -->
            <module name="Vendor_MailingExtesnion"/>
        </sequence>
    </module>
</config>

After all changes, the module structure should look like this:

Devto_ChangeMailingJobGroup/
│
├── etc/
│   ├── config.xml      # our config file
|   ├── crontab.xml     # our crontab definition file  
│   └── module.xml      
│
├── registration.php    
│
└── composer.json

Once Devto_ChangeMailingJobGroup is active and the cache refreshed, the send_mails_from_mailing_extension job will be disabled. Meanwhile, its counterpart, devto_send_mails_from_mailing_extension, will operate in the default cron group alongside the Feed cron job.

A Word of Caution on the Default Group

It's vital to recognize that Magento's native cron jobs already densely populate the default cron group. Overloading it might cause "traffic jams" if:

Jobs have lengthy runtimes: If your tasks take too long to finish, they could end up waiting in line, slowing things down.
Overfrequent Scheduling: Setting jobs to run too often can result in backlogs and potential overlaps.
Overpopulation: Filling the group with too many tasks, even if they are quick, might overwhelm the system.

So, while the default group is convenient, treat it like a city's main highway during rush hour. It can handle a lot, but there's a limit before things get clogged. If you have too many heavy tasks or they're scheduled too frequently, you'll likely end up with delays.

In short, be mindful of the workload you're placing on the default group and consider creating your own cron groups to avoid conflict of processes.

Conclusion

And that's the lowdown on Magento's cron jobs, how to shuffle their schedules, and adjust their groups without diving into third-party code directly. With the knowledge you've gathered here, tweaking those pesky cron jobs should be much easier 😊

Thank you for reading!

Preventing Transaction Loss: Unleashing the Power of Resilient Transactions with RabbitMQ dead-letter Exchanges in Magento 2

Cristiano Pacheco — Mon, 28 Aug 2023 10:57:40 +0000

This post will guide Magento 2 developers through implementing a message queue retry mechanism using the RabbitMQ dead-letter exchange feature.

To assist us in this endeavor, we will leverage the Message Queue Retry module by run-as-root.

What is RabbitMQ?
What is a dead-letter exchange?
Examples of transaction loss scenarios
Practical case scenario
Crafting an effective retry strategy for queue messages
Practical case scenario with queue retry

What is RabbitMQ?

RabbitMQ is a widely embraced open-source Message Queue software, facilitating seamless interaction and scalability across numerous applications. As a proficient messaging broker, RabbitMQ furnishes a unified platform catering to Asynchronous messaging needs. Within this platform, applications gain the capability to transmit and receive messages securely, with RabbitMQ diligently ensuring the sanctuary of your messages until they are consumed.

Since version 2.1.0, Magento 2 has offered native RabbitMQ support, enabling effortless queue configuration through XML settings. For further details, you can explore additional information here

What is a dead-letter exchange?

This feature functions as a safety net within the messaging architecture, intended to manage messages that encounter unsuccessful processing by the consumer.

The dead-letter exchange operates as a mechanism for directing these problematic messages to a designated queue referred to as a dead-letter queue. This facilitates developers in isolating and inspecting troublesome messages without interrupting the primary processing flow, enabling them to analyze these messages, identify the underlying reasons for processing failures, and implement essential adjustments to ensure smooth operation.

It's particularly recommended for scenarios where upholding message integrity, diagnosing processing errors, and averting message loss is paramount.

Examples of transaction loss scenarios:

Exporting the customers' data to a CRM
- Possible issue: The CRM credentials were modified, but the changes were not reflected in the Magento integration settings.
- Consequence: This mismatch in credentials will result in failed API calls due to the usage of incorrect or invalid authentication information.
Exporting the orders to an ERP
- Possible issue: During a marketing campaign, the store's throughput surged by 200%, leading to a substantial rise in incoming orders; however, the ERP system could only accommodate its standard order import capacity.
- Consequence: Only a fraction of the increased order influx will be successfully exported to the ERP system, potentially causing delays and incomplete order processing.
Scheduled price uplift

Imagine having the capability to schedule a comprehensive price uplift by 10% for the entire catalog of your store.
- Possible issue: Deadlock encountered while saving product and price information.
- Consequence: The product prices may not be accurate.

Practical case scenario

A typical case in e-commerce is exporting orders to an ERP or OMS. We will implement an asynchronous order export architecture, where for each placed order, a message is created with the order id and sent to a queue. So that a consumer can later read this message and export it to the external system.

To simplify this example, let's declare the topic's name, exchange, queue, and bindings with the same name: erp_order_export.

To handle this messaging, we'll utilize RabbitMQ as the broker.

You can find the complete code for the module on this repository.

We have to create four files to declare our queue.

etc/communication.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
    <topic name="erp_order_export" request="CristianoPacheco\OrderExport\Api\Data\OrderExportDataInterface"/>
</config>

etc/queue_consumer.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
    <consumer name="erp_order_export" queue="erp_order_export" connection="amqp"
              handler="CristianoPacheco\OrderExport\Queue\Consumer\ErpOrderExportConsumer::execute"
    />
</config>

etc/queue_publisher.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/publisher.xsd">
    <publisher topic="erp_order_export">
        <connection name="amqp" exchange="erp_order_export"/>
    </publisher>
</config>

etc/queue_topology.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
    <exchange name="erp_order_export" connection="amqp" type="topic">
        <binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export"/>
    </exchange>
</config>

Run the command below in order to Magento send the command to create the exchange, queue and bindings on RabbitMQ:

bin/magento setup:upgrade

It will look like this on RabbitMQ admin:

Visually, our configs will generate something like this:

Note: It would make more sense to configure a direct exchange. However, up to the current version of Magento (2.4.6), only topic exchanges are supported.

Now that the queue is set up, we must send the order ID to the erp_order_export queue every time a successful order is placed.

We can easily do this by listening to the event sales_model_service_quote_submit_success via an observer:

etc/events.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Event/etc/events.xsd">
    <event name="sales_model_service_quote_submit_success">
        <observer name="erp_export_order" instance="CristianoPacheco\OrderExport\Observer\SendOrderToQueueObserver"/>
    </event>
</config>

I won't post here the Observer code or the entire implementation. The complete working code is in the repository. If you want to simulate the scenarios, please install the module.

Our consumer will use a service to perform our "fake" order export. To simulate an error, our service will always throw an exception:

<?php

declare(strict_types=1);

namespace CristianoPacheco\OrderExport\Service;

use CristianoPacheco\OrderExport\Api\Data\OrderExportDataInterface;
use Exception;

class ExportOrderToErpService
{
    public function execute(OrderExportDataInterface $orderExportData): void
    {
        $orderId = $orderExportData->getOrderId();
        throw new Exception("The order $orderId id is invalid");
    }
}

With the current configuration, orders will enter the erp_order_export queue and can be "exported" during the execution of our consumer.

To run our consumer and export only one order per execution, run the following command:

bin/magento queue:consumer:start erp_order_export --single-thread --max-messages=1

What will happen at the end of the execution?

The service will throw an exception, and it will be handled here:

vendor/magento/framework-message-queue/Consumer.php:261

As you can see, the message will be rejected, and the requeue flag will have false as the value. In other words, the message will be removed from the queue, and the order will not be exported to the ERP.

Well done! We managed to simulate a transaction loss.

In day-to-day scenarios, probably (I hope so), exceptions are handled at the service or the consumer's execution entry point, and the message would be sent back to the original queue or even taken manually in cases of error, such as sending an alert, or persisting the message in a database, etc.

What if there was a way to automatically identify without manual work when there are errors in message processing, allowing the possibility to optimize and standardize this process?

Yes, there is. We can delegate this hard work to RabbitMQ via a dead-letter exchange.

Crafting an Effective Retry Strategy for Queue Messages

So, how do we ensure that our messages reach their destination in the face of these intermittent failures? This is where the concept of a retry strategy comes into play. A retry strategy outlines how we handle failed operations by making subsequent attempts at sending the same message. When exporting an order to ERP, adopting a well-defined retry strategy can significantly enhance the reliability of our application.

For this specific case, three attempts are reasonable.

So let's define our requirements, it should be possible to:

Reprocess the messages
Set processing retry limit
Set an interval in minutes for each retry attempt
Find out what error was generated during message processing
See what the message payload is
If the limit of retries is reached, we must provide a way to manage message processing manually
- Resend the message to the queue
- Discard the message

Steps to achieve our goal:

We will add a second queue (erp_order_export_delay) in RabbitMQ to store the message for a specific lifetime. This queue will not have a consumer. It will only serve to "delay" processing the message in the source queue (erp_order_export).
When the time to live is reached, RabbitMQ will automatically send the message to the source queue (erp_order_export).
If the retry limit is reached, the message will be stored in a database table for manual management.
- Maybe the ERP server is down, or some order data is incorrect, so we have to reprocess the messages later.

Steps 1 and 2 can be configured in the queue declaration, and RabbitMQ natively supports it.

Step 3 is a capability provided by the run as root Message Queue Retry module.

Practical case scenario with queue retry

Firstly we need to install the message queue retry module and enable the behavior to persist the messages on the database after the retry limit is reached.

No we have to configure our dead-letter exchange. The complete implementation of code below can be found here.

We'll use the exp_order_export_delay as the name for the topic, exchange, queue and bindings.

Declaring the topic:

etc/communication.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
    <topic name="erp_order_export" request="CristianoPacheco\OrderExport\Api\Data\OrderExportDataInterface"/>
    <topic name="erp_order_export_delay" request="CristianoPacheco\OrderExport\Api\Data\OrderExportDataInterface"/>
</config>

Declaring the retry limit:

etc/queue_retry.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:RunAsRoot:module:RunAsRoot_MessageQueueRetry:/etc/queue_retry.xsd">
    <topic name="erp_order_export" retryLimit="3"/>
</config>

Declaring the dead-letter exchange:

etc/queue_topology.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
    <exchange name="erp_order_export" connection="amqp" type="topic">
        <binding id="erp_order_export" topic="erp_order_export" destinationType="queue" destination="erp_order_export">
            <arguments>
                <argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export_delay</argument>
                <argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export_delay</argument>
            </arguments>
        </binding>
    </exchange>
    <exchange name="erp_order_export_delay" connection="amqp" type="topic">
        <binding id="erp_order_export_delay" topic="erp_order_export_delay" destinationType="queue"
                 destination="erp_order_export_delay">
            <arguments>
                <argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export</argument>
                <argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export</argument>
                <argument name="x-message-ttl" xsi:type="number">60000</argument><!-- 1 minute in  milliseconds -->
            </arguments>
        </binding>
    </exchange>
</config>

Let's take a closer look at each change in the file above.

1- the arguments x-dead-letter-exchange and x-dead-letter-routing-key will tell RabbitMQ to automatically send the messages to the erp_order_export_delay exchange in case the message is negatively acknowledged (rejected or nack with requeue parameter set to false).

...
<arguments>
    <argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export_delay</argument>
    <argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export_delay</argument>
</arguments>
...

2- We declared a the erp_order_export_delay exchange and bound it to the erp_order_export_delay queue

...
<exchange name="erp_order_export_delay" connection="amqp" type="topic">
    <binding id="erp_order_export_delay" topic="erp_order_export_delay" destinationType="queue" destination="erp_order_export_delay">
...

3- The arguments x-dead-letter-exchange and x-dead-letter-routing-key defines the return path for the message and the x-message-ttl has an integer number in milliseconds that defines when the message will return to the exchanged defined in the previous two arguments.

Remembering that this queue does not have a consumer.

...
<arguments>
   <argument name="x-dead-letter-exchange" xsi:type="string">erp_order_export</argument>
   <argument name="x-dead-letter-routing-key" xsi:type="string">erp_order_export</argument>
   <argument name="x-message-ttl" xsi:type="number">60000</argument><!-- 1 minute in  milliseconds -->
</arguments>
...

Visually, our new configs will generate something like this:

Note: We always send the messages to an exchange and the exchange will forward the messages to one or more queues. It is the same with dead-letter exchanges.

Now we have to manually delete the erp_order_export exchange and queue in RabbitMQ, because it won't be updated. You can do it in the RabbitMQ admin panel or via CLI.

After you did it, you have to run the command below to reflect our changes in RabbitMQ:

bin/magento setup:upgrade

Now if we navigate again in the RabbitMQ panel we should see:

Exchanges:

If we open our exchanges an expand the bindings, is it possible to see the arguments we have declared before:

Queues:

Did you notice that queues now have a few more features?

A homework for you, hover the mouse over each feature and wait a few seconds, a caption will be displayed informing what it is about. Also access each queue and expand the bindings section, see the values contained there.

Now that we finished our RabbitMQ configuration, let's place a new order and start the erp_order_export consumer to see what will occur.

After place an order a new massage is in the erp_order_export queue:

Now let's start our consumer:

bin/magento queue:consumer:start erp_order_export --single-thread --max-messages=1

Now if we take a look at the queues we'll see:

During message consumption, our service threw an exception, simulating a problem and the message was rejected. RabbitMQ sent the message to the erp_order_expor_delay queue.

After one minute the message was automatically sent to the erp_order_export queue.

Let's inspect the message headers:

The x-death header has interesting information about the retries.

Relevant attribute for us to analyse are:

count - it is the retry number
reason - In our case, can be rejected or expired
time - The timestamp of the occurrence
exchange, queue, routing-keys - Where the message was at the time of occurrence

So, what that image says is:

Reading from bottom to top

First occurrence

The message was in the erp_order_export queue and while processing it, it was rejected on Saturday, August 26, 2023 9:12:33 PM GMT-03:00

Second occurrence

The message was in the erp_order_export_delay queue, and it was rejected on Saturday, August 26, 2023 9:12:53 PM GMT-03:00

It has 20 seconds difference because I configured the TTL for 20 seconds :)

This is the message headers after the second retry:

After the third retry:

What will occur in the next consumer execution?

So, the run-as-root module intercepted the message processing failure, verified that the message has reached the retry limit and then persisted the message in the Magento database.

Now it is possible to manage the message through the admin panel, in the path:

Run as Root > Manage Messages

Now to can decide if you'll discard the message or send it back to the erp_order_export queue.

In summary, while multiple approaches exist to tackle this issue, my aim has been to show a straightforward method for to implement a retry mechanism.

That's it for today, happy coding!

Adobe Commerce CLI commands - or how to create a problem out of nothing

Vladyslav Podorozhnyi 🇺🇦 🌻 — Tue, 15 Aug 2023 19:45:38 +0000

Intro
The Underlying Problem & Its Roots
The Classic "Chicken or the Egg" Dilemma
The Pitfall of CLI Command Constructors
Strategies to Prevent These Errors
Simplify with Automated Proxy Injection
Summary

Intro

Hello everyone,

In this article, I'll provide practical examples showcasing how unexpected challenges can arise in Magento 2 & Adobe Commerce projects while building and/or using custom CLI commands from 3rd parties. This becomes especially significant while working having CI/CD and performing project setups and test execution in pipelines.

Introducing the main "hero" of our article: ISSUE: magento setup command is throwing magento.flag does't exist.. We'll take a deeper look into this issue, understanding its roots and finding ways to address it.

Disclaimer: Just a quick heads-up on naming - both Adobe Commerce and Magento 2 are covered in this article. After all, Adobe Commerce is Magento. Long Live Magento! 😁

The Underlying Problem & Its Roots

While working on various Magento 2 projects, I frequently encountered snags when performing a clean setup of a project using a freshly created database. This usually happens during integration or functional test setups in pipelines & local environments. Still, this issue isn't exclusive to that scenario and can pop up during upgrades, migrations, or deployments.

The culprits? Here are two pesky errors you might recognize:



The default website isn't defined. Set the website and try again.

For Commerce editions, there's also:



SQLSTATE[42S02]: Base table or view not found: 1146 Table
 'magento2.flag' doesn't exist, query was: 
SELECT flag.* FROM flag WHERE (flag.flag_code='staging')

When you see these during the php bin/magento setup:install command execution, they offer little insight into the real issue. After a few hair-pulling hours of research, you'd discover these errors stem from the premature invocation of the DB resource, triggering the initialization of config and state entities from the Magento DB.

But here's the conundrum: You can't fetch data from the DB at this point because there aren't any tables present yet. They only appear and populate during the installation process, not beforehand. ¯_(ツ)_/¯

The Classic "Chicken or the Egg" Dilemma

So, we're caught in a circular problem: trying to install the system but getting stopped because it's not installed. Confusing, right? 🤪

This happens because someone, somewhere, is prematurely calling the DB connection resource. It is obvious, and I hope you would be as grateful as Sam Eliot is for this revelation insight.

This "someone" should be inside bin/magento and called before actual command execution. Usually, CLI command constructors are guilty here. Let me explain why.

The Pitfall of CLI Command Constructors

For clarity, take a look at this code:



class DoSmthing
{
    private ?string $dummyConfigValue;

    public function __construct(ScopeConfigInterface $storeManager)
    {
        $this->dummyConfigValue = $storeManager->getValue('some/dummy/path', ScopeInterface::SCOPE_STORE);
    }

    public function getValue(): ?string
    {
        return $this->dummyConfigValue;
    }
}

class DummyCommand extends Command
{
    ...
    public function __construct(DoSmthing $model, string $name = null)
    {
        $this->model = $model;
        parent::__construct($name);
    }
    ...
}

The root of the problem lies in the constructors of both the command and model classes.

In the DoSmthing class, the system attempts to retrieve a config value. This action calls the store, store_website, and core_config_data tables when creating the DummyCommand object.

And guess what happens when this code runs before the tables are ready? Yep, errors appear!

Lets repeat one more time:

We execute php bin/magento setup:install ;
bin/magento application creates all instances of CLI commands;
DummyCommand instance depends on the DoSmthing model. So DoSmthing should be created before DummyCommand object creation;
Application performs creation of DoSmthing model and executes DoSmthing::__construct method;
DoSmthing::__construct method calls to the config manager;
Config manager calls to a chain of tables that are not created yet;
ERROR

Strategies to Prevent These Errors

Remember this golden rule: Every CLI command class initializes every time you invoke php bin/magento. So, here's how to save yourself future headaches:

Rule 1 - Be Wise

It is not the most wise solution to call for potential resource consuming operations like DB/API calls in __construct methods of CLI command classes. Those operations will be constantly executed even once you'd have a wish to get only a list of available commands.

Rule 2 - Be Ready

Steer clear of resources in __construct that might NOT be available during runtime, especially since Magento tables aren't present during setup:install.

Rule 3 - Fix The Problem

If you're not in control of the troublesome code, inject a Proxy using di configs.

We can easily solve our problem from the example with DummyCommand by just adding those lines in di.xml:



    <type name="AbcTest\DummyConsoleCommand\Console\Command\DummyCommand">
        <arguments>
            <argument name="model" xsi:type="object">\AbcTest\DummyConsoleCommand\Model\DoSmthing\Proxy</argument>
        </arguments>
    </type>

Proxy is a well-known pattern, and in Magento, it is widely used to avoid unneeded code execution before we actually request it. In our example - DoSmthing class object will not be instantiated until we do call for any method of this class.

No config retrieving while CLI class object creation - No error.

More details about proxy here: Proxies .

Simplify with Automated Proxy Injection

For those with a small number of extensions on their project, pinpointing the rogue CLI commands isn't too challenging. However, for more extensive, legacy projects or ones brimming with third-party extensions, identifying the culprits becomes an ordeal.

To save time, we've crafted a solution to automate the proxy injection process. Introducing: run-as-root/magento-cli-auto-proxy.

This package injects Proxy into all __construct arguments of CLI commands, eligible for proxying. The result? No errors related to early resource access and, as a side affect, faster execution of php bin/magento and php bin/magento setup:install.

Check out the README for more details. We always appreciate feedback and contributions!

Summary

Let's sum it up:

Unthoughtfully created CLI commands can unexpectedly lead to problems, even if they appear harmless at first.
The core of the problem often arises from premature database calls, leading to a classic "chicken or the egg" scenario where system installation gets stalled.
One significant pitfall identified is the CLI command constructors and their potential to wreak havoc when invoked at the wrong time.
To combat these challenges, we laid out three guiding principles:
1. Be Wise by avoiding resource-intensive calls in constructors.
2. Be Ready by steering clear of resources that might not be available.
3. Fix The Problem by utilizing proxies for problematic third-party code.
To streamline and automate the solution, we introduced the package run-as-root/magento-cli-auto-proxy which aims to automate proxy injections, offering a more efficient and error-free Magento experience.

Thank you for diving deep into this article. I welcome all your questions and appreciate any feedback you might have.
Happy coding and... Long Live Magento! 😁

Configuring a docker environment for a Magento 2 project

Cristiano Pacheco — Tue, 25 Jul 2023 19:09:07 +0000

This post will demonstrate how to configure a docker environment with the Warden tool for an existing Magento 2 project.

What is Warden?
Why Warden?
Quickstart
Requirements
Project Stack
Configuring a single website/store Magento 2 project
- Creating the project folder
- Creating the project folder
- Clone the Magento 2 code-base into the code-base folder
- Clone the Warden Magento 2 template
- Update the warden .env file
  - A brief explanation of the keys in the .env file
- Run the docker environment
- Run composer install
- Copy and rename the env.warden.php file
- Import the Magento 2 database
  - An Example of the content of this set-default-config.sh script
- Sign the SSL certificate
- Update your /etc/hosts file with the store domain
- Installing the sample data (Optional)
Configuring a multiple website/store view Magento 2 project
- Create a project folder
- Enter the created multisite directory
- Clone the Magento 2 code-base into the multisite code-base folder
- Clone the Warden Magento 2 template
- Change the warden .env file
- Define custom variables for the store domains
- Updates the the defaults config script
- Run the docker environment
- Composer install
- Copy and rename the env.warden.php file into the etc folder
- Import the Magento 2 multisite database
- Sign the SSL certificates for all domains
- Append your /etc/hosts file with the store domains
- Configuring the application domains

What is Warden?

Warden is a CLI utility for orchestrating Docker-based developer environments that fully support Magento 1, Magento 2, Laravel, Symfony, and Shopware 6 on macOS, Linux and Windows (via WSL2).

Why Warden?

Warden lets you run multiple environments simultaneously without conflicting ports.
Allows you to configure stack versions easily via .env file configuration.
Provides the full stack needed for a Magento 2 development environment.
You can extend it and add new features according to your needs.
No need to version the docker environment in the project.

Quickstart

We will configure two Magento 2 projects, the first project will have one website and one store view, and the second project will have multiple websites and store views.

The commands exposed here will work in Linux or macOS environment.

We will use the ~/Sites directory to place the projects. Feel free to put your project in any path on your computer if you change the commands informed here.

For each project, the following directory structure will be used:

📦project-name
 ┣ 📂code-base
 ┗ 📂docker

The project-name/code-base directory will contain the Magento 2 project code
The project-name/docker directory will contain the warden template for our docker environment

Note: The structure above is just a personal taste to use in my projects, you can structure as you see fit. For example using the ~/.warden directory

Your Magento 2 project can be versioned alone in the repository, and there is no need to change the directory architecture.

Here we'll use a fork of the Warden Magento 2 Template that have a Makefile and a few common commands to facilitate the day-to-day work.

Requirements

All you need is Docker, docker-compose and Warden installed on your computer.

Project Stack:

For a practical example, we will set up the development environment for an existing Magento 2 project with the stack below:

Magento 2.4.4 open source
Nginx 1.16
PHP 8.1
- Composer 2
- Xdebug 3
MariaDB 10.4
Elastic Search 7.16
Redis 6.2
RabbitMQ 3.9

More details about the required stack for Magento 2 here.

Configuring a single website/store Magento 2 project

1 Creating the project folder

mkdir ~/Sites/magento244

2 Enter the created directory

cd ~/Sites/magento244

3 Clone the Magento 2 code-base into the code-base folder

git clone git@github.com:cristiano-pacheco/magento244.git code-base

4 Clone the Warden Magento 2 template

git clone git@github.com:cristiano-pacheco/warden-env-magento2.git docker

5 Update the warden .env file

Enter the docker directory

cd ~/Sites/magento244/docker

Update the .env file with the content below:

WARDEN_ENV_NAME=magento244
WARDEN_ENV_TYPE=magento2
WARDEN_WEB_ROOT=/../code-base

TRAEFIK_DOMAIN=magento244.test
TRAEFIK_SUBDOMAIN=app

WARDEN_DB=1
WARDEN_ELASTICSEARCH=1
WARDEN_VARNISH=0
WARDEN_RABBITMQ=1
WARDEN_REDIS=1

ELASTICSEARCH_VERSION=7.16
MARIADB_VERSION=10.4
NODE_VERSION=10
PHP_VERSION=8.1
PHP_XDEBUG_3=1
COMPOSER_VERSION=2
RABBITMQ_VERSION=3.9
REDIS_VERSION=6.2
VARNISH_VERSION=7.0

MYSQL_USER=magento
MYSQL_PASSWORD=magento
MYSQL_DATABASE=magento

WARDEN_SYNC_IGNORE=

WARDEN_ALLURE=0
WARDEN_SELENIUM=0
WARDEN_SELENIUM_DEBUG=0
WARDEN_BLACKFIRE=0
WARDEN_SPLIT_SALES=0
WARDEN_SPLIT_CHECKOUT=0
WARDEN_TEST_DB=0
WARDEN_MAGEPACK=0

BLACKFIRE_CLIENT_ID=
BLACKFIRE_CLIENT_TOKEN=
BLACKFIRE_SERVER_ID=
BLACKFIRE_SERVER_TOKEN=

A brief explanation of the keys in the .env file:

WARDEN_ENV_NAME It will be the prefix of the docker containers name and will also serve to configure the xdebug in your IDE Example:

WARDEN_ENV_TYPE
It will define the docker compose yaml files that will be used in the environment, which can vary according to the chosen platform.
WARDEN_WEB_ROOT
Self explained, you have to inform the root directory of the project.
TRAEFIK_DOMAIN
Traefik is used to route the requests to the services (containers) dynamically. For example, when there is the cookie XDEBUG_SESSION with the value PHPSTORM, it will send the requests to the php-debug container that has the xdebug enabled. On the contrary, the php-fpm container will receive the request.
You can set the domain you want, just pay attention to the .test suffix, it you configure a domain with a different value you will lost the Automatic DNS Resolution feature.
TRAEFIK_SUBDOMAIN
It is used to set a subdomain for your application, in our example the store can be accessed with the address https://TRAEFIK_SUBDOMAIN.TRAEFIK_DOMAIN that is: https://app.magento244.test
WARDEN_DB, WARDEN_ELASTICSEARCH, WARDEN_VARNISH, WARDEN_RABBITMQ, WARDEN_REDIS

The keys above you can use 0 to disable and 1 to enable. When the value is 0 the container will not run.

_VERSION
The keys with the suffix _VERSION is to set a specific version to the service. For Magento 2.4.3, for example, you may use PHP 7.4. To do it, you can leave the key as: PHP_VERSION=7.4
PHP_XDEBUG_3
If you set this key with value 1, a different tag of the container php-fpm image will be used. Let's look at the docker-compose yaml generated by warden with the command warden env config:

COMPOSER_VERSION If it has the value = 1 or you ommit this key composer in version 1 will be the default executable, however you can run composer in version 2 with the command composer2.

Example:
If you omit or set the value 1 for COMPOSER_VERSION

If you set the value 2 for COMPOSER_VERSION:

I prefer to inform the value 2 and already run composer in version 2 with the composer command.

There are more interesting things to configure such as Magento 2 tests, live reload, elastic search and more. You can see this information here.

6 Run the docker environment

Inside the ~/Sites/magento244/docker directory type:

make docker-up

7 Run composer install

Inside the ~/Sites/magento244/docker run the command
below to access the PHP container

warden shell

Run the composer install command

composer install

8 Copy and rename the `env.warden.php` file

It will copy the file to the code-base/etc directory with the command below:

cp ~/Sites/magento244/code-base/app/etc/env.warden.php ~/Sites/magento244/code-base/app/etc/env.php

The env.warden.php file has the default configuration entries with the warden hosts for the Mysql database, Redis (session and store cache), RabbitMQ, and Elastic Search.

9 Import the Magento 2 database

There is a sample database inside our Magento project, so copy the ~/Sites/magento244/code-base/dump.sql.gz file to ~/Sites/magento244/docker/backfill/magento-db.sql.gz with the command below:

cp ~/Sites/magento244/code-base/dump.sql.gz ~/Sites/magento244/docker/backfill/magento-db.sql.gz

Open a new tab in your terminal and enter the directory ~/Sites/magento244/docker and type the command below:

make db-import

The db-import command is one of the custom commands inside the Makefile.

This command imports the Magento-db.sql.gz database dump to the Magento database in the warden MySQL docker container.

In addition to importing the database, the ~/Sites/magento244/docker/tools/set-default-config.sh script is then executed to define the project's base URL and create a user for the Magento administrative panel.

In the ~/Sites/magento244/docker/tools/set-default-config.sh file, I usually put all the relevant settings to the Magento 2 environment to avoid errors and also to speed up the restoration of the environment as faithful as possible to the actual project.

Example of settings that are added in this file:

Project base URLs
Media and asset URLs
Payment test credentials
Changing the Algolia index to the development index, etc.

An Example of the content of this `set-default-config.sh` script:

#!/usr/bin/env bash

source $(dirname $0)/common.sh

:: Import the config from files
warden env exec -T php-fpm bin/magento app:config:import

:: Set BASE URL Config
warden env exec -T php-fpm bin/magento config:set web/unsecure/base_url ${URL_FRONT}
warden env exec -T php-fpm bin/magento config:set web/secure/base_url ${URL_FRONT}
warden env exec -T php-fpm bin/magento config:set web/unsecure/base_link_url ${URL_FRONT}
warden env exec -T php-fpm bin/magento config:set web/secure/base_link_url ${URL_FRONT}

:: Create an admin user
warden env exec -T php-fpm bin/magento admin:user:create --admin-user=admin --admin-password=123123q --admin-email=admin@example.com --admin-firstname=Admin --admin-lastname=Admin

10 Sign the SSL certificate

Still, inside the ~/Sites/magento244/docker directory, type the command below:

warden sign-certificate app.magento244.test

11 Update your `/etc/hosts` file with the store domain:

sudo echo '127.0.0.1 app.magento244.test' >> /etc/hosts

Note Do the step above only if dnsmasq is not being used.

12 Installing the sample data (Optional)

Access the php container:

warden shell

Run the command below to install the sample data:

bin/magento sampledata:deploy

Run the command below to update the project database

bin/magento setup:upgrade

Now you're able to access the store with the URLS \o/:

Store Address: https://app.magento244.test
Backend admin panel: https://app.magento244.test/backend

Configuring a multiple website/store view Magento 2 project

This Magento 2 store has been configured with the below structure of websites/stores/store views:

So, the main difference is how to configure a local warden environment to support multiple websites/store views with different domains.

1 Create a project folder

mkdir ~/Sites/magento244-multisite

2 Enter the created multisite directory

cd ~/Sites/magento244-multisite

3 Clone the Magento 2 code-base into the multisite code-base folder

git clone git@github.com:cristiano-pacheco/magento244-multisite.git code-base

4 Clone the Warden Magento 2 template

git clone git@github.com:cristiano-pacheco/warden-env-magento2.git docker

5 Change the warden .env file

Enter the docker directory

cd ~/Sites/magento244-multisite/docker

Update the .env file with the content below:

WARDEN_ENV_NAME=magento244-multisite
WARDEN_ENV_TYPE=magento2
WARDEN_WEB_ROOT=/../code-base

TRAEFIK_DOMAIN=magento244-multisite.test
TRAEFIK_SUBDOMAIN=app

WARDEN_DB=1
WARDEN_ELASTICSEARCH=1
WARDEN_VARNISH=0
WARDEN_RABBITMQ=1
WARDEN_REDIS=1

ELASTICSEARCH_VERSION=7.16
MARIADB_VERSION=10.4
NODE_VERSION=10
PHP_VERSION=8.1
PHP_XDEBUG_3=1
COMPOSER_VERSION=2
RABBITMQ_VERSION=3.9
REDIS_VERSION=6.2
VARNISH_VERSION=7.0

MYSQL_USER=magento
MYSQL_PASSWORD=magento
MYSQL_DATABASE=magento

WARDEN_SYNC_IGNORE=

WARDEN_ALLURE=0
WARDEN_SELENIUM=0
WARDEN_SELENIUM_DEBUG=0
WARDEN_BLACKFIRE=0
WARDEN_SPLIT_SALES=0
WARDEN_SPLIT_CHECKOUT=0
WARDEN_TEST_DB=0
WARDEN_MAGEPACK=0

BLACKFIRE_CLIENT_ID=
BLACKFIRE_CLIENT_TOKEN=
BLACKFIRE_SERVER_ID=
BLACKFIRE_SERVER_TOKEN=

More details about the configuration above was shared in the step 5 of Configuring a single website/store Magento 2 project

6 Define custom variables for the store domains

We will define custom variables in ~/Sites/magento244-multisite/docker/tools/common.sh to be able to reuse them later in our set-default-configs.sh file:

URL_WEBSITE_TWO_FRONT=https://app.magento244-website-two.test/
URL_STORE_VIEW_THREE_US_FRONT=https://app.magento244-store-view-three-us.test/
URL_STORE_VIEW_THREE_DE_FRONT=https://app.magento244-store-view-three-de.test/

7 Updates the the defaults config script

Add the content of this file to the ~/Sites/magento244-multisite/docker/tools/set-default-configs.sh file.

The goal of the set-default-configs.sh script was explained here.

8 Run the docker environment

Inside the ~/Sites/magento244-multisite/docker directory type:

make docker-up

9 Composer install

Inside the ~/Sites/magento244-multisite/docker run the command below to access the PHP container

warden shell

Run the composer install command

composer install

10 Copy and rename the `env.warden.php` file into the etc folder

cp ~/Sites/magento244-multisite/code-base/app/etc/env.warden.php ~/Sites/magento244-multisite/code-base/app/etc/env.php

11 Import the Magento 2 multisite database

There is a sample database inside our Magento project, so copy the ~/Sites/magento244-multisite/code-base/dump.sql.gz file to ~/Sites/magento244-multisite/docker/backfill/magento-db.sql.gz with the command below:

cp ~/Sites/magento244-multisite/code-base/dump.sql.gz ~/Sites/magento244-multisite/docker/backfill/magento-db.sql.gz

Open a new tab in your terminal and enter the directory ~/Sites/magento244-multisite/docker and type the command below:

make DB-import

12 Sign the SSL certificates for all domains

Still, type the commands below inside the ~/Sites/magento244-multisite/docker directory.

To sign the certificate to the main and the admin store:

warden sign-certificate app.magento244-multisite.test

To sign the certificate to the website-two:

warden sign-certificate app.magento244-website-two.test

To sign the certificate to the Store View Three US:

warden sign-certificate app.magento244-store-view-three-us.test

To sign the certificate to the Store View Three DE:

warden sign-certificate app.magento244-store-view-three-de.test

13 Append your `/etc/hosts` file with the store domains

127.0.0.1 app.magento244-multisite.test
127.0.0.1 app.magento244-website-two.test
127.0.0.1 app.magento244-store-view-three-us.test
127.0.0.1 app.magento244-store-view-three-de.test

Note Do the step above only if dnsmasq is not being used.

14 Configuring the application domains:

According to the Warden documentation we have to create a warden-env.yml file inside the .warden directory. Type the commands below:

Create the .warden-env.yml file

touch ~/Sites/magento244-multisite/docker/.warden/warden-env.yml

Put the content below to the .warden-dev.yml file:

version: "3.5"
services:
  nginx:
    labels:
      - traefik.http.routers.${WARDEN_ENV_NAME}-nginx.rule=
        HostRegexp(`{subdomain:.+}.${TRAEFIK_DOMAIN}`)
        || Host(`${TRAEFIK_DOMAIN}`)
        || HostRegexp(`{subdomain:.+}.magento244-multisite.test`)
        || Host(`magento244-multisite.test`)
        || HostRegexp(`{subdomain:.+}.magento244-website-two.test`)
        || Host(`magento244-website-two.test`)
        || HostRegexp(`{subdomain:.+}.magento244-store-view-three-us.test`)
        || Host(`magento244-store-view-three-us.test`)
        || HostRegexp(`{subdomain:.+}.magento244-store-view-three-de.test`)
        || Host(`magento244-store-view-three-de.test`)

You can have a copy of this file here.

When you're done with the settings, look at the traefik address to see where this mapping ended up. It will be like as:

Create stores.php the Add the Magento 2 run params to the project:

cd touch ~/Sites/magento244-multisite/code-base/app/etc/stores.php

Add the content of this file in the file above.

Add the app/etc/stores.php to the composer.json file.

It should look like the image below after the change:

It will force composer to load this file on every request. the file then sets up the store for Magento to load based on the domain, passing the MAGE_RUN_CODE and MAGE_RUN_TYPE variables to Nginx.

Now you're able to access the stores with the URLS \o/:

Default Store Address:
https://app.magento244-multisite.test

Backend admin panel:
https://app.magento244-multisite.test/backend

Website Two Address:
https://app.magento244-multisite.test

Store View Three US Address:
https://app.magento244-store-view-three-us.test

Store View Three DE Address:
https://app.magento244-store-view-three-de.test

Note:
Inside the directory ~/Sites/magento244-multisite/code-base, type the command make, and you will see an output like:

That's it. If you have any questions, please let us know in the comments 👋

Composable Commerce - or: MACH Architectures in E-Commerce

Rico Neitzel — Fri, 21 Jul 2023 10:38:12 +0000

Gone are the days when online shops were built with one massive program. Now, one can select a distinct provider for every software segment, thus creating a unique e-commerce world. Or can they? A new trend is on everyone's lips: MACH! In German that means "Do it!" … But what exactly are we supposed to "do"? And what does it have to do with Composable Commerce?

In our post, we delve into what MACH entails, its potential benefits in e-commerce, but also the drawbacks it currently carries.

Composable Commerce

Until now, a typical "online shop" project involved collaborating with a service provider to develop an extensive application (the online shop) covering all crucial aspects: product catalog, search, ordering and payment process, and order and customer data management.

A disadvantage of these monolithic solutions is the dependency on a single manufacturer, whose updates affect the entire application.

Composable Commerce aims to modularize this monolithic structure and make the individual parts interchangeable.

Technically, it's about separating the components of an eCommerce system (such as shopping cart, payment processes, product data management, and search) from each other and connecting them via APIs so that each section can be expanded, replaced, or optimized independently, without affecting the rest of the system.

The apparent advantage for the retailer is to be able to search for a specialized provider for each aspect and ultimately compile their own solution without depending on a single provider.

The Foundation: MACH Architectures

MACH architecture (Microservices, API-first, Cloud-native, Headless) is the foundation for Composable Commerce.

The concept is quite simple: each part of the system ("Microservice") operates independently and communicates exclusively via defined interfaces ("APIs") with the other components. These components are designed for deployment on distributed systems and leverage their advantages like scalability, reliability, and cost-efficiency ("Cloud-native").

The separation of frontend (the user interface that the customer sees) and backend (like data management and business logic) allows independent further development of each component ("Headless").

The MACH architecture primarily aims at investment, a secured future, and maximum flexibility.

MACH Alliance

However, one challenge of this approach is the collaboration of various manufacturers and service providers. To coordinate this collaboration internationally, an alliance has been formed, providing educational work and certifying service providers. This is intended to make it easier for companies to commission providers who understand and can implement the MACH standards.

Composable Commerce Projects

Many service providers suggest that with Composable Commerce, all components are ready and just need to be "put together". However, implementing such a platform requires both internal and external expertise in terms of both the industry and the technology. Therefore, the application itself must be programmed — and here lies the actual work: the program code, which connects all these microservices via their APIs, must be precisely planned and developed.

This is where the "composable" aspect comes fully into play, as we can decide which components from which provider we want to connect. Search? Algolia! — Product catalog? Magento! — Shopping cart and payment process? Commerce Tools! ... and the list of possibilities and decisions to be made does not end here.

Even after this decision, the possibility remains open to exchange the components in a few years if a more suitable solution is found.

Dependency Hell - Fool's Gold!

Even if all of this sounds like shining gold, closer examination reveals: it's just fool's gold. Because the interchangeability of components often praised in advertising is not currently available.

Many providers already include their own components and advertise a closed system. Yes, it is composable; but no: I cannot easily combine any manufacturers at will.

A crucial technical prerequisite for this interoperability is, as often: standardization. And precisely this is currently not available. There is no DIN "product catalog" defining data formats for the exchange between components. It's not even standardized which API commands (so-called endpoints) must be communicated.

Thus, at the moment, each manufacturer has their own API with their own formats.

And at this point, the great advantage of composable commerce is currently lost. One is trapped in provider dependencies and cannot simply replace components without ordering development work for the integration again.

The Costs of Composable Commerce

It's often said that costs with modern architectures should be lower in the long term than with classic monoliths. This effect comes into play nowadays – if at all – only in rare exceptional cases.

In many cases, MACH architecture means a significantly higher demand on companies and people. Additionally, integrating new functions often means working on several of these components. Usually, different teams then have to be coordinated.

So, when the entire system has a bug, the search begins in distributed systems. Is it the application itself? Or one of the components? If yes, which one? Is it the sole cause of the problem, or do several issues in different components need to come together to trigger the error? The error search is thus much more difficult in these distributed systems.

All these challenges also raise the required budget of the project. A cost-saving only occurs in doubt when everything works smoothly — a rare state.

Thus, the difficult question remains of how to realize your project with a view to the future and what budget you want to put into it.

Quo Vadis?

As long as the major players of the MACH Alliance do not all sit at a table and decide that the welfare of the retailers should be placed above their own financial interests, one has to live with the sour taste of microservice-based monoliths. Only with established and certified interfaces and data formats can the full potential of Composable Commerce be exploited in the future.

An Opportunity - but with Consideration

Even today, existing monoliths are equipped with additional components, bringing new capabilities to existing systems. Algolia is an excellent example of how an external provider can replace and drastically improve the shop's search. Such solutions work reliably and cost-effectively.

However, the integrations are always tailored to the respective system, sometimes even to the individual use case. But they beautifully show that the idea of "Composable Commerce" is heading in the right direction.

So it remains exciting, and ultimately time will tell whether the MACH Alliance will establish common standards that find cross-industry support. At the moment, companies need to take the time to understand the possibilities, impacts, and costs to make an informed decision. Every hype should be critically questioned, and always remember that technology should always be a tool for the company; not the other way around.

Rico Neitzel is Co-Founder of run_as_root GmbH in Germany. We specialize in software quality automation and high quality software development. We primarily focus on ecommerce.

Navigating Magento 2 Architecture: Discovering Plugin Architecture for Newcomers

David Lambauer — Mon, 17 Jul 2023 18:15:13 +0000

Hello again, and welcome back to our series on Magento 2 Architecture for newcomers! We're glad to see you return for the second part of our journey, a clear indicator of your dedication to understanding and mastering this powerful platform. In our first post, we delved into the fascinating world of Magento 2's Observer Architecture. We explored its unique functionality and got our hands dirty with practical examples to bolster our knowledge.

Today, we will be unraveling the intricacies of Magento 2's Plugin Architecture. Our aim is to ensure you not only understand the concept theoretically but also gain the practical skills necessary to apply them efficiently. So buckle up, as we gear up for another exciting ride through the world of Magento!

The Role of Plugins in Magento 2

In Magento 2, plugins, also referred to as Interceptors, play a critical role in extending the functionality of existing public methods within a class. They provide a mechanism that allows us to alter or extend the behavior of public methods without modifying the actual class.

The charm of plugins lies in their ability to interact with a method at three different points: before, after, and around. This allows developers to modify the input or output data or even change the flow of processing.

Registering a Plugin

To leverage the power of plugins, we need to register them first. This is done in the di.xml file of your module. Here's a basic example:

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <type name="Magento\Framework\App\Action\Action">
        <plugin name="vendor_module_slack_notifier" type="Vendor\Module\Plugin\SlackNotifier" sortOrder="10"/>
    </type>
</config>

In this example, we've registered a plugin named vendor_module_slack_notifier for the class Magento\Framework\App\Action\Action. The plugin's class is Vendor\Module\Plugin\SlackNotifier.

Writing a Plugin

Now, let's get down to the nitty-gritty of writing our plugin. Remember our goal here is to send a Slack message at a clever place. A good idea could be to notify a Slack channel whenever a new order is placed on the website. In this case, let's consider we are writing a plugin for the placeOrder method in the Magento\Sales\Model\Service\OrderService class.

namespace Vendor\Module\Plugin;

class SlackNotifier
{
    protected $slack;

    public function __construct(
        \Vendor\Module\Model\Slack $slack
    ) {
        $this->slack = $slack;
    }

    public function afterPlaceOrder(
        \Magento\Sales\Model\Service\OrderService $subject,
        $result
    ) {
        $message = "New order placed with ID: " . $result;
        $this->slack->sendMessage($message);
        return $result;
    }
}

In this plugin, we are using the after plugin to send a Slack message after an order has been placed.

Unit Testing a Plugin

After we've written our plugin, it's important to test it to ensure it behaves as expected. Here's how you can use PHPUnit to test your plugin.

namespace Vendor\Module\Test\Unit\Plugin;

class SlackNotifierTest extends \PHPUnit\Framework\TestCase
{
    protected $slackNotifier;

    protected function setUp(): void
    {
        $this->slackNotifier = $this->getMockBuilder(\Vendor\Module\Plugin\SlackNotifier::class)
                                    ->disableOriginalConstructor()
                                    ->getMock();
    }

    public function testAfterPlaceOrder()
    {
        $orderServiceMock = $this->createMock(\Magento\Sales\Model\Service\OrderService::class);
        $result = "100001";
        $this->slackNotifier->expects($this->once())
                            ->method('afterPlaceOrder')
                            ->with($orderServiceMock, $result);

        $this->slackNotifier->afterPlaceOrder($orderServiceMock, $result);
    }
}

Observers vs Plugins

While observers and plugins might seem similar, they serve different purposes and their usage depends on your requirements. Plugins are great for modifying the behavior of existing public methods. However, if you're interested in reacting to specific events in the application, observers would be a more suitable choice.

The observer pattern can be seen as a more global approach, as it enables a certain type of code (observers) to react to system-wide events without coupling the observers to the code that emits those events. Plugins, on the other hand, are more targeted, allowing us to interact with a method at specific points in its execution.

With the knowledge you've gained from this post and the previous one, you are now well-equipped to start implementing these concepts into your projects. Next in this series, we will dive deeper into the world of Magento 2 architecture. Stay tuned and happy coding!

Extension Attributes in Adobe Commerce: Achieving Clean and Decoupled Code

Vladyslav Podorozhnyi 🇺🇦 🌻 — Thu, 13 Jul 2023 20:21:44 +0000

Introduction
Chapter 1: Understanding the Adobe Commerce Service Layer
Chapter 2: Exploring the Idea of Extension Attributes
Chapter 3: Extension Attributes and Service Contracts
Chapter 4: Saving Data for Extension Attributes
Chapter 5: Key Takeaways

Introduction

Welcome to our blog post, where we delve into the practical realm of Extension Attributes in Adobe Commerce (formerly Magento). If you've been working with Magento 2 for a while, you've probably encountered Extension Attributes and harnessed their capabilities. However, it's surprising how often developers overlook the valuable benefits that Extension Attributes offer, especially when compared to outdated methods of entity extension.

In this article, we aim to shed light on the advantages of Extension Attributes and how they can enhance your development projects. We'll explore their integration with the Service Layer, providing you with insights on how to write clean, modular code that avoids unnecessary dependencies on other layers or extensions.

Whether you're an experienced Adobe Commerce developer seeking to broaden your understanding or a newcomer eager to learn more, this article guides you to a bit deeper understanding of Extension Attributes' place & role in Adobe Commerce.

Join us in exploring Extension Attributes. Let's dive in and discover the practical benefits they bring to your development workflow. Let's get started!

Chapter 1: Understanding the Adobe Commerce Service Layer

As you may know already, Adobe Commerce has four Architecture layers:

Presentation layer - point of user interaction with Adobe Commerce.
Service layer - it is a "glue" between layers and a set of APIs used by modules and external systems to communicate with each other. Service contracts are in use here.
Domain layer - contains the business logic of the module.
Persistence layer - resource level, knows about DB and other storages.

See more details here: Architectural layers overview

The service layer is vital for understanding extension attributes, especially the principle of "service contracts" that is used as a system pillar in Adobe Commerce. If you do not know about "service contracts," - read this article Service contracts and keep in mind that this is an agreement between parts of the system on the functionality list available for each party and how these functionalities could be triggered. REST API and API folder in modules are two bright examples of service contract implementation.

The service layer allows to build long term, strictly defined relations between services, both internal and external.

While developing your extension and needing to change/extend one or another functionality of the system - you would need to rely on the Service layer, not any other. Only the Service layer would be able to provide you with a warranty to be changed in the nearest perspective - so there are no troubles with upgrades all the time, as the API definition is
not changed often. I'm not even mentioning the benefits of explicit entry point definition, its parameters, and predictable output.

Chapter 2: Exploring the Idea of Extension Attributes

Imagine you're working with a fundamental entity like an order in Adobe Commerce. This entity has its dedicated table called sales_order and a considerable list of predefined properties. But what if you find yourself in need of adding another property to the order entity?

Sure, you could take the easy route and directly modify the sales_order table, implementing custom observers and models to handle the new property. However, doing so would take you into the Domain layer of another extension, which isn't good. The domain layer of other modules is the last resort for an extension, as they are changed quite often by the vendor. It means meddling with the system's inner workings - system you are out of control, risking compatibility issues during future upgrades. Open–closed principle - the system should be closed for direct changes but open for extending. In this case, modifying the entity directly would be considered changing the system controllable by the vendor Magento - not you.

So, how can we add a property to the entity using the "right" tools?

The answer lies in the power of Extension Attributes. 🙌

Extension Attributes provide a sanctioned approach for extending entities without tampering with the core system's code. Instead of directly modifying the entity table, we leverage Extension Attributes to add new properties in a clean and modular way. By utilizing this method, we ensure our modifications align with the principles of the system's architecture, reducing the risk of compatibility issues and enabling smoother upgrades in the long run.

In the following chapter, we'll explore how Extension Attributes work hand in hand with the Service Layer, enabling you to extend entities effectively and maintain a flexible and scalable codebase.

Chapter 3: Extension Attributes and Service Contracts

Extension Attributes provide a "legitimate" way to modify Service contracts while maintaining backward compatibility for both providers and consumers. Although a simplified definition, it captures the essence of Extension Attributes in Adobe Commerce.

Extension Object: Leveraging Composition over Inheritance

Adobe Commerce adopts the principle of "composition over inheritance" to achieve this flexibility within its development ecosystem. Each entity supporting Extension Attributes has a setter and getter for the Extension object. In the case of orders, it is an instance of a class that implements \Magento\Sales\Api\Data\OrderExtensionInterface.

Configuring Extension Attributes with XML

The behavior of the Extension object is configured through simple and straightforward XML configurations. It is in this configuration where we specify which extension attributes need to be added, define their types, and associate them with the respective entities.

For example:



// @see vendor/magento/module-sales/etc/extension_attributes.xml
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Api/etc/extension_attributes.xsd">
    <extension_attributes for="Magento\Sales\Api\Data\OrderInterface">
        <attribute code="shipping_assignments" type="Magento\Sales\Api\Data\ShippingAssignmentInterface[]" />
    </extension_attributes>
</config>

Support for Extension Attributes: The Role of ExtensibleDataInterface

How does Adobe Commerce knows that order entity supports extension attributes?

Well, \Magento\Sales\Api\Data\OrderInterface extends \Magento\Framework\Api\ExtensibleDataInterface, which informs Adobe Commerce that the order entity will have Extension Attributes. The XML configuration specifies that the property shipping_assignments is one of those Extension Attributes.

Generated Service Contracts

Now that Adobe Commerce is aware of the Extension Attributes, what comes next?

It generates \Magento\Sales\Api\Data\OrderExtensionInterface in the generated/code folder based on the collected XML configurations from extension_attributes.xml.

The generated interface looks like this:



interface OrderExtensionInterface extends \Magento\Framework\Api\ExtensionAttributesInterface
{
    /**
     * @return \Magento\Sales\Api\Data\ShippingAssignmentInterface[]|null
     */
    public function getShippingAssignments();

    /**
     * @param \Magento\Sales\Api\Data\ShippingAssignmentInterface[] $shippingAssignments
     * @return $this
     */
    public function setShippingAssignments($shippingAssignments);
...
}

Additionally, the implementation of the contract/interface, \Magento\Sales\Api\Data\OrderExtension, is also generated.

Summary

In essence, Adobe Commerce has automatically generated a new Service Contract for us based on our configurations. We can now use this contract to effortlessly set and retrieve values of Extension Attributes for the order entity, without the need to navigate through other layers apart from the Service Layer of the Magento_Sales model.

This outstanding move allows your newly added Extension Attribute to become an integral part of the Service Contracts and Service Layer. It remains independent of the implementation details of any modules, extensions, or the system itself that you do not have control over.

Congratulations on leveraging this remarkable feature of Adobe Commerce!

In the next chapter, we'll explore how to not get unwanted dependencies while further Extension Attributes implementation.

Chapter 4: Saving Data for Extension Attributes

It's not always necessary to save Extension Attribute data persistently. Sometimes, it suffices to calculate their values on the fly when needed. However, let's assume we do need to save an attribute.

Main Steps for Saving Extension Attribute Data

The Adobe Docs article, Add extension attributes to entities, provides a comprehensive guide on how to achieve this. Let's go through a quick recap. To add data to an Extension Attribute, you typically follow these steps:

Create a plugin that intercepts the save operation of the entity to retrieve the Extension Attribute data and save it at the resource level.
Create a plugin that intercepts the load operation of the entity to retrieve the Extension Attribute data and populate the entity using the Extension object.

While there can be variations, this is the main scenario, requiring at least two plugins for saving and loading operations.

Where to Save Extension Attribute Data?

Now, that's a brilliant question that leaves many stumped, but not you! 😄

One option is to save the Extension Attribute data in the same table as the entity—for example, the sales_order table for orders. However, this approach introduces a dependency on the Domain Layer of the Magento_Sales module.

And yes, you're right—it's not "legal" in our paradigm. We aim to extend the software, not modify it.

The Best Approach: Have Your Own Entity

In my opinion, the best approach is to have your own entity to save and retrieve the Extension Attribute data. It doesn't matter what persistence layer you choose—whether it's a database, session, file, API, carrier pigeons, or even Morse code written on paper.

It may seem like overkill, but it's the most sensible way to avoid unwanted dependencies on unmodifiable parts of the system. Your Extension Attribute implementation should not impact the core system or violate the Open-Closed principle. By cutting down all connections and dependencies on the extendable module, except for the Extension Attributes definition, you ensure the integrity of the Service Contracts and keep the system functioning as intended.

This approach allows you to maintain a clean and decoupled codebase, free from unnecessary entanglements with uncontrollable system components.

Chapter 5: Key Takeaways

As we conclude this article on Extension Attributes in Adobe Commerce, let's summarize the key takeaways:

Service Layer and Service Contracts: The Service Layer and Service Contracts serve as excellent foundations for extending functionality. They provide stability, predictability, and explicit definitions, making them ideal for seamless extension development.
Persistence, Domain, and Presentation Layers: On the other hand, the Persistence, Domain, and Presentation Layers are not suitable for direct extension. These layers can undergo changes with each release, making them less reliable for long-term extension development.
Extension Attributes and Service Contracts: Extension Attributes are an integral part of the Service Layer and Service Contracts. Leveraging Extension Attributes enables effective extension development and facilitates communication across services within Adobe Commerce.
Avoid Saving Extension Attributes Inside the Entity: It is crucial to refrain from saving Extension Attributes within the entity you are trying to extend. Doing so would introduce a dependency on the Persistence Layer of the extendable module, rendering the benefits of the Service Layer useless. Instead, opt for storing Extension Attribute data in a separate entity to maintain a clean and decoupled architecture.

By adhering to these takeaways, you can build robust and scalable extensions that align with the principles of Adobe Commerce's architecture. Remember, the Service Layer and Service Contracts offer a stable foundation, while Extension Attributes provide a flexible and effective approach to extending functionality.

Thank you for joining us in exploring Extension Attributes in Adobe Commerce. We hope you found this article informative and insightful.

Happy developing!

Kickstarting Magento 2 Architecture Journey: Mastering Observer Architecture for Newcomers

David Lambauer — Mon, 19 Jun 2023 21:36:09 +0000

Hello and welcome to the first post in our blog series dedicated to the architectural nuances of Magento 2. Aimed specifically at those taking their initial steps into the Magento ecosystem, this series will illuminate key architectural elements and concepts of the platform. Our inaugural topic is a vital part of Magento's Event-Driven Architecture - the Observer Pattern.

Magento 2 Observer Architecture: An Overview

The Observer Architecture, an event-driven design pattern in Magento, paves the way for effective communication between various sections of code. It adopts the principle of 'Loose Coupling,' promoting interactions without rigid dependencies. Essentially, it lets you extend and customize Magento's functionality without modifying the core system files significantly.

Various events, such as adding a product to the cart or placing an order, initiate specific sequences of actions within Magento 2. The Dispatcher identifies these events and calls the corresponding observers - class methods assigned to carry out actions related to the event.

Identifying Event Names

Magento 2 has predefined event names for numerous actions. To identify the correct event name, you need to understand the functionality you wish to extend or modify. Often, you can find these event names within Magento's core code files, specifically inside the dispatch function calls. A pro tip: For a more comprehensive list, refer to the Magento 2 Event Cheat Sheet available online.

Registering an Observer with `events.xml`

To declare an observer in Magento 2, you will utilize the events.xml file. You'll need to specify the event name you're observing and the class that will handle it. Here's a quick walkthrough on registering an observer:

Create an events.xml file in the etc directory of your module.

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Event/etc/events.xsd">
    <event name="checkout_cart_product_add_after">
        <observer name="slack_message_observer" instance="Vendor\Module\Observer\SlackMessageObserver" />
    </event>
</config>

Crafting an Observer: Sending a Slack Message on Cart Addition

Let's put this into practice by creating an observer that sends a Slack message whenever a product is added to the cart.

First, define your Observer class, SlackMessageObserver.php, within your module's Observer directory.

namespace Vendor\Module\Observer;

use Magento\Framework\Event\ObserverInterface;
use Vendor\Module\Model\SlackMessageSender;

class SlackMessageObserver implements ObserverInterface
{
    private SlackMessageSender $slackMessageSender;

    public function __construct(SlackMessageSender $slackMessageSender)
    {
        $this->slackMessageSender = $slackMessageSender;
    }

public function execute(\Magento\Framework\Event\Observer $observer)
{
    $product = $observer->getDataUsingMethod('product');

    if ($product) {
        $message = 'A product has been added to the cart: ' . $product->getName();
        $this->slackMessageSender->send($message);
    }
}
}

In this code, we've implemented the execute method, which gets invoked when the checkout_cart_product_add_after event is dispatched. The execute method gathers the product data and sends a message to a Slack channel using the SlackMessageSender service.

Testing Your Observer

Testing in Magento 2 typically involves two levels - Unit Testing and Integration Testing.

Unit Testing aims at the smallest testable parts of an application, in isolation. In our observer's case, we can create a mock of SlackMessageSender and confirm if the send method is called with the expected argument.

Integration Testing, conversely, examines the interaction between different application components. It could involve testing the entire process, from adding a product to the cart to sending the Slack message. Here's how an integration test for our observer might look:

namespace Vendor\Module\Observer\Test\Integration;

use Magento\TestFramework\Helper\Bootstrap;
use Magento\Framework\Event\ManagerInterface as EventManager;
use Magento\Catalog\Api\ProductRepositoryInterface;
use Magento\Checkout\Model\Cart;
use Vendor\Module\Observer\SlackMessageObserver;
use Vendor\Module\Model\SlackMessageSender;

class SlackMessageObserverTest extends \PHPUnit\Framework\TestCase
{
    private $objectManager;
    private $cart;
    private $slackMessageSenderMock;

    protected function setUp(): void
    {
        $this->objectManager = Bootstrap::getObjectManager();
        $this->cart = $this->objectManager->get(Cart::class);
        $this->slackMessageSenderMock = $this->createMock(SlackMessageSender::class);
        $this->objectManager->addSharedInstance($this->slackMessageSenderMock, SlackMessageSender::class);
        $this->objectManager->addSharedInstance($this->objectManager->create(SlackMessageObserver::class), SlackMessageObserver::class);
    }

    public function testObserverIsTriggeredOnAddToCart(): void
    {
        $productRepository = $this->objectManager->get(ProductRepositoryInterface::class);
        $product = $productRepository->get('simple'); // 'simple' should be a SKU of a product in your test database

        $this->slackMessageSenderMock
            ->expects($this->once())
            ->method('send')
            ->with($this->stringContains($product->getName()));

        $this->cart->addProduct($product, 1);
        $this->cart->save();
    }

    protected function tearDown(): void
    {
        $this->objectManager->removeSharedInstance(SlackMessageSender::class);
        $this->objectManager->removeSharedInstance(SlackMessageObserver::class);
    }
}

In addition, a basic unit test might look like this:

namespace Vendor\Module\Test\Unit;

use PHPUnit\Framework\TestCase;
use Vendor\Module\Observer\SlackMessageObserver;
use Vendor\Module\Model\SlackMessageSender;
use Magento\Catalog\Model\Product;
use Magento\Framework\Event\Observer;
use Magento\Framework\Event;

class SlackMessageObserverTest extends TestCase
{
    public function testExecute(): void
    {
        $productName = 'Test Product';

        // Mock product
        $productMock = $this->createMock(Product::class);
        $productMock->method('getName')->willReturn($productName);

        // Mock event
        $eventMock = $this->createMock(Event::class);
        $eventMock->method('getData')->with('product')->willReturn($productMock);

        // Mock observer
        $observerMock = $this->createMock(Observer::class);
        $observerMock->method('getEvent')->willReturn($eventMock);

        // Mock SlackMessageSender
        $slackMessageSenderMock = $this->createMock(SlackMessageSender::class);
        $slackMessageSenderMock->expects($this->once())
            ->method('send')
            ->with($this->stringContains($productName));

        // Instantiate the SlackMessageObserver with the mocked SlackMessageSender
        $observer = new SlackMessageObserver($slackMessageSenderMock);
        $observer->execute($observerMock);
    }
}

Testing is as critical as writing code itself. Tests ensure your modules' stability and reliability, and mitigate future issues arising from updates or changes in other parts of the system.

Best Practices

Declare the Observer in the Appropriate Scope

The first step in using Magento 2 observers effectively is to ensure you declare them in the right scope: global, frontend, or adminhtml. The scope of your observer depends on where you need it to operate. A global scope means the observer operates universally, both in the admin area and the frontend. If your observer is only needed in the backend, use the adminhtml scope. Meanwhile, if it's strictly for the user-facing part of your site, then frontend is the appropriate scope.

Make the Observer as Efficient as Possible

Efficiency is key when it comes to coding, particularly with observers in Magento 2. To increase the efficiency of your observer, streamline your code. Try to reduce unnecessary loops and calculations as these can slow down your web pages. Remember, a lean observer is a fast observer. For instance, if there's an SQL query within your observer code that's not essential or could be optimized, it might be worth revisiting.

Avoid Cyclical Event Loops

Avoiding cyclical event loops is another crucial best practice. In a cyclical event loop, an event triggers an observer which then dispatches another event, which then triggers the initial event, and so on. This can lead to significant performance issues. To prevent this, ensure that your observers are mindful of the events they're triggering and avoid creating unnecessary dependencies.

Do Not Rely on Invocation Order

Finally, when dealing with multiple observers in Magento 2, do not rely on their invocation order. The order in which observers are invoked in Magento 2 is not specified, so building dependencies based on order can lead to unpredictable results. Each observer should be designed to operate independently, regardless of when it's invoked in the sequence.

In conclusion, using Magento 2 observers effectively involves careful scope placement, lean and efficient coding, avoidance of cyclical event loops, and independent design to prevent reliance on invocation order. Following these best practices will help you optimize your Magento 2 observer use.

Come to an end

And there we have it - the first steps into Magento 2's intriguing observer architecture. Don't worry if it seems a bit overwhelming now; with each post in this series, the fog will clear, and the road will become more navigable. Remember, all great journeys start with a single step.

As we journey together, I'd love to connect with you on other platforms as well. You can follow me on Twitter for regular insights, quick tips, and all things Magento. Or join me on Github to explore my projects, learn from the code and maybe even contribute to some open source initiatives!

Stay tuned for the next post in this series, and don't forget, the world of Magento 2 is expansive and full of opportunities for those willing to explore. As always, happy Magento coding!

Understanding Magento 2 Modules: A Comprehensive Look into Key Class Types

David Lambauer — Thu, 15 Jun 2023 18:28:46 +0000

Greetings, Magento 2 enthusiasts. The life of a developer is laden with challenges, one of the most prominent being structuring and naming modules effectively. This task, albeit complex, is crucial in creating software that is easy to read, modify, and debug. Let's embark on an exploration of five pivotal class types that are instrumental in structuring Magento 2 modules efficiently.

1. Mapper Classes

The first class on our list is the Mapper Class. Mapper classes are invaluable tools that mediate and translate between different data types and structures. They provide an efficient way to convert data from one format to another, hence increasing the interoperability of your modules.

In a Magento context, Mapper classes can be utilized to translate data from a CSV file to a more structured format like RSS or even a product DTO (Data Transfer Object). The Mapper class ensures this conversion process is streamlined and efficient, reducing code redundancy and increasing maintainability.

class CsvToRssMapper {
    public function map($csvData) {
        // Conversion logic from CSV data to RSS format
    }
}

By encapsulating the mapping logic within its domain, a Mapper class promotes single responsibility and enhances code readability.

2. Validator Classes

Validator Classes play a crucial role in maintaining the integrity and consistency of data within the application. They enforce rules and constraints to ensure the incoming data adheres to the specified format and conditions.

For example, in an e-commerce store, a ProductValidator class might scrutinize whether the product data entered by the admin meets all requirements, such as non-empty product name and valid SKU, before it gets saved into the database.

class ProductValidator {
    public function validate($productData) {
        // Validate if the productData meets the requirements
    }
}

Validator classes not only safeguard the application from invalid data but also encapsulate validation logic, keeping the code clean and organized.

3. Command Classes (CQRS)

Command classes form a critical part of the Command Query Responsibility Segregation (CQRS) pattern, which segregates operations that mutate state (Commands) from operations that return data (Queries).

In a Magento store, you might have a command like "UpdateProductStockCommand" that takes a product SKU and the new stock quantity, updating the inventory.

class UpdateProductStockCommand {
    public function execute($sku, $newQuantity) {
        // Logic to update the stock quantity of the product
    }
}

Command classes, by focusing solely on state mutation, bring clarity to your code and reduce the likelihood of side effects in data manipulation operations.

4. Query Classes (CQRS)

Complementing Command classes in the CQRS pattern are Query classes. They are solely responsible for data retrieval and do not alter the state of the application.

For instance, a Query class in Magento could be "GetProductBySkuQuery," which fetches a product's data based on its SKU without affecting the underlying data.

class GetProductBySkuQuery {
    public function execute($sku) {
        // Fetch and return the product data using SKU
    }
}

By decoupling data retrieval from data mutation, Query classes enable safer and more efficient data handling, enhancing the performance and reliability of the application.

5. Service Classes

Service classes are the workhorses of any Magento application. They encapsulate business logic and serve as a bridge between the presentation and data access layers.

In a typical e-commerce scenario, a ProductService might orchestrate the process of creating a product, which could involve validating input data, mapping it to a product entity, and saving it to the database using a repository.

class ProductService {
    public function createProduct($productData) {
        // Business logic to create a new product
    }
}

Service classes bring structure and order to the application, encapsulating complex operations, and promoting code reusability and maintenance.

To sum up, understanding these five class types – Mapper, Validator, Command, Query, and Service – is essential to structure and name Magento 2 modules effectively. These classes, when utilized properly, can make your code more readable, maintainable, and scalable. Here's to writing cleaner and more efficient Magento 2 code!

Forem: run_as_root GmbH

Good-bye Docker, Hello Nix: Configuring a Magento 2 Development Environment with Rooter

Table of Contents

Introduction:

Requirements

Creating a Magento 2 dev environment

1. Creating the project directory

2. Entering in the created directory

3. Creating the environment with rooter

4. Allowing direnv to execute

5. Cloning the Magento 2 repository

6. Starting the environment stack

7. Setting up the database

Rooter Appendix

The .devenv directory

Versioning

Diving deeper into the most important files

.env file:

.envrc file:

devenv.nix file:

devenv.yaml file:

Accessing the traefik dashboard

Accessing the mailpit UI

Rooter commands

Xdebug

Frontend Performance Optimization tips | Magento 2 | Adobe Commerce

Some backstory

The beginning of the journey

The next step - the deep dive into the performance by myself

Decrease the DOM size whenever it's possible

Don't create unnecessary DOM elements

Try to avoid rendering elements that are not visible

Defer non-critical JS initialization and execution

Largest Contentful Paint (LCP) optimization

Conclusion

Should you upgrade your Magento store at all? Developer view

What does any upgrade bring us:

New features

What value does it bring to the project?

Price?

Do we need this value?

New Possibilities for Developers

What value does it bring to the project?

Price?

Do we need this value?

Bugfixes

What value does it bring to the project?

Price?

Do we need this value?

Security Fixes

What value does it bring to the project?

Price?

Do we need this value?

Agility - you can upgrade faster to all future NEW versions

What value does it bring to the project?

Price?

Do we need this value?

Sum Up

Cron Jobs in Magento 2: How to Adjust Schedules and Cron Groups Seamlessly

Introduction

Why Adjust the Cron Schedule?

Rescheduling Cron Jobs

The Need to Change Cron Group

Altering the Cron Job's Group

A Word of Caution on the Default Group

Conclusion

Preventing Transaction Loss: Unleashing the Power of Resilient Transactions with RabbitMQ dead-letter Exchanges in Magento 2

Table of Contents

What is RabbitMQ?

What is a dead-letter exchange?

Examples of transaction loss scenarios:

Practical case scenario

Crafting an Effective Retry Strategy for Queue Messages

Practical case scenario with queue retry

Adobe Commerce CLI commands - or how to create a problem out of nothing

Table Of Contents

Intro

The Underlying Problem & Its Roots

The Classic "Chicken or the Egg" Dilemma

The Pitfall of CLI Command Constructors

3. Creating the environment with `rooter`

The `.devenv` directory

`.env` file:

`.envrc` file:

`devenv.nix` file:

`devenv.yaml` file:

Accessing the `traefik` dashboard

Accessing the `mailpit` UI

8 Copy and rename the `env.warden.php` file

An Example of the content of this `set-default-config.sh` script:

11 Update your `/etc/hosts` file with the store domain:

10 Copy and rename the `env.warden.php` file into the etc folder

13 Append your `/etc/hosts` file with the store domains