Forem: Dzmitry Chubryk

Writing Custom PHPStan Rule to prohibit business logic in controllers

Dzmitry Chubryk — Fri, 21 Jul 2023 13:24:26 +0000

As the official website says - PHPStan scans your whole codebase and looks for both obvious & tricky bugs. Even in those rarely executed if statements that certainly aren't covered by tests.

To solve subject-problem we will use PHPStan static analysis tool. It has a powerful enough engine that allows us to find all controllers in our code where some business logic is stored. This way we will force developers to use services or actions. To implement this rule you must have PHPStan already installed and configured.

Let me remind you that PHPStan is a static code analyser. It means that it does not run code but only reads it. It performs a number of checks called rules. For example, if the code calls some method, it checks that its call matches the arguments of the method and that these arguments match in type. If any problems are found, the programme will report this in the final report.

You can find information how to install PHPStan on their official website, but the basic steps are:

Install the package via Composer: composer require --dev phpstan/phpstan
Create the phpstan.neon file in the root of your project
Run PHPStan: vendor/bin/phpstan

Example of base phpstan.neon file:

parameters:
    level: 9
    paths:
        - ./src/

Pay attention to the maximum validation level i.e. level: 9 to get all you can out of this tool and to keep the maximum strict validation - it's highly recommended to use this level.

Let's move on to writing rules.

A rule in PHPStan is a class that implements the PHPStan\Rules\Rule interface. You can read about how to write custom rules on the official website page.

<?php declare(strict_types=1);

namespace App\PHPStan\Rules;

use PhpParser\Node;
use PhpParser\Node\Stmt;
use PHPStan\Analyser\Scope;
use PHPStan\Rules\RuleErrorBuilder;
use PHPStan\Rules\Rule

/**
 * @implements Rule<Class_>
 */
class ProhibitBusinessLogicInController implements Rule
{
    public function getNodeType(): string
    {
        return Stmt\Class_::class;
    }

    public function processNode(Node $node, Scope $scope): array
    {
        $className = (string) $node->namespacedName;
        if (!$this->isController($className)) {
            return [];
        }

        foreach ($node->getMethods() as $method) {
            foreach ($method->getStmts() as $statement) {
                if ($this->isStatementWithBusinessLogic($statement)) {
                    return [
                        RuleErrorBuilder::message(sprintf(
                            'Method "%s::%s" contains business logic. Do something better.',
                            $className,
                            $method->name->toString(),
                        ))
                        ->build(),
                    ];
                }
            }
        }

        return [];
    }

    private function isController(string $class): bool
    {
        return str_ends_with($class, 'Controller');
    }

    private function isStatementWithBusinessLogic($statement): bool
    {
        return in_array($statement::class, [
            Stmt\If_::class,
            Stmt\For_::class,
            Stmt\Foreach_::class,
            Stmt\While_::class,
            Stmt\Switch_::class,
        ]);
    }
}

As you may notice, the rule should have two methods: getNodeType and processNode. It works similarly to an event dispatcher: you register an event type that interests you, and then you receive notifications when the event occurs. For your rule you register a node type that you need and when PHPStan encounters this node, it calls the processNode method. But what exactly is a "Node $node" in it?

Static code analysis means the possibility of passing through each element(node) of your code. E.g. a class is a separate node, and each class method is also a node. Every keyword or expression is also a node. Each PHP file can be represented by one large tree of certain nodes.

Inside PHPStan uses the PHP-Parser library to parse PHP files and create an AST(Abstract Syntax Tree) for each file. It then traverses this tree, asking each rule if it needs to process the current node or not (getNodeType). If needed, it passes the node further for processing (processNode). In return, we receive an array of errors if any were found.

In our example we will be analysing controllers, so we take the Class_ node type.

Next in the processNode method we need to determine whether the passed class is a controller or not. After all, our rule should only work with controllers and no other class.

Let's try to define what class we can consider a controller. Here we face a problem since it can vary from one project to another, and here I'm trying to simplify it and use a simple way to define a controller as the postfix "Controller".

Next we have to get all the statement of each controller method. If statement related to list of our forbidden statements (see isStatementWithBusinessLogic), then this method contains business logic.

To register our rule we need to write it in the phpstan.neon file:

  services:
    - class: App\PHPStan\Rules\ProhibitBusinessLogicInControllerRule
      tags:
        - phpstan.rules.rule

If you like TDD development PHPStan gives you a good opportunity to write tests for your custom rules. PHPStan rules are tested using PHPUnit.

Custom rule test code:

<?php declare(strict_types=1);

namespace Tests\PHPStan\Rules;

use App\PHPStan\Rules\ProhibitBusinessLogicInControllerRule;
use PHPStan\Rules\Rule;
use PHPStan\Testing\RuleTestCase;

class ProhibitBusinessLogicInControllerRuleTest extends RuleTestCase
{
    public function testSkipControllerWithNothingWrong(): void
    {
        $this->analyse(
            [
                __DIR__ . '/fixtures/skip-controller-if-everything-ok.php',
            ],
            []
        );
    }

    public function testControllerContainsForLoop(): void
    {
        $this->analyse(
            [
                __DIR__ . '/fixtures/controller-with-for-loop.php',
            ],
            [
                ['Method "WithForLoopController::indexAction" contains business logic. Do something better.', 4]
            ],
        );
    }

    /**
     * @inheritDoc
     */
    protected function getRule(): Rule
    {
        return new ProhibitBusinessLogicInControllerRule();
    }
}

As you can see from the example above all the tests accept the fixture and compare the response, whether an error is returned or not. If yes, then which one. I'll give example of fixture file fixtures/controller-with-for-loop.php.

<?php declare(strict_types=1);

class WithForLoopController
{
    public function __construct(
        private readonly Repository $repository
    ) { }

    public function indexAction(): array
    {
        $result = [];
        for ($i = 0; $i < 100; $i++) {
            $result[] = $this->repository->getIndex($i);
        }

        return $result;
    }
}

In this fixture we are testing the case where a for loop is used in the controller. Feel free to cover all "isStatementWithBusinessLogic" cases in your tests.

So, in this article we have written a fairly simple but functional rule for PHPStan. We have discussed the basics of PHPStan and now we can cover our project with more complex tests. I also recommend reading the developer documentation. It won't take you long to learn more rule development features.

Efficient counters storage for your entities in Redis

Dzmitry Chubryk — Thu, 06 Jul 2023 10:09:33 +0000

Once upon a time I needed to make an efficient solution that would allow me to store and add a massive amount of statistics on the ad view counter with the ability to view statistics by day.

Input parameters and restrictions:

Redis Cluster
10+ million entities with views to be counted
Time complexity on increment - O(1), on select - O(n)

Functions to be implemented:

Increase the view count
Get views for the last month
Get views for all time

I figured out right away that it could be done very effectively with simple hash stored keys, the major sticking point was how to effectively implement the cluster. Redis' ability to distribute keys across clusters via hash slots came in handy, allowing me to spread the load across the cluster evenly.

A bit of theory

Redis utilises hash slots to distribute keys across multiple nodes in a Redis Cluster. This distribution mechanism enables scalability and high availability. The allocation of hash slots is based on the key name of each Redis key.

When a Redis key is created, a hash function is applied to its key name to determine the hash slot. Redis employs a 2^14-slot hash space, where each slot is identified by a number from 0 to 16383. By using a hash function and the modulo operation, Redis maps the hash value of the key name to a specific hash slot.

Using the CRC16 algorithm as the default hash function. This algorithm ensures a well-distributed allocation of keys across the hash space. However, keys that contain curly braces {} introduce a concept called hash tags. Hash tags allow Redis to group keys with the same hash tag into the same hash slot, irrespective of the rest of the key name. For instance, if a key has the format {advert:314}:views, Redis only considers the hash tag advert:314 when calculating the hash slot. Consequently, keys like {advert:314}:bids or new:{advert:314}:bids will also be assigned to the same hash slot.

Hash slots play a crucial role in Redis Cluster's ability to distribute keys evenly among nodes, facilitating efficient data sharding and horizontal scaling. By considering the key name and incorporating hash tags, Redis ensures that related keys are stored on the same Redis node, optimising data access for logical groups of keys.

Implementation

The structure I used in this solution:
Key name: {advert:%d}:views
Data structure: hash map[string]int, where string is date in YYYYMMDD format and int is the counter.

Let's have a look at each item of functionality in practice.

Increase the view count

I used HINCRBY command to increment the counter for a particular date.

Example command:

redis> HINCRBY {advert:314}:views 20230606 1
---------- Response ----------
(integer) 1

Get views for the last month

To get last month's data I'm using HMGET command with a list of dates I want to get statistics for.

Example command:

redis> HMGET {advert:314}:views 20230606 20230605 20230604 20230603 20230602 20230601
---------- Response ----------
1) 1
2) 2
3) (nil)
4) 1
5) 6

Note that the list of values associated with the given fields in the same order as they are requested.

Get views for all time

The issue is much more difficult than the previous ones.
I could run 2 queries to get a list of fields from the key and iterate through them but I didn't like it. So I decided to use internal LUA script via EVAL command, it would be more efficient with Redis' client because it works with 1 query only.

Example command:

redis> EVAL "local totalView = 0
local dayViews = redis.call('hvals', KEYS[1])
for _, dayView in ipairs(dayViews) do
    totalView = totalView + tonumber(dayView)
end
return totalView" 1 {advert:314}:views
---------- Response ----------
(integer) 10

As a bonus I'll attach a ready-made PHP class with this implementation which was written using Illuminate Redis client from Laravel:

<?php declare(strict_types=1);

namespace App\Repositories;

use App\Contracts\Entity;
use Illuminate\Redis\RedisManager;

class ViewStatsRepository
{
    private const
        DATE_FORMAT = 'Ymd',
        VIEW_STATS_DAYS = 30
    ;

    public function __construct(
        private readonly RedisManager $redis
    ) { }

    public function getTotalViewsCount(Entity $entity): int
    {
        $evalScript = <<<LUA
local totalView = 0
local dayViews = redis.call('hvals', KEYS[1])
for _, dayView in ipairs(dayViews) do
    totalView = totalView + tonumber(dayView)
end
return totalView
LUA;

        return (int) $this->redis->eval($evalScript, 1, $this->viewCountKey($entity));
    }

    public function getTodayViewsCount(Entity $entity, DateTimeInterface $now): int
    {
        return $this->getViewsCountByDate($entity, $now);
    }

    public function getViewsCountByDate(Entity $entity, DateTimeInterface $now): int
    {
        return (int) $this->redis->hget($this->viewCountKey($entity), $now->format(self::DATE_FORMAT));
    }

    public function getViewStats(Entity $entity, DateTimeInterface $now): array
    {
        $keys = [];
        for ($i = self::VIEW_STATS_DAYS - 1; $i >= 0; $i--) {
            $keys[] = $now->modify("-{$i} days")->format(self::DATE_FORMAT);
        }

        $stats = array_combine($keys, array_values($this->redis->hMGet($this->viewCountKey($entity), $keys)));

        return array_map('intval', $stats);
    }

    public function incrementViewsCount(Entity $entity, DateTimeInterface $now, int $incrementBy = 1): int
    {
        return (int) $this->redis->hIncrBy(
            $this->viewCountKey($entity),
            $now->format(self::DATE_FORMAT),
            $incrementBy
        );
    }

    private function viewCountKey(Entity $entity): string
    {
        return sprintf('{advert:%d}:views', $entity->getId());
    }
}

Useful links to enhance your knowledge:

How we test code in a fintech startup with PHP

Dzmitry Chubryk — Sun, 02 Jul 2023 22:36:39 +0000

I often hear developers justify themselves with different arguments as to why they don't write tests.

The most popular of these are:

I'm sure my code is written without bugs
Writing tests is difficult
Businesses don't want to spend money on writing tests
I don't understand how to write them
Here can be your option

Just forget about it! Tests are the mandatory part of your code and your responsibility as a software engineer.

In this article I'd like to describe how we test our code in the fintech start-up, which allows us to guarantee a high level of confidence that any regression is safe for us.

Let's start with some basic concepts and principles

Why do you need unit-tests?

Tests help make your production more stable
- regression testing
- contract fixing
Tests increase the usability of development
- faster feedback
- simpler refactoring
Tests organise poorly written code
- make you think about test cases
- make you think about code quality

What does a quality unit-test look like?

easy to understand
fail on bugs
resistant to refactoring
fast
environment independent

What does a normal test look like?

hard to understand
doesn't fail on bugs
any refactoring forces you to fix the tests
slow
environment dependent

So how do you write a quality unit-test?

Use contracts. A unit contract is the unit's expectations about you and also your expectations about the unit. Hence your code must be contractual if you want to test it with unit-tests.

The contract terms you need to test are:

Useful work (e.g. database calls or other third party services)
Expected result
Result on boundary conditions
Exception conditions

Try to test your class contract, not its implementation.

The functions you unit-test must be clean, otherwise your unit-tests will test your function's implementation.
A clean function a.k.a. deterministic function - function always give the same output (Y) for an input (X). Nondeterministic functions produce variable outputs. Deterministic means the opposite of randomness, giving the same results every time and without interacting with the environment in any way, for example:

Input: [1, 2, 3, 4] > array_reverse > Output: [4, 3, 2, 1]

This is the example of a good unit-test which tests a clean function:

final class ItemGrouperTest extends TestCase
{
    public function testGroupReturnsGroupedItemsGroupedByNameAndCurrency(): void
    {
        $itemGrouper = new ItemGrouper();

        $items = [
            $item1 = new Item(1, 'Nike AF 1', 'GBP', 13000),
            $item2 = new Item(1, 'Nike AF 2', 'GBP', 13000),
            $item3 = new Item(1, 'Nike AF 3', 'EUR', 14000),
            $item4 = new Item(2, 'Nike AF 1', 'GBP', 13000),
        ];

        $groupedItems = $itemGrouper->group(
            ItemGrouper::GROUP_BY_NAME & ItemGrouper::GROUP_BY_CURRENCY,
            ...$items
        );

        $this->assertCount(3, $groupedItems);

        $this->assertCount(2, $groupedItems[0]->getItems());
        $this->assertContains($item1, $groupedItems[0]->getItems());
        $this->assertContains($item2, $groupedItems[0]->getItems());

        $this->assertCount(1, $groupedItems[1]->getItems());
        $this->assertContains($item3, $groupedItems[1]->getItems());

        $this->assertCount(1, $groupedItems[2]->getItems());
        $this->assertContains($item4, $groupedItems[2]->getItems());
    }
}

We know the behaviour of our method, we give the input data and we check the expected result, perfect! This is the test I would always like to see in a deterministic function.

But what about functions that aren't clean?

These functions are usually called in service classes or as they are also named class-manager. They perform more complex operations: call one or more external services, aggregate results of these external services, use results through chain calls, etc.

Here is the description of such class:

A controller class that calls the external service userRepository to get the list of users, then this list of users is passed through usersFormatter class to prepare a response.

If we follow the basic rules for these classes, we can easily write such tests.

The basic rules are:

All external dependencies must be mocked.
We test the external dependency solely by contract: we check the call, the expected result and the exception conditions.

Let's write the tests for given class above:

final class UserControllerTest extends TestCase
{
    public function testListActionReturnsSuccessfulJsonResponseAndCallsAllRelatedDependencies(): void
    {
        $userController = new UserController(
            $userRepository = $this->createMock(UserRepository::class),
            $usersFormatter = $this->createMock(UsersFormatter::class),
        );
        $userController->setContainer($this->createMock(ContainerInterface::class));

        $request = $this->createMock(Request::class);

        $userRepository->expects($this->once())
            ->method('getList')
            ->willReturn($users = [
                $this->createMock(User::class),
                $this->createMock(User::class),
            ]);

        $usersFormatter->expects($this->once())
            ->method('format')
            ->with(...$users)
            ->willReturn($formatterResponse = [
                'key' => 'any response here',
                'why' => 'cuz we dont case about result of 3-rd party services here, just any expected results'
            ]);

        $response = $userController->listAction($request);

        $expectedJson = json_encode($formatterResponse, JsonResponse::DEFAULT_ENCODING_OPTIONS);

        $this->assertSame($expectedJson, $response->getContent());
    }
}

Conclusion

Try to write more deterministic methods that are easily covered by tests.
Non-deterministic methods shouldn't be left untested, simple checks for expected behaviour can save your code from unexpected bugs and errors.
And remember the better your code is covered by tests, than more you are protected from future bugs and refactoring.

Developing Modular Monolith vs. Traditional Monolith in Software Engineering, pros and cons

Dzmitry Chubryk — Thu, 22 Jun 2023 13:07:05 +0000

Monolithic architecture has been a widely adopted approach for developing complex applications. However, as technology evolves, engineers have explored alternative architectures to address scalability, maintainability, and deployment challenges. Two prominent approaches are developing a modular monolith and a traditional monolith. In this article, we will examine the pros and cons of each approach to help software engineers make informed decisions when choosing between them.

Understanding Traditional Monolith:
A traditional monolithic architecture is characterised by an application composed of tightly-coupled modules that are deployed and scaled as a single unit. All components, such as the user interface, business logic, and data access layer, reside within the same codebase and are developed, tested, and deployed together. While this approach simplifies development and deployment, it can lead to challenges as the application grows larger and more complex.

Understanding Modular Monolith:
Modular monolith, also known as a monolith with bounded contexts or microservice-like architecture, aims to address some of the limitations of a traditional monolith while retaining its benefits. In this approach, the monolithic application is split into loosely-coupled modules, each representing a distinct domain or bounded context. These modules can be developed and deployed independently, leveraging the advantages of modular design while avoiding the complexities of a distributed system.

Pros of Modular Monolith:

Simplified Development: modular monoliths provide a balance between the simplicity of traditional monoliths and the modularity of microservices. Developers can work on independent modules with well-defined boundaries, enabling faster development and testing cycles.
Easier Deployment and Scalability: while a traditional monolith requires deploying the entire application even for small updates, modular monoliths allow for individual module deployment. This granular deployment approach simplifies versioning, reduces downtime, and offers better scalability options.
Improved Maintainability: with modular monoliths, the separation of concerns makes it easier to maintain and update specific modules without affecting the entire system. Teams can focus on individual modules, reducing the risk of regression bugs and ensuring faster maintenance and bug fixes.
Shared Resources and Reduced Overhead: Modules within a modular monolith can share resources such as databases, caching layers, and infrastructure, reducing duplication and operational overhead. This can lead to more efficient resource utilisation and cost savings.

Cons of Modular Monolith:

Limited Independence: while modular monoliths promote independent development and deployment, they are still tightly coupled compared to true microservice architectures. Changes in one module may require coordination with other modules, which can limit the true potential of independent development.
Complexity with Scaling: modular monoliths can face challenges when it comes to scaling specific modules. As the size of individual modules increases, it becomes harder to scale them independently, potentially requiring scaling of the entire monolith.
Increased Complexity in Testing: testing a modular monolith requires considering inter-module dependencies, which can complicate the testing process. Ensuring proper integration testing and maintaining a comprehensive test suite across modules becomes crucial.
Challenges in Inter-module Communication: communication between modules within a modular monolith introduces complexity. Developers need to establish well-defined communication patterns and enforce proper contracts to maintain loose coupling and minimize unintended dependencies.

Choosing between a modular monolith and a traditional monolith depends on the specific requirements of your project. While modular monoliths offer benefits such as simplified development, easier deployment, and improved maintainability, they also come with challenges in terms of limited independence and increased complexity in testing and communication. Consider the size and complexity of your application, scalability requirements, and development team’s expertise when making this architectural decision.

How does a modular monolith work in practice?

Let's break down the example of an online shop domain.

Consider the following project structure:

Basic rules for transferring information between domains:

there must be a single point of communication with the domain, in our case it is *LocalClient
all data that is transferred from the domain must be in a shared space, see Shared namespaces.

By following these rules you can isolate the logic of certain domains, leaving the place of communication only through the local client.
Here's a roughly example of what an OrderService might look like that communicates with other services exclusively through their local clients.

namespace ModularMonolith\OrderDomain\Services;

use ModularMonolith\OrderDomain\Services\Exceptions\ProductNotAvailableException;
use ModularMonolith\OrderDomain\Shared\Models\OrderDto;
use ModularMonolith\ProductDomain\Shared\Models\ProductDto;
use ModularMonolith\UserDomain\Shared\UserLocalClient;
use ModularMonolith\WarehouseDomain\Shared\WarehouseLocalClient;

class OrderService {
    public function __construct(
        private readonly UserLocalClient        $userLocalClient,
        private readonly WarehouseLocalClient   $warehouseLocalClient,
    ) { }

    /**
     * @param ProductDto[] $products
     *
     * @throws ProductNotAvailableException
     */
    public function createOrderForUser(string $userUuid, array $products): OrderDto {
        $user = $this->userLocalClient->getUserByUuid($userUuid);

        foreach ($products as $product) {
            $productUuid = $product->getUuid();
            if (!$this->warehouseLocalClient->isProductAvailable($product->getUuid())) {
                throw new ProductNotAvailableException($productUuid);
            }

            $this->warehouseLocalClient->withdrawProduct($product->getUuid());
        }

        //to do something to store order to certain $user
        //$this->saveOrder($user, $products);
    }
}

Using a modular monolith is a great way to prepare your project for the future transition to microservices.

For a deeper understanding, I recommend reading the book

amazon.co.uk

Cheers.