Forem: Matteo Kovačić

5 reasons why we ended up rewriting the whole application before its release

Matteo Kovačić — Mon, 22 Nov 2021 23:24:57 +0000

As developers, sooner or later we will end up in a situation where we have to work with a legacy codebase. That can be the whole application or just part of the application written at the very start of the project. It's hard to fight legacy as we don't know how each part of the application will grow and at which scale. Making everything easy to scale is not very efficient so you need to guess which parts of the application are most likely to be improved in the future.
But on one of our projects, we faced a unique challenge where we created a legacy even before the initial release. For quite some time we tried to fix it or improve it, but in the end, we decided that we should rewrite the whole project from scratch.

Here are the top 5 reasons why we ended up doing it and changes we made in a rewrite.

1. Unknown territory

Before this one, we successfully developed more than 100 projects and we thought this should be just one more of them. The initial estimate was much bigger compared to anything else we were working on before. Just a few months into the project there were already red flags. The team was not comfortable with the chosen technologies as they had little experience with them. From the project very beginning, we used agile methodology in contrast to a waterfall on our previous projects. We started with very little documentation (as agile suggests) and worked from sprint to sprint. When the project grew in size, we faced constant refactors of our previous work to introduce new features. The combination of these few problems drastically increased feature estimates.
We also didn't have a lot of experience with the business domain, so we needed client input to discover even the basic information about their day-to-day work.

To fight these challenges, a few new roles were introduced: product manager, business analyst, and tech lead. Some were given to the experienced people already on the project, but some required completely new colleagues. Also, we decided to create a project specification with all the features, and a long-term development plan. This plan enabled us to develop features with upcoming features in mind. Together with all the organizational changes, we introduced new processes and improved old ones. This was supported by new tooling (Jira, Confluence) and integrations with tools already in our stack (GitHub, CI:CD).

2. Technology choice

One of the main reasons why we faced some walls during development is technology choice for a project of such size. Our core technologies were not a problem, but JSON:API as API standard gave us plenty of issues. We had a lot of actions on top of domain models and this didn't work very well with the resource-based approach. In addition to this, maintaining API documentation took us too much time. Generators helped with new resources, but not so much with modifying old ones.

In a rewrite, we swapped JSON:API with GraphQL which gave us much-needed flexibility in communication between API and front-facing app. Accessing the data remained very similar to what we used to in JSON:API, but mutations allowed us to modify data in a more action-focused way.

3. Architecture

We were very comfortable with MVC, so we decided to keep it in this project as well. The only difference is that we switched to a headless architecture where a back-end team develops API and a front-end team develops a front-facing app. When the project scaled, folder structure became very messy and there were no restriction to use a logic of multiple domain contexts in the same place. With everything mentioned, adding new features to such codebase became very hard even with a good code coverage in tests.

This was one of the hardest challenges to fix as we had to change it from the ground up. After some research and a few prototypes, we decided to use DDD and Layered architecture principles in our codebase. It gave us a very descriptive folder structure in every bounded context and boundaries for cross-context communication.

4. Onboarding difficulty

In a project with a complex domain and messy codebase with no clear structure, onboarding of new colleagues was very difficult. It took us around 3 months before a new person on the project became productive. As this was a long-term project, it's expected that from time to time we will need to onboard a new person, and losing too much time on it is something we wanted to avoid.

Just by implementing a new architecture, everything mentioned here was drastically improved. We had an option to onboard people only in one specific domain context and they could finish their tasks without worrying about breaking other parts of the application.

5. Long-term motivation

Every developer wants to work on an interesting, challenging, well-structured project with little to no stress. With everything we were facing, it was very hard to keep a good team spirit. Instead of feeling proud on finding the solution for a complex domain problem, we faced technical challenges where domain logic was very simple. This made us feel like we are not progressing as we should.

Clear hierarchy, roles, and responsibilities in combination with modern technologies drastically improved team morale. To ensure it stays this way, after every sprint, we grade our happiness and try to figure out what could be improved.

Conclusion

It's never an easy decision to abandon everything you were building for years, but sometimes this is the best possible decision. It took us 10 months to rewrite everything and during this process, we didn't face any major blockers or technical limitations. The team became more stable and fueled by the same goal. By looking at the project's future, the rewrite was the best decision we could make. The mistake was that we didn't do it earlier.

How to execute multiple GraphQL mutations in a transaction using Overblog GraphQL Bundle

Matteo Kovačić — Tue, 06 Jul 2021 11:43:26 +0000

When we started working with GraphQL, we experienced common issues like N+1 problem, error handling, etc. As our API started growing, new challenges occurred. They were mostly related to the schema design and how to have a schema that is easy to read and also functional.

By specification, GraphQL allows us to send multiple mutations in a single request and these mutations will be executed in the order in which they were sent. GraphQL specification doesn't guarantee us that if some of these mutations fail, all others will fail as well. In other words, GraphQL doesn't specify that all mutations should be executed in the transactions. Luckily, this can be easily enabled in the Overblog GraphQL Bundle using built-in events.

First, in services.yaml, you need to register the event listener for two events: graphql.pre_executor and graphql.post_executor.

services:
    ...

    App\EventListener\GraphQLExecutorListener:
        tags:
            - { name: kernel.event_listener, event: graphql.pre_executor, method: onPreExecutor }
            - { name: kernel.event_listener, event: graphql.post_executor, method: onPostExecutor }

Then, in the event listener, you have a method for each event. In onPreExecutor we start the transaction. After mutations are resolved, in onPostExecutor we commit the changes if there are no errors or rollback if an error occurred.

class GraphQLExecutorListener
{
    private EntityManagerInterface $entityManager;

    public function __construct(EntityManagerInterface $entityManager)
    {
        $this->entityManager = $entityManager;
    }

    public function onPreExecutor(): void
    {
        $this->entityManager->getConnection()->beginTransaction();
    }

    public function onPostExecutor(ExecutorResultEvent $event): void
    {
        $connection = $this->entityManager->getConnection();

        if ($connection->isTransactionActive() === false) {
            throw new LogicException('Transaction not active');
        }

        if (count($event->getResult()->errors) > 0) {
            while ($connection->getTransactionNestingLevel() > 0) {
                $connection->rollback();
            }

            return;
        }

        $connection->commit();
    }
}

That's it, all mutations should now be executed in the transaction.
If necessary, the event listener can be extended to conditionally enable the transaction, for example using a header or some other flag.

Access control in GraphQL using Symfony

Matteo Kovačić — Tue, 02 Feb 2021 20:48:24 +0000

Authorization is part of almost any web application and controlling access to specific data is essential for application security. The same goes for GraphQL APIs and with help of Overblog's GraphQL Bundle, this can be done easily.

Field access control

Every GraphQL API has at least one root type, Query. Root types are the most common place where we would want to control access by setting rules to a specific field. Some examples could be controlling access to some admin-related queries by allowing only users with ROLE_ADMIN role or allowing access to user query only if a requested user is currently authenticated user or has ROLE_ADMIN role. This can be done using expression language functions in the field configuration option called resolve.

Using `hasRole`

This expression language function is provided by the bundle and is self-explanatory - it checks if the currently authenticated user has the role you provide as an argument.

Query:
  type: object
  config:
    fields:
      activityLog:
        type: "[Activity!]!"
        access: "@=hasRole('ROLE_ADMIN')"
        resolve: "@=resolver('ActivityLog')"

Using `isGranted`

This function is not documented in the official documentation, but it actually exists if you look closely in the codebase. Sometimes checking role is not sufficient and we want complex logic to determine if a user has access or not. This can be done using voters and isGranted expression language function.

Query:
  type: object
  config:
    fields:
      user:
        type: 'User'
        access: "@=isGranted('user_access', args['id'])"
        args:
          id:
            type: 'ID!'
        resolve: "@=resolver('User', [args['id']])"

If you have any questions, comments or experiences with using GraphQL you'd like to share, put them in the comments section below!

Using Pusher Channels with Symfony

Matteo Kovačić — Mon, 26 Oct 2020 11:10:34 +0000

In modern web applications, updating parts of the pages in real-time is a common requirement. There are many options, from more low-level open-source projects like socket.io to fully managed services like PubNub, Ably or Pusher Channels.

After trying a few options, here at Bornfight we chose Pusher Channels for all our ongoing projects. Some of the key reasons are:

Pusher is present for a long time and Channels are their most stable and oldest product
Private channels with auth per channel which allows authorization on the fly
Developer friendly client and server libraries in almost every modern language
Free sandbox and fair pricing for paid plans

Creating a new Pusher app

To follow this tutorial, you will need a free sandbox account which you can create here. After you have your account ready, from the channels dashboard you can create a new app.

Installation

If you still don’t have Symfony project ready, you can initialize one:

composer create-project symfony/website-skeleton my_project_name

Pusher provides the official PHP library for interaction with its API. The library can be installed from the root project directory of your newly created project:

composer require pusher/pusher-php-server

The library also has a Symfony recipe which will create a new service and add environment variables to .env file.

Configuration

The recipe gave us a good starting point, but it’s a good practice to have service definitions only in services.yaml. Following this, we can remove a file config/packages/pusher_php_server.yaml generated by the recipe and add a new service to services.yaml

services:
    ...

    Pusher\Pusher:
        arguments:
            $auth_key: '%env(PUSHER_KEY)%'
            $secret: '%env(PUSHER_SECRET)%'
            $app_id: '%env(PUSHER_APP_ID)%'
            $options:
                cluster: '%env(PUSHER_CLUSTER)%'
                useTLS: true

In your .env file, the recipe already added a new pusher section with 3 environment variables.

###> pusher/pusher-php-server ###
PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
###< pusher/pusher-php-server ###

Because we also use PUSHER_CLUSTER, add it at the end of the section and add values with your app credentials provided in the app dashboard under App Keys section.

After adding values, your .env file should look something like:

###> pusher/pusher-php-server ###
PUSHER_APP_ID=12345
PUSHER_KEY=your-key
PUSHER_SECRET=your-secret
PUSHER_CLUSTER=eu
###< pusher/pusher-php-server ###

Now you have configured your service and you can use is it with dependency injection.

Demo

To verify that everything works, you can create a simple demo page.

Use a maker to create a new controller:

php bin/console make:controller DemoController

In the newly generated controller, add the following two methods:

/**
 * @Route("/demo", name="demo", methods={"GET"})
 */
public function index(): Response
{
    return $this->render('demo/index.html.twig');
}

/**
 * @Route("/demo/say-hello", name="demo_say_hello", methods={"POST"})
 */
public function sayHello(Pusher $pusher): Response
{
    $pusher->trigger('greetings', 'new-greeting', []);

    return new Response();
}

Now we only need to modify our template which should already be created by maker. In templates/demo/index.html.twig add the following content and replace your-key with the key you already added as PUSHER_KEY environment variable.

{% extends 'base.html.twig' %}

{% block title %}Pusher Demo!{% endblock %}

{% block body %}
    <button type="button" id="say-hello-button">Say Hello!</button>
{% endblock %}

{% block javascripts %}
    <script src="https://code.jquery.com/jquery-3.5.1.min.js"></script>
    <script src="https://js.pusher.com/7.0/pusher.min.js"></script>
    <script>
        var pusher = new Pusher('your-key', {
            cluster: 'eu'
        });

        var channel = pusher.subscribe('greetings');
        channel.bind('new-greeting', function() {
            alert('Hello!');
        });

        $('#say-hello-button').click(function () {
            $.post("{{ path('demo_say_hello') }}");
        });
    </script>
{% endblock %}

Your demo should be ready to go. Start Symfony built-in server with symfony server:start and navigate to /demo page with at least two browser windows or tabs. After clicking on the button, you should get an alert on all opened tabs.

Conclusion

You have a lot of options when choosing technology for real-time updates and Pusher is one of them. You get a lot of low-level WebSocket issues handled out of the box, without compromising a great number of features and flexibility.

If you already use Pusher, what do you think about it? Or if not, which technology do you use to send real-time updates to your clients?

Uploading files with GraphQL in Symfony

Matteo Kovačić — Mon, 03 Aug 2020 09:09:07 +0000

File uploads are a common thing in web development and you can most often see it in action when you try to change your account’s profile picture. GraphQL doesn’t have a file as a type defined by the specification, but a GraphQL community found a solution.

There is a spec that’s widely used in the Apollo ecosystem with apollo-upload-client and apollo-server, which allows us to send multipart requests to the GraphQL server. In the Symfony world, Overblog’s GraphQLBundle is one of the most popular options with support for both Apollo and Relay. This bundle also supports file uploads out of the box by providing Upload type and resolving it to UploadedFile object that’s commonly seen in Symfony when working with uploaded files.

Define your schema

In this example we will work with a single Author entity that consists of 3 fields: firstName, lastName and pictureFilename. GraphQL type definition should have exactly the same fields.

Author:
  type: object
  config:
    fields:
      id:
        type: "ID!"
      firstName:
        type: "String!"
      lastName:
        type: "String!"
      profileFilename:
        type: "String!"

Here, we also need only one mutation with arguments for required data and Author type as output. Instead of the pictureFilename defined in Author type, here we will use picture as an argument name.

Mutation:
  type: object
  config:
    fields:
      CreateAuthor:
        type: 'Author!'
        resolve: '@=mutation("App\\GraphQL\\Mutation\\AuthorMutation::createAuthor", [args["firstName"], args["lastName"], args["picture"]])'
        args:
          firstName:
            type: String!
          lastName:
            type: String!
          picture:
            type: AuthorPictureUploadFile!

As you can see, we used custom type for the profile picture field and that is where most of the magic happens. Now, the only thing missing in the schema is to add a definition for it by extending the Upload type provided by bundle.

AuthorPictureUploadFile:
  type: custom-scalar
  config:
    scalarType: '@=newObject("Overblog\\GraphQLBundle\\Upload\\Type\\GraphQLUploadType")'

Handling files in mutation

Now, when we have the schema defined, we can add the mutation logic. In the Mutation type definition, we already specified our mutation class and how it will pass arguments to its createAuthor method. Because the bundle will transform uploaded files to UploadedFile objects before passing them to the mutation, we can easily move them to the desired folder and use the filename to populate entity data.

class AuthorMutation implements MutationInterface
{
    ...

    public function createAuthor(string $firstName, string $lastName, UploadedFile $picture): Author {
        $filename = uniqid('author_picture_').'.'.$picture->guessExtension();
        $picture->move($this->uploadsPath, $filename);

        $author = new Author();
        $author->setFirstName($firstName);
        $author->setLastName($lastName);
        $author->setPictureFilename($filename);

        $this->entityManager->persist($author);
        $this->entityManager->flush();

        return $author;
    }
}

Conclusion

Uploading files with GraphQL is simple and easy with multipart specification widely supported by Apollo and its ecosystem. Even without the official specification regarding uploads, there are a few options available, but the one shown in this post is probably the simplest. Additionally, Symfony also has a great GraphQL bundle to provide server-side runtime with features like batching, file uploads and much more.

If you have any questions, comments or experiences with using GraphQL you'd like to share, put them in the comments section below!

Forem: Matteo Kovačić

5 reasons why we ended up rewriting the whole application before its release

1. Unknown territory

2. Technology choice

3. Architecture

4. Onboarding difficulty

5. Long-term motivation

Conclusion

How to execute multiple GraphQL mutations in a transaction using Overblog GraphQL Bundle

Access control in GraphQL using Symfony

Field access control

Using hasRole

Using isGranted

Using Pusher Channels with Symfony

Creating a new Pusher app

Installation

Configuration

Demo

Conclusion

Uploading files with GraphQL in Symfony

File uploads are a common thing in web development and you can most often see it in action when you try to change your account’s profile picture. GraphQL doesn’t have a file as a type defined by the specification, but a GraphQL community found a solution.

Define your schema

Handling files in mutation

Conclusion

Using `hasRole`

Using `isGranted`