Forem: ManoMano Tech Team

5 Atlassian MCP Hacks to supercharge your workflow

Danielo Artola — Thu, 16 Oct 2025 12:35:56 +0000

I hate to be switching over and over between the IDE, the task in Jira to review details, Confluence, or other docs to ensure I have the details, finding sometimes outdated docs, so I was curious how I could save time on that through AI. Every time we jump from writing code to updating a ticket or checking a requirement, we lose focus and momentum.

After some weeks testing, configuring, failing, and enjoying the process, I bring you here 5 situations that I found useful and saved me time. Your time is also important, so I added TL;DR in all the sections!

The case: During a recent migration of a microservice from Node.js to Kotlin, I put this to the test. Basically, the AI should help not only to do a plan for migration, but also we needed ideas for a safe rollout, a list of all the endpoints, business rules, external dependencies, and adapt the plan for the team size, etc. I created a single workspace in my IDE containing the legacy Node.js project, the new Kotlin boilerplate, and another small service we were absorbing to provide all the context I thought relevant for once I start prompting in the IDE. By giving the agent access to this full context, along with a link to our "Backend Guidelines" page in Confluence, I could kickstart the documentation process. A migration is a big topic, with several steps:

Create a plan to migrate it, and add it to somewhere easy to edit, share, review with the team, update, comment -> a Confluence page
Divide the work according to the team size to achieve in a determined time -> a Feature in Jira with all the tasks related
Follow our internal guidelines and avoid generic results -> use the current documentation in Confluence to consider them in the code
Ensure the information in the tickets: Edit tasks descriptions, info, and adding comments with the info in the code
Code fast small features or tech debt -> Implementing a Jira ticket

1. Documentation on Autopilot: From Code to Confluence

TL;DR: Use the AI to analyze projects (or specific pull requests) to generate or update technical documentation in Confluence, provide it with the best context, ask it to improve the prompt; this will help to have your docs sync with your code with minimal effort.

Generating and updating documentation is a task many of us postpone or sometimes defer to the bottom of the backlog. With an integrated AI agent, you can delegate a significant portion of this work.

Why in a document? Because it's easier to read and review than in the agent chat, and I can edit what I want to precise or correct easily.

Why a conversation? Because I involved the agent in the decisions.

Why did you mention a conversation? Because I shared my idea with Gemini 2.5 Flash in the web to help me generate the best prompt for the Cursor Agent (which primarily runs on Gemini 2.5 Pro for coding tasks) with my ideas, and I asked it to ask me questions to ensure the best result.

After refining the prompt and editing some things, I ended with a big prompt with around 4k characters. It’s big, but needed to have a good start. Having the output in Confluence helped me to edit easily some sections, add or remove parts, add comments in some parts, and copy-paste some of the text to ask the AI if that was an assumption or something in the code (and guess what, there were some wrong assumptions, AI is not perfect, folks, yet).

Here is the final, structured prompt used to generate the migration plan:

You are a Senior Solutions Architect tasked with creating a comprehensive 
and actionable migration plan. 

Project: Migrate the existing [Legacy Service Name] microservice 
(Node.js/TypeScript, GraphQL, REST) to the new [Target Service Name] microservice 
(Kotlin, Ktor, GraphQL). 

Primary Goal: Complete the migration and have the new service fully operational 
in Production within a three-month timeline. 

Key Context & Constraints: 
Team: The project will be executed by a team of 4 backend developers. 
Technology: We must maintain GraphQL compatibility. 
The goal is to reuse the existing GraphQL schema and as much of the domain logic 
and DTOs as possible to ensure a smooth transition for frontend clients. 
Development Velocity: 
The plan should incorporate the use of AI-powered code generation tools 
to accelerate the translation of business logic from TypeScript to Kotlin 
and to generate boilerplate code. 
Standards: The entire plan, architecture, and proposed implementation must 
strictly adhere to the best practices documented in the [Your Company Name] 
Backend Guidelines found here: [Link to Backend Guidelines] 

Required Plan Structure: Please generate the plan with the following 
distinct sections: 
1. Executive Summary 
2. Gap Analysis & Feature Parity Strategy 
3. Development & AI-Assisted Strategy 
4. Phased Rollout Strategy (Zero-Risk Production Deployment) 
5. Monitoring, Logging, and Alerting Strategy ([Observability Platform]) 
6. Timeline: Sprints & Milestones (3 Months) 
7. Risks & Dependencies. 

Write the output in **Confluence**, in the space for 
[Wiki Space ID] > [Parent Page], called "[Project Name] Node <> KT migration".

🔖 Add this to your prompts to improve results:

Ask me any questions you find relevant have better information.

Then add them to a Google Doc or similar, it will be easier to answer. Then just copy all the question and answers

‼️ I prefer to be specific with the destination link. Vague instructions can sometimes lead the AI to edit the wrong page. Double-check the target, I edited wrong documentation because there were two pages with the same title in different folders!

2. Intelligent Task Creation, Right From the Codebase

TL;DR: Define high-level constraints (time, team size) to have the AI decompose epics into manageable tasks. When you spot bad code, select it and instantly create a detailed tech debt ticket in Jira.

Whether you're breaking down a massive epic or flagging unexpected tech debt, you can create perfectly detailed Jira tickets without ever leaving your code.

For the migration project, I tasked the AI with planning. Here is the prompt used to decompose the epic into manageable, one-week tasks:

Act as a senior project manager responsible for planning a microservice migration.

**Context:**
Based on the migration plan from Node.js to Kotlin for the [microservice A] to [microservice B] that we just finalized in our previous conversation, your task is to generate a detailed list of Jira tasks required to execute this plan.

**Jira Target Information:**
- Jira Feature/Epic Link to associate tasks with: [Link to Feature/Epic]

**Rules and Constraints:**
1.  **Adherence to Plan:** All tasks must directly correspond to the phases and steps we previously agreed upon.
2.  **Team & Timeframe:** Remember, our team consists of 4 developers and the target timeframe is 12 weeks.
3.  **Task Granularity:** No single task should exceed one week of effort for a single developer. If a step from our plan is too large, break it down into smaller, logical sub-tasks.
4.  **Dependencies:** Identify and order the tasks logically.

**Final Step (Review):**
Before you execute the creation in Jira, please present the full list of proposed tickets here in the chat for my review and final approval. This is a dry run.

After a bit of back-and-forth, I gave the final command to execute the creation in Jira.

A more powerful feature is on-the-fly tech debt flagging. We've all been there: you're working on a feature and stumble upon a piece of convoluted "spaghetti code." Instead of letting it slide, I now select the code block and prompt the agent:

Create a tech debt ticket in Jira to refactor this. 
Explain that it has high complexity and doesn't follow our guidelines. 
Reference the file @path/to/the/problematic/file.js 
in the description and propose a solution explaining the why and how.

Using the @ symbol (formerly # in some tools) to reference specific files is always better if you want to tag several files, try always to provide the best context for best results. It gives the AI precise, unambiguous context, leading to faster and more accurate results. You're not making it search; you're telling it exactly where to look. This focused context approach is a key feature of modern AI IDEs, allowing you to scope the AI's attention precisely.

3. Weaving Confluence Knowledge into Your Code

TL;DR: Provide the AI with links to relevant Confluence pages (e.g., coding guidelines, business logic) to use as context when generating or refactoring code.

How many times have you had to dig through Confluence to find that one specific business rule or coding standard? I don’t remember 100% of things, and I hate to be searching some specific info in Confluence, switching over and over screens. Now, you can bring that knowledge directly to the agent that writes your code.

When I need the AI to generate code that must adhere to complex business logic documented in Confluence (or follow several internal guidelines), I start a conversation by providing the link. I first verify it has understood the content with a prompt like

Summarize the key business rules from this document regarding user authentication.

Once I confirm the agent has "ingested" the knowledge correctly, I can proceed with my code generation request, confident that it will respect the documented constraints.

🔖 If you don’t trust the AI answers 100% like me, I recommend you to ask something like

Explain to me how you took the info provided and how you adapted it 
and considered it in the code

This is currently the best way to provide external knowledge through Atlassian MCP. While Atlassian Intelligence is moving towards a future where the AI can automatically index and understand your entire Confluence space, the "link and verify" method is the best practice at the moment of writing this.

4. The IDE as Your Jira Cockpit

TL;DR: Use natural language commands in your IDE to add comments to tickets, change their status, or clarify descriptions, drastically reducing the need to switch to the Jira web UI.

Small, repetitive Jira actions add up, pulling you out of your coding mindset. You can execute most of these tasks with a single command from your editor.

The actions that save me the most time are:

Adding Contextual Comments: Highlighting a discrepancy in the code and telling the agent, "Add a comment to ticket XYZ explaining that the current implementation doesn't fit our business rules and requires extra development not considered in the task."
Changing Status: Simple commands like Move this ticket to 'In Progress' and give me a summary of the app or Move this ticket to 'Code Review' and push the code (I don’t waste tokens just to move a ticket, I do it with a relevant action).
Clarifying Ticket Descriptions: Sometimes a ticket's description is too vague or you forgot to mention something, or you have to correct a wrong assumption (yes, all of that sometimes happens). After discovering the necessary technical details, I'll ask the agent to update it. This is a great practice for enriching the ticket with implementation details.

‼️ However, be mindful of when editing a description is appropriate.

Good Practice (Clarification): Adding technical notes, like "The new field must be encrypted in the database using AES-256." This is the precise command used for ticket refinement:

Edit the description removing assumptions and adding only things in the code

Bad Practice (Scope Creep): Changing the core requirement, like turning "Add a 'First Name' field" into "Add 'First Name' and 'Date of Birth' fields and a new validation flow." The latter should be a new ticket.

Jira ticket created with generic description, generated in a bundle with several tasks:

A different task, but this with detailed description after asking to improve it with code, links and relevant info:

5. From Jira Ticket to Code, and Beyond with Automation

TL;DR: For simple and well-scoped Jira tickets, you can ask the AI to generate the code just by providing the ticket link. This opens the door to automating entire parts of your development lifecycle.

For well-defined, simple tasks, you can go from a Jira ticket link to functional code in minutes. This is where we can start thinking about true workflow automation.

This works best for tasks like:

Adding a new optional field to an API endpoint.
Changing UI label text for clarity or rewording.
Adding missing translations.
Generating unit tests for a function or improving test coverage.

🔖 The prompt is as simple as

Implement the changes described in this Jira ticket: [link].

The most important thing is to ensure a good description in the ticket.

This leads to the next logical step: automation. For example, here is a powerful multi-action prompt that links code analysis, documentation, and task creation:

Document the endpoints of the [Legacy Service Name] Node.js project for migration. 

Follow these steps to generate a **Confluence** page and a **Jira** task. 

1. **Confluence** Page Documentation: 
Generate a new page at the URL: [link here] 
For each endpoint in the [Legacy Service Name] project, create a new section with a descriptive title. Within each section, provide the following details: Route, Method, Parameters, Response Example, OpenAPI Link. 
At the end of the page, create a sortable table with the following columns: Status, Method, Endpoint, and Link to openapi. Use the sortable table macro or a similar feature if available in **Confluence** to enable sorting. 

2. **Jira** Task Creation: 
Create a new **Jira** task with the following specifications: 
Parent Epic: [link here] 
Title: Endpoints to migrate from [Legacy Service Name] 
Description: Include a TLDR section with a brief summary of the task. After the TLDR, include the same table you generated for the **Confluence** page.

This is the next frontier: a truly automated, intelligent development environment.

6. Setting Up Your AI Cockpit: MCP Atlassian Integration

TL;DR: Install the specific Atlassian Marketplace Connect Program (MCP) agent/plugin in your preferred editor (Cursor, IntelliJ, VS Code) and authenticate with your Atlassian workspace.

As always, there are requirements, some steps to follow depending on the IDE you use, etc. Here you'll see the instructions for all the IDEs. Let's see the example for Cursor

A. Cursor

Since Cursor is built with AI capabilities from the start, the integration is often seamless. The Cursor agent uses Gemini 2.5 Pro as its underlying model for the deep reasoning and code generation tasks described. (you also can add it easily from the cursor mcp list).

Open Settings: Navigate to Settings (usually Cmd+, or Ctrl+,).
Find Integrations: Look for "MCP".
Connect Atlassian: Select the Atlassian integration option (Jira/Confluence) and follow the prompts to sign in with your Atlassian credentials. This connects the internal atlassian-mcp-server to your agent, granting it read access to your permitted documentation and ticketing.

Yes, if you have troubles installing the mcp in your ide, you can ask to your favourite model in your IDE copy pasting the errors or explaining the problems! Or, of course, you can comment in this post and we'll try to help

Conclusion

Integrating Atlassian's AI tools into your IDE isn't about replacing the developer. It's about augmenting our abilities by removing friction, having the developer focus on development, and avoid the satellite tasks that are repetitive and from what you learn little to nothing. By handling the overhead of documentation, task management, and context gathering, it allows us to dedicate our most valuable resource—our focused brainpower—to what we do best: building bold and ingenious software.

What are your favorite AI-driven workflows? Share them in the comments below!

AI created, Human-enhanced.

Daniel Artola.

Wiremock + testcontainers + Algolia + Go = ❤️

Grégoire Paris — Fri, 20 Jun 2025 08:16:02 +0000

When dealing with a SaaS like Algolia, testing can be a hassle. Ideally, you should not "mock what you do not own". In other words, you should not mock libraries such as the Algolia SDK, not just because it might evolve in unforeseen ways, but also because writing unit tests for a piece of code where the logic is dictated by something external to the code is not a good idea: you would not be testing the part that has the most complexity.

To take a concrete example, let's imagine you want to index documents in Algolia. There is an end goal behind that, and the end goal is that it is possible to search for these documents.

Ideally, you would have a Docker container running Algolia locally that would be super fast at indexing and use the same code your production Algolia app uses, but sadly that does not exist, and I'm not hopeful it ever will.

In a legacy service I worked on, we have a test Algolia app that we use for integration tests. It worked great, but in the past years, Algolia introduced a new cloud-based architecture, and with this architecture, an indexing task can take a lot more time to be "published". As a result, using a test application on the cloud-based architecture is not an option anymore, as it slows the test suite down to a crawl. 🐌

On a new project, I decided to re-evaluate my options, and remembered a tool that seems to be the next best thing for the job: Wiremock.

In this post, I will guide you through the process of setting Wiremock and testcontainers to test Algolia's own quickstart guide for Golang.

Wiremock: a VCR for HTTP

Wiremock is a testing tool that comes with a so-called "record and
playback" feature.

It means you can do this once in your local environment:

┌────────────┐          ┌────────────────────────────┐       ┌─────────┐
│            ├─────────►│                            ├──────►│         │
│Your service│          │ Wiremock in recording mode │       │ Algolia │
│            │◄─────────┤                            │◄──────┤         │
└────────────┘          └────────────────────────────┘       └─────────┘

In recording mode, you give Wiremock a URL to record, and it will store files representing the requests you made, and the corresponding responses. With Algolia, it can be quite long, especially if you wait for operations.
What happens in practice is that the SDK will use a polling mechanism to check if your task is published. This will result in a lot of similarly looking files.
This is not very interesting to reproduce in your test, so I recommend simply deleting files representing a negative response to the question: "are the changes published yet?". Those typically contain a JSON field called status set to notPublished in their body, like so:

{"status": "notPublished", "pendingTask":false}

When the file is published, this becomes:

{"status": "published", "pendingTask":false}

The files have names that are a bit ugly, so I usually rename them for clarity.
For example, you might rename 1_indexes_test-index_task_226434943725-6e8689fa-9bbb-43fb-9d24-6824c02fc7d5.json
to index_test_task_published.json.

Once your recording is done, you can run your tests like this:

┌────────────┐          ┌───────────────────────────┐
│            ├─────────►│                           │
│Your service│          │ Wiremock in playback mode │
│            │◄─────────┤                           │
└────────────┘          └───────────────────────────┘

In playback mode, Wiremock will respond to your request with the mappings it has stored previously, and pretend to be Algolia. 🥸

While this does not shield you against breaking changes in the Algolia HTTP API, it does come with a few advantages:

It shields you against breaking changes or bugs in the Algolia SDK.
You no longer have to mock the SDK, which is a bad practice and a pain to do. A consequence of that is that your tests become easier to understand, and more expressive, and that they check things at a higher level rather than focusing on implementation details.
It still means that at least once, you do run the tests against the real thing, so if there is some issue that can only be detected at runtime, you will know about it.

Wiremock is a java application, but that shouldn't matter too much, especially given there is an official Docker image you can use.

Testcontainers: Docker for your tests

At ManoMano, we use Gitlab CI. While it is possible to define a Gitlab CI service with the aforementioned Docker image, that's not a great solution because Gitlab services do not expose the full power of Docker. For instance, mounting a volume is not possible, probably not without heavy involvement of privileged users.

A great alternative is testcontainers + testcontainers Cloud. Testcontainers is a library available in many languages that allows you to start and stop Docker containers during your tests, making it possible to get good isolation between tests.
Testcontainers Cloud is a service that allows you to run said containers on a remote infrastructure, as opposed to running them on your own infrastructure, which, if you want to use Kubernetes runners for Gitlab, implies using Docker in Docker, which is not great from the security standpoint.
Locally, you would still use a local docker container, but in the CI,
tescontainers will send requests to testcontainers cloud, to start and stop containers. Enough unpaid endorsement, let's get to the code.

Demo time

Let us follow Algolia's quickstart guide for Golang and see how easy it is to
test.

If you want to follow along, install mise-en-place and let's go!

For the sake of brevity, I will not systematically show the entirety of a file I edit in all snippets, however I have tried to create one commit per step in this Github repository, in case you would like to play with the code or simply read it in your own editor.

Installing Go

$ mise use go@1.24

Creating a new project

$ go mod init algolia-wiremock-testcontainers

Installing the Algolia SDK

$ go get github.com/algolia/algoliasearch-client-go/v4

Setting up the environment

At this point, you will need to set up a test Algolia application. Once you are done, you should have an application ID and an API key.

Let us use an unversioned env file to store our credentials.

# mise.toml
[env]
_.file = ".env"

# .env
ALGOLIA_APP_ID=changeme
ALGOLIA_API_KEY=changeme

You will need to replace ALGOLIA_APP_ID and ALGOLIA_API_KEY with values from your account.

# .gitignore
/.env

Writing the code to be tested

Let us take the code from Algolia's quickstart guide and split it into two files:

First, we have the code under test where the only changes are getting the environment variables from the actual environment, and renaming packages and functions.

// indexer.go

package indexer

import (
    "os"

    "github.com/algolia/algoliasearch-client-go/v4/algolia/search"
)

func indexRecord() {
    // Get Algolia credentials from environment variables
    appID := os.Getenv("ALGOLIA_APP_ID")
    apiKey := os.Getenv("ALGOLIA_API_KEY")
    indexName := "test-index"

    record := map[string]any{
        "objectID": "object-1",
        "name":     "test record",
    }

    // Create a new Algolia client
    client, err := search.NewClient(appID, apiKey)

    if err != nil {
        panic(err)
    }

    // Add record to an index
    saveResp, err := client.SaveObject(
        client.NewApiSaveObjectRequest(indexName, record),
    )

    if err != nil {
        panic(err)
    }

    // Wait until indexing is done
    _, err = client.WaitForTask(indexName, saveResp.TaskID)

    if err != nil {
        panic(err)
    }
}

To make it work, you will need to install the Algolia SDK:

$ go get github.com/algolia/algoliasearch-client-go/v4
$ go mod tidy

That call to WaitForTask is what is going to take the most time, and a good reason not to use a real Algolia instance in your test suite. That's what we are going to try first though.

Writing the test with a real Algolia instance

Let's start simple and write a first version of the test that talks directly to Algolia:

// indexer_test.go

package indexer

import (
    "os"
    "testing"

    "github.com/algolia/algoliasearch-client-go/v4/algolia/debug"
    "github.com/algolia/algoliasearch-client-go/v4/algolia/search"
)

func TestIndexRecord(t *testing.T) {
    // helps with seeing the progress, since this is super long
    debug.Enable()

    appID := os.Getenv("ALGOLIA_APP_ID")
    apiKey := os.Getenv("ALGOLIA_API_KEY")
    indexName := "test-index"

    client, err := search.NewClient(appID, apiKey)
    if err != nil {
        t.Fatalf("Failed to create client: %v", err)
    }

    // Create index indirectly by setting settings
    _, err = client.SetSettings(client.NewApiSetSettingsRequest(
        "test-index",
        search.NewEmptyIndexSettings().SetSearchableAttributes(
            []string{"name"},
        ),
    ))

    if err != nil {
        t.Fatalf("Failed to set settings: %v", err)
    }

    t.Cleanup(func() {
        // Ensure the test index is deleted after the test completes
        _, err := client.DeleteIndex(client.NewApiDeleteIndexRequest(
            indexName,
        ))
        if err != nil {
            t.Fatalf("Failed to delete index: %v", err)
        }
    })

    // Call the function under test
    // this should index a record in Algolia
    indexRecord()

    // Verify the record was indexed by searching for it
    // This search should return record with "test" in their contents
    // The quickstart guide currently uses a more complex version of this
    searchResp, err := client.SearchSingleIndex(
        client.NewApiSearchSingleIndexRequest(indexName).
            WithSearchParams(
                search.SearchParamsObjectAsSearchParams(
                    search.NewEmptySearchParamsObject().
                        SetQuery("test"),
                ),
            ),
    )

    if err != nil {
        panic(err)
    }

    // Assert that there are hits
    if len(searchResp.Hits) == 0 {
        t.Fatal("No hits found")
    }
}

aaaaand that doesn't work:

panic: The maximum number of retries exceeded. (50/50) [recovered]
        panic: The maximum number of retries exceeded. (50/50)

goroutine 7 [running]:
testing.tRunner.func1.2({0x800600, 0xc00028d640})
        /home/gregoire/.local/share/mise/installs/go/1.24.2/src/testing/testing.go:1734 +0x21c
testing.tRunner.func1()
        /home/gregoire/.local/share/mise/installs/go/1.24.2/src/testing/testing.go:1737 +0x35e
panic({0x800600?, 0xc00028d640?})
        /home/gregoire/.local/share/mise/installs/go/1.24.2/src/runtime/panic.go:792 +0x132
algolia-wiremock-testcontainers.indexRecord()
        /home/gregoire/Documents/blogging/wiremock/indexer.go:39 +0x166
algolia-wiremock-testcontainers.TestIndexRecord(0xc000198540)
        /home/gregoire/Documents/blogging/wiremock/indexer_test.go:40 +0x20a
testing.tRunner(0xc000198540, 0x8a6c10)
        /home/gregoire/.local/share/mise/installs/go/1.24.2/src/testing/testing.go:1792 +0xf4
created by testing.(*T).Run in goroutine 1
        /home/gregoire/.local/share/mise/installs/go/1.24.2/src/testing/testing.go:1851 +0x413
FAIL    algolia-wiremock-testcontainers 185.745s
FAIL

I have many applications on this instance, some of which are very busy, let us patch that real quick:

// Wait until indexing is done
_, err = client.WaitForTask(
    indexName,
    saveResp.TaskID,
    search.WithMaxRetries(100),
)

Exactly the type of thing that unit tests will not catch.

After that, the test passes (but it takes between several seconds or several minutes to run depending on how busy the instance on which the application is running is). Great! Now, let's add a proxy in the middle, and record all this.

Adding Wiremock in record mode 📼

We are using Docker, so if we want to obtain the so-called "mapping files" Wiremock will create, we need to mount a volume on our Docker container, and mount it in the right location.

Let us add 2 new dependencies to our project:

We could interact with Wiremock by calling the REST API with the net/http package, but as it turns out, there is a dedicated SDK for that, and it supports recording since this pull request I sent. Let's install it.

$ go get github.com/wiremock/go-wiremock@v1.13.0

Next, we will need a way to start and stop the Wiremock container, and for that
as well, there is a library:

$ go get github.com/wiremock/wiremock-testcontainers-go@v1.0.1

Let us start the container, with a volume mounting testdata in the current directory on /home/wiremock/mappings in the container. This is where Wiremock will create json files.

// indexer_test.go

// for some reason wiremock doesn't like the testing context
ctx := context.Background() 

absolutePath, err := os.Getwd()
if err != nil {
    t.Fatalf("Failed to get current working directory: %v", err)
}

// Start the Wiremock container,  using the testcontainers library
container, err := testcontainers_wiremock.RunContainerAndStopOnCleanup(
    ctx,
    t,
    testcontainers.WithHostConfigModifier(func(
        hostConfig *container.HostConfig,
    ) {
        hostConfig.Binds = []string{
            absolutePath + "/testdata:/home/wiremock/mappings",
        }
    }),
    testcontainers_wiremock.WithImage("wiremock/wiremock:3.12.1"),
)

if err != nil {
    t.Fatalf("Failed to create wiremock container: %v", err)
}

// The endpoint changes every time, so we need to obtain it at runtime
host, err := container.Endpoint(ctx, "")

if err != nil {
    t.Fatalf("Failed to get wiremock container endpoint: %v", err)
}

Next, we need to change how we instantiate the Algolia client, so that it calls Wiremock instead of Algolia:

algoliaClient, err := search.NewClientWithConfig(
    search.SearchConfiguration{
        Configuration: transport.Configuration{
            AppID:  appID,
            ApiKey: apiKey,
            Hosts: []transport.StatefulHost{
                transport.NewStatefulHost(
                    "http",
                    host,
                    func(k call.Kind) bool {
                        return true
                    },
                ),
            },
        },
    },
)

if err != nil {
    t.Fatalf("Failed to create client: %v", err)
}

Note that I have renamed the client to algoliaClient to avoid confusion with the Algolia client and the Wiremock client.

Let us also refactor our indexRecord() function to take the client as an argument:

// indexer.go

func indexRecord(client *search.APIClient) {
    indexName := "test-index"

    record := map[string]any{
        "objectID": "object-1",
        "name":     "test record",
    }

    saveResp, err := client.SaveObject(
        client.NewApiSaveObjectRequest(indexName, record),
    )

    if err != nil {
        panic(err)
    }

    _, err = client.WaitForTask(
        indexName,
        saveResp.TaskID,
        search.WithMaxRetries(100),
    )

    if err != nil {
        panic(err)
    }
}

Next, let's start the recording, and for that we need a client to call Wiremock's administration API:

// indexer_test.go

wiremockClient := wiremock.NewClient("http://" + host)
t.Cleanup(func() {
    err := wiremockClient.Reset()
    if err != nil {
        t.Fatalf("Failed to reset wiremock: %v", err)
    }
})

// Call Wiremock's administration API to start recording
err = wiremockClient.StartRecording(fmt.Sprintf(
    "https://%s-dsn.algolia.net",
    appID,
))

if err != nil {
    t.Fatalf("Failed to start recording: %v", err)
}

t.Cleanup(func() {
    // Call Wiremock's administration API
        // to stop recording and dump the mappings
    err := wiremockClient.StopRecording()
    if err != nil {
        t.Fatalf("Failed to stop recording: %v", err)
    }
})

// code that interacts with Algolia must be after

Now, let's run our tests again, check our testdata directory, and see what's new.

$ ls -1 testdata
1_indexes_test-index-4c8f560d-010a-4db9-916c-f5c112481fc8.json
1_indexes_test-index-e766b4bf-6087-499b-975f-722c451f1d4a.json
1_indexes_test-index_query-e08439bf-fd60-4fac-8181-11b72f7a7a1c.json
1_indexes_test-index_settings-9de043d2-5803-40a0-b19b-49d0d2a17dde.json
1_indexes_test-index_task_228603771246-0004a7a6-3759-452c-8d0d-ea2a2f948ea1.json
1_indexes_test-index_task_228603771246-012f5d71-2645-496c-ba68-b85dcafbce51.json
1_indexes_test-index_task_228603771246-050f8624-f6f5-4815-96ed-03f319cdbda0.json
1_indexes_test-index_task_228603771246-05510684-a31f-41a6-96aa-d722a7527e87.json
1_indexes_test-index_task_228603771246-07651b21-23b0-45b4-9e13-386775c37432.json
1_indexes_test-index_task_228603771246-0bfcad3d-4d19-403f-9197-2fe6214beeb2.json
1_indexes_test-index_task_228603771246-0d450b99-1af0-4761-8d92-d7b92a68c702.json
1_indexes_test-index_task_228603771246-0dd66ddc-d7e2-4f70-8a3f-f5934ade7ac1.json
1_indexes_test-index_task_228603771246-1490e160-63e7-466d-9405-9437efd31c68.json
1_indexes_test-index_task_228603771246-1d25eb69-2a0f-448d-ae60-7075c380837d.json
1_indexes_test-index_task_228603771246-1f675d54-54bc-41cc-a3b4-f3dcc153cb0b.json
1_indexes_test-index_task_228603771246-1f829991-f442-45b6-9463-6ad064eb7576.json
1_indexes_test-index_task_228603771246-22837672-68b3-4684-a1f0-5bf19a8a3e8d.json
1_indexes_test-index_task_228603771246-25162e07-d196-4527-b476-173b7d62acf7.json
1_indexes_test-index_task_228603771246-26087497-e8ee-429a-a98e-a0100219c112.json
1_indexes_test-index_task_228603771246-264ce294-058a-482e-8d33-8be46740c7e9.json
1_indexes_test-index_task_228603771246-272cd045-d145-4b26-86b7-accad674a2db.json
1_indexes_test-index_task_228603771246-2ac3189e-9737-46df-a1b5-269e63a4d36a.json
1_indexes_test-index_task_228603771246-30a023df-b21b-43e7-b1a0-3f3c6ceec6d4.json
1_indexes_test-index_task_228603771246-315a9105-a169-4228-9d6a-7f20e9db3e4a.json
1_indexes_test-index_task_228603771246-3358c4a2-9c5d-4c7f-982f-1ca49c413b08.json
1_indexes_test-index_task_228603771246-34ef8259-1204-4026-9c1d-d43731a8d489.json
1_indexes_test-index_task_228603771246-37be5ee5-2f56-43a9-afe3-866859945d72.json
1_indexes_test-index_task_228603771246-3ba76a13-6052-4b9d-a7f2-61d3d364714d.json
1_indexes_test-index_task_228603771246-3be6df0a-3bcc-4104-81e0-28462875ed86.json
1_indexes_test-index_task_228603771246-3da6c29e-5603-4262-bfa8-37a06b021297.json
1_indexes_test-index_task_228603771246-3e321244-de3c-420e-978e-60b995839ca2.json
1_indexes_test-index_task_228603771246-41ed4f34-d05b-4e30-a9da-fd1b84d85aaa.json
1_indexes_test-index_task_228603771246-420e2d2c-2aa0-4afd-9cdd-5ac6ecc7bd20.json
1_indexes_test-index_task_228603771246-4220ae1a-447c-419f-86f9-6ee5bc40a121.json
1_indexes_test-index_task_228603771246-42b9612d-005a-4e5e-bb8c-e0e431790404.json
1_indexes_test-index_task_228603771246-450ab804-2980-478f-b1f2-c9cef5ba021c.json
1_indexes_test-index_task_228603771246-472b813d-f05a-4bca-bc14-c8f0b085388c.json
1_indexes_test-index_task_228603771246-4802a463-b49f-4ddf-83cd-49af7615c66a.json
1_indexes_test-index_task_228603771246-49ed408e-d531-4bc5-a711-2f1e4acc4ae2.json
1_indexes_test-index_task_228603771246-4a9c9104-f042-4f9a-a56b-dc4a9976bde7.json
1_indexes_test-index_task_228603771246-4ae60200-dbed-4c1b-878f-6dcf53cf4954.json
1_indexes_test-index_task_228603771246-4df81e83-7ce3-4ed1-9102-48c987a675de.json
1_indexes_test-index_task_228603771246-4e11ef64-ade6-40ca-b5c0-8a35684f73cd.json
1_indexes_test-index_task_228603771246-5383a9a9-7023-41f9-8092-09bbde387e7d.json
1_indexes_test-index_task_228603771246-55f8777f-8ff1-4fdb-ae35-86bd8d1641f6.json
1_indexes_test-index_task_228603771246-581cce11-3bbe-4492-b9b2-4bb4db47e50f.json
1_indexes_test-index_task_228603771246-58a94473-05f6-460d-b595-87d67a6389cc.json
1_indexes_test-index_task_228603771246-5aadb748-f27e-40b5-8166-33e61de59888.json
1_indexes_test-index_task_228603771246-5eabca6c-637d-4a3d-a883-9a0ac8d40449.json
1_indexes_test-index_task_228603771246-628fb430-b307-4257-8a23-becfcd8c1649.json
1_indexes_test-index_task_228603771246-660b5321-6c43-41e1-83d8-02f936a09bc9.json
1_indexes_test-index_task_228603771246-6b15a9eb-886f-43ff-af1e-2b82a8dacdf2.json
1_indexes_test-index_task_228603771246-6c610574-d0cf-4272-9260-cd5c42954b1c.json
1_indexes_test-index_task_228603771246-6e0d4b06-c3dc-47b8-858b-1d66ed65f708.json
1_indexes_test-index_task_228603771246-70303855-6eb1-465e-949b-40c85e6b5d2e.json
1_indexes_test-index_task_228603771246-712fc20e-8877-43e0-998b-3408c85a1645.json
1_indexes_test-index_task_228603771246-71ede81b-df18-4f39-97ea-757967a84599.json
1_indexes_test-index_task_228603771246-72c592d9-6d61-4a22-bf3e-3d2559b319a8.json
1_indexes_test-index_task_228603771246-72de2fd6-675c-42e6-85d0-21b3947b3fc1.json
1_indexes_test-index_task_228603771246-74ca6363-881f-49b8-8dac-efd6f5c8d449.json
1_indexes_test-index_task_228603771246-75f4709b-ef34-4eed-9d2e-002a3237a880.json
1_indexes_test-index_task_228603771246-773d2b22-6e45-4b0b-943b-b8e261c38d9b.json
1_indexes_test-index_task_228603771246-7accc591-6809-4483-9b39-578557dcabd3.json
1_indexes_test-index_task_228603771246-7bd2b559-9f16-4c57-92bb-0020763419b1.json
1_indexes_test-index_task_228603771246-837f6509-a8b8-4f0a-98fd-4f3b82349acc.json
1_indexes_test-index_task_228603771246-8875e9eb-07f3-4565-9e45-9db6b8c4a662.json
1_indexes_test-index_task_228603771246-8b464ab1-bef7-4975-972a-c472bfca9a90.json
1_indexes_test-index_task_228603771246-8c42ea24-0b42-4ed3-91a3-7cbce16828e0.json
1_indexes_test-index_task_228603771246-932fe93f-2121-4b1a-bedb-334a4518b986.json
1_indexes_test-index_task_228603771246-9697ecdf-4079-4af0-9df5-2d3f44f2b26c.json
1_indexes_test-index_task_228603771246-9bbbf58d-a4ea-47ba-af43-d7a455cc333a.json
1_indexes_test-index_task_228603771246-9d4fbd76-3f4c-4605-aa88-49d105bf718c.json
1_indexes_test-index_task_228603771246-9f126ad6-8a7a-4a37-80ef-7528c62f939d.json
1_indexes_test-index_task_228603771246-9f247f3d-9ba1-447f-a9d1-f61919a40d65.json
1_indexes_test-index_task_228603771246-a01ab516-b606-47b2-b4db-c1188fd9527a.json
1_indexes_test-index_task_228603771246-a057d6b6-00fe-40ea-aee8-bc0873bfe66d.json
1_indexes_test-index_task_228603771246-a460d1b7-6d5f-46b4-9196-a0a49c6c9e10.json
1_indexes_test-index_task_228603771246-a7c15221-1a8f-48b2-9c20-c90bdb6f0074.json
1_indexes_test-index_task_228603771246-a90bdd7d-7be0-428b-973f-1ba429c10f99.json
1_indexes_test-index_task_228603771246-b06c842b-102e-4ca9-99b2-38d838166480.json
1_indexes_test-index_task_228603771246-b2fbbe73-e3cb-49ee-a008-05a3eb66c93d.json
1_indexes_test-index_task_228603771246-b4a8ce95-12f3-4786-9dca-e989f86873ef.json
1_indexes_test-index_task_228603771246-baf4c6a3-ac78-44df-9e5d-8798e9dcf1ff.json
1_indexes_test-index_task_228603771246-bee44a4d-b090-4ebf-b2c7-3b85a0c92000.json
1_indexes_test-index_task_228603771246-c367963c-3aca-41dd-a570-c3135702c841.json
1_indexes_test-index_task_228603771246-c4f8d672-f2ed-4b37-ad51-04ceacf8f3e0.json
1_indexes_test-index_task_228603771246-cbf506c8-70c9-406a-afaa-7db6b7212345.json
1_indexes_test-index_task_228603771246-cc952870-85f4-4057-bb42-24940e3a9050.json
1_indexes_test-index_task_228603771246-d46e09e2-0634-40c6-b121-9c488000a698.json
1_indexes_test-index_task_228603771246-d85de2d1-2dbd-472f-b7d7-e288f833867c.json
1_indexes_test-index_task_228603771246-d998fa67-ef81-407d-9ced-4549b63be22e.json
1_indexes_test-index_task_228603771246-dbebd7ae-ef9a-41b4-9f67-8e8cb871522f.json
1_indexes_test-index_task_228603771246-dec87bcc-2c17-4f38-97dc-f00d3346a00b.json
1_indexes_test-index_task_228603771246-e1228862-9f69-4224-8f46-135b307edd5a.json
1_indexes_test-index_task_228603771246-e4ea7ed5-6f42-4eb7-a54c-f25609c54f3a.json
1_indexes_test-index_task_228603771246-e4fdbcf5-1941-44bd-87bb-8f7c9b0c4718.json
1_indexes_test-index_task_228603771246-ea16fd95-c2ca-474f-a4d2-e556c5bd158c.json
1_indexes_test-index_task_228603771246-f657c74c-0e47-4a6b-887d-e003ba7118c6.json
1_indexes_test-index_task_228603771246-fa7feea5-229a-49af-ac9d-f4754539eb55.json
1_indexes_test-index_task_228603771246-fb081872-590e-443a-8c8b-957ac58fa541.json
1_indexes_test-index_task_228603771246-ff880e8d-f1cf-43ad-9e21-dfe5dab7a3c7.json

… OK that is quite a lot of files. 😅 As mentioned earlier, a lot of them are about polling.

Let's find the one that we should keep:

$ grep -i published testdata/*task*

testdata/1_indexes_test-index_task_228603771246-0004a7a6-3759-452c-8d0d-ea2a2f948ea1.json:    "body" : "{\"status\":\"notPublished\",\"pendingTask\":false}",
testdata/1_indexes_test-index_task_228603771246-012f5d71-2645-496c-ba68-b85dcafbce51.json:    "body" : "{\"status\":\"notPublished\",\"pendingTask\":false}",
…
testdata/1_indexes_test-index_task_228603771246-3ba76a13-6052-4b9d-a7f2-61d3d364714d.json:    "body" : "{\"status\":\"published\",\"pendingTask\":false}",
…
testdata/1_indexes_test-index_task_228603771246-fb081872-590e-443a-8c8b-957ac58fa541.json:    "body" : "{\"status\":\"notPublished\",\"pendingTask\":false}",
testdata/1_indexes_test-index_task_228603771246-ff880e8d-f1cf-43ad-9e21-dfe5dab7a3c7.json:    "body" : "{\"status\":\"notPublished\",\"pendingTask\":false}",

After removing the files with notPublished, we are left with the following mapping files:

$ ls -1 testdata
1_indexes_test-index-4c8f560d-010a-4db9-916c-f5c112481fc8.json
1_indexes_test-index-e766b4bf-6087-499b-975f-722c451f1d4a.json
1_indexes_test-index_query-e08439bf-fd60-4fac-8181-11b72f7a7a1c.json
1_indexes_test-index_settings-9de043d2-5803-40a0-b19b-49d0d2a17dde.json
1_indexes_test-index_task_228603771246-3ba76a13-6052-4b9d-a7f2-61d3d364714d.json

Switching to playback mode 📺

Now that we have our mapping files, we can switch to playback mode. Let us introduce a constant to turn recording and Algolia debugging on and off:

// indexer_test.go

const record = false

// …

if record {
    debug.Enable()

    err = wiremockClient.StartRecording(fmt.Sprintf(
        "https://%s-dsn.algolia.net",
        appID,
    ))

    if err != nil {
        t.Fatalf("Failed to start recording: %v", err)
    }

    t.Cleanup(func() {
        err := wiremockClient.StopRecording()
        if err != nil {
            t.Fatalf("Failed to stop recording: %v", err)
        }
    })
}

Note that I also moved the call to debug.Enable() to the recording block, when replaying the tests, we do not really need to clutter the output with Algolia debug information.

And now the test fails, with a rather clear error: apparently deleting the files was not enough, and we need to also edit the scenario name to outline that this is no longer the 43rd attempt.

--- FAIL: TestIndexRecord (1.87s)
panic: API error [404]
                                                       Request was not matched
                                                       =======================

        -----------------------------------------------------------------------------------------------------------------------
        | Closest stub                                             | Request                                                  |
        -----------------------------------------------------------------------------------------------------------------------
                                                                   |
        1_indexes_test-index_task_226434943725                     |
                                                                   |
        GET                                                        | GET
        /1/indexes/test-index/task/226434943725                    | /1/indexes/test-index/task/226434943725
                                                                   |
        [Scenario                                                  | [Scenario                                           <<<<< Scenario does not match
        'scenario-1-1-indexes-test-index-task-226434943725'        | 'scenario-1-1-indexes-test-index-task-226434943725'
        state:                                                     | state: Started]
        scenario-1-1-indexes-test-index-task-226434943725-43]      |
                                                                   |
        -----------------------------------------------------------------------------------------------------------------------
         [recovered]
        panic: API error [404]
                                                       Request was not matched
                                                       =======================

        -----------------------------------------------------------------------------------------------------------------------
        | Closest stub                                             | Request                                                  |
        -----------------------------------------------------------------------------------------------------------------------
                                                                   |
        1_indexes_test-index_task_226434943725                     |
                                                                   |
        GET                                                        | GET
        /1/indexes/test-index/task/226434943725                    | /1/indexes/test-index/task/226434943725
                                                                   |
        [Scenario                                                  | [Scenario                                           <<<<< Scenario does not match
        'scenario-1-1-indexes-test-index-task-226434943725'        | 'scenario-1-1-indexes-test-index-task-226434943725'
        state:                                                     | state: Started]
        scenario-1-1-indexes-test-index-task-226434943725-43]      |
                                                                   |
        -----------------------------------------------------------------------------------------------------------------------

After dropping "requiredScenarioState" : "scenario-1-1-indexes-test-index-task-226434943725-43", from the mapping file about polling, the test passes again, only this time, it passes in under 2 seconds.
It is possible to mention which scenario a mapping belongs to, allowing to do things like "On the first 2 calls respond A, and on the 3rd return B". Based on that, it is possible to build a complex choreography of requests/responses, fulfilling all sorts of requirements.

Making it work in the CI

After pushing the code, I got a bad surprise: the test fails in the CI, with the following message:

tc-wiremock.go:73: create container: container create: Error response from daemon: Invalid bind mount config: mount source "/builds/product-discovery/ms.indexer/internal/import/brandsuggestion/testdata" is forbidden by the allow list [/home /tmp] - update the bind mounts configuration and restart the agent to enable

It would seem that we cannot use a bind mount in the CI. Let us use our record constant to make the container options conditional:

// indexer_test.go

if record {
    // Use a bind mount
    opts = append(opts, testcontainers.WithHostConfigModifier(func(hostConfig *container.HostConfig) {
        hostConfig.Binds = []string{
            absolutePath + "/testdata:/home/wiremock/mappings",
        }
    }))
} else {
    // Use a copy operation
    mappingFiles, err := os.ReadDir(absolutePath + "/testdata")
    if err != nil {
        t.Fatalf("Failed to read testdata directory: %v", err)
    }
    for _, mappingFile := range mappingFiles {
        opts = append(opts, testcontainers_wiremock.WithMappingFile(
            mappingFile.Name(),
            "testdata/"+mappingFile.Name(),
        ))
    }
}

When recording, we mount the volume, which is not an issue because we are not in the CI.
Otherwise, we use the WithMappingFile function which relies on a copy operation.
That function is provided by the
wiremock-testcontainers-go library, which abstracts away the low-level testcontainers API so that we can think in terms of mapping files rather than just JSON files.

Not super satisfying, but it works.

Wrapping up

The test is a bit long now, but some parts look generic and reusable. Let us extract them to helpers.

// indexer_test.go

package indexer

import (
    "context"
    "fmt"
    "os"
    "testing"

    "github.com/algolia/algoliasearch-client-go/v4/algolia/call"
    "github.com/algolia/algoliasearch-client-go/v4/algolia/debug"
    "github.com/algolia/algoliasearch-client-go/v4/algolia/search"
    "github.com/algolia/algoliasearch-client-go/v4/algolia/transport"
    "github.com/docker/docker/api/types/container"
    "github.com/testcontainers/testcontainers-go"

    "github.com/wiremock/go-wiremock"
    testcontainers_wiremock "github.com/wiremock/wiremock-testcontainers-go"
)

const record = false

func spinUpContainer(t *testing.T) string {
    t.Helper()

    ctx := context.Background()

    absolutePath, err := os.Getwd()
    if err != nil {
        t.Fatalf("Failed to get current working directory: %v", err)
    }

    var opts []testcontainers.ContainerCustomizer

    opts = append(opts, testcontainers_wiremock.WithImage(
        "wiremock/wiremock:3.12.1",
    ))

    if record {
        opts = append(
            opts,
            testcontainers.WithHostConfigModifier(
                func(hostConfig *container.HostConfig) {
                    hostConfig.Binds = []string{
                        absolutePath + "/testdata:/home/wiremock/mappings",
                    }
                },
            ))
    } else {
        mappingFiles, err := os.ReadDir(absolutePath + "/testdata")
        if err != nil {
            t.Fatalf("Failed to read testdata directory: %v", err)
        }
        for _, mappingFile := range mappingFiles {
            opts = append(opts, testcontainers_wiremock.WithMappingFile(
                mappingFile.Name(),
                "testdata/"+mappingFile.Name(),
            ))
        }
    }

    container, err := testcontainers_wiremock.RunContainerAndStopOnCleanup(
        ctx,
        t,
        opts...,
    )

    if err != nil {
        t.Fatalf("Failed to create wiremock container: %v", err)
    }

    host, err := container.Endpoint(ctx, "")

    if err != nil {
        t.Fatalf("Failed to get wiremock container endpoint: %v", err)
    }

    return host
}

func startRecording(t *testing.T, host string, appID string) {
    t.Helper()

    wiremockClient := wiremock.NewClient("http://" + host)
    t.Cleanup(func() {
        err := wiremockClient.Reset()
        if err != nil {
            t.Fatalf("Failed to reset wiremock: %v", err)
        }
    })

    if !record {
        return
    }
    debug.Enable()
    err := wiremockClient.StartRecording(fmt.Sprintf(
        "https://%s-dsn.algolia.net",
        appID,
    ))

    if err != nil {
        t.Fatalf("Failed to start recording: %v", err)
    }

    t.Cleanup(func() {
        err := wiremockClient.StopRecording()
        if err != nil {
            t.Fatalf("Failed to stop recording: %v", err)
        }
    })
}

func newTestClient(
    t *testing.T,
    host,
    appID,
    apiKey string,
) *search.APIClient {
    t.Helper()

    algoliaClient, err := search.NewClientWithConfig(
        search.SearchConfiguration{
            Configuration: transport.Configuration{
                AppID:  appID,
                ApiKey: apiKey,
                Hosts: []transport.StatefulHost{
                    transport.NewStatefulHost(
                        "http",
                        host,
                        func(k call.Kind) bool {
                            return true
                        },
                    ),
                },
            },
        })

    if err != nil {
        t.Fatalf("Failed to create client: %v", err)
    }

    return algoliaClient
}

func TestIndexRecord(t *testing.T) {

    appID := os.Getenv("ALGOLIA_APP_ID")
    apiKey := os.Getenv("ALGOLIA_API_KEY")
    indexName := "test-index"

    host := spinUpContainer(t)

    startRecording(t, host, appID)

    algoliaClient := newTestClient(t, host, appID, apiKey)

    _, err := algoliaClient.SetSettings(
        algoliaClient.NewApiSetSettingsRequest(
            "test-index",
            search.NewEmptyIndexSettings().
                SetSearchableAttributes(
                    []string{"name"},
                ),
        ),
    )

    if err != nil {
        t.Fatalf("Failed to set settings: %v", err)
    }

    t.Cleanup(func() {
        _, err := algoliaClient.DeleteIndex(
            algoliaClient.NewApiDeleteIndexRequest(indexName),
        )
        if err != nil {
            t.Fatalf("Failed to delete index: %v", err)
        }
    })

    indexRecord(algoliaClient)

    searchResp, err := algoliaClient.SearchSingleIndex(
        algoliaClient.NewApiSearchSingleIndexRequest(indexName).
            WithSearchParams(
                search.SearchParamsObjectAsSearchParams(
                    search.NewEmptySearchParamsObject().
                        SetQuery("test"),
                ),
            ),
    )

    if err != nil {
        panic(err)
    }

    if len(searchResp.Hits) == 0 {
        t.Fatal("No hits found")
    }

    firstHit := searchResp.Hits[0]

    // Assert that the first hit has the expected name
    if firstHit.AdditionalProperties["name"] != "test record" {
        t.Fatalf(
            "Expected name to be 'test record', got '%s'",
            firstHit.AdditionalProperties["name"],
        )
    }
}

And now our test fits on a single screen 🙂
I also added an extra assertion just to be sure we get the expected record, and that's OK, since it does not mean extra calls to Algolia.
Now that we have paid the cost of writing that first step, writing more tests should be easier, and bring a lot of value to the project.

Why you should stop doing code reviews!

Victor Gallet — Wed, 03 Jan 2024 13:32:00 +0000

I would like to thank Baptiste Barbieri, Antoine Guy, Maxence Zajdlic and Rafael S. Ward for their time <3.

Code review is one of the most used methods in software development. We all know what are the benefits of doing code review.

To sum up, doing code review enables verification of new code quality by identifying future bugs, ensuring tested code, and ensuring readable code.

Moreover, by reading code written by others, developers can learn from others. A junior developer can learn by reading the code of a more experienced one. They can ask questions using comments.
Vice versa a senior developer can check the code of a junior developer and provide constructive feedback.

I won’t go deeper into code review benefits in this article. For more information, you can read the amazing work done by Trisha Gee. For instance this blog post.

However, is code review really worth doing? What are the drawbacks we faced doing code review? I tried to list all code review antipatterns I faced and I’m sure you faced some of them. Several times.

Code review pitfalls

The “ping pong” review

Some comments are added, you take them into account, some new comments are added, you take them into account, new comments are added…and so on. It’s really frustrating for the merge request author, even more when it happens with several reviewers.

The “re-design” review

Once the merge request opens, you receive comments indicating a large design issue. It’s not a simple correction, but the whole implementation must be redone. This is a demoralizing situation for the author and the reviewers and a waste of time.

The “ghosting” review

Comments are added, taken into account and then you wait for the approval. The reviewer has disappeared and you have to wait before merging. Even if other reviewers have approved the merge request, you have to wait for the ghost reviewer.

The “waiting for” review

Your merge request is not the most attractive and you are still awaiting approval.

The “TL;DR” review

Changes are very important but reviewers didn’t read it before approving it because it was too long.

The presentation review

You have done a huge refactoring or the changes you bring are really complex to review and your team is asking you for a presentation or a mob review. Now you have to find a time where everyone is available to do the code review, it’s only the beginning of your merge request journey.

The “convention style” review

Some comments on the merge request are merely about coding style. For instance, there are too many break lines or spaces. Those kinds of reviews could be avoided with a linter.

The “too small” merge request

If one line of code has been modified, you tend to think that it will be approved and merged fast. BUT! Instead, it triggers a lot of discussions. We are typically in a bike-shed effect.

The “keep rebasing” review

Your merge request is waiting for approval, while other merge requests are merged and you have to keep your code up to date and potentially resolve conflicts.

The “inconsistent feedback” review

Sometimes you get a comment and don’t know how to handle it. Most of the time, the review expressed a feeling more than a clear solution. For example “This function seems too complicated for me”.

The “after merged” review

The merge request has been approved and merged. Great! But a new challenger has appeared! This challenger is perhaps knowledgeable about the project or on the technologies used and he wants to make some changes. Most of the time, a new merge request will be opened by this person without sharing it with the first developer.

The “conflict” review

Expressing or receiving feedback can be complicated, people can take it personally and it can lead to some tensions in the team.

The “elderly” merge request

The merge request is open for a long time, it could be a couple of weeks, a month or even a year. The merge request is so old that most of the developers don’t know if it’s still relevant to merge it so the merge request will just sit there.

The “not working” review

The CI pipeline passed, and the merge request has been approved but once merged and deployed... it doesn’t work! Let’s do a full cycle again.

The “Full-time job” review

Code review can be a full-time job! A developer in a team could spend their entire time doing code review. Furthermore, it’s not uncommon to have developers in a team who are spending way more time than others doing code reviews. Not everyone does code review in a balanced way and it can lead to struggles within the team.

The worst thing about all those patterns is that they can be accumulated. You can be in a ping pong review, suddenly land on with a ghost review, keep rebasing and finish by having some tensions in your team.

Stop doing code review?

So what are the solutions to avoid those situations? Most of these problems could be avoided by sharing early. Before what?
For instance, sharing about coding guidelines, architecture, coding practices or design before jumping into the code. All of these methods and practices are living, moving with time and have to be updated during the project lifecycle.

It’s merely an Agile value: having feedback as soon as possible. Feedback coming from code review is too late.

In my experience, code review creates more problems than it solves. Code review is only good when developers work mostly in an asynchronous way and in different time zones. But in most cases, developers are working in the same team, on the same project at the same time and code review is only a way to avoid exchanges with colleagues. Nowadays, code review is a cargo cult.

Most of the time code review should be replaced by pair programming or better still ensemble programming (also known as mob programming).
In that way, you have at least two people working on the same piece of code and you always have:

knowledge sharing, both business and technical knowledge
proof-reading
developing cohesion and team spirit

Doing pair programming, doing synchronous development saves from ghosting, from ping pong comments, “TL;DR” review, “waiting for” review, “convention style” review….and so on. Actually, working together avoids all of the pitfalls of code review.

If you are developing using TDD, pair programming can even be more fun by doing ping pong pair programming.

Lastly, I won’t debate about the simplistic idea that having two developers working together on the same task is more expensive compared to having them working alone. The synchronisation process in all steps is one of the most expensive and code review is one of them.

Originally published at https://vgallet.github.io on December 14, 2023.

Feedback on sfCon 2022

Grégoire Paris — Thu, 15 Dec 2022 10:18:34 +0000

This year, the SymfonyCon was even more magical than it usually is, as it took place in Disneyland Paris, thus contributing sparkles on top of the nice features announced during the keynote.

Did you know that Symfony currently has no less than roughly 200 components? At the SymfonyCon, 2 new components were announced, that should make it into Symfony 6.3:

Webhook: can be used to host a webhook, or call a webhook, typically with payment providers. Handles validity, security and signature checks.
RemoteEvent: used to represent events produced outside Symfony

Here is the diff if you want to deep-dive head first!

This is a big event, so there were no less than 4 tracks, which means I had to choose.

First, I went to see a conference about Sylius and learnings the main maintainer has gathered over the past few years.
He mentioned ADRs, which we do use at ManoMano, and which would certainly be useful to also use in an open source setup (Doctrine maybe?).

Next, I went to see Nicolas Grekas' talk about proxies: he presented different ways to implement a proxy system, and examples of each one that can be found in Symfony or Doctrine.
Doctrine has an old implementation in doctrine/common that we are trying to get rid off, and Nicolas has proposed to replace those with Lazy Ghost proxies.
I know, I know, funny name. 👻
Lazy ghost proxies cannot be used with final classes, or internal classes, but do allow to use a fluent API, which makes them a good fit for doctrine/orm entities.

If you know me, you know I literally cannot shut up about Git, so for my next choice, I picked "advanced git magic" by Pauline Vos.
The highlight was definitely this explanation about git bisect:

After lunch, I heard there was a talk by one of my Doctrine fellows, Claudio Zizza a.k.a SenseException, which I first met in person at that conference, so of course, I had to pick that one.
Main takeways:

Always clear your entity manager at the end of a business class to avoid nasty side effects with $entityManager->clear();
(Postgre)SQL is very powerful, for instance you don't necessarily need to resort to the nested set model to build a menu, you can use WITH RECURSIVE

That was followed by a frightening talk form Sebastian Bergmann (himself!), who gave us a presentation of PHP's supply chain, and how each part of that chain has been exploited in the past, in PHP or for other ecosystems.
Some of these vulnerabilities are so bad they have got their own website:

All the links above are safe to visit, I swear! That last one might require a bit more processing power though.

Sebastian told us about the concept of Software bill of materials a.k.a. SBOM, as a way for you to know your dependencies and if they are vulnerable. You can try ./phpunit.phar --sbom to see it in action (only available when using the PHAR though).

I had already seen Kevin Dunglas' talk about FrankenPHP at the "Forum PHP", which also took place at Disneyland the previous month (yes, really), but I decided to watch it again because of how packed with information it was.
FrankenPHP provides a new PHP SAPI, that comes with a worker mode, that behaves much like Swoole or Roadrunner.
Unlike php-fpm, it does not require a separate nginx container to run: it's built on top of the Caddy webserver, which is written in Go, and is also written in Go.
Thanks to Kévin, Caddy is capable of forwarding 1xx HTTP responses it receives from its upstream, which unlocks using 103 Early Hints in PHP, a status code that aims at providing the browser with the urls of CSS files, JS files and other resources it might need before even sending the final response.

// Detect dark theme var iframe = document.getElementById('tweet-1557064869035347969-206'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1557064869035347969&theme=dark" }

Next, I went on a conference about the impostor syndrome, and one of the main takeaway was that it's so common and equally spread that we should rather call it impostor phenomenon. It's estimated to be have been experienced by 70% of people (and yes, that includes me). If you're unfamiliar with it, you can tell yourself that it is close to atychiphobia or kakorraphiophobia. Hope this helps.

That concluded the first day of conference. The 2 days were separated by a very nice evening with exclusive access to the Walt Disney Studios Park, which I must say is amazing, but I'll stop here because I don't want to make you jealous. But it was cool.

On the second day, I started with a conference about climate change and IT. As a staff engineer, I'm currently working on upgrading many of our microservices from PHP 7 to PHP 8, and I was glad to learn that PHP 8.1 is estimated to be 30% faster/less electricity-consuming than PHP 7.4.
There wasn't a similar figure to share about Symfony because it's harder to determine, but upgrading it is highly recommended as well.
I learnt that the carbon footprint can be evaluated in several scope, and that for an IT company like ManoMano, most of it is in Scope 3, which is all about the cloud, Scope 1 and 2 being about office heating and electricity. To lower
our impact, we should:

Upgrade our software
Optimize our apps (with a profiler)
Deploy to greener AWS regions: France, the Scandinavian peninsula (or Fennoscandia for geography nerds), the Netherlands.

This was very interesting, so I continued with a talk on the same theme: it was about computing carbon emissions in the cloud.
I learnt that "net zero" means the emissions of green house gases is equal to the removals, and that we are of course, very far from it, although it's a goal for 2050, that gets harder and harder to reach as time passes.
There is a formula to compute the carbon footprint of a VM. For each capacity, such as the CPU, storage, network, you have a matching emission factor expressed in watts, and a Power Usage Effectiveness that depends on the cloud provider you are using, which is a constant > 1 that is better the closest it is to 1. You then multiply that by the carbon intensity, that depends on the region where you are deploying, and is a factor that allows you to transform watts in CO2 emissions.

As the conference was getting closer to its end, I attended a talk about TDD.
The speaker dove into detail about TDD and gave us pointers about methods to do it properly, such as "fake it till you make it", which means you should write dummy code until you have written enough tests that force you to ditch your
initial, dummy implementation, or "triangulation", which consists in proving via tests that all parameters of a function are important, one by one.

Next we had a talk about automated upgrades by Tomáš Votruba, the author of Rector. He told us about tooling you can have in your projects to make it easier to upgrade. For instance, you can use so-called PHPStan collectors to measure type coverage, and ensure it only ever increases. Having type information is very important for static analysis, which Rector relies upon to understand your code and migrate it. Likewise, using PHP configuration instead of YAML configuration can help avoiding mistakes thanks to Static Analysis (which exists for PHP but not for YAML). He showed us a demo of using Rector in the CI to continuously upgrade the code.

The next talk I attended was by another fellow Doctrine maintainer: Alexander Turek. Alexander showed us strategies to modernize a legacy application with Symfony. It's possible for instance to introduce a global variable with the Symfony Kernel inside it to make Symfony services available inside the legacy application. That's called a micro rewrite because instead of migrating a full route to Symfony, you can migrate a service, deploy, rinse and repeat.
Something we might want to apply to our legacy ManoMano application?

The last talk I attended was by Mathieu Santostefano, a core team member. He presented the new AccessTokenAuthenticator shipped with Symfony 6.2, explaining how it had been waiting for other lower-level Symfony components to be ready over the years. Part of the talk was about the contribution process, and I remember a hilarious slide showing 2 contributors opening 2 pull requests about the very same thing (implementing that authenticator).

For the closing keynote, we had a light-hearted talk by Mickey Mouse himself presenting the evolution of Symfony over the past few months!

If this feedback piqued your interest, you can find many of the talks here:

SymfonyCon / 2022-talks

SymfonyCon - Paris 2022 talks

All talks are in english.
You can send feedback and love to speakers on their twitter account
See all tweets during the event

Keynote

Slides
~~Video~~

By Fabien Potencier
@fabpot
@fabpot

From monolith to decoupled…wait, why is that one getting bigger?!?

Description: Monolithic apps are getting broken down left and right into dedicated services and teams, all under the banners of separating concerns, higher efficiency, and more. The strategy that is implemented will be crucial to ensure decoupling is a beacon of efficiency and not a migration nightmare
In this talk, we’ll discuss how decoupling following the strangler fig approach will seem counter-intuitive, as your monolith continues to grow alongside your new decoupled architecture. But this approach, when done right, makes dismantling a monolith a process that is structured and safe, slow but agile, and without major service interruptions or massive interface changes that…

View on GitHub

Plotting the memory of a PHP process with Gnuplot 📈

Grégoire Paris — Fri, 26 Aug 2022 14:52:33 +0000

Lately, I have been troubleshooting memory issues on a process my team owns.
I started by watching this video about memory leaks from Benoît Jacquemont:

I learned a lot in the process, but also noticed the nice memory graphs in the video and figured it would be hard to troubleshoot anything if I didn't have them.
When using php extensions such as Benoît's or Arnaud Le Blanc's to take a snapshot of the memory, it's great to think about the most appropriate moment to take that snapshot in order to capture the memory leak you might be hunting.
Sure, you can use Monolog's MemoryUsageProcessor to that end, but I thought it would be more useful to get something a bit more ✨visual✨.

On our environments, we use Datadog, but on my development setup, I don't have that.

There are several metrics you can track for troubleshooting a memory issue, some provided by the OS, and typically reported by Datadog, some others by PHP through memory_get_usage() (which I don't currently have a way to monitor in production).

Measuring from PHP

PHP provides several methods to understand what is happening memory-wise. First, you have memory_get_usage(), which takes a boolean argument. Depending on that argument, the method returns the memory used by PHP, or the memory allocated to PHP. When freeing some memory, you will typically see the former decrease, while the latter stays stable.

Then, you have memory_get_peak_usage() which reports the highest value of used or allocated memory since the beginning of the script. That's useful because it can help the developer figure out that they are not calling memory_get_usage() where memory usage is at its highest.

Producing metrics

In the case of the script I was troubleshooting, I had a main loop that was frequently executed (but not at an even rate). That's still a good candidate for gathering metrics, as we will see.

Here is what I put inside that loop:

<?php
file_put_contents(
    'memory.tsv',
    time() . "\t" .
    memory_get_usage(true) / 1024 / 1024 . "\t" .
    memory_get_usage(false) / 1024 / 1024 . "\t".
    memory_get_peak_usage(true) / 1024 / 1024 . "\t" .
    memory_get_peak_usage(false) / 1024 / 1024 . "\n",
    FILE_APPEND
);

This produces a TSV file that looks like this:


1660831187  20  8.1252593994141 20.359375   19.608978271484
1660831187  20  8.1281814575195 20.359375   19.608978271484
1660831187  20  8.131103515625  20.359375   19.608978271484
1660831187  20  8.134033203125  20.359375   19.608978271484
1660831187  20  8.1369552612305 20.359375   19.608978271484
1660831187  20  8.1398773193359 20.359375   19.608978271484
1660831190  22  8.2328033447266 24.42578125 22.869613647461
1660831190  22  8.2357330322266 24.42578125 22.869613647461
1660831190  22  8.2386627197266 24.42578125 22.869613647461
1660831190  22  8.2415924072266 24.42578125 22.869613647461

Looking at the first column, what stands out is that there are groups of lines that can be several seconds apart, so the production of metrics is really, really not paced regularly.

Plotting graphs

Then, to create the graph, I turned to gnuplot, which seems like a whole universe of its own as well as a very robust piece of software. I started by creating the following configuration file:

# config.plt
set term png small size 800,600
set output "/tmp/memory_get_usage-graph.png"

set ylabel "memory in MB"

set yrange [0:*]

set xdata time # x is not just a random number
set timefmt "%s" # we use UNIX timestamps

plot "memory.tsv" using 1:2 with lines axes x1y1 title "memory_get_usage(true) in MB", \
     "memory.tsv" using 1:3 with lines axes x1y1 title "memory_get_usage(false) in MB", \
     "memory.tsv" using 1:4 with lines axes x1y1 title "memory_get_peak_usage(true) in MB", \
     "memory.tsv" using 1:5 with lines axes x1y1 title "memory_get_peak_usage(false) in MB"

As you can see, it's possible to let gnuplot know that the x axis represents time, which ensures you have a nicely formatted X axis.

The graph is created by running gnuplot config.plt.

Rendering the graph

What would be handy would be a graph that refreshes over time. For that, you will need 2 tiny programs: watch and feh.

Run watch gnuplot config.plt, to ensure a png is created every 2 seconds (which is watch’s default).
In parallel of that, you run feh /tmp/memory_get_usage-graph.png to display the png file. What's great with feh is that it refreshes automatically, so you don't need to do anything special to get your live graph. 🤯 feh does very little, but does it well.

2 interesting things to note here:

Only the graph for memory_get_usage(false) goes down, but it does go down, so there is no memory leak
The defaults of gnuplot are a bit ugly, and I am no frontend developer, so it will stay ugly.

Measuring from Linux

Producing metrics

Here, to produce the metrics, you can use ps.

while true; do
    ps --pid $(pgrep -f some_string_that_identifies_your_process) \
    -o pid=,%mem=,vsz= >> /tmp/mem.log
    gnuplot config.plt
    sleep 1
done

Note that you can of course use this for any process, not just PHP processes.

Plotting graphs

This time it's a bit more tricky, I am telling gnuplot to plot 2 metrics that
have different units on the same graph.

The left Y axis will have a scale for the first metric, and the right Y axis
will have a scale for the second metric.

I do not configure the X axis this time, since I'm producing metrics at a
regular pace.

This is all shamelessly stolen from Stack Overflow

set term png small size 800,600
set output "/tmp/mem-graph.png"

set ylabel "VSZ"
set y2label "%MEM"

set ytics nomirror
set y2tics nomirror in

set yrange [0:*]
set y2range [0:*]

plot "/tmp/mem.log" using 3 with lines axes x1y1 title "VSZ", \
     "/tmp/mem.log" using 2 with lines axes x1y2 title "%MEM"

Here you can see that the figures are different than from inside PHP. I will not get into this because that is off topic, but when troubleshooting memory issues, it can also be important to compare both aspects.

Takeaway

Those graphs helped me understand the differences between memory_get_usage(true) and memory_get_usage(false), and gave me a better understanding of my application. In particular, I understood that the batch processing I was doing relied on batches of objects that were not all the same size, and that making sure they were all roughly the same size would help avoid situations where a series of big objects caused an out-of-memory error.

How to work with redux on module federation

Danielo Artola — Thu, 17 Feb 2022 15:14:17 +0000

Module Federation is a plugin from Webpack 5 that helps to orchestrate micro-frontend architecture, thereby making it easier for organizational teams to decouple and share applications.

If you have a big monolith on frontend with react, you probably have redux to share info between components. What happens if we have to move some of those components to a module?

Introduction

This post discusses how to share info between modules via redux, so we’ll focus on how modules and base will work with the store, reducers, and actions. Also, the following sections will explain how modules can work with redux as isolated applications or a federated module with minimal code changes.

Index

Basic app architecture
Store for base and modules
Trigger an action from module
Inject reducer from module to base
Conclusions

Basic app architecture

In general, this approach consist of:

A NextJS project as a host/base project.
Federated modules for some features, such as Header, Homepage, Cart, and ProductPage.

Keep in mind that for this example we’ll be working with an NextJS base (React, Redux, etc) and a module with the same technologies (which exports only one component).

Our module will consist of a web app with react and redux, without NextJS, but the main point here will be only how to interact with redux.

Store for base and modules

The idea here is that all modules should work as an isolated app and as a federated module. Furthermore, we know redux actions can only be triggered under a redux provider.

Therefore, the strategy is to place a provider on sub apps, but export only the components under this provider to inject the module on our base.

The idea is to always have a redux provider, but avoid having two providers when a module is injected.

With this we’ll be able to trigger actions from modules, so no matter if is working as isolated or module.

Trigger an action from module

To trigger an action on Module, we can do it exactly as we do it on any react-redux app. There are only few things to keep in mind:

When we export a component to work as a module, we are exporting the component and not the reducers, so if we want to have same reducers on base and module, we need to inject it. As que saw in the last image, the exported component will have different provider depending how we work with it.
If we need a reducer and we don’t want to inject it because we have it from the beginning on the app, we have to search a way to export the reducer from module an include it on base to add it to the store when necessary, avoiding to have the same duplicated reducer on base and module.
We don’t need to change anything to trigger an action, but the available reducers will depend on where the component is mounted. If it's working isolated, we’ll need to mock or recreate the reducers that we need from base. If is working as module on base, we’ll need to add the reducers that we have on sub-app.
For testing you'll have to import or mock as in the point 3

Inject reducer from module to base

If you're working on big react app with redux, you probably faced the reducer injection:

const injectAsyncReducer = (store, name, asyncReducer) => {
  store.asyncReducers[name] = asyncReducer;
  store.replaceReducer(createReducer(store.asyncReducers));
}

You probably wrote a function similar to this, and that code will work perfectly for both of the cases that we're managing here: working isolated and as a module. Why? because in the two cases we'll have a redux provider, the only difference will be that each case will inject the reducer in a different store, but you'll have nothing to deal with.

NOTE: Yes, you can inject redux-sagas as well, if you're curious about I drop this line here

Conclusions

If you were worried about split your monolith in federated modules due to redux, there's nothing to be afraid. The most complicated part probably will be the Webpack configuration and not the redux.

Thank to english.with.yama@gmail.com for proofreading the article.

React 18 setTransition

Pascal Fong Kye — Thu, 27 Jan 2022 09:06:47 +0000

startTransition

React 18 introduces a new concurrent feature called startTransition which aims to solve problems we tried to solve using debounce and throttle, and much more.

This thread gives a deep dive example of how it solves the problem of UI responsiveness when there is expensive computation taking place.

Before React 18, we tend to use debounce (or throttle depending on the situation) to schedule this compute intensive task at a given time T so that the UI feels more responsive to other events. But every time the task will start after T has elapsed, even if it takes less time than T. And the task will still occur as a whole chunk, blocking the thread during this time.

This is where startTransition API brings a new approach to deal with these problems.
It can:

split tasks into small chunks and be smart enough to let the browser process incoming events (mouse events for example) -> yielding
manage an urgent update in priority skipping pending ones -> interrupting
update using new state by disposing of unfinished task with stale state -> skipping old results

Check my personal blog for more articles.

How to add custom types to a javascript library

Danielo Artola — Mon, 06 Dec 2021 16:12:52 +0000

Few weeks ago, I started contributing to an open source library called Teaful, a Tiny, EAsy, and powerFUL for React state management, with ambitious roadmap. Now Teaful reached more than 500 GitHub ⭐️ Stars, the library and his community are growing fast.

That means issues and pull requests are growing as well, and soon we realized that we need to improve dev-experience and provide tools for that reason.

Bear this in mind, implement custom types to allow all the benefits from TypeScript at Teaful is a big step on that way.

(Yes, I know, migrate a library to pure ts probably is a better solution, and it's on our roadmap before 1.0.0)

Index

Avoiding auto generated types
Where to place custom types
Create custom types
Conclusions

Avoiding auto generated types

In our case, an auto-generated custom type full of any was useless. So, we started implementing custom types.

We're using microbundle, they provide a flag to avoid auto-generate types, --no-generateTypes. Microbundle, according to docs, generally respect your TypeScript config at tsconfig.json (you can read more about here), but at this moment we don't need a specific configuration for TypeScript

Then we can inform on package.json where are our custom types with "types": "folder/index.d.ts".

Where to place custom types

Create a file with extension .d.ts , generally you'll put this file on dist folder. Now here you can add your custom types.

Create custom types

Here I'm going to explain how we created custom types specifics for Teaful and why some decisions were taken, if you're reading this to know how to add custom types to your js library and already know about TypeScript, feel free to skip this section.

InitialStore

The store is where Teaful saves data, is a key-value object (you can have more than one store). Easy to type:

type initialStoreType = Record<string, any>;

So far so good, nothing strange here. We want to store anything, and all keys will be string.

Then things become more complicated. In this article only things about creating types will be explained, so if you want to know more about how to implement Teaful I strongly recommend visit the README at github.

Hook Return

To create a new value on store is pretty similar to useState from React. Let's see an example:

const [username, setUsername] = useStore.username();

Easy right? Ok, so what have we here? useStore returns an array of two elements (Yes! Like useState!), the element in the store and the function to update it.

The type we need:

type HookReturn<T> = [T, (value: T | ((value: T) => T | undefined | null) ) => void];

If you're not familiar with TS, this could look a little cryptic. We're creating a new type called HookReturn which gets a generic type we called 'T' (from Type, but you can use any name).

This type is a tuple(a data structure that is an ordered list of elements with a fixed length, because we aren't going to add more elements for the return of our useStore), where first element is T, because we want to return a value with specific type that we don't know at the moment of creating the type, but we want to ensure, for example, that the setter function (the second element on this tuple) will get the same type we are using for the first element as param.

Then, let's pay attention on the second element of our tuple.

(value: T | ((value: T) => T | undefined | null) ) => void

Here, our type is a function that returns nothing ( () => void), but accepts one param (value: T | ((value: T) => T | undefined | null)), and this param could be a value of type T, or a function that get a value of type T and returns null, undefined or a value of type T ((value: T) => T | undefined | null).

What this means? what are we allowing here with this type? Ok, let's imagine a counter:

const [counter, setCounter] = useStore.counter();

//allowed by T
setCounter(counter+1);
//allowed by  ((value: T) => T | undefined | null)
setCounter((counter) => counter*2))
setCounter((counter) => undefined)
setCounter((counter) => null)

Yes, Teaful accepts a function as param on the setter function.

Hook type

When you create/call a new property with useStore, you call useStore.[newProperty](). This accepts two optional params, first for initialValue, and the second one is for updateValue (a function to update the store property indicated with the proxy). The hook looks easy to create here:

type Hook<S> = (
    initial?: S,
    onAfterUpdate?: afterCallbackType<S>
) => HookReturn<S>;

Both optional, but the second one is a specific function. Type onAfterUpdate, is a function with two params: store before and after the changes, both will be same type, extending our initialStore type.

type afterCallbackType<S extends initialStoreType> = (
    param: { store: S; prevStore: S; }
) => void

Finally, our type Hook will return a tuple [property,setter], so indeed, we're going to return our custom type HookReturn with our generic type. If we create a number, have sense to take care about number type in all places, for the initial value, the returned tuple... etc.

Hoc type

Teaful allows to use it as Hoc (as connect on Redux, code explain it by itself):

const { withStore } = createStore({ count: 0 });
class Counter extends Component {
  render() {
  const [store, setStore] = this.props.store;
    return (
      // [...]
    );
  }
}

// Similar to useStore()
const CounterWithStore = withStore(Counter);

The HOC withStore wraps a Component and returns the component with a prop called store. A second parameter for initial value is allowed, and a third one for onAfterUpdate callback.

type HocFunc<S, R extends React.ComponentClass = React.ComponentClass> = (
       component: R,
       initial?: S,
       onAfterUpdate?: afterCallbackType<S>
) => R;

We need two generic types, one for initial value and onAfterUpdate (both will use same generic, but onAfterUpdate will have a specific type, explained later) and the other one for React component to wrap that would be the same for the return, because we want the same component but with a new prop called store.

Look at the R type, is extending React.ComponentClass (type provided by React). This means that we are taking profit from that type and including it in our generic type called R.

Why extending component class only and not functional component?

Well, we didn't found a single situation when we wanted to wrap any component that doesn't extend Class with a HOC to get the store.

Ok, third type: onAfterUpdate. Here we need a function with two params store before and after the changes, both will be same type, extending our initialStore type. Same as first hook, we reuse same type for all callbacks params

Now we only have to export the a type to use

  export type Hoc<S> = { store: HookReturn<S> };

HookDry type

Teaful provides a helper called getStore, like useStore but:

It does not make a subscription. So it is no longer a hook and you can use it as a helper wherever you want.
It's not possible to register events that are executed after a change.

This means we don't want same as useStoretype, we return the same but we want to ensure we don't accept a second param as callback. Let's create another one:

  type HookDry<S> = (initial?: S) => HookReturn<S>;

The return is clear, same as Hook.

Let's type useStore, getStore and withStore

Ok, now we have almost all the work done. A custom type is needed for each tool, useStore, getStoreand withStore:

  type getStoreType<S extends initialStoreType> = {
    [key in keyof S]: S[key] extends initialStoreType
      ? useStoreType<S[key]> & HookDry<S[key]> : HookDry<S[key]>;
  };

  type useStoreType<S extends initialStoreType> = {
    [key in keyof S]: S[key] extends initialStoreType
      ? useStoreType<S[key]> & Hook<S[key]> : Hook<S[key]>;
  };

  type withStoreType<S extends initialStoreType> = {
    [key in keyof S]: S[key] extends initialStoreType
      ? withStoreType<S[key]> & HocFunc<S>
      : HocFunc<S>;
  };

The keyOf type operator ensures that our property will exist on store.

The ternary here looks weird if you're not familiar with Typescript, is used for conditional-types. The logic shared in three types is, get a generic type (S, that extends our initialStoreType), then get a keythat must be on S (the property should exists on our store).

Finally, this withStoreType<S[key]> & HocFunc<S> is a Intersection type. According to TypeScript documentation "An intersection type combines multiple types into one". So if S[key] extends initialStore, we set the intersection type, if not, the hook/hoc type only.

createStore

Last, the function to export from Teaful, the masterpiece:

function createStore<S extends initialStoreType>(
    initial?: S,
    afterCallback?: afterCallbackType<S>
  ): {
    getStore: HookDry<S> & getStoreType<S>;
    useStore: Hook<S> & useStoreType<S>;
    withStore: HocFunc<S> & withStoreType<S>;
  };

Conclusions

That's definitely not everything, but there are few steps that you'll face:

Check how to stop auto-generated types, check if types are generated by the bundler like our case, by tsconfig.json or whatever.
Create a custom types on a d.ts file.
Indicate to package.json the place of that file with property "types".

Adding custom types to a javascript library could be difficult at the beginning, but will improve the dev-experience from your users.

And most important, this could be a great opportunity to learn and improve your skills, to start networking with the community or a good way to help other devs.

I hope it was helpful to you, have a super nice day!

Thank to english.with.yama@gmail.com for proofreading the article.