Forem: Sebastian Rindom

Medusa: A New Way to Build Commerce

Sebastian Rindom — Tue, 04 Apr 2023 17:57:23 +0000

I am very excited to share an update to our website and announce our v1.8 release that came out today.

Medusa is a new way to build commerce, and our updated website and the release clarifies this with a sleek new design and powerful new features.

We build tools and modules that run in new, modern environments and make it easy for developers to create unique experiences.

Visit our new website to read about why we started Medusa and where we are headed.

Check out our Recap page on our website to get a peak of all the cool new features we put out in 1.8. This post also highlights some of those features.

Multi-Warehouse

The Multi-Warehouse feature allows businesses to manage their inventory across multiple locations and sales channels, reserving items in orders that aren’t fulfilled yet, choose where to fulfill items from and where to return items to, and more!

Learn more about Multi-Warehouse in the product announcement.

Product Categories

The Product Categories feature allows businesses to organize their products in categories of unlimited hierarchy. This improves product organization and allows customers to find products more easily.

Learn more about Product Categories in the product announcement.

Admin Plugin

We previously shipped the Medusa admin in an isolated GitHub repository. This caused a lot of issues related to updating and extending it.

So, we now ship the Medusa admin as an NPM package that can be installed as a plugin on the Medusa backend. This ensures that the admin now follows versioning similar to the Medusa core, and opens new potentials into its development and deployment.

Learn more about the Admin Plugin in the product announcement.

Modules

The Modules API gives developers more customization capabilities, allowing them to change how commerce features, or core logic is implemented. The Multi-warehouse feature is actually composed of two modules: inventory and stock location. Another example of modules are the cache and event bus modules.

You can learn more about Modules in the product announcement.

Cache Module

The cache module is used to temporarily store or cache data such as product prices and tax rates.

As the cache functionality is handled with a module, developers have the freedom to choose whichever third-party service or logic they used to handle caching. Medusa also provides two cache modules: local cache and Redis cache modules.

You can learn more about Cache Modules in the product announcement.

Event Bus Module

Medusa previously required using Redis for its event system. In this new release, the event system has been refactored to instead use an event bus module to handle the messaging system in Medusa.

Developers can create their own module to integrate their preferred third-party pub/sub service or custom logic. Medusa also provides two event bus modules: a local event bus module and a Redis event bus module.

You can learn more about the Event Bus Module in the product announcement.

Transaction Orchestrator

Transaction Orchestrator is a new feature in Medusa that allows handing transactions in an increasingly complex system and environment. This feature is a huge leap for Medusa to become more composable and robust, and it opened the door for the introduction and usage of modules.

You can learn more about the Transaction Orchestrator in the product announcement.

OAS Codegen Tools

At Medusa, we use OpenApi Spec (OAS) to generate SDKs and automate documentation, among other features. The new OAS tool allows developers to generate their own OAS files and TypeScript client types from their Medusa backend.

You can learn more about OAS Codegen Tools in the product announcement.

New Documentation

As mentioned in the beginning of this article, Medusa’s main goal has always been to provide the best developer experience. An essential part of that is constantly ensuring that we’re providing a reliable documentation with good user experience.

Earlier this month we’ve launched a new documentation with a new structure and design. You can learn more about the process behind that in the product announcement.

Interested in Learning More about Medusa?

If you’re interested in Medusa’s vision, and you want to learn more about it, check out our website for all the information you need. You can also refer to our GitHub Discussions to learn more about our Roadmap or give us any feedback you have. In addition, you can join our Discord and be part of our community.

Medusa’s New Feature: Order Editing

Sebastian Rindom — Wed, 25 Jan 2023 14:13:55 +0000

Medusa is happy to announce Order Editing and Payment Collections as the latest additions to the Order API, strengthening one of the most critical parts of any commerce application. This is an essential step in our mission to deliver commerce building blocks that developers can use to create bespoke digital commerce experiences without reinventing core commerce functionality.

medusajs / medusa

The world's most flexible commerce platform.

Medusa

Documentation | Website

Building blocks for digital commerce

Getting Started

Visit the Documentation to set up a Medusa application.

What is Medusa

Medusa is an ecommerce platform with a built-in framework for customization that allows you to build custom commerce applications without reinventing core commerce logic. The framework and modules can be used to build advanced B2B or DTC ecommerce stores, marketplaces, PoS systems, service businesses, or any product that needs foundational commerce primitives. All commerce modules are open-source and freely available on npm.

Learn more about Medusa’s architecture and commerce modules in the Docs.

Upgrades & Integrations

Follow the Release Notes to keep your Medusa project up-to-date.

Check out all available Medusa integrations.

Community & Contributions

The community and core team are available in GitHub Discussions, where you can ask for support, discuss roadmap, and share ideas.

Our Contribution Guide describes how to contribute to the…

View on GitHub

The Order Editing and Payment Collection features solve the problems faced when businesses need to change already placed orders. Systems that rely on order data, like fulfillment and accounting systems, will have to be updated, and payment discrepancies may arise. These updates and discrepancies often require error-prone, manual work.

Medusa’s new capabilities offer a simple and worry-free approach to creating order edits while giving your customers the best experience.

How Order Editing Works

The Order Edit flow consists of three simple steps:

Select the changes you wish to make.
Ask the customer to confirm your changes or, as an admin, force a confirmation.
See the order edit merged into the order and continue managing your order as usual.

Editing orders in Medusa allows you to update, add and remove line items on already placed orders with a few easy steps.

Helpful timeline events ensure that you and your team have visibility to understand the order’s history and assist customers with questions about their orders.

To learn more about how we designed Medusa’s order editing capabilities, check out this article that dives deeper into the solution.

Give your customers confidence and peace of mind

The confirmation step is essential to the order editing flow and ensures that your customer is always in the loop about changes to their order. You can configure notifications that automatically inform customers that they must accept changes to their orders.

They can confirm their changes with simple steps and pay for potential shipping and total differences.

Customers can accept order edits to stay in the loop about changes to their order and pay for potential differences.

For businesses, Medusa’s extensible API also provides a new opportunity to upsell customers and highlight products.

New possibilities with great APIs

Everything we build at Medusa is deliberately designed to give a robust starting point while remaining open for customizations and extensions that can make Medusa truly your own. This is also the case for Order Editing and Payment Collections, where plugins and projects can now listen to a comprehensive set of new events that can act as triggers for automations and integrations.

Accounting and fulfillment plugins, for example, will be able to listen for changes to orders and update systems to ensure consistency throughout your stack. This eliminates manual work while giving your customers a fast and smooth experience.

// src/subscribers/my-subscriber.ts

eventBus.subscribe(OrderEditService.REQUESTED, async ({ id }) => {
  const analyticsData = await getTrackingData(id)
  analyticsProvider.track("Order Edit Requested", analyticsData)
})

Finally, our Payment Collection API will soon expand to support broader use cases like re-authorizing expired payment authorizations and creating new payment experiences for your customers, like installments or deposit payments.

Try it out now

We are excited to see what unique new experiences and plugins will be built with the new Order Editing and Payment Collection capabilities.

Try it out now by upgrading to the latest version of Medusa:

yarn add @medusajs/medusa@latest

Or start a new project:

yarn create medusa-app

If you’re interested in learning more about this feature, check out this article which goes over how Git and GitHub inspired the design of the Order Editing capabilities.

You can also learn more about order editing in our documentation:

How to implement order editing functionalities for administrators
How to implement order editing functionalities in a storefront
OrderEdit Admin and Store API reference

Introducing Medusa.express: The easiest way to set up an ecommerce store

Sebastian Rindom — Mon, 14 Feb 2022 18:18:37 +0000

medusa.express automatically creates pages for the products in your catalog, each of them optimized to make the purchasing experience as frictionless as possible, by bundling the checkout flow alongside the product.

The underlying thesis is that not all customers and not all products benefit from a fully-fledged website experience. Consider three different examples:

Retargeting: Customers in the lowest part of the marketing funnel, say customers who have already shown an interest for a given product. Why not give these customers a super quick way to purchase your products.
Complex purchases: Consider stores with huge product catalogs where customers need guidance (algorithmic or human powered) to find the desired product - once the right product has been determined the goal becomes to complete the transaction both from the store’s and the customer’s perspective.
Simple store owners: Some people simply do not need a full store setup to sell their products; bloggers, one-item sellers or outbound salespeople might be sufficiently served by sending a link to their users for purchasing certain products while not having to maintain a full webshop frontend.

How it works

A customer visits a link e.g. https://medusa.express/basic-tshirt
The customer selects the variant of the product they wish to buy
Customer completes the checkout flow and the order is completed

Why did we build this concept?

One of the most liberating things about using a headless commerce engine, like Medusa, is that you are not constrained to follow any standards around how the shopping experience should be for your products. Standards are great in most cases, but they do also sometimes limit what is possible and therefore hinders you from opportunities if the standards are too tightly coupled with your software.

You are surely familiar with the standard 2-column product grid, that serves as entry points to a product page with an image on the left side a size and quantity ticker on the right side, which ultimately have the goal of taking you to a checkout flow that looks the same across all brands. This experience has not changed much over the past many years and a likely reason for this is that companies still treat their digital commerce channels as physical retail stores.

In physical retail stores, all customers are directed through the same path and products are placed along this path in a way that is meant to optimize the stores sales as carefully planned by a retail merchandiser. The same approach has been taken for digital commerce experiences to the point where even the most advanced personalization engines for ecommerce rarely offer solutions different from placing candy and chocolates along side each other in a supermarket does.

What is yet to be leveraged is the fact that the internet enables an infinite amount of purchasing experiences to be offered and medusa.express is a simplified demo that illustrates how headless commerce can make this possible.

Try it now!

Prerequisites: To use Medusa Express, you are required to have a running Medusa server. Check out our quickstart guide for a quick setup.

1. Create your Medusa Express project

git clone --depth=1 https://github.com/medusajs/medusa-express-nextjs medusa-express

2. Navigate to your project and install dependencies

cd medusa-express

yarn
# or 
npm install

3. Link your Medusa server

Create a .env.local file and add the following environment variables:

NEXT_PUBLIC_MEDUSA_BACKEND_URL=http://localhost:9000
# Stripe key is required for completing orders
NEXT_PUBLIC_STRIPE_API_KEY=pk_test_...

4. Try it out!

Spin up your Medusa server and Medusa Express project and try it out.

Ensure that your STORE_CORS configuration in medusa-config.js in your Medusa project includes the URL of your Medusa Express project (defaults to http://localhost:8000)

Conclusion

Medusa.Express is a solution for big and small stores alike. Whether you want an easy way for your customers to buy your store’s products, or you want a simple way to let people buy your products online, Medusa.Express is a fast way that lets you deliver a quick shopping experience for your users.

Should you have any issues or questions related to Medusa, then feel free to reach out to the Medusa team via Discord.

Wrapping up 2021 and what's in store for 2022

Sebastian Rindom — Tue, 04 Jan 2022 12:57:31 +0000

2021 has been quite overwhelming with all the interest we have received from the developer community - thanks a ton to all of you for that! Read below for a quick review of 2021 and a short glimpse into 2022.

Highlights of 2021 ⚡️

Product highlights:

Fully open-source; from core and admin to starters and design system
Created an onboarding flow that gets you up and running with only 2 commands
Developing starters that allow you to spin up a full store with a Gatsby or Next.js frontend
Building an extensive plugin library with integrations to Stripe, Contentful, Strapi, Algolia, Paypal, and many more
Building out our documentation, incl. deployment guides, setup tutorials, plugin guides
Migrating the project to Typescript

Community highlights:

Growing from 10 to +5,500 GitHub stars since our official launch 6 months ago
Launching our community Discord and welcomed ~700 amazing members
Hosting our first Hackathon and got 85 contributions to the core project

What is to come in 2022 🔮

Medusa 2.0

We aim to launch Medusa 2.0 by the end of Q1. A first core foundation for this will be released with our new Admin dashboard in January. We will share more details about the roadmap as we progress on our Discord.

Doubling down on community

Hackathon, conferences, merch store, video tutorials, and a heck of a lot more; community is king and we want to give back for all the contributions we have received through many more activities addressed specifically to our community.

Medusa Cloud

It should not take much to deploy Medusa and although our product will always remain open-source and available for self-hosting, we want to make it a one-click experience to get started with a Medusa store - and don’t worry you will always own the code that powers your store so that when it is time to customize your can quickly take over.
Plenty of exciting things to happen. We cannot wait to show it all to you!

Why we migrated to TypeScript after all

Sebastian Rindom — Fri, 19 Nov 2021 11:13:25 +0000

Over the past two weeks, we have been working hard on migrating parts of Medusa's codebase to TypeScript. We are doing this for a couple of reasons all of which have to do with improving the developer experience. Now that we are slowly merging the first parts of this migration we would love to share some insights into why we didn't just go with TypeScript in the first place and why we have decided that the time was right now.

A bit of background

Before working full-time on Medusa we did agency work for a handful of different e-commerce clients; one of them wanted to migrate away from their solution back then and when we couldn't find a new suitable platform for them we agreed to build a custom solution from scratch. We didn't intend for this to be anything other than a solution that would optimize and automate all the workflows for the client so we went straight into development mode. As the project grew and our client grew, new requirements emerged and we had to go back to the drawing board. This was the first time that we had to weigh our options in the context of the project being maintainable and easy to use in the long term. We evaluated approaches based on a couple of insights:

Growing businesses change (a lot)
- When you are on a growth path you can get by with basic tools in the beginning, but when processes get more complex you will typically need to adapt the tools in your stack. We wanted to build our architecture to accommodate this insight; making it easy to compose your e-commerce stack with the best-in-breed tools. Note:* best-in-breed for a small 2 person team is not the same as best-in-breed for a 25 person team, so your stack has to be dynamic.*
Growing businesses need control
- If you are growing rapidly you need to be in control of what happens when; you can't make a feature request and then sit around for 2 years waiting for it to be made available. You must be able to get the process moving straight away.
Customer Experience is king
- If you are trying to stand out in the crowd of standardized websites you need to have a razor-sharp focus on the customer experience. Customers should feel that they are entering into an on-brand universe when visiting an e-commerce site and to do so you may have to change up the standard browsing, purchasing, and checkout process to truly accommodate the product or brand.
Developers are the key to making all of the above succeed
- Developers get a bad rap for being expensive, slow, and complicated to work with and this makes sense to a certain extent; however, there is an incredible amount of projects - Medusa being one of them - being built that focus on making developers more efficient, making the requirements for developer capabilities lower and ultimately making it easier for merchants to start leveraging headless technologies earlier than they otherwise would have. As the ecosystem of great developer tools grow it will become increasingly appealing to new merchants to make the switch as early as possible.

The above requirements were foundational for some of the high-level decisions we took in the early days:

We created a simple and modular architecture that gives you all the basics out of the box and provides the interfaces necessary to easily integrate with other tools to give you the optimal stack.
We open-sourced everything to give users and merchants full control of what gets built
We provide a great APIs and tooling to create custom e-commerce experiences that move the needle
We think carefully about how to design things in the most developer-friendly way and choose tools in our stack that developers like.

Why we didn't go with TypeScript in the first place

You may be thinking now: okay but why on earth didn't you start with TypeScript if you wanted to use tools that developers like. The truth is that when we started developing we were optimizing for speed; with limited TypeScript experience back then we felt that there would potentially be many unknown issues to overcome if we had decided to go down that route. Furthermore, TypeScript was not as widely established as it is today and it would have been really sad if we had chosen a language that would wane in popularity after a couple of years (cough CoffeeScript and friends). That isn't to say that we didn't like TypeScript, on the contrary, we have always thought TypeScript was incredibly powerful, however, by Lindy, we decided that it was worth waiting a bit.

Not going with TypeScript in the first place allowed us to do other amazing things like creating an incredibly powerful core architecture and a handful of great plugins that provided integrations for systems like Stripe, Segment, Sendgrid, and others. We were able to do this at an incredible pace and iterate quickly, changing APIs on a day to day basis (one of the privileges of the early days with few customers) without too many complications which eventually got us to this point in time, where we have figured out the ins-and-outs of how data should be flowing and where we have a good understanding of the types and interfaces needed for modularity and composability.

Why is the time right now

When you are the lone developer and user of a project you know the APIs intimately because you have written them. Sometimes your memory might be a bit unclear, but generally, it is not a distractor for your workflow, so you don't think that much about IntelliSense not kicking in. That changes a lot, however, when you start working on things that are not your creation and not only from the user perspective also from the contributor perspective. Say you are working on code that calls some function that someone else from your team wrote. While you might have been in talks with your teammate about how the APIs are meant to work and what kind of dataflows should be going on, you might not know the exact implementation details, and that is when it starts being super useful to have all of your tooling in place.

We are so happy to see the community around Medusa growing and we are therefore very much aware that we need to improve all the tools in the ecosystem as much as possible to allow for the best developer experience. Furthermore, we feel confident that TypeScript will be around for a long time, and for this reason, now is the time!

How we are going about it

Migrating our entire codebase will take some time so we are starting with the parts that make the most sense and moving along. We are already using TypeORM for the data layer so all of our database entities are typed. We have gone ahead and added TypeScript to all of the core API's controllers typing all payloads and responses. This adds the amazing benefit of being able to share the type definitions between the core project and our JS client essentially ensuring that when you are making calls through our client library you are seeing the exact class properties that are being used to validate the request payload.

What's next

We are going to add TypeScript throughout the core package as we go along and our goal is to migrate all of our code eventually. If you are interested in helping out or want to try Medusa go check out our GitHub repo and join our Discord where you can get direct access to the engineering team!

An open-source implementation of idempotency keys in NodeJS with Express

Sebastian Rindom — Thu, 21 Oct 2021 14:59:44 +0000

Introduction

This post covers how idempotency keys are used in Medusa and how you can implement your own idempotency key logic in a NodeJS application to make your API more robust. This post and the implementation discussed here are inspired by this article by Brandur.

What is idempotency?

Idempotence is a mathematical term used to describe algebraic expressions that remain invariant when raised to a natural power - the word itself comes from the Latin words idem and potence meaning same and power respectively. In software and tech idempotency typically refers to the idea that you can perform an operation multiple times without triggering any side effects more than once. This is an extremely powerful property for fault tolerance in larger systems where service availability cannot be guaranteed. If you are familiar with RESTful design you have probably heard that DELETE requests should be idempotent, meaning that no matter how many times you make a DELETE request on a certain resource it should always respond with confirmation that the resource has been deleted (unless business rules don't allow it that is).

In fintech applications, idempotency is typically extended to other types of requests to ensure that sensitive operations like issuing money transfers, etc. don't erroneously get duplicated. For example, Stripe has support for idempotency on all of their requests controlled by an Idempotency-Key header. This allows you to safely retry requests if necessary, for example, you may be issuing an "Authorize Payment", but just after the request is sent your internet experiences an outage and you have no way of knowing whether the payment was successfully authorized or not; however, by using idempotency keys you can safely retry the "Authorize Payment" request without having to worry about making two payment authorizations.

One of the major benefits of headless commerce is that you can pick and choose the tools in your stack and have them integrate with each other for a best in breed stack; however, the more systems that are connected the more prone you will be to having inconsistencies across your tools, e.g. because of things out of your control such as server outages, connectivity issues or other unexpected situations. To solve this issue Medusa implements idempotency key support so that you can safely retry requests until consistency is confirmed.

How can idempotency keys be used?

There are two perspectives worth considering when answering the question of how idempotency keys can be used: one is from a client perspective, for example, when calling an API from a frontend, the other is from a server perspective when transferring data between systems. The purpose is the same in both circumstances, namely to ensure that an operation is completed correctly.

Client perspective

Imagine that you are adding a line item to a shopping cart through an API like Medusa's. You make a request to add the line item, but right after sending the request your internet drops resulting in a "Server not reachable" response - at this time it is not clear whether the request made it to the server and that the underlying database was able to successfully update your cart with the new item or if the internet dropped before sending the request and thus didn't result in an update in the backend. In the former case, a retry would result in your cart now having two items instead of one as expected, so if you retry the request you will have to have a compensating mechanism, which is tricky and tedious to build and test.

A typical retry flow

This is where idempotency keys come in handy as they can help you ensure that the intended state is reached even in fragile environments. In practice the requests would look something like this:

const idempotencyKey = uuidv4()

const makeRequest = async () => {
  return await api.post("/store/carts/[cart-id]/items", {
    variant_id: variant.id,
    quantity: 1
  }, {
    headers: {
      "Idempotency-Key": idempotencyKey
    } 
  })
}

let result
let shouldRetry = true
while (shouldRetry) {
  const { response, error } = await makeRequest()
  if (error) {
    shouldRetry = shouldRetry(response) // retry logic
    if (shouldRetry) {
      // expontential back-off
    }
  } else {
    shouldRetry = false
    result = response
  }
}

Notice that it is the same idempotency key that is being passed across all retries. This indicates to the backend: "Hey, I am only interested in this operation happening once - can you please check if the operation has already succeeded. If so just respond with the result of the succeeded operation, otherwise, perform the operation now and store the result under this key so subsequent requests with the same key don't perform the operation multiple times".

Server perspective

Now, shifting to the server perspective, imagine that you are integrating your commerce API with a payment provider like Stripe and you need to allow your API's consumers to be able to issue refunds. You both have to keep track of the refunds in your system but will also have to call Stripe's API to make sure that the refund of the money goes through to the customer's bank account. Consider what steps your backend would have to take when handling a refund request - you may come up with something along the lines of this:

Validate that the requested amount can be refunded (i.e. less than the original payment for the order minus what has already been refunded).
Create a record of the refund in your database.
Issue refund through the Stripe API.
Store Stripe refund id in the internal record of refund.
Dispatch job to send a refund confirmation email to the customer
Complete request and respond

A naive implementation would just execute each of the steps and hope for the best, but that would be a bad idea. Consider what would happen in a case where a server experiences an outage and you have to decide whether to retry the request or not - you don't know which of the steps failed so it is unclear whether the Stripe request has been processed; if it has a new request would duplicate the refund which is obviously bad. Alternatively, if the Stripe request hasn't been processed you may have stale data in your internal database.

A slightly better approach would be to wrap everything into an ACID transaction and roll back if something fails. This way you don't end up having records in your database if something fails unexpectedly; however, in the case of an error you are still left in the dark as to whether the Stripe request was successfully processed or not; so how might you safely retry your failed request? Luckily, Stripe has support for idempotency keys so if your implementation makes sure to forward the idempotency key to Stripe you can safely retry your request without having to worry about refunding the requested amount more than once; however, it is not all external systems that have support for idempotency keys and under such circumstances, you need to take additional measures for your requests to be idempotent. You will see how this can be accomplished through atomic phases shortly.

Idempotency Key implementation in Expressjs

The outline here will reveal how to implement idempotency keys in a NodeJS application with Express. It is assumed that the underlying database for the application is an ACID-compliant relational database like Postgresql. Before going further it should be clarified what exactly is meant by an idempotency key in this context: an idempotency key is a string that identifies a database entity that tracks the progress of an API request. By tracking the progress, idempotency keys can either pick up where previously failed requests left off or if a previous request succeeded they can be used to return a cached result of the request.

Building further on the idea of a Cart API, consider the API request needed to transform a Cart into an Order. The steps to take will be something like the following:

Consider the steps in the above request and what your system state and your payment providers state will have recorded in each of the failure points. You may consider each of them and find the following:

Failure point #1

You have created a record of the incoming request, but have failed to authorize the payment and no order has been created. You can safely retry the request.

Failure point #2

The payment has successfully been authorized and a record of the payment is stored. The order has not been created. If you retry the request now you will be authorizing the payment again - this may fail or worse authorize a new payment duplicating the payment from the previous request. Unless some compensation mechanism has been implemented in your authorization logic that checks for a previous payment it is generally not safe to retry the request.

Failure point #3

At this point you have both authorized the payment and created an order in your system. Retrying the request may result in both a duplicate order and a duplicate payment authorization.

Now consider what will happen if you wrap your entire request in a transaction that rolls back after each of the failure points. For failure point 1 you can safely retry, but rolling back at failure point 2 and 3 will result in your own state and the external state of the payment provider being out of sync. Namely the payment provider will have a payment that your internal system has no record of. In order to overcome this problem you must be able to recover from failed requests depending on whether the external system mutation has been completed or not. In simple terms a request retry should be able to say: "If the payment was already authorized skip that step and continue with creating the order. If the payment wasn't authorized do that now and continue". The points in the request lifetime where you wish to be able to retry from will be called recovery points in the following discussion.

Atomic phases

Between each recovery point you will complete an atomic phase, which is a set of operations that happen within a transaction. If one of the operations fail you will roll back the atomic phase and a retry of the request can then pick up from the recovery point that came before the atomic phase. Considering the request lifecycle above once again, you should realize that you will want 3 atomic phases. One before the payment authorization when the idempotency key is created, one containing the payment authorization and one after the payment authorization has been completed. The diagram below illustrates the atomic phases and each of the recovery points:

Retrying failed requests will now pick up from the most recently reached recovery point meaning that new requests will either skip the payment authorization or retry it if it failed, but will never duplicate it.

Now that you have a rough idea about the parts of the system that you will need to keep track of it is time to look at how you might implement this starting with a simplified database schema.

IdempotencyKey
- id
- idempotency_key
- request_path
- request_params
- response_code
- response_body
- recovery_point

Payment
- id
- payment_provider
- idempotency_key
- amount

Cart
- id
- items
- completed_at

Order
- id
- items
- payment_id

Note that the idempotency key entity notes which path and which parameters an API call is requesting. It also has fields for the response code and body to send after the API call has succeeded so that retries of completed requests can skip directly to the response.

To make atomic phases easy to work with consider the implementation below from Medusa's IdempotencyKeyService.

async workStage(idempotencyKey, func) {
    try {
      return await this.transaction(async (manager) => {
        let key

        const { recovery_point, response_code, response_body } = await func(
          manager
        )

        if (recovery_point) {
          key = await this.update(idempotencyKey, {
            recovery_point,
          })
        } else {
          key = await this.update(idempotencyKey, {
            recovery_point: "finished",
            response_body,
            response_code,
          })
        }

        return { key }
      }, "SERIALIZABLE")
    } catch (err) {
      return { error: err }
    }
  }

The IdempotencyKeyService in Medusa allows you to execute an atomic phase by using the service method called workStage, which takes an idempotencyKey string and a func function containing the operations to be executed inside the atomic phase. The function can return either a recovery_point string in which case the idempotency key's recovery point is updated to that value or alternatively a response_body and response_code in which case it is assumed that the operation is completed and we can allow the recovery point to be updated to "finished".

API controller implementation

Now it is time to implement the API controller that takes in the request to create an order from a cart. Below you are using a state machine pattern to step through each of the API request's atomic phases.

Notice that the first step in the implementation is to upsert the idempotency key: either by using a provided token in the Idempotency-Key header or alternatively by creating a new one at random (this happens in initializeRequest).

Once the idempotency key is retrieved the request moves into the state machine where the recovery point of the idempotency key determines which atomic phase should be executed first. If the most recent recovery point is "started" the request moves to authorization of the payment, if that has already been completed the request goes straight to creating the order.

The code snippet below is a simplified version of Medusa's request handler.

export default async (req, res) => {
  const { id } = req.params

  const idempotencyKeyService = req.scope.resolve("idempotencyKeyService")
  const cartService = req.scope.resolve("cartService")
  const orderService = req.scope.resolve("orderService")

  const headerKey = req.get("Idempotency-Key") || ""

  let idempotencyKey
  try {
    idempotencyKey = await idempotencyKeyService.initializeRequest(
      headerKey,
      req.method,
      req.params,
      req.path
    )
  } catch (error) {
    res.status(409).send("Failed to create idempotency key")
    return
  }

  res.setHeader("Access-Control-Expose-Headers", "Idempotency-Key")
  res.setHeader("Idempotency-Key", idempotencyKey.idempotency_key)

  let inProgress = true
  let err = false

  while (inProgress) {
    switch (idempotencyKey.recovery_point) {
      case "started": {
        const { key, error } = await idempotencyKeyService.workStage(
          idempotencyKey.idempotency_key,
          async (manager) => {
            let cart = await cartService.withTransaction(manager).retrieve(id)

            if (cart.completed_at) {
              return {
                response_code: 409,
                response_body: {
                  code: MedusaError.Codes.CART_INCOMPATIBLE_STATE,
                  message: "Cart has already been completed",
                  type: MedusaError.Types.NOT_ALLOWED,
                },
              }
            }

            cart = await cartService
              .withTransaction(manager)
              .authorizePayment(id, {
                ...req.request_context,
                idempotency_key: idempotencyKey.idempotency_key,
              })

            return {
              recovery_point: "payment_authorized",
            }
          }
        )

        if (error) {
          inProgress = false
          err = error
        } else {
          idempotencyKey = key
        }
        break
      }

      case "payment_authorized": {
        const { key, error } = await idempotencyKeyService.workStage(
          idempotencyKey.idempotency_key,
          async (manager) => {
            const cart = await cartService
              .withTransaction(manager)
              .retrieve(id, {
                select: ["total"],
                relations: ["payment", "payment_sessions"],
              })

            const order = await orderService
              .withTransaction(manager)
              .createFromCart(cart.id)

            return {
              response_code: 200,
              response_body: { data: order, type: "order" },
            }
          }
        )

        if (error) {
          inProgress = false
          err = error
        } else {
          idempotencyKey = key
        }
        break
      }

      case "finished": {
        inProgress = false
        break
      }

      default:
        idempotencyKey = await idempotencyKeyService.update(
          idempotencyKey.idempotency_key,
          {
            recovery_point: "finished",
            response_code: 500,
            response_body: { message: "Unknown recovery point" },
          }
        )
        break
    }
  }

  if (err) {
    throw err
  }

  res.status(idempotencyKey.response_code).json(idempotencyKey.response_body)
}

Notice how unexpected errors are bubbled out to the application controller - it is assumed that your Express app has an error boundary, that handles the error properly. Expected errors that are definitive, that is no matter how many calls you make it should always result in the same error code, which can be stored in the idempotency key so that subsequent requests can short circuit and send the cached response directly.

Using this pattern across your API endpoints will improve the robustness of your API by making it safe to retry all requests. This is useful for requests that modify the internal state alone, but the concept is especially powerful when dealing with requests that modify external states outside the control of your system. The key to making requests like these safe is to wrap external state modifications in atomic phases and allow retries to pick up both before or after such modifications, depending on the progress made from previous requests.

Idempotency in Medusa

In Medusa idempotency has so far been implemented for a handful of API requests, and support is continually being added to more endpoints. The goal is to support idempotency keys for all state-mutating requests so that you can be certain that retrying your requests is safe and harmless. The next step for Medusa will be to add idempotency patterns into the plugin APIs so that Medusa's core can implement self-healing logic that identifies and resolves inconsistencies between systems in your e-commerce stack. This will be a major improvement for the developer experience related to building headless commerce solutions, where there are lots of moving parts and hence lots of potential points of failure.

What's next?

If you wish to dive deeper into how idempotency keys are implemented in Medusa visit the Medusa GitHub repository. You are also more than welcome to join the Medusa Discord server, where you can get direct access to the Medusa engineering team, who will be happy to answer any questions you might have.

Thanks for reading and if you haven't already go check out the post by Brandur that inspired the implementation of idempotency keys in Medusa. Brandur also has a number of other articles that are definitely worth reading if you are looking to improve the robustness of your APIs.

Making your store more powerful with Contentful

Sebastian Rindom — Fri, 08 Oct 2021 16:19:28 +0000

In part 1 of this series you have set up Medusa with Contentful as your CMS system and added a Gatsby storefront. In this part you will get a further introduction to Contentful and learn how medusa-plugin-contentful can be leveraged to make your store more powerful. Apart from a front page, product pages and a checkout flow, most ecommerce stores also need miscalleneous pages like About and Contact pages. In this guide you will add a Rich Text content module to your Contentful space so that you can make this pages cool. You will also see how the content modules can be used to give your product pages more life.

What you will do in this guide:

Add a rich text content module
Add rich text to your /about page
Add a "Related Products" section to your product page

Topics covered:

Contentful Migrations
Product enrichment

Creating a rich text content module

In this guide you will make use of Contentful Migrations to keep a versioned controlled record of how your Content evolves over time. The Contentful app allows you to create content models straight from their dashboard, however, when using the migrations tool you will be able to 1) quickly replicate your Contentful space and 2) incorporate migrations as part of a CI/CD pipeline. You can read more about how to use CMS as Code here.

To prepare your migration create a new file at contentful-migrations/rich-text.js and add the following code:

// contentful-migrations/rich-text.js

module.exports = function (migration, context) {
  const richText = migration
    .createContentType("richText")
    .name("Rich Text")
    .displayField("title");

  richText.createField("title").name("Title (Internal)").type("Symbol");
  richText.createField("body").name("Body").type("RichText");
};

This small snippet will create a content model in your Contentful space with two fields: a title which will be used to name entries in a meaningful manner (i.e. it won't be displayed to customers) and a body which contains the rich text to display. To apply your migration run:

yarn migrate:contentful --file contentful-migrations/rich-text.js

If you go to your Contentful space and click Content Model you will see that the Rich Text model has been added to your space:

The validation rules in the Page model only allow Hero and Tile Sections to be added to the Content Modules fields so you will need another migration to make it possible for pages to make use of the new Rich Text modules. Create a new migration at contentful-migrations/update-page-module-validation.js and add the following:

// contentful-migrations/update-page-module-validation.js

module.exports = function (migration, context) {
  const page = migration.editContentType("page");

  page.editField("contentModules").items({
    type: "Link",
    linkType: "Entry",
    validations: [
      {
        linkContentType: ["hero", "tileSection", "richText"],
      },
    ],
  });
};

After migrating your space you are ready create your new contact page:

yarn migrate:contentful --file contentful-migrations/update-page-module-validation.js

Adding Rich Text to About

To use your new Rich Text module Content > Page > About, and click Add Content > Page. You will now make use of the new Rich Text module to add some more details about your store. You can write your own text or use the text provided below if you just want to copy/paste.

About Medusa

Medusa is an open-source headless commerce engine for fast-growing businesses. Getting started with Medusa is very easy and you will be able to start selling online with a basic setup in no time, however, the real power of Medusa starts showing up when you add custom functionality and extend your core to fit your needs.

The core Medusa package and all the official Medusa plugins ship as individual NPM packages that you install into a Node project. You store and plugins are configured in your medusa-config.js file making it very easy to manage your store as your business grows. Custom functionality doesn't have to come from plugins, you can also add project-level functionality by simply adding files in your src/ folder. Medusa will automatically register your custom functionalities in the bootstrap phase.

When you have added your text you can click Publish changes (make sure the About page is published too).

Updating the storefront to support the Rich Text module

If you want to jump straight to the final frontend code visit medusajs/medusa-contentful-storefront@part-2.

To display your newly created Rich Text module open up the storefront code and create a new file at src/components/rich-text/rich-text.jsx.

// src/components/rich-text/rich-text.jsx

import React from "react"
import { renderRichText } from "gatsby-source-contentful/rich-text"

import * as styles from "../../styles/rich-text.module.css"

const RichText = ({ data }) => {
  return (
    <div className={styles.container}>
      <div className={styles.content}>
        {data.body ? renderRichText(data.body) : ""}
      </div>
    </div>
  )
}

export default RichText

The renderRichText function is imported from the gatsby-source-contentful plugin to easily transform the text you entered in the Rich Text module to html. To make the Rich Text component render nicely add a style file as well at src/styles/rich-text.module.css.

/* src/styles/rich-text.module.css */

.container {
  display: flex;
  padding-top: 100px;
  padding-bottom: 100px;
}

.content {
  margin: auto;
  max-width: 870px;
}

If you restart your storefront server now you will not be able to see your new Rich Text module just yet. The last step to making that happen will be to let the Page component know to render the new Rich Text component when it encounters Rich Text in the Page's Content Modules. In your editor open up the file src/pages/{ContentfulPage.slug}.js and add the following:

At the top of the file import your RichText component:

...
import RichText from "../components/rich-text/rich-text"
...

Now in the contentModules.map function return the RichText component whenever a ContentfulRichText module is encountered. Add a case to the switch statement:

    case "ContentfulRichText":
      return <RichText key={cm.id} data={cm} />

Finally you will need to fetch the Rich Text data from Gatsby's data layer by modifying the GraphQL code at the bottom of the file after the line with contentModules { add:

    ... on ContentfulRichText {
      id
      body {
        raw
      }
      internal {
        type
      }
    }

Restart your local Gatsby server and visit http://localhost:8000/about, you will now see the your newly added Rich Text module.

Enriching your Product pages

You have now seen how the Page model in Contentful can be extended to include a new content module in a reusable and modular manner. The same idea can be extended to your Product pages allowing you to create completely bespoke universes around your products. You will use the same techniques as above to create a Related Products section below the "Medusa Shirt" product.

Migrating Products

First, add a new field to the Product content model. Using migrations you can create a file contentful-migrations/product-add-modules.js:

// contentful-migrations/product-add-modules.js

module.exports = function (migration, context) {
  const product = migration.editContentType("product");

  product
    .createField("contentModules")
    .name("Content Modules")
    .type("Array")
    .items({
      type: "Link",
      linkType: "Entry",
      validations: [
        {
          linkContentType: ["hero", "tileSection", "richText"],
        },
      ],
    });
};

Run the migration:

yarn migrate:contentful --file contentful-migrations/product-add-modules.js

Adding "Related Products" Tile Section

After the migration you can now add Content Modules to Products, to enrich the Product pages with relevant content. In this guide you will add a Tile Section that holds "Related Products", but the functionality could be further extended to showcase look book images, inspirational content or more detailed product descriptions.

In Contentful go to Content > Product > Medusa Shirt scroll all the way to the bottom, where you should be able to find the new Content Modules field:

Click Add content > Tile Section which will open a new Tile Section. For the Title write "Related Products", and for Tiles click Add content > Add existing content > Medusa Waterbottle > Insert 1 entry.

Click Publish and make sure that the Medusa Shirt product is published too.

Your data is now ready to be used in the storefront, but you still need to make a couple of changes to the storefront code to be able to view the new content.

Adding Content Modules to Product pages

Just like you did for the Page component, you will have to fetch the Content Modules from Gatsby's GraphQL data layer.

In the file src/pages/products/{ContentfulProduct.handle}.js add the following in the GraphQL query at the bottom of the file (e.g. after the variants query):

  # src/pages/products/{ContentfulProduct.handle}.js

  contentModules {
    ... on ContentfulTileSection {
      id
      title
      tiles {
        ... on ContentfulProduct {
          id
          title
          handle
          thumbnail {
            gatsbyImageData
          }
          internal {
            type
          }
        }
        ... on ContentfulTile {
          id
          title
          cta
          image {
            gatsbyImageData
          }
          link {
            linkTo
            reference {
              slug
            }
          }
          internal {
            type
          }
        }
      }
      internal {
        type
      }
    }
  }

This snippet will query the Content Modules defined for the product and will allow you to use the data in your components.

Next open up the src/views/products.jsx file and add the following snippets.

Import the TileSection component:

import TileSection from "../components/tile-section/tile-section"

Add the Content Modules in the JSX just before the final closing div:

  // src/views/products.jsx

  <div className={styles.contentModules}>
    {product.contentModules?.map((cm) => {
      switch (cm.internal.type) {
        case "ContentfulTileSection":
          return <TileSection key={cm.id} data={cm} />
        default:
          return null
      }
    })}
  </div>

Restart the Gatsby server and visit http://localhost:8000/product/medusa-shirt you should now see the new "Related Products" Tile Section below the Product page controls.

Summary

In this guide you created a new content model for Rich Text input in Contentful using contentful-migration. You further extended the storefront to render the new Rich Text plugin. The concepts in this guide are meant to demonstrate how Contentful can be used to make your store more powerful in a modular and scalable way. The content modules covered in this guide could be further extended to add other custom modules, for example, you could add a Newsletter Signup, module that when encountered in the code renders a newsletter form.

What's next

In the next part of this guide you will learn how to implement further commerce functionalities to your site such as adding support for discount codes, region based shopping and more. (Coming soon)

Introducing Medusa: Your building blocks for amazing digital commerce experiences

Sebastian Rindom — Mon, 06 Sep 2021 08:02:18 +0000

Over the past couple of years we have developed Medusa in close collaboration with customers that needed a headless solution that could support their growth for years to come. We are so excited to finally open up about what it is we have built, why we have built it, and how you can succeed with Medusa. Medusa is an open-source headless commerce engine that can power amazing digital commerce experiences and it is a great choice for any new headless ecommerce project out there.

Headless ecommere has gained plenty of interest recently and more and more businesses are looking towards making the shift to a headless solution to leverage a more modular setup and all the benefits it brings, such as: better page performance, greater flexibility, better support for unique customer experiences, support for best-in-breed tech stacks, etc. When we first started working on Medusa we had no idea what headless ecommerce meant, and initially, we set out to build a fully-fledged platform, but more on this later!

This is the first blog post we are publishing and it will stand as an important milestone for our work going forward. We have found that through modularity and simplicity our merchants end up being able to do much more with much less, which is why we will keep pursuing this approach when building out our product further. We want to provide a foundational set of building blocks that can be used to create unique, interesting, and powerful digital commerce experiences and we are so excited to see what amazing work we can help create.

Why we built Medusa

We didn’t wake up one morning thinking to ourselves that we wanted to build a headless commerce engine, but rather we got to build Medusa through experience with the limitations of existing platforms. Before transitioning to working full-time on Medusa, we operated as an agency, building all kinds of tools for our customers. We built everything from portfolio websites to custom integrations to ecommerce websites, but quickly started specializing in the latter; as we found new customers we also experienced that many of them were hitting limitations with the platforms available at the time.

This became evident when a client of ours expressed interest in switching platforms. We started looking for good alternatives but it quickly became clear that none of them would support the expansion across markets and brand ownership that they required so we decided with them that we should try to build a bespoke solution that would cater to their exact needs and so we began building a fully-fledged ecommerce platform with a CMS system, fulfilment system, etc. It was not soon after that we realized that we couldn’t do everything well at once so we started removing parts of what we’d built and replacing them with integrations to better tools. In the end, we were left with a core that was capable of processing and managing orders, ensuring payments were going through, and handling basic product and customer data. What was even more important was that this core was extremely good at integrating other tools and as such able to function as the glue between all the tools and services in your ecommerce stack. Essentially we had built a headless ecommerce engine, simply by following what was easiest for us as developers and most cost-efficient and scalable for our customers.

Our customers were really happy with the solution as they were able to expand to multiple markets while picking and choosing from any services they found interesting without having to worry about the cost of integration. Furthermore, our customers got a high degree of ownership when using our custom-built solution, as they could ask us to build features that they needed and have them released typically within a couple of days. When contemplating how we could ensure the same level of ownership going forward it became clear that we had to open-source the solution so that no central organization would ever be in control of what was possible for the merchant. Open-sourcing Medusa would also bring a bunch of other benefits such as leveraging the community for feedback and contributions which further strengthens the product.

After experiencing the issues with the existing platforms we were confident that our product would be a good alternative for many new ecommerce projects and shortly after we decided to open-source the product we started pulling out the generic parts of the implementations we had done, gave it the name Medusa and made it available through GitHub. Go to Github to view the open source project.

Headless: The next generation of commerce

In the early days of the internet operating ecommerce required big budgets and with customers still being new to the digital purchasing experience, only a small group of large enterprises set up online commerce channels by either building them from scratch or using solutions like Hybris. As more customers moved online, new platforms as Magento, Shopify, and WooCommerce emerged — these were easy to use and enabled merchants to make digital commerce available as a secondary sales channel.

In recent years, ecommerce has become a primary channel for many businesses leading to fierce competition in the space, increasing the need for custom solutions to give unique buying experiences. In the quest for customization, many businesses have been left frustrated with the lack of flexibility that existing platforms provide. While optimizing for ease of use, most traditional platforms is putting their focus on templated “out-of-the-box” solutions neglecting the constraints this inevitably put on the development flexibility. In many cases, businesses have been forced to converge towards compromised solutions, or even make entire platform shifts, as their needs grow beyond what a templated solution can offer. Simultaneously, a much wider variety of businesses (e.g. service providers, B2B companies etc.) requiring fully custom solutions from the beginning have started moving online.

The increasing demand for customization and flexibility has spurred an interest in headless commerce among performance-focused merchants that want differentiated and fully optimized customer experiences. Yet, many of them are still to make the move. One of the main reasons, why headless has yet to become the dominant type of ecommerce is that it can be expensive to operate a headless ecommerce stack as you need developers that can maintain the infrastructure that powers the integrations between systems in your stack and a high up-front investment for making the initial setup. This is something we are solving with Medusa by providing infrastructure that is preconfigured and optimized to run your projects.

Putting the developer first in your commerce setup

The modularity and ownership that Medusa can offer is capable of powering incredibly diverse applications ranging from ecommerce websites to more niche cases like in-restaurant experiences and will with high probability be suitable for whatever digital commerce experience you can think of. Many businesses postpone the transition to headless as it can seem daunting and complex when you need developers to maintain your infrastructure and operations; we will make it possible to take out these concerns completely by providing the infrastructure that is configured and optimized to run Medusa projects, thereby making it much cheaper and more accessible for businesses to make the transition to headless early on.

In more general terms we will be focusing on enabling the developer to do much more with less; this is in line with making sure that merchants can take ownership of their commerce setup as developers can help steer the direction and roadmap for your project without relying on us to create the features you need. By creating powerful tools for the developer we enable them to be more efficient and focused in their work. Examples of tools that we are building are; an infrastructure platform so developers don’t have to configure servers and databases, a powerful CLI that automates the redundant tasks developers face, a React component library to make it easy to build storefronts for Medusa, rich documentation that makes it easy to find what you are looking for and much more.

With the core project being open-source we will also be open for contributions from the community making sure that we always have a product that is loved by developers so that they can do more!

What’s next?

We are working hard on completing our infrastructure product so that projects can easily be deployed to the cloud. In the meantime, if you are a developer we encourage you to try out Medusa, you can get up and running within 5 minutes and get a quick overview of what features our platform will bring. Check out the tutorial to get started now.

If you are a merchant looking for a new solution we would love to get in touch and discuss what challenges we can help solve and guide you through our process.