Forem: Luiz Damim

Reorganizing your (Phoenix) Contexts as Use Cases

Luiz Damim — Tue, 04 Jun 2019 02:12:58 +0000

Contexts are a great concept to help you identify well defined boundaries between the different areas of your application. There is nothing new or specific about Phoenix Contexts, it's just a subset - identification and organization - of Bounded Contexts, defined by Eric Evans in the excellent Domain-Driven Design: Tackling Complexity in the Heart of Software.

In this post I'll show you another way to organize your Contexts by extracting actions into their own Use Cases modules to convey better clarity and make requirements explicit.

What is a Use Case?

A use case is a written description of how users will perform tasks on your website. It outlines, from a user’s point of view, a system’s behavior as it responds to a request. Each use case is represented as a sequence of simple steps, beginning with a user's goal and ending when that goal is fulfilled. - Usability.gov

In other words, a Use Case is an action that some user/actor performs on your application, like a button click in the UI, a command entered via CLI or even a response from a external API. It is something that the actor does and expects a side effect: a data change, sending an email, a background calculation.

Phoenix already creates three generic use cases when you use its generator to create or update a context: create_*/1, update_*/2 and delete_*/1 functions. In a Sales context with Client and Order, Phoenix defines the following functions:

create_client/1
update_client/2
delete_client/1
create_order/1
update_order/2
delete_order/1

Now add order items, payments, addresses and the concept of checkout to the context. Also add the other necessary accompanying functions like data validation, policies verification, external data fetching, email sending and event logging, and the number of functions will increase drastically.

There's nothing technically wrong defining all these functions inside a single module in Elixir, but as contexts grows larger you start to have too many functions related to each other but loosely related to the parent concept. The cognitive overhead to build a mental state of all these functions is too much.

Writing a Use Case

Steps for writing a Use Case:

identify the actions a user can perform
extract each action in its own Use Case module
...
PROFIT!

No joke, that's it. Here's an example for an order cancellation:

defmodule AlchemyReaction.Sales.CancelOrderManually do
  @moduledoc """
  Cancels an order with the given reason and notifies the client via email.
  """
  use Ecto.Schema
  import Ecto.Changeset
  alias Ecto.Multi

  alias AlchemyReaction.Sales.Order

  embedded_schema do
    field :reason
  end

  def changeset(attrs) do
    %__MODULE__{}
    |> cast(attrs, [:reason])
    |> validate_required([:reason])
    |> validate_length(:reason, min: 10)
  end

  def call(%Ecto.Changeset{valid?: true} = changeset, %{order: order, actor: actor} = _deps) do
    data = apply_changes(changeset)

    Multi.new()
    |> Multi.update(:order, Order.changeset(order, %{status: "cancelled"}))
    |> Multi.run(:audit, &cancel_order_audit(&1, &2, data.reason, actor))
    |> Multi.run(:email, &notify_client(&1, &2, actor))
  end

  defp cancel_order_audit(_repo, %{order: order}, reason, actor) do
    # ...
  end

  defp notify_client(_repo, %{order: order}, actor) do
    # ...
  end
end

The key benefits of writing Use Cases are:

Code organization and clarity

You can know, or at least have an idea, what each context do just by looking at the file structure. You can also group the Use Cases in sub contexts folders on larger contexts for better organization. In this case the CancelOrderManually would be nested in the sub context order and named as AlchemyReaction.Sales.Orders.CancelOrderManually.

You make the requirements explicit

It's clear that a reason, the order being cancelled and the actor who is doing it are neededed to manually cancel the order. Here I'm passing the order and the actor as dependencies to the call/2 function because I'm loading them outside, mainly for checking permissions, but you could have the order_id and actor_id defined in the schema and load them inside the use case.

Keep in mind that requirements varies depending on the execution context. A reason is required to manually cancel an order by an operator in the customer support, but it may not be required by the analist in the fraud detection department. This is a terrible example, but I hope I can make you through.

Also note that these requirements are strictly related to what is needed to execute the Use Case, there's nothing to do with roles and permissions. Both the operator in the customer support and the analyst in the fraud detection have permissions to manually cancel an order.

Use the Use Case to build your forms

The changeset serves as the data structure to build your forms the same way you would use an Order schema in a traditional way (you will need to set the as namespace and the submission method manually as the form_for/x helper can't infer from the struct).

You are decoupling the requirements to execute the use case from the underlying implementation, as oftentimes a change can span multiple schemas.

It's easy to reuse and compose Use Cases

You must always return an Ecto.Multi from the call/2 function. Always is a strong word, but I really mean ALWAYS!

Ecto.Multi makes it possible to pack operations that should be performed in a single database transaction and gives a way to introspect the queued operations without actually performing them. Each operation is given a name that is unique and will identify its result in case of success or failure. - Ecto.Multi docs

By always returning a multi you can reuse other Use Cases, composing them together by appending/merging them into the parent multi. Think of cancelling an order where you also have to cancel the payment, put back the items in stock and maybe scheduling an email for inviting the user to buy again in 2 weeks. Each of these use cases can be executed by themselves giving other contexts.

I use Ecto.Multi even when there's no database operation. The cost of opening/close an unutilized transaction is negligible.

Ecto.Multi is love. Ecto.Multi is life. ❤️ ❤️ ❤️

Don’t make your Use Cases generic or (overly) configurable

One of the key benefits of a Use Case is that it reflects a single, defined, finite execution path in the application. If you try to make it generic enough to handle multiple use cases, then you lose the benefit of clarity and context.

Having a flag to indicate that an email notification should be sent after performing the use case is ok. Having multiple options and knobs resulting in multiple execution paths and outputs is a red flag.

Identifying a Use Case

This is the hardest part. It's a constant process of mapping and understanding why and how data changes by analyzing the day to day of the application's users and the corner cases that surface from time to time.

Pay attention to the language they use, specially the verbs. Verbs denote actions and actions are what you're interested in.

It's ok if you don't get it right the first (or second, or third) time. But it's important you keep refining...

Some other examples of Use Cases:

Sales context: ShipOrder, RefundOrder, ChangeShippingAddress, CancelOrderViaPaymentGateway (naming is hard);
Accounts context: ChangePrimaryEmail, ChangePassword, DeactivateAccount, ConfirmEmailAddress, SetPreferredPaymentMethod, SetPreferredShippingAddress;

Conclusion

It may seem cumbersome to write this much code to do the same thing as the traditional way. I don't have any metrics, but my feeling is that it's not much more, it could even be the same. You're not just typing code out, you're mapping every operation and process of the application, identifying and documenting every execution path.

I've been using this concept of Use Cases with great success for multiple years, in applications written in Elixir, Ruby and even PHP.

Tracking changes with context using Phoenix and Ecto

Luiz Damim — Thu, 16 May 2019 00:00:00 +0000

In the previous post we discussed why it's important to add context to the application's data changes. Now we're going to see how we can do that in Elixir with Phoenix and Ecto.

Adding context to Context

Context is an overloaded term, specially in programming. Let's see the two definitions we will be using:

Context:

1) the situation within which something exists or happens, and that can help explain it; 2) the influences and events related to a particular event or situation;

(Cambridge Dictionary)

(Phoenix) Context:

Contexts are dedicated modules that expose and group related functionality. For example, anytime you call Elixir's standard library, be it Logger.info/1 or Stream.map/2, you are accessing different contexts. Internally, Elixir's logger is made of multiple modules, but we never interact with those modules directly. We call the Logger module the context, exactly because it exposes and groups all of the logging functionality.

(Phoenix docs)

They mean different things, but work really well together organizing and giving meaning to what can happen and why it happened. From now on I'll refer to contexts with the meaning of explaining a situation or something that happened as events.

Cancelling an order

Following our previous example of an online shop, we have two Phoenix Contexts: Accounts and Sales. In Accounts we have a User schema and in Sales, Client and Order schemas.

To cancel an order we could have a Sales.cancel_order/1 function as simple as:

def cancel_order(%Order{} = order) do
  order
  |> Order.changeset({status: "cancelled"})
  |> Repo.update()
end

Order cancelled. But why?

To know why an order was cancelled, let's refactor the function and add the reason and the user who did it.

def cancel_order(%Order{} = order, reason, %User{} = actor) do
  Multi.new()
  |> Multi.update(:order, Order.changeset(order, %{status: "cancelled"}))
  |> Multi.run(:audit, &cancel_order_audit(&1, &2, reason, actor))
  |> Repo.transaction()
end

defp cancel_order_audit(_repo, %{order: order}, reason, actor) do
  metadata = %{
    source: "manual",
    reason: reason
  }

  "order_cancelled"
  |> OrderAudit.event(order.id, actor.id, metadata)
  |> OrderAudit.save()
end

We're also using Ecto.Multi to make the operation atomic (it means that the event log will be saved only when the order is cancelled without errors) and added a cancel_order_audit/4 callback function that setups the metadata and saves the changes.

The OrderAudit module is a standard Ecto Schema, with a twist:

defmodule AlchemyReaction.Sales.OrderAudit do
  # highlight-range{1-6}
  use AlchemyReaction.Audit,
    repo: AlchemyReaction.Repo,
    schema: __MODULE__,
    events: [
      "order_cancelled"
    ]

  use Ecto.Schema
  import Ecto.Changeset

  alias AlchemyReaction.Accounts.User
  alias AlchemyReaction.Sales.Order

  schema "orders_audit" do
    field :event, :string
    field :metadata, :map
    belongs_to :row, Order
    belongs_to :actor, User

    timestamps(updated_at: false)
  end

  @doc false
  def changeset(%__MODULE__{} = order_audit, attrs) do
    order_audit
    |> cast(attrs, [:event, :row_id, :metadata, :actor_id])
    |> validate_required([:event, :row_id, :actor_id])
  end
end

We are using the Audit module and setting some options, including the events list.

There's only one event, order_cancelled, but we could have many more: item_removed, payment_type_changed, shipping_address_changed and so on.

I usually have multiple audit modules inside a context, one for each sub context.

Sub context?

For those familiar with DDD, they are like aggregate roots.

The Order schema is composed of other smaller schemas that makes no sense existing outside it, like order items and payments. So the Order aggregates this additional data, and as the most important one, it is the root, hence Aggregate Root. But I like to call them sub contexts because the terminology is closer to Phoenix Contexts. :)

Can you come with another good candidate for a sub context in Sales context?

IMPORTANT: a sub context is an internal concept and should only be interacted with from it's parent context. In this example, all calls from outside will still happen to Sales context.

Caveat: an Ecto schema should be a simple mapping between a data source and an Elixir struct, but I couldn't find a better place for this example. In my actual application I organize my contexts a little different so this doesn't happen. I'll write about it in the future.

The `Audit` module

So what is it? How does it work?

Here's the full code:

defmodule AlchemyReaction.Audit do
  require Logger

  defmacro __using__(opts) do
    repo = Keyword.fetch!(opts, :repo)
    schema = Keyword.fetch!(opts, :schema)
    events = Keyword.fetch!(opts, :events)

    if Enum.empty?(events), do: raise(ArgumentError, message: "event list empty")

    save_function =
      quote do
        # Output:
        #
        # def save(%Ecto.Changeset{} = changes) do
        #   AlchemyReaction.Audit.save(changes, repo)
        # end
        def save(%Ecto.Changeset{} = changes) do
          unquote(__MODULE__).save(changes, unquote(repo))
        end
      end

    event_signature =
      quote do
        def event(event, row_id, actor_id, metadata \\ nil)
      end

    event_log_functions =
      for event_name <- events do
        quote do
          # Output:
          #
          # def event("foo_event", row_id, actor_id, metadata) do
          #   AlchemyReaction.Audit.event(schema, "foo_event", row_id, actor_id, metadata)
          # end
          def event(unquote(event_name), row_id, actor_id, metadata) do
            unquote(__MODULE__).event(
              unquote(schema),
              unquote(event_name),
              row_id,
              actor_id,
              metadata
            )
          end
        end
      end

    [save_function, event_signature, event_log_functions]
  end

  @doc """
  Saves the event to the `Repo`.

  In case nothing changed, simply does nothing.

  It returns `{:ok, :no_changes}` if nothing changed, `{:ok, struct}` if the log
  has ben successfully saved or `{:error, changeset}` in case of error.
  """
  @spec save(Ecto.Changeset.t() | :no_changes, Ecto.Repo.t()) ::
          {:ok, :no_changes}
          | {:ok, Ecto.Schema.t()}
          | {:error, Ecto.Changeset.t()}

  def save(:no_changes, _repo) do
    {:ok, :no_changes}
  end

  def save(%Ecto.Changeset{} = changes, repo) do
    Logger.debug("Saving audit info...")
    repo.insert(changes)
  end

  @doc """
  Creates a `schema` changeset for the `event` identified by `row_id` and caused
  by `actor_id`.

  The given `metadata` can be either `nil`, `Ecto.Changeset`, struct or map.

  It returns `:no_changes` in case of an `Ecto.Changeset` metadata that changed nothing
  or an `Ecto.Changeset` with the event ready to be inserted.
  """
  @spec event(Ecto.Schema.t(), String.t(), non_neg_integer(), non_neg_integer(), Ecto.Changeset.t() | struct() | map() | nil) ::
          :no_changes | Ecto.Changeset.t()

  def event(schema, event, row_id, actor_id, metadata \\ nil)

  def event(_, _, _, _, %Ecto.Changeset{changes: changes} = _changeset)
      when map_size(changes) == 0 do
    :no_changes
  end

  def event(schema, event, row_id, actor_id, %Ecto.Changeset{} = changeset) do
    metadata = %{
      before: Map.take(changeset.data, Map.keys(changeset.changes)),
      after: changeset.changes
    }

    audit_changeset(schema, event, row_id, actor_id, metadata)
  end

  def event(schema, event, row_id, actor_id, %_{} = struct) do
    metadata =
      struct
      |> Map.from_struct()
      |> Map.drop([:__meta__])

    audit_changeset(schema, event, row_id, actor_id, metadata)
  end

  def event(schema, event, row_id, actor_id, metadata) when is_map(metadata) do
    audit_changeset(schema, event, row_id, actor_id, metadata)
  end

  defp audit_changeset(schema, event, row_id, actor_id, metadata) do
    schema
    |> struct()
    |> schema.changeset(%{
      event: event,
      row_id: row_id,
      actor_id: actor_id,
      metadata: metadata
    })
  end
end

Using this module defines a pair of event/3 and event/4 functions for each listed event, helping you avoid typing out a wrong event name thanks to pattern matching. The difference between them is the metadata parameter. Most events will need additional metadata, but some of them don't, the event name alone is sufficient to describe what happened.

A save/1 function is also defined. To save the event log. For real. Except when there's nothing to save because nothing changed. But saves most times...

The event/3 and event/4 functions delegate the calls to the event/5 inside the Audit module itself that creates the log changeset. Thanks to pattern matching again the metadata can be built from different sources:

an Ecto.Changeset: the changes and their original values are extracted in a %{before: %{}, after: %{}} map; or it returns a :no_changes atom in case nothing changed. The latter bypasses saving so it doesn't create a useless log entry
a struct: all fields from the struct are added to the metadata
a map: anything you need to contextualize the event. Take care to put only what's necessary, adding too little or too much hinders your ability to see what happened clearly

At first it may seem too much work to manually create the callback functions and build the events, but it pays off. The event log gives you multiple benefits: you can show the changes in a timeline, store changes over time for accountability reasons, have a traceability of changes for security, an much more.

I'm using this in one of my applications and it's been a blast.

Tracking changes with context

Luiz Damim — Wed, 01 May 2019 00:00:00 +0000

Imagine an online shop who sells... anything, really. Users place orders buying products they want. But sometimes they regret their decision ~~me 2min after buying something~~. Sometimes there's a problem with the payment gateway. Sometimes you have a problem in your warehouse and the products are out of stock. Sometimes you can't sell because you can't ship to the user's address or any other constraint that can't be verified at checkout time (please please PLEASE verify everything you can at checkout time, everyone gets happy :)).

All these possibilities result in the order cancellation. Behind the scene the application updates the order's status field from processing to cancelled and hopefully logs the date/time. That's it. Right? RIGHT?

Sometimes knowing What changed is not enough. It's essential to know Why it changed.

If you don't know why something happened you are losing insight about your application, making you unable to act upon it.

Example 1: User doesn't have sufficient funds

Payment declined because the user doesn't have sufficient funds. The application cancels the order automatically.

The fields gateway and reason could be set by your application's payment workflow and error_code returned by the gateway processor.

{
  "event": "order_cancelled",
  "id": 42,
  "actor_id": null,
  "metadata": {
    "reason": "payment_declined",
    "error_code": "INSUFFICIENT_FUNDS",
    "gateway": "IPayU"
    ...
  }
}

Although in this example actor_id is null, I usually have an application user that is assigned to every automated action in the application.

Example 2: Can't ship to the user's address

User changed their shipping address after placing the order and for whatever reason you can't ship there.

Here reason and description were informed by an employee after manually analyzing the problem.

{
  "event": "order_cancelled",
  "id": 42,
  "actor_id": 19,
  "metadata": {
    "reason": "shipping_address",
    "description": "User changed his address, can't ship to the new one."
    ...
  }
}

These are contrived examples. There are better ways to handle these cases than cancelling the order.

On the next post I'll show one way you can do it in Elixir using Ecto and Phoenix.

Many thanks to Allan Jorge, @gustavoaguiar and @tyurok from Elixir Brazil OffTopic @ Telegram for helping me overcome my writing block / fear of writing this first post.