Forem: Max

Getting React on Rails to work with Turbo Streams

Max — Thu, 15 Jan 2026 09:30:54 +0000

On a recent client project, redesigning a large Rails app, we found ourselves having to get Rails, Hotwire's Turbo and React on Rails to play together nicely. Each of the tools comes with its own amounts of magic™, so integrating them turned out to be a bit of a juggle.

Scenario

On this project, we were implementing the new design for a particular page. At it's core, it shows two lists, each with a "Load More" button to asynchronously fetch and append more items to it. The first list is just a stack of items, the second one also includes a grouping of the items. Imagine it roughly like this:

" button below. List B is divided into groups Group A, Group B and Group C"/>

In "List A", clicking "Load More" should append another batch of 5 items to the bottom of the list until the bitter end. For "List B", clicking "Load More" should also append another batch of 5 items, where some may still belong to "Group C" and others will form new groups, each with their respective group header. The list items are heterogeneous and contain complex markup with graphics, etc.

Pagination

We decided to implement the pagination logic using Turbo Streams, as they seem to be a good fit for our use case. They also integrate nicely into the Rails ecosystem without being too tightly coupled (e.g. to a specific version of Rails). From Rails 7 on, Turbo also replaced Rails' UJS (Unobtrusive JavaScript) functionality and is the recommended tool for sprinkling asynchrony and SPA (Single Page Application)-feelings. From the Turbo Stream docs:

Turbo Streams deliver page changes as fragments of HTML wrapped in <turbo-stream> elements. Each stream element specifies an action together with a target ID to declare what should happen to the HTML inside it. These elements can be delivered to the browser synchronously as a classic HTTP response, or asynchronously […]

And further on:

They can be used to surgically update the DOM after a user action such as removing an element from a list without reloading the whole page […]

This sounds like a match for what we are doing!

By having our server respond with turbo_stream templates for the pagination requests, we are able to use the same View Components and partials we already have in place from the initial server-rendered page. This is important to us because we're already using our component library throughout the whole project. Reimplementing the same components in a different framework or language just for this single page would be unnecessary duplication, not to mention the constant risk of the two implementations diverging.

Example

For the sake of simplicity, let's focus on the first list, "List A". The initially rendered page would have a markup like this for the list:

<h2>List A</h2>
<ul id="list-a">
  <li><abc-item/></li>
  <li><abc-item/></li>
  <li><abc-item/></li>
  <li><abc-item/></li>
  <li><abc-item/></li>
</ul>
<turbo-frame id="load-more-a">
  <load-more-button/>
</turbo-frame>
<!-- ... -->

When the user clicks the "Load More" button, we want to retrieve the next batch of items, append them to the list and replace the button with one to request the next batch. If there is no next batch of items, we remove the "Load More button.

In the controller taking care of the pagination request, we can render a response template of the format turbo_stream. A simplified version could look like this:

<%= turbo_stream.append 'list-a' do %> <%= render
AbcItem.with_collection(@items, as: :item) %> <% end %> <% if @pagy.next %> <%=
turbo_stream.replace 'load-more-a', method: :morph do %>
<turbo-frame id="load-more-a">
  <%= render LoadMoreButton.new(path: '/', params: { page: @pagy.next }) %>
</turbo-frame>
<% end %> <% else %> <%= turbo_stream.remove 'load-more-a' %> <% end %>

Note that the example is View Components and the pagy pagination library. In the template, we're appending items to "List A" and replace the "Load More" button with a new one. If there aren't any items left the "Load More" button gets removed.

The response from the server would then look roughly like this:

<turbo-stream action="append" target="list-a">
  <template>
    <li><abc-item/></li>
  </template>
</turbo-stream>
<turbo-stream action="remove" target="load-more-a">
</turbo-stream>

It appends one item to the list and removes the button to load more.

Problem

Our <abc-item/> element contains a React component, MyComponent in its markup, rendered via React on Rails' react_component helper. In the final HTML those React components appear as three elements each. Two script tags with context and props and one wrapper element for the rendered HTML of the component:

<script type="application/json" id="js-react-on-rails-context">
  …
</script>
<div id="MyComponent-react-component-eaca5b56-0732-4674-b9da-f858c224d410">
  <!-- MyComponent markup -->
</div>
<script
  type="application/json"
  class="js-react-on-rails-component"
  data-component-name="MyComponent"
  data-trace="true"
  data-dom-id="MyComponent-react-component-eaca5b56-0732-4674-b9da-f858c224d410"
>
  …
</script>

For any <abc-item/> element added to the page via Turbo Streams, those three elements are also added to the DOM, but the wrapping <div> stayed empty: the component wasn't actually mounted! 🙀

<script type="application/json" id="js-react-on-rails-context">
  …
</script>
<div
  id="MyComponent-react-component-eaca5b56-0732-4674-b9da-f858c224d410"
></div>
<script
  type="application/json"
  class="js-react-on-rails-component"
  data-component-name="MyComponent"
  data-trace="true"
  data-dom-id="MyComponent-react-component-eaca5b56-0732-4674-b9da-f858c224d410"
>
  …
</script>

Turbo is injecting markup into the already fully rendered page and none of the DOM events React on Rails is hooking into to execute the JavaScript to mount React components on the page are being fired.

Options

React on Rails already has support for Turbo Streams, but only as part of their paid pro plan. Going "pro" wasn't an option for the project as they are on a tight budget and only using small parts of React on Rails in isolated areas of their app. So we're not really qualifying for "pro" here…

Server-side rendering the React components would also have been an option to tackle the problem. But this again was out of scope for our project and we did not want to run JavaScript on the server.

So what now?

JS to the rescue

With the help of some AI coding agents, we explored the internals of the react-on-rails and @hotwired/turbo-rails NPM package to see what was actually happening. The rough idea was that, if any events were emitted by Turbo when Stream elements were injected and rendered, we could hook into them and then manually call React on Rails to mount all the newly inserted components on the page.

In Turbo, we found the StreamElement. It has a render function and a target pointing to the ID of the DOM-element it's rendered to. Combining these two things, the plan was to extend the render function to also take care of mounting any React elements within the updated target element. This means, when the StreamElement contained any React components, we want to make sure they are correctly mounted after the rendering of the StreamElement itself was done.

/**
 * Gets the current target (an element ID) to which the result will
 * be rendered.
 */
get target() {
  return this.getAttribute("target")
}

So we wrote our own tiny JS module to patch Turbo with some additional logic for our React on Rails integration:

// First we get hold of the original render function of Turbo's StreamElement
const originalRender = Turbo.StreamElement.prototype.render;

// Then we override it to extend it with our own additional logic:
Turbo.StreamElement.prototype.render = function () {
  // First we call the original render function
  originalRender.call(this);

  // Then we get hold of the target element
  const target = document.getElementById(this.getAttribute("target"));

  // Then we wait for the next frame, to make sure that the DOM is fully loaded
  // and we call a function to have React on Rails mount all components within
  // the newly inserted DOM snippet.
  if (target) {
    requestAnimationFrame(() => mountReactOnRailsComponents(target));
  }
};

The custom mountReactOnRailsComponents function is pretty much a manual re-implementation of what's happening when React on Rails is client-side rendering components:

function mountReactOnRailsComponents(domNode) {
  domNode
    // First we get all React on Rails components within our node
    .querySelectorAll("script.js-react-on-rails-component")
    .forEach((script) => {
      // Next we get the name of the component and the DOM ID of the placeholder
      const name = script.dataset.componentName;
      const { domId } = script.dataset;
      const shouldHydrate = script.dataset.hydrate === "true";

      // Then we put together the component props
      let props = {};
      try {
        props = JSON.parse(script.textContent || "{}");
      } catch (error) {
        console.error("ReactOnRails props JSON parse error for", name, error);
        return;
      }

      // Now we need the placeholder element
      const placeholder = document.getElementById(domId);
      if (!placeholder) return;

      // We check whether the component was already mounted
      // in this case we don't want to re-render it.
      const hasMounted = placeholder.dataset.reactOnRailsMounted === "true";
      if (hasMounted) return;

      // Then we call React on Rails' render function to mount the component
      ReactOnRails.render(name, props, domId, shouldHydrate);

      // Finally, we mark the placeholder as mounted, so that we don't accidentally
      // re-render it.
      placeholder.dataset.reactOnRailsMounted = "true";
    });
}

With this in place, our Turbo Streams based pagination works like charm. Any React component newly added to the DOM is mounted as normal 🎉.

Conclusion

Drilling around like this in the internals of Turbo and React on Rails of course yields its risks. Now we're maintaining additional code which may break any time we update our Turbo or React on Rails dependences. But as we're only using the module in a small isolated area of the application, it's a risk we can accept for now. Our test coverage of the feature will inform us early enough.

Using open source technologies, we have the chance to peak under the hood of the tools and can if necessary adapt them to our needs. And working with legacy codebases and legacy Rails apps, their may be situations with unconventional needs…

We hope this post was helpful and inspires you to explore the details of the tools you're using the next time you hit a wall with them. Migrating to an entirely different tool is often a way bigger operation, especially in large monoliths.

Loading Chrome extensions for development in 2025

Max — Wed, 03 Dec 2025 08:25:24 +0000

For loading a development build of our open source browser extension Tickety-Tick into Google Chrome during local development, we used to use Chrome's handy --load-extension flag. It allowed to load a local unpacked build of the extension via the command line. As this feature was recently removed from all branded builds of Chrome, we needed to update our workflow to use the-new-way™ - "remote debugging pipes".

Background

Tickety-Tick is on the nichier side of tech within our fleet of open source projects: it's a browser extension for establishing conventions on naming branches and writing commit messages in Git. And while we're slightly leaning towards team Firefox among us developers, Tickety-Tick is of course also supporting Google Chrome (and others…). For this, we as developers have to be able to run a local build of the extension in Chrome during development.

The old way

For local development, we had a yarn open:chrome command which would open Chrome with the current local build of the extension already installed, ready to click around. Under the hood, we utilized Google's chrome-launcher package to start a fresh, unpersonalized instance of Chrome for us. The code looked about like this:

const chromeFlags = ChromeLauncher.Launcher.defaultFlags()
  .filter((flag) => flag !== "--disable-extensions")
  .concat(["--no-default-browser-check", "--load-extension=/dist/chrome"]);

ChromeLauncher.launch({chromeFlags}).then((chrome) => console.log("Chrome running")

The important part is the --load-extension flag which points to the local Chrome build of the extension.

Some day, this just stopped working. Chrome would start as usual, but without the browser extension loaded.

What happened?

Some debugging and internet research brought us to this post by the Chrome team from March 2025:

RFC: Removing the --load-extension flag in branded Chrome builds

Turns out the --load-extension flag was commonly abused by malicious parties and the Chrome team was looking into ways to further restrict this. It lead them to removing this feature from all branded Chrome builds (so Chromium would still have it…) and instead promote other means for loading extensions for testing and development.

What now?

To figure out the new™ way to load an unpacked, locally build extension in Chrome, we just naively searched for the --load-extension flag on GitHub, hoping to find hints how other people updated their code. A promising hit was in the chrome-launcher package itself. If somebody knew how to do it, they should be the ones. The searching pointed us to a really helpful code comment in their tests:

// Note: --load-extension in chromeFlags used to be the primary method of
// loading extensions, but this is removed from official stable Chrome builds
// starting from Chrome 137. This shows the officially supported way to load
// extensions, with --remote-debugging-pipe.
// See: "Removing the `--load-extension` flag in branded Chrome builds"
// https://groups.google.com/a/chromium.org/g/chromium-extensions/c/aEHdhDZ-V0E/m/UWP4-k32AgAJ

As this was in a load-extension-test.ts file, we figured we'd just try to adopt what they are doing in their test setup for our open:chrome script. And as the comment indicates, we need to utilize the --remote-debugging-pipe flag, something we already knew from ChromicPDF our PDF rendering library for Elixir based on headless Chrome.

Following pretty much exactly what the loadExtension via remote-debugging-pipe test was doing, we ended up with an updated open-in-chrome script which would:

Start Chrome with the respective flags

   const chromeFlags = launcher.Launcher.defaultFlags()
     .filter((flag) => flag !== "--disable-extensions")
     .concat([
       "--remote-debugging-pipe",
       "--enable-unsafe-extension-debugging",
       "--no-first-run",
       "--no-default-browser-check",
     ]);

   const options = {
     chromeFlags,
     ignoreDefaultFlags: true,
     startingUrl: url,
   };

   const chrome = await launcher.launch(options);

Get hold of the debugging pipes

   const pipes = chrome.remoteDebuggingPipes;
   if (!pipes) {
     throw new Error("Chrome did not expose remoteDebuggingPipes");
   }

Put together the command to load the extension

   const requestId = Math.floor(Math.random() * 1e6);
   const request = {
     id: requestId,
     method: "Extensions.loadUnpacked",
     params: { path: dir },
   };

Send the request and listen on the pipes

   const firstResponse = new Promise((resolve, reject) => {
     let buffer = "";

     pipes.incoming.on("error", reject);
     pipes.incoming.on("close", () =>
       reject(new Error("Pipe closed before response")),
     );

     pipes.incoming.on("data", (chunk) => {
       buffer += chunk;
       let end;
       while ((end = buffer.indexOf("\x00")) !== -1) {
         const message = buffer.slice(0, end);
         buffer = buffer.slice(end + 1);
         try {
           const parsed = JSON.parse(message);
           if (parsed.id === requestId) {
             resolve(parsed);
           }
         } catch {
           // ignore non-JSON noise
         }
       }
     });
   });

   pipes.outgoing.write(JSON.stringify(request) + "\x00");

Check that the request was successful

   const response = await firstResponse;
   if (response.error) {
     throw new Error(`Failed to load extension: ${response.error.message}`);
   }

With that a fresh Chrome should be running with our extension loaded! 🎉

Our script

In our final script, we also added some emoji-powered checks and debugging statements, so that the CLI output aligns with our other tasks and the webpack build. Here's our final script, maybe you can draw inspiration from it for your own usecase:

#!/usr/bin/env node

// usage: open-in-chrome [extension-dir] [starting-url]

import * as path from "path";
import * as launcher from "chrome-launcher";

const dir = process.argv[2] || path.join(__dirname, "..", "dist", "chrome");
const url = process.argv[3] || "https://github.com/bitcrowd/tickety-tick";

async function launchChrome() {
  const chromeFlags = launcher.Launcher.defaultFlags()
    .filter((flag) => flag !== "--disable-extensions")
    .concat([
      "--remote-debugging-pipe",
      "--enable-unsafe-extension-debugging",
      "--no-first-run",
      "--no-default-browser-check",
    ]);

  const options = {
    chromeFlags,
    ignoreDefaultFlags: true,
    startingUrl: url,
  };

  const chrome = await launcher.launch(options);

  if (chrome.port !== 0) {
    console.warn(
      "⚠️ Expected remote-debugging-pipe mode on port 0, but got a debug port.",
    );
  }

  const pipes = chrome.remoteDebuggingPipes;
  if (!pipes) {
    throw new Error("Chrome did not expose remoteDebuggingPipes");
  }

  console.log("🚀 Chrome launched with remote-debugging-pipe.");
  console.log(`📂 Loading extension from: ${dir}`);

  const requestId = Math.floor(Math.random() * 1e6);
  const request = {
    id: requestId,
    method: "Extensions.loadUnpacked",
    params: { path: dir },
  };

  // --- Send command and wait for response
  const firstResponse = new Promise((resolve, reject) => {
    let buffer = "";

    pipes.incoming.on("error", reject);
    pipes.incoming.on("close", () =>
      reject(new Error("Pipe closed before response")),
    );

    pipes.incoming.on("data", (chunk) => {
      buffer += chunk;
      let end;
      while ((end = buffer.indexOf("\x00")) !== -1) {
        const message = buffer.slice(0, end);
        buffer = buffer.slice(end + 1);
        try {
          const parsed = JSON.parse(message);
          if (parsed.id === requestId) {
            resolve(parsed);
          }
        } catch {
          // ignore non-JSON noise
        }
      }
    });
  });

  pipes.outgoing.write(JSON.stringify(request) + "\x00");

  const response = await firstResponse;
  if (response.error) {
    throw new Error(`Failed to load extension: ${response.error.message}`);
  }

  console.log(`✅ Extension loaded (id: ${response.result.id})`);
  console.log(`🌐 Opening: ${url}`);

  chrome.process.on("exit", () => {
    console.log("💨 Chrome closed.");
    process.exit(0);
  });
}

launchChrome().catch((err) => {
  console.error("❌ Error:", err);
  process.exit(1);
});

What we learned

Comments in code can make other people's lives significantly easier. Think about it the next time you're hesitating to write one!

And: searching GitHub for something as specific as a Chrome flag, can yield helpful results in any kind of file, even in tests.

A RAG for Elixir in Elixir

Max — Tue, 07 Jan 2025 11:29:07 +0000

Note: This post was written by my teammate joel on the bitcrowd blog

A RAG for Elixir

Recently, we told you about our wonderful RAG for Elixir that you can use to chat with your codebase.
It has one single flaw. We built our RAG for Elixir using Python.
Don't get me wrong, Python is alright and there are reasons why as of today it's the most popular programming language on GitHub.

But there are also many reasons why Elixir is our programming language of choice.
So, wouldn't it be great if we could build our RAG for Elixir in Elixir?

The components

Let's start by revisiting the components we chose to build our original system.

It was based on LangChain.
We contributed a language parser for Elixir to the project to be able to chunk our code into tidy pieces.
To capture the semantic meaning of the code chunks, we used the renowned Jina embeddings model in version 2.
We persisted the resulting embeddings in a locally running chroma vector store.
To have an LLM to talk to, we employed llama3:8b locally with ollama.

Alright, let's have a look what's available in Elixir:

As we can see, all of the components are available in Elixir, although we will run chroma using docker as we've done before.

There is one small change: we will replace Meta's Llama3 with Microsoft's Phi-3.5 mini instruct.
While we can run Llama3 in Elixir, you must sign up on Hugging Face, request access from Meta, wait for approval, and get an authentication token to use it.
We don't want this barrier for anyone who wants to follow this blog post (and run the resulting script that's waiting at the end 😉).
With Phi-3.5 you won't need any of that. The only requirement is a machine with enough RAM.

With all that said, it's time to build a local Retrieval Augmented Generation system in Elixir.

Building a RAG system in Elixir

Our RAG system consists of three parts:

An ingestion pipeline to read the codebase, chunk the code, and store it in Chroma
A retrieval pipeline to accept an input query, find relevant pieces of code and retrieve them from Chroma
A generation pipeline to feed the relevant information and the query into an LLM and generate a helpful response for the user

Servings

We will run two models using Bumblebee. An embedding model that we will configure as RagTime.EmbeddingServing and an LLM that we will configure as RagTime.LLMServing.
We create a module that contains two builder functions to build the servings.
You can add servings to your supervision tree as described in the Nx documentation.

defmodule RagTime.Serving do
  def build_embedding_serving() do
    repo = {:hf, "jinaai/jina-embeddings-v2-base-code"}

    {:ok, model_info} =
      Bumblebee.load_model(repo,
        spec_overrides: [architecture: :base],
        params_filename: "model.safetensors"
      )

    {:ok, tokenizer} = Bumblebee.load_tokenizer(repo)

    Bumblebee.Text.TextEmbedding.text_embedding(model_info, tokenizer,
      compile: [batch_size: 64, sequence_length: 512],
      defn_options: [compiler: EXLA],
      output_attribute: :hidden_state,
      output_pool: :mean_pooling
    )
  end

  def build_llm_serving() do
    repo = {:hf, "microsoft/phi-3.5-mini-instruct"}

    {:ok, model_info} = Bumblebee.load_model(repo)
    {:ok, tokenizer} = Bumblebee.load_tokenizer(repo)
    {:ok, generation_config} = Bumblebee.load_generation_config(repo)

    generation_config = Bumblebee.configure(generation_config, max_new_tokens: 512)

    Bumblebee.Text.generation(model_info, tokenizer, generation_config,
      compile: [batch_size: 1, sequence_length: 6000],
      defn_options: [compiler: EXLA],
      stream: false
    )
  end
end

Ingestion

In our ingestion pipeline, we want to accept a path to an Elixir codebase.
We find all Elixir files in the codebase using Path.wildcard/1 but ignore every file that's inside _build or deps.
Then, we build a map that contains the path of the file at the source key and the content of the file at the content key.
We chunk the code into pieces, calculate embeddings which capture the semantics of the chunk of code, and store the chunk and the embedding in Chroma.

defmodule RagTime.Ingestion do
  def ingest(collection, input_path) do
    files =
      Path.wildcard(input_path <> "/**/*.{ex, exs}")
      |> Enum.filter(fn path ->
        not String.contains?(path, ["/_build/", "/deps/"])
      end)

    files_content = for file <- files, do: File.read!(file)

    documents =
      Enum.zip_with(files, files_content, fn file, content ->
        %{content: content, source: file}
      end)

    chunks = chunk_with_metadata(documents, :elixir)

    embeddings = generate_embeddings(chunks)

    store_embeddings_and_chunks(collection, embeddings, chunks)
  end

  def chunk_with_metadata(documents, format) do
    chunks = Enum.map(documents, &TextChunker.split(&1.content, format: format))
    sources = Enum.map(documents, & &1.source)

    Enum.zip(sources, chunks)
    |> Enum.flat_map(fn {source, source_chunks} ->
      for chunk <- source_chunks do
        %{
          source: source,
          start_byte: chunk.start_byte,
          end_byte: chunk.end_byte,
          text: chunk.text
        }
      end
    end)
  end

  def generate_embeddings(chunks) do
    chunk_text_list = Enum.map(chunks, & &1.text)

    Nx.Serving.batched_run(RagTime.EmbeddingServing, chunk_text_list)
    |> Enum.map(fn %{embedding: embedding} -> Nx.to_list(embedding) end)
  end

  def store_embeddings_and_chunks(collection, embeddings, chunks) do
    documents = Enum.map(chunks, & &1.text)
    ids = Enum.map(chunks, &chunk_to_id(&1))

    Chroma.Collection.add(collection, %{documents: documents, ids: ids, embeddings: embeddings})
  end

  defp chunk_to_id(%{source: path, start_byte: start_byte, end_byte: end_byte}) do
    file_content = File.read!(path)

    start_line =
      file_content
      |> String.byte_slice(0, start_byte)
      |> String.split("\n")
      |> Enum.count()

    end_line =
      file_content
      |> String.byte_slice(0, end_byte)
      |> String.split("\n")
      |> Enum.count()

    "#{path}:#{start_line}-#{end_line}"
  end
end

Retrieval

Alright, now that we've got the code chunks and corresponding embeddings in our Chroma collection, we want to retrieve relevant code chunks for a given query.
So, we'll build our retrieval pipeline next.
For that, we create a new module with a single retrieve/2 function that takes a Chroma collection and a query and returns relevant chunks and their sources.

We compute the embeddings that correspond to the query using the same embedding serving we employed for ingesting.
The embedding is an Nx.Tensor, so we convert it to a list and query Chroma with the list.
We limit the results to the 10 most relevant code chunks and return the code chunks and the sources.

defmodule RagTime.Retrieval do
  def retrieve(collection, query) do
    %{embedding: query_embedding} = Nx.Serving.batched_run(RagTime.EmbeddingServing, query)

    query_embedding = Nx.to_list(query_embedding)

    {:ok, results} =
      Chroma.Collection.query(collection,
        results: 10,
        query_embeddings: [query_embedding]
      )

    [code_chunks] = results["documents"]
    [sources] = results["ids"]

    {code_chunks, sources}
  end
end

Generation

We already arrived at the last piece of our system.
It's time to generate a helpful response.
Again, we'll define a new module.
This time it has a single generate_response/3 function that takes the query, context_documents, and context_sources.
We concat the code chunks (our context documents) and inject them into a prompt together with our query.
We pass the resulting prompt to an LLM and return query, context, context source, and the response.

defmodule RagTime.Generation do
  def generate_response(query, context_documents, context_sources) do
    context =
      Enum.map(context_documents, fn code_chunk ->
        """
        [...]
        #{code_chunk}
        [...]
        """
      end)
      |> Enum.join("\n\n")

    prompt =
      """
      <|system|>
      You are a helpful assistant.</s>
      <|user|>
      Context information is below.
      ---------------------
      #{context}
      ---------------------
      Given the context information and no prior knowledge, answer the query.
      Query: #{query}
      Answer: </s>
      <|assistant|>
      """

    %{results: [result]} = Nx.Serving.batched_run(RagTime.LLMServing, prompt)

    %{
      query: query,
      context: context,
      context_sources: context_sources,
      response: result.text
    }
  end
end

To have a public interface for our RAG system, we define a RagTime module with ingest/2 and query/2 functions.

defmodule RagTime do
  def ingest(collection, path), do: RagTime.Ingestion.ingest(collection, path)

 def query(collection, query) do
   {context, sources} = RagTime.Retrieval.retrieve(collection, query)

   RagTime.Generation.generate_response(query, context, sources)
 end
end

And that's it, a fully local RAG system in Elixir.

Running the RAG system

For demo purposes, we built a simple UI using phoenix_playground, you can download the resulting file from here.

Remember that we need a running Chroma instance.
You can start it with docker run -p 8000:8000 chromadb/chroma.
Then you can run elixir rag_time.exs to start the RAG system.

In our previous blog post, we used the system to get help for a Phoenix PR.
We will ask our Elixir RAG the same question to see if it works.

We clone the Phoenix repository and checkout the commit right before the PR got merged.
Then, we ingest the codebase into our RAG system using the UI.

Afterwards, we ask the exact same question as last time:

We: Please help me understand the following briefing:

Currently, if you set remote_ip during a test when building a conn, calling a dispatch call, e.g get, will run the conn through recycle which will lose this information. Instead, we want the remote_ip maintained, similar to host. Can you see why remote_ip is not preserved in test cases where recycle is used?

Chatbot: The remote_ip is not preserved during the recycle process in test cases because the recycling mechanism is designed to emulate the behavior of browsers, which typically recycle connections by using cookies. When a connection is recycled, the response cookies, including the host header, are saved and sent in subsequent requests. However, the remote_ip is not automatically included in these cookies. The recycling process, as implemented in the provided context, primarily focuses on preserving and transferring response headers like "accept", "accept-language", and "authorization". It does not explicitly handle the remote_ip header. This is why when you set remote_ip during a test and then dispatch a request (e.g., using get), the recycle function will process the connection and potentially discard the remote_ip information. To maintain the remote_ip during tests, you would need to implement a custom solution that explicitly preserves and transfers this header during the recycling process. This could involve modifying the recycling function or creating a wrapper function that handles the remote_ip separately. Alternatively, you could avoid using the recycle function altogether and rely on Phoenix's built-in connection recycling, which might preserve the remote_ip by default. However, this approach might not be suitable for all test scenarios, especially if you need to test specific behaviors related to the remote_ip header. In summary, the reason remote_ip is not preserved during recycling is that the recycling mechanism is not designed to handle this header, and it does not automatically include it in the response cookies. To address this issue, you would need to implement a custom solution or adjust your testing approach accordingly.

As you can see, we get a different response, but remember that we're using a different LLM.
We also get slightly different sources, which might be due to different chunking.

You can play around with the RAG system and swap out the embeddings model or LLM.
If your machine does not have enough memory to run the RAG system, try using smaller models, such as thenlper/gte-small for embeddings and HuggingFaceTB/SmolLM2-135M-Instruct as LLM.
Remember to adapt the prompt accordingly in that case.

In any case, start a discussion with us on GitHub or reach out via bluesky or slack if you need help or just want to let us know what you think.

Tip:
If you haven't done so already, check the rest of our ongoing series on RAG!

Meet Tickety-Tick

Max — Mon, 04 Nov 2024 11:01:57 +0000

Tickety-Tick is a tool so essential and at the same time so natural and obvious for our work at bitcrowd, that we actually never really introduced it much. So here is a very late “meet Tickety-Tick”, a tool we love, take for granted and whose usage is part of the collective muscle memory at bitcrowd since what feels like forever ❤️.

Basics

Tickety-Tick is quite an exotic member of the fleet of open source projects at bitcrowd: itʼs a browser extension to generate Git branch names and commit messages for the ticket one is working on. It allows us to align branch and commit message naming patterns throughout all projects and have a fixed and easy to follow relationship between the ticket - may it be a bug description or a feature - and its corresponding change in the source code, regardless of the ticketing system used and its possibilities or impossibilities to integrate with code repositories.

The idea is based on the “Convention over Configuration” design pattern which used to be super popular in the Ruby community where bitcrowd hung out too in 2015 when the project was born. You agree upon a convention on how to name branches and commit messages and then have a tool in place to take care of enforcing this convention without you ever having to think about it or alternative ways to name branches and commit messages ever again.

If your into history, there is also a slidedeck Bodo, the original author of the browser extension, used for a lightning talk at Ruby Conf Portugal in 2016. I was in the audience back then and immediately sold on the idea 🥰.

Convention

At bitcrowd, branch names follow the format of <type>/<id>-<title> and the subject of the first commit message on that branch would be [#<id>] <title>:

type: the type of ticket

This may be feature, bug, task, chore or whatever else the ticketing system uses.
id: the identifier of the ticket in the ticketing system

This may be TT-5045 in Jira, a numeric ID like on GitHub, a UUID on Notion, etc.
title: the title or subject of the issue

On GitHub, this would for instance be the issue title, something like “Add support for Safari”. When used for the branch name, the title gets lowercased and dasherized to add-support-for-safari.

So for an imaginary feature ticket about adding support for Safari, Tickety-Tick would generate the branch name feature/tt-504-add-support-for-safari and a commit message subject [#TT-504] Add support for Safari, when used with the default configuration.

Origin

The convention was derived from existing workflows at bitcrowd. And while having aligned branch names already makes sense for collaboration alone, the aligned commit message subject is a bit more special. Itʼs useful for us at bitcrowd, because we squash-merge 99.9% of our pull requests. This means each merged pull request is represented by a single commit in the Git history. Respectively, each ticket we work on results in a single commit and the history can be read as something like this:

[#504] Add support for Safari
[#503] Fix installation instructions for Chrome
[#501] Remove cookie banner
…

We also use the generated subject as the title for our pull or merge requests and for time-tracking. But more on that later…

Workflow

Letʼs walk though a possible workflow together.

Setup

First we need to add the Tickety-Tick web extension to our browser. Install it via Firefox Add-ons, the Chrome Web Store or follow the manual installation instructions for Chrome, Opera, Firefox or Safari. Then we need an ticket to work on. Tickety-Tick supports a constantly growing range of ticketing systems from GitHub to YouTrack.

Usage

Letʼs say weʼre working on Tickety-Tick itself. We have the project checked out locally, are on the latest changes of the main branch and have a terminal window open in the projectʼs root directory. Then we browse the open issues on GitHub:

We want want to work on the first issue to migrate the ESLint configuration (itʼs already assigned to me…) and click on the issue to read what needs to be done:

The description is only pointing to a series of blog posts from the ESLint team explaining the new default configuration format. Letʼs assume we read it and are ready to start working on the ticket. So we open the Tickety-Tick browser extension either by clicking on the ticket icon in our browserʼs tool-bar or, if weʼre already among the power users, use the nifty ctrl t keyboard shortcut to open the extension popup:

It offers three buttons: one to copy the generated branch name to the clipboard, one to copy the generated commit message and one to copy the “command”. We click the last button, and paste into our terminal:

git checkout -b 'feature/427-migrate-eslint-config-to-flat-format' && git commit --allow-empty -m '[#427] Migrate ESLint config to flat format

https://github.com/bitcrowd/tickety-tick/issues/427
'

Power users / keyboard lovers can also tab through the buttons of course. No need to reach the popup window with the mouse.

Broken down into pieces, this will:

Check out a new branch with the name feature/427-migrate-eslint-config-to-flat-format
Create an empty commit with this message:

   [#427] Migrate ESLint config to flat format

   https://github.com/bitcrowd/tickety-tick/issues/427

Check our post on commit messages for more info about commit messages, subject vs. body, formatting, etc.

With that, we can start coding away. Whether we decide to later amend the initial commit, add our changes and write a proper commit message body, or if we just keep the empty commit as a starting point for a series of smaller commits afterwards, that depends on the use case and personal preference.

I personally only amend the empty commit with my changes if the changes are super small. Otherwise, I prefer to have a series of commits which follow my thought process and just keep the empty commit as a starting point for my pull request.

Done?

Once done with the implementation, we can open a pull request with our changes. If our changes are small enough to fit into a single commit, GitHub will even pre-fill the pull request title with the commit messageʼs subject [#427] Migrate ESLint config to flat format and the description with the commit message body we provided.

We at bitcrowd reuse the first commit messageʼs subject for the pull request title. It has proven to make browsing the list of open pull requests easier to grasp for our colleagues. So again a convention…

Habit

For pretty much every dev team that we introduced to Tickety-Tick as part of our consulting work, using the tool became muscle memory quite fast. Especially the ctrl t shortcut plus tab quickly wired up as the go-to move whenever people would start working on a ticket or want to check out branches of a currently worked on ticket from another team member. Having the naming convention established and enforced, frees the developers from having to spent any attention or brain power onto them. Pick a ticket, read and understand it and paste a command to check out a branch for the ticket to start implementing it.

When adapting Tickety-Tick as a team with already established rules around branch names and commit messages, it may make sense to look into the advanced configuration of Tickety-Tick. It allows to adjust the naming patterns to the teamʼs needs, so that the tool can be used to enforce already existing conventions. The configuration options are powerful enough to cover any naming pattern your team may have come up with. Yet they are simple and limited enough to not have you spent hours on them.

bitcrowd can help you to optimize your workflows and team organization! Just hit us up and we see what we can do for you!

Why use it?

So if you already have a ticketing system and a place to host your repositories, why should especially a browser extension between them be of so much value for you as a developer and/or your team? Arenʼt you already keeping track of your work and juggling around too many tools?

In that regard, Tickety-Tick is not another tool™ in the garage of tools youʼre already using. Itʼs more like a loose bandage to tie your tools together, so that they donʼt drift apart. As a side-effect, it may also loosely bandage your team members together, so that they donʼt fully drift apart concerning naming conventions…

Enforce conventions

Adapting Tickety-Tick helps to enforce conventions, which frees developers from having to think about these naming questions themselves day to day. This reduces the cognitive load when working with ticketing systems and Git. And this freed head-space is something both, a dev team or an individual developer on their own can benefit from.

Having the naming done automatically via an agreed upon set of rules, also makes the project easier accessible for new people. Plus, it helps to keep things nice and tidy on the long run, without having to constantly have an eye on it.

Traceability

Another huge and probably the most important benefit of using Tickety-Tick is the connection between ticket and code contribution that it helps to establish. Combined with a habit of squash-merging pull requests, the team has a linear and easy to follow Git history: changes in code point to their commit and the commit message points to the ticket on the tracker. Looking at a commit message then allows to quickly dive into further detail about the change on the ticketing system, to find out where a feature originated from, etc. The quality and usefulness of this connection of course also depends on the quality of ticket descriptions, but thatʼs a whole different topic…

Having this fixed connection between ticket and source code and having all people in the team know about it, turned out to be extremely beneficial for pretty much every project or team weʼve seen so far. Itʼs an additional mean for knowledge sharing, makes onboarding newcomers on the codebase faster and is generally a trusted reference for any member of the team, newcomer to long term contributer. Itʼs an additional source of documentation for the code with a level of detail which would not be possible within the code itself.

Further usage

The encoded information about the connection between a ticket on the tracker and the code which later was written for it, can be used for further applications, like timetracking, work reports, etc.

A whole new information heaven can be reached once RAG (retrieval-augmented generation) comes into play. Looking at both, the ticket description and the resulting code contribution, a RAG system can learn a lot about the domain of a project and how the team communicates. Teams who adapted Tickety-Tick early enough in their work, now have a huge individual dataset of how they work right in front of them! This opens up quite some possibilities for knowledge sharing, analysis, automation and what not. We are super excited about the topic ourself and are currently doing a series of blog posts on it!

Try it!

After all the praise, the only thing left to say is: Just try it out yourself! Tickety-Tick works in all major browsers people use nowadays (Firefox, Chrome, Opera, Safari). It supports a lot of different ticketing systems and if yours is not in the list yet, we can help you to put together a pull request for it. Weʼre always curious to find out which ticketing systems and workflows people use.

If you have more questions about browser extensions or workflows, just hit us up!

A RAG for Elixir

Max — Wed, 26 Jun 2024 13:50:49 +0000

Abstract

This is the second part of a series of blog posts on using a RAG (Retrieval Augmented Generation) information system for your codebase. Together we explore how this can empower your development team. Check out the first post for an introduction into the topic if you haven't already.

In this episode we explore how we can adapt our RAG system for Ruby codebases from the first episode to read and understand Elixir code. We will take a look at LangChain and text "splitting" and "chunking".

Let's dive right into it.

Background

Our RAG system was built with the idea to discover Ruby codebases. In order to have conversations about Elixir codebases as well, we need to make sure our LLM "understands" Elixir code. This is where LangChain comes into play.

LangChain is a toolkit around all things LLMs, including RAG. We use it to parse our documents or codebase and generate a vector database from it. In our simple RAG system, we specify which file endings (.rb) and which programming language (Ruby) our documents have.

The ingestion of programming source code into an LLM with LangChain was initially only supported for Python, C and a few others languages. Then this issue proposed the usage of a parser library like Tree-sitter to facilitate adding support for many more languages. The discussion is worth reading.

Finally, this pull request introduced support for a lot more languages, including Ruby, based on this proposal. It was a school project:

I am submitting this for a school project as part of a team of 5. Other team members are @LeilaChr, @maazh10, @Megabear137, @jelalalamy. This PR also has contributions from community members @Harrolee and @Mario928.

Our plan is to use this as a starting point to enable basic parsing of Elixir source code with LangChain. With that, we should be able to have conversations with our RAG system about Elixir codebases as well.

Splitting / Chunking text

To read our Elixir codebase, the parser needs some rules on where to split the provided source code files at. Generally for RAG, when ingesting (reading in) a text file, PDF, etc. it will try to split it into chunks, ideally along it's semantic meaning. In text documents, the meanings are often grouped along:

chapters
paragraphs
sentences
words

If your embedding model has enough context capacity, you would try to split along chapters or paragraphs, because human readable text often groups meanings that way. If those are too big, you would try to break between sentences, and, as a last resort, words. One would generally try to avoid splitting inside words. Take for instance "sense" and "nonsense", which carry quite a different meaning.

Splitting / Chunking code

Embedding code is a bit underdeveloped, but the strategy is to break the code into pieces by inserting new lines so that it looks a bit more like natural text, and then let the embedding model deal with the task of making sense (inferring meaning) from it. Interestingly, the models trained on that task do that surprisingly well.

As said, LangChain has dedicated document loaders for source code and a guide on how to add new ones based on Tree-sitter. So we went ahead and implemented a document loader and parser for Elixir source code in LangChain. It only covers the core basics of the language, but it was already enough for our proof-of-concept RAG application. With LangChain now supporting Elixir out of the box, people can use the parser in a variety of different scenarios and will come up with ways to improve it to fit more use cases. Our implementation is only the ground work. You can have a look at the PR if you're interested in what's necessary to add parsing support for another programming language in LangChain. Spoiler: not much if you can utilize Tree-sitter.

The core of LangChain's programming language parsers based on Tree-sitter is their CHUNK_QUERY. For our Elixir parser it looks like this:

CHUNK_QUERY = """
    [
        (call target: ((identifier) @_identifier
            (#any-of? @_identifier "defmodule" "defprotocol" "defimpl"))) @module
        (call target: ((identifier) @_identifier
            (#any-of? @_identifier "def" "defmacro" "defmacrop" "defp"))) @function
        (unary_operator operator: "@" operand: (call target: ((identifier) @_identifier
              (#any-of? @_identifier "moduledoc" "typedoc""doc")))) @comment
    ]
""".strip()

We are using Tree-sitter's own tree query language here. Without diving into the details, our query makes sure to distinguish top level modules, functions and comments. The document loader will then take care of loading each chunk into a separate document and split the lines accordingly. The approach is the same for all programming languages.

Test drive

Let's take this for a spin in our RAG system scripts from episode one of this series.

Just as a refresher: the idea is to have a RAG system for your team's codebase using LLMs locally without exchanging any data with third parties like OpenAI and the like. It includes a conversational AI built with Chainlit, so that members of the team can "chat" with the LLM about the codebase, for instance to get information about the domain or where to find things for the ticket they are working on.

For testing purposes we will use our RAG system on a popular open source Elixir package, the Phoenix Framework.

Get the RAG ready

First we need to get our local RAG system ready for operating on an Elixir codebase. It needs to know:

Where is the code?
Which programming language is it?
Which suffixes have the source code files?

We provide this information via environment variables in a .env file:

OLLAMA_MODEL="llama3:8b"
CODEBASE_PATH="./phoenix"
CODEBASE_LANGUAGE="elixir"
CODE_SUFFIXES=".ex, .exs"

We just cloned the current state of the Phoenix Git repository right next to our RAG code. We also keep using Meta's Llama3 model, and instruct the document loader to look at Elixir files.

ℹ️ Info

At the time of testing our PR on LangChain was not released yet. So we were pointing to our fork's local code for the langchain, langchain-community and langchain-text-splitter Python packages from the requirements.txt file.

For the sake of simplicity we assume the project documentation to be in Markdown and hard-coded this information into our code ingestion logic.

With that, we can set up our vector database for the Phoenix codebase:

python ingest-code.py

Then we start the chat bot:

chainlit run main.py

Chatbot:

Hi, Welcome to Granny RAG. Ask me anything about your code!

Now we are ready to have a conversation about the codebase.

Ask questions

Let’s take an original PR from Phoenix to evaluate the quality of our solution, similar to what we did with Ruby in the previous post.

The pull request fixes a small issue in the recycle/1 function of the Phoenix.ConnTest module at phoenix/lib/phoenix/test/conn_test.ex:

Currently, if you set remote_ip during a test when building a conn, calling a dispatch call e.g get will run the conn through recycle which will lose this information. This PR changes that so it is maintained similar to host.

We reset our clone of the Phoenix repository (and our vector database) to the state right before the PR was merged and then ask the RAG system for help with the issue:

Understand the problem

We:

Please help me understand the following briefing:
Currently, if you set `remote_ip` during a test when building a `conn`,
calling > a dispatch call, e.g `get`, will run the `conn` through `recycle`
which will > lose this information. Instead, we want `the remote_ip`
maintained, similar to `host`.
Can you see why remote_ip is not preserved in test cases where recycle is used?

Chatbot:

The briefing explains that when running a test with Phoenix's Conn and Endpoint frameworks, the remote_ip information is lost during the test. This happens because of the way Phoenix's Conn.recycle/1 function works.
Here's what's happening:

In your tests, you create a new conn with a specific IP address (e.g., remote_ip = "192.168.0.100").

You then dispatch an HTTP request to this connection using something like get("/").

When you do this, Phoenix's Conn framework calls the Conn.recycle/1 function behind the scenes.

This recycle function is designed to clean up and reset the connection for subsequent requests.

However, as part of this cleanup process, it loses the previously set remote_ip information.

The reason why host is preserved but not remote_ip is that Phoenix's Conn framework has a special handling for the host attribute. When you set the host attribute, it gets stored in the connection's internal state. This means that when you recycle the connection, the host information is still retained.

On the other hand, remote_ip is not preserved because it's not explicitly stored in the connection's internal state. Instead, it's just a temporary value set during the test execution.
To maintain the remote_ip information across tests, you can create a custom Conn module that overrides the default behavior of Conn.recycle/1. This would involve storing the original remote_ip value and restoring it after recycling the connection.

In summary, remote_ip is lost in test cases because Phoenix's Conn.recycle/1 function doesn't explicitly preserve this information. To keep remote_ip intact, you'll need to create a custom Conn module that overrides the default behavior of recycle.

Sources:

/Users/joshuabauer/code/rag_time/phoenix/test/phoenix/test/conn_test.exs,

[…]

Implement a solution

This would give us a hint on how to solve the issue in our app if we were "using" the Phoenix framework. But let's say we want to fix the issue "for good" in the framework itself. We now know the Conn.recycle/1 function is what we're interested in. So we look at the listed sources, search for the recycle/1 function in the code and find ConnTest/recycle/1 as the test-conn equivalent to Conn.recycle/1. This is pretty much already be the "custom Conn module" the RAG was referring to and it's just used for tests. And after all, we only want to change the behavior for tests. So let's ask the RAG what to do with it:

We:

Given the following code from phoenix/lib/phoenix/test/conn_test.ex, what would I need to change to persist conn.remote_ip in the same way as conn.host?
def recycle(conn, headers \\ ~w(accept accept-language authorization)) do
  build_conn()
  |> Map.put(:host, conn.host)
  |> Plug.Test.recycle_cookies(conn)
  |> Plug.Test.put_peer_data(Plug.Conn.get_peer_data(conn))
  |> copy_headers(conn.req_headers, headers)
end
Chatbot:

A Phoenix ConnTest question!

To persist conn.remote_ip in the same way as conn.host, you would need to add a line to the recycle/2 function to copy the remote IP address.

Here's how:
def recycle(conn, headers \\ ~w(accept accept-language authorization)) do
  build_conn()
  |> Map.put(:host, conn.host)
  |> Map.put(:remote_ip, conn.remote_ip)  # Add this line
  |> Plug.Test.recycle_cookies(conn)
  |> Plug.Test.put_peer_data(Plug.Conn.get_peer_data(conn))
  |> copy_headers(conn.req_headers, headers)
end
Now, when you recycle a connection, both conn.host and conn.remote_ip will be preserved.

Sources:

[…]

Compare solution

Looking at the PR's file changes, this is exactly what the person came up with:

So the conversation with our Elixir RAG was quite helpful for guiding us through the code and finding an adequate solution to the problem.

Try it yourself!

It is really easy! Just clone our repo, follow the README and tell the script where to find your codebase:

CODEBASE_PATH="./path-to-my-elixir-codebase"
CODEBASE_LANGUAGE="elixir"
CODE_SUFFIXES=".ex, .exs"

We kept the scripts basic, so that they are easy to understand and extend. Depending on your codebase, the results might not always be perfect, but often surprisingly good.

Outlook

In this post we saw how we can extend a simple off-the-shelf system to better fit the needs of our dev team. We enabled our RAG system to read and understand Elixir code! Text splitting and chunking is just one possible example of where to start when it comes to adjusting a RAG system for your specific needs. What we got is already quite useful, but it's definitely still lacking precision.

We will explore possibilities for further improvements and fine tuning in the next episodes of this blog post series.

Or, if you canʼt wait, give the team at bitcrowd a shout via granny-rag@bitcrowd.net or book a consulting call here.

How even the simplest RAG can empower your team

Max — Wed, 26 Jun 2024 13:34:01 +0000

Abstract

This is the first part of a series of blog posts on using a RAG (Retrieval Augmented Generation) information system for coding. Find out how this can empower your development team.

In this episode, we will discuss at a very simple RAG system for Ruby made with LangChain, JinaAI embeddings and a very light local LLM served via Ollama.

If you donʼt know what any of this means, this blog post is for you. You will see how to benefit from the technology without uploading/donating your codebase to external providers like OpenAI.

It is an introduction, the coding follows in the next episodes.

Naming is hard, and I wanted something that goes against the flashy-ai-y. So letʼs call this a Granny RAG.

The next parts will then move on to the following topics:

examine different embedding models
splitting and chunking strategies
RAG for and with Elixir

Letʼs start.

A Use Case

Imagine you are developer and you just started a new job. The code base looks well maintained, the team is nice. Could be bitcrowd. You grab your first ticket, and it looks like this:

Uff. No description. Normally, you would reach out to your onboarding-buddy and ask for advise. This ticket is clearly written with a developers who knows the application in mind.

But ... you could also quickly investigate. Your team does have a Granny after all!

You go to your local granny. Granny RAG is in fact just some light off-the-shelf application wrapper around a simple RAG system, but you are used to call her Granny RAG, because she is patient, kind and knows all the stories.

Letʼs ask her:

Hy Granny, I need to show the end date of the subscriptions in the “subscription list”. What is a subscription in the application?

Granny: To display the end dates of subscriptions in the list of subscriptions, you can modify the COLLECTION_ATTRIBUTES constant in your SubscriptionDashboard class to include the active_until field. Hereʼs an updated version of the SubscriptionDashboard class with the modified COLLECTION_ATTRIBUTES:
class SubscriptionDashboard < ApplicationDashboard
 ATTRIBUTE_TYPES = {
   course: Field::BelongsTo.with_options(order: :name),
   user: Field::BelongsTo.with_options(order: :name),
   user_id: Field::Number,
   id: Field::String,
   active_until: Field::DateTime,
   deactivated_at: Field::DateTime,
   active?: Field::Boolean,
   product_type: EnumField,
   created_at: Field::DateTime,
   updated_at: Field::DateTime
 }.freeze

 COLLECTION_ATTRIBUTES = %i[
   course
   user
   id
   active?
   product_type
   active_until
 ].freeze

Wow, this is magic!

Without knowing the application too well, you came from a non-workable ticket to a plan of action. You can now chose to update the ticket description, or create a PR and use that for the discussion.

ℹ️ Note

This is a real life ticket from a client.

This ticket came from the work at our client MedGurus. When I tried out my Granny RAG, I browsed through the tickets in progress and found this one. I compared the solution Granny RAG proposed with the PR that came out of this ticket. I was actually thrilled how well this works.

My conversation with Granny RAG was actually a a bit longer. Here is the full conversation with Granny RAG.

How does it all work?

Granny RAG is a RAG system. That is short for Retrieval Augmented Generation. If you are looking for a quick intro, here is the nice video by Marina Danilevsky.

In essence, RAG improves the quality of LLM responses by enriching user prompts with relevant contextual information. It retrieves this information from an efficiently searchable index of of your entire project, generated with the help of an embedding model.

Embedding models

Itʼs not easy to say something simple about the embedding process without being incorrect. Embedding models are models that generate a representation of the “meaning” sequence of text. This “meaning” is represented as a vector called “embedding”. It is a long array of numbers that represent semantic meaning within the given context.

Tokens with a similar meaning in the source document get embedding vectors “close to each other” by some distance measurement.

A suitable model will place expressions with similar meaning in similar spaces of its vector space. So subscription will be next to activation and active_until.

You can think of the process as hashing with hashing function that understands the input.

Retrieval

Instead, when the user asks a question, we throw it into the same embedding function to get an index for it. With that, we do a lookup what sequences of text occupy a similar space in the memory.

There are multiple strategies for this similarity criteria. We will explore similarity in more depth in the second post of this series. For now, letʼs assume we found entries “close” to the index we got for the search term.

Each of those entries carries a piece of text and some metadata. The metadata tells us more about the source, e.g. which file it came from. Until now, we have build a more intelligent search function. It finds active_until even if you searched for end date. Something, a classic fulltext index would not find.

In an “old fashioned” information system, we would output those magical pieces of text and leave it to the reader to go through them, understand their meaning and evaluate their relevance.

“But wait”, you say, “are there not these new cool kids on the block, The LLMʼs™, that are brilliant at exactly that?”. You are right, this is exactly what RAG systems do.

Context

Attention: We will be simplifying heavily. If you would like to get a l ightweight intro head over to this huggingface course, or this series of videos from three blue one brown.

It boils down to this: When LLMs generate, they find the next word, or gaps in a text. They take this a step at a time, a bit like friends finishing each otherʼs sentences.

Then, they look at the text created, including the new word, and compile the next word, and the next. Put differently, they try to find the piece of text or the character that is most likely to make sense in the previously generated context.

Here is an example for a prompt that uses RAG:

You are an assistant for question-answering tasks. Use the following pieces of
retrieved context to answer the question. If you donʼt know the answer, just
say that you donʼt know.
Use three sentences maximum and keep the answer concise. # (1)
--
Question: “What would I need to change to show the active_until date in the list
of subscriptions?” # (2)

Context: {context} # <- The RAG magic happens here

Answer: # (3)

ℹ️ Info

A system prompt tells the LLM what is expected from it (1), then a question is specifying the task (2) and the “please fill in your answer here”-part (3) is what LLMs are used to work with.

LLMs do so, again, based on vector representations. Starting from a seed, often the system prompt, and the userʼs instructions.

The idea of RAG is that if you include facts that you researched into your prompt, the context for the generation is narrowed down significantly compared to a prompt that does not include those facts. Retrieval Augmented Generation is an effective countermeasure against hallucinations. It does not stop them, but makes them less likely.

💡 Tip

Outline the Important Findings in your follow up

All LLM based systems hallucinate at some point. RAG helps to avoid that, but as you can see in 5. Follow Up, even Retrieval Based Systems stray from the truth at times. You can detect that because the information in 5. Follow Up does not align with the previous answers.

If this happens, it helps to outline the previous facts in the next prompt, as I did in 6. Follow Up:

...

“You said before that the COLLECTION_ATTRIBUTES are responsible for the list of Subscriptions (...)”

...

Mentioning previous findings in the new prompt amplifies them in the context. This steers the conversation in the direction you like and helps helps the LLM to let go of the hallucinations.

The Code

You can find the code on bitcrowdʼs GitHub, which is probably easier to read than going through the inline version here.

We will use LangChain, which is a Python library. There are also implementations of it in JavaScript and Elixir. We use Python to start with, because it is the common language for machine learning, so prototyping is easy. Once we know what we want, we will explore the Elixir way. So please make sure you have Python installed.

We use JinaAI embeddings, which is a very capable model that can be used locally under the Apache 2 licence.

Finally, you need Ollama, or any other tool that letʼs you run a model and expose it to a web endpoint. In the example, we use Meta's Llama3 model. Models like CodeLlama:7b-instruct also work. Feel free to change the .env file and experiment with different models.

Test Drive

You need some Ruby library that we can try out here. Letʼs go for Discourse.

Weʼre interested in the logic Discourse uses prevent users from re-using an old, expired password and which files in the code are relevant for this. So we ask it:

How does Discourse keep track of which passwords a user already used? So that when changing their password, a user does not use the same, expired password again.

Granny RAG answers:

According to the context you provided, Discourse keeps track of which passwords a user already used by storing the old password hash and algorithm in the user_passwords table. This is done through the following mechanisms:

The user_passwords table stores the userʼs previous passwords along with their corresponding salts and algorithms.

When a user changes their password, Discourse generates a new password hash using the new password, salt, and algorithm.

Discourse then checks if the new password hash is already present in the user_passwords table for that user. If it is, it means the user has used this password before, and therefore, itʼs considered expired.

By storing the old password hashes and algorithms in the database, Discourse can easily verify whether a user has used a particular password before and prevent them from using an expired password again.

This approach ensures that users cannot reuse the same password twice, even if they change their password multiple times.
…

Without knowing all the details of Discourseʼ codebase, this seems a perfectly valid answer. It gives us a nice summary of the feature an idea on where to go looking for the implementation in Discourse otherwise overwhelmingly big codebase.

Wrap up

For our InfoServer Granny RAG we created a sophisticated search index for our codebase. This index allows us to infer meaning. We can now search for an answer by infering the meaning of the question and retrieve relavant information by looking what is close in the vector space.

We fed this information to an LLM to understand and prioritize the sources. We made the LLM create answers to our questions, based on its training data and the information retrieved.

So with a few, off-the-shelf, open source components, we manage to get expert knowledge about our code base, and get it delivered to us by our friendly LLM companion, hosted on our own premises.

How is this different from Copilot, Duo and friends?

If you ask Granny RAG a question, it can draw information from the whole codebase. It is able to incorporate all kinds of documents, and can be extended to incorporate additional data-sources. Granny RAG operates on resource effective, local LLMs.

No data needs to leaves your control.

The scripts that ingest and embed your data and code can be specific to your needs - as is your codebase. That way, you can even specify what should, and what should not, find its way into your RAG knowledge base.

Copilot and GitLab Duo have a much narrower angle of vision. Their context is primarily the opened files of the editor, or the PR. That means, once you know where to look, they can be helpful. Both to you and their creators, which can (and probably will) use some data to improve their models. Even if, per contract, your data and code should not be shared with GitLab or Microsoft, you lost all control once your data leaves the premises.

If you set theses concerns aside, you still have little control about what makes its way into the LLMs that are hosted on remote servers.

Here again, Granny RAG is different. You can collect data from usage and reactions, and you can use that data to train both, LLM and embedding model, on your data and needs.

That way, new arrivals in your dev team get an assistant that is steadily improving. Granny RAG can integrate into a Slack channel to provide a first opinion, and take feedback from the more seasoned developers to improve.

All in all, Granny RAG is a concept that can (and should) be adopted to your use-case and needs. Itʼs not a subscription you buy, but a technique your team learns to master. You invest in consulting or learning time, and you get control and excellent knowledge about the core or your business logic.

Try it yourself!

It is really easy! Just clone our repo, follow the README and tell the script where to find your codebase:

CODEBASE_PATH="./path-to-my-codebase"
CODEBASE_LANGUAGE="ruby"

We kept the scripts basic, so that they are easy to understand and extend. Depending on your codebase, the results might not always be perfect, but often surprisingly good.

Outlook

In this introductory post, we saw what a little off-the-shelf system can achieve. Itʼs already impressive, and it only uses local models, namely Llama3 and JinaAI Code.

You will find that this off-the-shelf solution is lacking precision in some use cases. To improve this, we will explore how changes in the parsing, chunking and embedding strategies will change performance in the next episodes of this blog post series.

Or, if you canʼt wait, give the team at bitcrowd a shout via granny-rag@bitcrowd.net or book a consulting call here.

Git, GitHub and your work email

Max — Mon, 09 Aug 2021 16:35:55 +0000

A possible setup for separating private and work-related contributions on GitHub: associate commits with your "private" vs. your "work" email.

GitHub being such a convenient place to collaborate on software, it's likely that both, work and "leassure coding" take place on the same platform. Do you need two separate accounts for both activities? Eventually yes… But to spare the hassle of switching accounts back and forth and remembering to actually do that, a lot of people - including myself - prefer to use a single account (as long as this is legally aligned with your work place). However, aside from keeping that yin-yang of a work-life-balance, fully blending in your work-related coding into your private ramblings on GitHub and vice versa feels wrong from multiple angles, mainly the legal one. In which role did you contribute this piece of code, who "owns" this contribution now? You, your employee, a client?

So a bit of separation can come in handy. But how to get there without juggling accounts? Say hi to your old friend from the 80s: email 📧

Different to what one would expect on first glance, GitHub does not directly associate contributions with accounts. It's rather just "connecting" the contribution to the account that happens to own the email address associated with a Git commit ^¹. Listen to the GitHub Ate My Résumé episode of The Bike Shed podcast for some wild situations this can bring…

It's probably save to assume most people have a dedicated email address for work. Using your private email address for work-related communication uncomfortably stresses the yin-yang and should be avoided. And since we're already there, why not apply the same email hygiene to your commits? Use your work email address for work contributions and your riotcat1991@smthng address for your private coding adventures.

Here are some tips how I set this up for myself. Disclaimer: people are different. While this works fine for me, other solutions may eventually work better for you - find what feels comfy 🧘.

Git Config ⚙️

The setup mainly lives in the gitconfig files in my home directory:

~/
├── .gitconfig
├── .gitconfig_personal
└── .gitconfig_work

Gitconfig

In .gitconfig, set a name via user.name and deliberately not configure any email address in user.email. I combine this with setting user.useConfigOnly to true:

Instruct Git to avoid trying to guess defaults for user.email and user.name, and instead retrieve the values only from the configuration

Recent versions of Git don't allow to commit without having a name and email address configured for the repository. The email address is an integral part of a commit. By not configuring one globally and instructing Git to not magically figure one out itself, I can avoid accidentally committing with the "wrong" email address.

With this configuration, I now have to manually set user.email for every repository. For convenience, I have two alias set up for this:

[alias]
  personal = "config user.email 'me@private.internet'"
  work = "config user.email 'me@work.internet'"

Inside a Git repository, running git work configures the email to be used for commits in that repository to my work address.

Directories

In most cases, it's obvious which email address I want to use. On disk, my projects are organized in a folder structure like this:

~/code/
├── work/
│   ├── client_a
│   │   ├── projetc_a
│   │   └── projetc_b
│   ├── internal
│   │   ├── project_c
│   │   └── project_d
│   └── client_b
└── personal/
    ├── project_a
    └── project_a

In my .gitconfig file I include private or work-related additional configuration depending on the directory path of a repository:

[includeIf "gitdir:code/personal/"]
  path = ~/.gitconfig_personal
[includeIf "gitdir:code/work/"]
  path = ~/.gitconfig_work

These personal and work sub-configurations only configure user.email. For example:

# .gitconfig_personal
[user]
  email = me@work.internet

This spares manually running git work or git personal in every repo. Cloning a new project at work into a subdirectory of ~/code/work, the configuration already tells Git to use my work email.

GitHub 🐙

Your work email needs to be added to your GitHub account: https://github.com/settings/emails

GitHub UI

Merging pull requests in the GitHub UI or accepting change suggestions also creates a commit. Similar to Git itself, GitHub doesn't associate commits with your account, but associates the account via the email address the commit is made with 🤯. For UI interactions, this is a bit tricky to get right.

On the first UI interaction in a repository, GitHub shows a dropdown to select the email address to use for the resulting commit. It then stores this email address for that repository and continues to use it for all UI interactions without showing a dropdown to change it ever again. This means, after having used the wrong email once, it is a bit hard to get out of this again.

I already reached out to GitHub support for this problem, but they could not help me. For now, the only "hack" I found to switch the email again is to:

Go to the respective repository
Click on the 🖊 icon to edit a file (e.g. README.md) in the web editor
You can choose your email again from a dropdown when commiting
GitHub will continue to use this email from now on for this repo

Disclaimer: this email juggling in the UI is what worked for me back then, it may already be different now.

References

1: https://docs.github.com/en/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address#about-commit-email-addresses

Script up your projects

Max — Tue, 02 Feb 2021 18:08:51 +0000

At bitcrowd, we love conventions. Being an agency, we jump projects, codebases, frameworks and languages quite a lot - to a point where it's impossible to keep all the details about every project one's head. It's not necessary to actually have this knowledge in practise though. Some people knowing the details of some projects is enough. They can onboard others who then slowly forget the details of their previous projects… A well-established, semi-stable cycle of knowledge transfer with some silos here and some information lost there - but overall working.

Still, there is this nice and fluffy feeling of coming to a new project, seamlessly setting it up and immediately finding your way around. Taking developers by their hands upon entry, these hygge codebases appear nice, friendly and motivating. And developer happiness aside, even on the dark and cold business side of things, people coming to these "welcoming" codebases are probably likely to be more productive with the head start they get there.

How can we get to this shiny place?

Conventions ⚖️

Notorious self-optimization and reaching for ultimate "productivity" are neoliberal derailments. But decluttering daily tasks from unnecessary cognitive overhead, can actually have a calming, decelerating effect on our personal work life: it gives us a cleaner, virtual "desk", less stress and easier time to set details aside and focus on the important things.

In software development, conventions can be a useful tool to reduce the cognitive effort involved when switching between projects. Backed by a known outline of what to expect where, developers can, for the moment, put aside the nitty-gritty details of which version X of framework Y a project uses and instead focus on things like domain or datamodel to easier wrap their head around the new problem space. Utilizing agreed upon patterns, we can take load off peoples' shoulders for the ultimate "hygge" developer experience.

Situation 🚣

In agency work, with its fast and frequent project switches, the "onboarding" and "getting started" phase on a codebase are especially crucial. But product teams shouldn't overwhelm new members with days of fiddling with different package managers either.

First Contact 🐣

Coming to a new project. What are the first steps you always take?

You most likely have a look at the README first (hopefully it's a good one) 🔦
You search the README for installation and setup instructions (the classic "getting started") 🔎
You spend a few ~minutes~ hours copy-pasting things from the README into your shell to get to a state where you can finally run the project 🚜
You nag your colleagues for the steps which are missing, outdated or simply not documented 🕵️

All in all you pretty much spend around half a day setting up the project and already lost the joy of contributing to it. Time, effort and motivation you could have spent on getting to know the domain better and that will be missing in later phases of the project.

A sad situation for a group of people aiming to utilize computer to "solve problems". Projects with a bad setup and onboarding experience are likely to distract and discourage people. Coming out of a rough setup safari, one may feel insecure and as if one knew "nothing" about the project.

Day-in day-out 🌚🌝

Once actually working on a project, one has to run certain development tasks now and then: database migrations, the test suite, managing translations, etc. One may want utilize all possible brain capabilities on memorizing those for each possible framework or alternatively just search up and down the shell history for the one command one at some point managed to successfully copy-and-paste. But what if you don't even know what to search for?

A classic example: You come back to the project after a few days on a different one. You fetch the latest changes on the main branch and then… Run the migrations? Update your JavaScript version? Update packages and then run migrations? Is it npm install or yarn install? A steam of questions and decisions to take before you can even start your editor or run the first test. And there is also the extra cognitive overload of context switches within the project: switching between feature branches, pairing with a colleague on their ticket, etc.

Agreeing on conventions, on a common way to do and approach things across projects may reduce this overload and give people an easier time thought the day 🐖.

Scripts to the Rescue 🚑

People at Github made an attempt to fix this situation: scripts to rule them all. The idea is to have common set of executable scripts for common developer tasks in a script/ directory in the root of every project:

A consistent bootstrapping experience across all our projects reduces friction and encourages contribution.¹

While every project may use different tools or languages, the script directory consistently following the same pattern everywhere gives developers something to hold on to. Scripts as an anchor in the sea of cognitive overload after checking out a codebase:

script/bootstrap to install/update dependencies
script/setup to set up a project for the first time
script/update to update a project to run at its current version
script/test to run tests
script/console to opens a console
…

Technical setup instructions can of course live as 100+ copy-pasteable steps in the README. Putting them into a script has roughly the same effort but a huge benefit: it can do the work for future developers 🤖. Scripts can work well as a runnable documentation for a project's tooling setup. Imagine switching between Python and JavaScript projects. How to run migrations here, what are the default CLI flags there? Instead of infinitely searching the shell history, simple scripts in the project can wrap and document common developer tasks.

The selection of files in your script directory of course depends on every team's individual situation and workflow. Just go with Marie Kondo, get rid of scripts which don't "spark joy" and add others where you feel pain.

Recommendations 🛒

We adapted this pattern for our own workflows at bitcrowd. Here are some loose recommendations on what we find useful:

script/test-e2e

On a lot of projects, high level end-to-end tests are slower. So it's nice to be able to run them with a separate command. With that, script/test can focus on running unit tests only, finish faster and provide important feedback earlier.

script/lint

We are obsessed with linters. Running them as part of the test suite does not fit our way of working. For us it makes sense to run tests and linters separately as they hint at different problems in your code. Maybe one wants to not care about linting until one has a working implementation or tests and prefers to do linting and code cosmetics later?

script/format

Some ecosystems support formatters. If machines can do the formatting for us, why not accept their help?

Your own 🎨

Add your own scripts, like script/deploy, script/psql, etc., there are no limits. Come up with whatever suits your organization and team. The only thing: stay consistent and have conventions. That's where the real benefits lie.

For instance script/manage to facilitate the manage.py utility in Django projects:

#!/usr/bin/env bash

# Run manage.py tasks
# Usage: scripts/manage [...args]

set -o errexit
set -o pipefail
set -o nounset

SCRIPTDIR=$(cd "$(dirname "$0")"; pwd)

cd "$SCRIPTDIR/.."

exec poetry run python manage.py "$@"

Or on a project which requires a specific version of a PostGIS database, using a utility script to to conveniently run the database in a Docker container. Developers may run script/db start to start the database and script/db stop to stop it:

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail

CONTAINER_NAME=project_x_db

find_container() {
  docker ps \
    --all \
    --quiet \
    --filter name="$CONTAINER_NAME"
}

start_db() {
  if [ "$(find_container)" = "" ]; then
    exec docker run \
      --interactive \
      --tty \
      --name "$CONTAINER_NAME" \
      --env LC_ALL=C.UTF-8 \
      --env POSTGRES_USER="$POSTGRES_USER" \
      --env POSTGRES_PASSWORD="$POSTGRES_PASSWORD" \
      --env POSTGRES_DB="$POSTGRES_DB" \
      --publish 5432:5432 \
      postgis/postgis:9.6-2.5-alpine \
      postgres
  else
    exec docker start \
      --interactive \
      "$CONTAINER_NAME"
  fi
}

stop_db() {
  exec docker stop "$CONTAINER_NAME"
}

print_usage() {
  echo "Usage: db [<option>]"
  echo ""
  echo "Options:"
  echo "  start   Start database."
  echo "  stop    Stop database."
  echo "  --help  Show this message and exit."
}

if [ $# -lt 1 ]; then
  print_usage
  exit 1
fi

case $1 in
  start)
    start_db
    ;;
  stop)
    stop_db
    ;;
  *)
    print_usage
    exit
    ;;
esac

How to do scripts? 🚌

Who would ever blindly run a script on their machine? Better take a look at the contents first. If written, formatted and documented with care, scripts work as a great way to consistently document development workflows. And since we never get things right the first time: continue to update your scripts (e.g. when new people are onboarded and run into issues). Integrating your scripts into your daily workflows and running them regularly also helps to iron out rough edges. We for instance usually run script/test, script/test-e2e and script/lint as part of our CI pipeline.

When it comes to writing scripts, pick whatever suits your usecase and what you feel comfortable with. For the sake of compatibility, Bash can be a good choice. But be aware, it can be tricky at times… ⚠️:

The weird thing about shell scripts is that even strong advocates of good practices gladly forget all they know when it comes to shell scripting.²

Linting your shell scripts with Shellcheck can improve the situation a lot though. It is also a great resource and opportunity to learn about best practices.

Here are some resources we found useful for leveling up our scripting:

Conclusion 🍼

Scripting is not a silver bullet, but it can make developer lives a bit easier. And it's a great rabbit hole to get lost in 🕳

References

Prepending Modules to the Rescue

Max — Thu, 26 Nov 2020 14:26:25 +0000

Ruby's ability to "overlay" default implementations of constants, methods and variables via the prepend or prepend_features method on Module can be a helpful tool when dealing with gems overriding setter methods for attributes.

Scenario 🧫

On a Ruby on Rails project, we were recently migrating from the attr_encrypted to the lockbox gem for encrypting database fields. For live applications which cannot accept hours of downtime, the migration path is a multi-step process which includes a period where both gems are used side by side until all data has been migrated from the old columns to the new columns.

Both gems integrate into Rails models via their own macro-style methods one is supposed to add to the models' class definitions:

class User < ApplicationRecord
  attr_encrypted :email, key: key
  attr_encrypted :phone, key: key

  encrypts :email, :phone, migrating: true
end

Under the hood, both gems then dynamically generate the respective getter and setter methods for the attributes (email, email=, phone and phone= in this example).

In addition to that, we were also overriding the setter methods email= and phone= ourselves to do some normalization on the provided values before assigning them. Combining this with the generated setters from attr_encrypted introduces a lot of fuzz: in what order are the implementations called - if at all - and what does super mean in which context? In order to eliminate all this confusion from the start, we previously decided to make our own implementation the "source of truth" and instead of relying on super calls, just integrate the respective parts of attr_encrypted's implementation into our own:

def phone=(value)
  normalized_number = Normalizers::PhoneNumber.normalize(value)
  self.encrypted_phone = encrypt(:phone, normalized_number)
  instance_variable_set(:@phone, normalized_number)
end

Problem 💥

Now, during the migration phase were we utilize both gems in parallel, our own implementation would of course also need to integrate the internals of both gems in our own overriden implementation. Plus we would need to ensure the attr_encrypted related code is removed again once the migration phase is over 🤯.

This seemed overwhelming and just way too many things to take care of for our own tiny model implementation. Integrating gems should ideally not interfere too much with our own plans to normalize attributes before assigning them. In addition to that, integrating so deeply with a gem, that understanding the code required reading the gems internals beyond the "normal" instructions in the README also comes with a high price on maintainability.

So we needed to find another way to use both gems in parallel while also guaranteeing our values are normalized before assigning and before encrypting them.

Rescue 🚑

A post on the arkency blog describes how Ruby's prepend method on Module can be utilized to override or better "overlay" methods added directly onto the class by a gem. One can prepend an anonymous module inline with the own implementation to either fully override the gem's implementation or just "prepend" one's own implementation and then call super() to still invoke the code generated by the gem.

We simply want to "prepend" our own normalization step before the gems start to do their magic and ideally don't want to get into the details of what they are actually doing. So utilizing super() after the normalization step fits our use case perfectly. In the model class definition, this could look like this:

class User < ApplicationRecord
  attr_encrypted :email, key: key
  attr_encrypted :phone, key: key

  encrypts :email, :phone, migrating: true

  prepend(Module.new do
    def phone=(value)
      normalized_number = Normalizers::PhoneNumber.normalize(value)
      super(normalized_number)
    end
  end)
end

This "overlays" or "prepends" our normalization step before the class' implementation of the setter method, even when it's changed by any of the included gems. So we make sure the value is normalized before it is assigned and the gems' implementation of the setter invoked afterwards without any need for us to fiddle with internal details.

⚠️ Note however, that for the sake of readability and consistency, our model class definition follows Rubocop's Rails Style Guide:

Group macro-style methods (has_many, validates, etc) in the beginning of the class definition.

Following this class layout, our own implementation is sure to be processed after the gems' methods were defined and we are prepending our normalization step before the final definition of the setter method. Ordering the definitions differently in our class would impact the value of super() here.

Cleanup 🧽

While prepending an anonymous module directly inside the class definitions works perfectly fine for our use case here, it still looks very verbose and suspiciously distracting for anyone reading over the model definition. And as we made use of this technique in multiple models within the project, we extracted the boilerplate into a model concern. The goal was to hide the details of the prepending trick while at the same time making the normalization step more visible and explicit to the reader.

We extracted a more general Normalizable concern. We already had multiple normalizer classes in the project. They all follow the same pattern and expose a single normalize class method as their public API. So it just made sense tie the implementation of the Normalizable model concern close to those normalizers:

module Normalizable
  extend ActiveSupport::Concern

  class_methods do
    def normalize(attr, with:)
      prepend(Module.new do
        define_method("#{attr}=") do |value|
          normalized_value = with.public_send(:normalize, value)
          super(normalized_value)
        end
      end)
    end
  end
end

Utilizing this, we can change the previous example to:

class User < ApplicationRecord
  include Normalizable

  attr_encrypted :email, key: key
  attr_encrypted :phone, key: key

  encrypts :email, :phone, migrating: true

  normalize :phone, with: Normalizers::PhoneNumber
end

This hides the internal details of our own implementation of the setter while still making it explicit that we are doing a normalization on the attribute. Similar to the original implementation inlining prepend with an anonymous module, this approach of course still only works as intended if the normalize macro in the class definition is defined after any other setter methods generated by gems are defined. However, in our case it seemed most fitting to place the `normalize calls at the end of the macros section anyways.

Improving callable service objects with private constructors in Ruby

Max — Wed, 25 Nov 2020 10:06:41 +0000

We recently found a nice practical use case for private class methods in Ruby: the constructor methods of what we will refer to here as "callable services" in Ruby on Rails projects.

Callable Services ☎️

First off: what do we mean with "callable services" in Ruby on Rails applications?

Drawers 🗄

In the real world, hardly any Ruby on Rails web application only consists of simple, atomic CRUD (create, read, update, delete) operations on plain resources. Even the famous blog engine built in 15 minutes may at some point incorporate logic beyond the complexity of creating or deleting posts. Take for instance translations, previews, comment moderation, etc. Real world applications model a specific domain and therefore include control flows, procedures and "business logic" tied to that domain. These bits are in fact what makes an app unique and interesting.

When it comes to organizing code, Rails people traditionally aim for "skinny controllers and fat models". But as models packed with responsibilities beyond modeling the domain soon tend to get overweight and hard to maintain, the community started to reach for additional concepts and patterns to organize their domain specific code. One "drawer" one may come across in a lot of Rails applications are "services", also referred to as "service objects" or "procedures". They are usually organized under the app/services/ directory and encapsulate functionality to handle domain-specific logic, such as checking our a cart, registering for the site or starting a subscription. ¹

Services are usually implemented as POROs ("plain old Ruby objects") which, upon a given input, perform a set of operations and return a predictable response. They are easy to unit test and help developers to maintain a sense for the bigger picture by hiding away the internal details in what appears to the outside as a large black-box-function. That is why people started to implement them as classes exposing only a single method, call or run. Depending on its internals and their complexity a service may create a new instance for each call or just utilize a single class method for its work:

class Authenticator
  def self.call(user)
    # complex logic
  end
end

# invoking the service:
Authenticator.call(user)

The same service implemented to use a new instance for each call:

class Authenticator
  def initialize(user)
    @user = user
  end

  def call
    # complex logic, operate on @user
  end
end

# invoking the service:
Authenticator.new(user).call

Conventions 🧘

At bitcrowd we - as you know - love conventions and therefore usually strive for a common API for our classes in app/services in Rails projects. So even if we don't know what's in the box, we at least know we're dealing with a box… Picking up the previous example, this could look like this:

class Authenticator
  def initialize(user)
    @user == user
  end

  def call
    # complex logic
  end

  private

  attr_reader :user

  def private_helper_method
    # some bits of logic, can operate on user
  end
end

So within the project all services follow the same general structure:

a constructor takes all the core input data the service needs to do its work
a single exposed call method invokes the service to perform the actual work

Simplifications 🧹

The functionality encapsulated within a service is seen as one operational unit on the outside. So we probably won't interfere between initializing the service and calling it. If the data needed to be manipulated between new and call, we should probably rather think about drawing the boundaries between our objects differently instead. But having things clearly encapsulated, we could also simplify the service' API to Service.call(<input-data>) and hide the implementation details of it using a new instance for each call inside of it. Since Ruby 2.7, we also make use of the 3 dots argument forwarding syntax ² - also referred to as "forward everything" - for the .call class method:

class Authenticator
  def initialize(user)
    @user = user
  end

  def self.call(...)
    new(...).call
  end

  def call
    # complex logic
  end
end

This API makes our service more streamlined and predictable. The service is easier to read and less verbose on the outside, hiding implementation details inside the class itself. We're also less likely to accidentally sneak code between new and call when invoking the service. And while it's aesthetically pleasing on the eye, it also allows us to write less verbose expectations in our unit tests for code which interacts with the service:

# Before
let(:service_instance) { instance_double(Authenticator) }

it 'calls the service' do
  expect(Authenticator).to receive(:new).with(user).and_return(service_instance)
  expect(service_instance).to receive(:call)
  # ...
end

# After
it 'calls the service'
  expect(Authenticator).to receive(:call).with(user)
  # ...
end

While we previously needed two expectations to ensure both, the service being initialized with the right data and then being invoked, we can now do both in one step.

Communication 📢

Even with the new simpler API, our services can of course still be called in "the old" way, initializing and calling the service in two steps. The API for this approach is public and aside from examples in the code or documentation, we don't have anything at hand to ensure the service is used as intended. People may still happily do Authenticator.new(user).call and use instance doubles in their tests… The new API only gives a hint on how to use services in the project, it does not actually encourage or enforce one unified style.

Private Constructors to the Rescue 🚒

Turns out we can make use of Ruby's private_class_method method on Module to hide the constructor and make our intentions on how to use the services more obvious:

Makes existing class methods private. Often used to hide the default constructor new.

Adapting our example:

class Authenticator
  def initialize(user)
    @user = user
  end

  private_class_method :new

  def self.call(...)
    new(...).call
  end

  def call
    # actual logic
  end
end

With this we introduce a new problem though: the visual overhead of the additional boilerplate code makes the actual service implementation harder to read and understand. One way to circumvent this and allow the readers to stay focussed on the actual business logic, would be extracting the boilerplate into a concern:

module Callable
  extend ActiveSupport::Concern

  included do
    private_class_method :new
  end

  class_methods do
    def call(...)
      new(...).call
    end
  end
end

We can then shorten the service to:

class Authenticator
  include Callable

  def initialize(user)
    @user = user
  end

  def call
    # actual logic
  end
end

This ensures our service is now used as intended. Trying to call its parts separately as Authenticator.new(user).call results in an error:

NoMethodError: private method `new' called for Authenticator:Class

Doing so, we can a clean and concise outer API for our services while not sacrificing readability on its internals. Invoking a service with its single exposed call class method ensure we pass the initial data to set up the state it needs to perform its work and then immediately trigger the actual "work" part.

TL;DR

Use private_class_method to hide the initializer of your service objects for a clearer API and less boilerplate in tests 💅.

References

Folding sections of Markdown in Vim

Max — Mon, 25 May 2020 07:22:45 +0000

TIL: Vim is able to fold sections or Markdown on their headings 💡

Modern version of Vim and Neovim support folding .md Markdown documents on their section headings #, ##, ### etc. out of the box, even without installing the additional vim-markdown-folding plugin.

What is folding 🤔

Vimʼs documentation describes “folding” as:

Folding is used to show a range of lines in the buffer as a single line on the
screen.  Like a piece of paper which is folded to make it shorter:

    +------------------------+
    | line 1         |
    | line 2         |
    | line 3         |
    |_______________________ |
    \            \
     \________________________\
     / folded lines       /
    /________________________/
    | line 12        |
    | line 13        |
    | line 14        |
    +------------------------+

The text is still in the buffer, unchanged.  Only the way lines are displayed
is affected by folding.

The advantage of folding is that you can get a better overview of the
structure of text, by folding lines of a section and replacing it with a line
that indicates that there is a section.

When editing large documents, I personally find it handy to temporarily fold and “hide away” certain parts I currently donʼt care about.

Folding Markdown 🗂

Out of the box, Vim and Neovim currently wonʼt know “how” to fold Markdown sections. Their included default vim-markdown filetype and syntax plugin however offers an undocumented setting to enable just this:

let g:markdown_folding = 1

There is an open pull request include this in the official documentation, but it is not merged yet.

Flicking this switch turns my .md buffers into something like this:

+--- 32 lines: ## What is folding 🤔···········································
+--- 23 lines: ## Folding Markdown 🗂···········································

Pressing zR will recursively unfold all sections again. From there on you may continue to happily fold and unfold. Drew Neil from Vimcasts put together a nice and short overview over the most important folding commands:

command	effect
`zo`	open current fold
`zO`	recursively open current fold
`zc`	close current fold
`zC`	recursively close current fold
`za`	toggle current fold
`zA`	recursively open/close current fold
`zm`	reduce `foldlevel` by one
`zM`	close all folds
`zr`	increase `foldlevel` by one
`zR`	open all folds

First look 👀

With Markdown-folding enabled, Vim will default to the fully “folded” view when opening a Markdown buffer. I personally prefer to start with the “full” document though. According to this Stackoverflow thread there are a few ways out of this behavior.

I went for setting a custom foldlevelstart value in my configuration. It instructs Vim how or if a buffer should be folded when one starts editing:

'foldlevelstart' 'fdls' number (default: -1)
            global
    Sets 'foldlevel' when starting to edit another buffer in a window.
    Useful to always start editing with all folds closed (value zero),
    some folds closed (one) or no folds closed (99).
    This is done before reading any modeline, thus a setting in a modeline
    overrules this option.  Starting to edit a file for |diff-mode| also
    ignores this option and closes all folds.
    It is also done before BufReadPre autocommands, to allow an autocmd to
    overrule the 'foldlevel' value for specific files.
    When the value is negative, it is not used.

I went for starting with all folds open in Markdown buffers:

au FileType markdown setlocal foldlevel=99

Happy folding 🗺

A note on commit messages

Max — Wed, 20 May 2020 07:49:52 +0000

This post is based on the talk “My Message on Commit Messages” I gave at the Ruby User Group Berlin.

Git 🌳

Peeking around the world of software development, one may sooner or later come across the term git. If you havenʼt yet, you have done so now and are prepared to continue this journey here ⛵️.

Software development is a collaborative process where we create, edit and delete text files that constitute our project. With multiple people doing this in parallel in different corners of the project, things can get confusing. We need to keep track of what is happening and inform the others about changes which may impact their work and the changes they are planning. Put into technical terms, we need a so called Version Control System (VCS) (or “Revision Control System”) if we want to build software together. Git is one flavour of these:

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.¹

Git was developed by Linus Torvalds to facilitate working on the Linux kernel with other developers. They initially managed changes to their project by passing around patches and archived files before introducing a proprietary version control system named “Bitkeeper”. Git was born in 2005 when Bitkeeper stopped offering their tool free of charge to the community.² Git is open source, continuously improved and the de facto (industry-) standard when it comes to version control systems nowadays.

Git in 2020 📠

Why are version control systems like Git still relevant in our fast-moving and hyper-modern times of 2020? Building and maintaining software is still a collaborative effort. People need to know about the changes others did to the project and why they did it. Without this, itʼs hard to make good decisions for oneʼs own changes and the project may already be doomed to end up as a digital junkyard. Access to this information at any time without having to ask everyone in the team individually is a key building brick for developing software together.

Without version control systems, we would send around tarballs of source code or mob-access the same files on a shared server. The first is tedious and uninclusive and the second turns developing software into an “open heart surgery” with people yelling “donʼt touch file xzy, I am working on it” through offices, living in constant fear of overwriting and breaking each otherʼs work and somewhere in this struggle naming files algorithm_final_final_2.js. Version control systems give teams structures and processes to “control” the “version” of their project.

Git with its ecosystem of tools and established workflows to document, decide on and reason about code changes has proven to be a good idea. To the point where there is hardly a way around it in todayʼs world of programming.³

A Commit 📸

Before diving into commit “messages”, we need to know about commits:

A “commit” is a snapshot of your files.⁴

A commit is a uniquely identifiable snapshot of the project at a certain moment. It contains the projectʼs current state encoded as information about how it differs from a previous state, along with a log message from the author describing the changes.⁵ The latter part is referred to as the commit “message”.

Commit messages 💌

Many developers treat commit messages more like an annoying “are you sure” confirmation dialogue on the otherwise interruption free autobahn to get their code out into the world and can easily get by with that forever. Commit messages arenʼt absolutely necessary for building software, but they can make life so much easier.

Communication cut ✂️

Software projects often have people working on different tasks in parallel. These tasks are rarely fully independent and can affect each other. Not caring about commit messages, one may experience scenarios where important information about a decision or change in the codebase is missing or not communicated yet. This knowledge gap can have negative impact on following tasks.

Imagine backend and frontend developers collaborating on a feature and neither can read code from the othersʼ domain. The backend people frequently update the codebase with changes labelled “fix”, “polish”, “make work” or “f*ck linters”. Now imagine yourself in the group of frontend developers, waiting for an API specification. How can you know there is something ready to base your work on? Is the API ready, broken or work in progress? Unable to reason about the raw diff, you constantly have to nag the backend developers about these missing pieces of information. This makes collaboration unnecessarily hard, cuts down possibilities for parallelism and introduces hard dependencies on personal availability, mood and memory. Coming in after the weekend, the backend team doesnʼt know about the current state from looking at their commit history either. They have to dig into the code to find out where they left things.

Useful commit messages can help to prevent situations of missing important information. Unlike a Slack message or a coffee chat in the kitchen, they are persisted with the code change as an always available way to retrieve contextual information about it - for your teammates or for yourself, coming back after the weekend.

Information hunt 🔍

Or imagine you need to adapt a piece of code in a legacy codebase. Looking at it for hours, you still donʼt understand the code and why it is as weird as it is. You donʼt dare to touch it as it could impact other areas of the project. With no one around to ask, you look up the commit that introduced the change to see whether it can enlighten you. Depending on the grade of “legacy-ness”, the message of the commit may read:

commit e3aa57ee56e8d9aed89560d1ab4702068ce65d12
Author: Hannah Operator <hannah@dev.internetz>
Date:   Mon May 4 22:44:05 2005 +0200

    make things finally wrk also linter and change UI, fix logging 🍻

This does not help. There is no useful contextual information in this commit message. The author of the code may either long be gone or - surprise - was you a couple of years ago. Nevertheless, you have to continue your search for information elsewhere and eventually just leave the piece of code untouched (and rotting). A good commit message would have been helpful here.

Commit messages can be a useful tool to provide contextual information about a change for others or your future self reading the code. Ignoring them when committing changes is like throwing a boomerang - it will come back at you:

Through its lifetime, code is read far more often then it is written…⁶

Mindset 🧠

Writing good commit messages is more a question of awareness than skill. It needs the right mindset to understand the value and why itʼs worth the effort.

With collaboration no longer tied to physical presence and teams distributed across the globe, good habits of communication - especially asynchronous communication - are more important than ever. Version control systems act as a medium for asynchronous communication among developers working on the same codebase. A Git commit links changes in the code to the people behind and the plans and intentions they had. It stays with the change forever, even if the authors may have left long ago. It can act as a powerful communication channel, even between generations of developers on a codebase.

The diff of a commit states “what” changed, the commit messages the context, the “how” and “why”. Both are important for anyone (including the authorʼs future self) reasoning about the code. A clean commit history with clearly scoped commits and meaningful commit messages acts as automated documentation for the life cycle of the project. It allows people to git blame a line of code to find out when, why and by whom it was introduced or changed.

Unlike API docs or architecture diagrams which usually document a desired or planned state of the project, commit messages reflect the current state of the project and how it was achieved. They can be read as a “process log” for the project. A project manager may ask: “What did you actually ship in the last two weeks?”. A well maintained Git history is able to answer this question within seconds without having to dig through wikis, issue trackers or Slack conversations.

Read vs. Write ↩️

Developing awareness for the value of commit messages means empathizing with readers of the code (including yourself). It relates to how high we value the maintainability and liveliness of software: Is it supposed to last for a while or is it a throw-away piece of digital junk? This of course depends on the project, but all in all, code is read far more often than written:

Indeed, the ratio of time spent reading versus writing is well over 10 to 1. We are constantly reading old code as part of the effort to write new code. …[Therefore,] making it easy to read makes it easier to write.⁶

We should therefore aim to cater the needs of developers reading code, not only the single author writing it. A clean commit history with meaningful commit messages facilitates reading, understanding and maintaining code. To read a piece of code, people, including the author themself, often need to re-establish the context around a change at a later point in time. Reasonably scoped commits with meaningful commit messages give them a chance to do so.

Software we build today usually has an intended lifespan of at least a couple years to decades. This inevitably requires people to read and understand the existing code. In solidarity with future generations of developers, we should aim to this as easy as possible. Writing meaningful commit messages has a significant impact on this and helps preventing frustration. Without this explicit documentation of the commit history, the codebase may easily slide towards big-L legacy, where code is doing things, but itʼs impossible to tell what and why.

🎥 Also watch Tekin Süleymanʼs awesome talk A Branch in Time (a story about revision histories) from Ruby Australia for an illustrative example of the benefits of good commit messages for other developers and your future self. Writing good commit messages makes lives easier.

Writing commit messages ✍️

Commit messages document “why” a change was made. Keeping in mind that this information stays with the change forever, the people at Thoughtbot propose three questions as an orientation for what to actually put into a commit message:⁷

1. Why is the change necessary?

A commit message informs about the purpose of the change and summarizes what the commit is about on a conceptual level beyond what is already visible in the raw diff.

2. How does this address the issue?

A commit message may also give a high level overview of what has been done. Technical detail should be left out as it is better visualized in the diff. It can also be useful to mention alternative approaches one considered. This helps to focus later discussions and makes the solution more transparent to future readers.

3. What are side effects of this change?

A commit message should also list side effects of a change if there are any. Having this close to the change helps when hunting down eventual regression bugs later. Discussing side effects in the code review or inside the team isnʼt sufficient as itʼs rather difficult to dig up this information again later. Noted in the commit message however, the information stays persisted close to the change where people can easily find it.

The questions indicate: commit messages are a powerful communication tool. All developers on the team, including yourself in two weeks will thank you for caring and using them. Therefore - and this applies to a lot of things in life - if you can make the lives of others a bit better, just immediately do it!

The Looks 💅

A good commit message documents the “why” and enables others to understand the context of a change. But how should it look?

Here is an example from the Rails codebase:

Convert configs_for to kwargs, add include_replicas

Changes the `configs_for` method from using traditional arguments to
using kwargs. This is so I can add the `include_replicas` kwarg without
having to always include `env_name` and `spec_name` in the method call.

`include_replicas` defaults to false because everywhere internally in
Rails we donʼt want replicas. `configs_for` is for iterating over
configurations to create / run rake tasks, so we really don't ever need
replicas in that case.

The first line, Convert configs_for to… is referred to as the “subject”, the rest as the “body” of the message. Tim Pope, of general open source fame and the author of the - at least among commit-message-lovers - legendary Note about Git Commit Messages with sensible and concise guidelines for formulating and formatting commit messages states:

The subject/body distinction may seem unimportant but itʼs one of many subtle factors that makes Git history so much more pleasant to work with than Subversion.

Being able to distinguish between subject and body allows for more control about how much detail about a change is displayed in which context.

Subject 📬

The subject is supposed to give a short summary of what the commit is about. You see it in various places in Githubʼs UI or command line tools like git rebase or git reflog where users need a summary of a list of commits, not the individual details.

According to Tim Popeʼs de-facto standard guidelines, the subject should contain an around 50 characters long concise but meaningful summary of the change. Given the size limits, this usually wonʼt make a valid full featured sentence and trying to formulate it as one may indeed be hindering. So itʼs recommended to not treat it as a sentence in the first place and also not end it with a dot. If the subject gets too long, most tools truncate it with ….

Tim Pope also suggests to use imperative forms, such as “fix bug” instead of “fixing bug” and formulate the subject to complete the sentence: “If applied, this commit will …”. This makes it align nicely with Gitʼs auto-generated commit messages like merge and revert and produces a consistent and easy to read commit history.

The subject line is followed by a blank line separating it from the body. This is absolutely crucial: Without this line the message canʼt be parsed correctly and may generate weird cut offs, line breaks and indentation. This brings us to the most important advice in this post. You may forget everything you read so far, but remembering the following will make you a better human being and spark joy in your and other peopleʼs lives:

🚨 Let go of the git commit -m shortcut forever and for good! 🚨

git commit -m <msg> is a shortcut to inline the commit message with the git commit command in one go. This is normally a two-step process where git commit opens your favorite text editor - at least the one configured via the GIT_EDITOR or EDITOR environment variable - to let you enter and save the log message before generating the commit.

It may be challenging to overcome the muscle memory, but youʼll immediately see drastic improvements in your commit message game. A text editor is by far more convenient and inviting for formulating and formatting prose. It gives you spell checking, indentation, line breaks, syntax highlighting and other goodies from the modern world. Multi line commit messages written on the command line will most likely not have the right format and wrapping and end up as a gibberish mess in the Git history.

TL;DR Use a text editor for writing commit messages! 💡

Body 🎒

For a commit small enough that a subject line is already enough to summarize it, one may of course fully omit the body of the commit message. For instance when fixing a small typo in the README.md of the project - a rare case for git commit -m. Most commits are complex enough to require a longer commit message though.

The commit message body explains the “why” and the “how” of the commit in more detail. It can be a multi line text with its paragraphs separated by blank lines and wrapped at 72 characters. Gitʼs default pager less may choke and show hard to read output on other formats, so this is the agreed upon standard and your text editor will most likely already adhere to it by default.

The whole commit message could then be formatted like this example from Tim Pope:

Capitalized, short (50 chars or less) summary

More detailed explanatory text, if necessary.  Wrap it to about 72
characters or so.  In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body.  The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.

Write your commit message in the imperative: "Fix bug" and not "Fixed bug"
or "Fixes bug."  This convention matches up with commit messages generated
by commands like git merge and git revert.

Further paragraphs come after blank lines.

- Bullet points are okay, too

- Typically a hyphen or asterisk is used for the bullet, followed by a
  single space, with blank lines in between, but conventions vary here

- Use a hanging indent

Extras 🍦

Depending on the tool you are using, commit messages can also include additional meta information. Github for instance supports closing issues and citing co-authors from commit message bodies formatted accordingly. The developers at thoughtbot already have the respective lines for pairing partners in their default .gitmessage template:

# 50-character subject line
#
# 72-character wrapped longer description. This should answer:
#
# * Why was this change necessary?
# * How does it address the problem?
# * Are there any side effects?
#
# Include a link to the ticket, if any.
#
# Add co-authors if you worked on this code with others:
#
# Co-authored-by: Full Name <email@example.com>
# Co-authored-by: Full Name <email@example.com>

Configuring a .gitmessage file can be useful to raise awareness and establish certain rules for commit messages within a team. Git uses the contents of the file to pre-fill its commit messages. It can be used as a template, checklist or guideline for writing the actual commit message.

Conclusion 👏

Empathize with the readers of your code and write good commit messages. A clean Git history with meaningful, well formatted commit messages is a great way to show solidarity with fellow developers including your future self and consistently document the life cycle of your project. Adapt a healthy mindset for writing commit messages and make other peopleʼs lives easier ☯️.