Forem: Titouan CREACH

Part 2 - How I replaced tRPC with effect rpc in a Next.js app router application - Streaming responses

Titouan CREACH — Tue, 22 Oct 2024 09:21:16 +0000

If you haven't read the first part yet: https://dev.to/titouancreach/how-i-replaced-trpc-with-effect-rpc-in-a-nextjs-app-router-application-4j8p

With Effect-RPC, it is possible to define a handler that returns a stream, which can be particularly useful when dealing with AI completion or other services that emit chunks over time.

Create Rpc router

In the first part, we defined a tagged request—an object containing a schema for success, payload, and failure.

For a streaming request, we define a request that extends Rpc.StreamRequest.

In this example, let's create an API route that recognizes text from a base64 image. This process is known as OCR (Optical Character Recognition).


import { Rpc } from "@effect/rpc";
import { Schema as S } from "effect";

export class OcrReq extends Rpc.StreamRequest<OcrReq>()(
"OcrReq",
  {
     payload: {
      imageB64: S.String,
    },

     success: S.Struct({
      textChunk: S.String,
    }),

    failure: S.Never,
  },
) {}

Now, we'll create a service to handle the domain logic. In this example, we’ll simulate a stream with a fake AI service.

class AiService extends Effect.Service<AiService>()("AiService", {
  succeed: {
    ocr: (_publicImageUrl: string): Stream.Stream<string, never, never> => {
      return Stream.fromIterable([
        "The",
        "quick",
        "brown",
        "fox",
        "jumps",
        "over",
        "the",
        "lazy",
        "dog",
      ]).pipe(Stream.tap(() => Effect.sleep("1 second")));
    },
  },
}) {}

This service, called AiService, has an ocr method that returns a stream. The stream will emit "The", then "quick", and so on, with a one-second delay between each chunk to simulate AI processing.

Now let’s create our RPC handler. The handler needs to:

Upload the image to obtain a public URL
Call our AI service
Stream the response
Delete the file afterward

Let’s start by creating a new RPC router:

const aiRouter = RpcRouter.make(
  Rpc.Stream(OcrReq, ({ imageB64 }) =>
    Effect.gen(function* () {
      // imaginary service to upload things to a bucket
      const uploadService = yield* UploadService;

      const aiService = yield* AiService;

      // will delete the file once the scope is closed
      const publicUrl = Effect.acquireRelease(
        uploadService.uplad(imageB64),
        (name) => uploadService.delete(name),
      );

      const stream = aiService.ocr(publicUrl).pipe(
        Stream.map((word) => ({
          textChunk: word;
        }),
      ));
    }),
  ).pipe(
    Effect.scoped, // we need to provide a scope, so that when the scope will be closed, the finalizer will run 
    Stream.unwrap, // unwrap the stream if it's wrapped in an effect
    Stream.orDie // since Failure is never, we die instead of returning the error
  ),
);

Next.js handler

Now, we need to convert a Next.js request into an RPC request, and the RPC response into a Next.js response.

In the first part, we created a file route.ts with a POST function.

The function prototype looks like this:

export const POST = async (
  request: NextRequest,
  { params }: { params: Promise<SegmentParams> }, // Next.js 15 async api
) => {
  // ...
}

In this function, we’ll create a handler, which is a function that takes a request and returns a stream. In the first part, we used RpcRouter.toHandlerNoStream. This time, we will use RpcRouter.toHandler.

const handler = Rpc.toHandler(aiRouter);


export const POST = async (
  request: NextRequest,
  { params }: { params: Promise<SegmentParams> }, // Next.js 15 async API
) => {
  const req = await request.json();

  const stream = handler(req);
}

Fortunately, Next.js supports streaming via the standard Response object.

To handle this, we need to encode the stream to a ReadableStream. Here's the code:

const respStream = await Stream.toReadableStreamEffect(
  stream.pipe(
    Stream.chunks,
    Stream.map((_) => `${JSON.stringify(Chunk.toReadonlyArray(_))}\n`),
    Stream.encodeText,
  ),
).pipe(ServerRuntime.runPromise);

Finally, we can return this ReadableStream directly in a response, along with custom headers:

return new Response(respStream, {
  headers: {
    "Content-Type": "application/ndjson; charset=utf-8",
    "Transfer-Encoding": "chunked",
  },
  status: 200,
});

Rpc resolver in the client

To call this on the client side, we need to create an RPC client using a resolver (HTTP resolver). In Effect, there are two types of HTTP resolvers:

A no-stream resolver (for handling payload and response as JSON bodies).
A stream resolver (for handling the payload as a JSON body and the response as ndjson (newline-delimited JSON)).

In the first part, we created a client using the no-stream resolver:

// ❌ From part one

import * as HttpResolver from "@effect/rpc-http/HttpRpcResolverNoStream";
import { Config, Effect } from "effect";
import * as Resolver from "@effect/rpc/RpcResolver";

const frontEndConfigHost = Config.succeed(process.env.NEXT_PUBLIC_HOST);

const makeHelloClient = ({ cookies }: { cookies: string }) => {
  const helloClient = Config.string("HOST").pipe(
    Config.orElse(() => frontEndConfigHost),
    Effect.andThen((host) =>
      HttpResolver.make<HelloRouter>(
        HttpClient.fetchOk.pipe(
          HttpClient.mapRequest(
            HttpClientRequest.prependUrl(`${host}/api/hello`),
          ),
          HttpClient.mapRequest(HttpClientRequest.setHeader("cookie", cookies)),
        ),
      ).pipe(Resolver.toClient),
    ),
  );

  return helloClient;
};

In this case, we will update the import to use the stream-capable HTTP resolver:

+import * as HttpResolver from "@effect/rpc-http/HttpRpcResolver";
-import * as HttpResolver from "@effect/rpc-http/HttpRpcResolverNoStream";
import { Config, Effect } from "effect";
import * as Resolver from "@effect/rpc/RpcResolver";

const frontEndConfigHost = Config.succeed(process.env.NEXT_PUBLIC_HOST);

const makeAiClient = ({ cookies }: { cookies: string }) => {
  const aiClient = Config.string("HOST").pipe(
    Config.orElse(() => frontEndConfigHost),
    Effect.andThen((host) =>
      HttpResolver.make<AiRouter>(
        HttpClient.fetchOk.pipe(
          HttpClient.mapRequest(
            HttpClientRequest.prependUrl(`${host}/api/ai`),
          ),
          HttpClient.mapRequest(HttpClientRequest.setHeader("cookie", cookies)),
        ),
      ).pipe(Resolver.toClient),
    ),
  );

  return aiClient;
};

In the first part, we also made the client available via React.Context.

Consuming the stream

Once all the boilerplate is done, we can move on to what we’re really here for: consuming the stream.

Here’s how you can consume the stream:

import { AiClientContext } from "./RpcClientProvider";
import { useMutation } from "@tanstack/react-query"


export function OcrImage() {
  const rpcClient = React.useContext(AiClientContext);
  const queryClient = useQueryClient();


  const [transcription, setTranscription] = useState(""); 

  const transcribeMutation = useMutation({
    mutationKey: ["transcribeMutation"],

    mutationFn: ({ imageB64 }: { imageB64: string }) => Effect.gen(function* () {

      // Remember, rpcClient is an effect, as it needs the config to be created
      const client = yield* rpcClient;

      // Call the api route
      const stream = yield* client(new OcrReq({ imageB64 }));

      // Then handle the stream as needed
      yield* Stream.runForeach(stream, ({ textChunk: word }) => {
        setTranscription((old) => `${old} ${word}`)
      });

    }).pipe(BrowserRuntime.runPromise)
              // ^ My custom managed runtime for browser 
  })

  return (
    <div>
      <UploadImageB64 onUpload={b64 => startTransition(async () => 
        transcribeMutation.mutateAsync({ imageB64: b64 })
      )} />

      <Input readonly value={transcription} />
    </div>
  )
}

How I replaced tRPC with effect rpc in a Next.js app router application

Titouan CREACH — Wed, 21 Aug 2024 08:04:39 +0000

Part 2 available: https://dev.to/titouancreach/part-2-how-i-replaced-trpc-with-effect-rpc-in-a-nextjs-app-router-application-streaming-responses-566c

I recently fell in love with Effect and began incorporating it into my development projects. As my codebase grew, I found myself using Effect more and more throughout the application.

However, I encountered frustration when one of the layers of my application didn’t integrate well with the others.

While almost every part of my application was using Effect schemas, I discovered that tRPC doesn’t play well with Effect schema transformations. The issue lies in how tRPC infers types.

The input type in tRPC is determined by the return type of the validator function, which is also used by the client to send data to the procedure. However, with Effect schemas, the output of the validator function (decodeUnknownSync) can differ from its input. This is because, in addition to validation, Effect schemas can encode and decode schemas into different types.

I faced a similar issue with the output type. tRPC infers the output type based on the return type of my handler, but I wanted the type to be inferred after the encoding process.

Quick example of this issue:

import { Schema as S } from "effect";

const MySchema = S.Struct({
  createdAt: DateFromNumber
})

const MyRouter = createTRPCRouter({
  myProcedure: t.procedure
    .input(S.decodeUnknownSync(MySchema))
    .query(async ({ input }) => {
    //                👆 { readonly createdAt: Date }
    })
});

// ❌ But in the client :
api.myRouter.myProcedure.useQuery({ createdAt: new Date() })
//                                                 👆 but here createdAt is required of type Date instead of type number.

To address this issue, the solution I found was to manually encode and decode data on both the client and server sides whenever necessary.

However, this approach proved to be very error-prone, and a significant portion of my code ended up being dedicated to encoding and decoding rather than focusing on the core business logic.

RPC Layer

Effect offers an RPC layer called @effect/rpc. Instead of directly calling functions from the frontend, you define requests with schemas and send them. These requests are automatically encoded and decoded at the appropriate stages.

Create an rpc router

The first step is to create an RPC router. To start, let's define a request using a schema. These schemas should be placed in their own file, as they will be imported by both the backend and frontend.

import { Schema as S } from "effect";

export class SayHelloReq extends S.TaggedRequest<SayHelloReq>()("SayHelloReq", {
  payload: { name: S.NonEmptyString },
  success: S.NonEmptyString,
  failure: S.Never,
}) {}

Next, create a service to handle the business logic. I recommend doing this so that you can call the service directly from the backend without relying on the RPC layer.


export class HelloService extends Effect.Service<HelloService>()("HelloService", {
  succeed: {
    sayHello: (name: string) => Effect.succeed(`Hello ${name}`)
  }
}) {}

Afterward, add the service to your managed runtime. If you don't have a managed runtime yet, I suggest creating one and including all the dependencies your application requires.

Finally, set up the router in /app/api/hello/route.ts.

import { Rpc, RpcRouter } from "@effect/rpc";
import { Effect } from "effect";

export const helloRouter = RpcRouter.make(
  Rpc.effect(SayHelloReq, ({ name }) =>
    HelloService.pipe(Effect.andThen((_) => _.sayHello(name))),
  ),
);


export type HelloRouter = typeof helloRouter;
const handler = RpcRouter.toHandlerNoStream(helloRouter);

Api route to handle rpc requests

Once the router is created, we need to route the HTTP requests to it. To do this, we need to export a POST function from our /app/api/hello/route.ts file.

import { NextRequest, NextResponse } from "next/server";
import { Effect } from "effect";
import { ServerRuntime } from "~/server/ServerRuntime"; // my managed runtime

export const POST = async (req: NextRequest) => {
  const data = await req.json();

  return await handler(data).pipe(
    Effect.andThen(Response.json),
    Effect.tapErrorCause(Effect.logError),
    Effect.andThen(ServerRuntime.runPromise)
  )
}

Now, we have an endpoint on /api/hello that can handle a SayHelloReq.

Client side

On the client side, we need to create an RPC client. A custom HTTP client is necessary because we must explicitly pass cookies; otherwise, authentication won't work during SSR.

Here is the constructor for that client:

"use client";

import { HttpClient, HttpClientRequest } from "@effect/platform";
import * as HttpResolver from "@effect/rpc-http/HttpRpcResolverNoStream";
import { Config, Effect } from "effect";
import * as Resolver from "@effect/rpc/RpcResolver";


// juste because Next.js inline those variables, but we should probably use another config provider

const frontEndConfigHost = Config.succeed(process.env.NEXT_PUBLIC_HOST);

const makeHelloClient = ({ cookies }: { cookies: string }) => {
  const helloClient = Config.string("HOST").pipe(
    Config.orElse(() => frontEndConfigHost),
    Effect.andThen((host) =>
      HttpResolver.make<HelloRouter>(
        HttpClient.fetchOk.pipe(
          HttpClient.mapRequest(
            HttpClientRequest.prependUrl(`${host}/api/hello`),
          ),
          HttpClient.mapRequest(HttpClientRequest.setHeader("cookie", cookies)),
        ),
      ).pipe(Resolver.toClient),
    ),
  );

  return helloClient;
};

Finally, create an instance of the client and pass it through React Context.


export const HelloClientContext = createContext<
  ReturnType<typeof makeHelloClient>
>(makeHelloClient({ cookies: "" }));

export default function RpcClientProvider({
  cookies,
  children,
}: {
  cookies: string;
  children: React.ReactNode;
}) {
  const [client] = useState(() => makeHelloClient({ cookies }));

  return (
    <HelloClientContext.Provider value={client}>
      {children}
    </HelloClientContext.Provider>
  );
}

Then, use the client we just created with react query directly:

import { HelloClientContext } from "./RpcClientProvider";

export default function MyComponent({ name }: { name: string }) {
  const rpcClient = React.useContext(HelloClientContext);
  const queryClient = useQueryClient();

  const message = useSuspenseQuery({
    queryKey: ["hello", name],
    queryFn: () => {
      return rpcClient.pipe(
        Effect.andThen((client) => client(new SayHelloReq({ name: "Foo" }))),
        Effect.runPromise,
      );
    },
  });

  return <div>{message}</div>;
}

And that’s all.

I would also recommend wrapping queries in custom hooks and considering wrapping client creation if you have multiple clients.

Here are some links to code that inspired me:

Thanks for reading

After one year of using CSharpFunctionalExtension

Titouan CREACH — Mon, 18 Mar 2024 13:49:44 +0000

This article is not meant to tell you what you should do but it's a story of my journey of using CSharpFunctionalExtension as a lead dev, with a small team, not confortable with functional programming in general.

So why using CSharpFunctionalExtension ?

The main reasons that decided me to use CSharpFunctionalExtension were:

Error as value
Optional values (replacing nullable)
My background with functional programming
Pragmatism

Error as value

Error are simply object representing what happened. From a function, your return a Success representing the value, or an Failure representing an error. So you can't forget to handle the error.
In C#, nothing force you to catch an exception. You also don't know if the function you call will throw an exception and what kind of exception you may catch.
With error as value, you can see everything in the function declaration.

Optional values

Nullable is bad to handle the case of "optional" value. See: the billion dollar mistake for example.
For my case, the nullable flag was not enabled in our projects and it's impossible to tell if a object could be null or not. And it's so easy to forget.

Maybe<T> also give you a good documentation:

public Task<int> CreateUser(string firstname, string lastname, string email)
{ ... }

public Task<Result<int>> CreateUser(Maybe<string> firstname, Maybe<string> lastname, string email)
{ ... }

I love when a function signature describe perfectly the intent of the function.

My background with functional programming

I did some OCaml, Elixir and Haskell as a hobby. I think I understand why the concepts like Maybe or Result are useful and elegant. It's frustrating to me when I see a situation where they would have perfectly fit but I can't use it.

Pragmatism

I've tested several functional programming librairies for C# and I found CSharpFunctionProgramming to be pragmatic and very well designed for my use cases.
I would like to be able to use procedural programming or even OO when I need to. I don't want to much ceremony and I would like my code to stay in the C# standard.
I would like a new hired people to understand the code.

So for that, I don't want too much purity. For example, I don't want IO Monads.

Tips I would love to have before I started

Don't scare your team with functional programming concepts

I noticed that starting to explain what a Functoror a Monad is was a mistake. Because my team directly put the concept in a trash and said: "too complicated for me". So let's start to explain a Maybe represents an optional value, and a Result, a result of an operation, either an error or a success.

So basically, you can check if a Maybe has a value with .HasValue property. And if yes, you can access .Value.
Ok, we know this is not sexy, but this is the starting point.

Be pragmatic

My second mistake was making things complicated. I was liking one-lining everything in a single expression, and chain operations. This resulted and complex code, hard to maintain. C# has no syntactic sugar for this (like do expression in Haskell).

After one year of experience with the lib, I can tell the most readable is early return.

Select functions to teach

I also have noticed there is some function that fit well in some situation. Let have a look to a subset of function I use everyday and where are the situation I would use them.

.TryGetValue
.Deconstruct
.GetValueOrDefault
.Map

`.TryGetValue`

I think the most used function (and the one I discovered very late) in my codebase is: TryGetValue.

TryGetValue allows to test if a Maybe has a value, and bind this value to a variable, in a single operation.

if (maybeEmail.TryGetValue(out var email))
{
    _email.SendTo(email);
}

This function can also be used with Results and be combined with early return. The var you defined with out var ... is accessible in the enclosing scope, this allow you to do:

if (!emailSendingOperationResult.TryGetValue(out var recipientCount))
{
   // handle error here
   return whatever;
}

// use recipientCount here

`Deconstruct`

The Deconstruct function is not used directly but implicitely used in a switch expression or a if statement.

A Maybe<T> is deconstructed into a tuple where the first element is .HasValue and the second one is value

var canDrinkBeers = maybeUser switch 
{
  (true, { Age: >=18, Region: "EU" }) => true,
  (true, { Age: >=21 }) => true,
  _ => false
};

I use deconstruct to mix the Maybe usage and the native C# pattern matching (guard etc)

`GetValueOrDefault()`

The classic: .GetValueOrDefault(), returns the default of the underlying Maybe type, or the first argument you pass to it.

var cpuCount = configurationMaxCpu.GetValueOrDefault(4);

`Map`

The Map function is extremely useful but, I find myself not using it too much. Depends on the case. I often use it when my "mapping function" takes no argument.


HandleMessage(
    maybeMessage.Map(Encoding.UTF8.GetString)
)

Spend time to do code review

Some months after the switch, when I was more confident about how should we code, I tried to spend time in code review. Remember functional programming is not that intuitive at first.

Real code example from a junior dev:


Maybe<int> paymentMethodId = Maybe<int>.None;

if (paymentMethod.HasValue)
{
    paymentMethodId = paymentMethod.Value.Id;
}

var checkoutResult = await _checkoutService.CreateCheckout(paymentMethodId);

Just tell there is a function that fit exactly for this need, the map function:

var checkoutResult = await _checkoutService.CreateCheckout(
  paymentMethod.Map(x => x.Id)
);

Why is pattern matching so great

Titouan CREACH — Wed, 10 Jan 2024 23:06:21 +0000

Ok, let's start with a story. You are a front-end developper and you work with your mate Bob. Bob is a very nice guy but not that rigorous. He sometimes makes breaking changes, return null instead of the response you expected. But you know, life is not perfect.

Anyway, Bob and you are asked to show a profile page for a user.

So let's start to design the response:

{
  firstname: "John",
  lastname: "Doe",
  dob: "1990-03-11T00:00:00Z"
  address: {
    location: [48.864716, 2.349014],
    street: "43 Avenue",
  }
}

So after one or two coffee, you decide to write this typescript type:

export type UserProfile = {
  firstname: string;
  lastname: string;
  dob: string;
  address: {
    location: [number, number];
    street: string;
  };
};

Then, you come up with something like:

export function UserProfile({ profile }: { profile: UserProfile }) {
  return (
    <h1> Hello { profile.firstname } </h1>
    <Map lat={profile.address.location[0]} lng={profile.address.location[1]} />
  );
}

After few minutes you understand that a user may not have filled his profile. So let's check if location exist. Make it optional in typescript first:

export type UserProfile = {
  ...
  address: {
    location?: [number, number]  
  }
}

After the check, you see that Bob decided to, sometimes, returns "NaN" instead of a valid location because you know: typeof NaN === "number".

So now you don't trust Bob anymore and you decide to check everything.

In english you would say:

In order to display a map, I need profile to be an object, and then, check if a key address exist and the value is an object, in that object, I want a key: position to exist. I want the value to be an array of 2 elements. I want these two element to be number not "Nan".

So, basically (I know shorter syntaxes exist), if I want to express that I should write:

if (
  profile && profile["address"] && profile["address"]["location"] &&  Array.isArray(profile["address"]["location"]) && profile["address"]["location"].length === 2 && typeof profile["address"]["location"][0] === "number" && typeof profile["address"]["location"][1] === "number" && !isNaN(profile["address"]["location"][0]) && !isNaN(profile["address"]["location"][1])) {
  // display the map
}

Nobody wants to write this, because it's two long, not fun, and error prone.

This is exactly when pattern matching comes in. Instead of checking every properties, you match you object with a "pattern" (a shape) and see if it match.

Pattern matching is not native in Typescript, but the closest we can do is probably "ts-pattern" (that is excellent)

import { P, match } from "ts-pattern";

...


export function UserProfile({ profile }: { profile: UserProfile }) {
  return (
    <h1> Hello { profile.firstname } </h1>
    {match(profile)
      .with({ address: { location: [P.number, P.number] } }, (location) => {
        return <Map lat={profile.address.location[0]} lng={profile.address.location[1]} />
      })
      .otherwise(() => {
        return null;
      })
    }
  )
}

Using a value after matching it is so common there is a syntax to destructure the result of our matching:

match(x)
  .with({ address: { location: [P.number.select('lat'), P.number.select('lng')] } }, ({ lat, lng} ) => {
    return <Map lat={lat} lng={lng} />
  })
  .otherwise(() => {
    return null;
  })

This is a very clean approach to replace a group of "if".

I found that this way of thinking rest my brain because I don't need to think to every branching, I describe my need, if feels more natural.

Once understood, we see a lot of opportunities to use pattern matching. For example, If you want to show a special message when the user is named "Bob" and is age is greater than 18.

You can write something like:

{
  match(profile)
    .with({ firstname: "Bob", age: P.number.gte(18).select() }, (age) => {
      return `Hello Bob, you are now ${age}, and you can drink a beer !`;
    })
    .otherwise(() => null);
}

There is plenty different ways to match everything, you should check: https://github.com/gvergnaud/ts-pattern