Forem: Wang Sijie

Migrating from Node.js crypto to Web Crypto API: A guided experience

Wang Sijie — Wed, 13 Sep 2023 06:40:10 +0000

Introduction

The Web Crypto API is a new Javascript tool for cryptography. It is compatible with modern browsers and leading platforms, including Cloudflare Workers and Vercel, and also Node.js. Such interoperability implies a developer's dream - write your cryptography code once and run it across numerous platforms using the Web Crypto API.

However, Node.js already has a "crypto" module with a long history, leaving a significant amount of legacy code in need of migration. In this article, we will delve into the transition experience, providing a comprehensive guide focusing on 3 common scenarios. We aim to illuminate the pathway to successful migration.

What is Web Crypto API

Web Crypto API, an open standard by W3C for JavaScript, is a collection of standardised cryptography primitives defined in the Web Cryptography API specification. It was created after several browsers and platforms began adding their own non-interoperable cryptography functions.

The API provides primitives for key generation, encryption and decryption, digital signatures, key and bit derivation, and cryptographic digest. It is centered around an interface called SubtleCrypto, you can find more details and tutorials in the Mozilla MDN documentation.

But Node.js already has crypto module?

Node.js developers are typically familiar with the crypto module. It offers a comprehensive set of cryptographic primitives. This module not only provides mechanisms for the same cryptographic operations defined in the Web Crypto API but often includes a broader range of cryptographic algorithms.

So why do we still need Web Crypto API? As Javascript becomes popular across many platforms and environments , from the client-side to servers and especially on the edge, it's important to have a cross-platform cryptographic tool to simplify processes.

In addition, the functions in Web Crypto API standard all return promises and support the async/await syntax. This is a significant advantage over the crypto module, which is synchronous and can block the event loop.

And there is a interesting thing that Node.js adds its support for Web Crypto API, that means, in most cases, Web Crypto API is perfect in most known platforms.

Using Web Crypto API

On most platforms, the collection of Web Crypto APIs is accessible via the global crypto object, which includes 3 top-level utilities: getRandomValues, randomUUID and subtle.

There are many differences compared to the traditional crypto module. They can be summarized into 3 most common parts. Let’s go through them and see how to migrate from existing code.

#1 Generate random values

In crypto module, you can generate random values by calling randomBytes

import { randomBytes } from 'crypto';

const generateRandomString = (length = 64) => randomBytes(length).toString('hex');

There is a simalar function called getRandomValues in Web Crypto, but the return value is an ArrayBuffer, so we need an another step to convert it to string.

const generateRandomString = (length = 64) => {
  const array = new Uint8Array(10);
  crypto.getRandomValues(array);
  return Array.from(array)
    .map((byte) => byte.toString(16).padStart(2, '0'))
    .join('');
};

#2 Hasing (or digest)

createHash is easy to use in crypto module:

export const sha256 = (text: string): string => {
  return createHash('sha256').update(text).digest('hex');
};

In Web Crypto, we can use subtle.createHash

export const sha256 = async (text: string): Promise<string> => {
  const encoder = new TextEncoder();
  const data = encoder.encode(text);
  const hash = await crypto.subtle.digest('SHA-256', data);

  return Array.from(new Uint8Array(hash))
    .map((byte) => byte.toString(16).padStart(2, '0'))
    .join('');
};

As you can see, the transformation from ArrayBuffer to hex string is also needed.

#3 Encryption and decryption

In crypto module, AES encryption and decryption can be implemented with:

export const encrypt = (text: string, password: string) => {
  const iv = randomBytes(16);
  const cipher = createCipheriv('aes-256-gcm', password, iv);
  const encrypted = cipher.update(text, 'utf8', 'base64');
  return {
    ciphertext: `${encrypted}${cipher.final('base64')}`,
    iv: iv.toString('base64'),
  };
};

export const decrypt = (ciphertext: string, iv: string, password: string) => {
  const decipher = createDecipheriv('aes-256-gcm', password, iv);
  const decrypted = decipher.update(encrypted, 'base64', 'utf8');
  return `${decrypted}${decipher.final('utf8')}`;
};

In Web Crypto, we need to create key by importKey first:

async function encrypt(text: string, password: string) {
  const iv = crypto.getRandomValues(new Uint8Array(12));
  const encodedPlaintext = new TextEncoder().encode(text);

  const secretKey = await crypto.subtle.importKey(
    'raw',
    Buffer.from(await getKeyFromPassword(password, crypto), 'hex'),
    {
      name: 'AES-GCM',
      length: 256,
    },
    true,
    ['encrypt', 'decrypt']
  );

  const ciphertext = await crypto.subtle.encrypt(
    {
      name: 'AES-GCM',
      iv,
    },
    secretKey,
    encodedPlaintext
  );

  return {
    ciphertext: Buffer.from(ciphertext).toString('base64'),
    iv: Buffer.from(iv).toString('base64'),
  };
}

async function decrypt(ciphertext: string, iv: string, password: string) {
  const secretKey = await crypto.subtle.importKey(
    'raw',
    Buffer.from(await getKeyFromPassword(password, crypto), 'hex'),
    {
      name: 'AES-GCM',
      length: 256,
    },
    true,
    ['encrypt', 'decrypt']
  );

  const cleartext = await crypto.subtle.decrypt(
    {
      name: 'AES-GCM',
      iv: Buffer.from(iv, 'base64'),
    },
    secretKey,
    Buffer.from(ciphertext, 'base64')
  );

  return new TextDecoder().decode(cleartext);
}

Conclusion

As you can see, the migration is not hard, the main job is to change the syntax to fit the new API and resolve ArrayBuffer with TextEncoder.

As an identity product, Logto uses cryptography in many places. We have migrated from the crypto module to the Web Crypto API. This transition enables us to better adapt to edge environments and makes it possible to execute SDK code securely in the browser.

Implementing stateless session for Next.js using Server Actions

Wang Sijie — Wed, 23 Aug 2023 09:21:06 +0000

Introduction

Following the much-celebrated release of the App Router released, Next.js introduced another feature, Server Actions. This new innovation facilitates server-side data manipulations, reduces reliance on client-side JavaScript, and progressively enhances forms–all while using JavaScript and React to create robust web applications without the need for traditional REST APIs.

In this article, we tap into the wealth of advantages offered by this innovation and see it in action as we implement a cookie-based stateless session for Next.js. This piece serves as a step-by-step guide that will walk you through every phase of crafting a demo project using the App Router.

Our practical demonstration can be readily deployed to Vercel using Edge Runtime. And you can download the full source code from GitHub.

Steering away from REST APIs

Traditionally, if we want to create a Next.js web app that queries database in the backend, we may create some REST APIs to validate auth state and query database, then the front-end React app will call these APIs. But if there is no need to open API to the public and this react app is the only client, it seems redundant to use REST APIs for they will only be called by ourselves.

With Server Actions, the React component can now run server side code. Rather than needing to manually create an API endpoint, Server Actions automatically create an endpoint for Next.js to use behind the scenes. When calling a Server Action, Next.js sends a POST request to the page you're on with metadata for which action to run.

The need for stateless session

As a preferred framework for creating stateless applications, Next.js often means serverless, in which we can not use the memory to store session data. Traditional sessions necessitate the use of an external storage service, such as Redis or a database.

However, when the session data remains small enough, we have an alternative path: engineer a stateless session using secure encryption methods and cookies stored on the client-side. This method bypasses the need for external storage and keeps the session data decentralized, offering non-trivial benefits regarding server load and overall application performance.

So our target is a lightweight, efficient stateless session that capitalizes on the client-side storage capability while ensuring security through well-executed encryption.

Basic session implementation

First up, we need to initiate a new project:

npx create-next-app@latest

For more details on the installation, refer to the official guide.

Crafting a session library

To make understanding Server Actions easier, we'll create a simplified version of the session first. In this version, we'll use JSON to store session data in cookies.

Create a file called session/index.ts and include the following code:

'use server';

import { cookies } from 'next/headers';

export type Session = {
  username: string;
};

export const getSession = async (): Promise<Session | null> => {
  const cookieStore = cookies();
  const session = cookieStore.get('session');

  if (session?.value) {
    return JSON.parse(session.value) as Session;
  }

  return null;
};

export const setSession = async (session: Session) => {
  const cookieStore = cookies();
  cookieStore.set('session', JSON.stringify(session));
};

export const removeSession = async () => {
  const cookieStore = cookies();
  cookieStore.delete('session');
};

The first line "use server" marks this file’s functions as Server Actions.

Since we cannot access request and response directly, we use next/headers to read and write cookies. This is only available in Server Actions.

Incoming: two more Server Actions

With the session library in place, it's time to implement a sign in and sign out feature to showcase the session's usability.

Incorporate the following code into user/index.ts:

'use server';

import { removeSession, setSession } from '@/session';

export const signIn = async (username: string) => {
  await setSession({ username });
};

export const signOut = async () => {
  await removeSession();
};

Here, we are using a 'pretend' sign in process that merely requires a username.

Building the page

In App Router, the page is usually an asynchronous component, and Server Actions cannot be directly invoked from such a component. We need to create components using "use client":

components/sign-in.tsx

'use client';

import { signIn } from '@/user';
import { useState } from 'react';

const SignIn = () => {
  const [username, setUsername] = useState('');

  return (
    <div>
      <input
        type="text"
        value={username}
        placeholder="username"
        onChange={(event) => {
          setUsername(event.target.value);
        }}
      />
      <button
        disabled={!username}
        onClick={() => {
          signIn(username);
        }}
      >
        Sign In
      </button>
    </div>
  );
};

export default SignIn;

components/sign-out.tsx

'use client';

import { signOut } from '@/user';

const SignOut = () => {
  return (
    <button
      onClick={() => {
        signOut();
      }}
    >
      Sign Out
    </button>
  );
};

export default SignOut;

Finally, let's construct our app/page.tsx

import { getSession } from '@/session';
import styles from './page.module.css';
import SignIn from '../components/sign-in';
import SignOut from '@/components/sign-out';

export default async function Home() {
  const session = await getSession();

  return (
    <main className={styles.main}>
      {session ? (
        <div>
          <div>You have signed in as {session.username}</div>
          <div>
            <SignOut />
          </div>
        </div>
      ) : (
        <SignIn />
      )}
    </main>
  );
}

Implementing encryption

The job of Server Actions is done. Now, the final part involves the encryption implementation that can be achieved through crypto.

// session/encrypt.ts
import { createCipheriv, createDecipheriv } from 'crypto';

// Replace with your own key and iv
// You can generate them with crypto.randomBytes(32) and crypto.randomBytes(16)
const key = Buffer.from('17204a84b538359abe8ba74807efa12a068c20a7c7f224b35198acf832cea57b', 'hex');
const iv = Buffer.from('da1cdcd9fe4199c835bd5f1d56446aff', 'hex');
const algorithm = 'aes-256-cbc';

export const encrypt = (text: string) => {
  const cipher = createCipheriv(algorithm, key, iv);
  const encrypted = cipher.update(text, 'utf8', 'base64');
  return `${encrypted}${cipher.final('base64')}`;
};

export const decrypt = (encrypted: string) => {
  const decipher = createDecipheriv(algorithm, key, iv);
  const decrypted = decipher.update(encrypted, 'base64', 'utf8');
  return `${decrypted}${decipher.final('utf8')}`;
};

Next, modify the session library to implement the following:

'use server';

import { cookies } from 'next/headers';
import { decrypt, encrypt } from './encrypt';

export type Session = {
  username: string;
};

export const getSession = async (): Promise<Session | null> => {
  const cookieStore = cookies();
  const session = cookieStore.get('session');

  if (session?.value) {
    try {
      const decrypted = decrypt(session.value);
      return JSON.parse(decrypted) as Session;
    } catch {
      // Ignore invalid session
    }
  }

  return null;
};

export const setSession = async (session: Session) => {
  const cookieStore = cookies();
  const encrypted = encrypt(JSON.stringify(session));
  cookieStore.set('session', encrypted);
};

export const removeSession = async () => {
  const cookieStore = cookies();
  cookieStore.delete('session');
};

Conclusion

Congratulations! You've successfully implemented a stateless session for Next.js. Here is an online preview on Vercel, and you can download the full source code here. We hope this guide aids your understanding of the brand new Server Actions.

Next, we'll be exploring how to use Server Actions to integrate Logto for Next.js. Stay tuned!

Efficiently tracking DAU and MAU in high-traffic sites

Wang Sijie — Wed, 02 Aug 2023 04:03:40 +0000

There's no denying that as an identity provider, Logto processes a mountain of user data daily. One of the critical metrics we can't afford to overlook is 'Daily Active Users' (DAU) - it gives us a feel of how engaged our users are, and it is a basic factor of pricing. But when it comes to tracking DAUs and Monthly Active Users (MAU), things can get messy. Managing massive volumes of data without affecting user experience is not easy, but it's something we've learned to tackle efficiently.

In this read, we're going to take this chance to walk you through DAU related concepts, explain how DAU, MAU, and a monthly engagement rate can be calculated.

DAU, MAU, and Monthly Engagement Rate

Daily active users (DAU)

DAUs are unique users who interact with your platform on any given day. These interactions could be anything - from just logging in, posting or liking something, or any other specific action that signifies activity. Tracking DAUs lets us gauge user engagement on a daily basis and get a sense of our active user base's size.

Monthly active users (MAU)

MAU, on the other hand, is an indicator of unique users who engage with your platform at least once within a month. Just like DAUs, active users can range from anyone who has simply logged in to someone who has interacted with the platform in some way.

Monthly engagement rate

When it comes to understanding just how often users interact with our platform in a month, we find immense value in a particular metric - the Monthly Engagement Rate (MER). This is not merely a number for us, but a pivotal element that heavily influences our pricing strategy.

MER reveals the frequency of a user's engagement by pinpointing the count of unique days a user logs in or interacts with our platform within a month. This insight isn't just useful; it's a key differentiator for us. Unlike the typical approach that solely focuses on whether a user visited within a month, we dive deeper, we strive to understand 'how often.' This 'how often' enables us to gauge the depth of a user's interaction with our platform and provides us with tangible metrics to deliver more value to our users.

In Logto, MER is a significant factor of usage-based pricing. By anchoring our pricing to MER, we ensure our charges align with the user's level of engagement; you could say we've turned fairness into a metric we can measure. While others might count a user who's logged in only once the same as a daily user, we differentiate, pricing based on the actual usage and value provided. Through this, we ensure a more transparent, fair, and ultimately, a standout approach toward pricing.

From user logs

Usually, DAU and MAU are calculated directly from user logs. But as these logs pile up, the process becomes inefficient and presents challenges like data scaling and log retention. Not to mention, keeping records of user logs forever isn't practical due to privacy and security regulations.

Another approach: creating a DAU table

That's why we came up with the concept of a DAU table. This table mainly includes two columns - user_id and date. The date can be set to the start of one day.

We have two ways to keep this table updated. The first one is a daily automated job that sifts through the user logs from the previous day. The second one triggers an update whenever a user activity takes place.

Whenever we come across a unique user_id, we add a new entry to the DAU table, which includes the user_id and the date. This is done using a SQL "upsert" operation which requires a unique key of user_id and date.

As a result, tracking DAU and MAU just becomes a matter of counting rows in the table. And the monthly engagement rate calculation can be done by combining group by and count.

Results and benefits

The DAU table, which essentially is a condensed version of user logs, has dramatically streamlined our DAU and MAU calculation process. Not only did it make tracking user engagement much easier, but it also reduced the load on our system significantly.

Conclusion

In our journey of optimizing DAU and MAU calculations, we've always laid heavy emphasis on user privacy and legal compliance. We built our strategy around ensuring the DAU table wouldn't store sensitive user information. Instead, we just store anonymous identifiers and dates of activity, strictly adhering to all relevant data protection regulations.

We would highly recommend this privacy-oriented approach to others in the digital service industry. Regardless of the scale or volume of user data, privacy should always get top priority.

In the end, data tracking doesn't have to be complicated, even for high-traffic web applications. A stable and efficient strategy like ours can make tracking vital user engagement metrics a cakewalk.

We hope that sharing our experience can inspire other digital platforms and provide ideas on how to develop innovative solutions for effective and efficient data tracking.

How we support an array of diverse connectors

Wang Sijie — Wed, 19 Jul 2023 05:52:33 +0000

Introduction

Connectors play a critical role in Logto. With their help, Logto enables end-users to use passwordless registration or sign-in and the capabilities of signing in with social accounts. Logto allows users to install or build their own connectors, and there are already more than 20 official connectors. Our connectors are designed to be highly flexible, and allow users to install or even build their own. At the same time, it is easy to develop a new connector. To achieve that, we designed and implemented a solution called “Dynamic Form” that is inspired of Config Driven Development (CDD).

Background

The configuration for Logto's connector is a JSON which, though flexible, can be challenging to edit and validate. In the begining, that is not a big problem, because at that time, most JSON configs are like:

{
  "appId": "xxx",
  "appSecret": "xx"
}

But as we support more and more providers and protocals, things began to change. Take “SAML connector” as an example, there are more than 10 keys in the JSON config, and the type of value are complex, there are strings, numbers, JSONs, select values, and multi-line string for certificate.

So we think is time to bring in a fine-designed form to replace the JSON editor. The Logto Connector is designed to work as a third-party application, so hard-coding forms isn't a feasible solution. The result is a strong need for dynamic forms that are easy to manange, and have good user experience at the same time. The form for SAML connector that we mentioned above looks like:

What is Config Driven Development (CDD)

CDD, or Component Driven Development, presents an alternative approach to constructing applications. In the conventional method, lead architects create designs based on business needs, the application is then developed and deployed, and any modifications are executed either through additional elements or through difficult refactoring.

In contrast, CDD focuses on building independent components right from the beginning, starting at the most fundamental level. An interface, typically JSON, is established to assemble the higher-level user interface. This method, involving the combination of reusable components and a JSON blueprint, empowers developers to conveniently build applications in a more flexible and scalable manner.

The essence of CDD lies in its utilization of modularity to craft a loosely linked set of components, which are then brought together using a standardized interface.

What is a Dynamic Form?

Embracing the Config Driven Development (CDD) approach allows us to create dynamic web forms. These are not ordinary forms with static, unchanging fields; rather, they're dynamic entities with fields that are generated based on a JSON file. The beauty of these dynamic web forms lies in their flexibility – if a change in the form fields is required, all it takes is an update to the JSON configuration. This level of adaptability makes dynamic web forms an ideal solution for Logto Connectors, which require diffrent form configurations that are unknown to Logto.

Designing Schema

We designed the schema to be an array of form items, each item may have name, label, placeholder, and the most important, type. We have 6 types of form item, and defined as an enum:

enum FormItemType {
  Text = 'Text',
  Number = 'Number',
  MultilineText = 'MultilineText',
  Switch = 'Switch',
  Select = 'Select',
  Json = 'Json',
}

And the type definition of form item is:

type FormItem = {
  type: FormItemType;
  name: string;
  label: string;
  placeholder?: string;
  required?: string;
  description?: string;
  tooltip?: string;
};

Then the full schema is:

type Form = FormItem[];

But there is a small problem, the form type "select" requires a list of options, so add this field to FormItem:

type BaseFormItem = {
  name: string;
  label: string;
  placeholder?: string;
  required?: string;
  description?: string;
  tooltip?: string;
};

type FormItem =
  | ({
      type:
        | FormItemType.Text
        | FormItemType.Number
        | FormItemType.MultilineText
        | FormItemType.Switch
        | FormItemType.Json;
    } & BaseFormItem)
  | ({ type: FormItemType.Select } & BaseFormItem & {
        options: Array<{ value: string; title: string }>;
      });

Building the form with React-Hook-Form

Now that the schema is set, we can build the form in the front end based on the schema. We'll use React Hook Form to achieve that.

React-Hook-Form is a powerful tool for building efficient, easy-to-use forms in React.

The first thing is to init a form, that's assume that the JSON config is fetched and called formItems, and the data in the form is formData:

type FormData = Record<string, unknown>;

type Props = {
  formItems: FormItem[];
  formData: FormData;
};

const Form = ({ formItems, formData }: Props) => {
  const { register, control, handleSubmit } = useForm<FormData>({ defaultValues: formData });

  const onSubmit = handleSubmit(async (data: FormData) => {
    await submitFormToServer(data);
  });

  return <form onSubmit={onSubmit}></form>;
};

Then, implement form control element according to the type, here is a simple demonstration:

<form onSubmit={onSubmit}>
  {formItems.map((formItem) => (
    <div key={formItem.name}>
      <div className="form-label">{formItem.label}</div>
      <div className="form-input">
        {formItem.type === FormItemType.Text && (
          <input type="text" {...register(formItem.name, { required: formItem.required })} />
        )}
        {formItem.type === FormItemType.Number && (
          <input type="number" {...register(formItem.name, { required: formItem.required })} />
        )}
        {formItem.type === FormItemType.MultilineText && (
          <textarea {...register(formItem.name, { required: formItem.required })} />
        )}
        {formItem.type === FormItemType.Switch && (
          <input type="checkbox" {...register(formItem.name, { required: formItem.required })} />
        )}
        {formItem.type === FormItemType.Select && (
          <select {...register(formItem.name, { required: formItem.required })}>
            {formItem.options.map((option) => (
              <option key={option.value} value={option.value}>
                {option.title}
              </option>
            ))}
          </select>
        )}
      </div>
    </div>
  ))}
</form>

Conclusion

The versatility of Config Driven Development (CDD) shines through when applied to dynamic form creation, especially in the case of Logto's connectors. The advantages are twofold:

For developers, it simplifies the process of creating interactive and user-friendly forms. Instead of dealing with intricate coding complexities, developers only need to define a JSON file to get the better-designed, intuitive user interfaces designed by Logto team.
For users, this approach greatly simplifies the process of setting up a connector. It takes the complexity out of integration, making it easier for users to integrate Logto with social sign-in features, as well as email or SMS services.

Recently, articles promoting Low Code are ubiquitous on the internet. This solution was developed from the perspective of actual user needs, and we believe it is a great representation of Low Code.

Want to give it a try? Go to Logto Cloud and pick a connector.

Our experience adding Edge Runtime to Next.js SDK

Wang Sijie — Wed, 21 Jun 2023 05:04:25 +0000

Introduction

Edge Runtime has become a buzzword in the technology landscape, driving dynamic, low-latency functions in platforms from AWS Lambda@Edge and Cloudflare Workers to Vercel Edge. Emphasizing its importance, Vercel recently changed "experimental-edge" to "edge", signaling official support in their popular Next.js framework.

With our Next.js SDK gaining serious traction, we at Logto thought, "Why not add Edge Runtime support?" So, we rolled up our sleeves and jumped right in. In this article, we're going to share our adventure, looking at the hurdles we faced, how we overcame them, and the cool stuff we learned along the way.

Transitioning modules and dependencies for Edge Runtime support

Working with Edge Runtime poses some unique challenges, primarily because it doesn't support all modules and dependencies commonly used in Node.js. We ran into this issue with the crypto, lodash, and iron-session modules, necessitating some innovative workarounds.

Crypto

In a Node.js environment, the crypto module serves as a wrapper for OpenSSL cryptographic functions. Unfortunately, Edge Runtime doesn't support it. But don't fret - most Edge Runtimes come to the rescue with support for the Web Crypto API. Despite some minor differences, it's a solid stand-in for the crypto module. For instance, to generate random bytes:

// Node.js
crypto.randomFillSync(new Uint8Array(length);
// Edge Runtime
crypto.getRandomValues(new Uint8Array(length));

And hashing:

// Node.js
const hash = createHash('sha256');
hash.update(encodedCodeVerifier);
const codeChallenge = hash.digest();
// Edge Runtime
const codeChallenge = await crypto.subtle.digest('SHA-256', encodedCodeVerifier);

Lodash

Lodash is a favorite among many developers for its utility, but Edge Runtime isn't a fan. Our workaround? We swapped out Lodash functions with native JavaScript methods, keeping our code both efficient and readable.

While replacing most Lodash functions wasn't a Herculean task, it did require some finesse. Let's take a peek at how we recreated the utility of "once" in our own way:

type Procedure<T> = (...args: unknown[]) => T;

export function once<T>(function_: Procedure<T>): Procedure<T> {
  let called = false;
  let result: T;

  return function (this: unknown, ...args: unknown[]) {
    if (!called) {
      called = true;
      result = function_.apply(this, args);
    }

    return result;
  };
}

Iron Session

The iron-session module's latest version is Edge Runtime-friendly, so all we had to do was update our version. Simple as that!

Navigating the Intricacies of "Response" in Edge Runtime

Another challenge that we faced when adapting our SDK for Edge Runtime was handling the differences in the "Response" object. Here's how we overcame these differences:

Creating a response manually

Unlike in Node.js, a request in Edge Runtime doesn't come with a comming request. This meant that we had to create it by calling new Response(), here is an example of returning data:

return new Response(JSON.stringify(context), {
  status: 200,
  headers: {
    'content-type': 'application/json',
  },
});

Letting go of "withIronSessionApiRoute"

In the Edge Runtime, the Response.body is a read-only affair. This means that we couldn't initialize a response before the data was prepared. As a result, our trusty "withIronSessionApiRoute" (along with other middleware) had to be benched.

To understand what we replaced, let's first unpack what withIronSessionApiRoute actually does:

It takes a peek at the cookie, constructs a session object, and ties it to res.
It automatically appends the "set-cookie" header to res if there's a change in the session.

So, how did we emulate this functionality in our new Edge Runtime setting?

Read: We utilized the existing getIronSession function. By giving it an empty and fake response, retrieves the session as needed. This replaced the "get" method from req.session.
Writing: We prepared a response with data upfront, then used getIronSession on this response instance to obtain the session object. Once we had this object in our hands, we could modify the session as required.

// Read
const getLogtoContext = async (request: NextRequest) => {
  const session = await getIronSession(request, new Response());
  const context = await this.getLogtoUserFromRequest(session);

  return context;
};

// Write
const response = new Response(JSON.stringify(user), {
  status: 200,
});
const session = await getIronSession(request, response);

// Modify session
session.userId = 'foo';

return response;

Redirecting

Redirection in Edge Runtime required us to manually add a Location header to our responses.

response.headers.append('Location', navigateUrl);

One package, two runtimes

In this journey of ours, we decided to stick to a single package to support both Edge and Node.js runtimes.

Here’s why

We thought about creating a separate package for Edge, but quickly realized it was unnecessary. Most of our code was shared between the two runtimes, with only a handful of lines needing tweaks. Plus, using the SDK remains pretty much the same across both runtimes, so maintaining a unified package made the most sense.

Here's what we did

Instead of duplicating efforts, we decided to expand the existing package. We added an "edge" folder right in the package's root, cozying up next to the old "src" folder. Then, we updated the package.json file, adding a new path to the "exports". This way, both Edge and Node.js runtimes could live harmoniously within the same package, with minimal fuss.

{
  "exports": {
    ".": {
      "require": "./lib/src/index.js",
      "import": "./lib/src/index.mjs",
      "types": "./lib/src/index.d.ts"
    },
    "./edge": {
      "require": "./lib/edge/index.js",
      "import": "./lib/edge/index.mjs",
      "types": "./lib/edge/index.d.ts"
    }
  }
}

Wrapping up

You can check out the full source code of our Next.js SDK edge part here.

By sharing our journey of embrace Edge Runtime, we hope to inspire and guide others exploring similar paths. Stay tuned for more updates with our Next.js SDK.

Our Journey Migrating Logto SDK Sample to Next.js 13 App Router

Wang Sijie — Thu, 25 May 2023 05:49:37 +0000

Introduction

At Logto, we offer a Next.js SDK named @logto/next, accompanied by a sample project. This project effectively demonstrates how to integrate Logto with Next.js using the traditional "pages" folder, offering examples of API routes, getServerSideProps, Client-side Fetching, and even edge runtime.

Next.js 13 brought us the groundbreaking App Router (previously called the app directory), introducing new features and conventions that quickly became popular in the developer community. Given that Logto’s Next.js SDK fully supports these new features, we were inspired to build a new sample project utilizing this App Router.

In this post, we'll walk you through the details of migrating our old sample project to the new one using the App Router. The project was built upon the old sample project and followed the official migration guide. The process involved several steps: creating pages and layouts, migrating API routes, and utilizing server and client components.

The Migration Process

Page Layout: A New Structure

In our original setup, we utilized an _app.tsx file for setting up the SWR fetcher and an index.tsx file served as the home page.

With the App Router, these become layout.tsx and page.tsx. The layout.tsx file contains the core information, while page.tsx mirrors the functionality of the old index.tsx file.

Client component

One hiccup arose during the first step: setting an "onClick" handler for the button was unsuccessful, yielding an error message stating, "Event handlers cannot be passed to Client Component props. If you need interactivity, consider converting part of this to a Client Component."

To remedy this issue, we extracted the problematic section into a separate component and prefixed the file with use client:

'use client';

type Props = {
  isAuthenticated: boolean;
};

const Nav = ({ isAuthenticated }: Props) => {
  return (
    <nav>
      {isAuthenticated ? (
        <button
          onClick={() => {
            window.location.assign('/api/logto/sign-out');
          }}
        >
          Sign Out
        </button>
      ) : (
        <button
          onClick={() => {
            window.location.assign('/api/logto/sign-in');
          }}
        >
          Sign In
        </button>
      )}
    </nav>
  );
};

export default Nav;

API routes

Transitioning the API routes was as simple as transferring the previous files from /pages/api to /app/api with a few tweaks:

index.ts was renamed to route.ts.
The exported function was renamed to GET or another relevant method name.

Server component

Within the api folder, we have the capability to add server-only functions, which allow React Server Components to fetch data.

In the /app/api/logto/user directory, there's a get-user.tsx file:

import { type LogtoContext } from '@logto/next';
import { cookies } from 'next/headers';

// `server-only` guarantees any modules that import code in file
// will never run on the client. Even though this particular api
// doesn't currently use sensitive environment variables, it's
// good practise to add `server-only` preemptively.
// eslint-disable-next-line import/no-unassigned-import
import 'server-only';
import { config } from '../../../../libraries/config';

export async function getUser() {
  const response = await fetch(`${config.baseUrl}/api/logto/user`, {
    cache: 'no-store',
    headers: {
      cookie: cookies().toString(),
    },
  });

  if (!response.ok) {
    throw new Error('Something went wrong!');
  }

  // eslint-disable-next-line no-restricted-syntax
  const user = (await response.json()) as LogtoContext;

  return user;
}

This function can then be invoked in page.tsx:

import { getUser } from './api/logto/user/get-user';
import Nav from './nav';

const Page = async () => {
  const user = await getUser();

  return (
    <div>
      <header>
        <h1>Hello Logto.</h1>
      </header>
      <Nav isAuthenticated={user.isAuthenticated} />
      {user.isAuthenticated && user.claims && (
        <div>
                    ...
        </div>
      )}
    </div>
  );
}

export default Page.

Conclusion

Upon completing the migration, we found our code and structure significantly more streamlined and intuitive. Although it seemed challenging at the outset, the process was far from daunting. We hope our experience can serve as a valuable guide, helping you confidently migrate your projects to the App Router.

For comparison, here are the links to our projects, both before and after the migration:

Before: https://github.com/logto-io/js/tree/master/packages/next-sample

After: https://github.com/logto-io/js/tree/master/packages/next-app-dir-sample

By examining these projects, you can gain a clearer understanding of the changes and benefits introduced by this migration.

Mastering RBAC in Logto: A Comprehensive Real-World Example

Wang Sijie — Tue, 16 May 2023 07:03:58 +0000

Introduction

Access control and security are essential aspects of modern applications, ensuring that users have appropriate access to resources. Logto's Role-Based Access Control (RBAC) offers developers an efficient way to manage access control and security in their applications. In this article, we'll explore the powerful features of Logto's RBAC implementation using a real-world example, helping you understand and apply these concepts to your projects.

By examining both front-end and back-end code snippets, you'll gain a comprehensive perspective on integrating Logto's RBAC into your application stack. By the end of this article, you'll be well-equipped to harness Logto's RBAC features to enhance your project's security and access control.

Try Logto Cloud Preview to implement RBAC

Introducing BookHarber: An Online Bookstore Use Case

To effectively demonstrate Logto's RBAC features, we'll use a real-world example: BookHarber, an online bookstore. BookHarber offers a wide range of features for customers and staff, ensuring a seamless and secure shopping experience.

Key features of BookHarber include:

Browsing and Purchasing Books: Users can easily search for and purchase books from a diverse collection, spanning various genres and authors.
Order Management and Logistics Tracking: Registered customers can manage their orders, track shipping, and receive updates on their purchases.
Special Offers and Holiday Activities: BookHarber provides exclusive discounts and promotions during special events and holidays to engage and reward its customer base.
Customer Support: Customers can open support tickets to address any concerns or issues they may encounter, receiving prompt assistance from BookHarber staff.
Customer Management: Staff members with different roles have the ability to manage various aspects of the platform, such as customer accounts, order processing, and issue resolution.

Roles

In the BookHarber ecosystem, we can identify several key user roles, such as:

Guest: Unregistered users who can browse the website, search for books, and view special offers.
Customer: Registered users who can purchase books, manage orders, track logistics, and open support tickets.
Store Admin: Staff members responsible for overseeing the overall management and operations of the platform. With full access.
Books Manager: Staff members in charge of book and category management.
Customer Service Agent: Staff members tasked with responding to support tickets.
Third-Party Logistics Provider: External partners responsible for managing and tracking the shipping and delivery of orders.
Marketing Staff: Staff members responsible for promoting BookHarber, responsible for managing special offers and events.

Designing Scopes for BookHarber REST APIs

To effectively implement Logto's RBAC system for BookHarber, we need to design scopes that correspond to the various REST APIs. Scopes are permissions that define the level of access a specific role has for each API endpoint. By assigning the appropriate scopes to each user role, we can ensure that users only have access to the actions and resources relevant to their role.

Let's design scopes for the following REST APIs:

Categories API:
- create:categories: POST /categories
- write:categories: PUT /categories/:id
- delete:categories: DELETE /categories/:id
- list:categories: GET /categories
Books API:
- create:books: POST /books
- write:books: PUT /books/:id
- delete:books: DELETE /books/:id
- list:books: GET /books
- read:books: GET /books/:id
Customers API:
- list:customers: GET /customers
- write:customers: PUT /customers/:id
- delete:customers: DELETE /customers/:id
- read:customers: GET /customers/:id
Orders API:
- create:orders: POST /orders
- list:orders: GET /orders
- read:orders: GET /orders/:id
- write:orders: PUT /orders/:id
Events API:
- create:events: POST /events
- write:events: PUT /events/:id
- list:events: GET /events
- delete:events: DELETE /events/:id
Order Tracks API:
- read:orderTracks: GET /orders/:id/tracks
- create:orderTracks: POST /orders/:id/tracks
- write:orderTracks: PUT /orders/:id/tracks/:trackId
Tickets API:
- create:tickets: POST /tickets
- list:tickets: GET /tickets
- read:tickets: GET /tickets/:id
- write:tickets: PUT /tickets/:id

Assigning Scopes to Roles

Now that we have defined the appropriate scopes for each REST API, we can assign these scopes to the respective user roles in the BookHarber ecosystem:

Scopes	Guest	Customer	Store Admin	Books Manager	Customer Service Agent	Third-Party Logistics Provider	Marketing Staff
create:categories			✓	✓
write:categories			✓	✓
delete:categories			✓	✓
list:categories	✓	✓	✓	✓	✓	✓	✓
create:books			✓	✓
write:books			✓	✓
delete:books			✓	✓
list:books	✓	✓	✓	✓	✓	✓	✓
read:books	✓	✓	✓	✓	✓	✓	✓
list:customers			✓		✓
write:customers			✓
delete:customers			✓
read:customers			✓		✓
create:orders		✓	✓	✓	✓	✓	✓
list:orders			✓
read:orders			✓		✓
write:orders			✓
create:events			✓				✓
write:events			✓				✓
list:events	✓	✓	✓	✓	✓	✓	✓
delete:events			✓				✓
read:orderTracks			✓		✓	✓
create:orderTracks			✓			✓
write:orderTracks			✓			✓
create:tickets		✓	✓	✓	✓	✓	✓
list:tickets			✓		✓
read:tickets			✓		✓
write:tickets			✓		✓

Understanding the Differences Between "List" and "Read" Scopes

To further illustrate the differences between "list" and "read" scopes in the context of REST API design and RBAC, let's consider a real-world example involving an online bookstore, BookHarber.

Suppose BookHarber has two types of users: customers and customer service agents. Customers can create orders, while customer service agents are responsible for helping customers with their orders. Let's take a look at how "list" and "read" scopes apply to the orders API resource in this scenario.

List Scopes: A "list" scope allows the user to access a collection of entities in the system. For example, the list:orders scope permits a user to retrieve a list of all available orders. In the context of BookHarber, this scope could be useful for store administrators or other staff members who need to have an overview of all orders in the system. However, customer service agents should not be able to access the entire list of orders, as their role is to assist individual customers with their specific orders.
Read Scopes: A "read" scope grants the user permission to access a single entity with a given ID. For instance, the read:orders scope allows the user to view detailed information about a specific order by its ID. In BookHarber's case, this scope is ideal for customer service agents who need to access information about a particular customer's order. When a customer opens a support ticket, the customer service agent can use the order ID provided in the ticket to access and view the details of that specific order.

Understanding Ownership: Why Customers Don't Need "Read" or "List" Scopes for Their Own Orders

In many applications, it is common for users to have access to their own resources without explicitly granting them the corresponding "read" or "list" scopes. This is because users are considered the owners of these resources and should naturally have access to them. In the case of our BookHarber example, customers can create orders but do not possess the "read:orders" or "list:orders" scopes.

The concept of ownership plays a crucial role in defining access control for specific resources in a REST API. By acknowledging that users can always access their own resources, we can implement more efficient and secure access control without granting unnecessary permissions. In BookHarber's case, this means that customers can still view and manage their orders without needing any additional scopes.

To demonstrate how this works, let's consider the GET /orders endpoint:

If a user has the list:orders scope (e.g., store administrators or staff members), they will be able to view all orders in the system. This provides them with a comprehensive view of the order data necessary for their role.
If a user does not have the list:orders scope (e.g., regular customers), the system will only return the orders that belong to the user. This ensures that customers can still access their order information without being granted unnecessary permissions.

By implementing this ownership-based access control, the API can provide the appropriate level of access to different user roles while maintaining security and a tailored user experience. In BookHarber's scenario, the ownership model allows customers to access their order information without the need for "read:orders" or "list:orders" scopes, simplifying the access control design and enhancing the overall user experience.

Configuring Settings in Logto Admin Console

To complete the configuration in Logto's Admin Console, follow these steps:

Create a Single Page Application (SPA) for React: Set up an SPA in the Logto Admin Console for your React application.
Create an API Resource: Add a new API Resource with the identifier https://api.bookharber.com.
Define Scopes for the Resource: Create the necessary scopes under the newly created API Resource.
Create Roles and Assign Scopes: Define the user roles for your application, and assign the appropriate scopes to each role.
Assign Roles to Users: Assign the relevant roles to the users in your application, ensuring that each user (Especially staff member) has the correct permissions based on their role.

Protect API using scopes

In our example project, BookHarber, we use Express for the backend service and React for the frontend web page. This section will provide a brief overview of how we can integrate Logto's RBAC features into these popular technologies to secure our application.

The full doc: https://docs.logto.io/docs/recipes/rbac/protect-resource

Front-end

To initialize Logto in your React application, follow the documentation provided here:: https://docs.logto.io/docs/recipes/integrate-logto/react/

In addition to the basic setup, you will need to specify the "resource" and "scopes" in the configuration:

const config: LogtoConfig = {
  endpoint: '<your-logto-endpoint>', // E.g. http://localhost:3001
  appId: '<your-application-id>',
  resources: ['https://api.bookharber.com/'],
  // For "customer" user
  scopes: [
    'list:categories',
    'list:books',
    'read:books',
    'create:orders',
    'create:tickets',
    'list:events',
  ],
};

Here's an example of how to make an API request using Logto:

const { getAccessToken } = useLogto();

const getCategories = async () => {
  const accessToken = await getAccessToken();
  const response = await fetch('https://api.bookharber.com/categories', {
    headers: {
      Authorization: `Bearer ${accessToken}`,
    },
  });
  const categories = await response.json();
  return categories;
};

Back-end

To protect API, follow: https://docs.logto.io/docs/recipes/protect-your-api/

In addition to the example code (https://docs.logto.io/docs/recipes/protect-your-api/node), we’ll need to add the scope validation:

async function parseUserScope(req, res, next) {
  const { payload } = await jwtVerify(/*...*/);
    req.scopes = payload.scope?.split(' ') || [];
  next();
}

function verifyScope(scope) {
  return (req, res, next) => {
    if (req.scopes.includes(scope)) {
      next();
    } else {
      return res
        .status(403)
        .json({ message: "Forbidden: Insufficient scope." });
    }
  };
}

app.get('/tickets', parseUserScope, verifyScope('list:tickets'), (req, res) => {
  // Fetch all tickets
  // ...
  res.status(200).json({ tickets: /*...*/ });
});

app.get('/orders', parseUserScope, (req, res) => {
  if (req.scopes.includes('list:orders')) {
        const orders = await getAllOrders();
    res.status(200).json({ orders });
  } else {
    const orders = await getUserOrders(req.user.id);
    res.status(200).json({ orders });
  }
});

Conclusion

Logto's RBAC system is a powerful tool for managing access control and security in modern applications. By leveraging Logto's RBAC features, you can ensure that users have appropriate access to resources based on their roles, protecting sensitive data and functionality from unauthorized access.

In this article, we explored a real-world example of an online bookstore, BookHarber, and demonstrated how to design scopes, assign them to user roles, and implement Logto's RBAC features in both the front-end and back-end of the application.

By applying these concepts and techniques to your projects, you can enhance the security and access control of your applications, providing a seamless and secure user experience. Whether you're working on an e-commerce platform, a content management system, or any other project requiring role-based access control, Logto's RBAC system offers a flexible and efficient solution to meet your access control needs.

Try Logto Cloud Preview to implement RBAC