Forem: Duy K. Bui

Access log middleware for Deno Fresh

Duy K. Bui — Sun, 16 Oct 2022 22:25:51 +0000

TL/DR: If you read the title and know exactly what I mean already, simply go to https://deno.land/x/fresh_logging and follow the README.

This is part 3 of a series about my Deno Fresh plugins, so I will not reiterate on the topics already defined in part 1. If you are not interested in my Cloudflare Turnstile plugin, please at least read part 1's Background section if you are unsure what Deno Fresh is.

Background

I was looking for a Deno middleware to help me easily handle the request's Accept header and couldn't find any. As a good citizen of the open-source community, I set out to create one myself and started reading about Deno middlewares. As soon as I read the official docs, however, I realized what the community is missing: an access log middleware! (Side note: as a security software engineer by day, it's my occupational habit to care about access logs.)

As of this post's writing, my plugin supports the two most commonly used (in my daily work) access log formats: the W3C Common Log Format and a variant by Apache called Combined Log Format.

Prerequisites

By now you should already know the drill.

Create a Fresh app if you haven't.

deno run -A -r https://fresh.deno.dev sample-app

Now that you have created your Fresh app, add fresh-logging to your import_map.json for convenience:

{
  "imports": {
    "$logging/": "https://deno.land/x/fresh_logging@1.1.2/"
  }
}

Then consume the middleware in your app's routes/_middleware.ts.

import * as getLogger from "$logging/index.ts";
// or
// import { getLogger, ... } from "$logging/index.ts";

export const handler = [
  getLogger(),
  // ... other middlewares
];

Use Case 1: Common Log Format

You do not have to do anything, this is the default format.

Use Case 2: Apache Combined Log Format

Specify LoggingFormat.APACHE_COMBINED for the format option like this:

import { getLogger, LoggingFormat } from "$logging/index.ts";

export const handler = [
  getLogger({
    format: LoggingFormat.APACHE_COMBINED,
  }),
];

The default two headers included are Referer and User-agent. You can override that by optionally providing the combinedHeaders option, which expects a string array of length 2.

Limitations

As of v1.1.2, the following fields are completely omitted (hard-coded to -):

rfc931 (client identifier): not sure how to obtain this
authuser (user identifier): not sure how to obtain this either
bytes (response content length): one way I can think of is to use res.clone() then read its as ArrayBuffer and get the byteLength, but that is both time and memory consuming. Until I can find a more efficient way to obtain this piece of information, omission is the decision.

Users can use the resolvers to provide custom resolutions of the missing fields. For example, the following code snippet allows logging the response bytes:

import { getLogger, ResolutionField } from "$logging/index.ts";

export const handler = [
  getLogger({
    resolvers: {
      [ResolutionField.bytes]: async (_req, _ctx, res) => `${(await res.clone().arrayBuffer()).byteLength}`,
    },
  }),
];

Again, please note that the example above only serves to illustrate how to provide customer resolvers for the missing fields, the actual implementation is sub-optimal. Otherwise, it would have been included as default resolver for that field.

Additional Features

Response Time

Regardless of log format, you can set the option includeDuration to true to include the handler response time at the end of each log entry. The header Server-Timing will also be set on the response.

import { getLogger } from "$logging/index.ts";

export const handler = [
  getLogger({includeDuration: true}),
  // ... other middlewares
];

Note: if includeDuration option is ON, getLogger() will also count the time taken by all of its subsequent middlewares.

For example, putting getLogger() at the beginning of your handler array will count the time taken by all middlewares, while putting it at the very end of your handler array will yield the time taken only by the route handler.

Custom Logger

By default, the middleware uses an Optic logger that sends output to the console at INFO level, with the color, but without the INFO prefix.

You can provide a log function with the signature (message: string) => string to the logger option to use your own log writing implementation.

import { getLogger } from "$logging/index.ts";

export const handler = [
  getLogger({
    logger: (message: string) => {
      console.debug(message);
      return message;
    },
  }),
];

The Optic logging library is full of features (seriously, look at it) and is highly recommended, but you can literally choose any implementation in your custom logger function.

Custom Resolver

As briefly discussed in the Limitations section above, you can provide a custom resolver for fields that the middleware is unsure how to populate. The following code snippet illustrates how to resolve the authuser field.


import { getLogger, ResolutionField } from "$logging/index.ts";

const auth: MiddlewareHandler = (req: Request, ctx: MiddlewareHandlerContext) => {
  ctx.state.user = "-";
  if (req.headers.has("Authorization")) {
    // inspect req.headers.get("Authorization") and override ctx.state.user
  }
  return ctx.next();
};

export const handler = [
  auth,
  getLogger({
    resolvers: {
      [ResolutionField.authuser]: (_req, ctx, _res) => ctx.state.user as string,
    },
  }),
];

Appendix: Options

The getLogger() function accepts an optional object {} with the following options:

Option	Default Value	Notes
`format`	`LoggingFormat.COMMON`	The log format to use, defaulting to the W3C Common Log Format.
`utcTime`	`false`	Whether to log timestamps in UTC or server timezone.
`includeDuration`	`false`	Whether to include handler response time.
`resolvers`	`{}`	Selectively supply customer resolvers for the missing fields. See the section on limitations for more details.
`logger`	`console.info` +color -level	Optionally supply a custom logger function of type `(message: string) => string`. See the section on custom logger for more details.
`combinedHeaders`	["Referer", "User-agent"]	Optionally supply custom request headers to include. Requires setting `format: LoggingFormat.APACHE_COMBINED`.

That's it, my friends. Thank you for staying with me till the end of it. Feel free to start a discussion or file an issue! You may also hit me up on dev.to messaging anytime too!

Cloudflare Turnstile plugin for Deno Fresh (p2)

Duy K. Bui — Sat, 08 Oct 2022 03:58:16 +0000

TL/DR: If you read the title and know exactly what I mean already, simply go to https://deno.land/x/fresh_turnstile and follow the README.

This is part 2 of a series about my Deno Fresh plugins. If you are not sure about some of the topics mentioned here, you are very likely to find them explained in part 1. In this post, I will continue to discuss the remaining use cases supported by the first release of my Turnstile plugin.

Use Case 2: JavaScript-based form submission.

In the previous post, we discussed how to protect a form traditionally submitted. Now imagine the same form, but submitted via AJAX / fetch / REST APIs / GraphQL / etc.

You can use the implicit rendered <CfTurnstile/> component similar to use case 1, but this time we will add an ID and a callback property.

import { useId, useState } from "preact/hooks";
import CfTurnstile from "$turnstile/components/CfTurnstile.tsx";
// ...
const turnstileElementId = useId();
const [turnstileToken, setTurnstileToken] = useState("");
// ...
<CfTurnstile id={turnstileElementId} sitekey="..." callback={setTurnstileToken} />

By simply adding the callback, you can easily access the token at your turnstileToken state variable, which will be set as soon as the Turnstile widget decides that the actor is a human.

However, the token can only be validated at most once, so if the form submission fails for some reason when validating on the server side, you will need a way to retrieve another token. That's where the ID is useful. Get access to the turnstile object with the getTurnstileAsync helper function and call reset with the ID.

import { getTurnstileAsync } from "$turnstile/plugin.ts";
// ... imagine this is inside your form submission action
try {
  if (!turnstileToken) {
    throw new Error("Turnstile is not ready");
  }
  await submit_the_form_with_ajax_or_fetch({
    // other form fields
    "cf-turnstile-response": turnstileToken
  });
} catch (e) {
  // form submission fails, get a new Turnstile token to be ready for the next try
  setTurnstileToken("");
  const turnstile = await getTurnstileAsync();
  turnstile.reset(turnstileElementId); 
}

Now your form is ready for the next submission without having to reload / re-render the page.

Use Case 3: custom interactions.

By now you should have had a fairly good idea on how to do this already, but for completeness's sake let's put it down in writing so that there won't be any misunderstanding.

Similar to the previous case, use the component with an ID and a callback. To demonstrate the idea that you can use the component as freely as any preact component you write yourself, I will use a signal in this example instead of a state variable.

import { useId } from "preact/hooks";
import { useSignal } from "@preact/signals";
import CfTurnstile from "$turnstile/components/CfTurnstile.tsx";
// ...
const turnstileElementId = useId();
const turnstileToken = useSignal("");
// ...
<CfTurnstile id={turnstileElementId} sitekey="..." callback={(token) => turnstileToken.value = token} />

And again, inspect your signal and use the getTurnstileAsync be able to reset the widget as needed.

import { getTurnstileAsync } from "$turnstile/plugin.ts";
// ... imagine this is inside your custom interaction
try {
  if (!turnstileToken.value) {
    throw new Error("Turnstile is not ready");
  }
  await validate_token_on_server_side({
    "token": turnstileToken.value}
  );
} catch (e) {
  // validation fails, get a new Turnstile token to be ready for the next try
  turnstileToken.value = "";
  const turnstile = await getTurnstileAsync();
  turnstile.reset(turnstileElementId); 
}

Note that in order to validate the token, you need to make a call to Turnstile API with your Turnstile site's secret key (see official instructions here). It is not a good idea to retrieve this secret client-side at any point in time. Instead, I highly recommend sending the token to the server side to perform the validation step.

BONUS: explicit rendering

The token from Turnstile has a relatively short validity period (5 minutes in v0). Therefore, if you want to use it with a blog post, an email, or an otherwise time-consuming form, you may want to reset the widget right before form submission.

Alternatively, you may choose to control when the widget gets rendered and time it according to your specificities. The idea is similar to what we've done before: leveraging the useId hook and the getTurnstileAsync helper. I am going to show a very simple example resembling use case 1 except that our user needs to click the "Check my form" button first before zhe can "submit" it.

import { useId } from "preact/hooks";
import { useSignal, computed } from "@preact/signals";
// ...
const turnstileElementId = useId();
const turnstileToken = useSignal("");
const formReady = computed(() => turnstileToken !== "");
const startTurnstile = async () => {
  const turnstile = await getTurnstileAsync();
  turnstile.render(
    turnstileElementId,
    {
      sitekey: "...",
      callback: (token) => turnstileToken.value = token
    }
  ); 
// ...
<form action="..." method="POST">
  // other form fields such as name, email, message, etc.
  <div id={turnstileElementId} />
  <button type="button" onClick={startTurnstile}>Check my form</button>
  <input type="submit" disabled={!formReady} />
</form>

Once the "check my form" button is clicked, the widget rendering will kick in. Once it has determined that the actor is a human, it will invoke the callback with the token, which is sent to the turnstileToken signal, which is chained to the formReady signal, turning the "submit" button from its disabled state into enabled. At this point, you have the token to use dynamically as needed; you also have the hidden input named cf-turnstile-token inside your <div/> ready to be submitted alongside your other form fields.

This approach is very flexible. I'm sure you will find a creative way to solve most of your captcha use cases with this approach if the basic use cases don't fit your bill. Feel free to drop a line and share your insight, or prove me wrong with an unsolvable use case, I challenge you 😉.

That's it, my friends. Thank you for staying with me till the end of it. As I've stated in the beginning this is the first release, so I am sure there are a lot more things I have not covered. Feel free to start a discussion or file an issue! You may also hit me up on dev.to messaging anytime too!

Cloudflare Turnstile plugin for Deno Fresh (p1)

Duy K. Bui — Fri, 07 Oct 2022 16:04:26 +0000

TL/DR: If you read the title and know exactly what I mean already, simply go to https://deno.land/x/fresh_turnstile and follow the README.

DISCLAIMER: Hi all, this is my first post after 3 years of reading exclusively on dev.to, so all criticisms are welcome! Coincidentally, I'm also writing about my first open-source project release, so again provide feedback by all means.

Background

Alright, now that the "noob" disclaimer is out of the way, let me quickly clarify all the nouns I mentioned in the post title:

Cloudflare: CDN, DOS-protection, 1.1.1.1, VPN, etc.
Turnstile: Cloudflare's recently released CAPTCHA alternative.
Deno: The hopeful successor of NodeJS.
Fresh: Deno's recently released web framework with a focus on island architecture.
plugin: 3rd-party library to inject CSS and JS into every page in Fresh.
CAPTCHA (not in the title, but for completeness' sake): a common instrument to guard against machine-driven web interactions.

OK, class is dismissed; you will be quizzed on these terms next week.

Kidding 😂

Now that we are absolutely on the same page (we are, right?) about what I made, let's talk about how it's supposed to be used.

In this first release, I'm covering three basic use cases of Turnstile:

Protecting a traditional form submission.
Protecting a form submitted with AJAX / fetch / etc.
Protecting a custom interaction.

Use Case 1: traditional form submission.

Err not so fast, before you can do anything with the plugin you need to have your Turnstile configured, which leads us to ...

Prerequisites!

You need to give up an email address (to Cloudflare, not me).
You need a Cloudflare account (not my call, don't argue).
You need a Turnstile site (free for now, again not my call).

Follow the instructions on the official page and you will be good to go in 30 seconds: https://www.cloudflare.com/lp/turnstile/

Oh, and you also need a Fresh app (not any fresh app, the Deno Fresh app, ok?).

Simply run this command to create one if you haven't.

deno run -A -r https://fresh.deno.dev sample-app

Now that you have created your Fresh app, add fresh-turnstile to your import_map.json for convenience:

{
  "imports": {
    "$turnstile/": "https://deno.land/x/fresh_turnstile@1.0.0-0/"
  }
}

Then consume the plugin in your app's main.ts.

import { TurnstilePlugin } from "$turnstile/index.ts";

await start(manifest, {
  plugins: [
    // ...
    TurnstilePlugin(),
    // ...
  ],
});

Now you are ready to use the plugin.

Use case 1 (for real this time)

This is the most common use case for Turnstile.

The basic idea is you have a form, such as a "leave us a note" form, which you only want human beings to be able to submit as that should largely reduce the amount of spam you receive from the form.

To achieve that with Turnstile, on the client side, you put a Turnstile widget inside your form, which will talk to Turnstile API, do the magic, and decide whether the actor is a human. There are several modes of operation for the widget: interactive, non-interactive, and completely hidden, which you chose when you created the Turnstile site in your Cloudflare dashboard. Once the widget has verified (or it thinks it has) that the actor about to submit the form is a human, it will append a hidden input to your form with a token from Turnstile, which gets included as the form is submitted. Then, on the server side, you can validate the token by making a call to the Turnstile API.

This plugin provides a <CfTurnstile/> component for you to use on the client side (fill in sitekey from your Turnstile site, which can always be retrieved from your Cloudflare dashboard):

import CfTurnstile from "$turnstile/components/CfTurnstile.tsx";
// ...
<form action="..." method="POST">
  // other form fields such as name, email, message, etc.
  <CfTurnstile sitekey="..." />
  <input type="submit" />
</form>

Once the page is fully rendered, <CfTurnstile/> will be replaced with a DOM tree similar to this:

<div class="cf-turnstile">
  <iframe><!-- THE WIDGET --><iframe>
  <input type="hidden" name="cf-turnstile-response" value="<!-- THE TOKEN -->" />
</div>

As the form gets submitted, you can look for cf-turnstile-response in your route handler and follow Turnstile's instructions for validation.

Or, if your form is submitted to a POST route, you can use the provided handler generator like this (again, fill in cf_turnstile_secret_key with your Turnstile site's secret, also retrievable from your Cloudflare dashboard):

import { CfTurnstileValidationResult, generatePostHandler } from "$turnstile/handlers/CfTurnstileValidation.ts";

import Page from "$flowbite/components/Page.tsx";

export const handler = { POST: generatePostHandler(cf_turnstile_secret_key) };

export default function CfTurnstileValidation({ data }: PageProps<CfTurnstileValidationResult | null>) {
  /* 3 scenarios can occur here:
   * 1. data is null => the form was not submitted correctly, or the secret key was not provided.
   * 2. data.success is false => the form was submitted correctly, but validation failed. data["error-codes"] should be a list of error codes (as strings).
   * 3. data.success is true => the form was submitted correctly and validated successfully. data.challenge_ts and data.hostname should be available for inspection.
   */
}

A side note on the interactivity of the widget:

If you look at the <CfTurnstile/> component source code, you will not see an IS_BROWSER guard, which means that it will be interactive even outside an island. That might sound like a violation of the island architecture at first, but with implicit rendering, the actual augmentation of the widget on top of the placeholder <div/> is done by Turnstile's own JavaScript, making it very difficult to render a "disabled" widget.

Therefore, I made a decision to leave that alone and rely on the plugin consumers' mental efforts to only use this component inside their islands. If you strongly believe that this decision is a mistake, please file an issue on the plugin repo:
https://github.com/khuongduybui/fresh-turnstile/issues/new

That's it for tonight; thank you very much for your time.
In the following posts, I will discuss the remaining use cases.