Forem: Taylor Beseda

Air Quality App with Enhance

Taylor Beseda — Wed, 12 Jul 2023 17:30:31 +0000

With wildfire season well upon us in North America, it’s a good idea to keep an eye on local air quality.
Let’s get some real time data from the US EPA’s AirNow program.
Even with a limited API request budget, we can get snappy results by caching and refreshing data on demand.
All with features already built into Enhance.

Try the AQI app now.

AirNow API

https://www.airnowapi.org/aq/observation/zipCode/current/?format=application/json&API_KEY=_SECRET_&zipCode=90210

That’s it, that’s the URL we’ll use to request the air quality index (AQI) for any given US zip code*.
We get back an array of one to three measurements if a weather station is found near the requested zip code.

{
  AQI: 58,
  Category: {
    Name: 'Moderate',
    Number: 2,
  },
  DateObserved: '2023-06-08 ',
  HourObserved: 10,
  LocalTimeZone: 'MST',
  ParameterName: 'O3',
  ReportingArea: 'Denver-Boulder',
  StateCode: 'CO',
}

We’ll need to keep in mind that this API limits us to 500 requests per hour, however the docs also let us know that most data points are updated once each hour.
So if our application caches its copy of that data for 15 min, we can query 125 unique zip codes each hour - probably a good start, at least until it goes viral 😉

*For simplicity, I’m sticking with US data, but international data is available.
I recommend checking out IQAir.

Databases in Enhance Apps

To cache AQI info we’ll need to stash data in a database.
And it needs to be fast!

Enhance is built on top of Architect and comes with all of Arc’s superpowers for free.
These features (database access, scheduled functions, event queues, and more) are all opt-in and don’t bloat your deployed project.

The data layer is powered by DynamoDB from AWS, so it’s incredibly quick to set and get data for a page request.
Like, <10ms fast.

The simplest way to start with database operations is to install @begin/data (this library can be used with any Arc/Enhance app, even if you deploy to your own AWS account).

Here’s a sample of usage:

import data from '@begin/data'

const table = 'pizza'

await data.set([ // save multiple pizzas at once
  {table, key: 'bbq-chkn', toppings: ['chicken', 'chz', 'bbq sauce']},
  {table, key: 'southwest', toppings: ['chilis', 'red onion', 'corn', 'chz']},
  {table, key: 'hawaiian', toppings: ['chz', 'ham', 'pineapple']},
])

const bestPizza = data.get({table, key: 'hawaiian'})

console.log(bestPizza.toppings.at(-1)) // 'pineapple'

Cache API Requests

I’ve created a simple form component that GETs the /us route with a zip query parameter.
My Enhance API function will then lookup the most recent AQI data for that zip code.
But a visitor could refresh that page several times over the course of an hour, burning through my allotted AirNow API limit.

So when the data is fetched the first time, I’ll cache the response with a time-to-live (TTL) value of 15 min - AirNow’s data rarely changes for a given location more than once in an hour.
Then, when that user refreshes or someone else requests the same zip code, I’ll check the database first and only query the API if a record doesn’t exist for that zip.

import data from '@begin/data'

const { AIRNOW_URL } = process.env
const table = 'aqi'

export async function get({ query: { zip } }) {
  if (!zip) return {} // just render the form

  const cacheKey = `zip:${zip}`

  let aqiData
  try {
    // first, check the cache
    const cached = await data.get({ table, key: cacheKey })

    if (cached?.aqiData) {
      aqiData = cached.aqiData // continue with data we have
    } else {
      // no cache, go get some data from AirNow
      const response = await fetch(AIRNOW_URL + `&zip=${zip}`)

      aqiData = await response.json()

      await data.set({ // cache the new data
        table,
        key: cacheKey,
        aqiData,
        ttl: Math.round(Date.now() / 1_000) + (60 * 15),
        // ↑ 15 minutes from now as seconds since epoch
      })
    }
  } catch (error) {
    return { status: 500 }
  }

  return { json: { aqiData } }
}

Granted this doesn’t have much error handling, but it serves as a great example of how simple my API layer is here.

Now we can present this data in our views powered by server-rendered custom elements and browser-native web components! I won’t get into the specifics of creating SVG meters (inspired by breathable.app), but here’s what I worked up:

Extended Example

In my full example I also grab a visitor’s likely zip code based on their IP address and serve that result from the index route.
Not only is the AQI data cached, but so is the zip code lookup since that service also has daily limits.

Interactive example here: https://invent-k6b.begin.app/

And the source code: https://github.com/enhance-dev/enhance-example-aqi

You can even see the range of meter values: https://invent-k6b.begin.app/test

Tested: Database Providers on Lambda

Taylor Beseda — Tue, 06 Jun 2023 23:27:09 +0000

The last couple years have seen the rise of third party database providers, Database as a Service (DBaaS).
Instead of hosting your database on the same box as your primary application server, developers can use an external db host.
Often called "serverless" databases, these offerings offload the responsibility of maintaining a database appliance.

Providers also often offer useful dashboards, data browsing, branching, schema versioning, and more.

Selecting a Database

When adding a data layer to any application, choosing a database type, engine, and, now, provider is an important choice.

Database Paradigm + Speed

Probably the most important factor when choosing a database is deciding on "relational" tables like SQL or "document" storage, often called NoSQL.
But for this experiment, we'll set that aside and focus on the second most important consideration: access speed.

Specifically, we'll look at how fast the simplest queries are from a Lambda (deployed to AWS with the vanilla Node.js runtime) to various third party db vendors.

The Tests

I've created an Architect application (hosted on Begin) that's made up of several functions: one for testing each provider and one to provide a web view of embedded <iframes> with the results of each.

Each test implementation performs essentially the same query:

SELECT * FROM things

The code and more technical explanation is available on the actual test page:

Sampled Speeds

Provider	Driver	Approx. Query Time
Neon ¹	`postgres`	300ms
	`@neondatabase/serverless`	100ms
Supabase ²	`postgres`	450ms
	REST API via `fetch`	25 - 375ms
PlanetScale	`mysql2`	125ms
	`@planetscale/database`	25 - 150ms
MongoDB	`mongodb`	725ms
DynamoDB	`@architect/functions`	10ms

¹ This does not include the cold start.
The "wake" time can exceed 5s (5,000ms), but once active is 0.
Neon is in early access and is working on various (paid) ways to manage this penalty.

² Supabase's @supabase/supabase-js was not tested as it requires a build step on install (my CD environment, Lambda, doesn't have node-gyp).
I expect it would perform similarly to their REST API.

Regional Differences

The above sampling is for tests where the database provider is always in a different US region from the Lambdas that connect to them.

The only provider sharing a region with the Lambda is Dynamo since both resources will naturally be created in the same AWS region.

All providers get a significant speed boost when in the same region as your Lambda and using the "native" driver.

For example, when both Lambda and Supabase are in us-east-1, the same query with postgres takes ~50ms: 9x faster 🔥

(Limited tests were conducted in shared regions but are not demonstrated live.)

Considerations

These tests do not:

attempt to pool or keep-alive connections
snapshot results or track variance over time
test subsequent queries
use a large dataset or a variety of DB operations
thoroughly consider resource regions

Conclusions

Not surprisingly, AWS's own DynamoDB is the fastest way to query data from a Lambda-based application.
Its repeatable 10ms query latency is 2.5 times better than the closest competitor's best results.
We acknowledge that it may be intimidating to get started with a NoSQL database, and that's why we provide @begin/data as an abstraction layer on top of DynamoDB.
For folks who want to learn more about DynamoDB we recommend Alex DeBrie's The DynamoDB Book.

That said, all tested provider queries are less than half a second (except MongoDB - however, their paid tiers do reach that 500ms threshold)!

Ultimately any database is better than no database.
Don't be paralyzed or resort to throwing the kitchen sink at the problem. Pick one and get to building the initial implementation.

Begin Domains beta

Taylor Beseda — Tue, 04 Apr 2023 17:39:32 +0000

Today we're excited to announce the beta release of Begin Domains. This early access feature allows you to subscribe to a domain name and link it to a Begin application. You can subscribe to a domain name using the new begin domains command.

Getting Started

Deploy an application

First, you'll need to have an application deployed to Begin; this is free. To learn about deploying applications with Begin, check out the Getting Started guide.

Subscribe to a domain

Once you have an application deployed, you can subscribe to a domain name. Use the begin domains add command to both check the availability of a domain and get a checkout link for the domain.
For this example, I'll use your-name.codes as the domain name.

> begin domains add --domain your-name.codes
your-name.codes is available!
Subscribe here: https://api.begin.com/checkout/XHB1WDPM

Follow the checkout link to subscribe to the domain name. If the domain is not available, some suggestions will be provided.

⏳ Wait for DNS propagation

Once you've subscribed to a domain name, you'll need to wait for DNS propagation. Unfortunately, we can't do much to speed this along, but you can check the status of your domain with the begin domains list command. Add the --verbose flag to see additional details.

> begin domains list --verbose

Link a domain to an environment

Now the fun part: link the new domain to an existing application environment. Use the begin domains link command to link a domain to an environment.

> cd my-app
> begin domains link --domain your-name.codes --env production

We'll provision the resources (SSL cert, DNS, etc.) for your app, so hang tight. Soon, you'll see your domain is linked!

> begin domains list
your-name.codes
└─→ my-app → "production"

Learn More

Find additional info on the new Domains feature in our docs. Including a roadmap of upcoming features.

Read up on the domains command reference.

Combine Utility CSS with Scoped Styles in an Enhance Project

Taylor Beseda — Tue, 20 Dec 2022 17:15:54 +0000

[A post from my drafts folder]

Enhance projects include a (totally optional) built-in CSS utility class system out of the box. By default, the generated stylesheet is very small, and developers can even configure their app’s design scale and color palette.

Let’s create a form layout with the available layout helpers and stylize the submit button with custom colors. For this approach, we’ll blend utility classes and a custom element’s local stylesheet.

Starting with a barebones pizza order form:

<main>
  <h1>Order a Pizza!</h1>

  <form action="/orders" method="POST">
    <fieldset>
      <legend>Pie Size</legend>

      <input type="radio" name="size" id="size_1" value="small" required />
      <label for="size_1">Small</label>

      <input type="radio" name="size" id="size_2" value="medium" />
      <label for="size_2">Medium</label>

      <input type="radio" name="size" id="size_3" value="large" />
      <label for="size_3">Large</label>
    </fieldset>

    <fieldset>
      <legend>Toppings!</legend>

      <label>Choose one or more <small>(cmd + click)</small>:</label>

      <select name="toppings" multiple size="12">
        <optgroup label="Premium toppings">
          <option value="pep">Pepperoni</option>
          <option value="jal">Jalapenos</option>
          <option value="oliv" disabled>Green olives</option>
        </optgroup>
        <optgroup label="Regular toppings">
          <option value="pine">Pineapple</option>
          <option value="feta">Feta cheese</option>
          <option value="bell">Bell pepper</option>
        </optgroup>
        <optgroup label="Free toppings">
          <option value="saus">Italian Sausage</option>
          <option value="ham" disabled>Ham</option>
          <option value="spin">Spinach</option>
        </optgroup>
      </select>
    </fieldset>

    <button type="submit">Place My Order</button>
  </form>
</main>

Well, that doesn’t look like much:

In order to give us a clean slate, Enhance Styles’ CSS reset is fairly aggressive. No problem, let’s start by building up a better layout.

<style>
  main { max-width: 50ch; }
</style>

<main class="mt4 m-auto font-sans">
  <h1 class="text2 text-center mb1">Order a Pizza!</h1>

Right away, I’ve added a <style> tag that scoped just to this page. I’ve set a max-width and then added our first utility classes to the <main> tag to get a bit of margin and use a sans-serif font stack. Similarly, I’ve set a text size and centering on the main <h1> header.

Let’s continue by adjusting each <fieldset>’s padding and a border:

<fieldset class="p1 border2">

All elements have a border-color of transparent by default, so the next step is to set the fieldset’s border-color in our <style> tag to a provided color swatch from Enhance Styles:

fieldset { border-color: var(--secondary-200) }

Taking a look now, and we’re in much better shape.

Alright, time for some grid layout! First up, the form:

<form action="/orders" method="POST" class="grid flow-row gap-2">

And with our web inspector’s help, we can see the grid is coming together:

We’ll use the flex class to do something similar with the “Pie Size” <fieldset>, notice the new <div> input/label wrappers:

<fieldset class="flex justify-around p1 border2">
  <legend>Pie Size</legend>

  <div>
    <input type="radio" name="size" id="size_1" value="small" required />
    <label for="size_1">Small</label>
  </div>

  <div>
    <input type="radio" name="size" id="size_2" value="medium" />
    <label for="size_2">Medium</label>
  </div>

  <div>
    <input type="radio" name="size" id="size_3" value="large" />
    <label for="size_3">Large</label>
  </div>
</fieldset>

And a similar treatment for the “Toppings!” section:

<fieldset class="grid flow-row gap-2 p1 border2">

We’re almost there

Now for that submit button, some added CSS in our style tag to set colors

button {
  color: var(--light);
  background-color: var(--primary-400);
}

And a wrapper div + some utility classes

<div class="mt-2 flex justify-around">
  <button type="submit" class="pt-4 pb-4 pl-1 pr-1 font-bold text1 radius1">
    Place My Order
  </button>
</div>

Finally, we’ve got a nice-to-use Pizza Order Form

The final markup, all in one Enhance page file:

<style>
  main {
    max-width: 50ch;
  }
  fieldset {
    border-color: var(--secondary-200)
  }
  button {
    color: var(--light);
    background-color: var(--primary-400);
  }
</style>

<main class="mt4 m-auto font-sans">
  <h1 class="text2 text-center mb1">Order a Pizza!</h1>

  <form action="/orders" method="POST" class="grid flow-row gap-2">
    <fieldset class="flex justify-around p1 border2">
      <legend>Pie Size</legend>

      <div>
        <input type="radio" name="size" id="size_1" value="small" required />
        <label for="size_1">Small</label>
      </div>

      <div>
        <input type="radio" name="size" id="size_2" value="medium" />
        <label for="size_2">Medium</label>
      </div>

      <div>
        <input type="radio" name="size" id="size_3" value="large" />
        <label for="size_3">Large</label>
      </div>
    </fieldset>

    <fieldset class="grid flow-row gap-2 p1 border2">
      <legend>Toppings!</legend>

      <label>Choose one or more <small>(cmd + click)</small>:</label>

      <select name="toppings" multiple size="12">
        <optgroup label="Premium toppings">
          <option value="pep">Pepperoni</option>
          <option value="jal">Jalapenos</option>
          <option value="oliv" disabled>Green olives</option>
        </optgroup>
        <optgroup label="Regular toppings">
          <option value="pine">Pineapple</option>
          <option value="feta">Feta cheese</option>
          <option value="bell">Bell pepper</option>
        </optgroup>
        <optgroup label="Free toppings">
          <option value="saus">Italian Sausage</option>
          <option value="ham" disabled>Ham</option>
          <option value="spin">Spinach</option>
        </optgroup>
      </select>
    </fieldset>

    <div class="mt-2 flex justify-around">
      <button type="submit" class="pt-4 pb-4 pl-1 pr-1 font-bold text1 radius1">
        Place My Order
      </button>
    </div>
  </form>
</main>

Selecting 3rd Party Web Components

Taylor Beseda — Mon, 12 Dec 2022 12:00:00 +0000

Cover image photo by Laam

As Web Components (WCs) continue to gain traction, it's becoming more common to see elements published as re-usable modules. This awesome trend helps new apps get off the ground and incorporate new features without having to implement complex functionality. Here are a few things to consider when selecting a third party WC for your project — Enhance or otherwise.

The dependency graph
Progressive enhancement-ability
Accessibility features
Styling interface
Component "entrypoint"

Don't ship a framework for a single component

Before npm installing your way to success, check what that new WC includes.

There are several lightweight libraries I might expect to see as dependencies (like Alpine.js or WC-specific utilities like Exalt and slimjs). A full front end framework, like React, for a component is wholly unnecessary.

Of course, there are specialized front-end frameworks specifically for Web Components: Lit, Stencil, and FAST to name a few. These can be very beneficial, especially if your app is already making use of one. However, the overhead should be proportional to the surface area of the new component. So if you'd like to use a full design system with several elements and it requires all of FAST, go for it! But if it's just a fancy dark mode toggle that happens to be "built with XYZ", it may not be worth the cost of a framework.

Additionally, many popular front-end frameworks support WCs by trying to export toWebComponent, but still require framework overhead — even if it's bundled, it's big.

Large, complex dependency graphs will only lead to upgrade woes and large server payloads for your users.

The component works without JS (or is the enhancement)

There should be a way to either partially render the WC from the server or use it in a baseline state. The exception would be if the external component library is working to augment an existing feature that has its own baseline.

Example: If the objective of the new component is to scroll a marquee of color-changing text, I wouldn't expect it to animate without JavaScript enabled. However, I would expect that my server can still render the text content so that it's readable by the user — even without that nostalgic text effect.

I want my server to transmit (or static file to contain):

<fancy-marquee>Web Components are RAD!</fancy-marquee>

I should not have to provide the text content via a JS-only API or as an attribute that won't be immediately rendered by the browser. Especially for such a simple element.

Accessible out of the box

Accessibility relates to the previous two points: don't transmit bloated, unused code to the browser and provide a baseline experience without JavaScript.

If those requirements are met, then the WC should also be accessible by default. This means that the component should be keyboard navigable and have a focus state. It should also be able to be used with a screen reader. Check to see that the component is using semantic HTML and ARIA roles/attributes where appropriate.

Example: As noted by MDN, if the component implements a progress bar, a native <progress> element should be used.

However, if the component can't necessarily be represented as a standard HTML element, then ARIA attributes should be used to provide information to assistive technology:

<fancy-marquee
  role="marquee"
  aria-live="off"
  aria-label="Web Components are RAD!">
  Web Components are RAD!
</fancy-marquee>

The component is style-able (or "headless")

If the added Web Component has an HTML interface, it should not be an exercise in browser dev tools gymnastics.

In many cases it's helpful to have some default CSS applied to custom elements and their children. But if my application requires adjustments to text shadow colors and input border radius, I'm going to benefit from a well thought out styling API.

Example: Ideally, well-structured WCs will document CSS variables that can be set within the scope of the custom element:

fancy-marquee { --shadow-color: Crimson; }
fancy-marquee { --container-radius: 0.25rem; }

Example: Another good approach is to provide a list of class names that can be augmented with custom definitions:

fancy-marquee .headline { text-shadow: 1px 1px 2px Crimson; }
fancy-marquee .container { border-radius: 0.25rem; }

Also, be wary of importing CSS into .js files…

Including the component is straightforward

Your new WC should not require a new build pipeline or several import statements.

Consider how your project includes external code, particularly how your web server sends code to the browser. Does the new component plug-and-play?

In Enhance, any file in your node_modules folder can be automatically bundled and server to the browser via the @bundles config in the project's .arc:

@bundles
fancy-marquee './node_modules/fancy-marquee/index.js'

Any page can now reference this bundle:

<script type="module" src="/_public/bundles/fancy-marquee.mjs">

Or from an Enhance element:

import FancyMarquee from '/_public/bundles/fancy-marquee.mjs`

Examples and Takeaways

I don't want to call out any specific libraries as "unworthy" because I'm stoked anyone is authoring reusable components with a focus on web standards and native browser features! So here are a few I've used and would recommend:

json-viewer - A Web Component to visualize JSON data in a tree view
mux-player - A drop in component for adding Mux videos into your web application
github-elements - GitHub's Web Component collection
wc-toast - Notifications component for everyone

Not all third party libraries will score an A+ on all of these points. But if you're looking for a new component, consider these factors when evaluating your options and use discretion when adding new dependencies to your project.

HTML Custom Element as a Feature API

Taylor Beseda — Fri, 09 Dec 2022 05:49:09 +0000

Say you've found a helpful front-end library you'd like to use for a feature in your web app, but you aren't thrilled with the interface.
It requires an awkward data structure or tedious markup.
Let's use an Enhance custom element definition to use as an adapter.

JavaScript-less Charts with charts.css

For this guide, I'll make use of a slick library called “charts.css” — it's a pure CSS way to turn tabular data into really nice charts.

Let's start with the end in mind. Here's the result we'd like to target using just HTML + charts.css:

Be sure to open and scroll through the HTML code here.
That's a lot of markup! And a lot of repetition.
But thankfully, it's just a standard HTML <table> with a few special classes and some CSS variable declarations mixed inline.

Data structure

Our 2016 Olympics medal data isn't likely in an HTML-ready format, so let's assume it comes from a database or API as an array of objects.
We can return that set from our Enhance API route:

export function get(req) {
  const medals = [
    {
      label: 'USA',
      values: [46, 37, 38],
    },
    {
      label: 'GBR',
      values: [27, 23, 17],
    },
    {
      label: 'CHN',
      values: [26, 18, 26],
    },
  ]

  return { json: { medals } }
}

Implementation

Again, we'll start with the goal in mind and author the desired code we want to write for the above chart (CodePen example):

<my-chart
  data-key="medals"
  type="bar"
  heading="2016 Summer Olympics Medal Table"
  value-key="Country"
  value-names="Gold,Silver,Bronze"
  multiple
  show-labels>
</my-chart>

Super simple interface and even follows charts.css conventions without all the markup.
Here is the Enhance element definition where I've implemented the above interface:

export default function MyChart({ html, state }) {
  const { attrs, store } = state

  const config = {
    dataKey: attrs['data-key'],
    type: attrs.type || 'bar',
    heading: attrs.heading || null,
    valueKey: attrs['value-key'],
    valueNames: attrs['value-names']?.split(',') || [],
    multiple: typeof attrs.multiple === 'string',
    showLabels: typeof attrs['show-labels'] === 'string',
  }

  const data = store.chartData[config.dataKey]
  const allClasses = [
    'charts-css',
    config.type,
    config.multiple ? 'multiple' : null,
    config.showLabels ? 'show-labels' : null,
  ]

  return html`
    <table class="${allClasses.join(' ')}">
      <caption>${config.heading}</caption>
      <thead>
        <tr>
          <th scope="col">${config.valueKey}</th>
          ${config.valueNames.map(v =>
            `<th scope="col">${v}</th>`
          ).join('\n')}
        </tr>
      </thead>
      <tbody>
        ${data.map(d => `
          <tr>
            <th scope="row"> ${d.label} </th>
            ${d.values.map((v, i) => {
              const style = [
                `--start: calc(${v}/100);`,
                `--size: calc(${v}/100);`,
                d.colors?.at(i) ? `--color: ${d.colors.at(i)}` : null,
              ]
              return `<td style="${style.join(' ')}">${v}</td>`
            }).join('')}
          </tr>
        `).join('')}
      </tbody>
    </table>
  `
}

At its core, it's a function to loop over the medals data structure to generate <tr> elements for charts.css.
The applied classes are informed by the element attributes added to the <my-chart> element.

Whenever we use the <my-chart> element in our app, this definition will be used by Enhance to render our custom element.
Bonus: this custom element ships zero browser JavaScript!

Not only have we vastly improved the experience of creating new charts in our application, but we've centralized this adapter.
We can iterate on it over time when we want to add features or when the shape of the data inevitably changes.

Tools for testing Functional Web Apps

Taylor Beseda — Mon, 20 Dec 2021 21:38:21 +0000

Photo by Nicolas Thomas

If you're building critical cloud functions to return API results, handle evented business operations (like Shopify webhooks), or render web views, you'll want to incorporate some tests. It's essential to test their internals, inputs, and outputs in a predictable context. We want a utilitarian toolchain to ensure core services function as expected. Where each test can run in isolation, in an unmodified Node.js context. The test suite should run quickly and deterministically; helpful in local development and ideal in CI, where computing resources might be limited.

Our tests should be proportionate to our functions in scope and size. Ideally, tests are fast and small, just like the services they're testing. (We're not building fat functions, right?)

For the sake of brevity, this discussion is limited to a Node.js runtime, but the principles are the same for other environments. Additionally, we won't worry about testing user interfaces or varying browser environments; those utilities are another post entirely.

So what's a good approach? Which libraries should be candidates?

A comparison

Several frameworks with performant runners help execute atomic tests, even concurrently. Some important considerations are library capabilities (like assertions), package size, maturity, and level of maintenance. Let's look at a collection of the most popular, up to date modules on npm today:

Library	Size	Concurrent	Version	Updated
Ava	281 kB	Yes	3.15.0	2021-11-01
Jasmine	47 kB	No	3.10.0	2021-10-13
@hapi/lab	160 kB	Yes	24.4.0	2021-11-09
Mocha	3.8 MB	Yes	9.1.3	2021-10-15
Node Tap	28.3 MB	Yes	15.1.5	2021-11-26
tape	248 kB	No¹	5.3.2	2021-11-16
uvu	46 kB	No	0.5.2	2021-10-08

achievable with tape-esque libraries like mixed-tape

A note about Jest

"But where’s Jest?" you ask. Don’t get me wrong, I understand the appeal of a framework with so many pleasantries. Jest’s feature-set is impressive and battle-tested. Unfortunately, tools like Jest, in order to accomplish so much, are opinionated. Jest uses implicit globals and its own context. It may not execute code the same way our servers will. This pattern can require all sorts of configuration bloat and transpilation, making debugging (especially in CI) tedious. In my view, Jest is not appropriate for what we're testing.

Unpacked module size

Emphasis on sizes > 1 MB in the above table is intentional.

Since we're running our tests in a cloud environment (in addition to locally), disk space matters.

Unfortunately, the library that most appeals to me, Node Tap, is just too large. At 28 MB, tap isn't very portable and will occupy a large part of allotted space in an environment like AWS Lambda. Hopefully, this limitation won't always be an issue, but it's an important factor for now.

A recommended testing “stack”

I think any of the above options are viable, depending on your use case and preference. For example, if BDD is preferable, jasmine has you covered. ava has excellent TypeScript support. uvu is super fast and works with ESM. And if you're looking for staying power, mocha has been around for nearly a decade!

For us at Begin and Architect, tape has been in use for several years. tape has a stable and straightforward API, routine maintenance updates, and outputs TAP, making it really versatile. While TAP is legible, it's not the most human-readable format. Fortunately, several TAP reporters can help display results for developers. Until recently, Begin's TAP reporter of choice was tap-spec. Sadly tap-spec wasn't kept up to date and npm began reporting vulnerabilities.

A new TAP reporter

Enter tap-arc. Heavily inspired by tap-spec (a passing suite's output is nearly identical), tap-arc is a minimal, streaming TAP reporter with useful expected vs. actual diffing. We're still improving the package, but it's definitely on par with tap-spec.

Feedback?

I'm super interested in what others are doing in this realm. How are you testing cloud functions? What factors are important when selecting test utilities? Do you test in the same environment you're deploying to?

Shopify Webhooks with Begin

Taylor Beseda — Tue, 30 Nov 2021 17:13:53 +0000

Photo by Tatiana Rodriguez

It’s always nice when data comes to you, and even better if your program can react in real-time. Shopify offers helpful webhooks for all sorts of store events. A developer just needs to set up a way to listen for those hooks and act on their data.

Shopify does offer support for AWS EventBridge and GCP Pub/Sub. Unfortunately, the process for both requires some web console spelunking, subscribing to webhook topics programmatically via the Shopify API (eventually, a good idea), and then setting up a mechanism to actually do work on the data.

Instead, let’s leverage the power and scale of AWS with the ease of Architect and Begin. Without managing AWS projects, IAM roles, or connecting EventBridge to a series of Lambdas.

Architect will help set up various HTTP routes (API Gateway endpoints backed by Lambda functions) to handle incoming webhook data. As an example, we’ll listen for customer updates and check if they're a VIP (10 or more lifetime orders).

Create a new Architect app with `npm init`

npm init @architect shopify-webhooks
cd shopify-webhooks

Architect has created the foundation for a new Arc project with a single HTTP endpoint. You can get a good view of what an Arc project is about by viewing the ./app.arc file.

Add an HTTP function called "customer-update"

Create a new entry in the app’s manifest. We'll use the "post" prefix to declare an endpoint that will respond to POST requests like webhooks from Shopify.

@app
shopify-webhooks

@http
get /
post /customer-update

Architect's CLI can help by creating some scaffolding. Just run arc create:

npx arc create
# ✓ Create Existing Architect project manifest found
# ✓ Create Created new project files in src/http/post-customer_update/
# ✓ Create Done!

Update the `customer-update` handler function

First, let's add the @architect/functions helper.

This library has some utilities to help parse request bodies, handle asynchronous code, and more. It's also available in Ruby and Python. Yep, this entire workflow can be done with alternative runtimes.

npm i @architect/functions

We'll add some very simple code to src/http/post-customer_update/index.js that will acknowledge a webhook request and decide what to log based on a customer's total order count.

const arc = require("@architect/functions");
function handler(request, response) {
  response({ status: 200 }); // respond immediately
  const customer = request.body;
  console.log(`${customer.id} was updated`);
  if (customer.orders_count > 9) {
    console.log(`${customer.id} is a VIP 🎉`);
  } else {
    console.log(`${customer.id} has ${customer.orders_count} orders`);
  }
}
exports.handler = arc.http(handler);

Test locally with Architect Sandbox

Architect has a Sandbox to help test our functions locally. Fire it up:

npx arc sandbox

Navigate to http://localhost:3333 to confirm the server is running and see get-index in action.

Next, we can test our new endpoint locally with a curl POST:

curl -X POST http://localhost:3333/customer-update \
   -H 'Content-Type: application/json' \
   -d '{"id": 706405506930370084,"email": "bob@biller.com","orders_count": 0}'

Checking the Sandbox logs, we can see the customer’s id was logged to the console.

706405506930370084 was updated
706405506930370084 has 0 orders

Try sending the curl command again but set “orders_count” to 10. This time the customer is logged as a VIP 🎉

706405506930370084 was updated
706405506930370084 is a VIP 🎉

Deploy to Begin.com

Let's get this to the cloud! Architect lets developers deploy directly to an existing AWS account if a local profile is set up.

Instead, we'll make it even easier and deploy it to Begin.com. First, push the Arc project to a new GitHub repository. Then head to Begin.com (and make an account if needed) to create a new app.

Select "Use an existing repo" and choose the GitHub repository we just created.

A staging build will kick off immediately. After that succeeds, go ahead and deploy to production. Once that's complete, you can click the production link to open the project's live version.

Configure Shopify with Begin’s new URL

Head over to the admin for the Shopify store you're working with. In store settings, under Notifications, add a new webhook. Set the topic to "Customer update" and use the production URL from Begin with the added /customer-update path.

Send a sample event from Shopify

After saving the webhook in Shopify, click "Send test notification" next to the new event. This will POST a test payload to the live project deployed on Begin.

From the Begin app dashboard, open the customer-update function logs from the HTTP view to check the production logs.

Just like that, we're catching Shopify webhooks in production with AWS Lambdas, no complex set up required.

Next steps

Okay, now what? Well, that's up to you and your app's needs. Here are some suggestions from my time in the Shopify ecosystem:

Verify the webhook signature.
Save data to a database; like Begin's built-in tables.
POST data back to Shopify's API, like adding a VIP customer tag.
Publish to an event queue (SNS) with the @events pragma.
Handle Shopify's mandatory webhooks.
Reduce repetition by sharing code between functions.

Inventory of Developer Discord Sidebars

Taylor Beseda — Sat, 04 Sep 2021 17:51:13 +0000

I've been doing some research on developer-centric Discord server setups. I had grabbed a snapshot of several servers I'm currently in, so I compiled them into a single image here in case it's useful to others.

By no means exhaustive, scientific, or in any particular order:

full res

Technical Writing Highlights from Large Orgs

Taylor Beseda — Sun, 01 Aug 2021 18:38:21 +0000

I've been reading through the style guides offered by larger tech companies. These summaries are a great place to start.

My takeaway: concise, conversational, active voice, you can use second person, and Oxford commas FTW.

CLI for Managing Shopify App Webhooks

Taylor Beseda — Sat, 10 Jul 2021 21:27:59 +0000

shopify-webhook-commander is a CLI to create, list, and delete Shopify Admin App webhook subscriptions.

Building out a new admin app often requires setting up listeners for Shop events but building out controllers and UI to manage webhook subscriptions isn't always at the top of the to-do list.

This CLI will let developers manage subscriptions from the terminal.

Install

🚫 This package is under active development; installer beware.

$ npm i -g shopify-webhook-commander

Note about auth tokens

This tool uses a single token to authenticate; this is easiest with a Shopify "Private" Admin App. Set the config secret to the private app password. There is no OAuth flow to help with "Custom" Partner apps.
You could provide an accessToken acquired from a Custom app's authentication flow.
shopify-webhook-commander doesn't really help for "Public" apps that typically manage webhooks for each shop install programatically.

Roadmap

tests 😬
replace Apollo client with a slimmer GQL lib
integrate @shopify/shopify-api types -- gotta get rid of those any types
webhooks:update command
autocomplete webhook topics
auth loop for custom apps

Deploying Nodewood

Taylor Beseda — Tue, 08 Jun 2021 03:54:02 +0000

[This was originally posted to a Github discussion on the Nodewood repository ; adding here for posterity.]

I recently deployed a vanilla instance of Nodewood, a JavaScript SaaS Starter Kit, to the cloud. I picked Render.com but this might help with other platforms (PaaS).

Things I set up before deploying:

A fresh Nodewood project running locally according to the Guide (great docs!)
My Nodewood project pushed to private Github repo (with changes outlined below)
Stripe account (activated, so I have production API keys)
Render account linked to my Github account
A Postgres db from Render (an addon for $7/mo)

This looks like a lot of work, but it's fairly quick-n-easy.

Project changes

`knexfile.js`

❗ Add knexfile.js to .gitignore so that it won't be a part of the production codebase. Knex configuration specific to production will be added to the Render service configuration (covered below).

Run > git rm knexfile.js if it has already been added to the repo.

`package.json`

Add some helpful commands to scripts for production use on Render:

+    "production:build": "yarn && yarn production:migrate && yarn production:stripe-sync && yarn production:build-ui",
+    "production:migrate": "knex migrate:latest --env production",
+    "production:stripe-sync": "npx @nodewood/cli stripe:sync --no-confirm",
+    "production:build-ui": "NODE_ENV=production yarn build-ui",
+    "production:start": "node app/api/api.js",

These are simple helpers that can be edited later if needed. Render will be configured to run these as a part of the build and deploy process.

Render Service Setup

Create a new "Web Service" linked to the (private) Github repo. $7/mo

Settings

❗ Change "Environment" option to "Node". Render will autodetect "Docker" but in this case it would add some unneeded overhead.

Build command: yarn production:build the custom script added to package.json that will run with each deploy

Start command: yarn production:start run the API

Advanced Settings

Environment variable:

key: NODE_ENV, value: 'production' -- just to be sure our code knows where it's at.

3 Secret files:

.env: copied from local project with updated DB_ variables for the Render database and updated production pk_ and sk_ keys from Stripe
.nodewood.js: copied exactly from local project 👍🏻
knexfile.js: with only a production object containing the Render database details. Looks similar to:

module.exports = {
  production: {
    client: "postgresql",
    connection: {
      host: "Hostname from Render database",
      database: "myapp",
      user: "myapp",
      password: "generatedpassword"
    },
    pool: {
      min: 2,
      max: 10
    },
    migrations: {
      tableName: "knex_migrations",
      stub: "migrations.stub",
      directory: ["./wood/migrations", "./app/migrations"]
    }
  }
};

Deploy 🚀

Commit and push your main git branch, Render will handle the rest.

Using your Nodewood App

Visit your Nodewood app (probably a .onrender.com URL) to see the static content (from ./www/dist/)
Sign up for an account
Edit the new user record in the database so that email_confirmed = TRUE and account_type = "admin". (I use Postico.app on my Mac to connect to the Render database to do this)
Visit the admin section of your Nodewood application in the browser!

This is as far as I've gotten so far, but from here I can just focus on my idea and not worry about deploying.

Notes/Ideas:

I didn't check if Knex could be configured from .env or similar -- this would eliminate the need for a production ./knexfile.js
No mailer was set up yet.
It's probably possible to deploy with Docker on Render, but that kind of defeats the purpose here 😄
I didn't check how many Postgres connections are allowed by Render's db service. So I may need to update the production Knex pool configuration