Forem: Jonathan Schneider

Boilerplate for complex hardware prototypes (RPi 5, 2 cameras, sensors, webapp)

Jonathan Schneider — Tue, 20 Aug 2024 16:57:41 +0000

You can't touch your raspberry pi 5, but you can touch your phone.
You can't hold 2 cameras, but you need control.
You know some part will break, but not sure which part.
Oh and maybe it'll float away and you'll need to catch it.

-- The ocean

If Use Cases similar to these are appearing in your project, I might have some code to get you started. Feel free to modify to your needs: Code on github (sbc-closed-system-controller)

I wrote this code because I had to throw a Raspberry Pi 5 in the waves while it was monitoring a prototype with 2 cameras and some sensors. Here's a video of the result:

Tech stack choice

A Python web backend (FastAPI) was chosen because the sensors I've used have good python support. The camera libraries for the raspberry pi 5 are also still in development and not yet as mature as the ones prior to the Pi5. However, only the Pi5 has two connectors for cameras over CSI/DSI, and synchronized recording from two cameras was important. Installing the libraries from the latest git branch can be done with this command: pip install git+https://github.com/raspberrypi/picamera2.git@next
A React WebApp that would poll individual sensors over REST was chosen to avoid having to refresh for updates. That is a drawback you see with some html templating engines. The Raspberry Pi would go underwater, and risked loosing the WiFi connection provided by the phone's hotspot. Having the frontend downloaded and only periodically updating its data allowed for each called endpoint to fail without the UI breaking down. It wasn't sure if the sensor and camera connection would survive when the prototype went underwater. This turned out to be easier for testing as well. Lastly, I'm used to React from my regular work as a frontend developer. The built frontend code can be served from FastAPI by using a symbolic link: ln -s ../../client-web-app/dist .
A service description to run the server at start of the Raspberry Pi. This actually took some searching before I found the right section, so here's the link

How the system behaves

The first behaviour when using 2 cameras on a Pi5 with a powerbank is... shutdowns. Most of them happen because of low voltage. The reason can be found in the system log: journalctl -S -10m --system.
failures are expected, so recordings are chopped into 5 second bits. The sensor readings go into a .csv-file, the raw .h264-files and their timestamps are saved separately. If everything had failed and got soaked in salt water, then the last barrier of water-proofing would have been the SD-card, which in theory should be waterproof. Luckily it didn't come to that. The combination of the files to bigger files is done with the python script convert_recordings.py, in the root of the repository.
The recording can be started and stopped from the web interface. The cameras stream a lowres-image to the web interface, both while recording and when idle. The latest sensor readings are summarized on the web interface as well.
There is a servo/PWM-controller board as well. This is used to control a pan-/tilt-head for one of the cameras, which uses 2 servos. The remaining channels are used for controlling the strength of LEDs to light the inner part.
It's possible to use a smart watch to display webapps on your local network. So far I only know how to do that with Samsung devices: Use Samsung internet to add the web address of the Pi to your favorites, while it's inside your hotspot's network. Samsung internet can be opened on the smartwatch as well, with limited functionality. For example the video display will have significant delay and the corners of the App are very hard to reach, so important buttons should be centered.

and that's about it

Conclusion

I hope this code helps someone get started with their own project. Hopefully you don't have to build something under water :)

Putting electronics in the water though is something I do not recommend, because waterproofing the components ended up taking the majority of the time I needed for this project. Extra care needs to be taken that the glues aren't water-solulable and other parts aren't harmful to the environment either.

In the end, the prototype showed that electricity can be generated in the surf on the shoreline. Recording the inside and outside of a wavebreaker produced some footage that makes this so much easier to explain.

So I hope you'll enjoy watching the above video. And let me know if this code has been helpful, I'd like to hear about similar use cases! :)

Underwater Voltmeter - high performance diagrams on Arduino

Jonathan Schneider — Mon, 03 Jul 2023 17:39:52 +0000

When you're developing for the web and mobile, you don't need to worry much about the performance of your display. This article covers the case when your device is offline, has limited resources and on top of it all... gets thrown into the water.

Video intro

It's a bit like coding in the early days of User Interfaces. This interface works like an oscilloscope that can measure the voltage of an analog input pin and display it as a waveform on the screen. But we need to treat data, time, and updates in other UIs as well. So I hope this article helps you with whatever you want to optimize.

Here's the link to the Arduino code

Challenges

One of the challenges of this project is to make the display as smooth and responsive as possible, without flickering or lagging. To achieve this, only updated, individual pixels are drawn in each loop, instead of clearing and redrawing the whole screen. This way, I can save a lot of time and resources, and make the display more efficient.

Calculating diagonal lines between data points would take time and resources. Especially because we would need to re-calculate them when we overdraw the previous pixels, so we're constrained to using pixels or horizontal bars.

It can cause some artifacts or glitches if the pixels are not updated correctly or in sync. So we need to keep track of time to make the display independent of the processor speed.

How the algorithm is faster

The algorithm does not redraw all of the pixels on each frame. Actually, it is pretty picky and benefits from the fact that we only show one value on the Y-Axis, and not multiple lines.

Here's how it works:

It reads the voltage from an analog pin and adds it to a buffer array.
It calculates and displays the max, average, and current voltage on the screen as text.
It draws a waveform on the screen that shows the voltage over time using iterative pixel drawing.
Iterative pixel drawing means that it only updates the pixels that change in each loop, instead of clearing and redrawing the whole screen.
To do this, it uses another array called yBuf that stores the height of each pixel on the waveform.
It shifts the values in yBuf to the left by a number of pixels that depends on the time difference between loops.
It loops through each pixel on the waveform and updates its color and height according to the values in yBuf and the buffer array.

The narrower the use case, the easier it is to improve performance

"Drawing the text actually competes for processor time with drawing the diagram, so this could be optimized"

But.... When we talk about "performance", these kinds of arguments are often brought up. Even if the diagram was quicker, would I actually use the 1 second diagram that often, or are the 5 seconds better? Is the accurate measurement or the moving average more important? Do I need a second value on the y-Axis, or is a single one just fine? All of these questions are more reasons to open-source the code, so your individual code can solve your individual problem. Are your problem and mine then similar enough to be solved with the same code?

These are the types of questions that developers of libraries, APIs and standards often face, and why custom solutions might have higher, "local" performance.

Why "local"? It's because it depends on the context, the resource limits at the time of writing the code, the device(s) available, what environment they'll be used in and the person who is implementing it. So it's not just about knowing the Tech, it's about knowing the User(s), the facilitators and people's Needs and Skills as well.

And someone who is doing this for many people... wow! My deep appreciation for all the API/library/standards-creators out there!

String-literal Union Types to Array - or - How to kick Pluto out of the list of planets

Jonathan Schneider — Fri, 07 Oct 2022 00:52:57 +0000

Pluto! Once a member of our own solar system's list of 9 planets, is now considered a dwarf planet. Who thought that list would change? And if Pluto is ever re-considered to be more than a dwarf, what happens to Planet Nine's name?

Similar to our solar system, we sometimes expect that some code or master data doesn't change. But when it does,

["Pluto"] is often hardcoded in arrays for dropdown-lists etc.
{"Pluto": {"discovered":1930, "namedAfter": "Pluto"}} can be found in keys or values of complex json objects
or it is a parameter of a function getLegendFromMythologyByDeity(f: "Pluto" | "Flora" | "Juno" | "Apollo" | "Minerva" | "Diana" | "Vulcan" | "Sol").

So to improve the situation, we can fill these types by hand. They're called string literal types, more on that topic here:

export type PlanetsType: "Mercury" | "Venus" | "Earth" | "Mars" | "Jupiter" | "Saturn" | "Uranus" 
  | "Neptune"; 
  //| "Pluto";
export type MythologicalDeityType: "Pluto" | "Flora" | "Juno" | "Apollo" | "Minerva" | "Diana" | "Vulcan" | "Sol";
export type MickeysFriendsType: "Pluto" | "Minnie" | "Donald";

If you additionally want an array that contains all possible permutations of a string literal union type, you could fill that by hand:

export const PlanetsTypeKeys: PlanetsType[] = [
  "Mercury", "Venus", "Earth", "Mars", "Jupiter", "Saturn", "Uranus"
  , "Neptune"];

...but you would have to remember to update that each time a new string literal type enters that union.

export type PlanetsType: "Mercury" | "Venus" | "Earth" | "Mars" | "Jupiter" | "Saturn" | "Uranus" 
  | "Neptune" 
  | "Planet9TheMisterious"; // <-- the new one, now the array has to be updated

A self-sustaining solution

Remembering to update the "right part" of the code is usually hard, even if you leave a comment. Throwing type errors is thus much better - and because we can't create an Array (runtime - javascript) from a union type (compile time - typescript), we'll use a helper, that'll finally bring us back to the master data.

/**
 * PlanetsType is a union type of string literals. This is useful for making switch-case statements exhaustive:
 * https://www.typescriptlang.org/docs/handbook/2/narrowing.html#exhaustiveness-checking
 * 
 * below however, we want to get an exhaustive list of all string literals in the union type. By building a helper
 * object with the key set to "s in PlanetsType", we effectively create an enum-like structure with all the
 * string literals of PlanetsType. When you create or change PlanetsType, this helps to keep the 
 * array in PlanetsTypeKeys up-to-date by throwing a typing error. The order of the entries is also preserved
 */
const PlanetsTypeHelperObj: { [s in PlanetsType ]: PlanetsType } = {
  "Mercury":"Mercury",
  "Venus":"Venus",
  "Earth":"Earth",
  "Mars":"Mars",
  "Jupiter":"Jupiter",
  "Saturn":"Saturn",
  "Uranus":"Uranus",
  "Neptune":"Neptune",
  //"Pluto":"Pluto",
  "Planet9TheMisterious":"Planet9TheMisterious",
}

export const PlanetsTypeKeys: PlanetsType[] = Object.keys(PlanetsTypeHelperObj) as PlanetsType[]

This turns PlanetsTypeKeys into an exhaustive Array of string literals in typescript. Because it's based on a json object (also called "dictionary" in this usage), there are no duplicates. Finally, the value-side of the json object ({"VenusKey":"VenusValue"}) can be anything you like. If you'd like to make the value-part your master data, the PlanetsTypeKeys will stay up to date, ready to be used ... with a dropdown for example.

The Hue-Light Mosquito Killer

Jonathan Schneider — Fri, 13 Aug 2021 14:29:36 +0000

Real-life problem: Mosquitos enter your bedroom at night, but electrical Mosquito killers consume electricity (and cartridges) all day. So let's put a smart plug in the middle and schedule the Moskito killers to run in the evening only.

If that's all you want, you can do it from an App. In the Hue App, you can create a zone for your mosquito killer and create an automation like this:

But we're programmers! So if you want finer control or create automations with devices from different IoT manufacturers, you can control individual plugs with node-red. It comes pre-installed on raspbian, or you can install it locally or in the cloud. I've used a Philips Hue smart plug because I've played around with Hue lights and node-red before. If you're using a different manufacturer, you can search the extensive library for a node.

Installing packages in node-red

Navigate to the URL of your node-red installation, which is http://localhost:1880 if you've installed it locally. Note that the port 1880 is most often also the port when you run it on a Raspberry Pi or NAS on your network. Head to "Manage palette" in the side-menu on the right.

Here you can see which nodes are available in your installation, in which package they come, or you can install new packages. The nodes for the Hue lights can be found in the node-red-contrib-huemagic package. When you click "install" the package will be downloaded and the nodes in it will be available shortly afterwards in the palette on the left of the main screen.

Connecting node-red with the Hue Bridge

Before we can use them, we need to configure the connection to the hue bridge, which is accessible from the "config" tab in the menu. A click on the "search"-icon will look on the local network for an available hue bridge, so you don't have to manually enter the IP. Once you have the IP, you can generate an API key by clicking on the "user" icon, and pressing the bridge's button within 20 seconds.

Adding and connecting nodes to send messages

From the panel on the left we drag and drop two inject nodes for testing the smart plug. Clicking the little rectangle to the left of the node will send a message into the flow, but for now the nodes are unconnected. Double-clicking the flow opens a property panel. The inject node which has msg.payload set to false switches the plug off, the node which has msg.payload set to true switches the plug on. The Name is just for yourself to tell the nodes apart.

Accessing a node's documentation

How do you know what msg.payload should be? That depends on the node that receives the messages. The smart plug can be controlled with the "Hue light" node (because switching on/off is the same behaviour for lights and for switches). To receive messages from the inject nodes, it needs to be connected. All messages flow left to right, so with the "Hue light" having one on the left and one on the right, it means that this node can also send messages. The current state "turned off" is displayed on the bottom. To find out what goes in and out of the "Hue light" node, we can look at the documentation, which you can find by clicking on the book icon.

Debugging a flow

From the documentation we can learn that the "Hue light" node has a "simple" and "extended" way of interpreting msg.payload. In the simple mode it takes a boolean (true/false, 1 or 0), in the "extended" mode a JSON object. The output always has the form of the "extended" JSON object, which you can try out by using a debug node, and checking the output in the "debug" tab. As a plus, if you have several debug nodes, hovering over a debug message will highlight the corresponding debug node in the flow.

The Mosquito Schedule mixed with our schedule

Now I don't know when exactly Mosquitos are active, but I roughly know mine (Sometimes I have the feeling they're active all night). It's not always regular and sometimes I'm not at home, so it would be great if the plug switched off then as well.

So as a basis a time-based schedule is necessary, and there are some options: You can install a scheduler-package that's configured inside the flows, or you could try out a dashboard scheduler. You can then still combine that with other nodes. With one of the schedulers and the HueMagic nodes alone we could set up a combined logic including a motion-, temperature- or brightness sensor. But with this article you should have the basics of setting up your very own Mosquito Killer Smart Home Defender™🤺🛡️ ... Let me know what you've built! 😊

While researching for this post I found...

This is also not the only way to protect yourself from mosquito bites. There are repellants and there's even
research about which type of bug is attracted by which type of light. So dimming your LEDs red apparently makes your home less attractive to blood-sucking mosquitos, but may attract other creatures of the night 😉.

Photo credits for part of the header go to:
Wolfgang Hasselmann

Asking for feedback: Visual editor + template for QR-Reader and -Generator Apps

Jonathan Schneider — Fri, 16 Jul 2021 00:02:59 +0000

Quick access or setup

Here's a live demo of the editor on the project homepage,
and of course the repository on github.
The repo also works as a template
for your own QR-enabled web apps. Everything open source.

Introducing: The MetaExplorer-starter template

I'm quite a fan of storybook, because you can communicate with your team mates about the features of frontend components. So everyone gets a clear idea of the UI component parts: But to talk about how two UI components look side by side, UX or measuring funnels I always wanted to have an editor for communicating more complex topics. Topics like how the application state changes, how good UX can be reused or if more copy moves the rest of the content "below the fold".

And so here's a simple App with a QR-reader and a QR-writer and some navigation. You can edit this App without code, and add your own UI components after reviewing them in storybook.

Feedback to make it more usable

Now it's mostly an editor for Application state, inspired by node-based editors such as in blender3D, node-red or camunda (BPMN).

Let me know your thoughts! Is it hard/easy to use, intuitive or not, are you against visual editors in general, what do you need in an editor? - and how long did it take you to crash it? 😉

Thanks! Wish everyone a productive time :) 📝

Devs go to stackoverflow, where do non-devs go?

Jonathan Schneider — Tue, 11 May 2021 10:06:48 +0000

We all know the process: Google, stackoverflow, copy&paste --> Time saved!
Recently I had to do a non-dev task that involved googling a lot, so I made a script to create the search-links in the browser console. That got me thinking that it might be useful for more people: So it's under permissive open source. Opening the console requires a bit of technical knowledge, so I also put together a little video of copying and running it.

So, how to use it is clear. Who would use it though?

Just listing it on github and youtube does NOT put it in front of people. "Built it and they will come" doesn't work for startups and it seems - neither for little stuff.

But where do you put your little scripts, so that they help someone?

Disclaimer: The links above will get me click-analytics, which I intend to share once I've learned something from the numbers.
Here are the full, non-tracked links: Script, Youtube-video

No code for nocode (1)

Jonathan Schneider — Fri, 05 Mar 2021 14:10:33 +0000

This post is part of a series:

Summary of the series

Nocode solves a business problem rather than a technical problem: It can be understood as a form of declarative programming
Instead of saving the declarative programs in a database, we could use git repos and our existing dev-toolset
We can build on top of existing data formats and create mappings between declarative and imperative programming
Different non-developers can get different shards of declarative data. That means: Custom nocode-tools for our team mates!
We'll look at a proof of concept with a popular tech stack. It shows that the concept works, what its possibilities and its boundaries are

The promise

"No coding needed" - does that mean thousands of devs will loose their jobs and dev.to can close their virtual doors?

Yes ;) All your fears will come true, the same way your past selves lost their jobs when you created that automation-script you're now a little proud of. But why was your automation-script so useful in the first place?

Did nobody like to do that task?
Your team mate often asked for A, B and C, and your script takes paramA, paramB, paramC as input?
Three different colleagues tried the task before, with three different results - and you had to combine their work?
It has run so often that you're thinking about version 2

Productivity

All of the above points don't have anything to do with code in the script. Code might have been important to build it, but the value your script creates for your team and organisation comes from something else. Your script touches the work of others, and makes their day a bit easier. This lets all of you work better together, but only if you're not on vacation. When your team mate sends you an email with A, B and C - your email-inbox is the current nocode-interface to that script.

The emails keep coming

Let's say word-of-mouth makes your script really popular, but now you get emails with B, C, D and E - and you've never heard of D and E, but the colleagues from the other department talk about it all the time. Your team mate has the Domain knowledge how D and E can be combined to form A. Wouldn't it be nice if your team mate could help improve the problems with your inbox filling up, and combine D and E?

Performance issues, outside computers

From a productivity perspective, once one bottleneck is fixed, another one needs fixing. What happened with your script, your colleagues and your team mate is the normal way an organization progresses. From a business perspective though, the power to progress is important. This power in the hands of more employees, that's what "Nocode" then sounds like.

Data about common problems

The problems that nocode-tools are currently trying to fix are mostly the ones that have existed across different industries. For example building web-interfaces. Centering elements in CSS? Just one click inside a nocode-tool.

Where does your data go in which format?

Now the nocode-tools may save element.centered = true somewhere in a database, somewhere in the cloud. Another one will save UIComponent.alignment = 'center' somewhere else, and both tools probably use a similar center-button. So there exists an unwritten industry-standard saying how a center-button should look like. However, there isn't an industry standard saying how or where to save that information.

Declaring what the program should do

"Alexa, center that element" is something we can't universally do because every tool implements the center-button-click in its own way. This is different when there is a commonly understood format for "centered-ness". How the button-click is turned into CSS or mobile elements is less important than what that button does.

Closed source and interfaces

With nocode-platforms, the declarative part of our program is hidden away in a database in a proprietary format. This is actually good in the Genesis-phase of tool-development, because changes happen often and code and interfaces stabilize slowly. The more mature a tool becomes, the more it has to stabilize though. For example a company working on a voice-plugin needs interfaces to "just work".

De-facto standards

At some point a tool reaches a point where it's a de-facto standard, where people say "Excel" instead of "spreadsheet". Excel's interfaces can't change that much any more, spreadsheet-tools are rather similar. This process is called commoditization. New tools based on spreadsheets can change a lot though, and a lot of nocode-tools actually make use of spreadsheets. How reliable can a proprietary de-facto standard be?

Remember Flash and Actionscript?

Flash offered something that was not possible on the web at the time. Nowadays, animations, video, audio and webcam-access seem normal, but then it was a substantial improvement. More importantly, in the visual editor it was quicker to design and animate than in code, and in the code-editor it was easier to create the behaviour.

What made it popular was the interface between designers and coders. What made it unpopular were security-issues, performance-issues and web-standards catching up. Although many were affected, there was only one company that could really fix Flash. When major parts were open sourced it was already too late, Flash was on the decline. Code-bases had to be rewritten rather than migrated because the custom tooling worked well with Flash, but not "the outside world". However, the source code for Flash-programs was saved in the file system. If a nocode-platform goes down it's up to the platform what happens with the nocode-programs it hosts.
Worst-case scenario: No more code.

We gladly have better open standards now, and the nocode-movement lets people solve more problems. If the nocode approach leads to many proprietary standards, that means integrators and customizers can get more business. Skillsets aren't as easily transferable between tools, which leads to rather closed communities around those standards.

Self-hosted vs cloud

The risk of a platform failing is lower with tools that can run both in the cloud, on-premise and locally. WordPress would be an example where anything "nocode" can be easily migrated. This is part of the design, thanks to established data formats. To migrate a mature data format would take quite an investment: The efforts for the Gutenberg editor from WordPress can give an indicator for that.

Open data formats

Open, mature data formats have a steeper learning curve, but it pays off to develop against established interfaces. Our IDEs are already one step closer to nocode-environments, especially if we count in extensions:

Frontend and backend devs interface with OpenAPI/graphql schemas
devs and devops interface with docker-compose files/kubernetes
devs and data/analytics interface with SQL queries
Pull requests get synced into JIRA for Project managers

So maybe the VS Code language server is a good starting point for the next low/nocode-tool. Asking

"What part of my IDE might be interesting for my colleagues?"

Also, when data formats are open and limited in their scope, it seems that dev-communities form around them. When the code fits into a stackoverflow-like answer, I guess it's easier to help each other.

To code or not to code, is this really a question?

All in all with nocode, we developers won't lose our jobs and Dev.to won't have to close their virtual doors either. There will be more problems to solve. Let's welcome new problem solvers, whether they're coders, hackers, software developers or the new citizen developers, and improve interfaces together.

schema.org-objects from GraphQL, typescript and a headless WordPress

Jonathan Schneider — Tue, 23 Feb 2021 17:30:14 +0000

Let's be lazy:

Here's a repo with a basic setup that maps from wpgraphql to json-linked-data in the schema.org vocabulary through a type-safe function. If that sounds complicated, here's the long explanation:

Intro to schema.org

To be more easily discoverably by search engines, WebSites can add structured data to their content. In this way google find's a site's own Ratings:

But it is also used to make the Web more interactive. Emails can include data about possible Actions, which GMail turned into this "View Pull Request" button here, so you don't have to open, read and click a link in the email:

The examples above are using a common, open vocabulary, schema.org, so other email-services could understand the structured data in github's email as well. More importantly, we can add data structured in json-ld and other formats to our own web pages, and for example make it easier for people to find an Event we're hosting close to them:

I don't know how many readers are hosting events, but I guess it would be easier to let Event hosts fill in the data dynamically, for example with WordPress.

Isn't there a plugin for it?

The short answer is Yes, many. This article is for typescript developers who work with GraphQL, we'll only use WordPress as a headless Content Management System (CMS). This article doesn't even include a single line of PHP, instead we'll generate typescript from a GraphQL schema. This approach actually works without WordPress, but it's easier to start with it.

From WordPress to GraphQL endpoint

The WPGraphQL Plugin is quickly installed from the admin panel of your WordPress installation.

You don't have to leave the Admin Panel to play around with queries either, you get a GraphQL IDE right inside WordPress:

That query retrieves all the posts in a given category, a short excerpt and a featured image - ideal to create a preview for a blog post.

From GraphQL endpoint to GraphQL schema

Now this is security relevant: When we have a graphql API, it can tell a client about its structure with a so-called introspection query. You can generate a graphql schema with such a query, if you don't already have access to the graphql schema. An attacker could do the same and try to find vulnerabilities in your API, that's why it's better to have it switched off in production. WPGraphQL has this switched off by default, you need to go into the settings to switch it on:

While it's switched on, we can use standard functionality from the graphql package and node.js to generate the schema and save it locally.

import fs from "fs";
import fetch from "node-fetch";
import {
    getIntrospectionQuery,
    printSchema,
    buildClientSchema,
} from "graphql";

/**
 * runs an introspection query on an endpoint and retrieves its result
 * thanks to this gist:
 * https://gist.github.com/craigbeck/b90915d49fda19d5b2b17ead14dcd6da
 */
async function main() {
    const introspectionQuery = getIntrospectionQuery();
    const response = await fetch("https://blog.example.com/graphql", {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
        },
        body: JSON.stringify({ query: introspectionQuery }),
    });
    const { data } = await response.json();
    const schema = buildClientSchema(data);
    const outputFile = "./wpgraphql-schema.gql";
    fs.writeFileSync(outputFile, printSchema(schema));
}

main();

Replace blog.example.com with your domain or localhost, whereever your graphql endpoint can be found. To be able to run that script, you need to set "type": "module" in your package.json (because it's in ES6 syntax).
And once you have the wpgraphql-schema.gql file, please, switch off introspection.

From GraphQL schema to typescript

With the wpgraphql-schema.gql file we can already create typescript types. In order for that to work we'll add graphql-generator to our project:

yarn add @graphql-codegen/cli @graphql-codegen/typescript @graphql-codegen/typescript-operations -D

To run the generator, it needs a little bit of configuration, this is done in the codegen.yml-file:

overwrite: true
schema: "./wpgraphql-schema.gql"
documents: "src/**/*.graphql"
generates:
  src/generated/wp-graphql.ts:
    plugins:
      - "typescript"
      - "typescript-operations"

The first line tells it to overwrite existing .ts-files, the second one points to the graphql schema we just got from the endpoint, and the third is the glob-pattern or file where our queries, mutations, fragments and subscriptions can be found. If we don't have any "documents", we can still run the generator with:

yarn graphql-codegen --config codegen.yml

and get types for our graphql schema. So getting all graphql schema-types is configured with the plugins: - "typescript" part. Getting queries etc is configured with the - "typescript-operations" part. That adds types for all "documents" it finds to our .ts-file. You could also generate separate .ts-files, this is up to you (to split server-features from client-features like queries and fragments, for example).

We could already use the wpgraphql Post-type to create a mapper-function to schema.org/BlogPosting. However, that type is pretty large and we might not always query all fields for a given blog post.
As you can see, we don't need to write documentation on the types from wpgraphql, it gets the documentation from the graphql schema.

To simplify our queries, we can create a graphql fragment like so:

fragment PostPreview on Post {
  title
  excerpt
  featuredImage {
    node {
      sourceUrl
      altText
      description(format: RAW)
      srcSet(size: THUMBNAIL)
    }
  }
  slug
}

and now any time we want to query these exact same fields we can just reference it in a query:

query wpPostPreviewByCategory {
  posts(where: { categoryName: "my-side-projects" }) {
    edges {
      node {
        ...PostPreview
      }
    }
  }
}

That query gets us the posts from a particular category, but only the fields from the fragment for each post. This is great for reusability in graphql-queries, and graphql-codegen generates the type PostPreviewFragment for us, so we can create a mapper-function that consistently works across queries.

We can now begin with the mapper-function:

import { PostPreviewFragment } from "./../generated/wp-graphql";
//other imports

export function mapWpPostPreviewToSchemaBlogPost(
    input: PostPreviewFragment,
    wpBaseURL?: string
): BlogPosting {
    // ...curious about wpBaseURL and BlogPosting? We'll use that to build our json-ld
}

From typescript to schema.org

Here's a good example of what a Blogposting can look like with json-ld and other formats that mix with HTML. The benefit of adding a script-tag with json-ld is that we can generate our website - the un_structured data - in another way than our publicly available _structured data. This gives you more flexibility, especially since it's allowed to add several script-tags with type="application/ld+json" to web pages.

Of course, for the human readers of our website we would want something visual. For our blog post we might display a link, an image, the post's title and a short preview. The machines reading our website don't know which is what. To help them read it properly as a BlogPosting, the equivalent json-ld would look like this:

<script type="application/ld+json">
{
  "@context":"https://schema.org",
  "@type": "BlogPosting",
  "@id": "https://shnyder.com/business-model-canvas-for-metaexplorer",
  "name": "Business Model Canvas for MetaExplorer",
  "abstract": "<p>Before starting out with my plans to turn metaexplorer into a business, I wrote a business plan &#8211; and because 2020 showed us what it thinks of plans&#8230;</p>\n",
  "image": {
    "@type": "ImageObject",
    "name": "Metaexplorer business model canvas, 2020-12-28",
    "contentUrl": "https://shnyder.com/wp-content/uploads/2020/12/IMG_20201228_234552-scaled.jpg",
    "thumbnailUrl": "https://shnyder.com/wp-content/uploads/2020/12/IMG_20201228_234552-150x150.jpg"
  }
}
</script>

As you can see in the example's "@id"-field and image urls, it includes a domain name - my blog's domain. Relative URLs are also possible in json-ld, so on shnyder.com I could use this:
"@id": "/business-model-canvas-for-metaexplorer"
Here on dev.to I would want to include the full domain name. To let our mapping-function include my domain, we add the optional parameter wpBaseURL.

So how do we get the json-ld from the graphql types? The library schema-dts helps us do exactly that, and in a typesafe way. That means we get intellisense on the actual schema.org vocabulary (ontology, even).

You can install it with:

yarn add schema-dts

A naive approach would be to map only the "happy path" from graphql to json-ld, where we always have the data that we want and need. That approach would look like this:

import { PostPreviewFragment } from "./../generated/wp-graphql";
import { BlogPosting } from "schema-dts";

/**
 * mapper-function to create schema.org/BlogPosting(s) for previews from fragments
 * @param input a fragment of a WordPress blog post
 * @param wpBaseURL the base domain of your wordpress installation. Used to add the slug
 */
export function mapWpPostPreviewToSchemaBlogPost(
    input: PostPreviewFragment,
    wpBaseURL?: string
): BlogPosting {
    let featuredImgNode = input.featuredImage.node;
    let thumbnailUrl = featuredImgNode.srcSet
        .split(" ")
        .find((val) => val.startsWith("http://"));
    let output: BlogPosting = {
        "@type": "BlogPosting",
        ...() => {wpBaseURL ? {"@id":`${wpBaseURL}/${input.slug}`} : undefined},
        name: input.title,
        abstract: input.excerpt,
        image: {
            "@type": "ImageObject",
            description: featuredImgNode.description,
            name: featuredImgNode.altText,
            contentUrl: featuredImgNode.sourceUrl,
            thumbnailUrl
        }
    };
    return output;
}

And this does compile in a standard typescript configuration. The thing is that we can't always be sure to get a preview image, and other fields might be empty as well. Handling empty fields will prevent errors in the future. So for the mapper function, we need to set compilerOptions.strict to true in our tsconfig.json. If we try to compile again, we'll receive lots of errors such as:

- error TS2533: Object is possibly 'null' or 'undefined'.
and
Type 'null' is not assignable to type 'string | PronounceableTextLeaf | readonly PronounceableText[] | undefined'.

Since we don't want something like "title": undefined in our json-ld, we have to remove the fields from the output-object altogether. The first thought to fix this might be to add lots of if-statements. But with the recent advances in ES6 and typescript, we have the option to create the output in a way that looks very much like a nested json-ld-object. Ant such a nested object is what we want to output. The main difference is that this nested object includes conditions for safe mapping as well:

/**
 * mapper-function to create schema.org/BlogPosting(s) for previews from fragments
 * @param input a fragment of a WordPress blog post
 * @param wpBaseURL the base domain of your wordpress installation. Used to add the slug
 */
export function mapWpPostPreviewToSchemaBlogPost(
    input?: PostPreviewFragment,
    wpBaseURL?: string
): BlogPosting | null {
    if (!input) return null;
    let featuredImgNode = input && input.featuredImage && input.featuredImage.node;
    let thumbnailUrl = featuredImgNode && featuredImgNode.srcSet && featuredImgNode
        .srcSet.split(" ")
        .find((val) => val.startsWith("http"));
    let output: BlogPosting = {
        "@type": "BlogPosting",
        ...(wpBaseURL && { "@id": `${wpBaseURL}/${input.slug}` }),
        ...(input.title && { name: input.title }),
        ...(input && input.excerpt && { abstract: input.excerpt }),
        ...(featuredImgNode && {
            image: {
                "@type": "ImageObject",
                ...(featuredImgNode.description && {
                    description: featuredImgNode.description,
                }),
                ...(featuredImgNode.altText && {
                    name: featuredImgNode.altText,
                }),
                ...(featuredImgNode.sourceUrl && {
                    contentUrl: featuredImgNode.sourceUrl,
                }),
                ...(thumbnailUrl && { thumbnailUrl }),
            },
        }),
    };
    return output;
}

23 lines without null-checks, compared to 32 lines with strict type checks. Plus, this notation scales into nested structures. Not much extra effort - but the reliability has improved greatly. Let's look at it the code detail. First, the output is an Object typed "BlogPosting", and it has to include a "@type": "BlogPosting". Typescript types are lost during compilation, whereas "@type" gets baked into json:

let output: BlogPosting = { "@type": "BlogPosting" };

The three dots that appear across the function are spread operators. Those operators are trying to spread the result of the evaluation in brackets ().

...(input.title && { name: input.title }),
```


Inside the brackets there's a logical operator, which returns the second part [if the first one is truthy](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Logical_AND#description), otherwise the first one. That means if one of our input fields like `input.title` is undefined, null or an empty string, this is what the spread operator gets to see:


```javascript
let mytestvar = {...undefined};
console.log(mytestvar);
mytestvar = {...null};
console.log(mytestvar);
mytestvar = {...""};
console.log(mytestvar);
mytestvar = {...false};
console.log(mytestvar);
mytestvar = {...true};
console.log(mytestvar);
//always an object with no fields
```


This way of converting is helpful to secure our string-based values, but it's not a magic syntax for API conversion. 

However, if we do have an `input.title`, the second expression `{ name: input.title }` is an object, and its key(s) and value(s) are added to the containing object.

## Conclusion
Now you have a pipeline to build structured data from your graphql API, independent of which frontend technology you choose. Structured data can have a good effect on SEO, some of it is displayed as special widgets in google. With `schema-dts` you don't only avoid typos while you build structured data, your IDE will help you find the right fields for your `@type`s. 

If you're thinking about where to put those conversions in your architecture, maybe ideas like the [middleman engine](https://css-tricks.com/reconciling-editor-experience-and-developer-experience-in-the-cms/#3-the-middleman-engine) can even help improve the handling of data in your organization as a whole.

Monorepos: Lerna, TypeScript, CRA and Storybook combined

Jonathan Schneider — Thu, 31 Oct 2019 16:16:10 +0000

Let’s be lazy:

repository on github
That’s the code for the starter repository.
Also made this repository a template repository

This post details why, how to prevent errors and how to do it yourself. It is useful if you want to set up a monorepo for an existing codebase, or if you run into errors when extending your monorepo.

Updated to use react-scripts v4.0.2!

With this update, the template contains:

the latest React@^17.0.1 and storybook
some example stories and components in the UI library part
those components can use css and scss, and CSS gets built into the output folder, along with type definitions
modifying the UI library triggers a storybook hot reload, building the UI library triggers a CRA hot reload

So, for the not-so-lazy:

If you've been using ReactJS in more than one project or are building multiple Apps, you've probably come across lerna already. Since setting up webpack can be tricky, the choice is usually to use create-React-app as long as possible. So we're going to look at how this works with a centralised TypeScript config that we'll also use for our ui component library, which we'll put in a separate repository. We’ll use yarn since we’ll make use of yarn workspaces as well.

yarn init

a private package as the root of our monorepo. Why private? Because private packages don’t get published to npm, our root is only there for organizing everything, and lastly defining yarn workspaces only works in a private package.

Introducing: Lerna

First of all, you’ll need to install lerna, and while you can do that globally, I recommend installing it in your monorepo unless you (and the contributors to your monorepo) want to author lots of monorepos with lerna and it’s part of your standard toolset.

yarn add lerna -D

Now we have lerna, which gives us organization tools for monorepos. For example initialization:

yarn lerna init

This will create a lerna.json file and a packages folder. Since we’ll use yarn workspaces, we need to define yarn as our npmClient and set useWorkspaces to true. Our lerna.json will end up looking like this:

{
  "packages": [
    "packages/*"
  ],
  "version": "0.0.0",
  "npmClient": "yarn",
  "useWorkspaces": true
}

And that is all the configuration we need for lerna.

Since we’re using yarn workspaces, we need to modify our package.json, by adding:

"workspaces": [
    "packages/*"
  ],

Note: your packages-folder doesn’t need to have that name. You could also have your ui-lib, apps and server code in different subfolders. For using workspaces and lerna together, you should however define them in both lerna.json and package.json.

Project Setup: UI component library package

Initializing sub-packages in monorepos is pretty similar to normal repos, with one thing to note when setting the name. You just change into the directory:

cd packages && mkdir my-monorepo-ui-lib && cd my-monorepo-ui-lib

And initialize a package:

yarn init

But with the name @my-org/my-monorepo-ui-lib. This is using a feature called npm organization scope and requires you to set up an organization with npmjs.com if you want to publish as the @my-org organization.

This is not mandatory, but it shows a source for bugs when we’re developing monorepos:

The package name isn’t always the same as the directory name
Configuration files and script parameters sometimes need a package name, sometimes a directory name
You can use this syntax even if you never intend to publish

Quick and dirty package installation

We want to build reusable react components in our UI library, but later our create-react-app package will determine which version of react we will use. That’s why react and react-dom can only be a peerDependency in the UI library. Storybook is our way to quickly try out our react components, so we’ll add that as a devDependency:

yarn add react react-dom -P
yarn add @storybook/react babel-loader -D

This is how we’ve always been doing it, right? Turns out, now there’s a node_modules folder in our ui-lib package, with react, react-dom and @storybook inside. But we wanted to have our packages at the root, so lerna will help us do that from the root package:

cd ../..
yarn lerna bootstrap

Now there’s a node_modules folder at the root, containing react, react-dom and @storybook. The node_modules folder inside our ui-lib package is still there, it contains a .bin-folder with storybook’s command line (bash/cmd) scripts for starting and building. All tools executing command line scripts such as storybook, tsc and create-react-app are not necessarily aware that they’re run in a monorepo, they execute commands on the operating system and are usually built for “normal” npm repos.

Troubleshooting bash and cmd scripts: storybook, tsc, react-scripts

Inside ui-lib, if we try to run

yarn start-storybook

it will execute the script but tell us that we have no storybook configuration file yet:

Create a storybook config file in "./.storybook/config.{ext}

We get the same error if we add it as a script in ui-lib’s package.json (naturally):

  "scripts": {
    "story": "start-storybook" 
  },

Let’s fix that error by creating the file packages/my-monorepo-ui-lib/.storybook/config.js

import { configure } from '@storybook/react'

const req = require.context('../src', true, /\.story\.(ts|tsx)$/)

configure(() => {
  req.keys().forEach(filename => req(filename))
}, module);

and packages/my-monorepo-ui-lib/src folder, that can be empty for now. Inside our ui-lib, running

yarn start-storybook

and

yarn story

works fine now, although it’s empty.

The difference becomes clear once we go to the root and try to run command line scripts from there:

cd ../..
yarn start-storybook

and we have the same error as before. The reason is that the node_modules-folder at the root also contains the command line script, and tries to look for a storybook config relative to the root package. Lerna will help us here as well, at the root we can call

yarn lerna run story --stream

That command will run ‘story’ relative to all packages in parallel, and ‘stream’ the script output to the console. This only works for so-called ‘lifecycle scripts’, i.e. scripts defined in one of the sub-packages' package.json, so the following command will not work:

yarn lerna run start-storybook

This is also the reason you’ll see scripts defined such as

"tsc": "tsc",

but it’s generally better to choose a different name to avoid confusion, especially because a lot of people install tsc and other tools globally.

Project Setup: CRA App

Take caution when using CRA for new packages in combination with yarn workspaces:

cd packages
create-react-app my-monorepo-cra-app

This will throw an error, since CRA copies files out of the node_modules folder where it’s installed in (here: packages/my-monorepo-cra-app/node_modules), while yarn workspaces make sure everything gets installed in the root-node_modules-folder. So in the root package.json delete

  "workspaces": [
    "packages/*"
  ],

and add it back in after you’ve run CRA. Then in the root folder run

yarn lerna bootstrap

and your dependencies will neatly be moved to the root-node_modules. Running

yarn lerna run start --stream

will start your CRA-App, the JavasScript version of it.

Adding Typescript

Monorepos can help centralize configuration, so we’ll create a general tsconfig.json at the root of our monorepo. It would be great if we could use that in every subproject, but CRA needs to make some assumptions about its TypeScript setup, so it adds/overwrites the values inside tsconfig. That’s also good news, since it doesn’t just overwrite the file - and we can extend from another tsconfig. In our library project on the other hand we are more free, we can change the webpack there if we have to.

How to structure your typescript-configurations

This decision depends on how many packages and what types of typescript-packages you want in your monorepo:

One CRA App, one UI library: Go for
- one tsconfig.json at the root with cosmetic settings like removeComments; settings that don’t conflict with CRA and which aren’t library-specific, like library export
- one extending from that, autogenerated in your CRA package
- Lastly one for your library that sets “outDir”:”lib” and configures declaration export. This needs to correspond with the settings in the lib’s package.json:

  "main": "./lib/index.js",
  "types": "./lib/index.d.ts",

Many CRA Apps: Same structure as the one above. The reason is, that right now using CRA means that you’ll have to recompile your library to make changes in your CRA App. When running react-scripts start though, the node_modules-folder is also being watched, so you can run tsc in your library in watch mode after starting CRA
Many libraries: Create an additional tsconfig.lib.json at the root, where you generalize your export settings. If one of your libraries depends on another one of your libraries, have a look at typescripts path-mapping and project references features

Apart from typescript, create-react-app supports css, scss and json-imports out of the box with just a little bit of configuration. We’ll add a typings.d.ts-file at the root for those types, so those file types are importable by default:

declare module "*.json" {
  const value: any;
  export default value;
}
declare module '*.scss' {
  const content: any;
  export default content;
}
declare module '*.css' {
  interface IClassNames {
    [className: string]: string
  }
  const classNames: IClassNames;
  export = classNames;
}

This is the minimal tsconfig.json we could work with:

{
  "exclude": ["node_modules"],
  "files":  ["./typings.d.ts"],
  "compilerOptions": {
    "jsx": "react",
    "esModuleInterop": true,
    "skipLibCheck": true
  }
}

We want to use typescript in all our packages, which is done by the lerna add command:

yarn lerna add typescript -D

We include skipLibCheck as well, because we want tsc to run fast.

UI-library with storybook and typescript

When structuring our UI library, it’s good to follow a consistent pattern. The goal is to just run ‘tsc’ and have working Javascript, no webpack needed if we can avoid it by clear structure.

It’s especially important to:

Separate concerns by usage (utils in one folder, React components in another)
Prevent cyclic imports/exports (utils exported before react components - if you use factories don’t put them in utils, export them after react components)
Make it easy for the next person to extend the library (group your react component with its story and its unit test)

So your folder structure may end up looking like this:

Any file named index.ts is either a leaf in the file tree and exports unit-tested code or is a branch and exports its subfolders. Unit-tests and stories are not exported and their files can be excluded from the compiled code via configuration. Here’s an example of what the files may look like:

However, we do need webpack for one thing: Storybook’s configuration for typescript. And since we’re at it, we can add support for scss and some file types as well.

cd packages/my-monorepo-ui-lib
yarn add @babel/core @types/storybook__react awesome-typescript-loader babel-loader node-sass sass-loader source-map-loader style-loader -D

Bootstrapping is not needed because we’re using yarn workspaces, and our packages can be found at the root’s node_modules folder.

Directly adding it inside the package is a workaround for an error in lerna add in combination with organization scopes:

lerna WARN No packages found where @babel/core can be added

The cleaner option would be to use lerna add with the --scope parameter, however this has been incompatible with how we set the organisation scope. The command would be:

yarn lerna add @babel/core @types/storybook__react awesome-typescript-loader babel-loader node-sass sass-loader source-map-loader style-loader --scope=@my-org/my-monorepo-ui-lib -D

Are you wondering, what the --scope-parameter is all about?
Here, --scope is the installation scope parameter, @my-org the npmjs-organization scope. So all those packages will be added to our UI library package.

Our UI lib’s webpack config is comparatively short:

const path = require('path');
module.exports = {
  module: {
    rules: [{
        test: /\.scss$/,
        loaders: ["style-loader", "css-loader", "sass-loader"],
        include: path.resolve(__dirname, '../')
      },
      {
        test: /\.css/,
        loaders: ["style-loader", "css-loader"],
        include: path.resolve(__dirname, '../')
      },
      {
        enforce: 'pre',
        test: /\.js$/,
        loader: "source-map-loader",
        exclude: [
          /node_modules\//
        ]
      },
      {
        test: /\.tsx?$/,
        include: path.resolve(__dirname, '../src'),
        loader: 'awesome-typescript-loader',
      },
      {
        test: /\.(woff|woff2|eot|ttf|otf|svg)$/,
        loader: "file-loader"
      }
    ]
  },
  resolve: {
    extensions: [".tsx", ".ts", ".js"]
  }
};

And we could use a minimal tsconfig.json that just extends from our root tsconfig.json, and puts the output in the lib-folder:

{
  "include": [
    "src"
  ],
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "outDir": "lib",
    "declaration": true
  }
}

This allows us to compile typescript-files and run storybook, but we want to do more! (to do less later on...)

For our library project, we need to emit declaration files (the files ending in *.d.ts). Otherwise we’ll receive errors such as:

Could not find a declaration file for module '@my-org/my-monorepo-ui-lib'. '.../lerna-typescript-cra-uilib-starter/packages/my-monorepo-ui-lib/lib/index.js' implicitly has an 'any' type.
my-monorepo-cra-app: Try npm install @types/my-org__my-monorepo-ui-lib if it exists or add a new declaration (.d.ts) file containing declare module '@my-org/my-monorepo-ui-lib'; TS7016

For clarification: Webpack isn’t used in our build-process, tsc is. The Webpack we’re configuring is used by storybook.

Typescript with CRA

The limits of centralizing our typescript configuration is determined by create-react-app’s use of typescript. At the time of writing this article, switching a CRA App from Javascript to Typescript is done by changing the index.js file to index.tsx and adding all the needed dependencies. Check CRA’s documentation for changes: https://create-react-app.dev/docs/adding-typescript

Inside our CRA-package, we run

yarn add typescript @types/node @types/react @types/react-dom @types/jest -D

then we copy our minimal tsconfig.json from the ui-lib over to the CRA App package. If we run

yarn start

Now, CRA’s compilerOptions will be added to our tsconfig.json.

Loading a component from our UI library

Now it’s time to load our UI library into our CRA App, it will be installed by running:

yarn lerna add @my-org/my-monorepo-ui-lib

But as you might have noticed, we haven’t done much build setup for the library yet. Why didn’t we do that earlier? The reason is pretty simple: CRA, lerna and Storybook are evolving, and so is typescript, npm and even Javascript. And with ES6 modules, we have a powerful new feature built into the language, replacing earlier module management solutions. The only problem is that it’s not 100% adopted, but as we want to be a good library provider, we offer a fallback. So let’s export our library to ES6 modules - and an “older” module management system. Otherwise we’ll run into errors such as:

Unexpected token “export”

If you want to deep-dive into that topic, this blog about nodejs modules and npm is a good start.

Npm as our package management solution has also been around since before ES6 and typescript’s rise, so we can set different entry points for our library project inside package.json:

“main” is the oldest one, it’ll point to our pre-ES6 export (“./lib/index.js”)
“types” is the place where our type declarations can be found ("./lib/index.d.ts")
“module” is the entrypoint for our ES6 modules ("./lib-esm/index.js")

Our project is written in typescript from the start, so we’re bundling the declarations with our package. If you’ve seen yourself importing @types-packages, this is because those projects are written in Javascript at the core, and type definitions have been added later on.

So we set a tsconfig.esm.json up to export as an ES6 module:

{
  "include": [
    "src"
  ],
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "lib-esm",
    "module": "esnext",
    "target": "esnext",
    "moduleResolution": "node",
    "lib": ["dom", "esnext"],
    "declaration": false
  }
}

This does the following:

Our modules will go into the lib-esm-folder, which we specified as our module-entrypoint in package.json.
Our module resolution strategy is “node”. If we don’t set it we’ll get an error such as:

src/index.ts:1:15 - error TS2307: Cannot find module './utils'.
1 export * from './utils';

Setting "esnext" targets latest supported ES proposed features: That means “features to be developed and eventually included in the standard”

This way, our library has one export for the latest Javascript features and one that is downwards compatible, so our library can have a bigger range of consumers. Note that for our own final App, CRA uses babel under the hood for compatibility in different browsers.

We’re already emitting our declarations in the lib-folder, so we won’t emit them another time here.

Finally, we’ll add a library-build-script in our library package.json:

"libbuild": "tsc && tsc --build tsconfig.esm.json"

And we’re ready to add our library package to our CRA package. We can set a wildcard for the package version so that it’s always going to be the latest version.

"dependencies": {
    "@my-org/my-monorepo-ui-lib": "*",

In our CRA App we can now add the component from the library, fully type-checked:

in a react render function"/>

And because monorepos should make our lifes easier, we’ll add scripts in our root-package.json to start storybook, and execute the library build before starting our CRA app:

  "scripts": {
    "story": "lerna run story --stream",
    "prestart": "lerna run libbuild --stream",
    "start": "lerna run start --stream"
  }

This will hopefully prevent the most common errors you can run into with this monorepo-setup. If you have additional tips, feel free to add them in the comments!