Forem: Andrew Evans

Building Microfrontends with Firebase

Andrew Evans — Tue, 19 Nov 2024 12:00:50 +0000

The post you are reading is cross posted from my blog site. Please see my original post at andrewevans.dev.

Recently I've been working with Microfrontends and wanted to put together a post that covers some basic concepts and includes an example project. I'm also a Firebase fan and wanted to include how you could build a Microfrontend with Firebase. I'm using React in this project, but if you wanted to pick a different Frontend Library or Framework the same basic principles should apply. Overall this post will cover what a Microfrontend is, and showcase an example project that shows you a working project. If you'd like to follow along or just jump straight into the code, checkout my GitHub repo firebase-mfe.

What is a Microfrontend?

Microfrontends (also called MFEs for short) have been around for several years now, but the core principles and pattern have remained the same. MFEs are a way to break down a large application into smaller pieces that can be deployed independently. The business value of MFEs, is the ability to break a large application into smaller teams that can be developed and released faster.

Consider as an example a web application that is maintained by 50 developers. Consider this application to actually consist of multiple pages that are independent parts (things like a shopping cart page, an info page, etc.). Anytime developers make changes, this could impact any and all parts of the same shared application. This is typically called a monolith whereby everything operates in the same codebase and any change impacts the entire application.

Conversely, consider taking that same application with the 50 developers and multiple pages and breaking it into pieces. Imagine having specific teams focus on specific parts of the application without worrying about potentially breaking or stopping another teams progress. The example of 50 developers could conversely be considered as a project with 5 teams of 10 developers. Each team focusing on a specific area (or page) of the Frontend application. This is the real value of this pattern in that it can stop many of the pain points of managing large systems.

To visually show this, one also has to point out that MFEs work with a host and remotes. The host manages the application as a while and consumes remote applications. The remotes expose components that are then consumed by the host. Modifying the earlier visualization to be shown in terms of MFEs:

Each of the remotes should be able to be independently developed and deployed. The host can also manage things like tokens or shared settings that can be consumed by the remotes. The remotes can take in values from the host as props within the components. This is really powerful because it means that in a corporate environment, a team could manage things like app settings and authentication and then the remote teams could all use the same values. This means each team doesn't have to go through the process of setting up their own resources.

At its core, MFEs also take advantage of Module Federation and Webpack. Module Federation has the following key concepts:

Sharing components (or modules) between pieces of an application through exposing or consuming (versus building everything at once)
Sharing dependencies within an application (not having to copy the same code in multiple areas)
On demand loading of dependencies (only load what you need)
Deploying and developing parts of an application independently

With Module Federation the host will consume remotes and the remotes will expose components to be consumed by the host.

If you do some Googling on this topic, you'll find that many people have different takes on the way that this is implemented. There are a variety of ways to manage an architecture like this. The overarching concepts are still the same. In the next sections, I'll share my example application firebase-mfe. and show how it accomplishes the MFE Architecture with Firebase.

Building an MFE with Firebase

Before you begin, you need to go to the Firebase console and create an application. If you're new to Firebase, I recommend checking out their fundamentals page. Firebase for the most part is free to initially get setup. Certain services have costs over time, but for the purposes of this project you shouldn't incur any cost.

Once you have the project setup, make sure to go and add an application which will result in configuration values like these:

const firebaseConfig = {
  // Add your Firebase config here
  apiKey: "<API_KEY>",
  authDomain: "<AUTH_DOMAIN>",
  projectId: "<PROJECT_ID>",
  storageBucket: "<STORAGE_BUCKET>",
  messagingSenderId: "<MESSAGING_SENDER_ID>",
  appId: "<APP_ID>",
};

With these values setup, you should also go over to the Authentication Service and add a test user. My application uses an Auth value that is passed into the remotes, so we need a test user to be able to validate everything.

With all of this setup, you can now move over to the application to get setup.

Building the Host

For my sample application I used React, but you could just as easily use a different library or framework of your choice. The most important part is just that it has to be able to use Webpack so you can make use of Module Federation.

Since I have a full working project to share, I'm just going to highlight the places that need to be configured for everything to work. The general setup of this sample project could be expanded to a much larger project if one wanted to. One other point is that my sample project is all in the same repository, but you could potentially have your projects in separate repos entirely if you want. One of the best parts about MFEs is that you have the flexibility to customize the setup based on a variety of project needs.

In my sample application I have three projects:

host
remote1
remote2

The host is where I store the Firebase configuration and you can see this in host/src/firebase.ts:

import { initializeApp } from "firebase/app";
import { getAuth } from "firebase/auth";

const firebaseConfig = {
  // Add your Firebase config here
  apiKey: "<API_KEY>",
  authDomain: "<AUTH_DOMAIN>",
  projectId: "<PROJECT_ID>",
  storageBucket: "<STORAGE_BUCKET>",
  messagingSenderId: "<MESSAGING_SENDER_ID>",
  appId: "<APP_ID>",
};

export const app = initializeApp(firebaseConfig);
export const auth = getAuth(app);

Next, you'll notice in the sample application I have a bootstrap.tsx file next to my index.tsx file. This file is read in by index.tsx to render my application. The remotes will also have a bootstrap.tsx file, but I will cover that after I finish this portion.

// src/bootstrap.tsx
import React from "react";
import { createRoot } from "react-dom/client";
import App from "./App";

const root = document.getElementById("root");
if (root) {
  createRoot(root).render(<App />);
}

Now if you go over to the App.tsx file, you'll note that you have 2 imports at the top that include 2 remotes:

const Remote1 = React.lazy(() => import("remote1/App"));
const Remote2 = React.lazy(() => import("remote2/App"));

You'll also notice that the remotes are defined in the "view" of the App.tsx component:

<ErrorBoundary>
  <div>
    {!authState.user ? (
      <LoginForm onLogin={handleLogin} />
    ) : (
      <div>
        <h1>Welcome {authState.user.email}</h1>
        <React.Suspense fallback="Loading Remote 1...">
          <Remote1 user={authState.user} />
        </React.Suspense>
        <React.Suspense fallback="Loading Remote 2...">
          <Remote2 user={authState.user} />
        </React.Suspense>
        <button onClick={async () => await signOut(auth)}>Logout</button>
      </div>
    )}
  </div>
</ErrorBoundary>

Also note that the user object is passed in as a prop to both remotes. We will need to account for that in the next section where I explain the remote1 and remote2 projects.

If we jump over to the host/config/webpack.config.js file you will note at the bottom how this is setup with the Module Federation plugin:

config.plugins.push(
  new ModuleFederationPlugin({
    name: "host",
    remotes: {
      remote1: isEnvDevelopment
        ? "remote1@http://localhost:3001/remoteEntry.js"
        : "remote1@https://remote1-11182024.firebaseapp.com/remoteEntry.js",
      remote2: isEnvDevelopment
        ? "remote2@http://localhost:3002/remoteEntry.js"
        : "remote2@https://remote2-11182024.firebaseapp.com/remoteEntry.js",
    },
    shared: {
      react: {
        singleton: true,
        requiredVersion: require("../package.json").dependencies.react,
        eager: false, // Add this line
      },
      "react-dom": {
        singleton: true,
        requiredVersion: require("../package.json").dependencies["react-dom"],
        eager: false, // Add this line
      },
      firebase: {
        singleton: true,
        requiredVersion: require("../package.json").dependencies.firebase,
        eager: false, // Add this line
      },
    },
  }),
);

Lets note a few things here:

the remotes block has a definition for remote projects with the prefix that is the name remote1 and remote2. You'll note that this is the same as the imports in the App.tsx file.
Both of the remotes are defined based on if the isEnvDevelopment flag is set. If in development, load from localhost otherwise point to the deployed sites.
Both of the remotes make note of a remoteEntry.js file, this is what is compiled and read in by the host.
Note also that in the shared section it specifies versions of react, react-dom, and firebase based on what is in the remotes.

Please note that the React apps shown here have been "ejected" as part of a create-react-app project. There are many ways one could have done this, but this was the easiest way I found to get setup with a sample project like this.

With the above setups, you'll also need to run firebase init and go through the terminal to get the project setup as a "firebase" project.

You'll then have to either go into the Firebase console, or run commands to generate "sites" within your project that you can deploy to. Here are the commands I ran:

firebase hosting:sites:create host-11182024
firebase target:apply hosting host-11182024 <PROJECT_ID>-host-11182024

Note that i named my host host-11182024 just as a value I could remember. You could name your host (and remotes) anything you want.

The names I made for the host project were arbitrary. I also had to run these commands a few times before I figured out the correct setting here. Ultimately I ended up with a firebase.json file that looked like this:

{
  "hosting": {
    "target": "host-11182024",
    "public": "build",
    "ignore": ["firebase.json", "**/.*", "**/node_modules/**"],
    "rewrites": [
      {
        "source": "**",
        "destination": "/index.html"
      }
    ]
  }
}

I also ended up with a .firebaserc file that looked like this:

{
  "projects": {
    "default": "<YOUR_FIREBASE_PROJECT_ID>"
  },
  "targets": {
    "<YOUR_FIREBASE_PROJECT_ID>": {
      "hosting": {
        "host-11182024": ["host-11182024"]
      }
    }
  },
  "etags": {}
}

Ultimately, this is all just setting it up so when you deploy you are sending the bundle to a specific place in your project. With my sample project I used host-11182024 which ended up me using the deploy command firebase deploy --only hosting:host-11182024. We will next do similar setups for the Remote projects.

Building the Remotes

The remote projects in the sample project are in the remote1 and remote2 folders. I'll just go through remote1 and you can follow the same process for remote2.

In remote1, look first at the bootstrap.tsx file:

import React from "react";
import { createRoot } from "react-dom/client";
import App from "./App";

const root = document.getElementById("root");
if (root) {
  // Don't render anything in standalone mode for production
  if (process.env.NODE_ENV === "development") {
    import("./devBootstrap");
  } else {
    createRoot(root).render(
      <div>
        This is a microfrontend that needs to be loaded from a host application.
      </div>,
    );
  }
}

Note that we are accounting for development mode. The devBootstrap.tsx file includes a mock user. This is because we are passing in the user object into the remote from the host. Having a mock value allows you to run the remote independently of the host for debugging etc. This is a great example of how you could pass things like tokens or other values from host to remote. Just for reference and to continue with the explanation this is what the devBootstrap.tsx file looks like:

import React from 'react'
import { createRoot } from 'react-dom/client'
import App from './App'
import { User } from 'firebase/auth'

const mockUser: User = {
    email: 'test@example.com',
    uid: 'test-uid',
    emailVerified: true,
    isAnonymous: false,
    metadata: {},
    providerData: [],
    refreshToken: '',
    tenantId: null,
    delete: async () => {},
    getIdToken: async () => '',
    getIdTokenResult: async () => ({
        token: '',
        authTime: '',
        issuedAtTime: '',
        expirationTime: '',
        signInProvider: null,
        claims: {},
        signInSecondFactor: null,
    }),
    reload: async () => {},
    toJSON: () => ({}),
    displayName: 'test user',
    phoneNumber: '123-123-1234',
    photoURL: '',
    providerId: '',
}

const root = document.getElementById('root')
if (root) {
    createRoot(root).render(<App user={mockUser} />)
}

Within the remote1 project we can next take a look at the host/config/webpack.config.js file where we define the Module Federation plugin:

new ModuleFederationPlugin({
  name: "remote1",
  filename: "remoteEntry.js",
  exposes: {
    "./App": "./src/App",
  },
  shared: {
    react: {
      singleton: true,
      requiredVersion: require("../package.json").dependencies.react,
    },
    "react-dom": {
      singleton: true,
      requiredVersion: require("../package.json").dependencies["react-dom"],
    },
    firebase: {
      singleton: true,
      requiredVersion: require("../package.json").dependencies.firebase,
    },
  },
});

Note that the name remote1 lines up with the name of the value in the host webpack config. Also note the exposes field points to the projects App.tsx file where the main application is loaded. The additional definitions in the shared section are similar to how we defined the values in the same place in the host config file.

Similar to how I setup the Firebase "site" for the host project, we can do the same here with similar commands:

firebase hosting:sites:create remote1-11182024
firebase target:apply hosting remote1-11182024 <PROJECT_ID>-remote1-11182024

Then when we want to deploy, we call the Firebase command firebase deploy --only hosting:remote1-11182024.

Going through the firebase init setup, for the remote1 project we get the following firebase.json values:

{
  "hosting": {
    "target": "remote1-11182024",
    "public": "build",
    "ignore": ["firebase.json", "**/.*", "**/node_modules/**"],
    "rewrites": [
      {
        "source": "**",
        "destination": "/index.html"
      }
    ]
  }
}

Similarly, we also get the following .firebaserc file values:

{
  "projects": {
    "default": "<YOUR_FIREBASE_PROJECT_ID>"
  },
  "targets": {
    "<YOUR_FIREBASE_PROJECT_ID>": {
      "hosting": {
        "remote1-11182024": ["remote1-11182024"]
      }
    }
  },
  "etags": {}
}

If you look at the remote2 project, you'll se basically the same setup. The only difference was that the name of the remote is remote2 and the Firebase "site" that I created was remote2-11182024.

Seeing everything in action

If you want to see my sample project working locally, you can just run each project in its own terminal session. Each project runs on a different port, and if you pull up the page for the host at port 3000 you'll see the app running and pulling in the remotes:

I created a sample account at hello@gmail.com just so I could login and verify the values were actually being passed in correctly.

Note that the host has props for the user that is logged in. These props are passed into the remotes without the projects needing to have the auth setup configured. This is just one example of what could be exchanged between the host and remote projects. Being able to centralize configuration like this, is one of the most powerful parts of MFE projects.

If you open up Chrome DevTools, you'll see that what you are seeing appears to be a regular React application:

If you open the network tab, however, you'll see that what was pulled in is from different endpoints:

When you look at the individual pages running locally, you'll see the devBootstrap in action as the value for the user is mocked:

Lessons Learned

So far I've discussed the general concepts behind MFEs, and also shown you how you can technically build your own. My general experience with MFEs are that the setup process tends to require some experimentation to get right. Fortunately, there are a fair amount of YouTube videos and blog posts that cover MFE concepts. I also was able to leverage Claude AI for help getting started and working through parts of the implementation. I recommend using all of these resources (and Claude) to help you with your initial setup.

Getting the values exposed by the Module Federation plugin can cause some issues. On a different MFE project I worked on, I had the incorrect name for the remoteEntry.js file which we didn't see until everything was deployed and the remote would not load.

Additionally, I ran into some an error about eager consumption:

Error: Shared module is not available for eager consumption: webpack/sharing/consume/default/react/react

This is why in the Module Federation part of my host webpack config you'll see eager: false like this:

    shared: {
      react: {
        singleton: true,
        requiredVersion: require("../package.json").dependencies.react,
        eager: false, // Add this line
      },
    }

It's also important to be able to properly handle errors when loading your components. If you notice in the host project, I have wrapped my "view" of the App.tsx values with an ErrorBoundary:

<ErrorBoundary>
  <div>
    {!authState.user ? (
      <LoginForm onLogin={handleLogin} />
    ) : (
      <div>
        <h1>Welcome {authState.user.email}</h1>
        <React.Suspense fallback="Loading Remote 1...">
          <Remote1 user={authState.user} />
        </React.Suspense>
        <React.Suspense fallback="Loading Remote 2...">
          <Remote2 user={authState.user} />
        </React.Suspense>
        <button onClick={async () => await signOut(auth)}>Logout</button>
      </div>
    )}
  </div>
</ErrorBoundary>

The ErrorBoundary component, is an implementation of React's Error Boundary and can be seen in the host/src/components/ErrorBoundary.tsx file:

import React, { Suspense } from 'react'

// Error Boundary Component
class ErrorBoundary extends React.Component<
    { children: React.ReactNode },
    { hasError: boolean; error: Error | null }
> {
    constructor(props: { children: React.ReactNode }) {
        super(props)
        this.state = { hasError: false, error: null }
    }

    static getDerivedStateFromError(error: Error) {
        return { hasError: true, error }
    }

    render() {
        if (this.state.hasError) {
            const styles = {
                container: {
                    padding: '1rem',
                    backgroundColor: '#fef2f2',
                    border: '1px solid #fecaca',
                    borderRadius: '8px',
                },
                heading: {
                    color: '#dc2626',
                    fontWeight: 600,
                    marginBottom: '0.5rem',
                },
                message: {
                    color: '#ef4444',
                },
                button: {
                    marginTop: '0.75rem',
                    padding: '0.5rem 1rem',
                    backgroundColor: '#fee2e2',
                    color: '#dc2626',
                    border: 'none',
                    borderRadius: '4px',
                    cursor: 'pointer',
                },
            }

            return (
                <div style={styles.container}>
                    <h2 style={styles.heading}>Failed to Load Component</h2>
                    <p style={styles.message}>{this.state.error?.message}</p>
                    <button
                        style={styles.button}
                        onClick={() => window.location.reload()}
                    >
                        Retry
                    </button>
                </div>
            )
        }

        return this.props.children
    }
}

export default ErrorBoundary

Wrapping Up

Overall, I hope this post demonstrated to you how MFE's work and also showcased how you can use Firebase to build one. MFEs are a powerful concept that teams can use to manage larger projects. MFEs also have a lot of flexibility, and you can fine tune your project to match a variety of needs.

In my Firebase implementation, I also hope you noticed how easy it was to get going with Firebase. I'm a long time Firebase fan, and loved how easy it was to just get setup with some basic configuration. I highly recommend Firebase for projects especially like the sample one I created for this post.

I recommend going through my sample project as well as looking at other posts and YouTube videos on how MFEs work. Jack Herrington has some great videos on YouTube like this one that MFEs with some higher level concepts and examples. Webpack also has documentation on the Module Federation plugin that you can check out.

I hope my post helped you better understand MFEs and how you can use Firebase to build them. Thanks for reading my post!

Intro to Azure Search

Andrew Evans — Mon, 17 Jun 2024 13:12:19 +0000

The post you are reading is cross posted from my blog site. Please see my original post at andrewevans.dev.

Azure AI Search (formerly Azure Cognitive Search) enables teams to improve the process of searching and analyzing data. Azure AI Search has a wide array of capabilities. I have been using Azure Search with an application I maintain, and wanted to put together a post that covers how Azure Search works and some of its features at a high level. The following post will walkthrough setting up Azure AI Search on a dataset, and how you can start utilizing Azure AI Search for your projects.

How does Search work?

Before I go into how Azure Search works, it helps to understand how you would approach implementing a full text search.

Most commerce websites have a "full text search" capability, and are a good example. When you "search" with these websites, applications take in text input and attempt to match that input to data in a data source like a NoSQL database or other storage mechanism. This is done through parsing data, and attempting to find a match based on operators like "==" or "!=" etc.

With any search service, you have (1) data and then (2) how you query the data. Consider using Amazon.com for example. Amazon.com has a lot of data between product listings, reviews, and more. When trying to search Amazon.com your search would include products but not necessarily reviews or other information. If Amazon wanted to expand their search to include the reviews (or more), they could modify their implementation to take in reviews and attempt to search based on both their list of products and product reviews. Further, when actually conducting a search in Amazon.com, what you input will find similar values (also known as "fuzzy search"). You could also be given the opportunity to do things like filtering, where you pick specific brands or results within a price point. The combination of these options act as query parameters to refine how the search is conducted.

It's also important to understand "fuzzy" searching, as that comes up a lot in search discussions. This is whereby an application searches not just on exact matches, but also similar matches within the parameters provided. For example, if you searched "soccer balls" this would return results that should include the text "soccer balls." However, if you searched "soccer balls2" without a fuzzy search you would get back results with "soccer balls2" but not with "soccer balls." If you ran the same query with "soccer balls2" in a fuzzy search, you would get results that match on "soccer balls" and "soccer balls2" since the terms are similar.

One final thing to note when generally speaking about "search" is optimizations. The reason services like Azure Search exist is because directly searching data with mechanisms like SQL queries are not always optimal. If you're using SQL in particular, there are issues with things like resource use or needing enhancements like database indexes. As data sources get larger, scaling also becomes a challenge. Services like Azure Search exist to provide optimizations for faster performance, scaling, and much more.

How does Azure AI Search work?

Azure Search implements the fundamentals of search with additional features that help teams optimize their search experience. Azure Search has also recently integrated with Azure AI Services, Azure OpenAI, and machine learning to provide additional intelligence in search results.

Azure Search starts with a data source, and then uses indexes (which you define) to "crawl" the data. Crawling here meaning building a data set from that original data source that is optimized and provides (potentially) faster queries with further customizations that teams can leverage in their applications. To update the dataset, you create an indexer which queries and updates the data periodically. Ultimately, you get a subset of your data that is customized based on how you want your users to interact with it. This is different than just a straight retrieval of all of your data, and can have major performance gains when doing queries.

The indexes setup the ability to interact with data at the field level. Each field you define in an index has different attributes for what you want to do with it. For example, if you wanted users to be able to filter a query on a specific field you would mark that field as "filterable." There are several options available. Copying the section from the field attributes in the Azure Search documentation:

Attribute	Description
"searchable"	Full-text or vector searchable. Text fields are subject to lexical analysis such as word-breaking during indexing. If you set a searchable field to a value like "sunny day", internally it's split into the individual tokens "sunny" and "day".
"filterable"	Referenced in $filter queries. Filterable fields of type Edm.String or Collection(Edm.String) don't undergo word-breaking, so comparisons are for exact matches only. For example, if you set such a field f to "sunny day", $filter=f eq 'sunny' finds no matches, but $filter=f eq 'sunny day' will.
"sortable"	By default the system sorts results by score, but you can configure sort based on fields in the documents. Fields of type Collection(Edm.String) can't be "sortable".
"facetable"	Typically used in a presentation of search results that includes a hit count by category (for example, hotels in a specific city). This option can't be used with fields of type Edm.GeographyPoint. Fields of type Edm.String that are filterable, "sortable", or "facetable" can be at most 32 kilobytes in length.
"key"	Unique identifier for documents within the index. Exactly one field must be chosen as the key field and it must be of type Edm.String.
"retrievable"	Determines whether the field can be returned in a search result. This is useful when you want to use a field (such as profit margin) as a filter, sorting, or scoring mechanism, but don't want the field to be visible to the end user. This attribute must be true for key fields.

In the next section I will go through an example setting up an index with fields.

An Example of Azure Search

It helps to view an example. I'll take some data from a previous post I wrote about python scripting with Cosmos DB that includes a set of Electric Vehicles (EV). In that post I loaded a free dataset into a Cosmos Collection that I'll use in the following example.

In order to setup the actual Azure Search instance, the index, and the indexer there are a few ways. I recommend following the Microsoft Tutorial to get started.

In the case of my example, I first made my Azure Search instance point to my Cosmos DB collection and do the following query:

Select c.id, c.VIN, c.State, c.ModelYear, c.Make, c.Model, c._ts From c WHERE c._ts > @HighWaterMark ORDER BY c._ts

The query I'm using here is very simple because the data is not very complex. What you select with this query will be become the dataset that runs your Azure Search instance. There are many more advanced things you could do with this query like using Cosmos DB's CONCAT statement to create fields that are actually combinations of values. The setup here is very flexible and can work with many scenarios.

With my query all setup, next I created an index on the data that looks like the following:

As you can see in my index, I have different properties for the different fields. Some examples include:

I can search on VIN but not sort or filter
I can filter, search, and sort on State, Make, and Model
I can only retrieve id
I can only filter and search based on ModelYear

With the index in place, I also defined an indexer that I just run manually. I could alternatively have defined this to run on a schedule.

Note I called my datasource "Toddy" as a fun reference to my son. I think it made the datasource better.

Azure provides information about the runs of your indexer as well with a nice graph:

After running the index one time, I can now do queries using the search explorer. I can do an example query like the following:

{
  "search": "*"
}

In that case I just passed the wildcard character * to select everything. If I wanted to do a more refined search, I could pass operators based on the different types of fields.

Here's a search for all Tesla vehicles that returns their State, ModelYear, and Model:

{
  "search": "Tesla",
  "select": "State, ModelYear, Model"
}

If I wanted to filter the results and only get ModelYear values greater than 2021, I could do the following:

{
  "search": "Tesla",
  "select": "State, ModelYear, Model",
  "filter": "ModelYear gt 2021"
}

We should also look at the data that is being returned from these queries. When I did the search with the wildcard character * you'll note that the data returned looked like similar to the following:

{
  "@search.score": 1,
  "id": "d7c852e0-0e6d-46e4-913c-f5dcc553c680",
  "VIN": "1FMCU0EZXN",
  "State": "WA",
  "ModelYear": 2022,
  "Make": "FORD",
  "Model": "ESCAPE"
}

The fields that were retrieved are all from the Cosmos DB collection, but the field @search.score is a measure of how that result compared to the others retrieved. There are more advanced ways to look into this value and I recommend checking out the Microsoft Article on how scoring works.

Azure Faceted Search

A more advanced concept with Search is to use Search Faceted Navigation. Facets allow you to control (server side) navigation that you would present to a Frontend that uses Azure Search.

In my example of a list of EV cars, I created a second index that included some fields that use Facets:

With that index in place, I then am able to run a query where I specify one of the Faceted values:

If you notice in the response there is a "facets" value that includes the Faceted field and a total. This provides data that could be used by a Frontend to show totals for specific values within the query result.

To provide a more refined search, I can then specify a filter to use with the facets:

There is a lot more that you can do with Facets. I recommend you check out the Microsoft Documentation.

Calling Azure Search from an API

Azure has SDKs that are available for interacting with the Search Service. The Azure Search Nuget Package provides the integrations necessary for you to use Azure Search with your C# applications.

Once you have created your project and included the Azure Search Nuget Package, you can call the service directly in your projects. Just to play around with this locally, I modified the Azure quickstart for .NET to suit the vehicles collection I had been working with.

I first created a class to represent the vehicles:

using System;
using System.Text.Json.Serialization;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;

namespace AzureSearch.Quickstart
{
    public partial class Vehicle
    {
        [SimpleField(IsKey = true, IsFilterable = true)]
        public string? Id { get; set; }

        [SearchableField(IsSortable = true)]
        public string? VIN { get; set; }

        [SearchableField(AnalyzerName = LexicalAnalyzerName.Values.EnLucene)]
        public string State { get; set; }

        public int ModelYear { get; set; }

        [SearchableField(IsFilterable = true, IsSortable = true)]
        public string? Make { get; set; }

        [SearchableField(IsFilterable = true, IsSortable = true)]
        public string Model { get; set; }
    }
}

I then modified the program's Main method to setup the package with the proper configuration:

using System;
using System.Text.Json;
using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;
using Azure.Search.Documents.Models;

namespace AzureSearch.Quickstart

{
    class Program
    {
        static void Main(string[] args)
        {
            // setup the configuration based on values from Azure
            string serviceName = "<AZURE_SEARCH_INSTANCE_NAME>";
            string apiKey = "<AZURE_SEARCH_API_KEY>";
            string indexName = "<AZURE_SEARCH_INDEX_NAME>";

            // create the actual client
            Uri serviceEndpoint = new Uri($"https://{serviceName}.search.windows.net/");
            AzureKeyCredential credential = new AzureKeyCredential(apiKey);
            SearchIndexClient adminClient = new SearchIndexClient(serviceEndpoint, credential);
            SearchClient srchclient = new SearchClient(serviceEndpoint, indexName, credential);

            // call the queries wrapped in a method
            Console.WriteLine("Starting queries...\n");
            RunQueries(srchclient);

            // End the program
            Console.WriteLine("{0}", "Complete. Press any key to end this program...\n");
            Console.ReadKey();
        }
    }
}

The example program writes values out to the console. I modified it to only show the values that were not null since the queries retrieved different sets of values:

// Write search results to console
private static void WriteDocuments(SearchResults<Vehicle> searchResults)
{
    foreach (SearchResult<Vehicle> result in searchResults.GetResults())
    {
        Vehicle vehicle = result.Document;
        Type type = vehicle.GetType();
        System.Reflection.PropertyInfo[] properties = type.GetProperties();

        string output = "{";
        foreach (System.Reflection.PropertyInfo property in properties)
        {
            object value = property.GetValue(vehicle);
            if(value != null){
                output += $"\"{property.Name}\":\"{value}\",";
            }
        }
        output += "}";
        Console.WriteLine(output);
    }

    Console.WriteLine();
}

With the client created and the output method setup, I then modified the queries in the RunQueries method similar to how we did in the search explorer.

// here is the first query we did with selecting all the values
SearchOptions options;
SearchResults<Vehicle> response;

// specify options like query size
options = new SearchOptions()
{
    IncludeTotalCount = true,
    Filter = "",
    OrderBy = { "" },
    Size = 10
};

options.Select.Add("VIN");
options.Select.Add("State");
options.Select.Add("ModelYear");
options.Select.Add("Make");
options.Select.Add("Model");

// passing in a wildcard character like we did in the search explorer
// also note I had to pass in a class based implementation of the vehicle values
response = srchclient.Search<Vehicle>("*", options);

The above will result in the following console output:

{"VIN":"1FMCU0EZXN","State":"WA","ModelYear":"2022","Make":"FORD","Model":"ESCAPE",}
{"VIN":"1G1FW6S03J","State":"WA","ModelYear":"2018","Make":"CHEVROLET","Model":"BOLT EV",}
{"VIN":"1FADP5CU8F","State":"WA","ModelYear":"2015","Make":"FORD","Model":"C-MAX",}
{"VIN":"WB523CF03P","State":"WA","ModelYear":"2023","Make":"BMW","Model":"IX",}
{"VIN":"1G1RC6S54J","State":"WA","ModelYear":"2018","Make":"CHEVROLET","Model":"VOLT",}
{"VIN":"WBY1Z4C56G","State":"WA","ModelYear":"2016","Make":"BMW","Model":"I3",}
{"VIN":"1G1RD6E44D","State":"WA","ModelYear":"2013","Make":"CHEVROLET","Model":"VOLT",}
{"VIN":"1G1FW6S00J","State":"WA","ModelYear":"2018","Make":"CHEVROLET","Model":"BOLT EV",}
{"VIN":"KNDCC3LG2L","State":"WA","ModelYear":"2020","Make":"KIA","Model":"NIRO",}
{"VIN":"WAUTPBFF3H","State":"WA","ModelYear":"2017","Make":"AUDI","Model":"A3",}

Here is the second query I did looking for Tesla vehicles:

options = new SearchOptions()
{
    IncludeTotalCount = true,
    Filter = "",
    OrderBy = { "" },
    Size = 10
};

options.Select.Add("State");
options.Select.Add("ModelYear");
options.Select.Add("Model");

response = srchclient.Search<Vehicle>("Tesla", options);

Writing the values out to the console, resulted in the following values:

{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2019","Model":"MODEL S",}
{"State":"WA","ModelYear":"2022","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2021","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2019","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2022","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2019","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}

Another example query I had was Tesla cars older than 2021.

options = new SearchOptions()
{
    Filter= "ModelYear gt 2021",
    Size = 10
};

options.Select.Add("State");
options.Select.Add("ModelYear");
options.Select.Add("Model");

response = srchclient.Search<Vehicle>("Tesla", options);

Writing this out to the console resulted in output similar to the following:

{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2022","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2022","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2023","Model":"MODEL X",}
{"State":"WA","ModelYear":"2022","Model":"MODEL Y",}
{"State":"WA","ModelYear":"2022","Model":"MODEL 3",}
{"State":"WA","ModelYear":"2023","Model":"MODEL Y",}

I also modified the configuration value to use my second index with Facets:

// setup options
options = new SearchOptions()
{
    Filter = "",
    Size = 10,
    IncludeTotalCount = true
};

// add Facet value of Make
options.Facets.Add("Make");

// specify fields to select
options.Select.Add("Make");
options.Select.Add("ModelYear");
options.Select.Add("Model");
options.Select.Add("VIN");

// do the actual search and write to the console
response = srchclient.Search<Vehicle>("*", options);
WriteDocuments(response);

For the console output I also had to modify the way it parsed the facets in the response with the following:

if(searchResults.Facets != null && searchResults.Facets.Count > 0){
    Console.WriteLine("FACETS");
    var makeFacets = searchResults.Facets["Make"];

    foreach(var facets in makeFacets)
    {
        Console.WriteLine($"  Value: {facets.Value}, Count: {facets.Count}");
    }
    Console.WriteLine("");
}

With everything setup for Facets, the resulting console output was the following values:

FACETS
  Value: TESLA, Count: 3293
  Value: NISSAN, Count: 941
  Value: CHEVROLET, Count: 752
  Value: BMW, Count: 453
  Value: FORD, Count: 399
  Value: KIA, Count: 384
  Value: TOYOTA, Count: 345
  Value: JEEP, Count: 302
  Value: VOLVO, Count: 272
  Value: VOLKSWAGEN, Count: 251

{"VIN":"1FMCU0EZXN","ModelYear":"2022","Make":"FORD","Model":"ESCAPE",}
{"VIN":"1G1FW6S03J","ModelYear":"2018","Make":"CHEVROLET","Model":"BOLT EV",}
{"VIN":"1FADP5CU8F","ModelYear":"2015","Make":"FORD","Model":"C-MAX",}
{"VIN":"WB523CF03P","ModelYear":"2023","Make":"BMW","Model":"IX",}
{"VIN":"1G1RC6S54J","ModelYear":"2018","Make":"CHEVROLET","Model":"VOLT",}
{"VIN":"WBY1Z4C56G","ModelYear":"2016","Make":"BMW","Model":"I3",}
{"VIN":"1G1RD6E44D","ModelYear":"2013","Make":"CHEVROLET","Model":"VOLT",}
{"VIN":"1G1FW6S00J","ModelYear":"2018","Make":"CHEVROLET","Model":"BOLT EV",}
{"VIN":"KNDCC3LG2L","ModelYear":"2020","Make":"KIA","Model":"NIRO",}
{"VIN":"WAUTPBFF3H","ModelYear":"2017","Make":"AUDI","Model":"A3",}

There are many more things you could do here, but I kept these queries necessarily simple just to showcase how this works.

Wrapping Up

In this post I hope you learned some details about how Azure Search works. I introduced search as a concept, and then went into detail about how search works with Azure. I covered an example that created an Azure Search instance and ran sample queries with the search explorer. I also included some examples of how you could use the Azure Search Nuget Package to query Azure Search in a C# .NET application. I only covered a high level of how to get started, and definitely recommend reviewing the various Microsoft tutorials and documentation that work with search. Other features that are worth noting are Autocomplete with Azure Search as well as several other advanced search options you can find in the official Azure Search documentation. Azure Search has a lot of potential to provide good solutions to searching needs. Thanks for reading my post!

How to use Cypress for E2E with React

Andrew Evans — Mon, 15 Mar 2021 09:38:53 +0000

The cover image was originally copied from https://pixabay.com/photos/lamp-light-lighting-light-bulb-3489395/

With any Frontend application, End to End (e2e) testing can really improve maintenance and the Software Development Life Cycle (SDLC) of the application. E2E allows you to quickly verify changes, and also works to document features in your application.

There are a few options for E2E frameworks today. Cypress is one of the newer and more exciting options for building E2E tests in your application.

Cypress is a great option for E2E because it:

runs in the same event loop as your frontend project (rather than externally "knocking on the door" like E2E Selenium based projects do)
Works with any Frontend project (install Cypress as a dependency and the test runner will automatically detect the corresponding configuration files)
All of the E2E is written in JavaScript (no need to have .feature or other associated files)
Cypress provides a hot reloading test runner allowing you to develop your tests in a very similar way to how you do local development already

The only limitations of Cypress is that it works with Chrome, Firefox, Electron, and the Edge browser. There are plans for it to support more browsers in the future. It is important to note, however, that those 4 browsers take up a big market share of the browser world and solve many usecases. Each project has different needs, but these 4 browsers provide a lot of potential coverage of E2E tests for Frontend projects.

Cypress has a load of features and a great community supporting it. I highly recommend checking out their docs

In this post I'm going to introduce Cypress and how you can use it with React. If you'd like to follow along, I've created a GitHub repo that has all of the examples I cover at https://www.github.com/andrewevans0102/cypress-react. The example I used also is a rebuild of a previous project I used for my post How to get started with Cypress.

I also created a YouTube video where I walk through this same sample project.

Sample Application

Before we dive into using Cypress, I just want to explain how the sample application works. You can reach the sample application at https://www.github.com/andrewevans0102/cypress-react.

The application has three major sections which have examples for:

Forms
Lists
Network Requests

In our Cypress Setup we're going to walkthrough building E2E tests for all three of these pages.

Installing Cypress

There are multiple ways to install Cypress. If you check out the docs, you'll see that you can install it with npm, yarn, or even a manual install of the binary.

In my project I used npm, so I installed it as a dev dependency with:

npm install cypress --save-dev

Once you do the npm install you should see a cypress folder and a cypress.json file created in your project's folder:

Please notice the "examples" folder, this is something I created and just copied the generated example files from integrations over to their own folder for simplicity.

The directories and files that are built contain different configuration that we'll use to build E2E with Cypress. They all correspond to:

fixtures is where you build mocks or stubbed responses for your tests
integration is where you place your actual test .spec.js files by default.
plugins allow you to extend Cypress behavior
screenshots and videos are where the test runner will store visual copies of test runs (more on that in the next sections)
support allows you to define "commands" or boil plate behavior you can reference in your tests avoiding the need to repeat startup tasks like login or similar flows

Once you've got Cypress installed, you can see your first interaction with Cypress by calling "open" with:

./node_modules/.bin/cypress open

First Tests

When you first install Cypress, you'll notice in the integrations folder that there is a set of examples. These are really useful because you can see a first interaction with the test runner. These examples hit the Cypress "kitchen sink" site, so there is no configuration required to run them. If you want to go ahead and play with them, you can use the cypress open command that was at the end of the previous section to see the test runner in action.

I normally go ahead and copy the "examples" over to its own directory. Assuming you've done that, the next step is to configure some scripts that automate working with Cypress.

I recommend creating the following npm scripts:

{
"scripts": {
    "cypress-open": "./node_modules/.bin/cypress open",
    "cypress-local": "concurrently \"npm run start\" \"npm run cypress-open\"",
    "cypress-run": "./node_modules/.bin/cypress run",
    "cypress-ci": "start-server-and-test \"npm run start\" http://localhost:3000 \"npm run cypress-run\""
}
}

Let's talk about what they do:

cypress-open opens the test runner by itself

cypress-local runs the test runner and the application locally (interactive)

cypress-run runs the test runner in CI (headless)

cypress-ci runs the application and the test runner in CI (headless)

Please note CI refers to "Continous Integration" and is just referring to a build pipeline

You'll also need to go ahead and install concurrently and start-server-and-test with:

npm install concurrently
npm install start-server-and-test

As I stated in the intro, Cypress runs in the same event loop as your application. This means that when you run your Cypress tests, you'll need to use some mechanism to run your application alongside the test runner. The use of concurrently and start-server-and-test allows for this behavior. When running locally concurrently keeps your application running alongside the test runner. When running in CI, the use of start-server-and-test will shut down your application and the test runner when the tests have completed.

Let's write our first test to see the "local" run of this in action.

Go over to the integrations folder and create a file first_test.spec.js with the following:

describe("First Test", () => {
  it("should visit home page", () => {
    cy.visit("http://localhost:3000/home-page");
  });
});

If you notice, this is following the standard mocha and jasmine syntax. This is nice because if you're familiar with other testing frameworks, there isn't a lot for you to learn beyond just building your tests and getting used to the cy test runner object.

In this first test, we're just visiting the sample applications home page. Once you've created your first_test.spec.js file, go ahead and run npm run cypress-local to see the test run.

When you do the first run, you'll note that by default Cypress will open a session of Chrome to actually test your application. The test runner then provides details on each step that is run, and you can even do "time travel" when you click through the different steps that ran.

Forms Test

So now that we have our first test running, let's go ahead and create our forms page test. Go to the integrations folder and create form.spec.js like the following:

describe('Form Test', () => {
  it('should visit home page', () => {
    cy.visit('/home-page');
  });

  it('should visit home page and click start', () => {
    cy.visit('/home-page');
    cy.get('#startButton').click();
    cy.get('h1').should('contain', 'Learn Cypress');
  });

  it('should go to the forms page and enter login information', () => {
    cy.visit('/home-page');
    cy.get('#startButton').click();
    cy.get('h1').should('contain', 'Learn Cypress');
    cy.get('#formsButton').click();
    cy.get("#email").type("HanSolo@gmail.com");
    cy.get("#password").type("password");
    cy.get("#submitButton").click();
    cy.on("window:alert", (str) => {
      expect(str).to.equal(
        "successfully entered input with email HanSolo@gmail.com and password password"
      );
    });
  });
});

Also, go ahead and modify the cypress.json file to have the following:

{ "baseUrl": "http://localhost:3000" }

What we just did there is created our first environment variable within Cypress. The cypress.json file allows you to create variables you can reference in your tests. The baseUrl is a built in variable, but you can create your own custom ones as well. By having the baseUrl defined, we can modify our "visit the homepage test" to be:

  it('should visit home page', () => {
    cy.visit('/home-page');
  });

instead of:

  it("should visit home page", () => {
    cy.visit("http://localhost:3000/home-page");
  });

If you go ahead and run npm run cypress-local then you should see the forms test run. If you hadn't stopped Cypress or the local application, you should've seen the test runner automatically load in the files and reload the page. This is one of the best parts of Cypress because it allows for "hot reloading" as you develop your tests.

If you notice in the code, we're passing commands to the Cypress test runner object cy like the following:

cy.get('#startButton').click();

What we are doing here is identifying the object on the page, and then passing events to it. We could also get information as you see with the asserts:

cy.get('h1').should('contain', 'Learn Cypress');

This is all similar behavior to the traditional:

arrange = setup your test environment
act = run the actual test
assert = verify the output result

There is a lot of documentation on how to setup tests and interact with the cy object. I recommend checking out the guide on writing your first tests.

Lists Test

So now we can also try testing the lists page. Create a file list.spec.js in the integrations folder:

describe("List Test", () => {
  it("should go to the list page and add a value", () => {
    cy.visit("/home-page");
    cy.get("#startButton").click();
    cy.get("h1").should("contain", "Learn Cypress");
    cy.get("#listsButton").click();
    cy.get("#createInput").type("use the force Luke!");
    cy.get("#createButton").click();
    cy.get("li").eq(4).should("contain", "use the force Luke!");
  });

  it("should go to the list page and delete a value", () => {
    cy.visit("/home-page");
    cy.get("#startButton").click();
    cy.get("h1").should("contain", "Learn Cypress");
    cy.get("#listsButton").click();
    cy.get("#createInput").type("use the force Luke!");
    cy.get("#createButton").click();
    cy.get("li").eq(4).should("contain", "use the force Luke!");
    cy.get(":nth-child(5) > .btn").click();
    cy.get("[data-cy=listValues]").children().should("have.length", 4);
  });
});

Once you've ran it, you should see something like the following:

If you run npm run cypress-local now, you can run the tests on the lists page. This is similar to the way that built our forms tests, with the additonal step that you notice we can count values as you see here:

    cy.get(":nth-child(5) > .btn").click();
    cy.get("[data-cy=listValues]").children().should("have.length", 4);

This just highlights one of the cool things you can do with the cy runner. Its particularly useful with lists since you'll often need to see the length of a list or if a value is present within the list in a frontend project.

Network Requests

With any Frontend application, network requests are always a key part of any workflow. You need to interact with the backend to get or push data.

Let's create our network requests test in the integrations folder by creating a file called network.spec.js with the following:

describe("Network Requests Page Test", () => {
  beforeEach(() => {
    // server starts to listen for http calls
    cy.server();
    // create route that cypress will listen for, here it is the films endpoint of the SWAPI
    cy.route("GET", "https://swapi.dev/api/films/**").as("films");
  });

  it("should go to the network requests page and select a movie", () => {
    cy.visit("/home-page");
    cy.get("#startButton").click();
    cy.get("h1").should("contain", "Learn Cypress");
    cy.get("#networkButton").click();
    cy.get("#movieSelect").select("A New Hope (1)");
    cy.get("#movieTitle").should("contain", "A New Hope");
    cy.get("#episodeNumber").should("contain", 4);
  });

  it("should go to the network requests page and verify the HTTP payload called", () => {
    cy.visit("/home-page");
    cy.get("#startButton").click();
    cy.get("h1").should("contain", "Learn Cypress");
    cy.get("#networkButton").click();
    cy.get("#movieSelect").select("A New Hope (1)");
    // await the response from the SWAPI http call
    cy.wait("@films").then((films) => {
      expect(films.response.body.title).to.equal("A New Hope");
    });
    cy.get("#movieTitle").should("contain", "A New Hope");
    cy.get("#episodeNumber").should("contain", 4);
  });
});

If you run it, you should see the following:

With this test, you'll notice that we have a beforeEach block:

  beforeEach(() => {
    // server starts to listen for http calls
    cy.server();
    // create route that cypress will listen for, here it is the films endpoint of the SWAPI
    cy.route("GET", "https://swapi.dev/api/films/**").as("films");
  });

This is standard with mocha and jasmine as it sets up the test runner before actually exercising the tests. Notice the use of the cy.server object. This allows Cypress to listen for network calls and specficially this test is looking for the "swapi.dev" API call with:

cy.route("GET", "https://swapi.dev/api/films/**").as("films");
  });

Then in the actual test runs, notice that there is a cy.wait which waits for the API call to complete to verify the results:

    // await the response from the SWAPI http call
    cy.wait("@films").then((films) => {
      expect(films.response.body.title).to.equal("A New Hope");
    });

This is very powerful in that it will allow you to test the payload of your API calls, and allows the test runner to be versatile enough to not only deal with the DOM on your page, but also the proper payloads that the HTTP calls should be returning.

Commands

So up until this point, all of our tests have had something like the following:

    cy.visit("/home-page");
    cy.get("#startButton").click();
    cy.get("h1").should("contain", "Learn Cypress");
    cy.get("#networkButton").click();

This is what I would consider "boiler plate" in that you always have to visit the home page and click one of the buttons to interact with the applicable page.

Cypress allows you to reduce that boilerplate by creating commands in the support folder. If you go ahead and open up the file cypress/support/commands.js you'll notice that there is some docs pointing to the commands guide. In Cypress, you can build commands which are basically just aliases to a set of steps. If you build a command here, you can then reference it in your tests and avoid having to copy and paste a lot.

Go ahead and add the following to the commands.js file:

Cypress.Commands.add("start", () => {
  cy.visit("/home-page");
  cy.get("#startButton").click();
  cy.get("h1").should("contain", "Learn Cypress");
  cy.get("#formsButton").should("contain", "Forms");
  cy.get("#listsButton").should("contain", "Lists");
  cy.get("#networkButton").should("contain", "Network Requests");
});

Here we create a start command which has the flow up to the content page. It then verifies that the correct values are present for the button labels.

We can then go back to our original Forms Test file (forst.spec.js) and remove the:

    cy.visit('/home-page');
    cy.get('#startButton').click();
    cy.get('h1').should('contain', 'Learn Cypress');

and add:

  before(() => {
    cy.start();
  });

This references the start command we created. Now if you run the test, you'll see the start command and test has run without you needing to reference in the associated spec file.

Running Cypress in CI

So the last thing I wanted to cover was what happens when you run Cypress in CI. In the beginning sections you notice we created a cypress-ci npm script which called cypress run instead of cypress open. This is Cypress mechanism to run "headless" in your project pipeline.

If you go ahead and take the tests we've written so far, you can run cypress-ci to see the output. First stop your app and Cypress (if you haven't already done so) and then run npm run cypress-ci to see the following:

This is all logged runs of all your spec files. Up until now, you would run each spec file individually with the test runner GUI. Now in CI, Cypress will run all of your spec files with console output that you can keep in your respective CI system.

You'll also note that there are mp4 files stored in the videos folder:

These are live runs of your CI tests. The cool part here is that you could script out this process such that you could copy these files and send them to a shared folder. This could also be sent directly to your project's Product Owner when you do deployments. There are a lot of options here.

Closing Thoughts

I hope you've enjoyed my post and learned something about Cypress. I've used it on several projects, and found it not only powerful but also fun. The tooling and community support with Cypress make it a very developer friendly tool. The support for logging and storing test run information also make it a powerful asset to any project team. I recommend checking out the docs and various guides on the Cypress Webisite.

Thanks for reading my post! Follow me on andrewevans.dev and on twitter at @AndrewEvans0102. Also check out Rhythm and Binary's new Twitter feed at @rhythmandbinary and YouTube channel. Thanks!

Celebrating Pi Day 2021

Andrew Evans — Sun, 14 Mar 2021 12:05:53 +0000

Every year on 03/14 everyone celebrates "Pi Day." This is a reference to the math number and also Raspberry Pi.

I thought it'd be fun to write a short post to celebrate Pi Day. As part of my celebration, I also 3D printed a "Pi Trophy" object that I found on thingiverse.

In addition to printing this, I also used a Raspberry Pi with the Camera Module to capture a time lapse of the print as it was happening on my 3D printer.

To create the time lapse, I setup a Raspberry Pi Zero W with the Pi Camera library. If you'd like to see how this works, check out my post Raspberry Pi Time Lapse.

In this post I just wanted to celebrate Pi Day and provide some fun background facts.

I created a YouTube video of my time lapse for my 3D print:

Pi Math

The number "Pi" is the ratio of a circle's circumference to its diameter and is normally equated to about 3.14.

You see Pi in many areas of Math and Physics. The use of the Greek later π representing Pi was first recorded by William Jones in 1706 (source wikipedia). The use of Pi as a number has been recorded across the ages, even back to ancient times.

The cool part about the Pi number is that it is actually the same for all circles! So this means that if you divide the circumference by the diameter for a penny its the same as a planet! Both always approximate to 3.14. This makes the standard number very powerful for applications in things like physics and math. The number itself has been traced across civilizations back to ancient times. If you'd like to learn more I recommend checking out the the Wikipedia Page and the Scientific American article.

Raspberry Pi

Raspberry Pis are mini computers that come with a variety of connections already setup. They are great for both teaching and hobbyists. If you get models like the Raspberry Pi Zero W, you can connect it to wifi and bluetooth right out of the box. If you get one of the more powerful models like the new Raspberry Pi 4 Model B, you get more processing power and can even connect it to two displays (mini hdmi)!

The fun thing with Raspberry Pi's is that you can easily get into the world of the Internet Of Things (IOT), and connect with a great community of developers. The Raspberry Pi foundation also does a great job of providing tutorials and materials for folks new to computers. Its a great first project for anyone interested in learning programming or anything technology related. If you're looking to learn, I recommend checking out the Raspberry Pi foundation tutorials and starting with small projects. Since the main memory that the Raspberry Pi runs on is an SD Card, it is also easy to swap out entire systems if you wanted to.

I've done quite a few projects with Raspberry Pis and highly recommend them. If you'd like to see some of my Raspberry Pi projects, check out the following links.

Closing Thoughts

Thanks for reading my post! I hope you enjoyed the video and get a chance to celebrate "Pi Day" today. Follow me on andrewevans.dev and on twitter at @AndrewEvans. Also check out Rhythm and Binary's new Twitter feed and YouTube channel. Thanks!

Raspberry Pi Weather Lamp

Andrew Evans — Mon, 01 Mar 2021 22:35:20 +0000

Recently I had a fun idea for an IoT project where I could have a lamp tell me what the weather was doing. My basic premise was that the lamp would turn different colors based on conditions like Rain, Snow, Clouds, or Sun. There are many APIs that can give me forecast information freely, so I just needed to pull together a way to power the lamp and a way to change colors.

In this post, I'm going to walkthrough how I built a weather lamp with a Raspberry Pi and a 3D printed lamp. To run this lamp I'm using Python and you can see a full copy of all the source code I go over at my repo https://github.com/andrewevans0102/weather-lamp.

I also created a YouTube video that covers this same project. Check it out at the following:

Required Materials

To do this project, you only need some basic materials that you can get either directly off of Amazon or at your local hobby store. If you wanted to 3D print the lamp itself, that would require a 3D printer. However, you can find votive lamps and covers on Amazon that provide the same look and feel and could be used instead.

For the hardware part of the project:

If you have a 3D printer, you can use the STL and GCODE file that I've included in my GitHub repo in the "lamp-model" folder. The original file was found at Thingiverse here.

Some notes on the hardware:

There are a lot of places to get all of these things.
The CanaKit package I linked is great and I've used it for several projects. I highly recommend that.
The Bread Board, Resistors, Jumper Wires, and LED lights can be all bought together. I linked them individually just to show what you would need.
The reason we need a resistor is because they are necessary to control the power consumption from the Raspberry Pi. The Raspberry Pi itself can only provide about 60mA but the LEDs will want to pull more. You'll need at least a 330Ω resistor. When setting this all up, I found the this post on PiHut super helpful with explaining why you need this and how it works.
As I stated in the intro, if you don't have a 3D printer there are a lot of alternatives to the cool lamp cover I printed. I recommend some googling and even other DIY ideas for fun.
If you notice the Raspberry Pi I'm using already has GPIO pins soldered on board. Usually if you buy a Raspberry Pi Zero W, it doesn't have these. You can solder them yourself very easily, or you could get a kit that has this all connected. You could also get a different Raspberry Pi that has the GPIO pins already onboard. If you want to go the soldering route, check out this kit on Amazon that has an iron and the necessary equipment.

Setting up your Raspberry Pi

Setting up for your Raspberry Pi project only takes a few steps. If this is your first Raspberry Pi project, you should first check out the Raspberry Pi website because they have some great docs on getting to know your device.

For this project I'm using what is called a "headless" setup where the Raspberry Pi doesn't have a desktop environment, and is controlled via SSH and the terminal. To do this, do the following:

Format the SD card of your Pi with the official Raspberry Pi Imager. Once you install the imager just choose the 32 bit Raspberry Pi Lite version that you see here:

Next, once your SD card is formatted, remove and reinsert the SD card into your computer and open the "boot" folder that opens up. This should automatically happen, but if not then just go into your Finder or File Explorer (windows) and navigate into the attached SD card and "boot" folder.
In the boot folder add a WPA_SUPPLICANT.conf folder that follows the official Raspberry Pi instructions.
Additionally, you need to add an empty file that is just named "SSH" into your boot folder. This is so that when the Raspberry Pi boots up it will automatically turn on SSH, and then you can directly SSH into the machine after seeing its IP address appear on your network. If you have questions or want to know more about this, check out the Raspberry Pi docs at https://www.raspberrypi.org/documentation/remote-access/ssh/README.md (look for "Enable SSH on a headless Raspberry Pi" on that webpage).
With all of that setup, you can now go ahead and plugin your Raspberry Pi and go to your terminal to get started!

Wiring up your Pi

So now you've got a Raspberry Pi all connected, but you need to wire it to your LEDs so we can write some code to turn them on and off.

The Raspberry Pi has GPIO headers which are basically several hot "leads" that you can connect wires or other peripherals for creating circuits. This is really intuitive and one of the coolest parts of the Raspberry Pi.

Here is a mapping of the GPIO headers you should have on the Raspberry Pi Zero W:

I copied this diagram at stack exchange

Borrowing this diagram from the PiHut's post on LED lights. Your first setup will look something like this:

this image was copied from the PiHut post here.

If you notice, the Raspberry Pi in the picture is a Raspberry Pi 4 (or at least some variation of it). That's fine for our purpose because it still has the same setup for a Raspberry Pi Zero W.

To map the connections on your Raspberry Pi Zero W, I recommend following the above picture and you can use the diagram available on pinout for a more interactive view.

What you see here is:

first a ground connection from the a ground pin on the Raspberry Pi to the breadboard
a power connection from one of the Raspberry Pi GPIO power pins to the breadboard
a resistor that runs between the breadboard connection for the LED light and the Raspberry Pi power source (reduces the energy pull from the Raspberry Pi)

If you're new to breadboards, they function as basically mediums that carry current. You should not two important concepts:

The two lines on the edge of the breadboard (denoted with the blue and red lines) are considered "rails". Anything you plugin to one of the pins on these "rails" is accessible to anything that connects to the rail. This is super useful if you want to have a ground connection for an array of circuits.
The vertical lines (that should have numbers next to them) also share a current. So basically because the resistor is plugged into the breadboard between the LED light and the grounded rail, the current is shared along that line in such a way that if you were to move the resistor connection a little farther down the same vertical line you could have a second LED draw power on this line.

If my attempt at explaining breadboards here is confusing, please check out the PiHuts post which also provides a similar (and probably a better) explanation at this website.

On my Raspberry Pi Zero W, for my first setup I made the following connections:

Jumper Wire from GND pin on Raspberry Pi (next to pin GPIO 26) to blue "rail" on breadboard
Resistor from blue rail to vertical column on bread board (mine had a #2 next to it)
LED with short leg (no kink) to `vertical column on bread board (mine had a #2 next to it)
Jumper Wire from GPIO pin 21 on Raspberry Pi to vertical column on bread board where LED long leg (no kink) was connected (mine had a #3 next to it)

So now you should have your Raspberry Pi wired up, so now we can write code to turn on the light!

First Program

So at this point you should have your Raspberry Pi wired up, and you can go ahead and turn it on. You should look at your router or whatever program you use to see connected IP addresses. Find your Raspberry Pi's IP address and then go to your computers terminal. You can SSH into your pi with ssh pi@<ip_address>. Then you should go ahead and run sudo raspi-config to change your SSH password, set your timezone, and (if you wanted to) change your hostname. I changed mine to "weatherlight" so I could just ssh into it with ssh pi@weatherlight.local. If you have more questions about raspi-config, check out the raspberry pi docs at https://www.raspberrypi.org/documentation/configuration/raspi-config.md.

Now we're going to be using Python's RPI.GPIO library, check out their page on how to install it at https://pypi.org/project/RPi.GPIO/. You should be able to use pip3 but if that's an issue, install that package with the instructions at https://www.raspberrypi.org/documentation/linux/software/python.md.

Once you've go the RPI.GPIO library setup, you can run your first program. So When I was just starting out, I followed the the PiHut post here. Copying basically the same program they provide there, my first program was this:

`py
import RPi.GPIO as GPIO
import time

white_light=21

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(white_light,GPIO.OUT)
print("LED on")
GPIO.output(white_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(white_light,GPIO.LOW)
`

If you're following along in my GitHub repo this is the program first_light.py.

All this does is pass the command to turn ON the LED light, wait 1 second, and then turn it off. If you run this, you should see your light turn on, and then off.

For this project, I also needed to connect multiple lights, so I followed the same process and connected several LEDs to my Raspberry Pi. I then extended the first_light.py program to include these connections to verify they were running correctly with:

`
import RPi.GPIO as GPIO
import time

white_light=21
green_light=20
yellow_light=16
blue_light=14
red_light=18
red_2=17
blue_2=27
yellow_2=22
green_2=10
white_2=9

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(white_light,GPIO.OUT)
print("LED on")
GPIO.output(white_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(white_light,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(green_light,GPIO.OUT)
print("LED on")
GPIO.output(green_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(green_light,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(yellow_light,GPIO.OUT)
print("LED on")
GPIO.output(yellow_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(yellow_light,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(blue_light,GPIO.OUT)
print("LED on")
GPIO.output(blue_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(blue_light,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(red_light,GPIO.OUT)
print("LED on")
GPIO.output(red_light,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(red_light,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(red_2,GPIO.OUT)
print("LED on")
GPIO.output(red_2,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(red_2,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(blue_2,GPIO.OUT)
print("LED on")
GPIO.output(blue_2,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(blue_2,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(yellow_2,GPIO.OUT)
print("LED on")
GPIO.output(yellow_2,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(yellow_2,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(green_2,GPIO.OUT)
print("LED on")
GPIO.output(green_2,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(green_2,GPIO.LOW)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(white_2,GPIO.OUT)
print("LED on")
GPIO.output(white_2,GPIO.HIGH)
time.sleep(1)
print("LED off")
GPIO.output(white_2,GPIO.LOW)

If you're following along in my GitHub repo this is the program verify_lights.py.

So now I had everything wired and could see everything was working with my program. I now needed to pull in a weather forecast and adjust the lights accordingly.

Weather Forecast

So in order to get the weather, there are many APIs that provide variations of weather information. I live in the US and decided to go with the (free) NOAA APIs that the National Weather Service provides. In order to use those APIs, you just need a Latitude and Longitude to be able to pass and get the current conditions. I'm not going to go into the ways to use their APIs, but will refer you to my previous post on how to find the weather forecast for your area at https://rhythmandbinary.com/post/National_Weather_Service_API. I specifically was looking at the "hourly" forecast endpoint, so I could get the conditions each hour and update my light accordingly.

The URL I ended up getting for my location (Richmond, Va) is https://api.weather.gov/gridpoints/AKQ/38,80/forecast/hourly.

Once you've got the URL for your location, you can now write some Python to call the weather service and pull out the necessary information.

I used Python Requests to make my API call and the basic call and data parsing looked like this:

weather call

weatherURL = "https://api.weather.gov/gridpoints/AKQ/38,80/forecast/hourly"

call weather service

response = requests.get(weatherURL)

parse JSON

hourlyWeather = response.json()
properties = hourlyWeather["properties"]
periods = properties["periods"]
rightNow = periods[0]
weatherCondition = rightNow["shortForecast"]

log weather condition that was just called

print(weatherCondition)

Now I had my weather forecast and current conditions, all I had to do was make a little conditional statement to take in the values and turn on the corresponding light.

I created an array of the lights I had connected, and then added the conditional to "turn on" the appropriate light:

weather call

weatherURL = "https://api.weather.gov/gridpoints/AKQ/38,80/forecast/hourly"

call weather service

response = requests.get(weatherURL)

parse JSON

hourlyWeather = response.json()
properties = hourlyWeather["properties"]
periods = properties["periods"]
rightNow = periods[0]
weatherCondition = rightNow["shortForecast"]

log weather condition that was just called

print(weatherCondition)

figure out which light to turn on

if (weatherCondition.find('Sunny')):
# yellow
turnOn = light_colors[0]
elif (weatherCondition.find('Clear')):
# white
turnOn = light_colors[1]
elif (weatherCondition.find('Cloudy')):
# green
turnOn = light_colors[2]
elif (weatherCondition.find('Showers')):
# blue
turnOn = light_colors[3]

I had that working, and so now I could connect this to my wired up Raspberry Pi. If you're following along in my GitHub repo this is the program local_weather.py.

Connecting it all Together

So now I had my wired up Raspberry Pi, my weather forecast, and now I just needed to (in code) connect the two.

At this point I also wanted to add some logging, and error handling since the Weather Service API's occasionally have issues.

I also wanted a way to see if the project was working with some kind of visual indicator. That way I understood if there was no light shown in my little lamp.

All of this ended up with the following:

`
import RPi.GPIO as GPIO
import time
import logging
import requests

definition of light object

class light:

def init (self, name, pin):

self.name = name
self.pin = pin

create a list of lights

lights = []

this covers two LEDs

lights.append( light("blue", 27))

this covers two LEDs

lights.append( light("white", 14))

this covers two LEDs

lights.append( light("yellow", 16))

green lights for sucess indicator

lights.append( light("green_1", 20))
lights.append( light("green_2", 10))

red lights for error indicator

lights.append( light("red_1", 18))
lights.append( light("red_2", 17))

logging

logging.basicConfig(filename='/home/pi/weather-lamp/history_daily.log',format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)

LED colors

light_colors = ['yellow', 'white', 'blue', 'green', 'red']

weather call

weatherURL = "https://api.weather.gov/gridpoints/AKQ/38,80/forecast/hourly"

figure out which light to turn on

turnOn = ''

def turnOffAllLights():
# loop through the list and turn em all off
for singleLED in lights:
logging.info("turning off " + singleLED.name)
GPIO.output(singleLED.pin, GPIO.LOW)

def turnOnGPIOLight( lightName ):
logging.info("looking for light name " + lightName)
for singleLED in lights:
if(singleLED.name.find(lightName) != -1):
logging.info("turning on " + singleLED.name)
GPIO.output(singleLED.pin, GPIO.HIGH)
return

try:
# official start of program
logging.info("starting program")

# turn on GPIO settings so we can work with the LEDs connected
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

# call setup to initialize all the lights
for singleLED in lights:
    GPIO.setup(singleLED.pin, GPIO.OUT)

# first turn off all the lights
logging.info("lets first turn all the lights off")
turnOffAllLights()

# call weather service
logging.info("now lets call the weather service")
response = requests.get(weatherURL)
logging.info("weather service API was called with a return status code of " + str(response.status_code))

# parse JSON
logging.info("parsing JSON")
hourlyWeather = response.json()
properties = hourlyWeather["properties"]
periods = properties["periods"]
rightNow = periods[0]
weatherCondition = rightNow["shortForecast"]

# log weather condition that was just called
logging.info("weather condition that was found is " + weatherCondition)

# figure out which light to turn on
if (weatherCondition.find('Sunny') != -1) or (weatherCondition.find('Clear') != -1):
    # yellow
    turnOn = light_colors[0]
elif (weatherCondition.find('Cloudy') != -1):
    # white
    turnOn = light_colors[1]
elif (weatherCondition.find('Rain') != -1) or (weatherCondition.find('Snow') != -1):
    # blue
    turnOn = light_colors[2]

# turn on green light to indicate success
turnOnGPIOLight(light_colors[3])

# log the output for successful run
logging.info("color to light has been found to be " + turnOn)
turnOnGPIOLight(turnOn)

except Exception as error:
# when error occurs in call, show red light so we know an issue happend
turnOnGPIOLight(light_colors[4])
logging.info('exception occured')
logging.info(error)

logging.info("program finished")
`

If you're following along in my GitHub repo this is the program weather_light.py.

I also rewired my original breadboard to have a shared connection for the LEDs so two LEDs could share the same power draw from the PI. This looks like the following:

I had a little wooden box that I had used previously to hide wires, it has some holes cut in the back of it where I can run jumper cables out of. So basically I put my Raspberry Pi and a breadboard in the wooden box. Then I ran a second set of Jumper cables out of the box to a second breadboard that I covered with my lamp.

The end product looked like this:

The wires and the Raspberry Pi were primarily in the box, and I ran some extra jumper cables out the back to connect to the breadboard under the lamp. The end product hid most of the wires and had a nice finished look in our kitchen like this:

Automating It

So for the last step, I wanted this to run every hour on the hour. To do this I just used a cronjob that runs on the Raspberry Pi. You can do this if you open your terminal (in your SSHed session) and run crontab -e.

I also wanted to make sure my log file doesn't get too big. Since the Raspberry Pi has limited file capacity, its important to keep track of how much memory is being used. To that end I wrote a small python program that deletes the logfile I write to when the program runs every hour:

`
import os
import logging

logging

logging.basicConfig(filename='/home/pi/weather-lamp/history_delete.log',format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)

logging.info("program started")

filename="/home/pi/weather-lamp/history_daily.log"

try:
logging.info("starting the removal of the file")
os.remove(filename)
logging.info("file has been deleted successfully")
except Exception as error:
logging.info('exception occured')
logging.info(error)

logging.info("program finished")

If you're following along in my GitHub repo this is the program cleanup_files.py.

The end product that I had with both the hourly run of my program running and the cleanup program in crontab looks like this:

`
0 * * * * python3 /home/pi/weather-lamp/weather_light.py
30 15 * * * python3 /home/pi/weather-lamp/cleanup_files.py

Closing Thoughts

So I hope you've enjoyed this post and learned something in the process. This project was a lot of fun and has been a fun talking piece in our kitchen. The light also looks really cool at night as the LED and the white fillament make it look like a small votive lamp. I highly recommend trying this project out, and looking at my GitHub repo. I also recommend looking at other cool things you can do with GPIO headers and projects on the Raspberry Pi.

Thanks for reading my post! Follow me on andrewevans.dev and on twitter at @AndrewEvans0102. Also check out Rhythm and Binary's new Twitter feed at @rhythmandbinary and YouTube channel. Thanks!

Making NodeJS Functions Sleep

Andrew Evans — Wed, 10 Feb 2021 15:41:30 +0000

Here are my two cats Chestnut and Rey. They're showing what a well rested NodeJS function looks like.

When learning JavaScript for the first time, one of the biggest challenges is understanding the event loop. Unlike many other languages, JavaScript operates inside a hosted environment which determines how the code is actually ran. This means that things like scope and order can make or break JavaScript programs. It makes it harder when you want to streamline processing to force control on things like threads or API calls.

One common approach to handling things like this is using async await. Which forces things that are async (like promises) to complete before getting a payload and making calls. I actually cover a basic walkthrough of this in my post Optimizing Angular with Async Await.

Recently, I was working on an NodeJS API that calls a set of endpoints to provide a weather forecast. I'm using some of the NOAA Endpoints that you can see here.

The issue I had was that one of the API calls would intermittently fail. What I noticed in testing was that if it failed, but I waited a few seconds and tried again, it worked. This was pretty frustrating because it would happen at different points in the day. My assumption was this was based on load etc. on the NOAA APIs. Regardless, I needed to build in some mechanism to handle this behavior. Otherwise, it would force my API to fail as a result.

So to do this I built a custom "sleep" function. There are several ways to do this (and npm packages as well). I liked my function because it was very simple and I understood what it was doing end to end.

In this post I'm just going to show what I did, and how you can use this in your programs.

Lets start with the code

So to start off with I originally had an API call that looked like this:

    const NOAAWeeklyForecast = await axios.get(endpoint);
    if (NOAAWeeklyForecast === undefined) {
        throw new Error('error when calling NOAA API for weekly forecast');
    }

I'm using axios to handle my AJAX calls here.

This was pretty simple, except (as i stated earlier) if an error happened, the rest of my endpoint would fail.

So to do this, I first created a "sleep" method that looks like this:

// wrap settimeout in a promise to create a wait
const sleep = (ms) => {
    return new Promise((resolve) => setTimeout(resolve, ms));
};

As the comment suggests, it wraps setTimeout with a promise and using setTimeout to delay activity. So if you call this with async await then it will pause or "sleep" any function that calls it.

I refactored my original axios call to have its own method and it looks like this:

const callNOAA = async (endpoint) => {
    let attempts = 5;
    let response = undefined;
    while (attempts > 0) {
        try {
            response = await axios.get(endpoint);
            break;
        } catch (error) {
            console.log(
                `error occured with message ${error} and attempts ${attempts.toString()}`
            );
            attempts = attempts - 1;
            console.log('waiting 5 seconds and then am going to try again');
            // wait 5 seconds before running again
            await sleep(5000);
        }
    }
    return response;
};

If you notice it calls my sleep function, and basically retries the HTTP call in 5 second intervals. Each time keeping track of a number of attempts. If it fails after 5 attempts, then it throws an exception (which can be handled by any function that calls this).

Axios Interceptors

So after adding this, I've been watching my APIs performance for some time. This basically resolved the intermittent failures that I saw originally.

I could have just as easily used something like axios-retry or several other packages that do something similar. However, I liked this implementation because it used some core JavaScript concepts and there was no "black box" that you often get with packages.

Another thing that I wanted to mention was that if you're using axios then there are also things called interceptors. They can be utilized with this kind of behavior, but also can be a great way to track HTTP calls in your NodeJS APIs.

I used axios request and response interceptors like you see here:

// request interceptor
axios.interceptors.request.use(
    function (config) {
        console.log(`request sent with config ${JSON.stringify(config)}`);
        return config;
    },
    function (error) {
        console.log(`request sent with error ${error}`);
        return Promise.reject(error);
    }
);

// response interceptor
axios.interceptors.response.use(
    function (response) {
        console.log(`response sent with return of ${response.status}`);
        return response;
    },
    function (error) {
        console.log(`response sent with ${error}`);
        return Promise.reject(error);
    }
);

If you'd like to learn more about axios interceptors, please check out their GitHub repo.

Closing Thoughts

I hope you enjoyed this post and also learned about a way to make your NodeJS functions sleep. I did quite a bit of googling leading up to this and found that there is a lot of documentation on this method as well as others to make your NodeJS functions sleep.

Thanks for reading my post! Follow me on andrewevans.dev and on twitter at @AndrewEvans. Also check out Rhythm and Binary's new Twitter feed at @rhythmandbinary and YouTube channel. Thanks!

Connecting Microsoft SignalR with Angular

Andrew Evans — Fri, 16 Oct 2020 20:25:54 +0000

Rey and Kylo Ren use SignalR to communicate with the force. Image was orginally copied from here

Microsoft's SignalR is a very powerful technology that enables websocket connections between clients. The technology has been around for quite some time, but now with Azure its even easier to get started.

Microsoft's @aspnet/signalr package makes it possible to leverage both Azure and Angular to create applications that utilize SignalR. In this post, I'm going to walkthrough setting it up in a chat application and discuss the technology along the way. If you want to follow along, please check out my sample application GitHub repo.

What is the SignalR Service?

SignalR is a technology from Microsoft that provides real time communication between client and server via WebSockets. You most often see it used for high frequency applications like chat applications, gaming, dashboards, anything that requires real time updates.

SignalR is offered as either a hosted technology, or you can leverage Azure to use their SignalR service. The Azure SignalR service is very easy to get started with and supports web clients, mobile apps, servers, and IoT devices.

SignalR works for both large and small applications. If you use the SignalR service that Azure provides, you get all of this technology without having to manage the underlying infrastructure.

image was originally copied from here

Azure's SignalR service provides broad language support to include SDKs for ASP.NET Core, ASP.NET C#, and JavaScript. You can also leverage serverless with Azure Functions to connect to the SignalR service, and also to handle the message negotiation.

SignalR can also deliver messages generically as they are sent to a hub, or it can send messages directly to specific clients.

If you'd like to see more on the SignalR technology, I recommend checking out the Microsoft page on it here.

How it works with Angular

So with SignalR you can connect webclients to send and receive messages. It's actually pretty easy to setup with the @aspnet/signalr package and a custom service.

The basic setup looks like the following:

On the left you see the connection process:

The client calls a negotiate function to setup the WebSocket connection between the client and the SignalR service.
The SignalR service connects and creates a connection through a second function that the webclient listens to
The webclient sends messages through the messages function which are then propagated to any other webclients connected to SignalR

On the right you see the end state after the clients have connected. Messages that are sent to the SignalR service are propagated out to the clients in a hub pattern. You can also set messages to only go between specific clients. In the example here and what I'm going to be walking through, I setup a hub whereby all messages are sent to all clients connected to the SignalR service.

This same setup could work with other frameworks than Angular. The key part is just using the @aspnet/signalr package and coordinating the handshake with the actual SignalR service.

Setting this up in Azure

So if you want to use the SignalR service with Angular, you first have to setup the infrastructure with Azure. One of the best parts is that Microsoft makes all of this very intuitive with the Azure Portal.

You first just literally create the SignalR service:

Then you make sure to go over to Keys and capture the connection string that Azure uses when doing the initial handshake:

Then you create two Azure Functions:

negotiate to handle the initial handshake process
messages to literally transport messages once the handshake is good

If you use the JavaScript option, the functions are incredibly simple with the negotiate looking like this:

and then the messages looking like this:

You also need to add AzureSignalRConnectionString as a application setting in your Azure Functions app instance:

The last step with regards to infrastructure is just to enable CORS if you're testing with a URL etc. You can do this with the CORS setting in the services.

I also highly recommend using the Azure Functions VSCode extension for developing the Azure Functions. It is super easy to work with, and makes building and deploying just a few steps.

Building Angular

Once you have your infrastructure setup, now it's time to connect your Angular application.

If you check out my sample application GitHub repo you'll see this in action.

I created a service that wraps the @aspnet/signalr package with two methods like the following:

export class SignalRService {
  private readonly _http: HttpClient;
  // private readonly _baseUrl: string = "http://localhost:7071/api/";
  private readonly _baseUrl: string = environment.azureConnection;
  private hubConnection: HubConnection;
  messages: Subject<string> = new Subject();

  constructor(http: HttpClient) {
    this._http = http;
  }

  private getConnectionInfo(): Observable<SignalRConnectionInfo> {
    let requestUrl = `${this._baseUrl}negotiate`;
    return this._http.get<SignalRConnectionInfo>(requestUrl);
  }

  init() {
    this.getConnectionInfo().subscribe((info) => {
      let options = {
        accessTokenFactory: () => info.accessToken,
      };

      this.hubConnection = new signalR.HubConnectionBuilder()
        .withUrl(info.url, options)
        .configureLogging(signalR.LogLevel.Information)
        .build();

      this.hubConnection.start().catch((err) => console.error(err.toString()));

      this.hubConnection.on("notify", (data: any) => {
        this.messages.next(data);
      });
    });
  }

When the service initializes, it gets a reference to the SignalR endpoint that is exposed from the SignalR service and negotiates the handshake. It then leverages an Angular Subject to emit any new messages received from the "notify" event from SignalR.

With regards to the actual messages, I chose to use localStorage to handle the conversation history as you see here:

  send(message: string): Observable<void> {
    console.log("called2");
    let requestUrl = `${this._baseUrl}messages`;
    return this._http.post(requestUrl, message).pipe(map((result: any) => {}));
  }

  receieve(message: Message): Message[] {
    // read in from local strorage
    const messages = this.load();
    messages.unshift(message);
    localStorage.setItem("messages", JSON.stringify(messages));
    return messages;
  }

  load(): Message[] {
    const messagesLocal = localStorage.getItem("messages");
    let messagesResponse = [];
    if (messagesLocal !== null) {
      messagesResponse = JSON.parse(messagesLocal);
    }
    return messagesResponse;
  }

  clear(): Observable<void> {
    const messagesLocal = localStorage.getItem("messages");
    let messagesResponse = [];
    if (messagesLocal !== null) {
      localStorage.setItem("messages", JSON.stringify(messagesResponse));
    }
    return of(null);
  }

In the actual components in the Angular application that operate the chat function, I create a reference to the SignalR service and handle the events that come in from the stream accordingly:

    this.signalRService.messages.subscribe((message) => {
      // create message
      const result = message.split("|");
      const sendMessage = new Message();
      sendMessage.sender = result[0];
      sendMessage.body = result[1];
      // this.messages.unshift(sendMessage);
      this.store.dispatch(
        MessagesActions.messageRecieved({ message: sendMessage })
      );
    });

I'm using NgRx to handle the various flows in the application, and you see this here with the dispatch of the messageReceived action.

Closing Thoughts

So in this post I introduced how you can use Azure's SignalR service with Angular. It is a very powerful service, that you can easily integrate with your frontend applications.

There are a lot of great things you can build with this technology. As I mentioned in the intro, this is great for any application that needs realtime feedback.

I covered Angular here, but there are similar patterns that you could apply to the other major frontend libraries out there. I encourage you to look at my sample application to learn more.

Thanks for reading my post! Follow me on andrewevans.dev and, feel free to message me on Twitter at @AndrewEvans0102 if you have any questions or wanted to learn more.

How to get Started with React Redux

Andrew Evans — Thu, 15 Oct 2020 23:49:00 +0000

The Mandalorian and Baby Yoda working with Redux on their ship the Razor Crest. The original image was copied from here

Redux is one of the most popular patterns that is in use in the frontend world today. You see the same pattern in not only React, but Angular and Vue as well. Redux is very powerful as it provides a routine way that you can manage state in your applications. Moreover, Redux scales as your projects get larger. So it works great for both small and enterprise applications.

This post is going to walkthrough how to use Redux in your React applications. I'm going to assume that you understand some basics about React Hooks as I'm going to be using the useState, useEffect, useSelector and useDispatch hooks respectively.

I'm also going to be walking through a sample project that I've setup at my GitHub repo here. We will be walking through different phases of the same project. I'm going to walk you through (1) setting up Redux, (2) adding actions and reducers, and (3) creating side effects.

As a Mandalorian fan myself, the sample project will be a mini fan site with pages for episode info, quotes, and more.

Redux Concepts

So before we dive into using Redux, we should cover some vocabulary that we'll be using in the subsequent sections.

image was copied from my article "How to Start Flying with Angular and NgRx" here

Redux is a way to centrally organize your applications state in what's called a store (in the diagram that's the block in pink). The idea is that everything about your application will be stored there, and then you'll use selectors in your components to access this state. The store is immutable which means that it cannot change. When you "change" the store, you are actually generating a new version. This is a concept you see in functional programming, and sometimes can be hard for newer folks to understand. I highly recommend watching Russ Olsen's talk on Functional Programming here for more on this concept.

Your components fire what are called actions that then go through reducers to modify the values in the store. The idea behind reducers is that the state is reduced from an action. An action can be any event that your application does from initial loading of data to responding to a button click. The reducers in your application handle the changes to the store that result.

Your components also subscribe to selectors which basically listen for any type of state change. Whenever the store updates, the selectors receive the updates and allow you to render your components accordingly.

Some actions can generate "side effects" which are usually HTTP calls. This would be when you want to call an API to get values to put in the store. The flow there is that you would (1) fire an action, (2) call an API through an effect, and then return an action that goes through a reducer to modify the store.

I know that this is a lot of vocabulary to start, but it will make more sense as we begin to add Redux to our application.

Starting Out

So if you view my sample project, you'll find that it has the following folders:

start
redux-setup
redux-actions
redux-http

We're going to walkthrough the folders in the project in this order. We will begin in the start folder, as that's a version of the application with no Redux at all. Then the three other folders are completed phases of this project:

redux-setup is the start with redux added and an initial set of actions, reducers, selectors, and effects for the episodes page.
redux-actions is the start with the episodes and quotes actions, reducers, selectors, and effects setup.
Finally, redux_http includes a set of actions, reducers, selectors and an effect that makes an actual HTTP call.

When you're finished, you'll have a mini Mandalorian fan page that includes a page for episodes, quotes, a video of Season 2, and even a way to send a contact message.

Initial Setup

We'll start by cloning the project, and then going into the start folder.

The initial project looks like this:

.
├── README.md
├── ReduxFlow.png
├── package-lock.json
├── package.json
├── public
│   ├── favicon.ico
│   ├── index.html
│   ├── manifest.json
│   └── robots.txt
└── src
    ├── App.css
    ├── App.js
    ├── App.test.js
    ├── components
    │   ├── Header.js
    │   └── index.js
    ├── config
    │   ├── episodes.json
    │   └── quotes.json
    ├── index.css
    ├── index.js
    ├── logo.svg
    ├── pages
    │   ├── ContactPage.js
    │   ├── EpisodesPage.js
    │   ├── HomePage.jpg
    │   ├── HomePage.js
    │   ├── QuotesPage.js
    │   ├── Season2Page.js
    │   └── index.js
    ├── serviceWorker.js
    ├── setupTests.js
    └── styles
        ├── _contact.scss
        ├── _episodes.scss
        ├── _header.scss
        ├── _home.scss
        ├── _quotes.scss
        ├── _season2.scss
        └── styles.scss

The first step is to add Redux to your application and then install the necessary libraries. Go ahead and install the libraries with npm by doing the following:

npm i react-redux
npm i redux
npm i redux-devtools-extension
npm i redux-thunk

Now, I also recommend the Redux DevTools extension for Chrome as that will help you see what happens with your store. I recommend installing that at this phase as well.

So now with your libraries installed, let's go over to the src/index.js file to setup our store.

To add Redux to React, you first need to wrap your entry component with a Provider as you see here:

// step 1 add these imports
import { Provider } from 'react-redux';
import configureStore from './redux/configureStore';

const initialState = {};
const { store } = configureStore(initialState);

ReactDOM.render(
    // step 2 wrap your app in the Provider here
    // <React.StrictMode>
    // <App />
    // </React.StrictMode>,
    <Provider store={store}>
        <App />
    </Provider>,
    document.getElementById('root')
);

Now, you'll notice that we're referencing a redux folder that hasn't been created yet. You'll need to ahead and set that up so we can begin the actions, reducers, and eventually effects that we'll be using.

Go ahead and create a src/redux folder as this will be where we put our Redux implementation. Now create the src/redux/configureStore.js file as you see here:

import { createStore, applyMiddleware } from 'redux';
import thunk from 'redux-thunk';
import { composeWithDevTools } from 'redux-devtools-extension';
import rootReducer from './reducers/index';

const middleware = [thunk];
const enhancers = [];

// create enhancers to include middleware
// thunk allows you to dispatch functions between the actions
const composedEnhancers = composeWithDevTools(
    applyMiddleware(...middleware),
    ...enhancers
);

// create the store and return it to the application onload
// note that here we are including our reducers to setup our store and interactions across the application
export default function configureStore(initialState) {
    const store = createStore(rootReducer, initialState, composedEnhancers);

    return { store };
}

As the comments point out, we first use the redux-devtools-extension library to create enhancers that we will use with Redux. This is a common way to start building your store, but there are other methods and enhancers you can include.

Then we create the configureStore method by using the createStore to build a root reducer and an initial state with our enhancers. Also note that we are using the redux thunk middleware so that we can return functions instead of just actions with our flows. There are a lot of options with middleware beyond thunk, but this is all we'll need for our application.

Once you've got configureStore all setup, let's go ahead and create our reducers folder in src/redux. Inside that folder create src/redux/reducers/index.js file with the following:

import { combineReducers } from 'redux';

export default combineReducers({
});

Now we've got the basic shell setup, and we have basically an empty store with no initial state except for {}.

Setting up the Actions

So with this basic shell, we now can go ahead and add actions. We're going to setup the episodes actions for the site.

Go ahead and create an actions and actionTypes folder in the src/redux folder that we created before.

Inside actionTypes folder create an Episodes.js file and copy and paste the following:

export const GET_EPISODES = 'episodes/GET_EPISODES';
export const SET_EPISODES = 'episodes/SET_EPISODES';
export const EPISODES_ERROR = 'episodes/EPISODES_ERROR';

export const initialEpisodesState = {
    episodes: [],
    errors: [],
};

I'm also using JavaScript modules, so add a index.js file next to it with:

import * as EpisodesActionTypes from './Episodes';

export { EpisodesActionTypes };

What is this doing? This is defining the action types we'll be using in our application. Notice that it is very simple and we have a GET_EPISODES and SET_EPISODES action along with an EPISODES_ERROR message. The initialEpisodesState is just defining what our store will look like when the application loads.

Next lets actually define the actions in a file src/redux/actions/Episodes.js file like so:

import { EpisodesActionTypes } from '../actionTypes';
import episodes from '../../config/episodes';

export function getEpisodes() {
    return { type: EpisodesActionTypes.GET_EPISODES };
}

export function setEpisodes(episodes) {
    return { type: EpisodesActionTypes.SET_EPISODES, episodes };
}

export function episodesError() {
    return { type: EpisodesActionTypes.GET_EPISODES };
}

// here we introduce a side effect
// best practice is to have these alongside actions rather than an "effects" folder
export function retrieveEpisodes() {
    return function (dispatch) {
        // first call get about to clear values
        dispatch(getEpisodes());
        // return a dispatch of set while pulling in the about information (this is considered a "side effect")
        return dispatch(setEpisodes(episodes));
    };
}

I'm also using JavaScript modules, so add a index.js file next to it with:

import * as EpisodesActions from './Episodes';

export { EpisodesActions };

So as you see here, we're defining a getEpisodes function that corresponds to the GET_EPISODES action, a setEpisodes function that corresponds to the SET_EPISODES action, a episodesError that corresponds to the EPISODES_ERROR action, and finally a side effect to retrieveEpisodes which will pull them from a local configuration file.

There are differing opinions as to where to place side effects in React projects. From the documentation I found on React Redux I found it was recommended to place them alongside your actions. In practice, I've experienced that having the side effects near your actions makes it easy as a developer to find and maintain them. In a more general sense, since React is a library, you can organize your application as you see fit and put them wherever it best works for you.

So now that we've defined our action types and actions, let's add reducers that use those actions. Create a src/redux/reducers/Episodes.js file as you see here:

import { EpisodesActionTypes } from '../actionTypes';

function Episodes(state = EpisodesActionTypes.initialEpisodesState, action) {
    switch (action.type) {
        case EpisodesActionTypes.GET_EPISODES:
            return Object.assign({}, state, {
                loading: true,
                episodes: [],
            });
        case EpisodesActionTypes.SET_EPISODES:
            return Object.assign({}, state, {
                ...state,
                loading: false,
                episodes: action.episodes,
            });
        case EpisodesActionTypes.EPISODES_ERROR:
            return Object.assign({}, state, {
                ...state,
                errors: [...state.errors, action.error],
            });
        default:
            return state;
    }
}

export default Episodes;

Since I'm using JavaScript modules, go ahead and modify the index.js file we had before to include the Episodes.js file as you see here:

import { combineReducers } from 'redux';
import Episodes from './Episodes';

export default combineReducers({
    Episodes,
});

What is all of this doing? The reducers are keyed based on action type. If you notice, the value that is returned from the action is then applied to the necessary place in the state. So in the case of SET_EPISODES you'll note that it is taking the action payload and putting it into the episodes portion of the state as you see here:

case EpisodesActionTypes.SET_EPISODES:
    return Object.assign({}, state, {
        ...state,
        loading: false,
        episodes: action.episodes,
    });

Connecting Redux to Your Components

So now we have all the pieces together, but we still need to add Redux to our actual components. So let's modify the src/pages/EpisodesPage.js as you see here:

import React, { useEffect } from 'react';
import { useSelector, useDispatch } from 'react-redux';
import { EpisodesActions } from '../redux/actions';
import '../styles/styles.scss';
// import episodes from '../config/episodes';

// const episodes = [
// { key: 'first', value: 'something here' },
// { key: 'second', value: 'something there' },
// ];

function EpisodesPage(props) {
    const dispatch = useDispatch();

    // first read in the values from the store through a selector here
    const episodes = useSelector((state) => state.Episodes.episodes);

    useEffect(() => {
        // if the value is empty, send a dispatch action to the store to load the episodes correctly
        if (episodes.length === 0) {
            dispatch(EpisodesActions.retrieveEpisodes());
        }
    });

    return (
        <section className="episodes">
            <h1>Episodes</h1>
            {episodes !== null &&
                episodes.map((episodesItem) => (
                    <article key={episodesItem.key}>
                        <h2>
                            <a href={episodesItem.link}>{episodesItem.key}</a>
                        </h2>
                        <p>{episodesItem.value}</p>
                    </article>
                ))}
            <div className="episodes__source">
                <p>
                    original content copied from
                    <a href="https://www.vulture.com/tv/the-mandalorian/">
                        here
                    </a>
                </p>
            </div>
        </section>
    );
}

export default EpisodesPage;

As you'll note there are a few changes that make Redux possible. First note that we are pulling in the necessary hooks at the top with:

import React, { useEffect } from 'react';
import { useSelector, useDispatch } from 'react-redux';
import { EpisodesActions } from '../redux/actions';

Next you'll note that we commented out the pull of the episodes information locally and instead are retrieving it from a selector:

// import episodes from '../config/episodes';

// const episodes = [
// { key: 'first', value: 'something here' },
// { key: 'second', value: 'something there' },
// ];

function EpisodesPage(props) {
    const dispatch = useDispatch();

    // first read in the values from the store through a selector here
    const episodes = useSelector((state) => state.Episodes.episodes);

Next you'll notice the use of useEffect which dispatches a retrieveEpisodes action as on load:

    useEffect(() => {
        // if the value is empty, send a dispatch action to the store to load the episodes correctly
        if (episodes.length === 0) {
            dispatch(EpisodesActions.retrieveEpisodes());
        }
    });

So now, if you run the application, and then go to the Episodes page you should see it in action. If you open the Redux Devtools Extension you'll see the flow:

So what happened and how does this work?

On load, you initialized your store with an area for episodes
The EpisodesPage component has subscribed to the store to listen for any new state changes
When you click on the "Episodes" page the retrieveEpisodes action fired which then actually triggers a side effect to first call GET_EPISODES to clear the episodes in the store and then SET_EPISODES which retrieves them from the config file and returns them to the component
The EpisodesPage component receives the new store and renders the component

If your end result did not do the above flow, please check out the redux-setup folder and see the end product there.

Adding Quotes

So now that you've got the episodes covered, you can now add quotes. The process is very similar and you'll create:

src/redux/actions/Quotes.js
src/redux/actionsTypes/Quotes.js
src/redux/actions/reducers/Quotes.js

Then in the QuotesPage component you'll setup the same action --> effect --> action --> reducer flow that we did before.

const dispatch = useDispatch();

// first read in the values from the store through a selector here
const quotes = useSelector((state) => state.Quotes.quotes);

useEffect(() => {
    // if the value is empty, send a dispatch action to the store to load the episodes correctly
    if (quotes.length === 0) {
        dispatch(QuotesActions.retrieveQuotes());
    }
});

To attempt to keep this post manageable, I won't add the implementation details here. I'll refer you to look at the redux-actions folder for what the finished product looks like.

Adding HTTP

So up until now the two flows that you've seen for episodes and quotes used local files and did not make any HTTP calls. One of the most common usecases you see with React Redux is to make HTTP calls to handle interactions with APIs.

If you go into the redux-http folder you'll see an example where we add HTTP calls for the "contact" page of the site.

The contact page actually adds messages to the page here. So when you've got this setup, you can see it in action by opening that page up alongside your local application.

When making HTTP calls with React Redux, the general best practice is to put the side effect alongside the actions. If you look in the redux folder you'll see Contact Actions, ActionTypes, and Reducers that are created.

A good convention to use with redux is to have an action that initializes the process, a second action that actually calls the process, and then a success and failure action to suit. You can see this here:

// here we introduce a side effect
// best practice is to have these alongside actions rather than an "effects" folder
export function sendContact(contact) {
    return function (dispatch) {
        // first call sending contact to start the process
        dispatch(sendingContact(contact));
        // actually call the HTTP endpoint here with the value to send
        return axios
            .post(contactEndpoint, contact)
            .then((response) => {
                dispatch(contactSuccess(response));
            })
            .catch((error) => {
                dispatch(contactError(error));
            });
    };
}

If you notice the sendContact action is called, then it calls sendingContact and then it makes the HTTP call and responds with either a contactSuccess or contactError response.

Once you've built out the redux parts, you can connect it to your component like so:

const dispatch = useDispatch();

// when you make the rest call, the response can be seen in the selector here
const response = useSelector((state) => state.Contact.response);

// when an error occurs it should appear here
const errors = useSelector((state) => state.Contact.errors);

const handleSubmit = (event) => {
    setProgress(true);
    event.preventDefault();
    const sendMessage = { firstName, lastName, message };
    dispatch(ContactActions.sendContact(sendMessage));
    // axios
    // .post(messageEndpoint, sendMessage)
    // .then((response) => {
    // alert('success');
    // setProgress(false);
    // })
    // .catch((error) => {
    // alert('error');
    // setProgress(false);
    // });
};

useEffect(() => {
    if (response !== undefined) {
        setProgress(false);
    }

    if (errors.length > 0) {
        setProgress(false);
    }
}, [response, errors]);

Then in your template you can catch the response or errors with a check on the selectors as happens with the following:

{
    response !== undefined && (
        <article className="contact__response">
            Success with a return of {response.status.toString()}
        </article>
    );
}
{
    errors.length > 0 && (
        <article className="contact__error">
            Error occured with message "{errors[0].message}"
        </article>
    );
}

This pattern scales well, and can be used throughout the HTTP calls in your components.

Again, to keep this post necessarily brief I'll refer you to the implementation in the redux-http folder.

Closing Thoughts

So as you see with this project, once you understand the parts to Redux it's not hard to follow the pattern. In our project we setup episodes, quotes, and even a contact page that used Redux in the process.

As I stated in the intro, this pattern enables you to have a common method of handling your applications state as you build more features and move it through its lifecycle. I have personally found that this pattern makes maintenance much easier than manually handling application state through custom services and event interactions.

I hope that this post and my sample project helped you in your journey to learn more about Redux. I recommend playing with the example project I have here, and building out additional pages or features to learn the process.

Thanks for reading my post! Follow me on andrewevans.dev and, feel free to message me on Twitter at @AndrewEvans0102 if you have any questions or wanted to learn more.

Rebuilding my Site with Netlify and Next.js

Andrew Evans — Mon, 08 Jun 2020 16:33:06 +0000

For quite some time now, I have wanted to rebuild my primary blogging site (www.rhythmandbinary.com) - I was previously using WordPress, but decided I wanted to host my own content and have more control over the publishing process. And so finally, over the past few weeks, I have rebuilt this site using Netlify and Next.js and wanted to highlight some cool things I learned along the way.

I chose Next.js because it has a lot of built in features that work well for static sites (check out their blog site example here). I also used Netlify because of the ease of their pipeline, and a lot of their built in features which make hosting and writing super fun.

From Build to Publish

With Netlify, you get a super user friendly console to manage your site out of the box. You connect Netlify to your site's GitHub repo, and set up a pipeline that listens to your repo's master branch. Then whenever you merge a PR into master, Netlify will kickoff a build and deploy of your site. You can customize this for more specialized setups, but this feature alone took a lot of the trouble out of managing a pipeline. There are also a ton of great features with Netlify, I recommend checking out their docs for more.

Since this site is a blog, I wanted some form of a CMS to make writing posts easier. After some googling I found Netlify CMS! It works nicely alongside Netlify hosting, and comes with a WYSIWYG editor that you can easily access directly from your site. Whenever you write posts, Netlify CMS makes PRs directly to your GitHub repo. It even has a publish workflow so you can work on posts over time, rather than having to write everything down in one setting. If you setup the workflow option, whenever you "save" Netlify's CMS pushes a commit to the PR that it opened.

To get started, you just need to enable Netlify to have access to your project's GitHub repo, and then add an associated yaml file for Netlify to read to understand how to work with your site. To get this setup with Next.js is super easy, check out Netlify's tutorial here.

The end product of using both Netlify CMS and Hosting is this:

The best part is that all of this is free and I still control all the source content. So you can actually edit posts directly in your project or through the editor.

Moving Posts from WordPress to Here

I had over 50 posts on my previous WordPress site that I had to move over. To do this, I used a little bit of JavaScript magic and leveraged the npm rss-parser package and turndown to basically read the RSS feed from my old site and make new markdown files. Once the RSS feed was converted into markdown, I used RegEx to replace punctuation and a few things I saw when initially testing the program. Then I used the Node.js "fs" package to write markdown files locally line by line. This was important since I'm also using Netlify's CMS, I'd need each post to have some metadata at the top of each markdown file. Despite leveraging all of this automation, I still ended up having to take an afternoon of walking through all the posts and fixing indenting errors and various image issues. That being said, this little bit of automation really helped. Here's the program I wrote to make it happen:

const Parser = require("rss-parser");
const parser = new Parser();
const fs = require("fs");
const TurndownService = require("turndown");
const turndownService = new TurndownService();
const striptags = require("striptags");

const callRSS = async (feedSource) => {
  const feedOutput = await parser.parseURL(feedSource);
  const output = feedOutput.items;
  return output;
};

const parseOutput = async () => {
  const feedSource = "https://rhythmandbinary.com/rss";
  const output = await callRSS(feedSource);
  return output;
};

const retrievePostsSaveMarkdown = async () => {
  const output = await parseOutput();

  output.forEach((post) => {
    // fileName
    const blankSpace = " ";
    const blankSpaceRegEx = new RegExp(blankSpace, "g");
    let fileName = post.title;
    fileName = fileName.replace(blankSpaceRegEx, "_");
    const singleColon = ":";
    const singleColonRegEx = new RegExp(singleColon, "g");
    fileName = fileName.replace(singleColonRegEx, "");
    fileName = fileName.replace(/,/g, "");
    fileName = fileName.replace(/\./g, "");
    fileName = fileName.replace(/[(.)]/gm, "");
    fileName = fileName.replace(/[(?)]/gm, "");
    fileName = fileName.replace(/[(!)]/gm, "");

    // title
    let title = post.title;
    title = title.replace(singleColonRegEx, "");

    // postDate
    const postDate = post.pubDate;

    // snippet
    let snippet = striptags(post["content:encoded"]);
    if (snippet.length > 200) {
      snippet = snippet.substring(0, 200);
    }
    const singleQuote = /(&#8217;)/gm;
    snippet = snippet.replace(singleQuote, "'");
    const leftQuote = /(&#8220;)/gm;
    snippet = snippet.replace(leftQuote, "");
    const rightQuote = /(&#8221;)/gm;
    snippet = snippet.replace(rightQuote, "");
    const imageSource = /(\(image source\))/gm;
    snippet = snippet.replace(imageSource, "");
    const multiSpace = /[\n]{2,}/gm;
    const multiSpaceRegEx = new RegExp(multiSpace, "g");
    snippet = snippet.replace(multiSpaceRegEx, "");

    // content
    const content = turndownService.turndown(post["content:encoded"]);

    fs.appendFileSync(`${__dirname}/../_posts/${fileName}.md`, "--- \n");
    fs.appendFileSync(
      `${__dirname}/../_posts/${fileName}.md`,
      "layout: blog \n"
    );
    fs.appendFileSync(
      `${__dirname}/../_posts/${fileName}.md`,
      `title: ${title} \n`
    );
    fs.appendFileSync(
      `${__dirname}/../_posts/${fileName}.md`,
      `date: ${postDate} \n`
    );
    fs.appendFileSync(
      `${__dirname}/../_posts/${fileName}.md`,
      `snippet: \"${snippet}\" \n`
    );
    fs.appendFileSync(`${__dirname}/../_posts/${fileName}.md`, "--- \n");
    fs.appendFileSync(`${__dirname}/../_posts/${fileName}.md`, `${content} \n`);
  });
};

retrievePostsSaveMarkdown();

Generating my own RSS Feed

As part of my new site, I knew I wanted to have an RSS feed. RSS feeds are great for followers and also for times when you want to retrieve a copy of all the content easily. To do this, I actually leveraged the getStaticProps() method that came with Next.js.

One of the coolest part's of Next.js is that it prebuilds a site to optimize performance. So that means if you use Next.js for your site, you're given a set of APIs that are called only at build time. When building static sites, this is very powerful because it lets you orchestrate data using all of the features you have on your local filesystem. So for my RSS feed, I created an "/rss" route which would retrieve all of my posts locally and build an XML file which I can then share directly form my site (see the Next.js docs on static file serving here). This ended up looking somewhat like this:

export async function getStaticProps() {
  const allPosts = getAllPosts(["title", "date", "slug", "content", "snippet"]);

  allPosts.forEach(async (post) => {
    unified()
      .use(markdown)
      .use(html)
      .process(post.content, function (err, file) {
        if (err) throw err;
        post.content = file;
      });
  });

  const XMLPosts = getRssXml(allPosts);
  saveRSSXMLPosts(XMLPosts);

  return {
    props: { XMLPosts },
  };
}

The specific file saving looks like this:

export function saveRSSXMLPosts(XMLPosts) {
  const publicDirectory = join(process.cwd(), "public/rss.xml");
  fs.writeFileSync(publicDirectory, XMLPosts);
}

To do the actual building of the RSS feed I had to do a fair amount of googling. I found this post super helpful in getting setup. I also used the W3C official RSS feed site to verify that my XML file was compliant with the standard (it is!).

Custom Navigation

I also built my own paging with custom navigation. I highlight this because it took quite a while to get the paging to work correctly. I know there are a lot of packages that have paging components, but I wanted to write my own. Doing this took a little bit of magic to manage finding the location and "page" of posts. I also made it responsive so that it looked good both in mobile and desktop views. To actually build my pages, I also used the "getStaticProps()" method that I mentioned earlier, and then built a local array that the main page on the site uses. The end result was a nice little set of page buttons:

Liking Posts

With my new site, I wanted a way for readers to "like" posts. So to do this I actually built a whole API with Firebase to save likes for each post. This was pretty straightforward. Just go to the little heart at the bottom of each post to see it in action.

Some Other Cool Features

In addition to the features I've highlighted:

I added a search feature that searches both the titles and content of my posts (using the standard JS filter method)
I made the site fully responsive. Special thanks to Mrs. Natalie Evans (my wife and best post reviewer around) for doing design reviews with me.
I made use of SCSS maps wherever possible. This was super helfpul as I was building the site's theme and also making it responsive.
Setup Google Analytics with regular reports. Next.js had a great example in their GitHub repo that got me going.

Closing Thoughts

So I will be using this new site as my primary blog platform from now on. Building my own site was quite an undertaking, but it is really cool to be able to say you built your own setup. From here, just follow this site and stay tuned for new posts. I have a Twitter feed that you can follow for any new posts I write (check it out here). You can also feel free to contact me on andrewevans.dev. Thanks again for reading my post!

Video Conferencing

Andrew Evans — Mon, 23 Mar 2020 12:25:08 +0000

In light of everything going on with the Coronavirus, I thought it might be helpful to do a short post covering different video conferencing applications. Many folks are working remotely, and it would help to have a quick rundown of some of the tools available. This list is by no means exhaustive, but hopefully will get you started with your home or business needs.

Apple FaceTime

(photo copied from Apple’s site here )

This is a given if you’re an Apple user. It is very straightforward and built into iOS. You can turn it on with your phone or other Apple device by going settings –> FaceTime.

You can call people directly or with their email. You can even use your Memoji as your avatar when talking.

I’ve had a great experience with this, and this is great for both individual and group chats.

For more info go to the official Apple support page here. If you want to do a group chat, then go to the link here for more info.

Zoom

(image originally copied from here)

I’ve used Zoom both in personal and professional settings. It has a super intuitive UI and works very well as long as you have a stable internet connection.

To get started just go to https://zoom.us/meetings and create a free account. Then you can either just use the browser or install desktop apps for your platform.

If you go over to the plans section you’ll see that you can get a lot with a free plan including:

Unlimited 1 to 1 meetings
40 mins limit on group meetings
Application Sharing
Custom Backgrounds (super cool feature if you turn it on)
Recording of video and audio if you enable it
Many other online collaboration options

The only real challenge with Zoom is that its only free for 1 on 1 calls. If you want to do a group chat (more than 2 people) then you only get 40 mins free.

I personally prefer Zoom if I’m meeting with someone one on one or providing IT support. The features are very intuitive and its easy to get up and running in no time.

For more on Zoom go here:

Google Hangouts

(image was copied from here)

If you have a google account (gmail, etc.) then you’re already setup to use Google Hangouts.

Hangouts is a great feature because its already built in to Google products, and there are associated apps for your phone, tablet, etc.

To get started, just go to Chrome and add the Hangouts extension. You can also just pull up Gmail on your browser and you’ll see it already integrated into the lefthand navigation.

Then you can directly message any of your friends that have a Google Email.

Google Hangouts has a lot of cool options that are absolutely free:

You can have group chats with up to 150 people
You can share photos, videos, GIFs, etc.
Record any conversation by turning on conversation history
Connect it to Google voice for even more features

For more info on how to get started, I recommend checking out these pages:

BlueJeans

(image was copied from here)

I was relatively new to BlueJeans until I used it recently. BlueJeans provides a dedicated desktop app as well as a web interface for meetings.

Bluejeans is a paid application that offers several collaboration features including screen and application sharing.

BlueJeans doesn’t have a free option (other than free trial offers). However, with the subscription you do get:

unlimited meeting time lengths
unlimited 1 to 1 meetings
unlimited group meetings
Most of the same features as the other products

BlueJeans also has integration offerings with Microsoft Teams to use in your professional workspace. You can connect conference rooms to Microsoft teams using the gateway product at https://www.bluejeans.com/products/integrations/microsoft-gateway.

To get started with Bluejeans, please check out their plans site at https://store.bluejeans.com/.

Microsoft Teams

(image was copied from here)

If you’re a big user of Office 365, then Microsoft Teams would be a natural product for you.

Microsoft Teams offers most of the same type of services that the other products I’ve mentioned cover. You can share screens, files and do general collaboration.

Microsoft Teams has both free and paid versions. The advantage of using Microsoft Teams is if you or your team is already integrated with Microsoft products.

Microsoft Teams offers a web interface and desktop applications.

To get started with Microsoft Teams I recommend you check out the following:

Wrapping Up

I hope the information I’ve provided here gives you a starting point for knowing which Video Conferencing software to use. There are a lot of options here and beyond the list I’ve provided. The biggest thing is to determine what solution works for your project or team. It also helps to consider what software applications that you want to share or collaborate with.

Thanks for reading my article! Follow me on twitter at @AndrewEvans0102!

Getting Started with RxJS

Andrew Evans — Wed, 12 Feb 2020 17:33:05 +0000

If you’re just starting or a seasoned JavaScript developer, chances are you’ve heard of RxJS.

RxJS is one of the most popular JavaScript Libraries that exists today. This post is going to cover a basic walkthrough of what it is, and how you can use it in your applications.

History

So before I begin, it helps to get a understanding of the history behind RxJS.

It all started with Reactive Extensions (or ReactiveX). ReactiveX was a concept that was originally invented byErik Meijer. It was an implementation of the Observer Pattern. After it was developed, subsequent programming Libraries were developed around the major languages like .NET and JavaScript.

RxJS is the implementation of Reactive Extensions for JavaScript. The RxJS project was originally started by Matthew Podwysocki and others as independent open source project. Starting around the time of RxJS version 5, Ben Lesh and others improved the project to be more of what it is today.

The RxJS Library implements both the Observer Pattern and the Iterator Pattern.

The RxJS library also uses Functional Programming to implement operators and functions for managing sequences of events (streams). For a great intro to Functional Prgoramming I highly recommend watching Russ Olsen’s video at GOTO 2018.

Imperative vs Declarative

When you hear people discuss RxJS, you commonly will hear them referring to imperative and declarative coding.

Imperative refers to code that you write in a specific way. This is code that you have manually piped the control flow similar to the way Promises work.

Declarative refers to using declared functions to perform actions. Here you rely upon “pure” functions that can define an event flow. With RxJS you see this in the form of observables and operators.

These definitions will be more apparent to you later on in this post, but its good to introduce them here.

Observables

When explaining RxJS it typically is easiest to do by showing code first.

Most people are typically familiar with a Promise implemented as follows:

Nothing super exciting here, just using the standard “resolve/reject” syntax. After the promise completes the output message is written to the console.

Now compare that to this:

Woah! What’s that? Well that’s RxJS! If you notice, the declarative practice is being used as the observable is first defined, and then the different hooks in the observer are used with next , error, and complete.

I’m going to come back to this example later in this article, but just wanted to introduce it first.

How does RxJS work?

So to start with RxJS, it helps to first have a few definitions:

Observable = a defined stream of events
Subscription = represents the actual execution flow of events (initiating a subscription basically “turns on” the execution)
Operators = are “pure” functions that can invoke flows with subscriptions. These have different forms that can either create a stream or reproduce a stream in a pipeable flow.
Subject = an event emitter that can be used for multicasting. These are special and used so that you can essentially inject emitters in your programs.
Schedulers = these help with concurrency and are really a more advanced RxJS topic. I’m just including it here for completeness.

For more info and examples please refer to the official RxJS Getting Started Guide here.

So with that vocabulary introduced, now we can formally discuss the example I introduced earlier.

Observables (again)

So lets go back to the code that I showed before:

This is a great example because it shows you an implemented Observable.

If you notice first you define the Observable with next and complete. Then when I start the execution flow with the subscribe I include definitions for what to do with the execution flow:

next = does a console.log of what is returned from the stream
error = does a console.log if an error occurs in the stream
complete = writes done to the console when execution is finished

This is one way to define an observable directly. Each observer has the three hooks of next, error, and complete that you can use to define execution behavior.

Operators

Obserables are great, but RxJS also offers operators which make defining observables much easier.

With operators there are two types:

creation operators = generated observables with predefined behavior
pipeable operators = observables that return other observables using the syntax “.pipe”

Here’s a creation operator in action:

Here we are using the of operator to emit values of 10, 20, and 30 in a sequence. This is super basic, but gives you an idea of how you could use this to emit a set of values in a stream without needing to manually define the observer hooks.

Here’s a pipeable operator in action:

So here, its a little more complicated, but I think you can figure it out.

We’re using the creation operator of that I referenced before to generate a stream of values a, b, c,
Next we take the output from of into the pipeable operator mergeMap
Then we’re letting mergeMap create a new observable and pipe it into interval
Then interval takes the output and console.log each value after a 1 second delay

So basically this creates a flow with the pipeable operators. The original source observable is used to recreate a new observable with added logic.

An easier way to think of this is as a stream is being defined here. Each pipe that is used with the stream adds value.

A more literal way to think of pipeable operators is as water flows through a set of pipes. Each pipe adds value to the water until it exits the flow.

Visually you can see this flow in the following diagram:

For a better visual of flows with RxJS, many people rely on Marble Diagrams. They’re a little daunting at first, but if you learn how to read them with the RxJS Guide they really help. I recommend checking out the RxJS guide at the bottom of the operators page here.

Subscriptions and Memory Leaks

So one big challenge that developers run into with RxJS is memory leaks with subscriptions.

Memory leaks are when you’ve forgotten to “unsubscribe” from a stream, and the process continues to run eating up your memory. Memory leaks can quickly eat up your browsers memory and slow down your application.

The best solution is to always make sure you have a .unsubscribe for your observables. You also can rely on prebuilt mechansims in frameworks like Angular’s async pipe.

Here’s some code that creates a memory leak:

This code can easily be fixed by adding a setTimeout that unsubscribes from the stream after a set interval of time like so:

I actually contributed an article on the RxJS DEVTO blog on this topic titled “Common Approaches to Handling Subscriptions” here. I highly recommend checking out my post when you have some time.

Advanced Topics

So far we’ve just covered some basic execution flows. If you combine RxJS operators, you can define some pretty complicated work in just a small amount of code.

The challenge happens when you create a group of Observables from a single Observable. This is called a Higher Order Observable. RxJS has operators that help you with flattening these situations to include:

I’m not going to dive into a deep example of Higher Order Observables here because I think it goes beyond the scope of an introductory article. I do, however, highly recommend checking out the RxJS Guide on operators that discusses this in more detail.

I also cover a more advanced topic in my RxJS DEVTO blog site post “From Promises to Observables” here. In that case I walkthrough setting up a scan operator to combine several HTTP calls.

I recommend reading posts on the RxJS DEVTO blog site for more advanced Observable cases and discussion as well.

Closing Remarks

I hope this post has helped you with learning RxJS. In this post I walked through what RxJS is, and ways you can use it in your programs.

Thank you for reading this post! Follow me on Twitter at @AndrewEvans0102, and check out what I’m working on at andrewevans.dev

Angular, Jest, Firebase, and ReyRey!

Andrew Evans — Mon, 06 Jan 2020 17:19:39 +0000

Rey after her latest restaurant adventure to our local Starbucks.

I’ve written several posts in the past using Firebase and the AngularFire2 library. The AngularFire2 library makes using and integrating Firebase with your Angular applications super fun and easy.

AngularFire2 also enables you to build JAMStack applications which only require a frontend and calls to the various Firebase services (Auth, Database, etc.). After following the docs on the AngularFire2 README, you can get up and running fairly easily. From there its just a matter of injecting the different services into your Angular Components.

I recently built an Angular application that uses AngularFire2. The application also uses Jest for unit testing. I learned some in the process of building it and wanted to share for future reference.

This post is going to cover my app and some basics about setting up Jest. I’m not going to cover the initial setup of AngularFire2 since their GitHub repo covers it. I’m also not going to go over a lot about integrating Jest with Angular, except to say that I’m using an Angular Builderfor Jest in lieu of Karma. Builders are great since they let you leverage the Angular CLI. I’ll cover briefly more on those and using Jest in the first section.

ReyRey’s Restaurants

The app that I’m going to cover is called “ReyRey’s Restaurants.” You can reach it by going to https://www.reyreysrestaurants.com. The application is a fun way to keep track of restaurants you visit in your city. The project is built and hosted with Firebase, and built withAngularFire2 to connect to the authentication and database services. I made it open source and you can check out the source code on GitHub here.

Also, the reason I built the application was to have a fun way to track restaurants in my city and incorporate my cat (Rey) into one of my projects.0. I already have Chessie Choochoo for my other cat (Chestnut) so I didn’t want to leave out Rey (checkout https://www.chessiechoochoo.com).

I setup some docs so you can easily see how to use the app here. The basic premise is that you create an account, then add restaurants to a “wanna go” section. When you visit your restaurant you can “promote” it to “been there” and add a review with comments and stars etc. here are some screenshots:

Project Setup

As I mentioned in the start, the two big things with this project were AngularFire2 and Jest.

I’m not going to go into how to setup AngularFire2 with your project since the README in their repo pretty much covers it. I will, however, point you to my post on How the AngularFire Library makes Firebase feel like Magic as that has a good set of instructions to get you started.

For setting up Jest with your project, there are a few ways to do this. I found using theJest Builder here was the easiest option for me. Other than the instructions on the README, I also did the following:

created a jest.config.js file based on the recommended example from the builder Repo
added a “–runInBand” option for the Jest CLIto resolve memory errors I was seeing with my CircleCI build

The cool part about using a builder was that I was able to leverage the existing Angular CLI to do this work. So anytime I called “ng test” it would invoke the Jest test runner rather than the Karma runner that is normally the default.

After playing with it some I have to say I really liked Jest because of the following:

The error messages and warnings were easy to understand
The test runner gives you more fine grained options

I’m not really going to go into a lot about Jest because several other folks have covered this very well. I recommend reviewing the post Angular CLI: “ng test” with Jest in 3 minutes (v2). Also (event hough the article doesn’t use builders) I recommend checking out the article Integrate Jest into an Angular application and library for more on Jest with Angular. Finally, the Jest Getting Started Docs are a great place to go for examples and much more in depth information.

Testing Angularfire2 with Jest

Normally, unit testing of libraries with different services was pretty straightforward. You mock the dependencies that you need to inject and use the various hooks (beforeEach, afterEach etc.) to handle the data you’re testing.

With AngularFire2 I had a number of issues trying to mock the different libraries because of the different methods that I needed to handle for my components etc. This wasn’t documented as much as I had hoped, and required a fairly extensive amount of googling. Fortunately, I found the GitHub issue here that discusses adding docs on testing to the project repo. Within this GitHub issue this response had a great example that helped me to learn how to do this for my project.

As a small disclaimer, before I go into my tests I want to note that there are still a lot of tests that could be added. As of this writing, time constraints have limited how many I can add. I’ve built out a successful set of tests for the service classes in my project. The setup I used for these tests can be used for the components that rely on them as well.

I wrote a set of service classes that pull out the AngularFire2 services into their own classes. This made it easier because then I had more flexibility with naming and how I wanted to use AngularFire2.

The basic process to test these services is to create stub, and mock the values for the AngularFire2 library methods. These mock values then actually mock the real values that would be returned from the Firebase service methods.

For the Authentication Service I have the following setup:

const credentialsMock = {
  email: 'abc@123.com',
  password: 'password'
};

const userMock = {
  uid: 'ABC123',
  email: credentialsMock.email
};

const createUserMock = {
  user: {
    uid: 'ABC123',
    email: credentialsMock.email
  }
};

const fakeAuthState = new BehaviorSubject(null);

const fakeSignInHandler = (email, password): Promise<any> => {
  fakeAuthState.next(userMock);
  return Promise.resolve(userMock);
};

const fakeCreateUserHandler = (email, password): Promise<any> => {
  fakeAuthState.next(createUserMock);
  return Promise.resolve(createUserMock);
};

const fakeSignOutHandler = (): Promise<any> => {
  fakeAuthState.next(null);
  return Promise.resolve();
};

const angularFireAuthStub = {
  authState: fakeAuthState,
  auth: {
    createUserWithEmailAndPassword: (email: string, password: string) =>
      fakeCreateUserHandler(email, password),
    signInWithEmailAndPassword: (email: string, password: string) =>
      fakeSignInHandler(email, password),
    signOut: () => fakeSignOutHandler()
  }
};

Then in my actual test describe block I reference the angularFireAuthStub value here:

describe('AuthenticationService', () => {
  let service: AuthenticationService;
  let afAuth: AngularFireAuth;

  beforeEach(() => {
    TestBed.configureTestingModule({
      providers: [{ provide: AngularFireAuth, useValue: angularFireAuthStub }]
    });

    service = TestBed.get(AuthenticationService);
    afAuth = TestBed.get(AngularFireAuth);
  });

  afterEach(() => {
    fakeAuthState.next(null);
  });

Then in the tests themselves I just call my service methods and check for the mock and stub responses:

test('should call the create user with email and password successfully', async () => {
  const response = await service.createUserWithEmailAndPassword(
    credentialsMock.email,
    credentialsMock.password
  );
  expect(response).toBe(createUserMock.user.uid);
});

Once I had the Authentication Service up and running I next built out the tests for the Database Service. The setup was similar to the Authentication Service and had the following:

let service: DatabaseService;
let savedValues = [];

const user = {
  uid: 'ABC123',
  firstName: 'first',
  lastName: 'last',
  email: 'abc@123.com'
};

const wgRestaurant: WgRestaurant = {
  id: '1234',
  uid: 'abc123',
  name: 'name',
  link: 'link',
  description: 'description',
  recorded: 1234
};

const btRestaurant: BtRestaurant = {
  id: '1234',
  uid: '5678',
  name: 'restaurant name',
  description: 'restaurant description',
  location: 'restaurant location',
  link: 'restaurant link',
  stars: 5,
  review: 'restaurant review',
  recorded: 1234
};

const fakeAddValueHandler = (value: any): Promise<any> => {
  return Promise.resolve(savedValues.push(value));
};

const deleteAddedValueHandler = (): Promise<any> => {
  return Promise.resolve((savedValues = []));
};

const firestoreStub = {
  collection: (name: string) => ({
    doc: (id: string) => ({
      valueChanges: () => new BehaviorSubject({ foo: 'bar' }),
      set: (d: any) => fakeAddValueHandler(d)
    })
  }),
  createId: () => {
    return new Promise((resolve, reject) => resolve('1234567890'));
  },
  doc: (idFirst: string) => ({
    collection: (name: string) => ({
      doc: (idSecond: string) => ({
        valueChanges: () => new BehaviorSubject({ foo: 'bar' }),
        set: (d: any) => fakeAddValueHandler(d),
        delete: () => deleteAddedValueHandler()
      })
    })
  })
};

If you notice I’m just generically using an array whenever values are saved to the Cloud Firestore database. This obviously be customized for more fine grained testing. I was just really concerned with basic calling of the different methods here so I left it this way.

I used a beforeEach and afterEach to setup the tests as you see here:

beforeEach(() => {
  TestBed.configureTestingModule({
    providers: [{ provide: AngularFirestore, useValue: firestoreStub }]
  });

  service = TestBed.get(DatabaseService);
});

// clear out any saved values
afterEach(() => (savedValues = []));

Then, finally when the tests are actually called you just call the service methods and check for the stub and mock values:

test('should call add user successfully', async () => {
  await service.addUser(user);
  expect(savedValues.length).toEqual(1);
});

I’ve had a lot of experience with Karma, so this was my first time really playing with Jest. Overall I found it very intuitive and it was fairly easy to work with. In particular I liked the warning and messages that the CLI gave me. They really helped to workout what I needed for configuration and building tests normally etc.

Closing Thoughts

So I hope you enjoyed this post and you learned something from it as well. I really enjoy using AngularFire2 with my projects because it makes it easy to integrate Firebase into Angular apps. It was also cool to use Jest for unit testing instead of Karma has I had always done before. My project here really just covers some basics and there is a lot more you can do with both AngularFire2 and Jest. Also I hope you check out ReyRey’s Restaurants, and possibly even use it as you’re checking out local restaurants!

Thank you for reading! Follow me on Twitter at @AndrewEvans0102!