Forem: John Carroll

Surprisingly simple method to increase Firestore performance & reduce cost

John Carroll — Thu, 27 May 2021 01:57:48 +0000

This post is intended for people already familiar with Firebase's Firestore as well as RxJS observables.

Recently, I realized that I could increase performance and reduce the cost of Firebase's Firestore using a simple query cache.

Edit: having used this in production for a while, I find this technique is mainly a performance improvement and does not have a large impact on cost (possibly because I've also enabled firestore persistence which already reduces cost).

The problem

While Firebase's Firestore is pretty fast to begin with, because it doesn't support automatically joining data together (i.e. SQL joins) a common workaround is to use observables to do something like the following (example taken from rxFire docs):

import { collectionData } from 'rxfire/firestore';
import { getDownloadURL } from 'rxfire/storage';
import { combineLatest } from 'rxjs';
import { switchMap } from 'rxjs/operators';

const app = firebase.initializeApp({ /* config */ });
const citiesRef = app.firestore().collection('cities');

collectionData(citiesRef, 'id')
  .pipe(
    switchMap(cities => {
      return combineLatest(...cities.map(c => {
        const ref = storage.ref(`/cities/${c.id}.png`);
        return getDownloadURL(ref).pipe(map(imageURL => ({ imageURL, ...c })));
      }));
    })
  )
  .subscribe(cities => {
    cities.forEach(c => console.log(c.imageURL));
  });

In this example, a collection of cities is being loaded and then image data needs to be separately fetched for each city returned.

Using this method, queries can quickly slow down as one query turns into dozens of queries. In a small app I made for my nonprofit, all these queries added up over time (as new features were added) and all of a sudden you'd be waiting 5-30 seconds for certain pages to load. Any delay is especially annoying if you navigate back and forth between pages quickly.

"I just loaded this data a second ago, why does it need to load everything again?"

What I wanted to do was cache query data for a period of time so that it could be quickly reused if someone navigates back and forth between a few pages. However, without giving it much thought, it seemed like implementing such as cache would take time and add a fair amount of complexity. I tried using Firestore persistence with the hope that this would automatically deduplicate queries, cache data, and increase performance, but it didn't have as much of an impact as I'd hoped (it did reduce costs somewhat, but it also unexpectedly reduced performance).

Turns out it was really easy to create a better cache.

The solution

I implemented a simple query cache that maintains a subscription to a query for some configurable amount of time, even after all observers have unsubscribed from the data. When a component executes a new query, instead of immediately calling Firestore, I check the query cache to see if the relevant query was already created. If it was, I reuse the existing query. Else, I create a new query and cache it for the future.

The code:

import { Observable, Subject } from 'rxjs';
import stringify from 'fast-json-stable-stringify';
import { delay, finalize, shareReplay, takeUntil } from 'rxjs/operators';

/** Amount of milliseconds to hold onto cached queries */
const HOLD_CACHED_QUERIES_DURATION = 1000 * 60 * 3; // 3 minutes

export class QueryCacheService {
  private readonly cache = new Map<string, Observable<unknown>>();

  resolve<T>(
    service: string,
    method: string,
    args: unknown[],
    queryFactory: () => Observable<T>,
  ): Observable<T> {
    const key = stringify({ service, method, args });

    let query = this.cache.get(key) as Observable<T> | undefined;

    if (query) return query;

    const destroy$ = new Subject();
    let subscriberCount = 0;
    let timeout: NodeJS.Timeout | undefined;

    query = queryFactory().pipe(
      takeUntil(destroy$),
      shareReplay(1),
      tapOnSubscribe(() => {
        // since there is now a subscriber, don't cleanup the query
        // if we were previously planning on cleaning it up
        if (timeout) clearTimeout(timeout);
        subscriberCount++;
      }),
      finalize(() => { // triggers on unsubscribe
        subscriberCount--;

        if (subscriberCount === 0) {
          // If there are no subscribers, hold onto any cached queries
          // for `HOLD_CACHED_QUERIES_DURATION` milliseconds and then
          // clean them up if there still aren't any new
          // subscribers
          timeout = setTimeout(() => {
            destroy$.next();
            destroy$.complete();
            this.cache.delete(key);
          }, HOLD_CACHED_QUERIES_DURATION);
        }
      }),
      // Without this delay, very large queries are executed synchronously
      // which can introduce some pauses/jank in the UI. 
      // Using the `async` scheduler keeps UI performance speedy. 
      // I also tried the `asap` scheduler but it still had jank.
      delay(0),
    );

    this.cache.set(key, query);

    return query;
  }
}

/** 
 * Triggers callback every time a new observer 
 * subscribes to this chain. 
 */
function tapOnSubscribe<T>(
  callback: () => void,
): MonoTypeOperatorFunction<T> {
  return (source: Observable<T>): Observable<T> =>
    defer(() => {
      callback();
      return source;
    });
}

I can use this cache like so:

export class ClientService {
  constructor(
    private fstore: AngularFirestore,
    private queryCache: QueryCacheService,
  ) {}

  getClient(id: string) {
    const query =
      () => this.fstore
        .doc<IClient>(`clients/${id}`)
        .valueChanges();

    return this.queryCache.resolve(
      'ClientService', 
      'getClient', 
      [id], 
      query
    );
  }
}

Now, when the ClientService#getClient() method is called, the method arguments and identifiers are passed to the query cache service along with a query factory function. The query cache service uses the fast-json-stable-stringify library to stringify the query's identifying information and use this string as a key to cache the query's observable. Before caching the query, the observable is modified in the following ways:

shareReplay(1) is added so that future subscribers get the most recent results immediately and also so that a subscription to the underlying Firestore data is maintained even after the last subscriber to this query unsubscribes.
Subscribers to the query are tracked so that, after the last subscriber unsubscribes, a timer is set to automatically unsubscribe from the underlying Firestore data and clear the cache after a user defined set period of time (I'm currently using 3 minutes).
delay(0) is used to force subscribers to use the asyncSchedular. I find this helps keep the UI snappy when loading a large dataset that has been cached (otherwise, the UI attempt to synchronously load the large data which can cause stutter/jank).

This cache could be further updated to allow configuring the HOLD_CACHED_QUERIES_DURATION on a per-query basis.

Conclusion

This simple cache greatly increases performance and potentially reduces costs if it prevents the same documents from being reloaded again and again in rapid succession. The one potential "gotcha" is if a query is built using Date arguments. In this case, you need to be careful about using new Date() as an argument to a query since this would change the cache key associated with the query on every call (basically, this would prevent the cache from ever being used). You can fix this issue by normalizing Date creation (e.g. startOfDay(new Date()) using date-fns).

Hope this is helpful.

Awesome Forms with Solidjs

John Carroll — Tue, 27 Apr 2021 18:56:15 +0000

Editors note 6/7/22: I overhauled this article in response to library improvements.

I recently started falling in love with Solidjs, a javascript library that looks like React but is significantly faster and, dare I say, has a notably better API. Unlike React, Solidjs component functions are invoked only once when the component is initialized and then never again.

I decided to take advantage of Solidjs' strengths, and build a 3.6kb min zipped library to aid with user input forms: solid-forms. Let's dive in and see what we can do (note, if you want an introduction to Solidjs, start here).

Let's create a simple TextField component in typescript.

import { IFormControl, createFormControl } from 'solid-forms';

export const TextField: Component<{
  control: IFormControl<string>,
  label: string,
  placeholder?: string,
}> = (props) => {
  return (
    <label>
      <span class='input-label'>{props.label}</span>

      <input
        type="text"
        value={props.control.value}
        oninput={(e) => {
          props.control.markDirty(true);
          props.control.setValue(e.currentTarget.value || null);
        }}
        onblur={() => props.control.markTouched(true)}
        placeholder={props.placeholder}
      />
    </label>
  );
};

This component tracks whether it has been touched by a user (notice onblur callback) and whether it has been changed by a user (oninput). When a user changes the value, we mark the control as dirty to track that the value has been changed by the user. We also have the ability to set a label on the input as well as a placeholder. Pretty straightforward stuff.

But text field's are rarely used in isolation. We want to build a component to collect some address information. This will involve asking for a Street, City, State, and Postcode. Lets use our TextField component to create our AddressForm.

import { withControl, createFormGroup, createFormControl } from 'solid-forms';

const controlFactory = () => 
    createFormGroup({
      street: createFormControl<string | null>(null),
      city: createFormControl<string | null>(null),
      state: createFormControl<string | null>(null),
      zip: createFormControl<string | null>(null),
    });

export const AddressForm = withControl({
  controlFactory,
  component: (props) => {
    const controls = () => props.control.controls;

    return (
      <fieldset classList={{
        "is-valid": props.control.isValid,
        "is-invalid": !props.control.isValid,
        "is-touched": props.control.isTouched,
        "is-untouched": !props.control.isTouched,
        "is-dirty": props.control.isDirty,
        "is-clean": !props.control.isDirty,
      }}>
        <TextField label="Street" control={controls().street} />
        <TextField label="City" control={controls().city} />
        <TextField label="State" control={controls().state} />
        <TextField label="Postcode" control={controls().zip} />
      </fieldset>
    );
  },
});

Note that the address form is wrapped with withControl(), a higher order component. This streamlines the process of creating reusable form components.

We want our AddressForm to use a FormGroup control rather than the default FormControl so we provide a controlFactory function which initializes the control.

const controlFactory = () => 
    createFormGroup({
      street: createFormControl<string | null>(null),
      city: createFormControl<string | null>(null),
      state: createFormControl<string | null>(null),
      zip: createFormControl<string | null>(null),
    });

export const AddressForm = withControl({
  controlFactory,
  component: (props) => {
    // continued...

All we needed to do to connect our AddressForm control to the TextField's control was to use the control={/* ... */} property to specify which FormControl on the parent should be connected with the child TextField.

const controls = () => props.control.controls;
// ...
<TextField label="Street" control={controls().street} />
<TextField label="City" control={controls().city} />

We also set the component up to apply css classes based on if the AddressForm is valid/invalid, edited/unedit, and touched/untouched.

Say we want to hook our AddressForm component into a larger form. That's also easy!

// factory for initializing the `MyLargerForm` `FormGroup`
const controlFactory = () => 
    createFormGroup({
      firstName: TextField.control(),
      address: AddressForm.control(),
    });

// the form component itself
export const MyLargerForm = withControl({
  controlFactory,
  component: (props) => {
    const controls = () => props.control.controls;

    // because we can
    const lastNameControl = createFormControl<string | null>(null);

    return (
      <form>
        <fieldset>
          <TextField label="First name" control={controls().firstName} />
          <TextField label="Last name" control={lastNameControl} />
        </fieldset>

        <AddressForm control={controls().address} />
      </form>
    );
  },
});

And, with just a few steps, we have a very powerful, very composible set of form components. As changes happen to the TextField components, those changes flow upwards and automatically update the parent FormGroup components.

You may have noticed that we did something new in the controlFactory function.

const controlFactory = () => 
    createFormGroup({
      firstName: TextField.control(),
      address: AddressForm.control(),
    });

We could have defined this controlFactory function the same way we defined the others (i.e. using createFormControl() and createFormGroup()), but withControl() helpfully adds the control's controlFactory function to a special control property on a component created using withControl(). This DRYs up our code and allows easily creating FormControl's for any components you create withControl().

So what if we want to listen to control changes in our MyLargerForm component? Well that's easy. For example, to listen to when any part of the form is touched, we can simply observe the isTouched property inside a Solidjs effect.

createEffect(() => {
  if (!props.control.isTouched) return;

  // do stuff...
});

To listen to when the "firstName" control, specifically, is touched

createEffect(() => {
  if (!props.control.controls.firstName.isTouched) return;

  // do stuff...
});

Here's a more complex, advanced example: if we want to listen for value changes, debounce the rate of changes, perform validation, and mark the control as pending while we wait for validation to complete, we can do the following. Note, when we set errors on the firstName control, that will result in the "First name" TextField being marked as invalid (score!).

import { myCustomValidationService } from './my-validation-service';

export const MyLargerForm = withControl({
  // ...hiding the controlFactory boilerplate...
  component: (props) => {
    const control = () => props.control;
    const controls = () => control().controls;

    // This is a simplified example. In reality, you'd want
    // to debounce user input and only run async
    // validation when it made sense rather than on every
    // change. There are multiple ways to accomplish this
    // but none of them are specific to Solid Forms.
    createEffect(async () => {
      const firstName = controls().firstName;

      firstName.markPending(true);

      const response = await myCustomValidationService(
        firstName.value
      );

      if (response.errors) {
        firstName.setErrors({ validationFailed: true });
      } else {
        firstName.setErrors(null);
      }

      firstName.markPending(false);
    });

    const onsubmit (e) => {
      e.preventDefault();
      if (control().isPending || !control().isValid) return;

      // do stuff...
    };

    return (
      <form onsubmit={onsubmit}>
        <fieldset>
          <TextField label="First name" control={controls().firstName} />
          <TextField label="Last name" control={controls().lastName} />
        </fieldset>

        <AddressForm control={controls().address} />
      </form>
    );
  },
});

This is really just scratching the surface of what you can do with solid-forms. Check out the repo to read the documentation and learn more.

Check out the repo

https://github.com/jorroll/solid-forms

Angular: How to save scroll position when navigating

John Carroll — Sun, 20 Dec 2020 11:40:43 +0000

If your app makes use of the Angular Router, you'd probably like for a user's scroll position to be remembered when they use the "back" button to navigate back to a page they were already on. In a standard "static" webpage, the browser does this for the user automatically. This doesn't happen automatically in an angular app however and there are a few reasons why.

Even for static sites, the browser will only refresh the scroll position of the <body> element. In a single page app, it's quite likely that you'd like to refresh the scroll position of other elements. For example, if you're showing an admin dashboard, maybe there are a number of divs on the page that you'd like to independently refresh the scroll position for.
Before you can refresh the scroll position of an element, you need to wait for all the data that element needs to load. For example, if you have a <my-list> component and you try to refresh the scroll position before the <my-list> items have finished loading, nothing will happen because the <my-list> component's height in the DOM will not account for all of the items that haven't been loaded.

So how can we accomplish our goal? Well, you could try the Angular Router's built in scroll position refresh functionality, but you'll probably be disappointed to learn that it only works for the page's <body> element and it also requires you to use Angular Resolvers for all of your data. In practice, I don't find it to be useful. There's also very little documentation for it.

Here's an alternative solution that's very flexible: if you're using the small IsLoadingService to manage your app's loading state (and I highly recommend that you do), then you have a simple, centralized way of checking if something is loading. This sets us up to build a ScrollPositionDirective (@service-work/scroll-position) which automatically saves an elements scroll position on router navigation, and automatically refreshes an element's scroll position after all the pieces of an element have finished loading.

This post builds off of Angular: How to easily display loading indicators. If you haven't read that post, I recommend you start there and then come back here.

Now, let's look at some code! Here are a few examples which build off of each other. You'll see that most of these examples also use the IsLoadingService from the @service-work/is-loading package, which the ScrollPositionDirective depends on.

Alternatively, here's a codesandbox demo: https://codesandbox.io/s/isloadingservice-example-ujlgm?file=/src/app/app.component.ts

Refresh the scroll position for a synchronous element

Let's start easy. We want to refresh the scroll position of an element that doesn't depend on any asynchronous data.

@Component({
  selector: 'app-lorum-ipsum',
  template: `
    <div
      swScrollPosition='lorum-ipsum'
      style="height: 35px; overflow: auto;"
    >
      <p>
        Class aptent taciti sociosqu ad litora torquent 
        per conubia nostra, per inceptos himenaeos. In 
        ultricies mollis ante. Phasellus mattis ut turpis 
        in vehicula. Morbi orci libero, porta ac tincidunt a,
        hendrerit sit amet sem.
      </p>
    </div>
  `,
})
export class LorumIpsumComponent {}

Simply by applying the [swScrollPosition] directive to the div element (seen here with the key "lorum-ipsum"), Angular will now automatically remember and refresh this element's scroll position when you navigate away from, and back to, this component (admittedly, this example component doesn't have that much text so I'd imagine most of the time everything can fit in the viewport without needing a scrollbar).

Refresh the scroll position for an asynchronous element

Let's look at a more realistic example, say we have a contact list component and we want to automatically refresh the scroll position when a user navigates back to this component.

@Component({
  selector: 'app-user-list',
  template: `
    <ul
      id="user-list"
      swScrollPosition='users-loading'
    >
      <li
        *ngFor='let user of users | async'
        [routerLink]="['/user', user.id, 'profile']"
      >
        {{ user.name }}
      </li>
    </ul>
  `,
})
export class UserListComponent {
  users: Observable<IUser[]>;

  constructor(
    private isLoadingService: IsLoadingService,
    // assume UserService is a service you've created to fetch
    // user data
    private userService: UserService,
  ) {}

  ngOnInit() {
    this.users = this.isLoadingService.add(
      // assume `UserService#getUsers()` returns `Observable<IUser[]>`
      this.userService.getUsers(),
      { key: 'users-loading' }
    );
  }
}

Similar to the previous example, the UserListComponent will automatically refresh the #user-list element's scroll position when a user navigates back to this component. Unlike in the previous example however, here the ScrollPositionDirective will wait until the "users-loading" key has finished loading (i.e. swScrollPosition='users-loading') before attempting to refresh the #user-list element's scroll position.

Refresh the scroll position for an asynchronous element and show a loading indicator while data is loading

Let's expand on the previous example. Say you want to show a loading indicator while the #user-list element is loading. Here, we'll use the Angular Material MatProgressSpinner component as our loading spinner, along with the IsLoadingPipe (i.e. swIsLoading pipe) from @service-work/is-loading.

@Component({
  selector: 'app-user-list',
  template: `
    <mat-spinner
      *ngIf="'users-loading' | swIsLoading | async; else showContent"
    >
    </mat-spinner>

    <ng-template #showContent>
      <ul
        id="user-list"
        swScrollPosition='users-loading'
      >
        <li
          *ngFor='let user of users | async'
          [routerLink]="['/user', user.id, 'profile']"
        >
          {{ user.name }}
        </li>
      </ul>
    </ng-template>
  `,
})
export class UserListComponent {
  users: Observable<IUser[]>;

  constructor(
    private isLoadingService: IsLoadingService,
    private userService: UserService,
  ) {}

  ngOnInit() {
    this.users = this.isLoadingService.add(
      this.userService.getUsers(),
      { key: 'users-loading' }
    );
  }
}

What's happening?

Behind the scenes, the ScrollPositionDirective will subscribe to the appropriate IsLoadingService loading state as specified by a string key you pass the directive. For example, if you setup the directive with swScrollPosition='users-loading', then the ScrollPositionDirective will use the IsLoadingService to subscribe to the "users-loading" loading state and wait for loading to emit false. When it does so, it will refresh any saved scroll position it has for that element.

Psyudo-code:

class ScrollPositionDirective {
  ngAfterViewInit() {
    this.isLoadingService
      .isLoading$({ key: this.key })
      .pipe(
        filter((v) => !v),
        take(1),
       )
       .subscribe(() => this.refresh())
  }
}

The ScrollPositionDirective will also subscribe to Angular Router navigation events. When the router emits a ResolveEnd event, then the directive will grab the host element's current scroll position and save it with a key derived from the provided loading key and the current URL. For advanced usage, if there are parts of your application's URL that you want ignored by the ScrollPositionDirective (e.g. specific query params), then you can provide a custom url serialization function to ScrollPositionDirective by re-providing the SW_SCROLL_POSITION_CONFIG.

Psyudo-code:

private getPositionKey(userProvidedKey: string) {
  return `${userProvidedKey}::` + this.config.urlSerializer(this.router.url);
}

The loading key (e.g. "users-loading") is also how the ScrollPositionDirective differentiates between different loading elements on the same page. And the URL is how the ScrollPositionDirective differentiates between the same element on different pages.

Conclusion

And that's pretty much it. There's some additional customization you can do for the ScrollPositionDirective. For example, you can set an optional swScrollPositionDelay which adds a delay (in milliseconds) before the scroll position is refreshed. You can also set swScrollPositionSaveMode='OnDestroy' to have the ScrollPositionDirective save its host's scroll position OnDestroy rather than OnNavigation. This is useful (and necessary) if the host component is inside of an ngIf structural directive and is being shown / hidden by logic other than page navigation.

You can check it out at: https://gitlab.com/service-work/is-loading

A proposal to improve Angular’s ReactiveFormsModule

John Carroll — Thu, 07 Nov 2019 09:25:05 +0000

This was originally published on Angular In Depth.

In the past, the AngularInDepth blog has included some very helpful articles showing how the ReactiveFormsModule in @angular/forms can make your life easier.

Today, we’re going to talk about some of the problems with the ReactiveFormsModule and discuss a proposal to fix many of these problems. The formal proposal can be found as an issue in the Angular repo #31963 (it appears to be the fastest growing issue at the moment¹). The goal of this post is to encourage feedback from the community on improving the ReactiveFormsModule and fixing some of its longstanding issues.

So you may be wondering, what issues are there with the ReactiveFormsModule? Some of the biggest issues are:

1. The module is not strongly typed

See issues #13721 #27389 #27665 #25824 #20040 #17000 #16999 #16933 relating to controls.
See issues #31801 #19340 #19329 relating to ControlValueAccessor.

2. It’s relatively complicated to display error messages, given how fundamental this task is.

See #25824 #24981 #22319 #21011 #2240 #9121 #18114.

3. It’s relatively complicated to add error messages, including interfacing with async services for validation (hence the need for different update strategies like “on blur" / “on submit").

See #31105 #29275 #26683 #23484 #20371 #17090 #13920 #9119 #6895 #19851 #18871 #10530 #6170.

4. Numerous annoyances with unfortunate API decisions.

You can’t bind a single form control to multiple inputs without ControlValueAccessor #14451
Can’t store arbitrary metadata on a control #19686
Calling reset() doesn't actually reset the control to its initial value #20214 #19747 #15741 #19251
Must call markAsTouched() / markAsUntouched() instead of simply markTouched(boolean), which is more programmatically friendly #23414 #23336
Creating custom form components is relatively complex #12248
etc. #11447 #12715 #10468 #10195 #31133

5. In addition to all the issues dealing with errors, the API does not offer low level programmatic control and can be frustratingly not extensible.

See issues #3009 #20230 related to parsing/formatting user input
See issues #31046 #24444 #10887 #30610 relating to touched/dirty/etc flag changes
See issues #30486 #31070 #21823 relating to the lack of ng-submitted change tracking
Ability to remove FormGroup control without emitting event #29662
Ability to subscribe to FormGroup form control additions / removals #16756
Ability to mark ControlValueAccessor as untouched #27315
Provide ControlValueAccessors for libraries other than @angular/forms #27672

Fundamentally, the existing AbstractControl class does not offer the extensibility / ease of use that such an important object should have. It’s unlikely that any one API could solve everyone’s problems all of the time, but a well designed API solves most peoples problems the majority of the time and can be extended to solve problems of arbitrary complexity when needed.

What follows is a proposal for a new AbstractControl API powered by a ControlEvent interface. In general, this proposal addresses issues 1, 3, 4, and 5, above. Importantly, this proposal is a completely community driven effort. The Angular team has not provided any feedback in regards to this proposal.

The Angular issue associated with this proposal can be seen here: https://github.com/angular/angular/issues/31963
The github repo for this proposal can be seen here: https://github.com/jorroll/reactive-forms-2-proposal. The repo includes working implementations of everything discussed here.
A prototype module for the proposal has been published on npm at reactive-forms-module2-proposal this is just suitable for experimentation!

The github repo also contains stackblitz examples of the proposed API in action. The stackblitz demo also contains an example compatibility directive, letting the new AbstractControl be used with existing angular forms components (such as @angular/material components).

The proposed new AbstractControl

The proposed AbstractControl class has a source: ControlSource<PartialControlEvent> property which is the source of truth for all operations on the AbstractControl. The ControlSource is just a modified rxjs Subject. Internally, output from source is piped to the events observable, which performs any necessary actions to determine the new AbstractControl state before emitting a new ControlEvent object describing any mutations which occurred. This means that subscribing to the events observable will get you all changes to the AbstractControl.

With this relatively modest change, we can accomplish a whole host of API improvements. Let’s walk through some of them by example, before looking at the ControlEvent API itself.

Alternatively, you can scroll down and skip to the “_Diving into the ControlEvent API” section, below._

Example 1

The new API is familiar for users of the old API

It’s important that the new API be very familiar to users of the existing ReactiveFormsModule, and be 100% usable by folks who don't want to use observables.

Example 2

Subscribing to nested changes

The new API allows us to subscribe to the changes of any property. When applied to ControlContainers such as FormGroup and FormArray, we can subscribe to nested child properties.

Importantly, in this example, if the address FormGroup is removed, then our subscription will emit undefined. If a new address FormGroup is added, then our subscription will emit the new value of the street FormControl.

This also allows us to subscribe to controls changes of a FormGroup/ FormArray.

Example 3

Linking one FormControl to another FormControl

Here, by subscribing the source of controlB to the events of controlA, controlB will reflect all changes to controlA.

Multiple form controls can also be linked to each other, meaning that all events to one will be applied to the others. Because events are keyed to source ids, this does not cause an infinite loop.

Example 4

Dynamically transform a control’s value

Here, a user is providing string date values and we want a control with javascript Date objects. We create two controls, one for holding the string values and the other for holding the Date values and we sync all changes between them. However, value changes from one to the other are transformed to be in the appropriate format.

Example 5

Dynamically parse user input

Manually syncing changes between controls, as shown in Example 4, above, can be somewhat of a hassle. In most cases, we just want to parse the user input coming from an input element and sync the parsed values.

To simplify this process, FormControlDirective/ FormControlNameDirective/etc accept optional "toControl", "toAccessor", and "accessorValidator" functions.

In this example, we provide a stringToDate function which receives an input string and transforms it into a javascript Date, or null if the string isn't in the proper format. Similarly, we provide a dateToString function to sync our control's Date | null values back to the input element. We also provide an optional accessorValidator function to validate the input element's strings and provide helpful error messages to the user.

Example 6

Validating the value of an AbstractControl via a service

Here, a usernameControl is receiving text value from a user and we want to validate that input with an external service (e.g. "does the username already exist?").

Some things to note in this example:

When a subscription to the usernameControl's value property emits, the control will already be marked pending .
The API allows users to associate a call to markPending() with a specific key (in this case "usernameValidator"). This way, calling markPending(false) elsewhere (e.g. a different service validation call) will not prematurely mark this service call as "no longer pending". The AbstractControl is pending so long as any key is true.
Similarly, errors are stored associated with a source. In this case, the source is 'usernameValidator'. If this service adds an error, but another service later says there are no errors, that service will not accidentally overwrite this service's error. Importantly, the errors property combines all errors into one object.

Diving into the ControlEvent API

Note: it’s important to emphasize that, for standard usage, developers don’t need to know about the existence of the ControlEvent API. If you don't like observables, you can continue to simply use setValue(), patchValue(), etc without fear. For the purposes of this post however, lets look under the hood at what is going on!

At the core of this AbstractControl proposal is a new ControlEvent API which controls all mutations (state changes) to the AbstractControl. It is powered by two properties on the AbstractControl: source and events.

To change the state of an AbstractControl, you emit a new PartialControlEvent object from the source property. This object has the interface

When you call a method like AbstractControl#markTouched(), that method simply constructs the appropriate ControlEvent object for you and emits that object from control's ControlSource (which itself is just a modified rxjs Subject).

Internally, the AbstractControl subscribes to output from the source property and pipes that output to a protected processEvent() method. After being processed, a new ControlEvent object containing any changes is emitted from the control's events property (so when a subscriber receives a ControlEvent from the events property, any changes have already been applied to the AbstractControl).

You’ll notice that only events which haven’t yet been processed by this AbstractControl are processed (i.e. !event.processed.includes(this.id)). This allows two AbstractControls to subscribe to each other's events without entering into an infinite loop (more on this later).

You can check out the github repo to see the full AbstractControl interface proposal, as well as working implementations of FormControl, FormGroup, FormArray, etc.

Now that we know a bit more about the ControlEvent API, lets look at some examples it allows…

Example 7

Syncing one FormControl’s value with another

Say we have two FormControl’s and we want them to have the same state. The new API provides a handy AbstractControl#replayState() method which returns an observable of the ControlEvent state changes which describe the current AbstractControl's state.

If you subscribe one FormControl’s source to the replayState() of another form control, their values will be made equal.

The replayState() method also provides a flexible way of "saving" a control state and reapplying all, or parts of it, later.

Example 8

Customizing AbstractControl state changes

Say you are changing a control’s value programmatically via a “service A”. Separately, you have another component, “component B”, watching the control’s value changes and reacting to them. For whatever reason, you want component B to ignore value changes which have been triggered programmatically by service A.

In the current ReactiveFormsModule, you can change a control's value and squelch the related observable emission by passing a "noEmit" option. Unfortunately, this will affect everything watching the control's value changes. If we only want componentB to ignore a values emission, we're out of luck.

With this new API, we can accomplish our goal. Every method which mutates an AbstractControl’s state accepts a meta option to which you can pass an arbitrary object. If you subscribe directly to a control's events, then we can view any passed metadata.

Here, the subscription in the ngOnInit() hook ignores changes with the myService: true meta property.

Example 9

Emitting “lifecycle hooks” from an AbstractControl

Let’s use this proposal’s FormControlDirective implementation as an example (full code can be seen in the github repo). Say you're creating a custom directive which exposes a public FormControl, and you wish to provide "lifecycle hooks" for subscribers of that FormControl.

In the specific case of the FormControlDirective, I wanted the ability for a ControlValueAccessor connected to a FormControlDirective to be notified when the "input" control of the FormControlDirective changed.

Admittedly, this is an advanced use case. But these are precisely the kinds of corner cases which the current ReactiveFormsModule handles poorly. In the case of our new API, we can simply emit a custom event from the control's source. The control won't actually do anything with the event itself, but will simply reemit it from the events observable. This allows anything subscribed to the events observable to see these custom events.

In this example, a custom ControlAccessor might want to perform special setup when a new input control is connected to MyFormControlDirective.

ControlValueAccessor

This far, we’ve focused on changes to the AbstractControl API. But some of the problems with the ReactiveFormsModule stem from the ControlValueAccessor API. While the ControlEvent API presented thus far doesn't rely on any assumptions about the ControlValueAccessor API, and it will work just fine with the existing ControlValueAccessor interface, it also allows for a big improvement to the ControlValueAccessor API.

At the risk of introducing too many new ideas at one time, lets look at how we can improve ControlValueAccessor using the new ControlEvent API...

As a reminder, the existing ControlValueAccessor interface looks like

The proposed ControlEvent API allows for a new ControlAccessor API which looks like:

With this update, the control property of a directive implementing ControlAccessor contains an AbstractControl representing the form state of the directive (as a reminder, components are directives).

This would have several advantages over the current ControlValueAccessor API:

1. Easier to implement

When the form is touched, mark the control as touched.
When the form value is updated, setValue on the control.
etc

2. Easier to conceptualize (admittedly subjective)

3. Allows a ControlAccessor to represent a FormGroup / FormArray / etc, rather than just a FormControl

A ControlAccessor can represent an address using a FormGroup.
A ControlAccessor can represent people using a FormArray.
etc

4. Very flexible

You can pass metadata tied to changes to the ControlAccessor via the meta option found on the new AbstractControl.
You can create custom ControlEvents for a ControlAccessor.
If appropriate, you can access the current form state of a ControlAccessor via a standard interface (and you can use the replayState() method to apply that state to another AbstractControl)
If appropriate, a ControlAccessor could make use of a custom control object extending AbstractControl.

Example 10

A simple example using the existing ControlValueAccessor API

As a refresher, here is a simple custom ControlValueAccessor implemented using the existing interface:

Example 11

A simple example using the proposed ControlAccessor API

Here is the same component implemented using the proposed ControlAccessor interface:

If we want to programmatically mark this ControlAccessor as touched, we can simple call this.control.markTouched(true). If we want to programmatically update the value, we can simply setValue(), etc.

Lets look at a few more advanced examples of the benefits of the new ControlAccessor API:

Example 12

An email address input with async validation

Here, we create a custom form control component for an email address. Our custom component performs async validation of input email addresses using a userService. Similarly to Example 6, we mark the component as pending and debounce user input so that we don't make too many requests to our external service.

Example 13

A form group control accessor

Here, we create a “user form” component which encapsulates the input fields for our user form. We also make use of our custom email address input component from the previous example. This control accessor represents its value using a FormGroup, something which is not possible using the current ControlValueAccessor API.

I’ll also note that, because this component is also a ControlContainerAccessor, the use of formControlName will pull directly from the app-user-form component's control property. I.e. in this case, we don't need to use a [formGroup]='control' directive inside the component's template.

Example 14

Nesting multiple form groups

Here, we utilize our custom “user form” component (created in the previous example) as part of a signup form. If the user attempts to submit the form when it is invalid, we grab the first invalid control and focus it.

Conclusion

While fixing the existing ReactiveFormsModule is a possibility, it would involve many breaking changes. As Renderer -> Renderer2 has shown, a more user friendly solution is to create a new ReactiveFormsModule2 module, deprecate the old module, and provide a compatibility layer to allow usage of the two side-by-side (including using a new FormControl with a component expecting an old ControlValueAccessor).

There is also a lot more to this proposal than what was covered here.

To take a look at the code and the current state of the proposal, as well as view stackblitz examples, head on over to the repo: https://github.com/jorroll/reactive-forms-2-proposal.
To provide your support or disapproval for the proposal, head on over to its Angular issue: https://github.com/angular/angular/issues/31963.
To provide feedback, make Pull Requests / contributions, etc, head on over to the github repo: https://github.com/jorroll/reactive-forms-2-proposal.

Things not covered: the validators API

A lot of the issues with the current FormControl API are ultimately issues with the current ValidatorFn / ValidationErrors API.

Examples include:

1. If a control is required, a [required] attribute is not automatically added to the appropriate element in the DOM.

Similarly, other validators should also include DOM changes (e.g. a maxLength validator should add a [maxlength] attribute for accessibility, there are ARIA attributes which should be added for accessibility, etc).
If you validate to make sure an input is a number, it’s appropriate to add a type="number" attribute on the underlying <input>.

2. Generating and displaying error messages is much harder than it should be, for such a fundamental part a Forms API.

Ultimately, I see these as failings of the current ValidatorFn / ValidationErrors API, and should be addressed in a fix to that API. Any such fix should be included in any ReactiveFormsModule2 and can be incorporated into this AbstractControl API, but are currently out of scope for this particular proposal.

To give your support or disapproval to the proposal:

head on over to Angular issue #31963.

Footnotes

The “fastest growing issue” statement is based off the fact that, in 3 months, the issue has risen to the second page of the Angular repo’s issues when sorted by “thumbsup” reactions. It is the only issue on the first 4 pages to have been created in 2019.

Advanced Typescript: dynamic return types without using generics

John Carroll — Tue, 15 Oct 2019 09:21:29 +0000

This post is aimed at library authors

While creating the rSchedule recurring date library, I ran into a problem. The library facilitates processing recurring events (e.g. "every Tuesday starting on 2019/10/10") and I created it to be date-library agnostic. Depending on what date library you are using, rSchedule's objects should return dates using the user's chosen date library. But how to accomplish this while maintaining proper typing?

For example, if you use the MomentDateAdapter with rSchedule, rSchedule objects should return Moment dates. The obvious way to do this is via generics. This quickly becomes annoying though, as every rSchedule type in a given application will be providing the same generic argument, over an over again. Want to create a new recurrence Rule? You'll need to do so via Rule<MomentDateAdapter>, etc. I wondered if there was a better way...

A much better approach would be if typescript realized you imported a particular date adapter, and then updated all objects to return the appropriate dates. Well, it turns out you can do this.

For example, if you use rSchedule after importing the MomentDateAdapter (once), then rule.occurrences() is typed as returning MomentDateAdapter objects for the entire application. However, if you were to use rSchedule after importing the LuxonDateAdapter, then rule.occurrences() is typed as returning luxon DateTime objects.

import '@rschedule/moment-date-adapter/setup'
import { Rule } from '@rschedule/core/generators';

const rule = new Rule();

const dates = rule.occurrences().toArray().map(({date}) => date);

// dates is typed as `Moment[]`

import '@rschedule/luxon-date-adapter/setup'
import { Rule } from '@rschedule/core/generators';

const rule = new Rule();

const dates = rule.occurrences().toArray().map(({date}) => date);

// dates is typed as `DateTime[]`

The trick comes from the ability to turn a typescript union into a typescript intersection, combined with typescript declaration merging.

For example, given the following interface:

interface MyCustomDateTypeInterface {
  std: object;
  moment: Moment;
}

We can produce the type object & Moment via

// taken from https://stackoverflow.com/a/50375286/5490505
type UnionToIntersection<U> =
  (U extends any ? (k: U) => void : never) extends ((k: infer I) => void)
    ? I
    : never;

type MyCustomDateType = UnionToIntersection<MyCustomDateTypeInterface[keyof MyCustomDateTypeInterface]> // equals `object & Moment`

From here, we can update MyCustomDateType using declaration merging to add key: value entries to MyCustomDateTypeInterface.

For example, if MyCustomDateTypeInterface started like this:

interface MyCustomDateTypeInterface {
  std: object;
}

We could update it to include moment: Moment by declaring

interface MyCustomDateTypeInterface {
  moment: Moment;
}

At that point, MyCustomDateType would update from being object to object & Moment! This is the black magic rSchedule uses to achieve generic status without using generics!

Angular: How to easily display loading indicators

John Carroll — Tue, 02 Jul 2019 21:16:30 +0000

A common task for Angular applications is showing a loading indicator during page navigation, or showing a loading indicator while POSTing data to a server, or showing a loading indicator while *something* is happening.

I recently read a great post by Nils Mehlhorn talking about various strategies that you can use to display/manage loading indicators. It's a great read, but I don't think any of the options discussed are quite as nice as using the small IsLoadingService I made for this task. Allow me to demonstrate using some examples which build off each other (some of the examples are taken from Nils' post).

Indicating that a page is loading

Let's start with page loading. Many (most?) Angular application's will make use of the Angular Router to allow navigation between sections of an app. We want to display a loading indicator while navigation is pending.

For our loading indicator example, we will use the MatProgressBar component from the @angular/material library, and display it along the top of the page.

@Component({
  selector: 'app-root',
  template: `
    <mat-progress-bar
      *ngIf="isLoading | async"
      mode="indeterminate"
      color="warn"
      style="position: absolute; top: 0; z-index: 5000;"
    >
    </mat-progress-bar>

    <router-outlet></router-outlet>
  `,
})
export class AppComponent {
  isLoading: Observable<boolean>;

  constructor(
    private isLoadingService: IsLoadingService,
    private router: Router,
  ) {}

  ngOnInit() {
    this.isLoading = this.isLoadingService.isLoading$();

    this.router.events
      .pipe(
        filter(
          event =>
            event instanceof NavigationStart ||
            event instanceof NavigationEnd ||
            event instanceof NavigationCancel ||
            event instanceof NavigationError,
        ),
      )
      .subscribe(event => {
        // If it's the start of navigation, `add()` a loading indicator
        if (event instanceof NavigationStart) {
          this.isLoadingService.add();
          return;
        }

        // Else navigation has ended, so `remove()` a loading indicator
        this.isLoadingService.remove();
      });
  }
}

In this example, we use the IsLoadingService to get our loading state as an observable, save it in the class property isLoading, and subscribe to it in our root component's template. When loading is false, the loading indicator will disappear.

The Angular IsLoadingService is a small (less than 1kb minzipped) service for angular that helps us track whether "something" is loading. We can subscribe to loading state via IsLoadingService#isLoading$() which returns an Observable<boolean> value indicating if the state is loading or not.

To indicate that something has started loading, we can call IsLoadingService#add().
When something has stopped loading, we can call IsLoadingService#remove().
IsLoadingService will emit true from isLoading$() so long as there are one or more things that are still loading.

Because we want to display a loading bar during router navigation, we subscribe to angular Router events. When a NavigationStart event is emitted, we say that loading has started (via add()). When NavigationEnd || NavigationCancel ||NavigationError event is emitted, we say that loading has stopped (via remove()).

With this code, our app will now display a loading bar at the top of the page while navigating between routes.

Note: if we called IsLoadingService#add() with a Subscription, Promise, or Observable value, we wouldn't need to call remove().
- When passed a Subscription or Promise, add() will automatically mark that the Subcription or Promise has stopped loading when it completes.
- In the case of an Observable value, add() will take(1) from the observable and subscribe to it, noting that loading for that Observable has stopped when it emits it's next value.
- More on this below...

Waiting for data

Having completed the task of adding a "page loading" indicator during router navigation, now we want to display a loading indicator when we asynchronously fetch a list of users from a service. We decide we want to trigger the same progress bar indicator that we just set up to indicate that a page is loading.

@Component({
  selector: 'user-component'
  template: `
    <list-component>
      <profile-component *ngFor="let user of users | async" [user]='user'>
      </profile-component>
    </list-component>
  `
})
export class UserComponent implements OnInit  {
  users: Observable<User[]>;

  constructor(
    private userService: UserService,
    private isLoadingService: IsLoadingService,
  ) {}

  ngOnInit() {
    this.users =
      this.isLoadingService.add( this.userService.getAll() );
  }
}

Note how, in this example, there are no additional subscriptions or variables we need to manage.

When you call IsLoadingService#add() with an Observable (or Promise or Subscription) argument, that argument is returned.

So this code...

this.users = this.isLoadingService.add(this.userService.getAll());

Is the same as this code, so far as this.users is concerned.

this.users = this.userService.getAll();

Seperately (as mentioned before), when IsLoadingService#add() is passed an Observable value, IsLoadingService will take(1) from the value and subscribe to the results. When this component is initialized, this.isLoadingService.add( this.userService.getAll() ) will be called triggering the "page loading" indicator we set up previously. When the observable returned from this.userService.getAll() emits for the first time, IsLoadingService will know that this observable has stopped loading and update the "page loading" indicator as appropriate.

If this.userService.getAll() returned a promise (or subscription), we could also pass it to this.isLoadingService.add() and achieve similar results.

Submitting a form

Next up, we want to let our users create a new User by submitting a user creation form. When we do this, we'd like to disable the form's "submit" button and style it to indicate that the form is pending. We also decide that, in this case, we do not want to display the same "page loading" progress bar as before.

One way of accomplishing this task, is the following...

@Component({
  selector: 'user-component'
  template: `
    <form [formGroup]='userForm' novalidate>
      <mat-form-field>
        <mat-label> Your name </mat-label>
        <input matInput formControlName='name' required />
      </mat-form-field>

      <button
        mat-button
        color='primary'
        [disabled]='userFormIsPending | async'
        (click)='submit()'
      >
        Submit
      </button>
    </form>
  `,
  styles: [`
    .mat-button[disabled] {
      // button pending styles...
    }
  `]
})
export class UserComponent implements OnInit  {
  userForm: FormGroup;
  userFormIsPending: Observable<boolean>;

  constructor(
    private userService: UserService,
    private isLoadingService: IsLoadingService,
    private fb: FormBuilder,
    private router: Router,
  ) {}

  ngOnInit() {
    this.userForm = this.fb.group({
      name: ['', Validators.required],
    });

    this.userFormIsPending =
      this.isLoadingService.isLoading$({ key: 'create-user' });
  }

  async submit() {
    if (this.userForm.invalid) return;

    const response = await this.isLoadingService.add(
      this.usersService.createUser(
        this.userForm.value
      ).toPromise(),
      { key: 'create-user' }
    )

    if (response.success) {
      this.router.navigate(['user', response.data.userId, 'profile']);
    }
  }
}

There are some new concepts here. For one, instead of subscribing to "default" loading state, we're subscribing to the "create-user" loading state by passing the key option to IsLoadingService#isLoading$(). In this case, userFormIsPending will ignore the page loading indicator state. It only cares about loading things added using add({key: 'create-user'}).

Next, when our userForm is submitted, we pass the form value to UsersService#createUser() which returns an observable. We transform that observable into a Promise, and await the result. We also pass this promise to IsLoadingService#add() with the key: 'create-user'.

If our mutation is a success, we navigate to the new user's profile page.

In the component template, we subscribe to userFormIsPending and, when a submit() is pending, the "submit" button is automatically disabled.

This example is pretty clean, but it still has this otherwise unnecessary userFormIsPending property. We can improve things...

Simplifying form submission with IsLoadingPipe

A quick way we can simplify the previous example is using the optional IsLoadingPipe alongside IsLoadingService. The IsLoadingPipe simplifies the task of subscribing to loading state inside a component's template.

The IsLoadingPipe is an angular pipe which recieves a key argument and returns an isLoading$({key}) observable for that key. Importantly, it plays nice with Angular's change detection. Because the IsLoadingPipe returns an observable, you should use it with the Anguilar built-in AsyncPipe.

@Component({
  selector: 'user-component'
  template: `
    <form [formGroup]='userForm' novalidate>
      <mat-form-field>
        <mat-label> Your name </mat-label>
        <input matInput formControlName='name' required />
      </mat-form-field>

      <button
        mat-button
        color='primary'
        [disabled]='"create-user" | swIsLoading | async' <!-- change is here -->
        (click)='submit()'
      >
        Submit
      </button>
    </form>
  `,
  styles: [`
    .mat-button[disabled] {
      // button pending styles...
    }
  `]
})
export class UserComponent implements OnInit  {
  userForm: FormGroup;

  constructor(
    private userService: UserService,
    private isLoadingService: IsLoadingService,
    private fb: FormBuilder,
    private router: Router,
  ) {}

  ngOnInit() {
    this.userForm = this.fb.group({
      name: ['', Validators.required],
    });
  }

  async submit() {
    if (this.userForm.invalid) return;

    const response = await this.isLoadingService.add(
      this.usersService.createUser(
        this.userForm.value
      ).toPromise(),
      { key: 'create-user' }
    )

    if (response.success) {
      this.router.navigate(['user', response.data.userId, 'profile']);
    }
  }
}

Notice, we no longer need to have the userFormIsPending property on our component.

However, we can improve this example further. In addition to simply disabling the userForm "submit" button, we'd like to add an "is-loading" css class to the button during loading. We'd also like to add an animated css spinner to the button while it is loading.

Improving form submission with IsLoadingDirective

Using the optional IsLoadingDirective alongside IsLoadingService, we can improve our previous example. The IsLoadingDirective allows us to easily style (and optionally disable) elements based on loading state.

@Component({
  selector: 'user-component'
  template: `
    <form [formGroup]='userForm' novalidate>
      <mat-form-field>
        <mat-label> Your name </mat-label>
        <input matInput formControlName='name' required />
      </mat-form-field>

      <button
        mat-button
        color='primary'
        swIsLoading='create-user' <!-- change is here -->
        (click)='submit()'
      >
        Submit
      </button>
    </form>
  `,
  styles: [`
    .sw-is-loading {
      // button styles...
    }

    .sw-is-loading .sw-is-loading-spinner {
      // spinner styles
    }
  `]
})
export class UserComponent implements OnInit  {
  userForm: FormGroup;

  constructor(
    private userService: UserService,
    private isLoadingService: IsLoadingService,
    private fb: FormBuilder,
    private router: Router,
  ) {}

  ngOnInit() {
    this.userForm = this.fb.group({
      name: ['', Validators.required],
    });
  }

  async submit() {
    if (this.userForm.invalid) return;

    const response = await this.isLoadingService.add(
      this.usersService.createUser(
        this.userForm.value
      ).toPromise(),
      { key: 'create-user' }
    )

    if (response.success) {
      this.router.navigate(['user', response.data.userId, 'profile']);
    }
  }
}

In this example, we apply the IsLoadingDirective to our "submit" button via swIsLoading='create-user'. We pass the 'create-user' key to the directive, telling it to subscribe to "create-user" loading state.

When the "create-user" state is loading, the IsLoadingDirective will automatically disable the "submit" button and add a sw-is-loading css class to the button. When loading stops, the button will be enabled and the sw-is-loading css class will be removed.

Because this is a button/anchor element, the IsLoadingDirective will also add a sw-is-loading-spinner child element to the "submit" button.

<sw-is-loading-spinner class="sw-is-loading-spinner"></sw-is-loading-spinner>

Using css, you can add a spinner animation (example here) to this element so that the button displays a pending animation during loading. All of these options are configurable. For example, if you don't want a spinner element added to buttons when they are loading, you can configure that globally by injecting new defaults or locally via swIsLoadingSpinner=false.

Triggering multiple loading indicators at once

As one last variation on the above, say we wanted to trigger both the page loading indicator as well as the "create-user" indicator during form submission. You can accomplish this by passing an array of keys to IsLoadingService#add().

In this case, you could update the previous example with...

export class UserComponent implements OnInit {
  async submit() {
    if (this.userForm.invalid) return;

    const response = await this.isLoadingService.add(
      this.usersService.createUser(this.userForm.value).toPromise(),
      { key: ['default', 'create-user'] }, // <-- change here
    );

    if (response.success) {
      this.router.navigate(['user', response.data.userId, 'profile']);
    }
  }
}

Calling IsLoadingService#isLoading$() without a key argument is the same as calling it with isLoading$({key: 'default'}), so you can trigger the "default" loading state by passing "default" as a key.

Conclusion

IsLoadingService has been created iteratively over the past year or so. At this point, I find that it greatly simplifies managing loading indicators in my apps.

You can check it out at: https://gitlab.com/service-work/is-loading

Poll: why does clicking a DEV comment link display the comment in isolation?

John Carroll — Thu, 27 Jun 2019 14:34:36 +0000

For those who aren't familiar, when you click on a comment link in DEV.to, you are taken to a page which displays only that comment in isolation (example here). If you want more context, you can (must) click VIEW POST or VIEW FULL DISCUSSION links. Medium works similarly.

Question: Do you like this UX?

To me, a comment is inherently contextual.

Is there ever a time when viewing a comment without context is desireable? When / why?
Is there a technical reason for the current UX?
- Github comment links are powered by anchor tags, so the browser simply displays the full page and scrolls to the relevant section (very intuitive).

Angular Size Observer: a library for styling DOM elements based on their size in the browser.

John Carroll — Wed, 12 Jun 2019 15:20:49 +0000

Angular Size Observer includes tools for Angular apps to monitor and react to the display size of elements in the browser. Think CSS media queries, except applying CSS based on DOM element size rather than browser screen size (tangentially related: see the CSS element query spec proposal).

The primary component of the library is a SWObserveSize directive. At its most basic, simply apply the directive to a DOM element and the directive will automatically apply a special CSS class based on the element's display width, and a second CSS class based on the element's display height.

Example:

@Component({
  selector: 'app-root',
  template: `
    <div id="the-div" swObserveSize>
      <!-- div content... -->
    </div>
  `,
})
export class AppComponent {}

Lets assume that the display size of 'the-div' is 1250 px wide and 650 px tall in the above example. SWObserveSize Directive will apply two CSS classes to 'the-div' element:

'sw-container-width-xlarge'.
'sw-container-height-small'.

When the display size of 'the-div' changes in the browser, SWObserveSize will automatically update the CSS classes of 'the-div' as appropriate.

By providing an optional SW_SIZE_OBSERVER_CONFIG config token, you can customize the CSS classes that SWObserveSize applies to elements, along with the width/height breakpoints associated with these classes.

The default config token which is automatically used if you do nothing:

const defaultSizeObserverConfig: ISWSizeObserverConfig = {
  defaultWidthClass: 'sw-container-width-xtiny',
  defaultHeightClass: 'sw-container-height-xtiny',
  widthBreakpoints: new Map([
    [1200, 'sw-container-width-xlarge'],
    [1024, 'sw-container-width-large'],
    [768, 'sw-container-width-medium'],
    [600, 'sw-container-width-small'],
    [500, 'sw-container-width-xsmall'],
    [250, 'sw-container-width-tiny'],
  ]),
  heightBreakpoints: new Map([
    [1200, 'sw-container-height-xlarge'],
    [1024, 'sw-container-height-large'],
    [768, 'sw-container-height-medium'],
    [600, 'sw-container-height-small'],
    [500, 'sw-container-height-xsmall'],
    [250, 'sw-container-height-tiny'],
  ]),
};

This config translates to, "If an element's display width is..."

>= 1200 px, apply the 'sw-container-width-xlarge' CSS class.
>= 1024 px, apply the 'sw-container-width-large' CSS class.
>= 768 px, apply the 'sw-container-width-medium' CSS class.
etc ...
Otherwise, apply the 'sw-container-width-xtiny' CSS class.

The same logic applies for the display element's height.

Learn more

You can find more information about the Angular Size Observer library, including the SizeObserverService for more advanced usage, in its repo on GitLab: https://gitlab.com/service-work/size-observer

How to share Firebase Authentication across subdomains

John Carroll — Sat, 09 Mar 2019 01:24:53 +0000

This post is intended for people who are already familiar with Firebase, Firebase Authentication, and who are using Firebase in their web apps.

If you use Firebase Authentication in your web apps, you may have run into the problem that Firebase only supports authentication with a single domain. This means that if your application experience is spread across multiple subdomains, your users must sign in to each subdomain separately. More problematically, your users must sign out of each subdomain separately.

If your applications share branding across subdomains, this could pose a security risk. It might be reasonable for a user to expect signing out of app1.domain.com to also sign them out of app2.domain.com. Many popular applications share signed in status across subdomains, e.g. Google itself.

Having spent much longer then I intended to getting single-sign-in working across subdomains, I'm writing this post so that the next person hopefully has it easier.

At a high level, this is our setup:

We have three applications at different domains.
- accounts.domain.com
- app1.domain.com
- app2.domain.com
We have three Firebase Functions
- ...cloudfunctions.net/users-signin
- ...cloudfunctions.net/users-checkAuthStatus
- ...cloudfunctions.net/users-signout

In order to sign in:

Someone navigates to the accounts.domain.com app
They provide their authentication information
That authentication information is sent to our /users-signin cloud function which verifies the information and, if valid, sets a signed __session cookie which contains the user's UID and returns a success indication to the client.
On success, the client calls the /users-checkAuthStatus cloud function which looks for the signed __session cookie, extracts the user UID, and uses the UID and the firebase-admin SDK to mint a custom auth token which it returns to the client.
When the client receives this custom auth token, it uses it to sign in using the firebase javascript SDK.

When someone navigates to one of the other apps, say app1.domain.com, the app first checks to see if the person is already signed in via Firebase Auth. If not, it calls the /users-checkAuthStatus cloud function which looks for the signed __session cookie and returns a custom auth token to the client if appropriate. The client then signs the user in using the custom auth token (if present).

If a user for app1.domain.com isn't signed in and wants to be, you send them over to accounts.domain.com and then redirect them back to app1.domain.com when sign in is complete.

In order to sign out, a client clears the local auth state by calling signOut() with the firebase-js-sdk and also calls ...cloudfunctions.net/users-signout, which clears the __session cookie. Additionally, the client needs to notify any other connected clients that the user has been signed out so that they can call signOut() using the firebase-js-sdk.

Actually making things work, with security.

That's the high level overview, but in order to actually make it work, we need to deal with some stuff like cross-site-scripting, cookies, handling provider auth, etc.

Signing in

To start off, you need to decide how to verify authentication on the server.

One possibility, is to authenticate someone on the accounts.domain.com client normally (using Firebase Auth), and then send their idToken to the server where you use the admin SDK to verify the ID token, verify the issuedAtTime associated with the ID token (e.g. make sure it was created in the last 5 minutes), and verify the provider associated with the ID token (e.g. make sure it wasn't created using a custom auth token).

Another possibility, if someone is authenticating via a provider like Facebook or Twitter, is to authenticate them using that provider's SDK, retrieve the authToken, and send the authToken to the server where you follow the provider's instructions for verifying the token on the server.

However you accomplish passing credentials to the server, if the server determines that authentication is valid, you need to set a __session cookie, which is equal to the Firebase Authentication UID associated with the user, as well as set a cross-site-scripting cookie, which we'll use to protect against cross-site-scripting attacks.

The __session cookie should be signed, secure (meaning HTTPS only) and httponly (meaning javascript can't access it). The cross-site-scripting cookie, we'll call it csst, should be secure but not signed or httponly. Instead, the csst cookie should be created using the jsonwebtoken library which will sign the token and record the token's subject (i.e. auth UID). The domain associated with both cookies should be your application's root domain (i.e. domain.com). This ensures that the cookies are shared between subdomains.

Browsers do not allow you to set cookies for another domain. This is a problem because, by default, your Firebase Functions are on a different domain from your app. You can follow these instructions to use Firebase Hosting & a custom domain for your Functions. Note also that using Firebase Hosting for your functions means that you are restricted to only reading a __session cookie on the server side (this won't impact our consumption of the csst token). Even if you set up Firebase Functions to use your custom domain though, you might still have trouble during development: in my original setup, I was hosting my app locally but deploying Firebase Functions to a special development firebase project. This meant that the domain associated with the functions was not localhost (meaning that a function couldn't set a cookie that the client could see).

There are two workarounds to this situation that I came up with.

Disable cross-site-scripting checks during development and remove the domain specification from the __session cookie. This works because the __session cookie is only read by the Firebase Functions anyway, so it's OK if the __session cookie isn't shared across subdomains (in this case, the domain associated with the __session cookie will be your Firebase Functions Domain).
Serve your functions locally during development. Firebase has a instructions for using their local-emulator in the docs.
- The local functions emulator now works with node 8+ ~~A problem with the local emulator is it only works for nodejs 6 (FYI, I found the current version of expressjs doesn't work in nodejs6).~~
- This is no longer necessary as the local functions emulator now works with node 8+ ~~Another option is to build your own express app to host your functions during development. This is the route...~~

In order to set cookies, you'll need to use an onRequest Firebase Function rather than an onCall Firebase Function. You'll also need to handle CORS and all that jazz. I'll also call out that Google Chrome has a very unexpected quirk in that it strips the set-cookie header from the response if the set-cookie header is for a different domain. I hate this quirk. I spent hours thinking the cookie wasn't being set when, in fact, it was. See this S.O. issue for more information. Another FYI, when performing CORS requests you need to specify that the request is being made "with credentials" in order for the cookies to be sent. The server also needs to specify that credentials are allows on CORS requests for the credentials to be received.

Checking auth status

Anyway, so after you set the __session cookie and csst cookies on the client. The client can now call the /users-checkAuthStatus endpoint. When doing so, the client will need to find the csst cookie, extract its token, and set the token in a Authorization: Bearer ${token} header for the request. When receiving the request, the checkAuthStatus endpoint extracts the csst token contained in the Authorization header, validates the signature on the token, and makes sure the token's subject matches the auth UID contained in the signed __session cookie. Assuming everything is valid, you use the firebase-admin SDK to mint a custom auth token and send it to the client. If things are invalid, clear any old __session / csst cookies on the client.

Finally, when the client receives one of these custom auth tokens, make sure the client signs in using SESSION auth persistence. This means that the auth state will be persisted through page refreshes, but the moment that every tab associated with a domain is closed, the auth state will be cleared. Whenever an app is initialized, you'll need to:

Check if the person is already signed in.
If not, call the /users-checkAuthStatus endpoint and, if you receive a custom auth token in response, use it to sign the user in.
- If you receive nothing, you know the user isn't signed in.

Signing out

When someone signs out, the client needs to call the /users-signout endpoint, which will clear any __session / csst cookies on the client, as well as call the signOut() method of the firebase-js-sdk. Additionally, you need to somehow ping any other apps which are open to let them know of the signout -- at which point they should call the signOut() method of the firebase-js-sdk to sign themselves out. As a reminder, if an app is closed, the firebase-js-sdk's auth state has already been cleared.

In order to ping the other open apps to tell them to signout of the firebase-js-sdk, I found the easiest method is to monitor the presence of the csst cookie. If the csst cookie disappears, you know that the person has signed out and your app should call the signOut() method of the firebase sdk.

Wrapping up

Anyway, this wraps up my overview. Getting Firebase Authentication to work across subdomains is not super straightforward, but it is doable without that much work. Unfortunately, you need to be familiar with a number of concepts such as CORS, cookies, JWTs, Firebase Authentication itself, etc.

Good luck!

Edit (5/14/21)

A few people have asked for examples (i.e. code) of a working setup. Obviously I can understand why this would be really helpful, but I don't have any plans to do this (i.e. I'm not willing to spend the time). If someone reading this puts together an example repo and pings me in a comment, I'll update this post with a link to your repo (and credit you) so that other people can benefit.
Another developer came up with a variation of this approach (which they feel is an improvement) and which you can read about here, Cross-Domain Firebase Authentication: A Simple Approach. I haven't tested this approach at all, so I'm sharing it without endorsing it (but always good to have options, right?).
Unrelated, I recently discovered that you can (pretty easily) implement a simple query cache for Firebase Firestore which increases performance and may reduce costs. You can see an overview here:
- https://dev.to/johncarroll/surprisingly-simple-method-to-increase-firestore-performance-reduce-cost-2gcg

Simple Angular "isLoading?" Service Library

John Carroll — Sun, 30 Sep 2018 11:08:55 +0000

IsLoadingService is a simple angular service for tracking whether your app, or parts of it, are loading. By subscribing to its isLoading$() method, you can easily know when to display loading indicators.

At its most basic, you can import the service into your root component and use ngIf + the AsyncPipe to show a loading indicator during page navigation.

Example:

import { IsLoadingService } from '@service-work/is-loading';

@Component({
  selector: 'app-root',
  template: `
    <mat-progress-bar
      *ngIf="isLoading | async"
      mode="indeterminate"
      color="warn"
      style="position: absolute; top: 0; z-index: 100;"
    >
    </mat-progress-bar>

    <router-outlet></router-outlet>
  `,
})
export class AppComponent {
  isLoading: Observable<boolean>;

  constructor(
    private isLoadingService: IsLoadingService,
    private router: Router,
  ) {}

  ngOnInit() {
    this.isLoading = this.isLoadingService.isLoading$();

    this.router.events
      .pipe(
        filter(
          event =>
            event instanceof NavigationStart ||
            event instanceof NavigationEnd ||
            event instanceof NavigationCancel ||
            event instanceof NavigationError,
        ),
      )
      .subscribe(event => {
        // if it's the start of navigation, `add()` a loading indicator
        if (event instanceof NavigationStart) {
          this.isLoadingService.add();
          return;
        }

        // else navigation has ended, so `remove()` a loading indicator
        this.isLoadingService.remove();
      });
  }
}

If you want to manually indicate that something is loading, you can call the loadingService.add() method, and then call the loadingService.remove() method when loading has stopped.

If you call loadingService.add() multiple times (because multiple things are loading), isLoading$() will remain true until you call remove() an equal number of times.

Internally, the IsLoadingService maintains an array of loading indicators. Whenever you call add() it pushes an indicator onto the stack, and remove() removes an indicator from the stack. isLoading$() is true so long as there are any loading indicators on the stack.

You can also pass a subscription or promise argument to loadingService.add(subscription). In this case, the loading service will push a loading indicator onto the stack, and then automatically remove it when the subscription closes / promise resolves (i.e. you don't need to manually call remove().

Advanced Usage

For more advanced scenarios, you can call add() with an options object containing a key property. The key allows you to track the loading of different things seperately. Any truthy value can be used as a key. The key option for add() is intended to be used in conjunction with key option for isLoading$() and remove().

Example:

class MyCustomComponent implements OnInit, AfterViewInit {
  constructor(
    private loadingService: IsLoadingService,
    private myCustomDataService: MyCustomDataService,
  ) {}

  ngOnInit() {
    this.loadingService.add({key: MyCustomComponent})

    const subscription = this.myCustomDataService.getData().subscribe()

    // Note, we don't need to call remove() when calling
    // add() with a subscription
    this.loadingService.add(subscription, {
      key: 'getting-data'
    })
  }

  ngAfterViewInit() {
    this.loadingService.remove({key: MyCustomComponent})
  }

  isDataLoading(): boolean {
    // the `isLoading()` method (without the `$`) returns a boolean
    // instead of an observable
    return this.loadingService.isLoading({key: 'getting-data'})
  }
}

Wrapping Up

This service is very simple, but I've found myself often recreating it in different applications so I decided to package it up and publish it. It also helps get around a few Angular quirks I've run into, by debouncing and deduplicating isLoading$ emissions which can otherwise mess up Angular's change detection.

Personally, I hook it up to a global loading indicator in my app's root component and let it automatically respond to router navigation. Additionally, when loading data from an API, I generally want to display that same app loading indicator, so I'll simply add the data fetch subscriptions with loadingService.add(subOne) + loadingService.add(subTwo). Though if I wanted to display a different loading indicator for data fetching, I could use the optional key argument when calling add() and isLoading$().

You can check it out on GitLab

rSchedule: a javascript recurring dates library

John Carroll — Mon, 03 Sep 2018 15:29:31 +0000

rSchedule is a javascript library, written in typescript, for working with recurring dates and ICal recurrences. Rules can be imported / exported in iCalendar RFC 5545 format, and Rule objects themselves adhere to the javascript iterator protocol.

Example usage:

const rule = new Rule({
  frequency: 'YEARLY',
  byMonthOfYear: [2, 6],
  byDayOfWeek: ['SU', ['MO', 3]],
  start: new Date(2010,1,7),
})

let index = 0;
for (const { date } of rule.occurrences()) {
  date.toISOString()
  index++

  if (index > 10) break;
}

rule.occurrences({
  start: new Date(2010,5,7),
  take: 5
})
  .toArray()
  .map(date => date.toISOString())

rSchedule makes use of a fairly simple DateAdapter wrapper object which abstracts away from individual date library implementations, making rSchedule date library agnostic. If your chosen DateAdapter supports time zones, rSchedule supports time zones.

StandardDateAdapter, LuxonDateAdapter, MomentDateAdapter, and MomentTZDateAdapter packages currently exists which provide a DateAdapter complient wrapper for the standard javascript Date object, as well as moment, moment-timezone, and luxon DateTime objects. Additionally, it should be pretty easy for you to create your own DateAdapter for your preferred library.

rSchedule has been coded from scratch to facilitate the creation of complex recurring schedules and there's a lot packed in. For a complete overview, check out the project on Gitlab.

Occurrence Stream Operators

Without delving too deeply into the library, I want to call out one, very cool feature which it has: occurrence stream operators.

Occurrence stream operators are inspired by rxjs pipe operators, and they allow you to combine and manipulate occurrence streams from different objects.

An example from my own app:

I want to get the volunteer schedule for a particular volunteer. A volunteer schedule is the intersection of the schedule a person signs up for and the occurrence schedule of the associated volunteer opportunity.

Put another way, a volunteer opportunity has its own occurrence schedule. Lets say: every other Tuesday as well as every Friday. Separately, someone might sign up to volunteer "every tuesday", as well as a few, specific, Fridays. The days that an individual is actually going to volunteer though, are the intersection of these two schedules.

Here's how you could build this new schedule using rSchedule:

declare const volunteerAvailability: Schedule[];
declare const opportunitySchedule: Calendar;

const volunteerSchedule = new Calendar().pipe(
    add(...volunteerAvailability),
    unique(),
    intersection({
      streams: opportunitySchedule
    })
  );

Breaking this example down:

volunteerAvailability contains an array different schedules when a volunteer has indicated that they are available.
opportunitySchedule contains a calendar describing when the volunteer opportunity actually occurs.

We want to create a calendar containing the times when the volunteer is available to participate in this volunteer opportunity (the volunteerSchedule).

const volunteerSchedule =

  // create a new calendar
  new Calendar().pipe(

    // add all times that the volunteer is available, in general
    add(...volunteerAvailability),

    // filter to get only unique times
    unique(),

    // get the intersection of these times with the
    // volunteer opportunity's schedule
    intersection({
      streams: opportunitySchedule
    })
  );

The resulting volunteerSchedule can be iterated over using volunteerSchedule.occurrences(). I can also easily group occurrences by month using volunteerSchedule.collections({granularity: 'MONTHLY'}) and iterate over the months.

Considering each volunteer that signs up will have their own schedule, I can then combine each of these schedules together to create a new calendar containing the dates which every person is scheduled to volunteer.

For example:

declare const volunteerSchedules: Calendar[];

const scheduleOfAllVolunteers =
  new Calendar({ schedules: volunteerSchedules });

I imagine most users will not need such powerful tools, and for them they can just use the provided Schedule object which implements RRULE, EXRULE, RDATE, and EXDATE from the ICAL spec.

For example:

const schedule = new Schedule({
  rrules: [
    {
      frequency: 'WEEKLY',
      start: new Date(2012, 5, 24),
      end: new Date(2012, 11, 31)
    },
    {
      frequency: 'DAILY',
      start: new Date(2011, 9, 2)
    }
  ],
  data: 'Holds anything I want',
})

schedule
  .occurrences({take: 10})
  .toArray()
  .map(date => date.toISOString())

To learn more, check out the rSchedule repo.

Forem: John Carroll

Surprisingly simple method to increase Firestore performance & reduce cost

The problem

The solution

Conclusion

Awesome Forms with Solidjs

Check out the repo

Angular: How to save scroll position when navigating

Refresh the scroll position for a synchronous element

Refresh the scroll position for an asynchronous element

Refresh the scroll position for an asynchronous element and show a loading indicator while data is loading

What's happening?

Conclusion

You can check it out at: https://gitlab.com/service-work/is-loading

A proposal to improve Angular’s ReactiveFormsModule

1. The module is not strongly typed

2. It’s relatively complicated to *display* error messages, given how fundamental this task is.

3. It’s relatively complicated to *add* error messages, including interfacing with async services for validation (hence the need for different update strategies like “on blur" / “on submit").

4. Numerous annoyances with unfortunate API decisions.

5. In addition to all the issues dealing with errors, the API does not offer low level programmatic control and can be frustratingly not extensible.

The proposed new AbstractControl

Example 1

The new API is familiar for users of the old API

Example 2

Subscribing to nested changes

Example 3

Linking one FormControl to another FormControl

Example 4

Dynamically transform a control’s value

Example 5

Dynamically parse user input

Example 6

Validating the value of an AbstractControl via a service

Diving into the ControlEvent API

Example 7

Syncing one FormControl’s value with another

Example 8

Customizing AbstractControl state changes

Example 9

Emitting “lifecycle hooks” from an AbstractControl

ControlValueAccessor

1. Easier to implement

2. Easier to conceptualize (admittedly subjective)

3. Allows a ControlAccessor to represent a FormGroup / FormArray / etc, rather than just a FormControl

4. Very flexible

Example 10

A simple example using the *existing* ControlValueAccessor API

Example 11

A simple example using the *proposed* ControlAccessor API

Example 12

An email address input with async validation

Example 13

A form group control accessor

Example 14

Nesting multiple form groups

Conclusion

Things not covered: the validators API

1. If a control is required, a [required] attribute is not automatically added to the appropriate element in the DOM.

2. Generating and displaying error messages is much harder than it should be, for such a fundamental part a Forms API.

To give your support or disapproval to the proposal:

Footnotes

Advanced Typescript: dynamic return types without using generics

Angular: How to easily display loading indicators

Indicating that a page is loading

Waiting for data

Submitting a form

Simplifying form submission with IsLoadingPipe

Improving form submission with IsLoadingDirective

Triggering multiple loading indicators at once

Conclusion

You can check it out at: https://gitlab.com/service-work/is-loading

Poll: why does clicking a DEV comment link display the comment in isolation?

Question: Do you like this UX?

Angular Size Observer: a library for styling DOM elements based on their size in the browser.

Learn more

How to share Firebase Authentication across subdomains

At a high level, this is our setup:

Actually making things work, with security.

Signing in

Checking auth status

2. It’s relatively complicated to display error messages, given how fundamental this task is.

3. It’s relatively complicated to add error messages, including interfacing with async services for validation (hence the need for different update strategies like “on blur" / “on submit").

A simple example using the existing ControlValueAccessor API

A simple example using the proposed ControlAccessor API