Forem: Phil Hardwick

A Java Library Release Process Which Doesn't Suck

Phil Hardwick — Fri, 07 Aug 2020 05:15:39 +0000

Everytime I read about Maven’s release process it makes me groan. There must be a better way than this. Something less manual, with less commits and less steps, right?

Requirements

I had some strict requirements for this release process:

no multiple commits
no commits by CI
should build snapshot versions when someone is building locally
should build snapshot versions as a default when building in CI so not every push to master is a “release”
versioning should be by semver, and shouldn’t include build numbers
shoudn’t be tied to the CI choice (i.e. no CircleCI specific release)

New solution

The best, simplest way of releasing Java apps I’ve found is via git tags and a lesser known Gradle plugin I’ve recently discovered, so I’ll show you how I’m now releasing Java libraries without the pain.

How it works

The current version of the project is determined by the most recent git tag on the branch. So if the last tag was 0.0.5, 2 commits ago, and you’ve configured the plugin to increment the patch version, it’ll be 0.0.6-SNAPSHOT. If you merge that commit to master, CI will also build 0.0.6-SNAPSHOT and publish it to your Maven repo so others can try it out. If you want to release that commit as a new version, just tag the commit with the version you want e.g. 0.0.6. CI will pick up the new git tag and re-build. It will see that the current commit has a semantically versioned tag and it will use that as the version of the project. You can also tag it with any version number e.g. 0.1.0 or 1.0.0. This keeps the developer in control of how the versions increment.

How to add it to a project

1. Add the semver git plugin

plugins {
    id "io.wusa.semver-git-plugin" version "2.2.1"
}

2. Configure the plugin

This is the simple configuration I decided on: that all branches behave the same way (incrementing the patch version when a tag isn’t on the current commit) and that the version is just major.minor.patch with a possible -SNAPSHOT suffix. More options are available as shown in the plugin README.md.

semver {
    snapshotSuffix = "SNAPSHOT"
    initialVersion = "0.0.1"
    branches {
        branch {
            regex = ".+"
            incrementer = "PATCH_INCREMENTER"
            formatter = { "${semver.info.version.major}.${semver.info.version.minor}.${semver.info.version.patch}" }
        }
    }
}

3. Use the version

Assign the version calculated by the plugin to the version of the project. I explicitly call .toString() here so it’s only evaluated once. If you have a multimodule build, the version will be evaluated for each subproject and the .toString() method will be called each time (which is where all the git commands and tag searching happens).

I also find it very useful to output the version being built for visibility.

version = semver.info.toString()
println "Building version $version of $rootProject.name"

4. Set up your CI to build when a new tag appears

This will be different based on what CI you use, but in Concourse I added a new git resource with a tag_filter: *.*.* and then added a new input to the build job which will also trigger the job:

resources:
- name: commons-utils.git-release-tags
  type: git
  check_every: 2m
  source:
    uri: git@github.com:PhilHardwick/commons-utils.git
    branch: master
    private_key: #private key config
    fetch_tags: true
    # only detect a new version when tag matches the semver pattern
    tag_filter: "*.*.*"

jobs:
  - name: build-commons-utils
    plan:
      - get: commons-utils.git
        trigger: true
        params:
          fetch_tags: true
      # use new resource as input and trigger job on new releases
      - get: commons-utils.git-release-tags
        trigger: true

Github Bonus

As a bonus, if you’re using Github, it will also pick up the tags and show them as releases on your Github repository so developers can easily check the current version.

Conclusion

I’m a big fan of this release process since it is simpler than the usual Maven releases process with multiple commits. It’s also not tied to any CI implementation, and it works the same in CI as it does locally.

What’s next? Still need to find a good way to do release notes! Maybe using conventional commit…

Refactoring Branches out of Spring Controllers with Params

Phil Hardwick — Fri, 17 Jul 2020 05:05:31 +0000

There’s a lesser known way of specifying which method in a Spring controller should be executed based on the query parameters sent to it.

Before knowing about this I would have used optional parameters and if statements inside the same handler - which increases complexity and decreases readability. For example, here I’m returning a bank account and there are two ways of searching for one, by nameOnAccount and by accountNumber.

@RestController
@RequestMapping(value = "/accounts")
public class AccountController {
    private AccountService accountService;

    public AccountController(AccountService accountService) {
        this.accountService = accountService;
    }

    @RequestMapping(method = GET)
    public ResponseEntity<AccountResource> searchForAccount(
            @RequestParam(value = "accountNumber", required = false) String accountNumber,
            @RequestParam(value = "nameOnAccount", required = false) String nameOnAccount) {
        // both params are optional so I need to manually check if neither or both exist
        if ((isBlank(accountNumber) && isBlank(nameOnAccount))
                || isNotBlank(accountNumber) && isNotBlank(nameOnAccount)) {
            // Returns a bad request
            throw new AccountSearchException(
                "Must provide either account number or name on account only.");
        }

        if (isNotBlank(accountNumber)) {
            return accountService.getAccountByAccountNumber(accountNumber);
        } else {
            return accountService.getAccountByNameOnAccount(nameOnAccount);
        }
    }
}

The above method has more if statements than necessary for a simple handler which makes it difficult to read. We can make it simpler by using the params field in the @RequestMapping annotation.

You can provide an array of strings in the params field of the @RequestMapping. This field can specify whether the request query parameters should have a specific value e.g. params = "accountNumber=12345678" or not have a specific value e.g. params = "accountNumber!=12345678" or whether it should have the query parameter present with any value e.g. params = "accountNumber" or not present e.g. params = "!accountNumber".

The example below shows how you can simplify the above code and still provide the same functionality. The other possible parameter needs to be negated so that if someone provides both query parameters they get a 400 status code back, just as they did before from the AccountSearchException.

@RestController
@RequestMapping(value = "/accounts")
public class AccountController {
    private AccountService accountService;

    public AccountController(AccountService accountService) {
        this.accountService = accountService;
    }

    @RequestMapping(method = GET, params = {"accountNumber", "!nameOnAccount" })
    public ResponseEntity<AccountResource> getAccountByAccountNumber(
            @RequestParam(value = "accountNumber") String accountNumber) {
        return accountService.getAccountByAccountNumber(accountNumber);
    }

    @RequestMapping(method = GET, params = {"nameOnAccount", "!accountNumber"})
    public ResponseEntity<AccountResource> getAccountByNameOnAccount(
            @RequestParam(value = "nameOnAccount") String nameOnAccount) {
        return accountService.getAccountByNameOnAccount(nameOnAccount);
    }
}

By using the params field we’ve been able to get rid of a number of branches in our code whilst still having handlers attached to the same url mapping.

Paging a http endpoint by recursion

Phil Hardwick — Fri, 10 Jul 2020 22:45:12 +0000

Paging the results of REST API calls is a good opportunity to show recursion and how it differs from just using loops.

Recursion requires 3 things:

a stop condition (also called a base case)
an accumulator (some number or value which changes) which is used as input into the stop condition
a method which can be called with the accumulator

Here’s an example where the goal is to get REST resources from a HTTP API. We have a client which has a method get(int offset, int limit) which gets the resource from the API and returns a Java object representation of the response. Here’s how to use that client recursively:

public List<Object> getResources() {
    List<Object> resources = new ArrayList();
    return retrievePagedResource(resources, 0);
}

private List<Object> retrievePagedResource(List<Object> resources, int offset) {
    ResourceResponse response = client.get(offset, limit);
    resources.addAll(response.getItems());
    if (response.getTotal > limit + offset) {
        retrievePagedResource(resources, limit + offset);
    }    
}

Let’s see how the above implementation of paging satisfies the requirements for recursion. The stop condition is in the implicit else of the if statement - recursion will stop when total results are less than or equal to the limit plus offset. We use the combination of the limit and offset as the accumulator and we use it as an input in the stop condition. Because the original method takes no method arguments we need a second method which takes the accumulator as an argument. This second method can then be called from the original function with a default value, 0, indicating we want to start paging from the 0th item in the available resources.

Using loops, this same example would look like:

public List<Object> getResources() {
    int limit = 50;
    int offset = 0;
    int totalResults;
    List<Object> resources = new ArrayList();
    ResourceResponse response = client.get(offset, limit);
    resources.addAll(response.getItems());
    totalResults = response.getTotal;
    while (totalResults > limit + offset) {
        offset = offset + limit;
        ResourceResponse response = client.get(offset, limit);
        resources.addAll(response.getItems());
    }
    return resources;
}

There are some clear differences - the loop implementation relies heavily on modifying variables and so since the variables are always changing it’s harder to reason about. The recursive example can be made purer by not adding items to an existing list but creating a new list every time which is concatenated with another list.

When paging, be wary of storing every object in-memory, in case there’s a lot of objects. If you are filtering the returned list, it is best to filter as each page comes in rather than gather a huge list and filter after everything has been returned.

Changing Bootstrap Theme at Runtime with CSS Variables

Phil Hardwick — Fri, 03 Jul 2020 06:40:59 +0000

Bootstrap is a well-known framework for quickly building websites and apps. The way to customise it is well documented - override the SASS variables which are provided. This allows customisation at compile-time - if you know your colours and branding when you develop the app. However, what if you want to change the theme of your Bootstrap components at runtime? For example, maybe you want to allow choice between a light or dark theme, or, as in my case, you have multiple tenants visiting your site, each with their own branding.

The Requirements

The first requirement is that the tenant branding should be stored in the app database, so it can be changed easily. Secondly I don’t want to have to redeploy the app when a new tenant is added. So that excludes adding a new a CSS file for every tenant. Dynamic theming is possible with CSS custom properties (CSS variables). You can change the values of the CSS variables in Javascript, and they will apply to the browser immediately. So, is this possible in Bootstrap?

CSS Variables Solution

I stumbled across the possibility of dynamic theming in an issue thread.

Top tip, don’t just search google for blog posts when you need ideas. Search issues in Github for the library you’re using and see if it’s been solved, answered or has a workaround.

The problem with using CSS variables in Bootstrap is that all the SASS colour functions require a colour type input - they can’t handle a string like var(--primary).

On this issue, the idea of using CSS variables to change the Bootstrap theme had been dismissed as too much work in the past but had just been reopened. A contributor to the project, johanlef, posted an idea for how to override the SASS functions to enable using hsl values assigned to CSS variables which could then be assigned to SASS variables.

Drawbacks

This way of dynamic theming uses the CSS function calc() which isn’t compatible on IE11.

How I implemented it

Firstly, take Johan’s SASS functions and put them in a file called _functions-override.scss.

Secondly, in _bootstrap-variables.scss, set your Bootstrap SASS variables to reference CSS variables:

$primary: var(--primary);

$theme-colors: (
  'primary': var(--primary)
);

$primary now references the string var(--primary), which can be set at runtime.

Thirdly, change the order of imports in your main SASS file:

// functions and mixins first
@import '~bootstrap/scss/functions';
@import '~bootstrap/scss/mixins';

// override bootstrap functions to comply with --vars
@import 'functions-override';

// Override Boostrap variables
@import 'bootstrap-variables';
// add other themes if you want
@import '~bootswatch/dist/sandstone/variables';
// Import Bootstrap source files from node_modules
@import "~bootstrap/scss/variables";
@import "~bootstrap/scss/root";
@import "~bootstrap/scss/reboot";
@import "~bootstrap/scss/type";
@import "~bootstrap/scss/images";
@import "~bootstrap/scss/code";
@import "~bootstrap/scss/grid";
@import "~bootstrap/scss/tables";
@import "~bootstrap/scss/forms";
@import "~bootstrap/scss/buttons";
@import "~bootstrap/scss/transitions";
@import "~bootstrap/scss/dropdown";
@import "~bootstrap/scss/button-group";
@import "~bootstrap/scss/input-group";
@import "~bootstrap/scss/custom-forms";
@import "~bootstrap/scss/nav";
@import "~bootstrap/scss/navbar";
@import "~bootstrap/scss/card";
@import "~bootstrap/scss/breadcrumb";
@import "~bootstrap/scss/pagination";
@import "~bootstrap/scss/badge";
@import "~bootstrap/scss/jumbotron";
@import "~bootstrap/scss/alert";
@import "~bootstrap/scss/progress";
@import "~bootstrap/scss/media";
@import "~bootstrap/scss/list-group";
@import "~bootstrap/scss/close";
@import "~bootstrap/scss/toasts";
@import "~bootstrap/scss/modal";
@import "~bootstrap/scss/tooltip";
@import "~bootstrap/scss/popover";
@import "~bootstrap/scss/carousel";
@import "~bootstrap/scss/spinners";
@import "~bootstrap/scss/utilities";
@import "~bootstrap/scss/print";
//other app specific css below

I’ve included all the Bootstrap SASS files above, but you can remove those you don’t need.

Lastly set the CSS variables based on some app state. I’m using React Helmet to change the page head and set the CSS variables in an in-line style. The below code mostly uses Johan’s code from his gist, with a few tweaks for Typescript and using it with React Helmet. I get my app state from a Redux store but this could just as easily be retrieved from React Context or other state management.

import React from 'react'
import { connect } from 'react-redux';
import { IRootState } from 'app/shared/reducers';
import { Helmet } from 'react-helmet';
import identity from 'lodash/identity'
import map from 'lodash/map'
import trim from 'lodash/trim'

const printCss = (suffix = '', convert: (string) => string = identity) => {
  return (value, property) => `--${property}${suffix ? '-' + suffix : ''}: ${convert(value)};`
}

const rgbToHsl = (red, green, blue) => {
  const r = Number(trim(red)) / 255
  const g = Number(trim(green)) / 255
  const b = Number(trim(blue)) / 255

  const max = Math.max(r, g, b)
  const min = Math.min(r, g, b)
  let h,
    s,
    l = (max + min) / 2

  if (max === min) {
    h = s = 0 // achromatic
  } else {
    const d = max - min
    s = l > 0.5 ? d / (2 - max - min) : d / (max + min)
    switch (max) {
      case r:
        h = (g - b) / d + (g < b ? 6 : 0)
        break
      case g:
        h = (b - r) / d + 2
        break
      case b:
        h = (r - g) / d + 4
        break
      default:
        break
    }
    h /= 6
  }

  h = Math.round(360 * h)
  s = Math.round(100 * s)
  l = Math.round(100 * l)

  return [h, s, l]
}

// from @josh3736 | https://stackoverflow.com/a/3732187
const colorToHsl = (color: string): any[] => {
  if (color.startsWith('#')) {
    if (color.length === 4) {
      const r = parseInt(color.substr(1, 1) + color.substr(1, 1), 16)
      const g = parseInt(color.substr(2, 1) + color.substr(2, 1), 16)
      const b = parseInt(color.substr(3, 1) + color.substr(3, 1), 16)
      return rgbToHsl(r, g, b)
    } else {
      const r = parseInt(color.substr(1, 2), 16)
      const g = parseInt(color.substr(3, 2), 16)
      const b = parseInt(color.substr(5, 2), 16)
      return rgbToHsl(r, g, b)
    }
  } else if (color.startsWith('rgba')) {
    const [r, g, b] = color.slice(5, -1).split(',')
    return rgbToHsl(r, g, b).slice(0, 3)
  } else if (color.startsWith('rgb')) {
    const [r, g, b] = color.slice(4, -1).split(',')
    return rgbToHsl(r, g, b)
  } else if (color.startsWith('hsla')) {
    return color.slice(5, -1).split(',').slice(0, 3)
  } else if (color.startsWith('hsl')) {
    return color.slice(4, -1).split(',')
  } else {
    // named color values are not yet supported
    console.error('Named color values are not supported in the config. Convert it manually using this chart: https://htmlcolorcodes.com/color-names/')
    return [0, 0, 16] // defaults to dark gray
  }
}

export const ApplyBranding = ({colors}) => {
  if (colors) {
    return (
      <Helmet>
        <style>
          {`:root {
          ${colors &&
          map(
            colors,
            printCss('', color => {
              const hsl = colorToHsl(color)
              return `hsl(${hsl[0]}, ${hsl[1]}%, ${hsl[2]}%)`
            })
          )}
          ${colors &&
          map(
            colors,
            printCss('h', color => {
              const hsl = colorToHsl(color)
              return hsl[0]
            })
          )}
          ${colors &&
          map(
            colors,
            printCss('s', color => {
              const hsl = colorToHsl(color)
              return `${hsl[1]}%`
            })
          )}
          ${colors &&
          map(
            colors,
            printCss('l', color => {
              const hsl = colorToHsl(color)
              return `${hsl[2]}%`
            })
          )}
          }`}
        </style>
      </Helmet>
    )
  } else return null
}

export const TenantAwareTheming = (props: StateProps) => {
  return <ApplyBranding colors={{
    primary: props.tenant.branding.primary,
    secondary: props.tenant.branding.secondary,
  }}/>
}

const mapStateToProps = ({tenant}: IRootState) => ({
  tenant: tenant.currentTenant
});

type StateProps = ReturnType<typeof mapStateToProps>;

export default connect(mapStateToProps)(TenantAwareTheming);

Conclusion

So really, most of this isn’t my work - but I wanted to draw attention to it since it took me so long to find it! Hopefully this can save time for someone else. Thank you, Johan, for producing this solution.

Using Micronaut Annotation Mapping to Automatically Add Security Info to Swagger Docs

Phil Hardwick — Fri, 26 Jun 2020 16:41:34 +0000

In my last post I introduced how to create a custom security rule in Mirconaut which allows you to secure an endpoint by requiring a particular permission for a resource ID. In Mettle, we wanted to automatically add some documentation to Swagger based on the custom security annotation so any person consuming the API documentation would know what permissions are necessary. This is a good example of documentation generated from code, so that it is always correct and up to date.

The OpenAPI Specification allows you to add extra information via “Specification Extensions”. In Java annotations these are specified like so:

import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.extensions.Extension;
import io.swagger.v3.oas.annotations.extensions.ExtensionProperty;

@Operation(
            extensions = {
                    @Extension(name = "extensionName",
                    properties = {
                            @ExtensionProperty(name = "name", value = "value")
                    })
            }
    )

To add these annotations automatically however, my colleague Nejc and I, used an annotation mapper to hook into the Micronaut compilation process. This is how we did it:

Dependencies

Start a new gradle project and add to your build.gradle:

implementation platform("io.micronaut:micronaut-bom:$micronautVersion")
implementation "io.micronaut:micronaut-inject"
implementation "io.micronaut:micronaut-runtime"
implementation("io.swagger.core.v3:swagger-annotations") 
implementation project(":custom-security-rule-lib")

This will add the necessary Micronaut and Swagger annotations to your classpath.

Annotation Mapper

Extend the TypedAnnotationMapper<RequiredPermission> interface and implement like:

import custom.security.rule.RequiredPermission;
import io.micronaut.core.annotation.AnnotationValue;
import io.micronaut.inject.annotation.TypedAnnotationMapper;
import io.micronaut.inject.visitor.VisitorContext;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.extensions.Extension;
import io.swagger.v3.oas.annotations.extensions.ExtensionProperty;

import java.util.Collections;
import java.util.List;

public class OpenApiAnnotationMapper implements TypedAnnotationMapper<RequiredPermission> {
    @Override
    public Class<RequiredPermission> annotationType() {
        return RequiredPermission.class;
    }

    @Override
    public List<AnnotationValue<?>> map(AnnotationValue<RequiredPermission> annotation, 
            VisitorContext visitorContext) {
        String resourceIdName = annotation.stringValue("resourceIdName").orElse(null);
        String permission = annotation.stringValue("permission").orElse(null);

        AnnotationValue<ExtensionProperty> extensionProp = AnnotationValue.builder(ExtensionProperty.class)
                .member("name", "AuthorisationDescription")
                .member("value", "Your JWT needs to have " + permission + " permissions for " + resourceIdName)
                .build();

        AnnotationValue<Extension> extension = AnnotationValue.builder(Extension.class)
                .member("name", "Security")
                .member("properties", new AnnotationValue[]{extensionProp})
                .build();

        AnnotationValue<Operation> operation = AnnotationValue.builder(Operation.class)
                .member("extensions", new AnnotationValue[]{extension})
                .build();
        return List.of(operation);
    }
}

Here I’ve decided to use TypedAnnotationMapper and have the jar which provides the custom security rule annotation on the annotation processor classpath, but it’s also possible to use NamedAnnotationMapper. NamedAnnotationMapper allows you to map the annotation without having it on the classpath - this is useful if you want to keep you annotation processor classpath small, as suggested in the Micronaut docs.

Implementing the mapping is a case of:

getting the information from the custom security rule
creating an extension property
creating an extension which holds a list of that extension property
creating an operation which holds a list of that extension
return that operation

Micronaut won’t remove the annotation you’re mapping, so your endpoint will still be secured by the security rule. It also won’t remove any existing Operation annotations, it will merge the new Operation annotation with the existing one.

Making the mapper discoverable

To make the mapper discoverable via the java service loader create a file in the resources folder called META-INF/services/io.micronaut.inject.annotation.AnnotationMapper and put the fully qualified name of the mapper inside:

custom.security.rule.openapi.OpenApiAnnotationMapper

Using your mapper in another project

Add the jar as an annotation processor, for example, in gradle add the following to your dependencies:

//to generate the openapi docsc
annotationProcessor("io.micronaut.configuration:micronaut-openapi:1.5.1")
//This shows using the jar from a multi-module project 
//if you're uploading this to a maven repo use the artefact coordinates
annotationProcessor project(":custom-security-rule-openapi")

What gets generated

When the above class and service descriptor is added to the annotation processor classpath, the annotation mapper will run before the Micronaut openapi annotation processor. This means the OpenAPI processor will parse the mapped @Extension annotation and generate the following:

openapi: 3.0.1
info:
  title: Example API
  version: v1
paths:
  /tenant/{tenantId}:
    get:
      operationId: index
      parameters:
      - name: tenantId
        in: path
        required: true
        schema:
          type: string
      responses:
        default:
          description: index default response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HttpStatus'
      x-Security:
        AuthorisationDescription: Your JWT needs to have READ_ONLY permissions for tenantId

Notice the x-Security key which holds the info specified in our AnnotationMapper.

Conclusion

Micronaut’s compilation process is really powerful and allows a lot of custom functionality to be created - this is just scratching the surface. Another way to do the above is to use a TypeElementVisitor to modify the @Operation annotation when a @RequiredPermission is present. I’ll aim put a blog post up about that to follow this one.

The code from above, including unit tests, is all in a public github repo: https://github.com/PhilHardwick/micronaut-custom-security-rule.

Custom Micronaut Security Rules

Phil Hardwick — Sat, 20 Jun 2020 10:03:23 +0000

Micronaut comes with a few useful security rules such as ip filtering, url pattern matching and an annotation inspecting rule. However, extending these by creating custom rules is also possible and is useful to lock down your application and reduce the amount of code needed for checking authorisation.

First, let’s look at customising the default rules in Micronaut security:

Customising @Secured

The usual way to secure endpoints is by using @Secured with the name of a role or roles e.g. @Secured("ADMIN"). This will look for a string of “ADMIN” in the roles key in the claims. Either

{
    "sub":"1",
    "roles":["ADMIN"]
}

{
    "sub":"1",
    "roles": "ADMIN"
}

would be allowed.

To customise this, you can either change the key the default roles finder looks in by setting the property micronaut.security.token.rolesName or implement io.micronaut.security.token.RolesFinder in a new Bean.

Custom Security Rules

To implement a new custom security rule you need to implement io.micronaut.security.rules.SecurityRule which only has one method:

SecurityRuleResult check(HttpRequest request, @Nullable RouteMatch routeMatch, @Nullable Map claims);

SecurityRule also implements Ordered so you can override the default getOrder method to define when your rule gets run.

When implementing the check method you need to return SecurityRuleResult. This is an enum with 3 values: ALLOWED, DENIED and UNKNOWN. You should return ALLOWED if your rule explicitly allows the action, DENIED if your rule specifically does not allow the action and UNKNOWN if you don’t have enough information to make a decision. Returning UNKNOWN allows the other security rules to also try make a decision, rather than immediately denying the request if there’s not enough information.

When you implement the rule you have 3 parameters passed to you: the request, the route match and claims. The request is simply the request made by the client which is being authorised. The route match is what gives you access to the method in your controller so you can get annotations or arguments. Claims is a map of the JSON from the JWT sent in the request. This can be null if there wasn’t a JWT, or it couldn’t be parsed or validated.

So when would you need a custom security rule?

Permissions on Specific Resources

If you want to implement permissions on specific resources then you need to implement a custom rule. This is useful if, for example, your system is multi-tenant and users are only allowed to manage their tenant. This is different from a list of roles because the user is only allowed to do something on a specific resource. Usually, JWTs with resource specific permissions would look like:

    {
      "sub": "",
      "https://your-domain.com/claims": {
        "123": "ADMIN",
        "234": "READ_ONLY"
      }
    }

The permissions are in a namespaced map (namespacing by domain is the conventional way to do it - Auth0 explains a bit more about this) where the keys are resource IDs. Following my example, 123 and 234 would be the IDs of two tenants. The user who asked for this JWT would be able to perform ADMIN actions on tenant with ID 123 and only view tenant with ID 234.

So how can we authorise this user in Mirconaut?

You’ll need two classes, one annotation and one SecurityRule implementation. The annotation will need to define the resourceIdName and the permission it’s looking for:

public @interface RequiredPermission {

    /**
     * The name of the parameter in the controller method which contains the 
     * resource ID this permisssion is required for
     * @return resourceIdName
     */
    String resourceIdName();

    /**
     * The permission required, e.g. READ_ONLY, ADMIN, WRITE
     * @return permission
     */
    String permission();

}

Then we define a security rule that looks for this permission and uses the values in the annotation to pull out the resource ID in the url and compare it with the claims in the token:

import io.micronaut.core.annotation.AnnotationValue;
import io.micronaut.http.HttpRequest;
import io.micronaut.security.rules.SecurityRule;
import io.micronaut.security.rules.SecurityRuleResult;
import io.micronaut.web.router.MethodBasedRouteMatch;
import io.micronaut.web.router.RouteMatch;

import javax.annotation.Nullable;
import javax.inject.Singleton;
import java.util.Map;
import java.util.Optional;

@Singleton
public class PermissionSecurityRule implements SecurityRule {
    @Override
    public SecurityRuleResult check(HttpRequest request, @Nullable RouteMatch routeMatch, @Nullable Map<String, Object> claims) {
        if (routeMatch instanceof MethodBasedRouteMatch) {
            MethodBasedRouteMatch methodBasedRouteMatch = (MethodBasedRouteMatch) routeMatch;
            if (methodBasedRouteMatch.hasAnnotation(RequiredPermission.class)) {
                AnnotationValue<RequiredPermission> requiredPermissionAnnotation = methodBasedRouteMatch.getAnnotation(RequiredPermission.class);
                // Get parameters from annotation on method
                Optional<String> resourceIdName = requiredPermissionAnnotation.stringValue("resourceIdName");
                Optional<String> permission = requiredPermissionAnnotation.stringValue("permission");
                if (permission.isPresent() && resourceIdName.isPresent() && claims != null) {
                    // Use name of parameter to get the value passed in as an argument to the method
                    String resourceId = methodBasedRouteMatch.getVariableValues().get(resourceIdName.get()).toString();
                    // Get claim from jwt using the resource ID
                    Object permissionForResource = ((Map) claims.get("https://your-domain.com/claims")).get(resourceId);
                    if (permissionForResource != null && permissionForResource.equals(permission.get())) {
                        // if the permission exists and it's equal, allow access
                        return SecurityRuleResult.ALLOWED;
                    }
                }
            }
        }
        return SecurityRuleResult.UNKNOWN;
    }
}

Annotating this rule with @Singleton means it will be picked up by Micronaut and executed as part of a list of rules. Notice how one only path in the code leads to the method returning ALLOWED and all others return UNKNOWN, meaning other rules can also be checked - if everything returns UNKNOWN the request will be rejected with a 403.

To put this rule before other rules, implement the getOrder method and return a number lower than the numbers returned from the other rules. Currently, this is the order of the rules, and the numbers returned from getOrder():

IpPatternsRule - Order number: -300
SecuredAnnotationRule - Order number: -200
ConfigurationInterceptUrlMapRule - Order number: -100
SensitiveEndpointRule - Order number: 0

You can have much more complicated JWT claims and still validate they are correct. For example, I would probably have a list of permissions in the JWT per resource ID like {"123":["READ_ONLY","WRITE","ADMIN"]} and so I would verify that the required permission is in the list. Also, unless your IDs are globally unique, I would put these in a nested object to show what resource they relate to, e.g. {"https://your-domain.com/claims": {"tenants": {"123": ["READ_ONLY"]}}}.

Conclusion

Micronaut makes it really easy to hook into the security and provides all you need to validate claims and authorise requests. All code for this, plus tests, is on Github at https://github.com/PhilHardwick/micronaut-custom-security-rule.

8 reasons you should use GitOps

Phil Hardwick — Fri, 12 Jun 2020 06:12:37 +0000

What is GitOps

Before saying why GitOps is so useful, I should explain what it is. At its most basic, GitOps is defining all the configuration for your system in Git. This includes deployments, versions of services, config and secrets. For example, if using Kubernetes, a GitOps repository will have some Kubernetes deployment files which define the images and versions to deploy. Each deployment file may also reference config maps or contain environment variables which specify the config required to run that service in each environment.

All this means you have a declarative way of defining the state of your system. It’s like saying, “I want the whole system to always reflect the settings in these files”. This is the opposite to the imperative way of defining your system where you tell you the system how to change e.g. helm deploy or helm rollback. In the imperative way, the system is a sum of all the things it has been told.

In GitOps, changes to the system are pulled in (via agents) rather than pushed in by developers, or a CI issuing commands. Agents run in the cluster, comparing the actual state of the cluster and to the state defined in Git. If there’s been changes to Git, it will apply them to the cluster.

Agents are also used in the deployment of services. New versions of services are deployed by tagging an image in the docker repository. An agent will be scanning for new tags in that repository. If the tag matches a predefined pattern then the tag gets applied to the deployment files in the Git repo and that change will be picked up and applied.

So why use GitOps?

1. Show me the config

It’s really easy to find and share the configuration of a service. I’ve had many instances when someone has asked me, “what configuration does Kafka have set?” Using GitOps doesn’t require me to look at the deployment descriptor in Kubernetes. I search in Git. When I’ve found it, I can easily share that file as a url with anyone who needs it - they don’t need special access to be able to inspect what’s running. GitOps is a great leveller, and it gives everyone the ability to see how things are running.

2. History of changes

When all changes are made through Git you have a ready-made history of all the changes to the system, including some description of what or why the changes happened in the commit messages. When something breaks, it’s then easy to look back at the last change and figure out what may have caused the problem.

3. Git revert

Git revert then becomes an instant rollback mechanism. So you can feel confident making changes knowing it can easily be undone too.

4. Using familiar tools

Git is a familiar tool to most developers and using GitOps saves having to learn a new CLI with all its syntax and options. It does still need developers to know how to define the deployment/configuration files though e.g. you need to know how to structure a Kubernetes deployment yaml file if deploying in Kubernetes - but at least you don’t have to learn how to use Helm too!

5. Declarative means rebuildable

Having all your configuration and state in a single place is the first step to making your environments rebuildable. When an environment rebuilds it doesn’t require a CI to kick in at some point and deploy all the latest apps - the agents that are already running simply apply the config that’s already in Git and the system will be exactly as it was before.

6. Git is the source of truth

I know it’s so tempting to make a sneaky change, in place, to try something out: kubectl set env .... If it works, “wonderful!”, but it’s possible you forget to apply the config to all services or add it to CI and then when there’s a cluster rebuild, or even a new deployment that change may be lost. It also works the other way. If an app breaks, you don’t need to check if any config has been manually set which is causing the problem. Git is the single source of truth for how your apps are running and that gives a warm sense of security and certainty about being able to reproduce, recreate and replicate entire systems in other environments, without any manual intervention.

7. Collaboration and sharing

Changes happen via pull requests so all changes are visible and can be discussed. GitOps is a great way to share knowledge because everyone can see how the system is being configured - no one can make changes secretly.

8. Self service

Anyone can change the state of the cluster, it just needs a pull request and some approvals. This is great because it means the organisation doesn’t rely on a few individuals to make changes, anyone can make the change. This is truly self-service. It also means people don’t need special permissions to be able to make changes. So you can lock down permissions whilst still allowing all developers to make changes via PRs.

React Map GL - How do I test this?

Phil Hardwick — Sat, 06 Jun 2020 08:01:41 +0000

How do I test this? This is a common question I ask myself when writing a new feature. Sometimes it’s not so easy and you need to truly think about what you’re testing and the level at which you’re testing (in the test pyramid).

I’m creating an app which draws layers on a map. Testing these layers are visible to the user is a difficult problem, so I’ll share how I solved it and why.

Libraries and Components

I’m using Typescript and React with react-map-gl to provide the map component. As a test framework I’m using Testcafe.

Make the map queryable

The first thing is to make the map available to the tests. This is done by using React’s ref prop. I have passed a function which takes the element and assigns it to the window. Casting the window to any is required to ensure the Typescript compiler allows us to assign a new variable on the window object.

<MapGL
    {...viewport}
    width="100%"
    height="100%"
    style={{ position: 'relative' }}
    mapStyle="mapbox://styles/streets-v11"
    onViewportChange={this._onViewportChange}
    mapboxApiAccessToken={process.env.REACT_APP_MAPBOX_ACCESS_TOKEN}
    ref={el => ((window as any).map = el)}
  >
    <div className="navigation-control-container">
      <NavigationControl showCompass={false} />
    </div>
    <MapLegend />
  </MapGL>

Querying the map for layers

Testcafe (and most ui test frameworks like Cypress or Selenium) allow you to define code which can be executed on the browser. We’ll leverage this to access the map and its layers. In Testcafe this is done using a ClientFunction. getMapLayer uses a ClientFunction by passing a function which returns a Promise. The most difficult thing about this setup was getting the transpiled code to be correct. Testcafe runs the code in the ClientFunction through Babel to allow you to use modern language features but it also means you have to structure your code quite specifically. I found it was easiest to define all called functions in the same file, pass those functions as dependencies and don’t call any other functions inside those functions (unless they’re defined in the scope of the function - like filterLayerFields).

//map-commponent.page-object.ts
import {ClientFunction, Selector, t} from 'testcafe';
import {IMapLayer} from "./map-types";

// declared so typescript allows you to access window.map from the client function
declare let map: any;

const getMapLayerAndData = (localMap, layerId): IMapLayer => {
  const filterLayerFields = layer => {
    return {
      id: layer.id,
      type: layer.type,
      paint: layer.paint,
      layout: layer.layout
    };
  };

  const layer = localMap.getMap().getLayer(layerId);
  const source = localMap.getMap().getSource(layerId);
  return {
    layer: layer && filterLayerFields(layer),
    geoJson: source && source._data
  };
}

const doOnMapOnceLoadedOrImmediately = (localMap, callback) => {
  if (localMap.getMap().loaded()) {
    callback()
  } else {
    localMap.getMap().on('idle', () => {
      callback();
    });
  }
}

const getMapLayer = layerId =>
  ClientFunction(() =>
    new Promise<IMapLayer>(resolve => {
      doOnMapOnceLoadedOrImmediately(map, () => resolve(getMapLayerAndData(map, layerId)))
    }), { dependencies: { layerId, doOnMapOnceLoadedOrImmediately, getMapLayerAndData }
  });

And the types used:

//map-types.js
import { FeatureCollection, Point, MultiPoint, LineString, 
    MultiLineString, Polygon, MultiPolygon, GeoJsonProperties } from 'geojson';

export interface IPropertyValue {
  kind: string;
  value: any;
}

export interface IPaintProperty {
  property: any;
  value: IPropertyValue;
  parameters: any;
}

export interface IPaintValues {
  'line-opacity': IPaintProperty;
  'line-color': IPaintProperty;
  'line-width': IPaintProperty;
  'line-floorwidth': IPaintProperty;
}

export interface IPaint {
  _values: IPaintValues;
  _properties: any;
}

export interface ILayer {
  id: string;
  type: string;
  paint: IPaint;
  layout: any;
}

export interface IMapLayer {
  geoJson: FeatureCollection<Point | MultiPoint | LineString | 
    MultiLineString | Polygon | MultiPolygon, GeoJsonProperties>;
  layer: ILayer;
}

Using this in your test

Now you can call this client function. As with a lot of UI testing it helps to build in wait mechanisms because it will take some time for the map to render all the layers. This will make your test less brittle. I’ve put this code in a page component, so it can be easily reused. This is the same file that has the functions above declared globally (but scoped to the file).

//map-commponent.page-object.ts
import {ClientFunction, Selector, t} from 'testcafe';
import {IMapLayer} from "./map-types";

export default class MapComponent {
  mapCanvas: Selector = Selector('.mapboxgl-canvas', { timeout: 1000 });

  private async waitForMapLayer(layerId: string, condition: (IMapLayer) => boolean) {
    let mapLayer = await getMapLayer(layerId)();
    let retries = 0;
    while (!condition(mapLayer) && retries < 20) {
      t.wait(100);
      retries++
      mapLayer = await getMapLayer(layerId)();
    }
    return mapLayer
  }

  mapLayerHasFeatures = (numberOfFeatures: number) => (mapLayer: IMapLayer): boolean => {
    return mapLayer.geoJson !== undefined && mapLayer.geoJson.features.length === numberOfFeatures;
  }

  async getLayer(layerId: string, waitUntilNumberOfFeatures: number): Promise<IMapLayer> {
    return await this.waitForMapLayer(layerId, this.mapLayerHasFeatures(waitUntilNumberOfFeatures));
  }
}

Finally, you can call this in a test:

import MapComponent from "../map-component.page-object";

test('map layer exists with correct coordinates', async t => {
  const mapComponent = new MapComponent()
  const mapLayer = await mapComponent.getLayer("my-road", 1);
  t.expect(mapLayer.geoJson.features[0].geometry.coordinates)
    .eql([[0.1872612, 51.9872986], [0.1876876, 51.9725736]])
});

Things to watch out for

You can’t currently pass a typescript class to the client function as a dependency otherwise it will hang. See this issue.

What you’re actually testing

There are other ways to test this too. I’ve seen a lot of screenshot comparing tests. Comparing screenshots is the ultimate way to definitely make sure your layers are drawn on the map. I’ve also thought about comparing the image data from a canvas and counting the number of colour pixels. However, I decided not to go either of those ways because I felt the tests would be too brittle and require too much maintenance. When using my approach I’m just testing my code and some of React Map GL’s code, then trusting mapbox to render the layers correctly on a canvas. I can imagine they already have a lot of sophisticated testing around this, so I don’t need to duplicate that by comparing pixels.

Adding Observability - Tracking Exceptions

Phil Hardwick — Fri, 29 May 2020 05:26:21 +0000

This post will show how to easily observe exceptions in a Spring Boot application, using Honeycomb and Aspect Oriented Programming.

Observability

Observability is understanding how your system is running in production without deploying changes. It’s about being able to answer questions you didn’t know you wanted the answer to, without changing code. Two things that enable observability are: exporting non-aggregated data (i.e. events not metrics) and high cardinality labels for those events. Events contain lots of data about what happened in a system which can then be queried or aggregated later. A high cardinality label, for example, would be user ID. That’s useful because if your system is showing slow responses, you can group the response times by user ID and see what proportion of users are being affected. If it’s only a small subset of users you can then investigate only those users - you’ve delved into the heart of the problem much quicker than you could have with low cardinality metrics.

To improve observability in a personal side project app, I decided to use Honeycomb. Honeycomb is super easy to export data into and has an intuitive UI for querying and graphing (I’m not affiliated with Honeycomb, it’s simply something I’ve wanted to try for a while).

Observing Exceptions

Another high cardinality label would be exception data. Adding exception data to events is also very useful for understanding why things are happening in your system and particular exceptions might be the cause or differentiator for you to be able to find issues and the causes of them.

I’m going to show how to easily add exception data to your events with Aspect Oriented Programming.

Aspect oriented programming (AOP)

Aspect oriented programming allows you to write code for cross-cutting concerns (non-functional requirements (NFRs) that are needed for all business functionality e.g. logging, transactions, observability) without adding extra code into the business logic - which would distract from others being able to read the intent of the logic. Instead, you can define actions to be taken before, after or around method calls in your system and keep business logic and NFRs separate.

Terminology: Pointcuts and Advices

Pointcuts are simply a filter, without which an advice would run on every method call (or join point as it’s called in AOP). An advice is the action that runs before, after, or around a method call.

Adding fields to spans after an exception

This is using Spring AOP. So firstly add the Spring Boot starter for AOP into build.gradle:

implementation 'org.springframework.boot:spring-boot-starter-aop'

Then add a new Aspect class (make sure to annotate with @Aspect):

@Aspect
public class BeelineAspect {

    private final Beeline beeline;

    public BeelineAspect(Beeline beeline) { // 0
        this.beeline = beeline;
    }

    @Pointcut("within(@org.springframework.stereotype.Repository *)" +
            " || within(@org.springframework.stereotype.Service *)" +
            " || within(@org.springframework.web.bind.annotation.RestController *)") // 1
    public void springBeanPointcut() {
    }

    @Pointcut("within(technology.wick.repository..*)" +
            " || within(technology.wick.service..*)" +
            " || within(technology.wick.web.rest..*)") // 2
    public void applicationPackagePointcut() {
    }

    @AfterThrowing(pointcut = "applicationPackagePointcut() && springBeanPointcut()", throwing = "e") // 3
    public void traceAfterThrowing(JoinPoint joinPoint, Throwable e) {
        beeline.getActiveSpan().addField("exception.message", e.getMessage()) // 4
            .addField("exception.cause", e.getCause())
            .addField("exception.name", e.getClass().getSimpleName())
            .addField("exception.happenedIn", joinPoint.getSignature().getDeclaringTypeName() + "." + joinPoint.getSignature().getName())
        ;
    }
}

Beeline gets autowired in since it is exposed as a bean by the beeline-spring-boot-starter.
Add pointcut to match any method call in beans annotated with @Repository, @Service or @RestController.
Then add a pointcut to match any method call in beans in the repository, service and web packages.
Use those pointcuts in an advice to run only after throwing an exception.
Add exception data to the current span using Beeline (Honeycomb’s interface to modify spans and traces).

The pointcut methods are purposefully empty because the actual action we want to take is in the advice.

This aspect can then be enabled with an @Configuration class, which exposes the aspect as a bean.

@Configuration
@EnableAspectJAutoProxy
public class AspectConfiguration {

    @Bean
    public BeelineAspect beelineAspect(Beeline beeline) {
        return new BeelineAspect(beeline);
    }
}

Conclusion

If you have the rest of the Honeycomb configuration set up, you can now query for exception events in the dashboard.

AOP is a nice way to add observability for exceptions since you don’t have to touch any of your current code. This can also be applied to other tracing providers like OpenTelemetry.

Facts about Pacts

Phil Hardwick — Fri, 22 May 2020 07:16:00 +0000

In Mettle, Pact was used to break up deployments and releases so that big bang releases were avoided. Previously testing had been done against a fixed set of versions of each of the services and if any version changed, testing would have to be re-done. I joined after this work had been done, so I had to catch up about how Pact worked.

Using consumer driven contracts is a key step towards getting to continuous delivery, where deployment to production is automatic. Continuous delivery is particularly excellent because changes going into production are small and incremental - if something is broken it’s easy to roll back to the previous version. Also, since the change was small, you know where to look for the bug - it makes the process of releasing much less risky.

These are some things I learnt about Pact, when first starting, which may help you get up to speed quicker!

It tests contracts

What’s particularly key about Pact is that it separates testing the functionality of a service (by feature tests) from the interfaces it provides or consumes. Pact is very specific about what it tests: it’s not testing authorization, authentication, standards, interactions with third party systems - it is testing the format of messages sent between services you own.

It provides fast feedback

Pacts sit in between the unit test and integration test levels of the test pyramid. It can therefore be used for really quick feedback: contract tests can be run as part of the build process, so you know immediately whether your service is compatible.

Consumers only publish contracts

It’s “consumer driven”, so it follows that consumers publish the pacts (contracts) first. Consumers don’t “verify” contracts, they only publish them. For the consumer, it’s actually quite a self-contained process.

Consumers create contracts by running an in memory server, setting up expected requests on the server and then invoking the production code to make that request.

This means that when the consumer build runs, it’s not really running against a provider, it’s doing its own thing by sending requests to a mock Pact server and checking the request is as expected. The only reason this can fail is if the code is sending a different request to what the mock server is expecting.

Providers only verify contracts

Providers only verify the contracts which have been published, they don’t publish contracts.

Providers verify pacts by running the exact request from the contract and then comparing the response they return with the expected response in the contract.

Providers pull contracts down from the broker and use them to create requests to send to the running service (in our case they run against a Spring Boot test which spins up the application in memory). This is where the contract test can fail because of a mismatch between the expected response in the contract compared to the response returned from the service.

Providers need to know when new contracts are published

To be able to get instant feedback, providers need to know when new contracts are published. Then they can run their own build to verify that they provide everything the new contract requires.

The Pact broker can show a lot of version matches in the compatibility matrix

Don’t be afraid of all the version matches shown - that’s normal, and the Pact broker is made to handle that. The reason for the number of versions is that when a provider runs a verification, it verifies against all the tags you specify and publishes each of these to the broker. So if you verify against tags master, dev, staging, prod then you may be creating 4 version matches every time you build the provider. Add that to the fact that we use a git sha as the version of the Pacticipant, and you get even more version matches. This is because each git commit will be a new version in the broker.

Including generated JWTs in your contracts will create a new contract on every consumer build

Because JWTs include time information such as expiry (exp) and issued at (iat), every time the test runs it will likely have a different JWT with a different exp and iat. Try to avoid generating JWTs and using them in your pact. Either don’t send it or use a static long-lived token. This enables Pact to do some optimisation - if the pact doesn’t change it automatically knows all the current providers fulfill it, so the provider tests don’t need to run again verify it.

Using tags it is easy to see the version of each service in each environment.

After a service is deployed to a new environment we tag its version in the Pact broker with the environment name (for consumers we tag the contract, for providers we tag the provider). This is to enable providers to verify against all the versions of the consumers we have deployed. This enables our pipeline to check in the Pact broker that a new version of a service can run against everything that is already running in dev, staging or prod.

There’s a side benefit in that you can see exactly what version of each service is deployed to which environment - pretty cool.

What to test in Pact

What to test in Pact is up to you - but try to keep it to the format of interactions rather than the functionality. For instance we decided to mock the spring service layer in pacts rather than seed the repositories with data to meet the provider states. This bypasses any logic in the services and just keep the test to exercising what the controller does which the HTTP interface.

Conclusion

These were, hopefully useful, facts about Pact!

Prompt to install a PWA on iOS and Android with React Hooks

Phil Hardwick — Wed, 13 May 2020 06:10:30 +0000

So you’ve got your service worker, your manifest.json and your app works offline, but how do you get your users to install the app?

This post will show you how to prompt your users to install your progressive web app (PWA) using React Hooks. I’m going to assume you already have all the necessary stuff for a PWA in place (manifest.json, a service worker, all served from https - when you’re not on localhost).

I’ve been creating an app and, instead of messing around with the App Store, React Native and two build pipelines, I decided to build a PWA so I could run one, web-based codebase on all devices. Users should be able to “install” the app if they wanted. So doing some research, I was surprised to find PWA support is still not universal or consistent across Android and iOS. So I created a cross-platform solution to prompt users based on their device. This solution uses React Hooks to be able to reuse and easily integrate this functionality into components.

Has the user been prompted recently?

The first bit of functionality is to store when a user has been asked to install, so we can check if they’ve been asked recently and therefore not show the prompt too often. This is common to both iOS and Android prompts, so I extracted it into a Hook of its own.

import { useState } from 'react';
import moment from 'moment';

const getInstallPromptLastSeenAt = (promptName: string): string => localStorage.getItem(promptName);

const setInstallPromptSeenToday = (promptName: string): void => {
  const today = moment().toISOString();
  localStorage.setItem(promptName, today);
};

function getUserShouldBePromptedToInstall(promptName: string, daysToWaitBeforePromptingAgain: number): boolean {
  const lastPrompt = moment(getInstallPromptLastSeenAt(promptName));
  const daysSinceLastPrompt = moment().diff(lastPrompt, 'days');
  return isNaN(daysSinceLastPrompt) || daysSinceLastPrompt > daysToWaitBeforePromptingAgain;
}

const useShouldShowPrompt = (promptName: string, daysToWaitBeforePromptingAgain = 30): [boolean, () => void] => {
  const [userShouldBePromptedToInstall, setUserShouldBePromptedToInstall] = useState(
    getUserShouldBePromptedToInstall(promptName, daysToWaitBeforePromptingAgain)
  );

  const handleUserSeeingInstallPrompt = () => {
    setUserShouldBePromptedToInstall(false);
    setInstallPromptSeenToday(promptName);
  };

  return [userShouldBePromptedToInstall, handleUserSeeingInstallPrompt];
};
export default useShouldShowPrompt;

This uses local storage to persist a user’s response across sessions. The useState hook is used to make sure the application has a way to check the state of a user’s response. Combining these means you have a persistent way to watch for updates.

iOS

The iOS version of detecting whether a user should be prompted is simply detecting whether they’re on an iOS device, and that they haven’t already “installed” the PWA.

import useShouldShowPrompt from 'app/shared/hooks/useShouldShowPrompt';

const iosInstallPromptedAt = 'iosInstallPromptedAt';

const isIOS = (): boolean => {
  // @ts-ignore
  if (navigator.standalone) {
    //user has already installed the app
    return false;
  }
  const ua = window.navigator.userAgent;
  const isIPad = !!ua.match(/iPad/i);
  const isIPhone = !!ua.match(/iPhone/i);
  return isIPad || isIPhone;
};

const useIosInstallPrompt = (): [boolean, () => void] => {
  const [userShouldBePromptedToInstall, handleUserSeeingInstallPrompt] = useShouldShowPrompt(iosInstallPromptedAt);

  return [isIOS() && userShouldBePromptedToInstall, handleUserSeeingInstallPrompt];
};
export default useIosInstallPrompt;

We return a hook which combines checking if the device is using iOS and whether the user has already been prompted, with a function to handle the user dismissing the prompt.

All other platforms

On all other platforms, PWA support is more consistent and uses web events. The key is to attach an event handler in a useEffect hook (using the clean up variation to remove the event handler) to catch and store the install prompt event. We also use the useState hook to store the event, and the hook we previously created useShouldShowPrompt. This hook returns the event, a method to handle a user wanting to install and a method to handle a user declining an install. You’ll notice the useEffect has a dependency on userShouldBePromptedToInstall so that it will run again when that changes, this is so that the user isn’t re-prompted right after they decline to install on the native prompt.

import { useState, useEffect } from 'react';
import useShouldShowPrompt from 'app/shared/hooks/useShouldShowPrompt';

const webInstallPromptedAt = 'webInstallPromptedAt';

const useWebInstallPrompt = (): [any, () => void, () => void] => {
  const [installPromptEvent, setInstallPromptEvent] = useState();
  const [userShouldBePromptedToInstall, handleUserSeeingInstallPrompt] = useShouldShowPrompt(webInstallPromptedAt);

  useEffect(() => {
    const beforeInstallPromptHandler = event => {
      event.preventDefault();

      // check if user has already been asked
      if (userShouldBePromptedToInstall) {
        // store the event for later use
        setInstallPromptEvent(event);
      }
    };
    window.addEventListener('beforeinstallprompt', beforeInstallPromptHandler);
    return () => window.removeEventListener('beforeinstallprompt', beforeInstallPromptHandler);
  }, [userShouldBePromptedToInstall]);

  const handleInstallDeclined = () => {
    handleUserSeeingInstallPrompt();
    setInstallPromptEvent(null);
  };

  const handleInstallAccepted = () => {
    // show native prompt
    installPromptEvent.prompt();

    // decide what to do after the user chooses
    installPromptEvent.userChoice.then(choice => {
      // if the user declined, we don't want to show the prompt again
      if (choice.outcome !== 'accepted') {
        handleUserSeeingInstallPrompt();
      }
      setInstallPromptEvent(null);
    });
  };
  return [installPromptEvent, handleInstallDeclined, handleInstallAccepted];
};
export default useWebInstallPrompt;

How to use the hooks

This is an example of how I use these two hooks in a modal asking the user if they want to install the app. This is using Reactstrap. The modal is always open because, if neither of the hooks return true, this component will return null. If the iosInstallPrompt is true then we show an instruction to add the web page to the Home Screen. The handleIOSInstallDeclined is wired up to the onClick of the “close” button to make sure the user won’t be shown it again once they dismiss the modal.

Otherwise, if webInstallPrompt exists the modal shows a modal with an “install” or “close” button. The handleWebInstallDeclined and handleWebInstallAccepted are wired up to the “close” and “install” buttons to either show the native install popup or register that the user has dismissed the modal and shouldn’t be shown it again.

This is how the component’s code looks:

import React from 'react';
import { Button, Modal, Card, CardText, CardBody, CardTitle } from 'reactstrap';
import useIosInstallPrompt from 'app/shared/hooks/useIosInstallPrompt';
import useWebInstallPrompt from 'app/shared/hooks/useWebInstallPrompt';

export const InstallPWA = () => {
  const [iosInstallPrompt, handleIOSInstallDeclined] = useIosInstallPrompt();
  const [webInstallPrompt, handleWebInstallDeclined, handleWebInstallAccepted] = useWebInstallPrompt();

  if (!iosInstallPrompt && !webInstallPrompt) {
    return null;
  }
  return (
    <Modal isOpen centered>
      <Card>
        <img
          className="mx-auto"
          style={{
            borderTopRightRadius: '50%',
            borderTopLeftRadius: '50%',
            backgroundColor: '#fff',
            marginTop: '-50px'
          }}
          width="100px"
          src="content/images/appIcon-transparent.png"
          alt="Icon"
        />
        <CardBody>
          <CardTitle className="text-center">
            <h3>Install App</h3>
          </CardTitle>
          {iosInstallPrompt && (
            <>
              <CardText className="text-center">
                Tap
                <img
                  src="content/images/Navigation_Action_2x.png"
                  style={{ margin: 'auto 8px 8px' }}
                  className=""
                  alt="Add to homescreen"
                  width="20"
                />
                then &quot;Add to Home Screen&quot;
              </CardText>
              <div className="d-flex justify-content-center">
                <Button onClick={handleIOSInstallDeclined}>Close</Button>
              </div>
            </>
          )}
          {webInstallPrompt && (
            <div className="d-flex justify-content-around">
              <Button color="primary" onClick={handleWebInstallAccepted}>
                Install
              </Button>
              <Button onClick={handleWebInstallDeclined}>Close</Button>
            </div>
          )}
        </CardBody>
      </Card>
    </Modal>
  );
};

You can find the iOS share icon in the Apple documentation or https://github.com/chrisdancee/react-ios-pwa-prompt has an svg version.

Conclusion

I’m happy with how this has turned out: cross platform and easy to include in my app. The use of hooks here allowed me to extract some common functionality really easily, e.g. the useShouldShowPrompt hook, which was used in both iOS and web prompts hooks.

Attribution and Further Reading

My code was inspired by https://jason.codes/2019/03/pwa-install-prompt/ and https://medium.com/swlh/a-simple-react-hook-to-prompt-ios-users-to-install-your-wonderful-pwa-4cc06e7f31fa.

Don't use Event Sourcing if you want auditability

Phil Hardwick — Tue, 05 May 2020 07:54:00 +0000

The reasons for choosing an architecture can be so varied. We take into account customer needs, stakeholder requirements, legislation and non-functional requirements like availability and security.

At its core though, choosing an architecture is about choosing what you optimise for and the best choice, in the long run, is to optimise for change.

Event Sourcing is pretty popular right now and one reason I hear for choosing it is auditability, “if we have a log of all the events of changes in the system, we get auditability for free!”

I think this is a bad reason to choose Event Sourcing - you’re choosing a particularly difficult pattern, simply so you can check what the system has done in the past.

Why it might not provide auditability

It’s possible, and probable, you won’t get Event Sourcing right the first time, and who’s to say you actually have recorded all the events of the system? If you’re not testing your golden source of truth regularly then you may not have all your data in events and therefore your audit trail may not be complete.

Why you should really choose Event Sourcing

A good reason to choose Event Sourcing is that you don’t know your future requirements. No one can fully predict how a system will evolve or what will be needed a year from now. Having a stream of events of all the previous changes which can be read from any service allows you to be flexible and implement any future requirement - it’s truly extensible. This is because any requirement can be implemented as a new service and it can load in data, transforming it and enriching it to meet what’s needed. For example, a requirement to send email reminders to fill in some questions: a new service can now consume user events to get email addresses and names, along with the events about when they last filled in the questions. The new service builds up an understanding of the current state of the system and then uses that to make decisions. This is all possible without changing the existing services, which helps to reduce the chance of introducing regressions, and makes it simpler to deploy since the change is only adding services, not modifying any.

In this way, Event Sourcing optimises for dealing with change in requirements.

Forem: Phil Hardwick

A Java Library Release Process Which Doesn't Suck

Requirements

New solution

How it works

How to add it to a project

1. Add the semver git plugin

2. Configure the plugin

3. Use the version

4. Set up your CI to build when a new tag appears

Github Bonus

Conclusion

Refactoring Branches out of Spring Controllers with Params

Paging a http endpoint by recursion

Changing Bootstrap Theme at Runtime with CSS Variables

The Requirements

CSS Variables Solution

Drawbacks

How I implemented it

Conclusion

Using Micronaut Annotation Mapping to Automatically Add Security Info to Swagger Docs

Dependencies

Annotation Mapper

Making the mapper discoverable

Using your mapper in another project

What gets generated

Conclusion

Custom Micronaut Security Rules

Customising @Secured

Custom Security Rules

Permissions on Specific Resources

So how can we authorise this user in Mirconaut?

Conclusion

8 reasons you should use GitOps

What is GitOps

1. Show me the config

2. History of changes

3. Git revert

4. Using familiar tools

5. Declarative means rebuildable

6. Git is the source of truth

7. Collaboration and sharing

8. Self service

Further reading

React Map GL - How do I test this?

Libraries and Components

Make the map queryable

Querying the map for layers

Using this in your test

Things to watch out for

What you’re actually testing

Adding Observability - Tracking Exceptions

Observability

Observing Exceptions

Aspect oriented programming (AOP)

Terminology: Pointcuts and Advices

Adding fields to spans after an exception

Conclusion

Facts about Pacts

It tests contracts

It provides fast feedback

Consumers only publish contracts

Providers only verify contracts

Providers need to know when new contracts are published

The Pact broker can show a lot of version matches in the compatibility matrix

Including generated JWTs in your contracts will create a new contract on every consumer build

Using tags it is easy to see the version of each service in each environment.

What to test in Pact

Conclusion

Prompt to install a PWA on iOS and Android with React Hooks

Has the user been prompted recently?

iOS

All other platforms

How to use the hooks

Conclusion

Attribution and Further Reading

Don't use Event Sourcing if you want auditability

Why it might not provide auditability

Why you should really choose Event Sourcing