For fun and learning: playing around with JavaScript face detection and html5 canvas

Markus Tiefenbacher — Fri, 10 Apr 2020 11:36:52 +0000

It’s easter and wanted to make some fun for my coworkers and what should be better than some Covid-19 fun in this surreal times.

I had the idea to put every model in our webshop a surgical mask on their face.

But how should I solve this?

Face Detection!
This was a topic I ever wanted to discover and so it was a perfect match for this. I started reading the face-api.js documentation and after some hours of playing around with it I was ready to ship the easter egg on our testing server.

Result:

Try it yourself:

I made also a version where you can try it yourself: https://tiefenb.github.io/use-a-mask/

... and here are the source code: https://github.com/tiefenb/use-a-mask

Reactions so far:

Learnings:

Loading images for HTML5 canvas is only possible with images with crossorigin attribute
crossorigin attribute has to be set on the image tag before the image is loaded - it’s not possible afterwards. That means also, if you create the image element with JavaScript you have to add the crossorigin-attribute before the src-attribute.
Rotating an single image in html5 canvas is not possible in the way most of us think. you have to rotate the canvas-context, than you put on the canvas the image and you rotate the canvas-context back to your normal angle
Putting an image on a canvas is only possible after the image is full loaded - so you have to listen in the image.onload event
face-api.js tensorflow.js-models are quite big - better to make detection it on the server not the client

Thank's for reading and happy easter!

Dear appear.js, I have met someone new. Her name is IntersectionObserver.

Markus Tiefenbacher — Tue, 14 May 2019 13:19:10 +0000

Dear appear.js, I have met someone new. It was more or less a good time with you. But I want to break up and you have to leave your stack.

The new one? Her name's IntersectionObserver. She is nice to me. She is smart, pretty syntax and very fast in doing things. I think I will have a good time with her.

Enough with joking.
At blue-tomato.com, we used some jQuery based image lazy loading library and appear.js for lazyloading things and doing effect based on elements in the viewpoint.

Both libraries listen on the scroll event and looking if the actual scroll position is in the range of the given elements position.

This technic was the default for long time, but since IntersectionObserver comes up, it was clear to switch sometimes to this. The main problem with the old method was the listing on the scroll-event was very expensive to browser performance.

Example

Making a navigation bar sticky with appear.js

In this example, after the .main-content became visible, the .navigation bar will get the CSS class .navigation--sticky and will become sticky. When .main-content disappers (e.g. by scrolling back to the top of the page) the .navigation bar will become static by removing the .navigation--sticky class.

appear({
  elements: function elements() {
    return document.querySelectorAll('.main-content');
  },
  appear: function appear() {
    $(".navigation").addClass('.navigation--sticky');
  },
  disappear: function appear() {
    $(".navigation").removeClass('.navigation--sticky');
  }
});

Making a navigation bar sticky with InteractionObserver

With InteractionObserver this would work in this way:

var element = document.querySelector('.main-content');
if(element) {
  let observer = new IntersectionObserver(entries => {
    entries.forEach(element => {
      if(element.intersectionRatio > 0) {
        $(".navigation").addClass('.navigation--sticky');
      } else {
        $(".navigation").removeClass('.navigation--sticky');
      }
    });
  }, { rootMargin: '0px 0px', threshold: 0.00 });

  observer.observe(element);
}

Whats that rootMargin and thresold?

Appear.js has only some "bounds" option to let it fire some pixels before the element is really in the viewport. IntersectionObserver has a really powerful and flexible method to control this via the root/rootMargin and thresold options. Have an eye on this article from Smashing Magazine where Denys Mishonnuc explains this very detailed: https://www.smashingmagazine.com/2018/01/deferring-lazy-loading-intersection-observer-api/#intersectionobserver-initialization

Forem: Markus Tiefenbacher