Forem: Justin Ribeiro

Faster YouTube embeds with lite-youtube web component

Justin Ribeiro — Mon, 18 Nov 2019 22:47:41 +0000

I was puttering around Github at the end of last week and I stumbled upon Paul Irish’s lite-youtube-embed. I heart the things that Paul comes up with and in the run around with Chrome Dev Summit and what not last week, I had likely missed it (and as of this writing, it’s come a long way since I saw it on Friday in a single html file).

What I really was intrigued about was the numbers: it was fast. Having been embedding YouTube videos in some of my posts recently, I wanted more speed as the YouTube embeds weren’t exactly blazing fast in my traces. So I decided I’d go a bit of a different direction than Paul and build a ShadowDOM version of the component (since Paul was going to only build the custom element version, I didn’t want to pester him with a shadowDOM argument in his issue tracker).

My encapsulated and responsive version is now available on NPM at @justinribeiro/lite-youtube, recently updated to version 0.3.1. Primary features include:

No dependencies; it’s just a vanilla web component.
It’s fast yo.
It’s Shadow Dom encapsulated!
It’s responsive 16:9
It’s accessible via keyboard and will set ARIA via the videotitle attribute
It’s locale ready; you can set the videoplay to have a properly locale based label

To use, you can install it via npm or yarn:

npm i @justinribeiro/lite-youtube # or yarn add @justinribeiro/lite-youtube

Or you can just include is directly with JSDelivr:

\<script type="module" src="https://cdn.jsdelivr.net/npm/@justinribeiro/lite-youtube@0.3.1/lite-youtube.js"\>

From there, just include it your page with the videoid and videotitle

\<lite-youtube videoid="guJLfqTFfIw" videotitle="Sample output of devtools-to-video cli tool"\> \</lite-youtube\>

If you’d like to contribute or find an issue, the project is available on Github or if you’re looking for the non-ShadowDOM version, do check out Paul’s repo and pending NPM package as well.

Fail a PR based on any audit score with v1.1.0 of Web Performance Audits with Lighthouse Github Action

Justin Ribeiro — Wed, 06 Nov 2019 18:09:03 +0000

While the budget.json support in my Web Performance Audits with Lighthouse Github Action was a good start point for many folks already using budgets, there was just something missing from it. How can I check and fail a pull request based on a specific audit or an overall category score?

I thought about this for some time and weighted the options. I could make Github Action input variables that allow setting, but that seemed rather convoluted and restrictive. I could come up with my own specific JSON file, but that too seemed ill conceived. What I settled on was the old adage dance with the one that brought; why not use the data model for Lighthouse?

With that, v1.1.0 introduces a new input, lighthouseScoringBudget that takes a file I call scores.js.

scores.js is based on the Lighthouse JSON output data model and maps one-to-one with the existing keys and definitions within lighthouse. This allows an easy to use, vastly configurable file to test any audit score or raw numeric value that you find important.

module.exports = { audits: { 'service-worker': { score: 1, }, 'first-contentful-paint': { score: 1, numericValue: 100, }, 'first-meaningful-paint': { score: 1, numericValue: 100, }, }, categories: { performance: { score: 0.95, }, accessibility: { score: 0.95, }, },};

No having to wait for me to add special parameters and easily comparable to the standard out from a JSON run of lighthouse:

$ lighthouse --output json --output-path=./sample.json --chrome-flags="--headless" https://YOUR\_URL\_HERE.com

Just pop open that JSON file, determine what audits or categories are important to you, and start failing those pull requests at will.g sb

In the current iteration of this feature, we can test for either one or two specific keys:

score: decimal ranging from 0 to 1. If you wanted to verify that your performance is above 95, you’d enter 0.95 like above.
numericValue: decimal starting at zero. Commonly holds the value out of the trace in milliseconds, good for testing if you want those raw performance numbers.

That’s all for now; stay tuned for more features. As always, PRs and issues are welcome. Now go trace a site and let’s make the web fast!

Web Performance Audits with Lighthouse for Github Actions Released

Justin Ribeiro — Sat, 02 Nov 2019 23:37:27 +0000

I’m just going to say it: I can pretty much integrate Lighthouse anywhere. I’ve put in tests, I’ve written it into Visual Studio code tasks, I’ve wired it into docker, I made it dance at Chrome Dev Summit on giant screens as a game, I’ve even made a few commits here and there.

Which is to say, I really like lighthouse. It’s a useful tool that I wish more developers would use. It’s one reason I keep writing integrations and examples, hoping that more buy-in for web performance is always just around the next corner.

In today’s case, I’m releasing Web Performance Audits with Lighthouse, a new Github Action that has a number of nifty features that I hope folks will enjoy.

The biggest feature takes a page from the lighthousebot style for Travis and injects a comment with the audit results right onto your pull request:

If you’re looking at the screenshot and wondering “wait, budget.json fails?”, you’d be correct. You can use your existing budget.json for Lighthouse (which you’re already using on the CLI right?) right into your configuration:

name: Audit Web Performance on: [pull\_request] jobs: perf: runs-on: ubuntu-latest steps: - uses: actions/checkout@v1 - name: Generate Lighthouse Report uses: justinribeiro/lighthouse-action@v1.0.1 with: with: secret: ${{ secrets.GITHUB\_TOKEN }} url: https://justinribeiro.com/ wptConnectionSpeed: threegfast lighthouseBudget: .github/test/budget.json - name: Saving Lighthouse Audit Artifacts uses: actions/upload-artifact@master with: name: lighthouse-artifacts path: './results'

The full feature list includes:

Uses Puppeteer to start up Chrome with network emulation settings defined by WebPageTest. This is similar connection setup to my existing lighthouse-jest-example.
Supports saving of artifacts to the Github Action run.
Supports custom Lighthouse configuration via JavaScript file.
Supports Lighthouse budget.json for failing PRs.
Posts results of audit run as a comment on your PR.

If you want to see an example of this action in an actual Github workflow, you can see it in use in my blog-pwa repo where I’m using it to test site changes on deployments of PRs to Google App Engine.

You can get started using the action today via the marketplace. The repo is also available and happy to accept feature requests and pulls, so let me know what you’re thinking. I’ve written this as simply and straight forward as I could think of, so if you’re looking to understand a little how it all works, this example is a great way to dive in.

If you happen to be at Chrome Dev Summit this month, find me and we’ll chat about all things Lighthouse, all things web performance.

Now get out there and start testing even more of your deployments and let’s make the web fast!

Six Months Using Firebase Web Performance Monitoring

Justin Ribeiro — Mon, 28 Oct 2019 14:46:28 +0000

When Firebase announced earlier this year that they were bringing real user monitoring of web performance into their stack of tools, I was ecstatic. Another tool to track what real users in the wild are experiencing with an easy setup? Yes, please.

Initially, I heard similar shared excitement. Now six months on since it was announced I can find very little chatter about it, and I’ve not run into anyone who’s tried it.

You’re in luck internet: I’ve been using Firebase web perf tracking since June on this very blog. Let’s take a look at the good, the bad, and the downright confusing portions of the Firebase web performance monitoring beta.

The Good

From the onset, Firebase web performance isn’t there to destroy your existing web performance. The library can be invoked with only the web performance tracking and not load anything related to the database or other Firebase-related products. This may seem like common sense, but then again, the third party web resource community is not exactly known for their light touch.

While there are many setups outlined in the docs, I delay loading the library even further with no detrimental effect. I do load the first input delay polyfill early on page load however (which is the same way I use it when tracking performance with tools like Google Analytics).

From a collection of numbers standpoint, you literally have no code to write beyond the invoke. Install script, verify it’s sending data, wait for numbers to appear in the panel:

In the screenshot above, we’re looking at three months worth of data from the entire domain. This paints a rather nice picture of performance for both the entire site and the specific blog-entry path, though we need to keep in mind we’re only looking at the median. All the action is in the tails when it comes to performance problems.

If we dive into a particular route, we can get a larger picture of our web performance.

These are the default performance numbers that Firebase Performance SDK will collect for the web. They’ll all probably look familiar if you’ve run a Lighthouse audit or looked a DevTools trace. Good numbers to have generally. Unfortunately, all is not great when it comes to those numbers.

The Bad

While it’s easy to collect numbers the further we dive into the numbers the more difficult it is to actually find actionable data. Actionable data is on the tail, where load times and potential bottlenecks within our web app code could be causing users on certain devices or networks problems. We want the bad cases so we can create fixes.

Let’s take for instance the last three months of First Paints:

We can move around to see the various percentiles by hovering over the graphs, but we really can’t dive into that tail to see the number of loads or other cross section of information when it comes to what’s going on with these users.

If we filter down into a single country, in the case below India, we see similar tools and not much more information. Shifting the trend over time doesn’t really paint us a better picture, leaving us to guess as to why and how that segment of data might have come to be.

Just export the data and process it you say? I agree, we should do that. Should be the opportune word.

The Downright Confusing

While the data segmentation tools are not great, the more confusing part is the data itself. While the documentation makes note that you can export the data into BigQuery, doing so doesn’t expose the collected data for web performance. The entire dataset is empty and without this working there’s no way to dig into or export the data into other tools.

This is further compounded because you an only see up to the last three months of data within the web UI. My hope is that this is just a case of “it’s a beta, we haven’t enabled all this yet”.

Then there is the question of the User Timing API. I love this API and use it a lot, as the marks and measures surface in my Lighthouse reports and many other custom tools I’ve written. Turns out the Firebase Performance SDK does support them, allowing you to use standard performance.mark and performance.measure to allow them to said data to appears as custom trace numbers in the panel. Note, this isn’t in the documentation but rather briefly makes an appearance in their codelab.

What confuses me is why they felt the need to define their own firebase.performance().trace method, which in combination with start and stop commands, allows custom trace logging. This is documented and it frustrates me. Why not simply use the standard User Timing API? My hunch is this is because people may use this in node outside the browser context, but I would hope this would be clearly defined.

Should You Use Firebase Web Perf SDK

Like most questions of this nature, it depends.

On the one hand, if you want to dabble and start to test the beta on things that aren’t mission critical, I’d say give it a spin. If you aren’t tracking data at all and you want to get a feel for just how your web site or PWA is working in the wild with users, then it’ll give you a pulse. Is it going to be actionable? My gut says no, but only you know how your app works.

If you’re in production right now on a site and you’re thinking you want to track performance, the answer is no. The data reporting and export to BigQuery do no appear to work (and are not documented in relation to their Android/iOS brethren), the panels are not really actionable or allow custom alerts to be set. It’s just not ready for prime-time. There are significantly better tools out there for monitoring web perf RUM; I recommend Speedcurve if you’re doing anything at scale.

Regardless of the gaps and initial issues with the beta, I applaud any company that’s will to take on web performance tracking and monitoring. It’s not easy to roll your own and often the alternatives are terrible.

While Firebase Performance SDK for the web is clearly a work-in-progress, I’m hoping that it improves. The more awareness we have about web performance, the more I’m hoping we can make the web faster and better for more users around the world.

Originally published on justinribeiro.com

Generating a Simple Font Report With a Devtools Snippet

Justin Ribeiro — Fri, 18 Oct 2019 16:23:42 +0000

As with most weekday mornings, links and articles are dropped into our random channel as either points of potential interest or swords drawn discussion. The topics have a lot of range, but the one that recently caught my eye was and article regarding finding improper font definitions in a page.

The article had a useful premise, thought I wasn’t big on the code example or the page injection (which I found hard to read personally, but your mileage will vary). Instead, I decided I’d just write a variation of the snippet to output a unique report and use the DOM’s TreeWalker instead.

TreeWalker has been in the spec and in browsers for a long time and allows you to get a given document’s subtree and take a look at the nodes within. The nice thing about TreeWalker is that it’s fast and if you’ve ever used say lit-html or other such libraries, congratulations, you’ve used TreeWalker without even knowing it.

With our API in mind, we can transverse and go about finding out what our getComputedStyle() is for a given text node as in the DevTools snippet below.

function findTextNodesFor(element){ let node; const discoveredTextNodes = []; const walkTree = document.createTreeWalker(element, NodeFilter.SHOW\_TEXT, null, false); while(node = walkTree.nextNode()) { discoveredTextNodes.push(node); } return discoveredTextNodes; } weights = { 100: "Thin", 200: "Extra Light", 300: "Light", 400: "Regular", 500: "Medium", 600: "Semibold", 700: "Bold", 800: "Extra Bold", 900: "Black" }; fonts = new Set(); findTextNodesFor(document.querySelector('body')) .filter(node =\> !['SCRIPT', 'STYLE', 'NOSCRIPT'].includes(node.parentNode.nodeName)) .forEach(node =\> { const computedStyle = window.getComputedStyle(node.parentNode); const font = computedStyle.fontFamily || ""; const size = computedStyle.fontSize; const weight = computedStyle.fontWeight; const weights = { 100: "Thin", 200: "Extra Light", 300: "Light", 400: "Regular", 500: "Medium", 600: "Semibold", 700: "Bold", 800: "Extra Bold", 900: "Black" }; fonts.add(`${size} ${weights[weight]} ${font}`); }); console.table([...fonts].sort());

This will output a nifty little table of only the unique fonts we find (if you’ve not used Set, I highly recommend you take a look at the docs to see how if can make your life easier).

Now, there are some gotcha’s to keep in mind here:

This won’t show you the browsers computed display. Chrome DevTools for instance will show you the font(s) that system are actually being rendered as opposed to just the font definition that is being applied to the node.
This won’t transverse the custom element with shadowDOM (because that’s a whole other can of worms).

Sans that, you get a nice little way to see exactly what’s being defined and if you may need to re-evaluate your font strategy.

Generating Video From a Chrome Devtools Trace With devtools-to-video

Justin Ribeiro — Wed, 16 Oct 2019 14:55:51 +0000

When it comes to diagnosing web performance problems, you do a lot of living in DevTools. Between the Performance Tab and the all powerful chrome://tracing, these are the sorts of things I work with on a daily basis.

The problem with living in this paradise of useful information is that the information has a tendency to only make sense to a few folks. When you talk to upper management, traces tell a story that visually can be hard to decipher in bit sized meetings.

Pictures of traces speak a thousand words. Video of what a user is experiencing get resources assigned to fix web performance problems.

In a perfect world, I’d have WebPageTest just rolling runs at my behest with video output, but in a lot of on many projects WebPageTest isn’t running internally and testing externally is a non-starter. We need a method to generate some video with some feeling. Enter devtools-to-video.

devtool-to-video is a little cli tool written in node that utilizes ffmpeg to generate a video from a Chrome DevTools trace and put a timer on the bottom that shows the time progression.

An example of the video ouput:

This is similar in concept to WebPageTest though their output video timer is way cooler. This simply utilizes the presentation time (pts) within the ffmpeg to give us a quick means to create the video without a lot overhead via drawtext.

Using devtool-to-video is pretty simple. Install the cli tool via npm or yarn:

npm i @justinribeiro/devtools-to-video # or yarn global add @justinribeiro/devtools-to-video

Once installed, we have a few command line options to deal with:

➜ devtools-to-video -h DEVTOOLS-TO-VIDEO Output a video file from screenshot frames within a Chrome DevTools JSON trace file. Usage: `devtools-to-video [options ...]` Global Options -i, --input string File path to Chrome DevTools trace JSON file. -o, --output string Output file name for video file. -c, --hideClock If set, hides the time scale clock on the output video file. -l, --label string If set, adds the label above the time scale clock in the output video file. -f, --frames number Sets the frames per second of the output video. -h, --help Print out helpful usage information for this program. -v, --version Print out current program version number. Version 0.2.1

In practice, we generate our trace with screenshots enabled from the performance tab in DevTools, export and download that file, and then run it through our tool:

➜ devtools-to-video -i profile.json -o sample.mp4 -l '3G Slow @ Moto G4' STARTING UP Checking environment and setting up params. CONVERT Spawning FFMPEG worker with open pipe. CONVERT Begin piping screenshots from DevTools trace to FFMPEG. SCREENSHOTS Processed 21 screenshots into video file. CONVERT DevTools trace successfully converted to MP4. LABEL PASS Adding label and time scale to MP4. LABEL PASS Finished adding label and time scale to MP4. COMPLETE DevTools trace to video is now complete! Your file sample.mp4 is ready

Take file, show stakeholders, make case that you need better web performance.

Generating a video from a trace is also not a new concept; I used a much more convoluted approach a year or so ago that was just unreliable. Then I saw Kris Selden’s trace-to-mp4.js gist, and my faith in ffmpeg was restored. From there it was just a matter of double passing the video with a filter within ffmpeg (which is it’s own can of worms, but I love ya ffmpeg).

So go out there and trace your sites and apps, and keep making the web faster.

Web Components Have Already Won Whether You Like It Or Not

Justin Ribeiro — Mon, 14 Oct 2019 15:15:43 +0000

With the announcement a last week that VMWare is taking their popular enterprise Clarity design framework from Angular to web components, a few emails have trickled into my inbox asking for my opinion on certain anti-web components threads on Twitter (which I refuse to link to because it’s all blah). My opinion is not a popular one, but here it goes.

Web components already won while you all were arguing about it on Twitter.

A lot of folks are going to disagree. More pieces of “why I’m not using web components” will be written by folks who seemed either misinformed or simply want to protect their own knowledge space and viability. Many will point to technical reasons that show their lack of understanding of not just web components but the web platform as a whole. Some will point to performance and say web components are slow and have no trace to prove their point.

No number of existing data that shows web component usage will matter, no number of large companies making announcements about their usage of web components will sway those opinions. I will probably be called names and my character attacked.

All of that won’t change the fact that web components already won.

Web components didn’t just win in the large public companies with splashy open source projects. They’ve won at private companies of all sizes, where line of business software keeps companies running. I’ve lost count of how many companies I’ve helped ship nearly a hundred pieces of LOBS that are built on top of web components. They won where integration was a challenge that frameworks could not (and frankly probably shouldn’t) deliver on. I’ve written a lot of components in all shapes and sizes for hundreds of companies at this point and I’m losing compared to some of the work I see developers doing on web components in so many organizations.

You know the place it wins that no one wants to talk about because it’s not cool? jQuery. Yeah that little library that it’s in use on in more places than anyone wants to admit. Turns out, web components are extremely pliable and companies with developers who still work on those system’s love them.

“No one uses jQuery anymore, especially not with web components!”

You live a sheltered development existence and I feel sorry for you. Not everyone is on the bleeding edge, not everyone can change their stack year after year. If you can’t have empathy for those developers, we can’t be friends.

Why don’t you hear about these cases? The developers and companies in question are not in the bubble. They don’t care about your argument; they have work to do and are succeeding. Drag them into that argument and they’ll just walk away. They don’t have an axe to grind.

When they do pop up from time to time, the brave step into a hostile online conversation, what happens? Drowned out by unwarranted judgement and pretty baseless gibberish. Then they shrug and go back to work on their web components.

Inevitably someone will say I’m against frameworks (you clearly have not listened to the podcast) and that I (and many others) who are helping companies build with web components (or even that raw web platform APIs) are doing these companies a disservice. I’ll let the CIO’s and CTO’s I’m meeting with this week know your concerned.

The reality is you want to argue and quite frankly I’m done arguing. Stop trying to make it an adversarial game that someone has to lose. We’re all working towards a common goal of happy users.

You don’t like the spec? Contribute. Meetings continue to happen, get a word in and let’s talk it out. Features are good on the platform for everyone.

You think it competes with your framework? Probably less than you think. We can coexist people and I don’t have to tear anyone or any piece of technology down to do so.

Use the right tools for the job, use others as warranted. Sometimes that’s going to be web components whether you like it or not.

The battle was hard fought and took a lot longer than anyone expected, but specs and false starts and missteps aside, web components did the unthinkable: they won.

You can keep arguing if you like. Me, I’m going to keep building things.

Improving RSS Reader Support and Adding New Topic Feeds

Justin Ribeiro — Wed, 09 Oct 2019 23:10:50 +0000

With my recent foray into more widespread writing topics, I decided it was time to revisit the concept of topic specific RSS feeds and tag pages. Yeah, you’re not going to get the nitty gritty of some of my more personal posts, but hey, at least maybe you’ll keep on reading the content you find is relevant for you.

Available now (and in the blog-pwa codebase) are a new set of topic specific pages and feeds listed on the tags page and accessible from the home page or the archive. Topics currently include:

🕸️ Web Dev, Web Perf, and Web Platform Articles
🥽 Internet of Things (IoT) Articles
📷 Photography Articles and Prints
👔 Management and Business Articles
👨 Personal Articles

Under the hood, there was actually not a great deal of change required for this.

Handling tag generation in Hugo

The tags are generated via Hugo in a similar fashion to how I generate all my JSON objects. The biggest difference you’ll find is that the taxonomy defines it’s own RSS template and handles some of the deployment paths via replace and we make our title more specific:

\<channel\> \<title\>{{ .Site.Title }} - {{ .Title }}\</title\> \<description\>{{ .Description }}\</description\> \<link href="{{ replace .Permalink "/tags" "/data/tags" }}index.xml" rel="self"/\> \<link href="{{.Permalink}}"/\> \<language\>en-us\</language\> \<id\>https://justinribeiro.com/\</id\>

Along those lines, instead of just generating a big list of tags which would includes a number of old tagging conventions from my perl/jekyll/wordpress post conversions over the years, I override the /tags/_index.md and simply made a nicely sectioned version for ease of use. I also decided to do this for each of the tag pages within the content folder, again to make things nicer.

In terms of problems within this approach is that fact that every post needs at least one tag. Sure, I could have written some case logic within the hugo templates, but I figured I might as well backfill as much of the missing tags as needed. Turned out, not that many posts to update.

There was also a more wide spread fix applied at this time as well; code-block tags were replaced with pre, allowing better code block rendering for CSS clients (which did not understand the web component obviously). Both tag and overall RSS feeds have this patch, allowing for a much nicer rendering remotely.

Handling the tags in the PWA

This part was actually rather easy because of the structure of existing views. The blog-chronicle.js web component already had a posts array parser, so with a little retrofitting of the data layer, it could now serve to display tags and tag specific pages as well.

At that point, it only required a couple new routes within vaadin-router in the blog-pwa.js web component, and everything worked without a hitch.

No additional JavaScript weight or logic required. You gotta love it when a plan comes together.

Future planning

I’m not sold on only those using those tags (and I really need to clean the old ones, which do properly function still within the Hugo generation and PWA). For now, this will at least give user a chance to pick the right feed for them.

What are you waiting for? Subscribe to a feed or read a tag! You know you want to.

The Four Cases of Progressive Rendering Fallbacks for Progressive Web Apps

Justin Ribeiro — Tue, 08 Oct 2019 16:46:13 +0000

I mentioned yesterday that blog-pwa uses a static rendering layer for cases when the browser or whatever is fetching the page doesn’t support JavaScript. This was a slight simplification on my part. blog-pwa defines four primary cases:

Are you a known bot or tool that isn’t going to understand JavaScript or understand the noscript redirect? STATIC
Are you a browser that supports JavaScript? PWA
Are you a browser with JavaScript support turned off? PWA + NOSCRIPT Redirect to STATIC
Are you a browser with JavaScript support but no ES Modules support? PWA + NOMODULE Redirect to STATIC.

These four primary cases are what I use for any content serving progressive web app I build today. Doesn’t matter the scale, if I’m building a PWA for someone, I consider these four cases the key to serving as many users and bots that I can (though in the case of #4 above, that might serve a different ES5 bundle based on what features I can patch).

Note: I said “content serving” because some PWA’s are offering more native app-like feature sets using things like WebUSB or WebBluetooth and the expectation that these should fallback the same way is not the same in my opinion.

In the case of blog-pwa, this takes place at both the server level (blog-pwa’s Google App Engine backend) and the client level (blog-pwa’s index.html shell). In most PWA buildouts in the last year or so, I always use server and client approaches.

In the case of server side, I check two cases:

Are you a bot or tool in the known list?
Are you a request with the url queryparam static set to true?

The second option works in conjunction with the blog-pwa’s index.html shell, which itself defines two cases to cause clients to redirect with static set to true based on enabled browser feature set:

In the no JavaScript support enabled case, we use noscript and a meta redirect tag.
In the no ES Module support case, we use script nomodule and location.href to handle the redirect.

To make this work, I cheat on first load by injecting a template variable server side into index.html that in modern browsers with JavaScript enabled won’t matter to handle the above two cases:

\<noscript\> \<p\>JavaScript appears to be turned off. No problem, this Progressive Web App is Progressive! Redirecting to static page...\</p\> \<meta http-equiv="refresh" content="0;url={{ noscript }}"\> \</noscript\> \<script nomodule\> // IE11 and anything that doesn't support ES Modules gets the no-JS // version of the site location.href = '{{ noscript }}'; \</script\>

This isn’t fancy or heavy engineering; this is just testing against features. No fancy user agent checking required.

With these cases and the static side setup, I’m able to safely say that the site renders in things like browsers like Lynx quite beautifully.

Even more crazy is that you still get a reasonably readable render in something like Internet Explorer 8.

Are there a lot of users viewing my site in these browsers? No. Does that mean I (or you) shouldn’t plan for this scenario? I’m of the opinion people should be able to read my content even on the most terrible of connections or browsers.

Should it all looks the same in every browser ever designed? No. A lot of folks might disagree, but the engineering effort to make that happen is significantly high and you better be doing some cost-benefit analysis before you jump into those waters. I’ve seen projects crash and burn and take down jobs. Don’t be that team.

Not every PWA might need these cases or offer more fine grained approaches. If you’re not thinking about these cases and you’re only relying on those JavaScript based analytics, do you really know how much syndication or user base you’re missing? Dig a little deeper and you might be surprised.

Adding Webmention support to a Progressive Web App

Justin Ribeiro — Mon, 07 Oct 2019 21:18:51 +0000

On my ongoing quest to independence from third party content silos, one of things that was inherently missing was engagement. Surely I would want some way to interact beyond @justin@ribeiro.social; people should be able to write replies and what not on their own blogs, and I should be able to surface that on the site. Enter Webmention.

Webmention is a W3C recommendation specification that at the top level allows notifications that one URL links to another. You can think of this as a primitive that allows us to build federated services around the things people might write or comment about all around the web.

A simple example is your write a response to one of my posts on your blog, linking to my original post. You then ping said url and express what exactly you did (maybe it was a comment, maybe it was a review), and then my endpoint accepts them and I can display them or reply back as I see fit.

The spec is one of many concepts to come out of IndieWeb, a community focused on people and the independent web. As you probably guessed I really like this concept. Problem was, how do I implement this when it comes to a progressive web app?

How our PWA is built

As I’ve written about before, blog-pwa is my little place I call home. I’ve worked on it for nearly three years and it’s powered my site for a long time with success. Built with web standards like web components and service worker, my custom progress web app blog setup is truly progressive, supporting no-javascript clients and bots via a nifty static generator. For a long time this was limited to linkbots for your social networks and what not, but over the years I’ve tweaked it to support a wide range of other services, including old IE (via no-module), lynx (just because) and Googlebot (static one-to-one representation of the PWA, which I technically don’t have to do anymore).

In IndieWeb parlance I’ve been told this is considered selfdogfooding. I was not aware this was a thing to be honest, but I use what I build to be sure. Term at will.

Having read a lot of documentation from the wiki, I set forth to handle what I hoped would be the easy bit: setting up the receiver.

Receiving webmentions with webmentions.io

webmention.io is an open source and hosted service for receiving webmentions. I’ve initially opted for the hosted version to jump start things and on the surface this seemed pretty simple.

After logging into webmention.io, we’re presented with a couple links to set in the head of our documents:

\<link rel="webmention" href="https://webmention.io/justinribeiro.com/webmention" /\> \<link rel="pingback" href="https://webmention.io/justinribeiro.com/xmlrpc" /\>

Note: you could also do this via http headers if you so please.

With the this now in the head of my document, I decided to just roll the dice and test to see what the various webmention tools out there would think about this. The end result will probably not shock you.

A receiver unseen

With blog-pwa now rocking the proper setup, I went over to Bridgy, an open source connector that takes a look at your social media reactions and turns them into webmentions you can use on your site. After login and little timeline parsing, Bridgy had found some interactions, but was reporting that webmention support on my domain was not setup.

A quick glance quickly revealed what I suspected: it was having trouble parsing the progressive web app output case. Bridgy wasn’t the only one; Telegraph, Webmention.app, IndieWebify.me all initially struggled to properly see the link references.

I could have sat here and complained, but this stuff is open source so I got down to work. blog-pwa can render static output as needed; what I needed was User Agents to point them to the correct case.

In the specification, the fetch GET case is not defined but senders should send “webmention” within their user agent. Since most of these were working with some type of GET to do the initial parsing and targeting of incoming mentions, I just needed to find these User Agents:

# web mentions handlers webmention\_bot\_hunt = ['webmention', 'node-fetch', 'guzzle', 'bridgy', 'go-http-client', 'ruby', 'appengine-google', 'xray']

You might be thinking that guzzle is a PHP HTTP client and node-fetch is a node client and you would be correct. What I found was that most of the GET processors used their respective library’s default User Agent, which means I had to stretch a bit to make them all work. In the case of Webmention.app its user agent was set to unknown (but I patched that last week, cheers Remy for the merge!).

The thing to keep in mind here is that these are fetch-and-parse operations; they’re not doing this via headless Chrome (my temptation to make a webmention parser with headless is running really deep at the moment), so we’re not going to get DOM parsing.

This also extends to h-card and h-entry microformats, which weren’t being read from my PWA body either. This too was needed within the static generation side, allowing for posts and replies to carry more context.

With those fixes in place, all the webmention services I tested had no trouble reading or sending webmentions to my site.

Note: say what you will, but always have a static rendering plan when it comes to PWAs that are displaying content (ala, the word progressive). There are lots of tools out there, including the fabulous Rendertron that can help with this if you’ve not engineering a specific solution and case for static generation.

A little manual webmention action

With data flowing, I wanted to continue tackling collection first. One thing I didn’t want to stop people from doing was just manually sending me a link letting me know “hey, I commented on this”.

To handle this, I wrote a little form (they one at the very bottom of this post) that handles this. By no means is the lower form idea a new one; I had originally seen this concept on Jeremy Keith’s site.

Writing such a form is actually pretty simple. Here, I just a use a standard form with a little click bind from my lit-element and a simple POST:

\<form id="webMentionForm" action="https://webmention.io/justinribeiro.com/webmention" method="POST"\> \<input type="hidden" name="target" .value="${this.metadata.permalink}" /\> \<label for="source"\> Written a response or comment to this post? Fantastic! I support \<a href="https://indieweb.org/Webmention"\>WebMentions\</a\>. Paste and send your URL here: \</label\> \<input type="url" name="source" placeholder="https://your-amazing-response-url-here/" id="webMentionSource" /\> \<button @click="${e =\> this.\_\_submitWebMention(e)}"\> 🚚 Send Webmention \</button\> \</form\>

From a JavaScript perspective, the thing to keep in mind is that we could technically change our headers to request Location header access to see allow the user to see the status of their webmention being processed, but I wasn’t really sold on that from a user experience perspective.

async \_\_submitWebMention(event) { event.preventDefault(); let message = 'Thank you for sharing! Your Webmention has been received and is currently be processed.'; const action = this.shadowRoot.querySelector('#webMentionForm').action; const target = this.metadata.permalink; const source = this.shadowRoot.querySelector('#webMentionSource').value; if (source !== '') { // technically, we could get the location header and show them the ticket, // but I'm not 100% sold on that as a user experience const response = await fetch(action, { method: 'POST', mode: 'cors', cache: 'no-cache', headers: { 'Content-Type': 'application/x-www-form-urlencoded', }, body: `target=${target}&source=${source}`, }); if (!response.ok) { message = "Oh no, your Webmention didn't seem to make it through. Please try again."; } // show our snackbar popup to let the user know it's all good this.dispatchEvent( new CustomEvent('display-snackbar', { bubbles: true, composed: true, detail: { message, }, }), ); this.shadowRoot.querySelector('#webMentionSource').value = ''; } }

Note: if you’re using Content Security Policy, make sure to allow webmention.io in your form target settings otherwise this will not work!

Rendering those mentions (or not)

With mentions now flowing, my final step was to actually render this at the footer of my blog entries. Most of the implementations I’ve reviewed of late handle this at build time (see examples for eleventy, jekyll, and hugo), but webmention.io has an API and it’s easy enough to write a fetch function to get that data in a number of different forms.

The problem became pretty clear to me: if I did this on a every render, I run the risk of pummeling that API for data in high traffic cases. I didn’t think that would be very cool. There is also the risk of large payloads based on interactions and I didn’t really dig that concept either for low bandwidth scenarios (of which I’d rather have content render rather than likes).

What I settled on at the moment is to only render the interaction counts (which currently is a small payload and given that I haven’t parsed a lot of webmentions data yet, not heavy).

async \_\_getInteractionCounts() { const response = await fetch( `https://webmention.io/api/count?target=${this.metadata.permalink}`, { method: 'GET', mode: 'cors', }, ); if (response.ok) { const data = await response.json(); if (data.count \> 0) { this.interactions = `There are currently ${data.count} interactions with this piece on the open web.`; } else { this.interactions = `There are currently no interactions with this piece. Be the first!`; } } }

I suspect I’ll put a tiny cache in blog-pwa’s GAE layer to handle this in the future, so I get a level of control and I can shrink the payloads to a smaller, more succinct set of data I can render. I don’t want to hide those responses.

Looking ahead

I have a few next steps that I’ll be tackling:

I have some PR’s I’m going to push to clean up a lot more user agents, that way if you have a PWA-style static render and are using bot string matching, that you’ll have an easier time.
Along the lines of above, I need to open a ticket and see if that makes sense to put into the spec.
I iced the web component I wrote for the interaction list, but that’s going to get published on NPM.
I’m probably going to implement Remy’s webmention.app CLI tool into my make deployment script. Testing it locally over the weekend, his tool (and site) work great.

In the meantime, if you have a mention you can ping me below in the form

¡Viva la web abierta!

Workaround for Karma and Chrome headless on Windows Subsystem for Linux, ala WSL

Justin Ribeiro — Wed, 02 Oct 2019 22:26:00 +0000

Windows Subsystem for Linux (WSL) is a vastly useful thing. When I do work on Windows for those certain projects, WSL has made my life significantly easier.

One the gaps however is that running Chrome headless within WSL is difficult. I’m not blaming WSL for this (it’s basically a namespace problem which I’d expect). There are workarounds to be sure, usually involving single process and disabling the GPU, but in a recent demo I was running showcasing open-wc and karma, I was in a bind for a workaround on short notice. The fix is it’s faster and easier to point to the pre-installed Chrome binary on Windows.

Doing this requires setting the CHROME_BIN env variable, which will allow chrome headless to work without issue when using karma:

~/src/jdr-wc-template master\* ➜ export CHROME\_BIN=/mnt/c/Program\ Files\ \(x86\)/Google/Chrome/Application/chrome.exe ~/src/jdr-wc-template master\* ➜ yarn test yarn run v1.19.0 $ karma start --coverage START: \<your-component\> ✔ has a default property hello ✔ allows property hello to be overwritten Finished in 0.055 secs / 0.038 secs @ 15:23:16 GMT-0700 (Pacific Daylight Time) SUMMARY: ✔ 2 tests completed TOTAL: 2 SUCCESS

This is actually one of the cooler things about WSL; you can integrate executables between environments.

Will this fix all your headless chrome problems in the WSL? Probably not; some of my basic headless examples don’t run in this configuration. But if you need to run some tests real quick like, this fix just might be for you.

Three Reasons You Betrayed Your Modern Web Tooling

Justin Ribeiro — Tue, 01 Oct 2019 16:46:54 +0000

No one wants to believe the DevTools trace. I have many theories as to why, some proven others less so. One prevailing theory I have is that people don’t understand how to read said trace, so I do the only reasonable thing and teach them. Surely that’ll work I always think, knowledge is power.

Apparently it’s not enough power, as they still won’t believe it.

Finger pointing will ensue. Passive aggressive comments will be made. Somehow it will become not only my fault, but my problem. Which is funny because if it’s really my problem I’ll fix it. I will propose a plan, people will immediately push back and protect their tiny kingdoms.

Among the ensuing fray of competing kingdoms waging their various troops forward shouting the battle cries of yore, “Ops, chaaarrrge”, “UX through the flank”, and “AGGGILEE!!!” it’s clear no one has a clue what’s going on.

Mind you the villagers do. Yeah, they’re your users suffering at the hands of your petty turf war.

A comment from the lead developer leaves their quiver and flies through the air destined for my heart, to end the ensuing threat of more of my trace analysis.

“We’re using modern web tooling and we followed so-and-so’s guide and they’re an expert.”

The arrow strikes true, resulting in a hushed silence upon the battlefield. A smirk comes across my face.

“A craftsperson never blames their tools.” The arrow falls lifeless to the floor, my armour in tact.

You see I hear this argument more often than not these days. Pick a fancy web tool and I’ve probably got a terrible story of poor engineering around it. The three rather generic reasons I’ve seen time and again for this poppycock:

Time investment for learning. Not all tools are intuitive; some are harder to learn than others and that requires time and invest to use correctly in a project to get the performance you may want. I sometimes call this the “WebPack conundrum”.
Single information source. Too many folks send cited sources and articles that gloss over or misstate the impact of said tool usage, yet these often single point information source then becomes the gospel truth in a given team.
Hope-based Development. People don’t put in the work to verify anything, resulting in “write once, it works on my machine, it built so it’s not my problem” thinking. I call it hope-based because I’m being optimistic that people actually are hopeful their code works better, but experience tells me they too often don’t care.

Ultimately these things conspire against the very best web tooling because just because a tool is good doesn’t mean it can’t be misused. A lot of work has been done on many such tools to push warnings and messages to developers in an attempt to prevent such harsh and poor performing development modes from slipping into production, but if you’ve got a team that’s done any or all of the above three points, a warning isn’t going to stop them.

What can you do to prevent such malfeasance? For starters, I’d never take any article or tutorial on the web as a single source of truth. There are rare exceptions, but even in those cases it comes back to doing your due diligence, not just from a single developer but rather a team effort.

Managers: Give your developers time to actually study what they’re working on. Tight deadlines are all fine and dandy until you ship a steaming pile of code that only functions on a set 12 core processors. You want cutting edge? Give them time.
Developers: Know your tools and and which ones are right for the job at hand. What every you choose out of your toolbox, you have to trace. Trace on your desktop, trace on device. Run tests. Find the edges of the performance and tooling. Study reasons why, test your hypothesis.
Designers: Get to know the web. Understand that for all our progress, we have constraints that differ on devices and network connections. Dig in, learn what works and what might not get you to 60 fps.
Quality Assurance: You’re the last line of defence. Get some Lighthouse tests in your mix and verify that web app is going to work for users in a wide range of places and device types. Look beyond upper management’s high speed Wifi and iPhone X existence.

Ultimately it’s not the tools that betrayed you, but rather your approach to them and the code you write. Tools won’t save you from poor engineering; you have to put in not just the work to write the lines of code, but also understand what those lines of code actually going to do. The tools will only get you so far, and then it comes down to actually verifying how that deliverable is going to perform.

When in doubt, run a trace and work through it. If everyone started doing these things, just maybe I’ll get less arrows shot at me at my next meeting.