Forem: Michael Crenshaw

The three meanings of "template" in Argo Workflows

Michael Crenshaw — Fri, 21 Jan 2022 17:22:07 +0000

The word "template" has 3 different meanings in Argo Workflows. Argo is much easier to understand once you know the differences between "template tags," "unit-of-work templates," and "(Cluster)WorkflowTemplates".

1. Template tags

In an Argo Workflow manifest (or in an Argo CronWorkflow, WorkflowTemplate, or ClusterWorkflowTemplate), some fields may contain "template tags". A "template tag" starts with {{ and ends with }}.

There are two types of template tags: variable-substitution template tags and expression template tags.

1.1 Variable-substitution template tags

Variable-substitution template tags look like {{some.variable.here}}. When a Workflow runs, Argo will replace the template tag with the value of the variable. Different variables are available in different parts of a Workflow manifest.

Example:

image: "ubuntu:{{inputs.parameters.version}}"

1.2 Expression template tags

Expression template tags look like {{=some expression here}}. Expressions may also use variables. But instead of simply inserting the variable's value, Argo executes some expression here as expr code. Then the expression template tag gets replaces with the result of that execution.

Example:

count: "{{=inputs.parameters['some-number'] * 3}}"

(Notice that there's a slightly different way to access parameters by name if the parameter name contains a hyphen. expr would have treated the hyphen as a subtraction operator.)

2. Unit-of-work templates

The second use of the word "template" in Argo Workflows is to refer to a unit of work. These templates are located in the Workflow manifest (or CronWorkflow, WorkflowTemplate, or ClusterWorkflowTemplate manifest) under spec.templates.

Example (from coinflip example:

spec:
  templates:
  - name: coinflip
    steps:
    - - name: flip-coin
        template: flip-coin
    - - name: heads
        template: heads
        when: "{{steps.flip-coin.outputs.result}} == heads"
      - name: tails
        template: tails
        when: "{{steps.flip-coin.outputs.result}} == tails"
  - name: flip-coin
    script:
      image: python:alpine3.6
      command: [python]
      source: |
        import random
        result = "heads" if random.randint(0,1) == 0 else "tails"
        print(result)
  - name: heads
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was heads\""]
  - name: tails
    container:
      image: alpine:3.6
      command: [sh, -c]
      args: ["echo \"it was tails\""]

Each item under spec.templates is a unit-of-work template (referred to in documentation simply as a "template".)

There are several template types. The example above contains two container templates, one script template, and a steps template (a type of composite template to execute the other templates in a certain order).

3. (Cluster)WorkflowTemplates

The third and final use of the word "template" in Argo Workflows is to refer to the WorkflowTemplate and ClusterWorkflowTemplate resources. A ClusterWorkflowTemplate is just a WorkflowTemplate that lives at the cluster level instead of a namespace. For the purposes of this description, they're effectively the same.

A WorkflowTemplate has two potential uses.

3.1 WorkflowTemplate as an invokable workflow

When a Workflow is installed on a cluster, the Argo Workflows controller picks it up and immediately starts running it. But sometimes you want a workflow available to be run without actually running it immediately.

You could store the Workflow in git. But then if you have multiple users, you have to rely on them to authenticate with git and pull the latest source whenever they need to run a workflow.

Instead, you could define a whole Workflow and change the kind field in the manifest to WorkflowTemplate. After it's applied to the cluster, anyone with cluster access can run kubectl submit --from workflowtemplate/your-workflow-template. There's no need for git access, and they'll always be running the latest version.

3.2 WorkflowTemplate as a template library

In this case, "template" means the unit-of-work template.

Suppose you have a script-type template that sends a Slack notification. You might want to use this in a lot of different Workflows. Rather than copying and pasting the same template YAML into every Workflow, you could reference a template which is defined in a WorkflowTemplate.

Example WorkflowTemplate:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: slack-workflow-template
spec:
  templates:
  - name: slack-template
    inputs:
      parameters:
      - name: message
    container:
      image: my-slack-notification-image
      command: [send-to-slack]
      args: ["{{inputs.parameters.message}}"]

Example Workflow that uses a template from a WorkflowTemplate:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: task-with-slack-notification-
spec:
  entrypoint: task
  templates:
  - name: task
    steps:
    # other steps here
    - - name: slack
        templateRef:
          name: slack-workflow-template
          template: slack-template
        arguments:
          parameters:
          - name: message
            value: "task is done"

Conclusion

The three uses of the word "template" in Argo Workflows can cause confusion. I recommend using the terms "tag template," "unit-of-work template," and "(Cluster)WorkflowTemplate" to help differentiate.

How to debug an Argo Workflow

Michael Crenshaw — Mon, 01 Mar 2021 14:38:31 +0000

Argo Workflows is a tool for running a series (or a graph) of containers on Kubernetes, tying them together into a workflow.

It's a relatively young project, so things like error messages and documentation still need some work.

In the meantime, here are some steps you can take when your Workflow doesn't behave as expected.

Inspect the Workflow with Argo CLI

For simple issues, the Argo CLI will include a short description in the MESSAGE column of its argo get output.

$ argo get workflow-template-dag-diamond 
Name:                workflow-template-dag-diamond
Namespace:           default
ServiceAccount:      default
Status:              Succeeded
Conditions:          
 Completed           True
Created:             Mon Mar 01 08:51:26 -0500 (7 minutes ago)
Started:             Mon Mar 01 08:51:26 -0500 (7 minutes ago)
Finished:            Mon Mar 01 08:51:36 -0500 (7 minutes ago)
Duration:            10 seconds
Progress:            1/1
ResourcesDuration:   5s*(1 cpu),5s*(100Mi memory)

STEP                              TEMPLATE                                               PODNAME                                   DURATION  MESSAGE
 ✔ workflow-template-dag-diamond  diamond                                                                                                      
 └─✔ A                            workflow-template-whalesay-template/whalesay-template  workflow-template-dag-diamond-2997968480  6s

If there's a message that's just not quite detailed enough to figure out the problem, copy the PODNAME of the failed step, and skip to the section about using kubectl to describe the Pod.

If there's no useful message, try describing the Workflow.

Use kubectl to describe the Workflow

Sometimes there's a problem with the whole Workflow that doesn't fit nicely in argo get's MESSAGE column.

kubectl describe workflow will print a lot more details about the Workflow, including a list of Events. The Events often contain details about what went wrong.

$ kubectl describe workflow workflow-template-dag-diamond
Name:         workflow-template-dag-diamond

...

Events:
  Type    Reason                 Age    From                 Message
  ----    ------                 ----   ----                 -------
  Normal  WorkflowRunning        8m42s  workflow-controller  Workflow Running
  Normal  WorkflowNodeSucceeded  8m32s  workflow-controller  Succeeded node workflow-template-dag-diamond.A
  Normal  WorkflowNodeSucceeded  8m32s  workflow-controller  Succeeded node workflow-template-dag-diamond
  Normal  WorkflowSucceeded      8m32s  workflow-controller  Workflow completed

Use kubectl to describe the Pod

If a Pod fails, there are a number of places which may hold clues. One is the Events associated with the Pods.

Pod names may be unpredictable (they often have random suffixes like -93750129), so use argo get to get the name of the suspect Pod.

Then use kubectl describe po to see the Pod details, including the Events.

$ kubectl describe po workflow-template-dag-diamond-2997968480
Name:         workflow-template-dag-diamond-2997968480

...

Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  36s                default-scheduler  Successfully assigned default/workflow-template-dag-diamond-2997968480 to docker-desktop
  Normal   Pulled     35s                kubelet            Container image "argoproj/argoexec:v2.12.9" already present on machine
  Normal   Created    35s                kubelet            Created container wait
  Normal   Started    35s                kubelet            Started container wait
  Warning  Failed     31s                kubelet            Failed to pull image "docker/whalesay": rpc error: code = Unknown desc = Error response from daemon: Head https://registry-1.docker.io/v2/docker/whalesay/manifests/latest: x509: certificate is valid for auth.docker.io, not registry-1.docker.io
  Normal   Pulling    17s (x2 over 35s)  kubelet            Pulling image "docker/whalesay"
  Warning  Failed     16s (x2 over 31s)  kubelet            Error: ErrImagePull
  Warning  Failed     16s                kubelet            Failed to pull image "docker/whalesay": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: x509: certificate is valid for auth.docker.io, not registry-1.docker.io
  Normal   BackOff    5s (x2 over 30s)   kubelet            Back-off pulling image "docker/whalesay"
  Warning  Failed     5s (x2 over 30s)   kubelet            Error: ImagePullBackOff

This Workflow failed because a proxy issue is preventing pulls from Docker Hub.

Sometimes the problem doesn't show up in the Events, because the failure is inside one of the step's containers.

Use kubectl to read the Pod logs

Pods run as part of Argo Workflows have two or three containers: wait, main, and sometimes init.

The wait sidecar is injected by Argo to keep an eye on the main container (your code) and communicate with the Argo Workflow controller (another Pod) about the step's progress.

The main container is the one you set up when you defined the Workflow in yaml. (Look for the image, command, args, and source items to see part of this Pod's configuration.)

The init container, if present, is also injected by Argo. It does things like pulling artifacts into the Pod.

To read the logs, use kubectl logs. For example:

$ kubectl logs workflow-template-dag-diamond-2997968480 init
error: container init is not valid for pod workflow-template-dag-diamond-2997968480
C02D507EMD6P:test_workflows ekmed$ kubectl logs workflow-template-dag-diamond-2997968480 wait
time="2021-03-01T14:09:18.339Z" level=info msg="Starting Workflow Executor" version=v2.12.9
time="2021-03-01T14:09:18.346Z" level=info msg="Creating a docker executor"
time="2021-03-01T14:09:18.346Z" level=info msg="Executor (version: v2.12.9, build_date: 2021-02-16T22:51:48Z) initialized (pod: default/workflow-template-dag-diamond-2997968480) with template:\n{\"name\":\"whalesay-template\",\"arguments\":{},\"inputs\":{\"parameters\":[{\"name\":\"message\",\"value\":\"A\"}]},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"docker/whalesay\",\"command\":[\"cowsay\"],\"resources\":{}}}"
time="2021-03-01T14:09:18.346Z" level=info msg="Waiting on main container"
time="2021-03-01T14:14:17.998Z" level=info msg="Alloc=4699 TotalAlloc=14633 Sys=70080 NumGC=6 Goroutines=7"

The logs from init and wait may be a bit difficult to read, because they come from Argo. The logs for main will be from your configured image, so they'll probably be more familiar.

Use kubectl to read the Workflow controller logs

Argo comes with a Pod called the "Workflow controller" to sort of usher a Workflow through the process of running all its steps.

If all the other debugging techniques fail, the Workflow controller logs may hold helpful information.

First, find the Pod name. If you used the default Argo installation command, the Pod will be in the argo namespace.

$ kubectl get po -n argo
NAME                                  READY   STATUS    RESTARTS   AGE
argo-server-6bb488c6c8-ff88g          1/1     Running   0          40m
workflow-controller-57db6b46f-7qfr9   1/1     Running   0          40m
$ kubectl logs workflow-controller-57db6b46f-7qfr9 -n argo

... lots of stuff here ...

Ask for help

If none of these solves your problem, ask a question on StackOverflow, start a discussion on GitHub, or ask in the Argo Slack.

These are just a few of my go-to tools. If I'm missing anything, please comment!

How do you know when to NOT refactor?

Michael Crenshaw — Tue, 02 Jul 2019 20:50:27 +0000

What Sparked the Question

I recently leafed through The Power of TED*," a book-length parable about destructive cycles of human psychology in relationships.

At first I was annoyed by the abstractions. The "Dreaded Drama Triangle" made up of Victim, Persecutor, and Rescuer roles just seemed too neat and tidy to actually describe reality. It seemed the author was under the delusion that they'd pushed aside some curtain and glimpsed Reality Proper.

But then I realized the abstractions are just abstractions - useful heuristics to help people ask and answer the right questions. "Am I thinking of myself as a victim, and how might that impact my relationships? Do I recognize this kind of cycle in my life?"

(Aside: recognition of victim-hood can be useful. From what I can tell, the book thoughtfully avoids judgment of people who recognize "persecution" and instead focus on how to break harmful cycles of behavior. If people choose to wield the book's ideas judgmentally, that's obviously bad.)

The Question

Someone recently remarked to me, "A developer's first reaction to anyone else's code is 'This is rubbage, let's redo it all.'"

Though hyperbolic, the remark hit home. I refactor by default, and I don't have enough tools to recognize when I should leave less-than-perfect code alone.

So I wonder: How do you determine when to not refactor? What questions do you ask yourself or others? What kind of "gut feelings" do you encounter?

I'm also interested in hard-and-fast rules, but I suspect many readers will have more approximate tools at hand.

Don't forget to ask Babel to compile .mjs files

Michael Crenshaw — Fri, 17 May 2019 15:29:02 +0000

I spent several hours trying to understand why Babel wasn't transpiling const statements in Jeremy Wagner's yall.js library.

It was because the actual filename is yall.mjs, and I hadn't configured Babel to handle that extension.

The correct test configuration ended up being as follows:

test: /\.(m?j|t)s$/,

Or, if you don't need TypeScript:

test: /\.m?js$/,

Suffer me not to ponder the ills and advantages of the .mjs extension. That's for smarter people.

But since some folks do use that extension, it's probably worth updating your Babel config and related documentation/tutorials.

Writing modular controls in ASP.NET Web Forms

Michael Crenshaw — Fri, 03 May 2019 16:06:21 +0000

Most sites have common components, like headers and footers. Some components exhibit only slight variations across the site.

You can copy and paste the common parts of the markup. But that can lead to headaches when it's time to make a change to each copy of the component.

ASP.NET provides a way to be more modular, with nested components.

A familiar example of nesting is with the built-in Repeater control.



<asp:Repeater ID="ExampleRepeater" runat="server">
   <HeaderTemplate></HeaderTemplate>
   <ItemTemplate></ItemTemplate>
   <FooterTemplate></FooterTemplate>
</asp:Repeater>

The Repeater has common behavior that's applied to the content in the Template tags.

Take a look at this markup from the boilerplate ASP.NET Web Form site that comes with Visual Studio 2017:



<div class="row">
    <div class="col-md-4">
        <h2>Getting started</h2>
        <p>
            ASP.NET Web Forms lets you build dynamic websites using a familiar drag-and-drop, event-driven model.
        A design surface and hundreds of controls and components let you rapidly build sophisticated, powerful UI-driven sites with data access.
        </p>
        <p>
            <a class="btn btn-default" href="https://go.microsoft.com/fwlink/?LinkId=301948">Learn more &raquo;</a>
        </p>
    </div>
    <div class="col-md-4">
        <h2>Get more libraries</h2>
        <p>
            NuGet is a free Visual Studio extension that makes it easy to add, remove, and update libraries and tools in Visual Studio projects.
        </p>
        <p>
            <a class="btn btn-default" href="https://go.microsoft.com/fwlink/?LinkId=301949">Learn more &raquo;</a>
        </p>
    </div>
    <div class="col-md-4">
        <h2>Web Hosting</h2>
        <p>
            You can easily find a web hosting company that offers the right mix of features and price for your applications.
        </p>
        <p>
            <a class="btn btn-default" href="https://go.microsoft.com/fwlink/?LinkId=301950">Learn more &raquo;</a>
        </p>
    </div>
</div>

That code renders the three columns visible on the homepage:

Let's consolidate the common parts of the markup into a modular user control.

Setting up the project

If you know how to set up an ASP.NET Web Forms project in Visual Studio 2017, or are doing so in a different IDE, feel free to skip to the next section.

First, create a new project. I'm using the C# flavor of an ASP.NET Web Application project.

File > New > Project...

Choose the Web Forms type of Web Application.

(I know Web Forms is passé, but many sites still use it. As far as I can tell, it's still a perfectly acceptable way to build a site.)

If you launch the site in a browser, you'll see the default homepage as shown above.

We're going to create a new user control to hold the common "column" code.

Setting up the modular control

I'm creating a new "Controls" folder, just to stay organized.

Solution Explorer > Right-click ModularControls project > Add > New Folder

Now create the control. I'm calling it a "homepage card," though you'd probably only ever bother to create a modular control for something that's used on multiple pages.

Solution Explorer > Right-Click Controls folder > Add > Web Forms User Control

Copy and paste the code for any of the columns to your new control. Then replace the contents of the header and paragraph tags with appropriately-named <asp:PlaceHolder> controls.



<div class="col-md-4">
    <h2>
        <asp:PlaceHolder ID="TitlePlaceHolder" runat="server"></asp:PlaceHolder>
    </h2>
    <p>
        <asp:PlaceHolder ID="ContentPlaceHolder" runat="server"></asp:PlaceHolder>
    </p>
    <p>
        <asp:PlaceHolder ID="FooterPlaceHolder" runat="server"></asp:PlaceHolder>
    </p>
</div>

Then set up the control's code-behind something like this:



using System;
using System.Web.UI;

namespace ModularControls.Controls
{
    [ParseChildren(typeof(Control), DefaultProperty = "Content", ChildrenAsProperties = true)]
    public partial class HomepageCard : UserControl
    {
        public HomepageCard()
        {
            Title = new ControlCollection(this);
            Content = new ControlCollection(this);
            Footer = new ControlCollection(this);
        }

        [PersistenceMode(PersistenceMode.InnerProperty)]
        public ControlCollection Title { get; private set; }

        [PersistenceMode(PersistenceMode.InnerDefaultProperty)]
        public ControlCollection Content { get; private set; }

        [PersistenceMode(PersistenceMode.InnerProperty)]
        public ControlCollection Footer { get; private set; }

        protected void Page_Load(object sender, EventArgs e)
        {
            foreach (Control control in Title)
            {
                TitlePlaceHolder.Controls.Add(control);
            }

            foreach (Control control in Content)
            {
                ContentPlaceHolder.Controls.Add(control);
            }

            foreach (Control control in Footer)
            {
                FooterPlaceHolder.Controls.Add(control);
            }
        }
    }
}

There's a lot going on, so I'll break down the key parts.



[ParseChildren(typeof(Control), DefaultProperty = "Content", ChildrenAsProperties = true)]

The three arguments of the ParseChildren attribute instruct ASP.NET to

parse nested content as Control objects (typeof(Control))
if no section is specified (we'll look at how that's done below), parse controls into the Content property
parse children into properties, as defined below



public HomepageCard()
{
    Title = new ControlCollection(this);
    Content = new ControlCollection(this);
    Footer = new ControlCollection(this);
}

The default constructor news up the control collections so they're ready to accept controls parsed by ASP.NET. The this argument specifies the "owner" control of these control collections.



[PersistenceMode(PersistenceMode.InnerProperty)]
public ControlCollection Title { get; private set; }

[PersistenceMode(PersistenceMode.InnerDefaultProperty)]
public ControlCollection Content { get; private set; }

[PersistenceMode(PersistenceMode.InnerProperty)]
public ControlCollection Footer { get; private set; }

The PersistentMode attributes tell ASP.NET how the contents are persisted into the rendered page. Note that the Content property gets marked as the InnerDefaultProperty.

Each ControlCollection needs the setter explicitly set to private. If you skip this step, you'll get difficult-to-debug runtime errors.



protected void Page_Load(object sender, EventArgs e)
{
    foreach (Control control in Title)
    {
        TitlePlaceHolder.Controls.Add(control);
    }

    foreach (Control control in Content)
    {
        ContentPlaceHolder.Controls.Add(control);
    }

    foreach (Control control in Footer)
    {
        FooterPlaceHolder.Controls.Add(control);
    }
}

The Load event handler loops through the parsed controls and drops them into the appropriate placeholders.

Using the modular control

First, register the control in Default.aspx.



<%@ Register Src="~/Controls/HomepageCard.ascx" TagPrefix="ex" TagName="HomepageCard" %>

Then replace the first column with the following code.



<ex:HomepageCard runat="server">
    <Title>
        <asp:PlaceHolder runat="server">
            Getting started
        </asp:PlaceHolder>
    </Title>
    <Content>
        <asp:PlaceHolder runat="server">
            ASP.NET Web Forms lets you build dynamic websites using a familiar drag-and-drop, event-driven model.
            A design surface and hundreds of controls and components let you rapidly build sophisticated, powerful 
            UI-driven sites with data access.
        </asp:PlaceHolder>
    </Content>
    <Footer>
        <asp:PlaceHolder runat="server">
            <a class="btn btn-default" href="https://go.microsoft.com/fwlink/?LinkId=301948">Learn more &raquo;</a>
        </asp:PlaceHolder>
    </Footer>
</ex:HomepageCard>

The Title, Content, and Footer tags tell ASP.NET where to place the contents.

The <asp:PlaceHolder> controls allow ASP.NET to parse the markup as controls.

It's a bit verbose. But once the verbose part is written, it never has to be changed. Common parts of the controls can be changed in HomepageCard.ascx.

If you rebuild and reload the home page, it should look precisely the same as before you wrote the modular control. That's the idea! This pattern doesn't change the rendered content - it just makes it more modular and maintainable.

Improvements?

When researching this technique, I found very little documentation. Since the first version of ASP.NET Web Forms was released in 2002, it's possible most of the guides for this technique are in printed form.

If you know tweaks to improve this pattern, please let me know! I'd love to drop the PlaceHolder tags to tighten up the markup. And I'd really love to drop all the properties and placeholder population from the code-behind in favor of some more expressive syntax. But I'm not sure either is possible.

I hope this is a helpful pattern! If so, please comment and let me know how you used it!

Should I finish loading lazy images while the browser is idle?

Michael Crenshaw — Tue, 16 Apr 2019 13:51:45 +0000

I've heard of JS packages that prefetch linked pages while the browser is idle.

Would there be any downside to doing something similar with lazily-loaded images?

At work, I'm on a fast connection. The Crutchfield home page loads very quickly, partly because below-the-fold images are lazy loaded.

But when I scroll down, even though I've been idle a few seconds, lazy-loaded images still "snap" into place. Not an ideal experience.

Shouldn't idle time be used to complete the page load?

Questions that come to mind:

How do I determine "idle"? requestIdleCallback, setTimeout(loadImages, 5000), etc.?
How do I performantly order the images to be eagerly loaded? Slowly expand the margin of an IntersectionObserver? (Is that even possible?)
How will this interact with the new loading attribute? Would I simply adopt the same strategy, just setting loading="eager" instead of changing the src attribute?

I'd love to know your thoughts, especially if you've implemented this - or explicitly chosen not to implement it for some reason.

How do you support detailed validation error messages?

Michael Crenshaw — Mon, 15 Apr 2019 18:17:04 +0000

I was irked by this Tweet from Sebastian McKenzie:

If you’re writing validation code, please never make it output “Incorrect foo” or “Invaid bar”. Pleeeaaase just tell me what I need to fix, and why it’s invalid.

It irked me not because Sebastian is wrong... on the contrary, it hits too close to home. I need to implement more detailed validation messages for some of Crutchfield's inputs, but it's not clear how I should do it.

For example, take this email validation regex:

^[-a-zA-Z0-9+.*!#$\/=_]+@((([a-zA-Z0-9][-a-zA-Z0-9]{0,61}[a-zA-Z0-9])|([a-zA-Z0-9]))\.)+(([a-zA-Z0-9][-a-zA-Z0-9]{0,61}[a-zA-Z0-9])|([a-zA-Z0-9]))$

(I know it's not great to validate email with regex - but it works for 99.999% of real-world email addresses, and it helps users catch common typos.)

A number of things can cause an address to fail validation, but the regex only communicates two states: valid or invalid.

So how do I communicate to the user what invalidated the input? I could imagine designing a series of validations using parts of the above pattern:

    [
        {
            "pattern": "^[-a-zA-Z0-9+.*!#$\/=_]",
            "message": "Your email must start with a letter, number, or one of these symbols: - + . * ! # $ \/ = _"
        },
        {
            "pattern": "@",
            "message": "Your email address must contain @"
        }
    ]

But there are a couple problems:

There's code duplication, complicating maintenance.
Applying the validation requires JavaScript (the pattern attribute won't suffice).
It becomes less clear for developers what is the "true" validation (which should also be used server-side) and what is just used for helpful messaging.

Here is a potential solution:

Call the additional patterns "typo checks" instead of "validation," making clear they're only used to help resolve simple typos, not provide a complete validity check.
Don't treat input as invalid if only the typo check fails. If the main validation pattern matches, treat the input as valid; if it doesn't match, check the other patterns to find a "friendly message."

At least that's my current plan of attack.

Does that make sense? How does your site/app handle this?

How much of a page is occupied by images?

Michael Crenshaw — Sun, 14 Apr 2019 16:41:51 +0000

I was curious how much of a page's area was occupied by images. So I hacked together a rough approximation:

function PercentOfPage(selector) {
    return [...document.querySelectorAll(selector)].reduce(
            (a, i) => a + i.offsetWidth * i.offsetHeight,
            0
        ) / (document.body.offsetHeight * document.body.offsetWidth);
}

For my purposes I'd use const imgArea = PercentOfPage('img');.

It doesn't take into account whether the target elements are actually visible (for example, they might be positioned off-page or behind another element). But it's good enough for my purposes.

Please offer suggestions! I'd love to fine-tune this utility a bit more.

P.S.: this page is ~1% images.

When the browser can't take a (preconnect) hint

Michael Crenshaw — Sun, 07 Apr 2019 22:23:05 +0000

Resource hints help pages load faster by telling the browser what assets it'll need in the future. For example, a preload hint for a font file tells the browser to start downloading the font before stylesheets are parsed.

<head>
  <!-- other stuff -->
  <link rel="preload" href="https://cdn.example.com/font.woff2" as="font" type="font/woff2" crossorigin>
  <link rel="stylesheet" href="https://cdn.example.com/styles.css" type="text/css">
</head>

Sometimes you don't know ahead of time what assets are needed in a page. But you know they'll be hosted on a particular domain. In that case, you can give the browser a head start with the preconnect resource hint.

<head>
  <!-- other stuff -->
  <link rel="preconnect" href="https://cdn.example.com/" crossorigin>
</head>

Without resource hints

Suppose a page on my site uses a stylesheet on another origin, and that stylesheet defines a custom font.

@font-face {
  font-family: 'Custom Font';
  src: url(https://mac9416.com/demo/preconnect/font.ttf) format('truetype');
}

body {
  font-family: 'Custom Font';
}

Without resource hints, my page's HTML must be parsed before the stylesheet can be downloaded.
(In this example, I've moved the stylesheet link below a massive block of "lorem ipsum" text, to simulate a late-discovered stylesheet and make the waterfall chart easier to read.)

In the above WebPageTest waterfall chart, the page is completely downloaded and parsed before Chrome connects to mac9416.com to download the stylesheet.

preconnect with crossorigin

The asset will load faster if I add a preconnect hint for mac9416.com. (Remember, we're pretending we don't know specifically which assets will be downloaded from mac9416.com. Otherwise we would add preload hints for an even greater performance boost.)

<head>
  <!-- other stuff -->
  <link rel="preconnect" href="https://mac9416.com/" crossorigin>
</head>

With a preconnect hint in place, the waterfall looks better. A DNS lookup for mac9416.com happens immediately after the first chunk of HTML is downloaded. And the connection used to download my custom font happens immediately after the DNS lookup is finished. But it looks like there's another connection to mac9416.com that's initiated after HTML is downloaded and parsed. That second connection is used to download the custom font.

preconnect without crossorigin

In the above code sample, I naiively copied an example and left the crossorigin attribute in place. I just assumed it meant "this connection is to a different domain" -- which it is. Let's see what happens when I remove that attribute.

<head>
  <!-- other stuff -->
  <link rel="preconnect" href="https://mac9416.com/">
</head>

In this waterfall chart, we seem to have the same problem, but with the connections swapped. The connection used to download styles happens immediately, but the connection used to download fonts begins after the stylesheet is downloaded and parsed.

Why did crossorigin break things?

What's going on? I incorrectly assumed crossorigin simply meant "the target is on another domain." But the browser could infer that by comparing the <link> element's href attribute to the current page's origin. So what is the crossorigin attribute for?

crossorigin actually tells the browser that "resources on this connection are downloaded using CORS."
By default, it specifically means "CORS without credentials."

CORS improves web security. That's all I'll say about it here, because smarter people have explained it better elsewhere.

To speed up this web page, all we need to know is that resources downloaded without CORS are downloaded on a separate connection from those that use CORS.

A quick glance at a list of requests that use CORS shows that our font request will use CORS, but the stylesheet request will not.

preconnect with crossorigin and without

So let's use two preconnect hints, one for non-CORS requests, and the other for CORS requests.

<head>
  <!-- other stuff -->
  <link rel="preconnect" href="https://mac9416.com/">
  <link rel="preconnect" href="https://mac9416.com/" crossorigin>
</head>

This waterfall chart looks much better. With two resource hints in place, both connections to mac9416.com are established immediately after the first chunk of HTML is parsed. Incidentally, we've achieved the fastest time to document complete (the blue line in the waterfall chart) of all the tests.

Just for good measure, I'll add a dns-prefetch hint for browsers that don't support the preconnect hint.

<head>
  <!-- other stuff -->
  <link rel="dns-prefetch" href="https://mac9416.com/">
  <link rel="preconnect" href="https://mac9416.com/">
  <link rel="preconnect" href="https://mac9416.com/" crossorigin>
</head>

There's no need for a crossorigin attribute, since DNS queries are performed without CORS.

For this experiment I've ignored crossorigin="use-credentials". I suspect CORS requests with credentials would require a third hint.

Summary

The crossorigin attribute, when used with rel="preconnect", doesn't describe where the target origin is but rather what kind of assets will be downloaded from that origin. If the assets use CORS, crossorigin is needed. If CORS won't be used, crossorigin should be omitted. If both types of assets will be present, two resource hints are necessary.

If you've found resource hints and crossorigin confusing, don't feel bad. There's a lot going on. If you find this guide confusing, please contact me! The fault is probably mine, and I'll be happy to clarify.

Originally posted on crenshaw.dev.