Forem: fiiv

So I made a remote cache service

fiiv — Thu, 29 May 2025 04:53:36 +0000

There's so many data APIs out there (check out Public APIs for some options) - near-infinite possibilities of great things you could build with them.

Building data-driven applications is fun and challenging at the same time. One of the challenges I have encountered again and again in building these types of applications concern grabbing the data itself.

Recently, I was working on one myself - Bangkok Rent Map. Unfortunately though, I ended up dealing with three issues that come with external APIs - quota/cost per-request, speed of the requests, and marshalling the data I need from several different endpoints.

Cache Horse is the solution to these problems - it's a remote caching service!

So what does it do?

Quite simply, Cache Horse allows you to cache every single HTTP request you make. Instead of this:

GET https://api.example.com/products

You'd do this:

GET https://api.cache.horse/get?uri=https://api.example.com/products

That's it! Now the GET /products request result will be nicely cached for you. Best part is, you can expire it at any time by adding a refresh=true - or by automatically expiring it by setting an expiry time with ttl=10000000.

But wait - there's more!

You can also batch requests together for your convenience. So, instead of making 2 separate requests:

GET https://api.example.com/categories
GET https://api.example.com/products

You can just combine it into one by sending it to us with a | (pipe character) separator:

GET https://api.cache.horse/get?uris=https://api.example.com/categories|https://api.example.com/products

Now these requests will be cached (individually) and returned to you like so:

// The returned data is in an array - with the shape of URI string followed by the data:
[
  "https://api.example.com/categories",
  { "categories": [ "shirts", "pants", "underwear" ] },
  "https://api.example.com/products",
  { "products": [ { "id": 1, ... }, { "id": 2, ... }, ... ] }
]

So, how would I use this?

Imagine you have a React component that needs to load both GET /products and GET /categories. It might look like so:

import { useState, useEffect } from 'react'

const getData = async (endpoint) => {
  const request = await fetch(`https://api.example.com/${endpoint}`)
  return request.json()
}

const MyComponent = () => {
  const [ products, setProducts ] = useState([])
  const [ categories, setCategories ] = useState([])

  useEffect(() => {
    getData('products').then(({ products }) => setProducts(products))
    getData('categories').then(({ categories }) => setCategories(categories))
  }, [])

  return (
    <div>
      <aside>
        <p>Categories:</p>
        {categories.map((category) => (<p>{category}</p>))}
      </aside>

      <ul>
        {products.map((product) => (<li>{product.name}</li>))}
      </ul>
    </div>
  )
}

export default MyComponent

The only thing we really need to do to make this work with Cache Horse is to re-write our fetching code. Instead of grabbing GET /products and GET /categories directly, we will simply route those requests through the Cache Horse proxy:

import { useState, useEffect } from 'react'

const CACHE_HORSE_API_KEY = '<your-api-key>'
const CACHE_HORSE_API = 'https://api.cache.horse/get'

const getData = async (endpoints) => {
  /*
    We need to build a query parameter like so:
    uris=https://api.example.com/products|https://api.example.com/categories
  */
  const uris = endpoints.map(path => encodeURIComponent(`https://api.example.com/${path}`))
  const request = await fetch(`${CACHE_HORSE_API}?uris=${uris.join('|')}&api_key=${CACHE_HORSE_API_KEY}`)
  return request.json()
}

const MyComponent = () => {
  const [ products, setProducts ] = useState([])
  const [ categories, setCategories ] = useState([])

  useEffect(() => {
    getData(['products', 'categories']).then(([ , products, , categories ]) => {
      setProducts(products)
      setCategories(categories)
    })
  }, [])

  return (
    <div>
      <aside>
        <p>Categories:</p>
        {categories.map((category) => (<p>{category}</p>))}
      </aside>

      <ul>
        {products.map((product) => (<li>{product.name}</li>))}
      </ul>
    </div>
  )
}

export default MyComponent

Done! Now the request comes back in one go, and is now cached.

The advantage of Cache Horse

Obviously, there's a lot of variables in measuring the impact of using a remote cache like Cache Horse. But to show it off a bit, here's a quick and easy benchmark we put together to show the difference (source code is available on Github).

Here, we pulled 4 data APIs we found. We chose these because they require no authentication and present unique challenges. For example, world.openfoodfacts.org sends a massive payload of data back.

+----------------------------+--------------------+---------------------+--------------------+
| Domain                     | Raw request time   | Cached request time | Time savings       |
+----------------------------+--------------------+---------------------+--------------------+
| api.domainsdb.info         | 1.5706900240002142 | 0.3069977329996618  | 1.2636922910005524 |
| world.openfoodfacts.org    | 27.751143773000877 | 1.1871313930005272  | 26.56401238000035  |
| api.openbrewerydb.org      | 0.9226720890001161 | 0.3071872029995575  | 0.6154848860005586 |
| api.carbonintensity.org.uk | 0.8186340400006884 | 0.4737608600007661  | 0.3448731799999223 |
+----------------------------+--------------------+---------------------+--------------------+

These requests were done separately to our API, but of course, if you batch them using our multiple HTTP request cache, you can expect to get all the data back in one big request.

Additionally, since we query things only if the cache has expired (or, you want to force a refresh), the quota problem with external API services is reduced. This can help with data intended to be fresh, but not too fresh.

Your mileage may vary - so Cache Horse has a free plan to try it out for yourself.

Give it a try, and most importantly, let us know how it goes!

One simple trick to increase Stimulus.js legacy browser compatibility

fiiv — Fri, 07 Feb 2025 05:04:25 +0000

Stimulus.js states in its installation documentation:

Stimulus supports all evergreen, self-updating desktop and mobile browsers out of the box. Stimulus 3+ does not support Internet Explorer 11.

In my latest project, I found that older iOS Safari browsers also had trouble with one particular part of the code. To understand it, here's the basic sample Stimulus hello_controller.js:

import { Controller } from "stimulus"

export default class extends Controller {
  static targets = [ "name", "output" ]

  greet() {
    this.outputTarget.textContent =
      `Hello, ${this.nameTarget.value}!`
  }
}

The problem is here:

static targets = [ "name", "output" ]

It will lead to the following error message on older Safari browsers:

SyntaxError: Unexpected token '='

The issue is the static keyword. However, there's a simple workaround for this:

import { Controller } from "stimulus"

export default class extends Controller {
  static get targets() {
    return [ "name", "output" ]
  }

  greet() {
    this.outputTarget.textContent =
      `Hello, ${this.nameTarget.value}!`
  }
}

So, what did we do?

static get targets() {
  return [ "name", "output" ]
}

Well, the static keyword to create static fields is only available in Safari after 14.4 (2020).

BUT - static has been around for much longer to define methods (including getter methods with the get keyword), in Safari 8.4+ (2014).

With this simple trick (and honestly, it's not that much of a pain to refactor), you can extend support of your applications for an admittedly small segment of legacy audiences.

Everything you've (n)ever wanted to know about favicons

fiiv — Tue, 26 Oct 2021 19:43:43 +0000

A favicon is quite simply an icon. The idea behind its inception was for browsers to use them to make a quick, easy to identify visual representation of a website in the browser's UI.

A little bit of history

The first favicons were implemented in 1999 by Microsoft in their Internet Explorer browser. Since then, they were widely adopted by almost all browsers, and today it is commonplace to find favicons being displayed all over the browser, such as in bookmarks menus, window and tab headings, address bars, history, etc.

Microsoft's original idea was to load that icon from a specific place relative to a website. If your website's address was pokemon.com, Internet Explorer looked for it at pokemon.com/favicon.ico.

This original implementation worked alright at first. But at the time, lots of websites were hosted with companies like Geocities, Angelfire and Tripod. They would give your website an address like tripod.com/my-pokemon-website. The problem with this was that the browser would always use Tripod's favicon located at tripod.com/favicon.ico and not one that the website's author might want to show!

Eventually, the people responsible for standardising HTML on the web, the World Wide Web Consortium (W3C), created a specification for how both browsers and website authors should use these icons. In the HTML of the page, an HTML tag would be included, telling the browser where to find that icon.

But, since the original Internet Explorer needed to work because lots of websites had been using it, browsers began supporting them both.

The advent of the smartphone (and other use-cases)

A lot of things changed when the smartphone became mainstream. One notable thing that changed for favicons was that people could save websites to their home screen. But the favicon.ico icons up to this point were all too small – sometimes they were as tiny as 16 pixels! These icons were supposed to be shown next to app icons, which are much more high fidelity images – up to 10 times bigger, in fact!

Android and iOS use different icon sizes. Also, Windows began using yet another icon type for displaying websites in its Metro tile display.

The chaos is real

As a result of all these different specifications, there are now a lot of places to look for favicons. And the different icons you can get will vary depending on which one you request. Some sites will have some but not others!

The list includes:

yoursite.com/favicon.ico
In the site's HTML code
In the site's web manifest file

Some sites end up with dozens of different icons. These vary, from the usually tiny favicon.ico to the higher resolution icon used by iOS's home screen feature. Additionally, a lot can go wrong with fetching these.

There might not be any icons at all.
The icons are specified but don't exist (return a 404).
The icons are the wrong size.
The icons are the wrong format for where you might want to use them.
The icons are corrupted or broken.
The icons specified might not have the one you might want to use.
The web server could be misconfigured (for example, resulting in redirect loops).
The web server could be slow to respond with the icons or the files specifying the icons.

So, requesting something as simple as a tiny little website icon can be a very complicated task! Clearly there is no one-size-fits-all solution to grab the icons.

How to make sure your website has all its icons

So all that being said, how do you make sure your website has all the different icons it should have, and they're properly referenced in your HTML markup?

Quite simply, the best way to do this is to use a service that generates the markup and icons for you. The best ones I've found so far are realfavicongenerator.net and Website Planet's Favicon Generator. You can upload an icon image, specify some preferences, and it will generate a ZIP file with your icons properly sized and set up, as well as the HTML to insert into your site's <head> section.

If you use Webpack, you could also try out the plugin jantimon/favicons-webpack-plugin that will automatically work its magic to give you favicons.

How to use other sites' favicons on your site

You could build your own API or serverless function to fetch a site's favicons. But, I gotta tell you from experience, there's a lot of edge cases and weirdness here.
You could always load domain.com/favicon.ico, but be prepared for 404s.
You could use an existing service. Shameless plug – I built Icon Horse for this purpose, you can use it free (or upgrade for additional features). Other services also exist, like Favicon Grabber. Some search engines like Google, Bing and DuckDuckGo also have APIs for this, but since they're intended for internal use and are undocumented and unsupported, be prepared that they might not work like you want or they might stop working at any moment.

Other cool stuff

Iconmap.io: a massive map of favicons from all over the web
realfavicongenerator.net: the best tool I've found for generating favicons for your website
Website Planet's Favicon Generator: another great tool for generating favicons for your site
Icon Horse: my super cool favicon grabber API
Let me know if you can think of a link for here and I'll add it

How & why I built a favicon fetching service

fiiv — Fri, 03 Sep 2021 20:27:28 +0000

A few days ago, I launched Icon Horse, which is a simple, free API to quickly get the favicon of any website. The launch itself went really well, and I had a lot of fun building it.

But inside the seemingly simple but glamorous life of favicons, there is a lot of complexity. I thought I'd share some of how it works with you all, and some of the hoops I had to hop to build Icon Horse.

What is a favicon, really?

The year was 1999. Britney Spears and Eminem were at the top of the charts, the world was introduced to Napster for the first time, and the "browser war" was heating up with Microsoft releasing Internet Explorer 5.

One of IE5's new features was the favicon, or a small icon which was displayed in the "Favourites" menus/bars next to the title of the site someone bookmarked. Things were simple back then, and if the favicon existed, it would be loaded from the site's root like so:

https://mysite.com/favicon.ico

Since then, a number of new circumstances have come up requiring new types of icons – one example is the advent of the smart phone, which allowed people to save a website shortcut on their home screen. But since favicons tended to be low resolution, a number of diverging standards came about and added these on both Android and iOS.

Now while technically a shortcut icon, they are usually lumped in with favicons in general internet terminology and also during the process to create them.

Today, there are three main places to look for icons:

https://mysite.com/favicon.ico
The HTML of the site in the <head>
A separate Web App Manifest file which is specified in the <head>

So what's the problem?

I'm currently working on a tool called Meeting Canary, from which a note-taking interface is displayed for a given calendar meeting. Many meetings have links to relevant places, such as video conferencing apps (especially since the COVID-19 situation has moved many people to remote work).

I decided it would be nice to render a small icon next to the links, as a way to tease the content to my users:

After hunting for a good way to fetch icons for a given link, I found one that was a JSON API endpoint, but I wasn't very satisfied with this solution – I did not want to complicate my life by using it, since I still needed to write code to figure out the best icon to display from the list I got back.

Also, I did not find a single service that provided fallbacks, or an icon that would be shown if the site is unreachable or if it had no icons at all.

After all, Gravatar does this very well for email addresses, and it was strange that no one had done this with favicons yet! So I got to work.

It was really simple to make, right?

When it comes to standards, the web is pure chaos. When building a product to fetch favicons, you should expect no mercy.

Some sites have no icons at all. Some sites have only a few of the icons from the spec. Some sites use strange sizes. Some sites don't even bother to tell you the size (in pixels) of the icons.

Some sites have completely broken DNS or server issues (like infinite redirect loops). Some send confusing or broken headers. Some sites serve invalid HTML or JSON. Some sites have 404 Not Found errors on their icons. Some sites used weird caching schemes. Some don't specify a MIME type or a file extension so you have to parse the actual image to know what you're dealing with.

The list of difficulties goes on – for example one prominent clothing retailer's icon is simply not loadable because they have a nasty bug in their site's redirects and headers meaning you cannot simply redirect: 'follow' your way to the HTML page, but must chain one request after the other manually.

But I persevered through all those.

I knew there was going to be even more edge cases I hadn't considered in the future, so it was very important to build in functionality for fallback images. I never wanted to serve a broken image or a timeout.

I also had to make some decisions. Since my service was making icons available to all who wanted them, there was no telling the different use cases they would be serving. So I made the assumption that the best icon to serve would be the most high fidelity image. Also, for some use cases (such as React Native), SVG format icons would not work out of the box, and needed something like react-native-svg to get them to load, so I stuck to raster formats only (for the first version at least, I plan to open up SVGs as well in the future via a query parameter).

And finally, pulling all this content is time intensive. Consider that to serve an icon, one needs to:

Load https://asite.com's HTML site
Parse the HTML and do a query lookup to get relevant icons and the manifest file
Load the manifest file and parse it
Check for the existence of the https://asite.com/favicon.ico icon
Merge all these icons together in a list and sort them based on a best to worst criteria, while also making sure the icons themselves are reachable (and don't 404 Not Found)
If all this fails, generate a JPEG file on the fly as a fallback
Serve the icon to the user

So when dealing with a slow server, it's possible that loading that icon could take quite a long time. To help with that, Icon Horse must cache the resulting list of icons and the resulting chosen icon.

I wanted to keep the functionality super simple, so when someone queried my API to get an icon:

https://icon.horse/icon/dev.to

I ended up with just the icon:

Putting it all together

From the very beginning, I knew I wanted to use a serverless approach to this service. The landing page is built out of Next.js and the service itself sits in a lambda function hosted on Amazon.

After struggling a little bit with getting the environment set up (it had to have image processing capability as well as a few other things), I managed to get it working and running properly.

And there you have it. I launched on ProductHunt (among other places) and was surprised by the overwhelming positive reception – I got almost 200 upvotes and almost 1000 unique visitors. What surprised me the most is how a favicon fetcher service I thought would be niche and developer only was actually really well understood by all kinds of people.

I learned a lot about the weird world of favicons and solved my own need, but above all had a lot of fun doing it.

Thanks for reading.