Forem: Alexa Steinbrück

Write your first CRON job & monitor product discounts with Nodejs

Alexa Steinbrück — Mon, 04 Jan 2021 22:10:15 +0000

There are (paid) services out there that let you monitor the price of a certain product on the internet. But did you know that you can write this kind of monitoring tool yourself?

This use case is perfect for a cron job: A cron job schedules the execution of code at specific times (e.g. once a day, every 5 minutes, etc). Cron jobs are usually used by system administrators to make backups of their servers etc. But it can be useful for other (fun) things too!

🗂️ 1. The `crontab` command

The single Unix utility to know in order to get started with cronjobs is called crontab.

It's named cron*tab* because it refers to the cron*table*, which is a configuration file (also called table). To get more information on the crontab utility, type man crontab into the Terminal.

Two important commands to remember:

✨ crontab -l This displays the content of your current cron table, meaning all your scheduled cron jobs. (If you have never initialized a cron job, it will say "crontab: no crontab for {user}")

✨ crontab -e with which you can edit your cron table, meaning adding and deleting cron jobs! (If you haven't initiated a crontab yet, it will say: "crontab: no crontab for {user} - using an empty one")

Notes about text editors

The command crontab -e will try to open the crontab file with a text editor. If you haven't configured this before, the default will be vim, which is fine. By contrast, I had difficulties using vscode.

Tip: If your default editor isn't vim but you want to use vim only once for this occasion (without changing the default editor settings in your bashprofile), type export EDITOR=vi into your Terminal, this will set your default editor to vim until the end of this terminal session (e.g. until you close the Terminal).

Questions
Why does crontab -e create a temporary file in /tmp? This is normal behavior. Instead of editing the crontab directly, they will let you write into a temporary file first, which will then be copied to the real crontab.

🤡 2. Writing your first (silly) cron job

To get started, let's do something silly. Let's create a cron job that makes your computer say a word out loud every minute. (Very useful to annoy a friend or roommate).

First, open the crontab with the crontab command:
crontab -e

Then add the following line to the file and save the file.
* * * * * say Hello
This will immediately start your first cron job!

But what does this mean? The stars here represent the time instructions (5 stars separated by a space mean „every minute“), followed by the actual shell command (say Hello) that will be executed in the specified rhythm.

Why does * * * * * mean every minute? Because this is the magic instruction syntax of crontabs! And the star means "every" like in regexes.

If you want to learn more about the syntax and how to express situations like every day at midnight, or every 15 minutes in August watch this video.

Okay, this cronjob was just a silly test, so let's delete it! You can just remove the line by editing the file (with crontab -e). Or remove the whole crontab (with crontab -r).

⛓️ 3. Write a cron job that runs code from another file

Instead of typing the actual code we want to execute into the crontab itself, we can write our code into a file and execute the file from the crontab. The file could be written in any programming language you like, e.g. shell, Node, Python etc...

Example 1 (Shell)
* * * * . path/to/my-shell-script.sh
will execute a Shell script. The single dot before the file path means execute („source“) the following script.

Example 2 (Node)
* * * * path/to/your/node/binary path/to/my-node-script.js
will execute a Node script.

Note Example 2: Note that you can't just type node path/to/my-node-script.js, but you have to insert the full path to the node binary too. Why is that? The fact that you can usually run node just by typing node is because you configured your PATH. Cron jobs do not read this PATH like you would expect, that's why they don't know what node or npm is.

You can get the path to your node binary by typing which node into the console. If you haven't installed node at all on your machine, you should do that first (ideally via nvm). But note that this is by no means a requirement to write a cron job. You can choose any other language!

🕵️‍♀️ 4. Writing the (Node) script that finds the discount

Now let's get to the actual heart of it all: Analysing a website to get details about a certain product. This part very much depends on the website you're analysing. Here is how I approach this:

(1) First I check with a browser plugin called "Wappalyzer“ which technologies are used on the site. This gives me hints about how the website works: Is it a Single-Page-App that consumes a Rest API or is it server side rendered. Does it use any known APIs?

(2) Then I check the network tab in the Browser Dev Tools and filter for „XHR“ requests, and quickly scan through their JSON responses. Is there something that looks like a Rest API to me? Are there any key names in data that seem relevant to me, e.g. product name, price, etc?

(3) If this is the case, we're lucky! We can just parse these JSON outputs with a (Node) script which we will build now.

Note: In case there aren't any JSON responses with relevant data, the site is probably server-side rendered. This means we have to do a little more work by parsing the HTML and look for relevant CSS classes and ids in order to get the data. I'd prefer to do this task in Node, because it is plain Javascript and it has the DOM API natively baked in. But if you write your script in Python, you would have to use a library called Beautiful Soup.

Our little node script is really a piece of cake. We need 1 library, which is called node-fetch, which makes http requests easier.
So let's create a new Node project by
yarn init
And then install the library
yarn add node-fetch

Now let's write our little Node script. Note that the content depends on your usecase, website, etc. You have to write your own! ;-)

const fetch = require("node-fetch");
const URL ="url-for-your-product-from-the-rest-api";
const MAX_PRICE = 999;

fetch(URL)
  .then((response) => response.json())
  .then((data) => {
    const item = data.result[0];
    if (item.price < MAX_PRICE) {
      const message = `PRICE ALARM :-) ${item.price}${item.currency}`;
      sendNotification(message);
    }
  });

function sendNotification(message) {
  /* Because this is a cron job this plain console.log is enough to send an email to me... */
  console.log(message);
}

📬 5. Let the cron job report back to you

Note that in our little script the function sendNotification is really just doing a plain console.log.

But ask yourself, where will a console.log end up if it is run by a cron job? All cron jobs are run in non-interactive shells, there is no terminal screen attached to it!

Most of the time cron emails the output to the user whose crontab the job came from, but only if they have set up a „mailer daemon“ (like sendmail, qmail, postfix). If this isn't the case, the output just lands somewhere on your computer, and nobody really knows where. :-D

In my case I already had such a mailer deamon installed, and the emails ended up in a file under /var/mail. Check out if you have such a file. If this is the case, then your mailer daemon works, and you can easily set the recipient to a real email address, e.g. your Gmail address, like so:

At the top of the crontab file write:
MAILTO="youremailadress@gmail.com"

Then check your Gmail spam folder, after the cronjob was executed (if necessary, change the cron job timing back to each minute for testing). Then click on „report as not spam“ and it will go to your regular inbox.

Yaay, you should now receive an email when the product has a discount!!

😴 6. But wait - what happens when I close my laptop?

Question: Will it run the cronjob nevertheless?
Answer: No! Cron doesn't execute while the computer is asleep.

But there is a trick you can do: You can schedule your computer to "wake up" 5 minutes or so before your cron job is scheduled.
On Mac you can do this via Preferences >> Energy Saver >> Schedule.

Happy cron jobbing!

Credits: Title image by David Vega under CC BY 2.0

Github authentication facts I forget every two months

Alexa Steinbrück — Mon, 10 Aug 2020 16:36:47 +0000

Git recommends SSH over HTTPS!
- You make the decision for SSH/HTTPS, when you
  - clone a remote repository, or
  - assign a remote to a local repository
When you do HTTPS:
- You can use the credential.helper in OSX to cache your password
- PITFALL POTENTIAL: If you use 2FA, then your password is not your password, but your Personal Access Token, which you can create here https://github.com/settings/tokens
When you do SSH:
- You need to have a key available. Otherwise you get "Not authenticated (public key error)" or something along these lines...
- Test if you have a key by typing in the console: ssh -vT git@github.com
- If you dont have a key, then go create one: https://github.com/settings/keys

Why you shouldn't use the file creation time to represent a date

Alexa Steinbrück — Mon, 10 Aug 2020 12:53:13 +0000

Let's say you build a blog with Gatsby where each article is written in markdown and queried from your local filesystem with GraphQL with the help of the plugin gatsby-source-filesystem.

Now you also want to show the date of when the article was created.

So you go to Gatsby's interactive GraphQL interface (see below) and search for a field name that looks like it's a date. You find birthTime. Sounds great! The intuition here is that the article creation date is identical to the date when the file was first created on a hard drive/file system, which is what birthTime signifies.

So you query this field and it gives you some dates that look about right. Then you render them in your frontend and everything looks great on the development server. Even when do a real build on your local machine (gatsby build & gatsby serve).

The surprise comes when you involve CI/CD. Let's say you deploy with Netlify. Once your site is build by Netlify based on your Git repository, you will see different dates - namely: The time it is right now!

Why is that? Remember that we said earlier that birthTime is referring to the date when the file was first created on a hard drive/file system. Now, if you have a CI/CD service like Netlify building your site, it first fetches your repository from Github and creates all those files again on its own filesystem. Those are entirely new files! That's why their birthTime is different.

So how can you add a creation date to your blog entries?

Hardcode a date string into the frontmatter of your markdown files
Use a CMS (like Netlify CMS) that offers a UI for these things, or adds dates to the data layer automatically

Just like @pentacular said: "If it's a meaningful part of the file content, it better be part of the file content."

Have you used Netlify for real client projects?

Alexa Steinbrück — Thu, 30 Apr 2020 12:00:26 +0000

I read a lot of case studies of developers who use Netlify for their personal projects (blog, portfolio, experiments, ...).

I've almost never heard anybody telling how they used Netlify as a hosting/CD solution for real-world client projects.

I would be curious to hear about your experience using Netlify for your client projects.

Which plan did you choose (free or professional)?
How did you explain Netlify to your client? I have the impression that the decoupled nature of JAM stack websites is confusing for clients (as opposed to a monolithic Wordpress site)
How was the handover? Did you make a Netlify account for your client?

Please share!

Understanding gatsby-image (Part 3): Controlling sizes, breakpoints and styling

Alexa Steinbrück — Mon, 09 Dec 2019 12:57:04 +0000

This is Part 3 of a three-part series covering the Gatsby plugin gatsby-image
Part 1: Graphql, generated files & markup
Part 2: Responsive images 101
Part 3: Controlling src-set, breakpoints and styling

In Part 1 we've learned what gatsby-image is outputting, in Part 2 we learned the basics of responsive images in order to understand what gatsby-image is outputting. In this part we're going to learn how to tweak what gatsby-image is outputting in order to get the images that serve our use case.

Gatsby Image: What src-set property does it generate?

If you’re dealing with a fixed image it is producing a src-set with a width-descriptor appended to each source (e.g. 600w)

"srcSet": 
 "/static/1234/1111/my-image.jpg 600w, 
  /static/1234/2222/my-image.jpg 900w”

In the case of a fluid image it is producing a src-set width a x-descriptor appended to each source (e.g. 1.5x)

"srcSet": 
 "/static/1234/1111/my-image.jpg 1x, 
  /static/1234/2222/myImage.jpg 1.5x, 
  /static/1234/3333/myImage.jpg 2x”

Gatsby Image: What sizes property does it generate?

This only applies to fluid images. Fixed images don’t have a size property, because they use x-descriptors in their srcSet property.

Gatsby image just takes the sizes property from what gatsby-plugin-sharp is outputting. Remember, in Part 1 we showed how the fragment GatsbyImageSharpFluid includes a size property.

If you haven’t indicated a maxWidth in your graphql query it will generate:

sizes="(max-width: 800px) 100vw, 800px"

If you have indicated a maxWidth of 1200px, it will look like this:

sizes="(max-width: 1200px) 100vw, 1200px"

You can see here that the 100vw is based on the assumption that your image will take up the full available viewport width. It doesn’t take into account that your full-width image might be smaller because it’s living in a container with paddings and margins.

Setting custom sizes and breakpoints

But you can adjust that to your needs, by overriding the sizes property in the object you pass to the fluid property of your gatsby image. In case your image lives in a container that has a margin of 20px on each side, you can substract 40px from the viewport width:

<Img
  fluid={{
    ...img.image,
    sizes: "(max-width: 1200px) calc(100vw - 40px), 1200px",
  }}
/>

This way you don’t load images that are much bigger than what you really need. You might also want to override the sizes property to add additional breakpoints:

sizes: "(max-width: 300px) calc(100vw - 20px), (max-width: 600px) calc(100vw - 40px), 1200px",

Another technique for overriding sizes is to include your custom sizes as a query parameter in your graphql query, like in this example:

childImageSharp { 
  fluid(sizes: “(max-width: 1200px) calc(100vw - 40px), 1200px”){ 
   ...GatsbyImageSharpFluid
  } 
}

Controlling which image sizes are generated

If you want more control over which image sizes are generated by gatsby-plugin-sharp you can use the srcSetBreakpoints parameter in your fluid query. This could be an array like [200, 350, 900]. You probably want to use this in combination with a custom sizes property in the gatsby-image React component, as described above. This way you can refer to concrete numbers in your sizes property.

Checkout the documentation of gatsby-plugin-sharp here: https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-plugin-sharp#parameters-2

Ways to style/layout the image created by gatsby-image

We’ve covered the markup and layout that gatsby-image creates in the first Part 1 of this series.

Gatsby image accepts the following properties related to styling:

style={} /* styles for the wrapper container */
imgStyle={} /* Styles for the actual img element */

Here’s an interesting discussion of how to apply custom styling to a gatsby image.

👉 That was the 3-part series about gatsby-image. I hope it was helfpul!

Understanding gatsby-image (Part 2): Responsive images 101

Alexa Steinbrück — Mon, 09 Dec 2019 12:39:20 +0000

This is Part 2 of a three-part series covering the Gatsby plugin gatsby-image
Part 1: Graphql, generated files & generated markup
Part 2: Responsive images 101
Part 3: Controlling src-set, breakpoints and styling

In this post we’ll cover these things:

<picture> with <source> as child elements
<img> with srcset and sizes attribute
Using the browser devtools to find out which image files are really fetched by the browser

This is providing some helpful background knowledge to use gatsby-image, but it is not specific to gatsby-image.

How does the `<picture>` element work?

The <picture> element contains zero or more <source> elements and one <img> element.

From MDN:

“The browser will consider each child source element and choose the best match among them; if no matches are found or the browser doesn’t support the picture element, the URL of the img element’s src attribute is selected. The selected image is then presented in the space occupied by the img element.”

“The img element serves two purposes: it describes the size and other attributes of the image and its presentation, and it provides a fallback in case none of the offered source elements are able to provide a usable image.”

How does `<img>` with the `srcset` and `sizes` attribute work?

Checkout the following image markup. The srcset and sizes properties use a distinct notation:

<img 
 srcset=
  "my-img-280w.jpg 280w, 
   my-img-440w.jpg 440w, 
   my-img-800w.jpg 800w"
 sizes=
  "(max-width: 320px) 280px, 
   (max-width: 480px) 440px, 
   800px"
/>

"srcset defines the set of images we will allow the browser to choose between, and what size each image is." (MDN)

It reads as a string with comma-separated pairs of information: The file path followed by the width of the actual image as a number (without px) but with ‘w’ at the end.

What about the `sizes` attribute?

"sizes defines a set of media conditions (e.g. screen widths) and indicates what image size would be best to choose" (MDN)

It reads as a string with comma-separated pairs of information: a media condition in brackets, followed by the width of the slot the image will fill when the media condition is true.

“The last slot width has no media condition (this is the default that is chosen when none of the media conditions are true). The browser ignores everything after the first matching condition, so be careful how you order the media conditions.” (MDN)

Another option: `<img>` with `srcset` but without `sizes`

Check out this srcset attribute:

srcset=
 "my-img-300w.jpg 1x, 
  my-img-450w.jpg 1.5x, 
  my-img-600w.jpg 3x"

Instead of providing a width, we’re providing a so called x-descriptor that informs the browser to check the device pixel ratio (DPR) of the client’s monitor, and choose the right image.

This is useful for optimization for retina displays.

When should I use `<picture>` and when `<img>` & `sourceset`?

On first sight both ways seem to provide a solution for the same problem: Offering a choice of image files to let the browser choose from. The main guideline is:

1. Use <picture> for art direction and image-format-switching.
2. Use <img> & sourceset for resolution-switching.

If you're wondering: "Art direction…? But I'm not an art director!" - Read this definition of art direction within the domain of web development:

Art direction: The problem whereby you want to serve cropped images for different layouts — for example a landscape image showing a full scene for a desktop layout, and a portrait image showing the main subject zoomed in for a mobile layout.

(MDN)

What is meant by image-format-switching? Let's say we want to serve our images as the fast newer image format webp to our readers. But this format isn't supported yet by all browsers, so we have to provide a jpg image as a fallback. In this case we would use a <picture> element with two <source> elements: One for the webp and one for the jpg.

Testing: How to check which images are REALLY requested by the browser?

⚠️ Attention: It is not what you see inside the src of the <img > tag (but it can be)! Even more confusing: It doesn’t even have to be one from the srcSet array of the img element (but it can be)!

Two options to find out:

Open your browser’s developer tools Network Panel and checkout which file was fetched over the network
Open your browser’s developer tools “Inspector” or “Elements” panel: Select the img element and type into the Browser console: $0.currentSrc ($0 stands for the currently selected element)

👉 Continue with Part 3 of this series about gatsby-image: “Controlling sizes, breakpoints and styling”

Understanding gatsby-image (Part 1): graphql, generated files & markup

Alexa Steinbrück — Mon, 09 Dec 2019 12:18:45 +0000

This is Part 1 of a three-part series covering the Gatsby plugin gatsby-image:
Part 1: Graphql, generated files & markup
Part 2: Responsive images 101
Part 3: Controlling src-set, breakpoints and styling

Disclaimer: This is for the reader who has used gatsby-image before & understands what it is capable of, but needs a more in-depth understanding in order to control it better. (Written in 12/2019 & based on gatsby 2.18.x, gatsby-image 2.2.x, gatsby-transformer-sharp 2.3.x and gatsby-plugin-sharp 2.3.x)

On first sight, the Gatsby plugin gatsby-image is a jack of all trades plugin. But gatsby-image is essentially just a node module that exports a React component, and it works hand-in-hand with 2 other plugins: gatsby-plugin-sharp and gatsby-transformer-sharp, which do the heavy lifting on the server/build side.

Used together they can achieve the following:

Generating a bunch of image files into your /public/static folder based on an input image file and a graphql query that describes what kind of images you want (gatsby-plugin-sharp)
Produce markup and css to take care of the appearance of these images on the page (full width, cover/contain, etc.) (gatsby-image)
Getting a smooth appearance of a newly loaded image, e.g. by first showing a blurred base-64 image and replacing it with a full-resolution one, once it is loaded. This is known as the "blur up" technique popularized by Medium and Facebook (gatsby-image)
Prevent image jumping, when the image loads due to the exact image size not known before loading(gatsby-image)
Give you lazyloading by using IntersectionObserver in the browser (gatsby-image)

Note: You could probably use gatsby-image's image generating capacities exclusively, without using the gatsby-image component (and the quite opinionated markup), by writing your own markup. To do this you would have to query the appropriate fields that would be needed to satisfy a native <img> element, or your custom markup with <picture> and <source> elements.

Fluid vs. Fixed

Before you work with gatsby-image you have to make a binary decision about the type of image you are working with: Is it fixed or fluid?
From the Gatsby Docs:

"To decide between the two, ask yourself: "do I know the exact size this image will be?" If yes, it's the first type [Fixed]. If no and its width and/or height need to vary depending on the size of the screen, then it's the second type. [FLUID]"

The result of your fixed or fluid graphql query is then passed respectively to the fixed or fluid property of your gatsby-image component. The markup that gatsby-image outputs will differ if it's a fluid or a fixed image.

The graphql query

Fixed

A query for a fixed image could look like the following:

{
  imageSharp {
    fixed(width: 400) {
      ...GatsbyImageSharpFixed
    }
  }
}

Here, GatsbyImageSharpFixed is a fragment. It is equivalent to this query:

{
  imageSharp {
    fixed(width: 400) {
      base64
      width
      height
      src
      srcSet
    }
  }
}

Fluid

The following is a query for a fluid image:

{
  imageSharp {
    fluid(maxWidth: 1200) {
      ...GatsbyImageSharpFluid
    }
  }
}

It is equivalent to this query (without using a fragment):

{
  imageSharp {
    fluid(maxWidth: 1200) {
       base64
       aspectRatio
       src
       srcSet
       sizes
    }
  }
}

(Note the additional "sizes" parameter in the fluid query above)
Here's the source code of these fragments, so you know what they contain.

The generated image files

With your graphql query you decide which images are being generated: the dimensions and the formats (jpg/png, webp, base64). The images will be saved into your /public/static folder. Each image will get its own folder, and each image version will get its own folder within this folder.

Fixed

In the case of a fixed query, it will generate images that have the width/height you specified in your graphql query PLUS it will try to generate higher resolution images for retina displays: 150% and 200% of the size you specified (but only if the source image actually has that many pixels available).

Fluid

If you haven't indicated a maxWidth in your graphql query it will generate images with the widths of 800px, 400px and 200px (in case your original image isn't smaller than 800px width). If you do have indicated a maxWidth (example), then it will generate images with 100% , 50% and 25% maxWidth.

If the indicated maxWidth exceeds the width of the original image, it will still try to produce 50% and 25% width images (as long as these don't exceed the original image width), but the maximum size will be the original image. If the indicated maxWidth is smaller than the width of the original image, it will also generate images bigger than the indicated maxWidth, namely 150% and 200%, this is to satisfy retina displays.

If you want more control over which sizes are generated, check out Part 3 of this series.

👀 TIP: If you're using VS Code you can see the dimensions of images in your workspace at the lower bar on the right side.

The generated markup

Fixed

A relatively positioned wrapper <div class="gatsby-image-wrapper"> that has the proportions of the image it contains (see width and height). Inside this wrapper we see a <picture> element that contains several <source> elements and one <img> element. The <img> element is positioned absolutely and takes up the whole width and height of its parent.

Fluid

This looks similar to the fixed one, except that <div class="gatsby-image-wrapper"> doesn't have a defined width or height anymore. Instead, we have an additional child element, an empty <div> that has a procentual padding-bottom. This percentual padding happens to be the exact aspect ratio of the image it contains. Thanks to this, the <img> element can be positioned absolutely again and take up the whole width and height of its parent.

👉 Continue with Part 2 of this series about gatsby-image: "Responsive images 101"

Debugging a live site by replacing remote files with local ones

Alexa Steinbrück — Fri, 17 May 2019 16:49:13 +0000

At one point in your web developer life you might come across the following scenario:

Your site is live and you need to fix a problem in a Javascript file.
For some reason you can't set up and run the whole project locally. Or maybe what you want is to run the Javascript in a production environment against a live database.

What can you do?

I recently discovered a browser extension named Requestly (available for Firefox and Chrome). It allows you to intercept and redirect network requests of your browser and replace the requested remote assets (like a JS File) with assets served from your own machine's localhost or any other remote server.

Remember, that's only for debugging and testing on the browser side, it will not actually change the original files on the server!

Let's exercise this scenario with some random website. I will use the website of Hackernews https://news.ycombinator.com/. As a proof of concept I just want to add an alert in their script saying "Hello world".

Step 1: Downloading the original file

In Firefox what I do is opening the Devtools, go to the network panel, filter for JS resources only, right-click on the relevant JS file, choose "Open in new tab", and then download the file from there.

In the case of our example (Hacker News) the file we want to replace is named hn.js followed by a query string including a hash ?2c3QnShZ2ErBfQjZ9gFC

Step 2: Editing the file

Let's do some changes to the downloaded file with our favorite code editor. Let's just add one line of code right at the beginning of the file. Obviously in reality you would do your foxy debugging magic here!

Step 3: Serving the file

When you tell Requestly to set up a redirect, the destination has to be a URL. A file from within your file system will not work. I use the super handy Live Server plugin for VS Code to serve my local files straight from my IDE in a matter of a click. Here we go, my edited JS file is served by localhost under: http://localhost:5500/hn.js

Step 4: Configuring the redirect with Requestly

When you have the Requestly browser extension installed, click on it in the browser toolbar, then choose "Open app". Select "New Rule", then the "Redirect Request" option from the menu.
Enter the URL of the file you want to replace (Request Url) and the URL of your replacement file (Destination Url). If the original Url contains a (versioning) hash you might want to select "contains" instead of "equals".

Now refresh the Hacker News site in the browser. Do we see the "Hello world" alert? Nope. But why? When we open the browser console we see a hint.

Step 5: Understanding Content Security Policy (CSP)

Let's look at this console error more closely:
Content Security Policy: The page’s settings blocked the loading of a resource at http://localhost:5500/hn.js (“script-src”).

The message refers to a HTTP header named Content Security Policy (CSP) which is attached to the HTTP response of the original site. The exact content of the header can be examined again with the browser Devtools: Go to the Network tab again, filter by HTML, click on the file, this will show the details to the right. Select the tab "Headers".

Here's what's in it:
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://cdnjs.cloudflare.com/; frame-src 'self' https://www.google.com/recaptcha/; style-src 'self' 'unsafe-inline'

The interesting part of this CSP string is the part that is following the keyword/directive script-src. It translates to:

This site is allowed to execute scripts from resources that are hosted on the same domain ('self'), or embedded as inline scripts ('unsafe-inline'). Furthermore the site will accept scripts from a few selected external providers (Google, Gstatic, Cloudfare).

Obviously, our localhost is not invited by this policy! That's why the request could not be redirected to localhost. Luckily we can also change that by telling Requestly to modify the HTTP headers of the site.

Step 6: Modify the CSP HTTP header to include localhost

Create a new Requestly rule. This rule should be of the type "Modify Headers". Then, inside the form enter "Content-Security-Policy" (without quotes) in the "Header" input field. In the "Value" input field enter the original CSP header of the site and finally add localhost after the script-src directive. At the input field for Request Url enter the URL of the site you want to change (in our example the site of Hacker News).
Select "Response", because we want to change the Reponse HTTP Headers.

The updated CSP header for our example would be:
default-src 'self'; script-src 'self' 'unsafe-inline' http://localhost:5500 https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://cdnjs.cloudflare.com/; frame-src 'self' https://www.google.com/recaptcha/; style-src 'self' 'unsafe-inline'

(Note that the existing CSP HTTP header of your site might be different. Use the one of your site!)

⚠️ A word of caution ⚠️
Be aware that tinkering with HTTP security headers can be dangerous if you don't know what you're doing. Modifying the CSP header by adding localhost to your trusted hosts is of relatively low risk (because localhost === you). But disabling the CSP header altogether for a site could expose you to a security risk. By entering the wrong parameters (like a wildcard * for the URL pattern) you could easily disable important security mechanisms for all sites you visit. So be very, very careful!

Step 7: HTTP + HTTPS = Mixed Content

This is the last hurdle finally. If the website is served over HTTPS, but your replacement file is served over HTTP, you have a situation called "Mixed Content". Most browsers are blocking mixed content to keep you safe. In this case you will see an error like this in the console:

Blocked loading mixed active content http://localhost:5500/hn.js

To solve this you have 2 options:

Serving your replacement file over HTTPS
Telling your browser to (temporarily) unblock mixed content

I used the second option for this demo (note the lock icon with the red line in the address bar). And voila we see the alert!

Nota bene
Don't forget to deactivate your Requestly rules once you're done! Speaking from experience! You will forget that you set up this redirect and wonder why things are behaving so strangely. That's because there is no visual hint on the site to remind you that you have tampered with it (unless you're checking the network panel)!

Happy (live) debugging!

Boilerplate/Starterkit for framework-less Frontend projects

Alexa Steinbrück — Mon, 01 Apr 2019 11:06:37 +0000

Hello lovely community!
Do you know about any boilerplate tools ("starter kits", "bootstrap tools") for kickstarting simple Frontend projects that don't require a JS-framework? Basically like create-react-app, but for vanilla javascript. My main motivation is to cut down on set-up time and to become productive immediately.

It should also provide support for:

Transpiling (Babel)
SASS
Module bundling
Linting
Minification

I was searching on Github, but couldn't really find any repositories matching my need. Maybe the framework-less approach is slowly becoming extinct?! Anyway, I am thankful for your ideas!

Forem: Alexa Steinbrück

Write your first CRON job & monitor product discounts with Nodejs

🗂️ 1. The crontab command

🤡 2. Writing your first (silly) cron job

⛓️ 3. Write a cron job that runs code from another file

🕵️‍♀️ 4. Writing the (Node) script that finds the discount

📬 5. Let the cron job report back to you

😴 6. But wait - what happens when I close my laptop?

Github authentication facts I forget every two months

Why you shouldn't use the file creation time to represent a date

Have you used Netlify for real client projects?

Understanding gatsby-image (Part 3): Controlling sizes, breakpoints and styling

Gatsby Image: What src-set property does it generate?

Gatsby Image: What sizes property does it generate?

Setting custom sizes and breakpoints

Controlling which image sizes are generated

Ways to style/layout the image created by gatsby-image

Understanding gatsby-image (Part 2): Responsive images 101

How does the <picture> element work?

How does <img> with the srcset and sizes attribute work?

What about the sizes attribute?

Another option: <img> with srcset but without sizes

When should I use <picture> and when <img> & sourceset?

Testing: How to check which images are REALLY requested by the browser?

Understanding gatsby-image (Part 1): graphql, generated files & markup

Fluid vs. Fixed

The graphql query

Fixed

Fluid

The generated image files

Fixed

Fluid

The generated markup

Fixed

Fluid

Debugging a live site by replacing remote files with local ones

What can you do?

Step 1: Downloading the original file

Step 2: Editing the file

Step 3: Serving the file

Step 4: Configuring the redirect with Requestly

Step 5: Understanding Content Security Policy (CSP)

Step 6: Modify the CSP HTTP header to include localhost

Step 7: HTTP + HTTPS = Mixed Content

Boilerplate/Starterkit for framework-less Frontend projects

🗂️ 1. The `crontab` command

How does the `<picture>` element work?

How does `<img>` with the `srcset` and `sizes` attribute work?

What about the `sizes` attribute?

Another option: `<img>` with `srcset` but without `sizes`

When should I use `<picture>` and when `<img>` & `sourceset`?