Forem: Juhana Jauhiainen

How to search files from Azure Blob Storage using Python

Juhana Jauhiainen — Tue, 13 Dec 2022 20:48:44 +0000

Searching for files using index tags is a common use case but the Azure documentation can be quite hard to grasp.

To search for blobs with specific tags in Azure Blob Storage using Python, you can use the find_blobs_by_tags method from the Azure Storage SDK. It takes as a single parameter filter_expression which can be used to query blobs with specific tags, container or name.

Here's an example of how to use find_blobs_by_tags to search for blobs with specific tags

from azure.storage.blob import BlobServiceClient

# Create a BlobServiceClient object
blob_service_client = BlobServiceClient.from_connection_string("<your_connection_string>")

# Get a reference to the container where your blobs are stored
container_client = blob_service_client.get_container_client("<your_container_name>")

# Use the find_blobs_by_tags method to search for blobs with a specific set of tags
results = container_client.find_blobs_by_tags(
    "key1 = 'value1' and key2 = 'value2'"
)

# Print the names of the blobs that were found
for blob in results:
    print(blob["name"])

In this example, the find_blobs_by_tags method is used to search for blobs that have the tags key1 with a value of value1 and key2 with a value of value2.

find_blobs_by_tags returns a iterable response of FilteredBlob objects. The return value is automatically paginated and you can control the number of Blobs per page with the results_per_page keyword argument. You can iterate the results by blob or by page using the iterator returned by the by_page function.

What are template literal types in TypeScript?

Juhana Jauhiainen — Thu, 21 Apr 2022 20:12:19 +0000

Before diving into template literal types, we first need to discuss string literal types

In TypeScript, string literal types are types that constrain the type of a variable to a string or a union of strings.

type Greeting = "hello world";

const text1: Greeting = "hello world"; // ✅  This is ok
const text2: Greeting = "hello"; // ❌ Type '"hello"' is not assignable to type '"hello world"'.

Constraining the value of a variable to a single string is not that useful but a union of strings is something you'll see often.

type GreetingType = "hi" | "hello" | "howdy";
function sendGreeting(greetingType: GreetingType, name: string) {}

So what are template literal types then? Well, just like in JavaScript, we can use template literals to construct strings from variables, in TypeScript we can use a similar syntax to create new types. A template literal type is a type created by combining types and strings using the template literal syntax.

type World = "world";

type Greeting = `hello ${World}`;
// type Greeting = "hello world"

Template literal types can be built using all simple types (or their unions) which can be converted to strings. However, using an object or an array type results in an error.

type Numbers = 1 | 2 | 3 | 4 | [1];

type NumberGreeting = `hello ${Numbers}`;
// Type 'Numbers' is not assignable to type 'string | number | bigint | boolean | null | undefined'.
// Type '[1]' is not assignable to type 'string | number | bigint | boolean | null | undefined'.(2322)

When we use a union with template literal types, we get a type that contains all combinations of the union and the string. And if we have multiple unions in our template literal type, it will contain all the combinations of those 🤯

Let's say we have a union of strings which represent translatable string ids and a union of strings which represent language ids. Now we want to create an object where the keys are combinations of string ids and language ids.

type MessageType = "error" | "notification" | "request";
type Lang = "en" | "fr" | "de" | "jp" | "cn";

We can use template literal types to create a union that includes all combinations of these.

type MessageTypeTranslations = `${MessageType}_${Lang}`;

/*
type MessageTypeTranslations = "error_en" | "error_fr" | "error_de" | "error_jp" | "error_cn" | "notification_en" | "notification_fr" | "notification_de" | "notification_jp" | "notification_cn" | "request_en" | "request_fr" | "request_de" | "request_jp" | "request_cn"
*/

We could now use this as a key type for an object which contains all the translations

type MessageTranslations = {
  [key in MessageTypeTranslations]: string;
};

Type inference

Another cool feature of template literal types is we can also use them with type inference.

type Mouse = {
  position: [number, number];
  leftButtonState: boolean;
  rightButtonState: boolean;
};

type MouseEventName = `on${Capitalize<keyof Mouse>}Change`;
// Here R is a inferred type
type MouseEventKey<T> = T extends `on${infer R}Change`
  ? Uncapitalize<R>
  : never;
// MouseEventKey<"onPositionChange"> is equal to "position"

Here we are using template literal types to generate event names from the keys of the object type Mouse. Then we use type inference (in MouseEventKey<T>) to retrieve the original object key when given an event name.

We can now use these types to define a function type for an event listener.

type MouseEventFunc<T extends MouseEventName> = (
  value: Mouse[MouseEventKey<T>]
) => void;

// MouseEventFunc<"onPositionChanged"> is equal to (value: [number, number]) => void

And now we can define a function for adding event handlers and get type errors if the handler has an incorrect type.

function addListener<T extends MouseEventName>(
  event: T,
  callback: MouseEventFunc<T>
) {
  //...
}

// ✅  value is of type [number, number]
addListener("onPositionChange", (value) => {
  console.log(value[0], value[1]);
});

// ❌ Property '0' does not exist on type 'Boolean' (value is of type Boolean)
addListener("onLeftButtonStateChange", (value) => {
  console.log(value[0], value[1]);
});

What can you do with template literal types?

Template literal string can be used to achieve some incredible stuff like JSON parsingor a working SQL database engine 🤯 For more awesome template literal type stuff, check the awesome-template-literal-type repo in GitHub

In my last article, I wrote on how to implement useMediaQuery hook in React. The useMediaQuery hook takes a single parameter, a string with a media query, and returns whether the query is true. Since the parameter is just a string, it's easy to make a mistake with the query because we can't type check it. But now, with the knowledge of template literal types, we can!

This is what the typing for useMediaQuery looks like currently.

function useMediaQuery(query: string): boolean {
  //...
}

Type checking for all the possible media features is outside of the scope of this article, but we can start by type checking for the most common one. By far, the most common use of media queries is checking if the width of the device is under a certain threshold with max-width or min-width.

First let's type the width queries

type CSSMaxWidth = `max-width: ${number}px`;
type CSSMinWidth = `min-width: ${number}px`;
type MediaWidthQuery = `(${CSSMaxWidth | CSSMinWidth})`;
// (max-width: 1024px) ✅
// (min-width: 768px) ✅
// (min-widht: 1024px) ❌ Type '"(max-widht: 1024px)"' is not assignable to type '`(max-width: ${number}px)` | `(min-width: ${number}px)`'.(2322)

Next, we combine the width queries with the possibility to define the media type and we have a simple type for a media query.

type MediaType = "all" | "print" | "screen";
type MediaOperator = "and" | "not" | "only";

type MediaQuery =
  | `${MediaType} ${MediaOperator} ${MediaWidthQuery}`
  | MediaWidthQuery;
// (max-width: 768px) ✅
// screen and (max-width: 1024px) ✅

MediaQuery could already be used as the type for the useMediaQuery hook if we only want to allow a single query. If we want to check multiple media queries in a single string, we need to introduce some recursion 😎

type UseMediaQueryParam<Str, Orig> = Str extends MediaQuery
  ? Orig
  : Str extends `${MediaQuery},${infer Rest}`
  ? UseMediaQueryParam<Rest, Orig>
  : never;

function useMediaQuery<Param extends string>(
  query: UseMediaQueryParam<Param, Param>
): boolean {
  //...
}

Here, we first check if the parameter extends MediaQuery, which means it's just a single media query, and return the original parameter if so. If it's not a single media query, we check if it's a combination of a media query and a string separated with a comma. Then we pass it to UseMediaQueryParam recursively. If the string doesn't have a MediaQuery at the start, we return never because the string isn't a valid list of media queries.

One last thing: string manipulation

Template literal types are a powerful feature of TypeScript and they become even more powerful with some of the built-in string manipulation types that come with TypeScript.

The types included are Uppercase<StringType>, Lowercase<StringType>, Capitalize<StringType>, and Uncapitalize<StringType>.

type MessageType = "error" | "notification" | "request";

type MessageId = `${Uppercase<MessageType>}_ID`;
// type MessageId = "ERROR_ID" | "NOTIFICATION_ID" | "REQUEST_ID"

type MessageName = `${Capitalize<MessageType>}`;
// type MessageName = "Error" | "Notification" | "Request"

type MessageIdLower = `${Lowercase<MessageId>}`;
// type MessageIdLower = "error_id" | "notification_id" | "request_id"

type MessageIdCapitalized = `${Capitalize<MessageIdLower>}`;
// type MessageIdCapitalized = "Error_id" | "Notification_id" | "Request_id"

String manipulation types make it easy to convert existing string literal types to new ones.

How to implement useMediaQuery hook in React

Juhana Jauhiainen — Sun, 13 Mar 2022 20:13:31 +0000

What are Media Queries?

Media Queries are a CSS feature that can be used to conditionally apply selected styles on an HTML element. Some examples of media queries include checking for the width of the browser window, checking for the media type (print, screen), or checking for dark/light mode preference.

The most common use case for media queries is using it to implement responsivity on a website. Checking for the width of the viewport and applying styles based on it allows us to define different styles on different devices (desktop, mobile, tablet).

The syntax for media queries is comprised of an optional media type and any number of media feature expressions. Media types include all,screen, and print. The default value for media type is all.

.header {
  font-size: 20rem;
}

@media print {
  .header {
    font-size: 15rem;
  }
}

The media type is followed by any number of media feature expressions enclosed in parenthesis.

.header {
  font-size: 20rem;
  color: pink;
}

@media (max-width: 800px) {
  .header {
    color: blue;
  }
}

Here, max-width: 800px is a media feature expression which signifies that this CSS will only be applied if the width of the viewport is equal to or less than 800 pixels.

There is a number of different media features we can use to apply CSS in specific situations. The most common is width because it is used when creating responsive and mobile-friendly websites.

How to use Media Queries from JavaScript?

If you want to check for a media query using JavaScript, you can use the window.matchMedia function. matchMedia takes a single parameter, the query string you want to check for, and returns a MediaQueryList object. The MediaQuery object can be used to check for a match or to attach a change event listener. The change listener is called every time the result of the media query changes.

const result = window.matchMedia("(max-width: 800px)");
if (result.matches) {
  // do something
}

result.addEventListener("change", (event) => {
  if (event.matches) {
    // do something
  }
});

Custom React hook for Media Queries

matcMedia makes it possible to implement a React Hook we can use to check for Media Query matches and change your application UI or behavior based on the results.

First, let's define the API for our hook. With TypeScript, our hooks type definition would look like this.

type useMediaQuery = (query: string) => boolean;

Our hook will take the query string as a parameter and return a boolean. Next, we'll need to add a React.useEffect call which calls matchMedia and adds an event listener for change. We also need a state variable to store the match.

function useMediaQuery(query) {
  const [matches, setMatches] = React.useState(false);
  React.useEffect(() => {
    const matchQueryList = window.matchMedia(query);
    function handleChange(e) {
      setMatches(e.matches);
    }
    matchQueryList.addEventListener("change", handleChange);
  }, [query]);

  return matches;
}

Great 🥳 This already works but we have one more important thing to add.. cleanup for the event handler. React.useEffect function can return a function used for cleanup. It's commonly used to unregister event handlers or unsubscribing from external data sources.

Let's add a cleanup function to our useEffect

function useMediaQuery(query) {
  const [matches, setMatches] = React.useState(false);

  React.useEffect(() => {
    const matchQueryList = window.matchMedia(query);
    function handleChange(e) {
      setMatches(e.matches);
    }
    matchQueryList.addEventListener("change", handleChange);

    return () => {
      matchQueryList.removeEventListener("change", handleChange);
    };
  }, [query]);

  return matches;
}

Now our useMediaQuery hook is finished. Here's how you would use it.

function SomeComponent() {
  const isMobile = useMediaQuery("min-width: 768px)");

  return <h1>Browsing with {isMobile ? "phone" : "desktop"}</h1>;
}

Links

MDN on window.matchMedia
MDN on MediaQueryList

Basics of React server-side rendering with Express.js

Juhana Jauhiainen — Sat, 05 Feb 2022 22:04:00 +0000

If you want to develop SEO friendly and fast websites with React, you have two choices: server-side rendering (SSR) or static site generation (SSG).

There are some awesome frameworks like remix.run , next.js, astro or 11ty, which allow you to use one of (or both) techniques. So if you're building a production app, I recommend using one of them because server-side rendering is quite hard to get right.

But if you want to understand how it works and what is happening under the hood in these frameworks, you definately should try it out. This article will focus on how SSR works and we will also go through a simple example of using SSR.

How React server-side rendering works?

Server-side rendering means rendering the initial HTML on the server instead of waiting for the JavaScript to be loaded in the browser and then rendering.

In client-side rendering, the browser makes a request for the index.html page, the server responds. the browser then reads this .html file and makes requests for any additional resources defined in it (CSS, JavaScript, favicon, images etc.). Only once the JavaScript is downloaded and can be executed, will there be anything rendered on the screen.

As we can see, the server doesn't really do anything here. That's why you can host a client-side rendered React app by just serving the static files using a web server like nginx.

With server-side rendering, you need a server side application which handles the initial rendering of your React application. The server application will import your React applications root component and render it into a HTML document which is then returned to the client.

Do I need to use server-side rendering in my React app?

If you're starting a new project and are serious about performance and SEO you should definately look into SSR. I'd recommend using one of the React frameworks tailored for SSR if they fit your needs.

For existing client-side rendered apps you should really weight the pros and cons. While SSR might provide some benefits (SEO, loading speed, social media previews), it will cost you some development time and will increase your server costs.

How to implement server-side rendering

We're going to go through a simple, but limited, implementation of server-side rendering just to get you an idea on how it works.

You can use any Node.js or Deno framework for the server code but in this example, we're using Node.js, express and esbuild. The full source code of this example can be found here

First let's look at the client side code.

Our main code in the client side is in App.jsx.



import * as React from "react";

export default function App() {
  const [times, setTimes] = React.useState(0);
  return (
    <div>
      <h1>Hello {times}</h1>
      <button onClick={() => setTimes((times) => times + 1)}>ADD</button>
    </div>
  );
}

App.jsx contains a small React component with a counter which is increased when the user clicks a button. The only other file in the client side we need is an entrypoint.



import * as React from "react";
import ReactDOM from "react-dom";
import App from "./App";

ReactDOM.hydrate(<App />, document.getElementById("root"));

index.jsx is the entrypoint for our client side code. Notice we're using ReactDOM.hydrate instead of ReactDOM.render. Instead of rendering the app (because it has already been rendered by the server) we are hydrating our app.

Hydrating refers to attaching all the event handlers to the correct elements of our server-side rendered DOM so our application will function correctly.

Next, let's take a look at the server code.



import path from "path";
import fs from "fs";

import React from "react";
import ReactDOMServer from "react-dom/server";
import express from "express";

import App from "../src/App";

const PORT = process.env.PORT || 3000;
const app = express();

app.get("/", (req, res) => {
  fs.readFile(path.resolve("./public/index.html"), "utf8", (err, data) => {
    if (err) {
      console.error(err);
      return res.status(500).send("An error occurred");
    }

    return res.send(
      data.replace(
        '<div id="root"></div>',
        `<div id="root">${ReactDOMServer.renderToString(<App />)}</div>`
      )
    );
  });
});

app.use(
  express.static(path.resolve(__dirname, ".", "dist"), { maxAge: "30d" })
);

app.listen(PORT, () => {
  console.log(`Server is listening on port ${PORT}`);
});

On the server side, we use express to define a root endpoint which serves a index.html file. When a request is received, we render our React app root component App to a string using ReactDOMServer.renderToString. The rendered string is then injected into our index.html file so that we replace the div with the id root with our rendered content.

We also setup static file loading from dist folder so that our client side JavaScript code in bundle.js will be loaded once the browser reads index.html.

index.html contains basic HTML structure, a <script> tag for bundle.js and a <div> element which the React app will be rendered to.



<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>SSR App</title>
  </head>

  <body>
    <div id="root"></div>
    <script src="bundle.js"></script>
  </body>
</html>

Now, when a request is made to the root of our app, the express server renders our React app into a string and injects that into the HTML which is returned to the browser. The browser then loads our JavaScript file (bundle.js) which contains the ReactDOM.hydrate call. After hydrate is called, our application is fully interactive and works just like it before we moved to server-side rendering.

This setup is enough for a simple example, but falls down pretty quickly with a more complext app. For example, it has no support for routing, which means we would render the same HTML no matter which URL the user is loading. It's also missing setup for loading static files imported in React components.

Summary

Server-side rendering is a useful technique you can use when you want to improve the load times and SEO of your React application. It is however, hard to implement well and might not be needed if your client-side rendered application is performing well and you don't have issues with SEO.

I strongly recommend you to try remix or next.js if you want to build a server-side rendered app.

Links

React docs on ReactDOMServer
React docs on hydrate
remix.run
next.js
Is SSR with React worth it? (Jakob Lind)

Photo by Sean Sinclair on Unsplash

Use a Model Manager to filter queryset by default in Django

Juhana Jauhiainen — Thu, 20 Jan 2022 19:17:35 +0000

How can you filter a queryset by default so that no matter where you use the model, the filter will be applied?

This can be achieved by creating a custom Manager and overriding its get_queryset method.

For example, if we have a soft delete feature in our app implemented by adding a deleted column, we can force only returing non-deleted objects by creating a custom model manager which filters for deleted=false

class BookManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(deleted=False)

class Book(models.Model):
    # ...
    deleted = models.BooleanField(default=False)
    objects = BookManager()

Now everytime you query Book objects in your app with Book.objects , you will only receive books that haven't been deleted.

Links

Model managers

Should you use React.FC to type your components?

Juhana Jauhiainen — Tue, 07 Dec 2021 21:44:57 +0000

TLDR: No. You should probably not.

const Dialog: React.FC = ({children}) => {
    return (
        ...
    )
}

The consensus in the React community now seems to be that, you should avoid using React.FC because it causes the component to implicitly take children. This means the component will accept children, even if they're not supposed to.

Let's take a look at the pros and cons of React.FC.

Pros of React.FC

Excplicit return type

In addition to the children prop, React.FC declares an explicit return type for the component. The return type is ReactElement | null; and we will get a type error if we try to return something else from the component.

const Header: React.FC = () => {
    return "test";
  )
}

/*
  Type '() => string' is not assignable to type 'FC<{}>'.
  Type 'string' is not assignable to type 'ReactElement<any, any> | null'.ts(2322)
*/

Component static properties

Here's the full definition of React.FunctionComponent which React.FC is a shorthand for.

interface FunctionComponent<P = {}> {
  (props: PropsWithChildren<P>, context?: any): ReactElement<any, any> | null;
  propTypes?: WeakValidationMap<P> | undefined;
  contextTypes?: ValidationMap<any> | undefined;
  defaultProps?: Partial<P> | undefined;
  displayName?: string | undefined;
}

In addition to the function signature, React.FC provides type checking for some static properties. propTypes and defaultProps define the prop types based on the generic type P. These provide type validation for props you pass to the component and default prop values you can define with Component.defaultProps = {}.

contextTypes and displayName are less useful in most situations.
contextTypes is related to a legacy Context.

displayName is used in debug messages.

Cons

Implicit children

The biggest downside to using React.FC is that it causes the component to implicitly take children. This means all components defined using React.FC will happily accept children, even when they're not supposed to.

const Icon: React.FC = <span className="icon">...</span>;

const OhNoh = <Icon>This should be a error</Icon>;

This won't be a runtime error, but TypeScript would catch this error if React.FC had not been used.

Component as namespace

React.FC also doesn't accept generics (GeneriComponent<T>) and makes using the "component as namespace" pattern awkward to type.

<Dialog>
  <Dialog.Content></Dialog.Content>
</Dialog>;

const Dialog: React.FC<DialogProps> & { Content: React.FC<ContentProps> } = (
  props
) => {
  /*..*/
};
Dialog.Content = (props) => {
  /*...*/
};

//vs

const Dialog = (props: DialogProps) => {
  /*...*/
};
Dialog.Content = (props: ContentProps) => {
  /*..*/
};

Alternatives

Explicit props and return type

If you want to have the benefits of explicit return types, you can use JSX.Element to explicitly type the return value of your components. children should be typed with React.ReactNode since it accepts all valid children values

interface Props {
    children: React.ReactNode;
}

const Header = ({children}: Props): JSX.Element => {
    return "test";
  )
}

/*
  Type 'string' is not assignable to type 'Element'.ts(2322)
*/

Explicit props but implicit return type

This is my preferred way of typing React components. I find the implicit return type is usually not a problem and haven't had any bugs where a component is returning something it's not supposed to.

interface Props {
    children: React.ReactNode;
}

function Header({children}: Props) {
    ...
}

In my opinion, this is a pretty clear and clean way of typing React components. What this way loses in typesafe, it gains in readability. I also prefer using named functions because I think they are more readable than the array functions.

Photo by Ash Edmonds on Unsplash

Why you should use the URL to store state in your application

Juhana Jauhiainen — Sun, 28 Nov 2021 16:45:57 +0000

It seems we have, in the web community, collectively forgotten the role of the URL in storing the state of our application. For good or bad, we've started developing websites like they are apps and forgotten one of the key elements of the web. I've seen (and built) applications where opening modals, searching, filtering, or navigating a wizard doesn't change the URL. And this is wrong because it doesn't fit the web paradigm.

Websites work in the context of the browser, and can never control the user flow completely as desktop or mobile applications can. At any point, the user can hit refresh, navigate back/forward or close the tab. These are all things we, as web developers, need to take into account. We should take ownership of these interactions and design our applications so these actions result in an outcome the user expects!

How can we do better? We need to think about the actions our users take, and the expectations they have when using our application. What should happen when you hit refresh? Or navigate back? What state do we need to retain? What inputs need to be initialized from the URL?

Search and filtering

Let's start with something clear-cut in my opinion. Search forms should be saved to the URL because the expectation is that you can copy-paste and share the URL and if you refresh the page, you will get the same search results.

How do you do it then? Well, this is what the url search params are for! Store each field of the search query in the URL and initialize the inputs from the URL when the page loads.

The same concept can be applied to filtering data in a view. For example, if we have a view with rows of invoices the user might want to filter out the invoices with which have been paid before a specified date (for example /invoices?paidBefore=01-01-2021). Now our users share this URL and the other user will see the same invoices.

Detail views

Another clear-cut case is detail view layouts. In a detail view layout, you have the main layout with items and a child layout with a detail view of the selected item. In this case, we can use the URL to store the id of the selected item.

For example, if we have a list of invoices at the URL /invoices and want to display the details when an invoice is clicked, we can have a link to /invoices/1 and render the detail view when the URL has the id at the end.

Modals

With modals, we can use the URL to store the open state. This is useful when the modal is used to add/edit entries or to show a detail view. With other modals, like alert/notification modals, it doesn't make sense because they are used to inform the user about some action they tried to perform.

When a user clicks a button or a link that opens the modal, the URL should change from for eg. /invoices/ to /invoices/add and this should trigger the opening of the modal. On edit modals, we can use the id of the data were editing in the URL. So for example /invoices/1/edit.

This allows our users the share a link that opens the edit dialog of a specific invoice. Notice that while in the search interface we stored the input values in the URL, in other forms/modals we don't want to do this because the expectation is that if you cancel the modal or refresh the page, the data is lost. It is, however, a good idea to inform the user before a page reload about the possible loss of data.

Wizards and stepped user interface

As with modals, there are many types of wizards or stepped interfaces we might have in our applications. The problem we face with wizards is that unless you use Local Storage or Session Storage to save the data, you will lose it on page refresh even if the position in the wizard is retained. The wizard position should therefore only be saved if we can also save all the data the user has entered in previous steps.

On some wizards, it might even make sense to store the state in the database. This would enable the user to come back to the wizard later, and continue the process where they left last time.

Summary

The main benefits of this approach are a better user experience and productivity. When our users can share deeper links they are more productive and less frustrated with our application. At the same time, the browser controls (refresh, navigate back/forward) result in an application state which the user can expect.

Photo by Javier Allegue Barros on Unsplash

What are HTTP cookies

Juhana Jauhiainen — Thu, 28 Oct 2021 18:03:45 +0000

A HTTP cookie is a small piece of data that a server sends to a user's web browser. The browser may then store the cookie and send it back to the same server with later requests.

Cookies are used typically to tell if two HTTP requests come from the same browser/user. So for session management, tracking and to a lesser extent personalization

For general data storage on the client, there are more modern APIs.

How cookies are sent between server and client

When client(browser) and the server communicate with HTTP calls (GET, POST etc) the calls include content, or a body and headers

The headers contain a lot of information on the request or response currently being transported. Things like content type, cache control and user agent. Headers are present both when client mades a request to the server, and when the server responds.

Cookies are also set and transported using HTTP headers. Setting cookies on the server is done by adding a Set-Cookie header to a response. The browser will then attach a Cookie header, with the value sent by the server in the Set-Cookie header, to every request it will make to a url with the same domain.

Set-Cookie has a number of optional attributes which can be used to configure how the the browser treats the cookie. These include for example Max-Age, which can be used to set the lifetime of the cookie.

Using cookies in Node.js and Express

Setting cookies in Express.js is done by calling the cookie method on a Response object.

app.get('/', (req, res) => {
    const value = "something";
    res.cookie("somecookie", value, { maxAge: 1000 * 60 * 15 })
    res.send('Hello World!')
});

Reading cookies can be done with the cookies property of the Request object When using the cookie-parser middleware, cookies becomes a object that contains the cookies senty by the browser.

var app = express()

app.use(cookieParser())
app.get('/', (req, res) => {
    const value = req.cookies.somecookie;
    res.send('Hello World!');
});

Cookie security

Cookies are automatically sent by the browser when a request is made to the domain which set the cookie originally. This is really nice, but can result in security issues. You don't want the browser to allow accessing the cookie from JavaScript or sending the cookies over an insecure connection.

To make your cookies secure, you should use the optional attributes passed to Set-Cookie.

HttpOnly
Forbids JavaScript access to cookie using Document.cookie

Secure
Sends the cookie only over HTTPS

SameSite=Strict | Lax | None
Strict: Only send the cookie for same-site requests
Lax: Don't send the cookie on cross-site requests but send it when navigating to our server using a link
None: Always sends the cookie

Expires=date
Sets the maximum lifetime of the cookie as a HTTP-date timestamp

It's important to set HttpOnly for any cookies containing authentication tokens or sensitive information to prevent JavaScript access to them in the client.

SameSite can be used to prevent some Cross-Site Request Forgery attacks. Setting it to Strict might not be ideal, because then a user navigating from a link to your site, will have to login even if they have a valid session. Setting it to Lax will only send the cookies when a user is navigating to your site SameSite defaults to Lax if not set.

If Expires is not set, the cookie will be deleted when the browser session ends.

With Express.js you can set these using the options object passed to cookie method of the Response object.

app.get('/', (req, res) => {
    const value = "something";
    res.cookie("somecookie", value, { 
        httpOnly: true,
        secure: true,
        sameSite: "strict",
        expires: date
    })
    res.send('Hello World!')
});

If you're using cookies to create a user session, consider using a middleware like express-session

References

MDN on Cookies

Photo by Christina Branco on Unsplash

How to make TypeScript understand Array.filter

Juhana Jauhiainen — Mon, 04 Oct 2021 19:36:50 +0000

If you've ever used Array.filter to filter a list to certain type of items, you've probably been hit by TypeScript not realizing what type of items your list contains after filtering. To fix this, you can use user-defined type guards.

Let's say we have content items of two type: Post and Image.

interface Post {
    text: string;
    title: string;
    date: Date;
}

interface Image {
    url: string;
    alt: string;
    date: Date;
}

We are going to store items of both types in a array.


type Content = Post | Image;
const content: Content[] = [
    {
        title: "A post",
        text: "...",
        date: new Date()
    },
    {
        alt: "A image",
        url: "...",
        date: new Date()
    }
]

Now if we want to get a list of all the post titles, we can do it by first filtering filtering with "title" in obj. But even though we know this works and there's no way title is undefined, we still get a type error.

const postTitles = content.filter(c => "title" in c).map(c => c.title);
// Property 'title' does not exist on type 'Content'.
// Property 'title' does not exist on type 'Image'.(2339)

This is because TypeScript can't deduce that all the remaining items are of the type Post. We can fix this issue with a TypeScript feature called user-defined type guards

What are user defined type guards?

Type guards allow narrowing a type with differenty ways. For example, you can use typeof or instanceof.

User-defined type guards are functions whose return type is a type predicate.

function isPost(content: Content): content is Post {
    return "title" in content;
}

Notice the return value of the function is content is Post this tells TypeScript the function will only return true if content is of type Post. The first part of a type predicate (here content) must be a name of parameter of the function.

Now we can use this type guard in our Array.filter and TypeScript won't yell at us anymore 🥳

const postTitles = content.filter(isPost).map(c => c.title);

If you want to play with the code in this article, checkout this playground

Why Docker is eating your disk space

Juhana Jauhiainen — Thu, 23 Sep 2021 20:08:11 +0000

I learned recently, in less than ideal circumstances, that Docker doesn't do any log rotation by default 😱

So if you're running your application in production with Docker and using Docker's default logging driver you might be awakened some night at 3 AM to fix a service outage when you can barely think straight. 😅

The default logging driver is json-file which caches the container logs as JSON. In practice, this means Docker is writing all the container stdout and stderr to a JSON file located in the host systems /var/lib/docker/containers folder.

You can find out the locations and sizes of these log files with docker inspect. You might need to run this with sudo.

du -h $(docker inspect --format='{{.LogPath}}' $(docker ps -qa))

If you need to delete logs of a container before continuing, you can use echo to overwrite them as empty. This might require stopping the containers first.

echo "" > $(docker inspect --format='{{.LogPath}}' container_name)

Changing the default logging driver

You can change the default logging driver by adding a configuration for it in /etc/docker/daemon.json. Docker supports multiple logging drivers but the simplest solution to our disk gorging logs problem is to use the local driver. The official Docker documentation explicitly refers to using it to prevent disk-exhaustion.

Let's create a basic config for local logging driver.

{
  "log-driver": "local",
  "log-opts": {
    "max-size": "100m",
    "max-file": "3"
  }
}

We are defining the logging driver with log-driver and setting the maximum size of a log file before it is rolled. This means, when the file reaches 100 megabytes, a new file is created and the old one is archived. max-file is here set to "3", so at any point in time there will be only three log files stored. When the third file reaches 100 megabytes, a new file is created and the oldest log file is deleted.

Next, we need to reload the config for the Docker daemon. The Docker documentation suggests we can use SIGHUP signal to reload the configuration. Unfortunately, this doesn't work on all configuration options with logging driver being one of them.

This means we will have to restart the Docker daemon to reload the config. The command for restarting the daemon depends on the system it's running on. On Linux systems using [systemd] docker daemon can be restarted using the service command.

service docker restart

Now if we check the logging driver currently in use with info, it should print local

docker info --format '{{.LoggingDriver}}'

Now we have changed the default logging driver but if we have running containers this is not enough. The configuration change will only affect new containers. This means we will have to recreate all our existing containers. Depending on your setup you can do this using docker-compose or by running docker build

docker-compose up -d --force-recreate

If you're using docker-compose and get an warning WARNING: no logs are available with the 'local' log driver when you're trying to read logs from a container, it means you're running an older version of docker-compose which doesn't support local logging driver. Try upgrading your docker-compose

How to type an object with exclusive-or properties in TypeScript

Juhana Jauhiainen — Sun, 12 Sep 2021 20:34:47 +0000

How can we define a type for an object, which has some required attributes and some attributes, of which one and only one must be defined?

Let's say we want to define a type for a message that has a timestamp and either a text or an id.

// ✅ This should be valid
const messageWithText = {
    timestamp: "2021-08-22T19:58:53+00:00",
    text: "Hello!"
}

// ✅ This should be valid
const messageWithId = {
    timestamp: "2021-08-22T19:58:53+00:00",
    id: 123
}

// ❌ This should be a type error 
const messageWithBoth = {
    timestamp: "2021-08-22T19:58:53+00:00",
    id: 123,
    text: "Hello!"
}

// ❌ This should be a type error 
const messageWithoutEither = {
    timestamp: "2021-08-22T19:58:53+00:00",
}

Union type

One way to approach this would be to use a union type

type Message = {timestamp: string}
type TextMessage =  {text: string};
type IdMessage = {id: number};
type TextOrIdMessage = Message & (TextMessage | IdMessage);

Now, timestamp is a required attribute, and one of id and text attributes must also be present in an object of type TextOrIdMessage.

const msg1: TextOrIdMessage = {timestamp: "2021-08-22T19:58:53+00:00", text: "Hello!"}
const msg2: TextOrIdMessage = {timestamp: "2021-08-22T19:58:53+00:00", id: 1234}

// ❌ Type error!
const msg3: TextOrIdMessage = {timestamp: "2021-08-22T19:58:53+00:00"}

This does not however prevent us from creating an object with both id and text 😔

// This is still valid..
const msg4: TextOrIdMessage = {timestamp: "2021-08-22T19:58:53+00:00", id: 1234, text: "Hello!"}

Using never to forbid a key from a type

What we can do to prevent this, is to use the never type

type Message = {timestamp: string}
type TextMessage =  {text: string, id?: never};
type IdMessage = {id: number, text?: never};
type TextOrIdMessage = Message & (TextMessage | IdMessage);

We've added id with the type never to TextMessage and text as type never to IdMessage. This means id and text can never be in the same object just like we wanted!

// ❌ Type error!
const msg4: TextOrIdMessage = {timestamp: "2021-08-22T19:58:53+00:00", id: 1234, text: "Hello!"}

This is a great solution and most of the time it would probably be enough. But is there a way to make a more generic, reusable solution?

A reusable exclusive-or utility type

As it turns out, there is. This is a solution I came across in a Stack Overflow question Why does A | B allow a combination of both, and how can I prevent it?. The solution was provided by jcalz.

The solution is a bit more involved than the previous one, so we will go through it step by step. Here's the full code to create a utility type called ExclusifyUnion.

type AllKeys<T> = T extends unknown ? keyof T : never;
type Id<T> = T extends infer U ? { [K in keyof U]: U[K] } : never;
type _ExclusifyUnion<T, K extends PropertyKey> =
    T extends unknown ? Id<T & Partial<Record<Exclude<K, keyof T>, never>>> : never;
type ExclusifyUnion<T> = _ExclusifyUnion<T, AllKeys<T>>;

ExclusifyUnion<TextOrIdMessage> will give us a union type in which id and text are defined as optional undefined types.

type ExclusifiedMessage = ExclusifyUnion<TextOrIdMessage>;
/*{
    timestamp: string;
    text: string;
    id?: undefined
} |
{
    timestamp: string;
    id: number;
    text?: undefined
}*/

That's a lot to unpack 😅 Let's look at AllKeys and Id first and then go through step by step, how ExclisifyUnion works.

On the first line type AllKeys<T> = T extends unknown ? keyof T : never; we are creating a utility for getting all the keys from a type. Just using keyof T isn't enough because in our example with the type TextOrIdMessage this would result in only the key timestamp. This is because keyof a union type only returns the keys which are shared by all the types in the union.

But with the magic of distributive conditional types we can extract all the keys from a union. This works because when T extends unknown ? keyof T : never is given a union, it applies keyof T to each part of the union. AllKeys<TextOrIdMessage> gives us a union of all the keys "timestamp" | "id" | "text".

type AllKeys<T> = T extends unknown ? keyof T : never;
type AllTheKeys = AllKeys<TextOrIdMessage>
// "timestamp" | "id" | "text"

Now, let's look at the Id utility type.

type Id<T> = T extends infer U ? { [K in keyof U]: U[K] } : never;

At first look, Id looks pretty baffling 🤔 It seems to just give you back exactly the type you give to it..

Id<number> // number
Id<number[]> // number[]
Id<{a: string}> // {a: string}

But when you give Id a more complex type like TextOrIdMessage, the result is a "compiled" version type. While TextOrIdMessage and Id<TextOrIdMessage> are functionally the same, the latter is easier to work with because your editor can show you a more readable version of the type.

The purpose of Id is to eliminate intersections in order to make the type easier to read. You can read more about it at the end of this Stack Overflow answer.

Id<TextOrIdMessage>
/*
{
    timestamp: string;
    text: string;
} | {
    timestamp: string;
    id: number;
}
*/

How does Id work then? Well, it uses a conditional type with infer to extract all the keys and value types from the given type in order to eliminate intersections.

type SomeIntersectionType = {id: number, timestamp: string} & {timestamp: string, text: string}
type ReadableIntersectionType = Id<SomeIntersectionType> // {id: number, timestamp: string, text: string}

Now that we've looked at what AllKeys and Id do, let's go over what ExclusifyUnion does to TextOrIdMessage.

type _ExclusifyUnion<T, K extends PropertyKey> =
    T extends unknown ? Id<T & Partial<Record<Exclude<K, keyof T>, never>>> : never;
type ExclusifyUnion<T> = _ExclusifyUnion<T, AllKeys<T>>;

The key to understanding ExclusifyUnion is noticing that it uses a distributive conditional type to apply Id<T & Partial<Record<Exclude<K, keyof T>, never>>> to each part of a union given as the generic type T. Inside the conditional, T refers to the part of the union it is being applied to.

Remember, K here is AllKeys<TextOrIdmessage> which resolves to "timestamp" | "text" | "id".

Our type TextOrIdMessage, is a union type with two parts, {timestamp: string, text: string} and {timestamp: string, id: number}.

So when we apply the inside of Id<...> to each part of TextOrIdMessage, we will get

{timestamp: string, text:string} & Partial<Record<Exclude<"timestamp" | "id" | "text", "timestamp" | "text">, never>>
{timestamp: string, id:number} & Partial<Record<Exclude<"timestamp" | "id" | "text", "timestamp" | "id">, never>>

After resolving Exclude we will have

{timestamp: string, text:string} & Partial<Record<"id", never>>
{timestamp: string, id:number} & Partial<Record<"text", never>>

Resolving Record gives us

{timestamp: string, text:string} & Partial<{id: never}>
{timestamp: string, id:number} & Partial<{text: never}>

And finally resolving Partial and the intersection operators

{timestamp: string, text:string, id?: undefined}
{timestamp: string, id:number, text?: undefined}

The final result will be the union of these types just like we wanted 🥳

type ExclusifiedMessage = ExclusifyUnion<TextOrIdMessage>;
/*{
    timestamp: string;
    text: string;
    id?: undefined
} |
{
    timestamp: string;
    id: number;
    text?: undefined
}*/

Advanced Django queries

Juhana Jauhiainen — Tue, 27 Jul 2021 21:12:10 +0000

Django Framework comes with a powerful ORM and query capabilities built-in. If you're only familiar with the basics of Djangos Query API, this article will introduce some more advanced queries and methods you can use.

In the examples, I'll be using the following data models.

class Author(models.Model):
    nickname = models.CharField(max_length=20, null=True, blank=True)
    firstname = models.CharField(max_length=20)
    lastname = models.CharField(max_length=40)
    birth_date = models.DateField()


class Book(models.Model):
    author = models.ForeignKey(Author, related_name="books", on_delete=models.CASCADE)
    title = models.CharField(unique=True, max_length=100)
    category = models.CharField(max_length=50)
    published = models.DateField()
    price = models.DecimalField(decimal_places=2, max_digits=6)
    rating = models.IntegerField()

Filtering

Basic filtering in Django can be done using something called field lookups with the filter method. Field lookups consist of the field and a suffix defining the lookup type. If no suffix is defined, the default behavior is equivalent to using the exact suffix.

books = Book.objects.filter(title="Crime and Punishment")
books = Book.objects.filter(title__startswith="Crime")

Querying by using field lookups is already very powerful. We can filter with different suffixes like gte, lte, contains, and a myriad of other lookups depending on the field type.

# Books published in 2021
books = Book.objects.filter(published__year=2021)
# Books published in the year 2000 or after
books = Book.objects.filter(published__year__gte=2000)
# Books published before the year 2000
books = Book.objects.filter(published__year__lt=2000)

With time fields we can even filter using a range lookup to query objects within a specific time range.

start_date = datetime.date(2021, 1, 1)
end_date = datetime.date(2021, 1, 3)
books = Book.objects.filter(published__range=(start_date, end_date))

Using the built-in expressions and Q objects we can do some more advanced queries.

Q Objects

Keyword arguments in a filter query are "AND"ed together. If we want to execute OR queries we can use the Q object. Q objects encapsulate keyword arguments for filtering just like filter, but we can combine Q objects using & or |.

from django.db.models import Q
# Get all books published in 2018 or 2020
books = Book.objects.filter(Q(published__year=2018) | Q(published__year=2020))
# Get all books published in

This will query all books published in 2018 or 2020.

You can combine any number of Q objects into more complex queries.

F expressions

F expressions represent a value of a model field. It makes it possible to use field values in queries without actually pulling the value from the database. This is possible because Django creates a SQL query that handles everything for us.

from django.db.models import F
# Query books published by authors under 30 years old (this is not exactly true because years vary in length)
books = Book.objects.filter(published__lte=F("author__birth_date") + datetime.timedelta(days=365*30))

We can also use F expressions when updating data to avoid performing multiple queries.

from django.db.models import F
book = Book.objects.get(title="Crime and Punishment")
book.update(rating=F("rating") + 1)

This will result in an SQL query that will add one to the rating of the book without first querying the current value.

Annotation

Most often we just want to query the values defined in a model with some filtering criteria. But sometimes you'll be calculating or combining values that you need from the result of the query. This can, of course, be done in Python but for performance reasons, it might be worth it to let the database handle the calculations. This is where Djangos annotations come in handy.

from django.db.models import F, Value as V
from django.db.models.functions import Concat
author = Author.objects.annotate(full_name=Concat(F("firstname"), V(" "), F("lastname")))

Here, we are adding a new dynamic field full_name to our query results. It will contain a concatenation of the authors firstname and lastname done using the Concat function, F expressions, and Value.

Concat is just one example of a database function available in Django. Database functions are a great way to add dynamic fields into our queries. Django supports a number of database functions like comparison and conversion functions (Coalesce, Greatest, ...), math functions (Power, Sqrt, Ceil, ...), text functions (Concat, Trim, Upper, ...), and window functions (FirstValue, NthValue, ...)

Here's an example on how we can use Coalesce to get either the nickname or the firstname of a author.

from django.db.models import F
from django.db.models.functions import Coalesce
books = Author.objects.annotate(known_as=Coalesce(F("nickname"), F("firstname")))

Now our results have a new field known_as, that holds the value we wanted for each Author in the result set.

Using F expressions we can also do basic arithmetic to calculate the value for a dynamic field.

# Add  a new field with the authors age at the time of publishing the book
books = Book.objects.annotate(author_age=F("published") - F("author__birth_date"))
# Add a new field with the rating multiplied by 100
books = Book.objects.annotate(rating_multiplied=F("rating") * 100)

Aggregation

While annotate can be used to add new values to the returned data, aggregation can be used to derive values by summarizing or aggregating a result set. The difference between aggregation and annotation is that annotating adds a new field to every row of a result set and aggregating reduces the results into a single row with the aggregated values.

Common uses for aggregation are calculating counting, averaging, or finding maximum or minimum.

from django.db.models import Avg
result = Book.objects.aggregate(Avg("price"))
# {'price__avg': Decimal('13.50')}
result = Book.objects.aggerate(Max("price"))
# {'price__max: Decimal('13.50')}
result = Book.objects.aggerate(Min("published"))
# {'published__min': datetime.date(1866, 7, 25)}

Aggregation can also be done without the aggregate method by using the database functions we talked about earlier. This way we can add a dynamic field into our results with annotate and calculate the value for it with aggregation. 😎

from django.db.models import Count
authors = Author.objects.annotate(num_books=Count("books"))

This will add a dynamic field num_books to every row of the result set with the number of books the author has.

Using values, annotate and a aggregation function allows us to do a GROUP BY query with a dynamic field.

# Calculate average prices for books in all categories.
Book.objects.values("category").annotate(Avg("price"))
# {'category': 'Historical fiction', 'price__avg': Decimal('13.9900000000000')}, {'category': 'Romance', 'price__avg': Decimal('16.4950000000000')}

Case...When

The last example I want to show is a bit more complex but showcases some of Djangos query features together. It includes using conditional expressions when calculating the value of a dynamic field.

from django.db.models import F, Q, Value, When, Case
from decimal import Decimal
books = Book.objects.annotate(discounted_price=Case(
    When(category="Romance", then=F("price") * Decimal(0.95)),
    When(category="Historical fiction", then=F("price") * Decimal(0.8)),
    default=None
))

Here we're calculating a discounted prices on all books based on the category using Case and When

Forem: Juhana Jauhiainen

How to search files from Azure Blob Storage using Python

Further reading

What are template literal types in TypeScript?

Type inference

What can you do with template literal types?

One last thing: string manipulation

Further reading

How to implement useMediaQuery hook in React

What are Media Queries?

How to use Media Queries from JavaScript?

Custom React hook for Media Queries

Links

Basics of React server-side rendering with Express.js

How React server-side rendering works?

Do I need to use server-side rendering in my React app?

How to implement server-side rendering

Summary

Links

Use a Model Manager to filter queryset by default in Django

Links

Should you use React.FC to type your components?

Pros of React.FC

Excplicit return type

Component static properties

Cons

Implicit children

Component as namespace

Alternatives

Explicit props and return type

Explicit props but implicit return type

Why you should use the URL to store state in your application

Search and filtering

Detail views

Modals

Wizards and stepped user interface

Summary

What are HTTP cookies

How cookies are sent between server and client

Using cookies in Node.js and Express

Cookie security

References

How to make TypeScript understand Array.filter

What are user defined type guards?

Further reading

Why Docker is eating your disk space

Changing the default logging driver

Further reading

How to type an object with exclusive-or properties in TypeScript

Union type

Using never to forbid a key from a type

A reusable exclusive-or utility type

Further reading

Advanced Django queries

Filtering

Q Objects

F expressions

Annotation

Aggregation

Case...When

Further reading