Forem: monty5811

rich progress and multiprocessing

monty5811 — Thu, 24 Mar 2022 00:00:00 +0000

How to use rich with python’s multiprocessing:

What is this?
- Track progress of long running tasks when using multiprocessing
Why would you want to do this?
- When you are doing lots of things with multiprocessing and each task can take a long time - feedback makes it easier to see that things are actually happening

How to do it:

import multiprocessing
import random
from concurrent.futures import ProcessPoolExecutor
from time import sleep

from rich import progress

def long_running_fn(progress, task_id):
    len_of_task = random.randint(3, 20) # take some random length of time
    for n in range(0, len_of_task):
        sleep(1) # sleep for a bit to simulate work
        progress[task_id] = {"progress": n + 1, "total": len_of_task}

if __name__ == " __main__":
    n_workers = 8 # set this to the number of cores you have on your machine

    with progress.Progress(
        "[progress.description]{task.description}",
        progress.BarColumn(),
        "[progress.percentage]{task.percentage:>3.0f}%",
        progress.TimeRemainingColumn(),
        progress.TimeElapsedColumn(),
        refresh_per_second=1, # bit slower updates
    ) as progress:
        futures = [] # keep track of the jobs
        with multiprocessing.Manager() as manager:
            # this is the key - we share some state between our 
            # main process and our worker functions
            _progress = manager.dict()
            overall_progress_task = progress.add_task("[green]All jobs progress:")

            with ProcessPoolExecutor(max_workers=n_workers) as executor:
                for n in range(0, 20): # iterate over the jobs we need to run
                    # set visible false so we don't have a lot of bars all at once:
                    task_id = progress.add_task(f"task {n}", visible=False)
                    futures.append(executor.submit(long_running_fn, _progress, task_id))

                # monitor the progress:
                while (n_finished := sum([future.done() for future in futures])) < len(
                    futures
                ):
                    progress.update(
                        overall_progress_task, completed=n_finished, total=len(futures)
                    )
                    for task_id, update_data in _progress.items():
                        latest = update_data["progress"]
                        total = update_data["total"]
                        # update the progress bar for this task:
                        progress.update(
                            task_id,
                            completed=latest,
                            total=total,
                            visible=latest < total,
                        )

                # raise any errors:
                for future in futures:
                    future.result()

Use iconic.app icons in your Django templates

monty5811 — Tue, 17 Aug 2021 08:54:14 +0000

iconic.app icons are a wonderful, free, set of icons. You can use them wherever you like. How you use them is by copy and pasting and SVG into your templates. That can be a bit cumbersome.

But there is a better way for your Django (and Jinja) templates: use iconic and you can include any icon by name:

{% load iconic %}

{% iconic_icon "announcement" size=48 class="h-4 w-4 inline" %}

Easy peasy.

Credits

Thanks to @jamesm and @ormanclark for the fantastic icons. You should definitely buy the "Pro" icons
The python package is a fork of the heroicons package by Adam Johnson. Check out his announcement post for more information

Caveats

Right now this package does not support the Pro icons in any way - you still need to copy and paste those SVGs.

Full Instructions

Install the package

pip install iconic[django]

Add it to your INSTALLED_APPS:

# settings.py
INSTALLED_APPS = [
    ...,
    'iconic',
    ...,
]

Then just load it and use it in your templates:

{% load iconic %}

{% iconic_icon "announcement" size=48 class="h-4 w-4 inline" data_controller="announcement" %}

Matlab: getting rid of nested loops

monty5811 — Mon, 01 Jun 2020 23:00:00 +0000

Ever seen matlab code that looks like this? Maybe you are doing a grid search?

for x = 1:10
  for y = 1:5
    for z = 1:100
      for t = 1:2
        do_something(x, y, z, t)
      end
    end
  end
end

This is pretty nasty, can we make this look better?

How about something like this?

params.X = 1:10;
params.Y = [2, 4, 8];

vectorized_parameters = build_vecd_params(params);
N = size(vectorised_parameters, 1);
for x = 1:N
    candidate = get_row_as_struct(params, vectorised_parameters, x);
    do_something(candidate);
end

This looks much nicer, we don’t have nested loops and adding a new parameter to loop over is much cleaner.

We can even randomise the looping order pretty easily:

N = size(vectorised_parameters, 1);
iteration_order = randperm(N);
for x = 1:N
    idx = iteration_order(x);
    candidate = get_row_as_struct(params, vectorised_parameters, idx);
    do_something(candidate);
end

How it works

We need to create the two functions:

build_vecd_params to create a matrix where each row is a single permutation of all variables we want to loop over
get_row_as_struct: to extract a row of the matrix and give us back a struct we can use to easily access each looping variable

function [vectorised_parameters] = build_vecd_params(params)
% params is a struct like
% params.X = 1:10;
% params.Y = [2, 4, 8];

all_ranges = struct2cell(params);
N = numel(all_ranges);
[grid{1:N}] = ndgrid(all_ranges{:});
vectorised_parameters = reshape(cat(N + 1, grid{:}), [], N);
end

function [candidate] = get_row_as_struct(params, vectorised_parameters, idx)
% return a `candidate` permutation of our loop variables
row = vectorised_parameters(idx, :);
fns = fieldnames(params)';
assert(numel(row) == numel(fns)); % sanity check
c = 1;
for fn_ = fns
    fn = char(fn_);
    candidate.(fn) = row(c);
    c = c + 1;
end
assert(c - 1 == numel(row)) % sanity check
end

How to use it

Just copy the two functions above into your matlab file and you’re off to the races.

Django: Query ImageField by height or width

monty5811 — Thu, 16 Jan 2020 23:20:14 +0000

Django has a built in ImageField - it is great & makes dealing with images very easy. However, out of the box, you can't query the database for images of a certain dimension (width or height). For example, you can't find all the large images you are storing.

The solution is hiding in plain sight in the Django docs.

An ImageField accepts two arguments: height_field and width_field. Using these are pretty easy:

from django.db import models


class Person(models.Model):
    avatar = models.ImageField(
        blank=True,
        null=True,
        height_field="avatar_height",
        width_field="avatar_width",
    )
    avatar_height = models.IntegerField(blank=True, null=True)
    avatar_width = models.IntegerField(blank=True, null=True)

Every time we save an image to the avatar field, Django will automatically update the avatar_heigh and avatar_width fields with the new image dimensions.

This means we now have an entry in the database that we can query! If we wanted to find all the images with at least one large dimension, we can do something like:

from django.db.models import Q


big_dim = 500;
big_images = Person.objects.filter(
    Q(avatar_height__gt=big_dim) | Q(avatar_width__gt=big_dim)
    )

Couple of final notes:

The fields are only updated on "save", so if you are adding this to an existing model, you may want to loop through each instance & call save() so that the fields are populated
Normal usage of person.avatar.height would load the whole image into memory in order to figure out how big it is. This can be slow if you are using a remote storage such as S3 or Digital Ocean Spaces. But using person.avatar_height skips all that. This can lead to a big performance increase if you need to know the size of your images but don't need to operate on the image itself.

Put some Tailwind in your Elm

monty5811 — Sat, 26 Oct 2019 09:18:07 +0000

tl;dr a postcss plugin to put your Tailwind classes into Elm

monty5811 / postcss-elm-tailwind

put some tailwind in your elm

postcss-elm-tailwind

tailwind + elm = 🚀

See the demo and repo.

view : Model -> Html Msg
view model =
    Html.div [ TW.h_screen, TW.w_screen, TW.flex, TW.justify_center, TW.items_center, TW.bg_gray_200 ]
        [ Html.div []
            [ Html.button
                [ E.onClick Decrement
                , TW.px_2
                , TW.px_4
                , TW.text_white
                , TW.bg_blue_500
                , TW.w_full
                ]
                [ Html.text "-" ]
            , Html.div
                [ TW.text_2xl
                , TW.text_center
                , TW.my_4
                ]
                [ Html.text (String.fromInt model) ]
            , Html.button
                [ E.onClick Increment
                , TW.px_2
                , TW.px_4
                , TW.text_white
                , TW.bg_blue_500
                , TW.w_full
                ]

…

View on GitHub

What

Go from this:

Html.div [ A.class "container mx-auto" ] []

To this:

Html.div [ TW.container, TW.mx_auto ] []

Why

More compiler guarantees - catch typos in your class names. Elm can't tell you that A.class "mx-uato" is a typo, but it will tell you TW.mx_uato doesn't exist.
Code completion - leverage code completion for Elm to know which tailwind classes are available

How

We create a new postcss plugin that grabs all the classes in your css
For each class we create a new Elm function that looks something like this:

{-|Tailwind class:
    xl:z-40"
-}
xl_z_40 : Html.Attribute msg
xl_z_40 =
    A.class "xl:z-40"

We then write out each of these new functions to create an Elm module that you can import like any other

The Elm module generated with the default Tailwind build is about 900kb, but the Elm compiler removes any functions that are unused: so you don't need to worry about your asset size. It can take a few seconds to compile the module - but this only needs to happen once for every time you change your Tailwind config.

Get Started

Install: yarn add postcss-elm-tailwind
Add require("postcss-elm-tailwind")({ elmFile: "src/TW.elm" }) to your postcss.config.js file
Start using the generated Elm module import TW!

Check out the

Building a serverless donate form

monty5811 — Wed, 18 Sep 2019 15:59:34 +0000

Learn how to put a donation form on a website - using Netlify & Stripe - fully SCA compliant and no servers to manage!

We will:

Host a static site on Netlify
Use Stripe Checkout to handle the donation
Wire it together with a serverless Netlify Function

tl;dr jump straight to the code here:

monty5811 / donate-form

example severless donate form with stripe & netlify

donate-form

This repo is an example of how to build a donation form with Stripe & Netlify.

DEMO
More info here

There are two main components:

A super simple form with some JS to handle the redirect flow
A netlify function to talk to the Stripe API

View on GitHub

Step 1: Setup

First of all we need a form where the user can choose how much to donate:

<!DOCTYPE html>
<html>

<head>
    <meta charset="utf-8">
    <title>Serverless Donate Form</title>
    <link href="https://unpkg.com/tailwindcss@^1.0/dist/tailwind.min.css" rel="stylesheet">
    <style>
        html {
            font-family: 'Lucida Grande', Verdana, sans-serif;
        }
    </style>
</head>

<body class="h-screen flex flex-col justify-center">
    <form class="antialiased max-w-xs mx-auto" id="payment-form">
        <input class="block w-64 py-2 px-3 mx-auto mb-4 border-gray-300 border rounded-md" type="number" min="0" placeholder="$50" id="giving-amount" />
        <button class="bg-green-500 hover:bg-green-600 text-white py-2 px-4 rounded-full mx-auto block" id="giving-button">
        Donate
      </button>
    </form>
</body>

</html>

Which looks something like this (we've used Tailwind for styling):

Step 2: Adding Stripe to the form

Now we need some javascript to handle the interaction with Stripe. We do a few different things to hook our form up to Stripe:

Listen for the user submitting the form
Update the form to a "waiting" state
Get a Stripe Checkout Session ID from our lambda function
Redirect to Stripe's hosted checkout with the Session ID
Handle any errors

Required changes:

    <!-- rest of content as above -->
    <!-- add jquery & stripe -->
    <script src="https://code.jquery.com/jquery-3.4.1.min.js" crossorigin="anonymous"></script>
    <script src="https://js.stripe.com/v3/"></script>
    <script type="text/javascript">
        var errorText = "Failed. You have not been charged.";

        // look out for submit events on the form
        document.addEventListener("DOMContentLoaded", function(event) {
            var submitButton = document.getElementById("giving-button");

            var stripe = Stripe("<YOUR_STRIPE_PUBLISHABLE_KEY_HERE>");

            var form = document.getElementById("payment-form");
            form.addEventListener("submit", function(event) {
                event.preventDefault();
                const buttonText = submitButton.innerText;
                submitButton.innerText = "Working...";

                var data = {
                    amount: document.getElementById("giving-amount").valueAsNumber * 100,
                };

                // create a stripe session by talking to our netlify function
                $.ajax({
                    type: "POST",
                    url: "/.netlify/functions/get_checkout_session/",
                    data: JSON.stringify(data),
                    success: function(data) {
                        // we got a response from our netlify function:
                        switch (data.status) {
                            case "session-created":
                                // it worked - send the user to checkout:
                                stripe
                                    .redirectToCheckout({
                                        sessionId: data.sessionId
                                    })
                                    .then(function(result) {
                                        submitButton.innerText = result.error.message;
                                    });
                                break;
                            default:
                                submitButton.innerText = errorText;
                        }
                    },
                    dataType: "json"
                });
            });
        });
    </script>
</body>

</html>

Step 3: Add our lambda function

Now we have a form that will take the donation amount & redirect to Stripe's hosted checkout. However, in order to use a custom "price" with Stripe Checkout we need a server-side component.¹

Setting up a whole server seems like overkill for this - a serverless function is ideal.

The serverless function simply takes the amount and gets a Session ID from Stripe. This Session ID is then sent back to the browser where the user is redirected to complete their donation.

const stripe = require("stripe")(process.env.STRIPE_SECRET_KEY); // get from ENV

const headers = {
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Headers": "Content-Type"
};

exports.handler = function(event, context, callback) {
  // some error checking:
  if (event.httpMethod !== "POST" || !event.body) {
    callback(null, {
      statusCode: 400,
      headers,
      body: JSON.stringify({ status: "bad-payload" })
    });
  }
  // Parse the body contents into an object.
  const data = JSON.parse(event.body);

  // Make sure we have all required data. Otherwise, escape.
  if (!data.amount) {
    console.error("Required information is missing.");

    callback(null, {
      statusCode: 400,
      headers,
      body: JSON.stringify({ status: "missing-information" })
    });

    return;
  }
  // actually create the session with Stripe
  // we need to provide a couple of redirect urls:
  stripe.checkout.sessions.create(
    {
      success_url: "https://donate-form-example.netlify.com/success",
      cancel_url: "https://donate-form-example.netlify.com/cancel",
      payment_method_types: ["card"],
      billing_address_collection: "required",
      payment_method_types: ["card"],
      submit_type: "donate",
      line_items: [
        {
          name: "Donation!",
          amount: data.amount,
          currency: "usd",
          quantity: 1
        }
      ]
    },
    function(err, session) {
      // asynchronously called
      if (err !== null) {
        console.log(err);
        callback(null, {
          statusCode: 200,
          headers,
          body: JSON.stringify({ status: "session-create-failed" })
        });
      }
      // woohoo! it worked, send the session id back to the browser:
      callback(null, {
        statusCode: 200,
        headers,
        body: JSON.stringify({
          status: "session-created",
          sessionId: session.id
        })
      });
    }
  );
};

You can see how this is hooked up to Netlify in the full repo:

monty5811 / donate-form

example severless donate form with stripe & netlify

donate-form

This repo is an example of how to build a donation form with Stripe & Netlify.

DEMO
More info here

There are two main components:

A super simple form with some JS to handle the redirect flow
A netlify function to talk to the Stripe API

View on GitHub

Conclusions

That's it! We have built a donation form where a user can choose how much they would like to donate and we have done it without ever having to worry about running our own server.

You can just as easily do this for a non-static site - you just need to replace the serverless function with a route on your site that will create a Stripe Session & return the id to the frontend.

If you have fixed prices, or fixed donation amounts, then you don't need any server side components. You can do everything client side. See the Stripe docs for more info ↩

Expiring (Signed) Links with Digital Ocean Spaces

monty5811 — Wed, 12 Jun 2019 13:47:22 +0000

Very quick guide on how to create expiring links to an object on Digital Ocean Spaces with Django. (<-- that's an affiliate link, btw, you'll get $50 to spend in 30 days if you signup)

tl;dr:

import boto3
from botocore.client import Config

session = boto3.session.Session()
client = session.client(
    "s3",
    region_name=AWS_S3_REGION_NAME,
    endpoint_url=AWS_S3_ENDPOINT_URL,
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    config=Config(signature_version="s3"),
)

url = client.generate_presigned_url(
    ClientMethod="get_object",
    Params={
        "Bucket": "bucket-name",
        "Key": "object-key",
    },
    ExpiresIn=60 * 60,
)

Why

If you store your assets (images, PDFs, audio files, etc) in an object store like Spaces, but you want to restrict access to these objects for some reason you can create a signed link that expires after some defined time.

For example, I host all the audio files from Yet Another Sermon Host on Spaces, but I need to keep track of how many downloads occur to provide analytics for my customers. I can do this by registering a download in a Django view that then redirects to the expiring Spaces URL.
This way, all the downloads have to pass through my server that generates the expiring URL and I can record every download.

How

Digital Ocean Spaces is API compatible with Amazon S3, so we can use boto3 & django-storages - I'll assume you already have django-storages setup (instructions for DO Spaces).

After installing django-storages your settings.py should contain something like this;

# settings.py

AWS_QUERYSTRING_AUTH = False
AWS_DEFAULT_ACL = "public-read"
AWS_ACCESS_KEY_ID = os.environ["DO_SPACES_ACCESS_KEY"]
AWS_SECRET_ACCESS_KEY = os.environ["DO_SPACES_SECRET"]
AWS_STORAGE_BUCKET_NAME = "bucket-name"
AWS_S3_REGION_NAME = "do-spaces-region"
AWS_S3_ENDPOINT_URL = f"https://{AWS_S3_REGION_NAME}.digitaloceanspaces.com"
AWS_S3_CUSTOM_DOMAIN = (
    f"{AWS_STORAGE_BUCKET_NAME}.{AWS_S3_REGION_NAME}.cdn.digitaloceanspaces.com"
)
AWS_S3_OBJECT_PARAMETERS = {"CacheControl": "max-age=86400"}AWS_MEDIA_LOCATION = "yash-media"
AWS_S3_FILE_OVERWRITE = False # safer behaviour than the default
AWS_S3_OBJECT_PARAMETERS = {"CacheControl": "max-age=86400"} # do some caching, why not

And we'll add a boto3 client:

# settings.py

# client for expiring urls:
aws_session = boto3.session.Session()
AWS_CLIENT = aws_session.client(
    "s3",
    region_name=AWS_S3_REGION_NAME,
    endpoint_url=AWS_S3_ENDPOINT_URL,
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    config=Config(signature_version="s3"),
)

And we should have a model with a FileField

# models.py

from django.db import models


class Asset(models.Model):
    name = models.CharField(max_length=1000)
    private_file = models.FileField()
    # other fields here

Generating the expiring link is then a matter of using the boto3 library, so
let's add it as a method to the model:

# models.py

from django.conf import settings
from django.db import models


class Asset(models.Model):
    name = models.CharField(max_length=1000)
    private_file = models.FileField()
    # other fields here

    @property
    def storage_key(self):
        return f"{self.private_file.storage.location}/{self.private_file.name}"

    def get_expiring_url(self, expires_in=1800):
        """
        Generates a permissioned url that will expire at `expires_in` seconds.
        Defaults to 30 minutes.
        """
        if self.private_file is not None:
            url = settings.AWS_CLIENT.generate_presigned_url(
                ClientMethod="get_object",
                Params={
                    "Bucket": settings.AWS_STORAGE_BUCKET_NAME,
                    "Key": self.storage_key,
                },
                ExpiresIn=expires_in,
            )
            return url

        raise Http404

And we can redirect users like this:

# views.py

from django.shortcuts import get_object_or_404
from django.views.generic import RedirectView


class Download(RedirectView):
    def get_redirect_url(self, *args, **kwargs):
        """
        Override this method so the view doesn't try to do any 
        string interpolation.
        """
        return self.url

    def get(self, request, *args, **kwargs):
        asset = get_object_or_404(models.Asset, pk=kwargs["pk"])
        # TODO: register a download or check user has permission here
        self.url = asset.get_expiring_url()
        return super().get(request, *args, **kwargs)

Note: you have to set the ACL on the object to private otherwise anyone can still access the files without the signed URL.

We can do this by making the entire bucket private:

# settings.py

AWS_DEFAULT_ACL = "private"

Or we can toggle individual objects with a couple of helper methods:

# models.py

class Asset(models.Model):
    # other methods from above

    def set_private(self):
        self._set_acl("private")

    def set_public_read(self):
        self._set_acl("public-read")

    def _set_acl(self, acl):
        settings.AWS_CLIENT.put_object_acl(
            Bucket=settings.AWS_STORAGE_BUCKET_NAME,
            Key=self.storage_key, ACL=acl
        )

Parsing Bible References with Elm

monty5811 — Fri, 04 Jan 2019 17:04:38 +0000

This was originally posted at deanmontgomery.com.

This post will give a brief intro into how to write a simple parser in Elm.
The thing we will be parsing are Bible references: a Bible reference is a shorthand that let's one quickly look up a specific verse or range of verses in the text of the Bible.

Sounds simple...

What's a reference look like?

A Bible reference can be broken down into a start location and an end location. Each location consists of a book, a chapter and a verse.

So a reference might look like Genesis 1:1 - Exodus 2:1 which tells us to start at the first verse of the first chapter of Genesis and end at the first verse of the second chapter of Exodus.

But the reference might also look like any of these:

Genesis 1 - A whole chapter
Genesis 1:1 - A single verse
Genesis 1:1-20
Genesis 1:20-2:24
Genesis 1-5 - Multiple whole chapters
Genesis 1 - Exodus 5
Genesis 1:1 - Exodus 5:20
Genesis 1:1 - Exodus 5
Genesis 1 - Exodus 5:20

Additionally, some books of the Bible only have a single chapter (e.g. Jude) and, by convention, the chapter number is dropped from the reference. So Jude 2 is the second verse of the first (and only) chapter of Jude, not all of Jude chapter 2.

We'll aim to handle all of these cases when we write our parser.

How do we do parsing in Elm?

elm/parser is a super nice parsing library written by the creator and maintainer of Elm. I won't go into details on it here - there is a nice tutorial and conference talk if you want to dig deeper.

We will parse the Bible reference in two steps:

We parse the string into a list of statements
Then we validate the list of statements to check it is a valid reference

Getting a list of statements

A Bible reference can have a space, a colon, a hypen, a book name and a number, so we define:

type Statement
    = BookName Book
    | Num Int
    | Dash
    | Colon

and a parser to turn a string into a list of statements:

{-| A `List Statement` parser. We use `P.loop` to consume the whole string
-}
parser : P.Parser (List Statement)
parser =
    P.loop [] statementsHelp


statementsHelp : List Statement -> P.Parser (P.Step (List Statement) (List Statement))
statementsHelp revStmts =
    P.oneOf
        [ P.succeed (\stmt -> P.Loop (stmt :: revStmts))
            |. P.spaces
            |= statement
            |. P.spaces
        , P.succeed ()
            |> P.map (\_ -> P.Done (List.reverse revStmts))
        ]

{-| A `Statement` parser
-}
statement : P.Parser Statement
statement =
    P.oneOf
        [ P.map BookName (P.oneOf bookTokensList)
        , P.map (\_ -> Dash) (P.symbol "-")
        , P.map (\_ -> Colon) (P.symbol ":")
        , P.map Num P.int
        ]

With this parser we can now turn a string into a List Statement:

parse : String -> Result String (List Statement)
parse str =
    P.run parser str

Validating the list of statements

Now we will either have a list of statements, like [Book Genesis, Colon, Num 1] or [Book John, Colon, Num 2, Dash, Num 2], etc. But there is nothing to guarantee that we have a valid collection of statements. For example, we could have [Colon, Colon, Colon] which is obviously not valid, or [Book Genesis, Num 52] which appears to be valid, but Genesis only has 50 books - so it is invalid.

First we will define a Reference type:

type alias Reference =
    { startBook : Book
    , startChapter : Int
    , startVerse : Int
    , endBook : Book
    , endChapter : Int
    , endVerse : Int
    }

And a function processStatements : List Statement -> Result String Reference that will validate our list of statements. This function is rather large to account for all the potential formats available and to handle single chapter books, but the function is essentially a case statement:

processStatementsHelp : List Statement -> Result String Reference
processStatementsHelp stmts =
    case stmts of
        -- Gen
        [ BookName bk ] ->
            reference
                bk
                1
                1
                bk
                (numChapters bk)
                (numVerses bk (numChapters bk))

        -- Gen 1
        [ BookName bk, Num ch ] ->
            if numChapters bk == 1 then
                reference
                    bk
                    1
                    ch
                    bk
                    1
                    ch

            else
                reference
                    bk
                    ch
                    1
                    bk
                    ch
                    (numVerses bk 1)

    -- truncated for brevity (full function can be seen: https://github.com/monty5811/elm-bible/blob/2.0.0/src/Internal/Parser.elm#L38-L243)

        -- Genesis - Revelation
        [ BookName startBk, Dash, BookName endBk ] ->
            reference
                startBk
                1
                1
                endBk
                (numChapters endBk)
                (numVerses endBk (numChapters endBk))

        [] ->
            Err "No reference found"

        _ ->
            Err <| "No valid reference found"

Now we have a Reference that contains a start book, start chapter, start verse, end chapter and end verse but we haven't checked that all of these are in order (e.g. the reference cannot end before it starts) so we use one last function to validate the reference:

validateRef : Reference -> Result String Reference
validateRef ref =
    validateBookOrder ref
        |> Result.andThen validateChapterOrder
        |> Result.andThen validateVerseOrder
        |> Result.andThen validateChapterBounds
        |> Result.andThen validateVerseBounds

-- see each validate function here: https://github.com/monty5811/elm-bible/blob/2.0.0/src/Internal/Parser.elm#L363

Finally! We have a validated Bible reference!

Note I think it should be possible to move all of this validation inside the parser and do everything in one step, but I think this is a cleaner approach.

Conclusion

This post has shown you how to create a parser in Elm so we can validate Bible references. Hopefully this will help you get started building a parser.

If you don't care about building a parser and just want an elm package to do this for you, then check out monty5811/elm-bible that provides a parser, nice formatting and a compact encoder/decoder.

monty5811 / elm-bible

elm-bible

Parse and format Bible references in Elm.

Features

Parse a reference from a string
Nicely format a reference to a string
Convert a reference to an encoded representation for sorting/comparing/storage

The following reference formats can be parsed:

Genesis 1
Genesis 1:1
Genesis 1:1-20
Genesis 1:20-2:24
Genesis 1-5
Genesis 1 - Exodus 5
Genesis 1:1 - Exodus 5:20
Genesis 1:1 - Exodus 5
Genesis 1 - Exodus 5:20

Examples

(fromString "Gen 1:1" |> Result.map format)
     == Ok "Genesis 1:1" 

(fromString "Gen 1:1 - Rev 5") |> Result.map format)
    == Ok "Genesis 1:1 - Revelation 5:14" 

(fromString "Gen 1:1 - Rev 5") |> Result.map encode) 
    == Ok {start = 1001001, end = 66005014}

Contributing

Contributions welcome, please open an issue to get started.

View on GitHub