Forem: Miles Watson

URL Shortener with Rust, Svelte, & AWS (6/): Deploying to AWS

Miles Watson — Fri, 08 Oct 2021 15:31:31 +0000

In the last post, we created a static front-end application with Svelte + Bulma, and then integrated it into our Rust code + Dockerfile. In this final post, we will cover the process of actually deploying the app to AWS Elastic Beanstalk.

What is Elastic Beanstalk?

Elastic Beanstalk is a service provided by AWS which enables developers to focus purely on application code, without worrying about the infrastructure the code is being run on. Although we won't be exploring the full power of EB, it is capable of capacity provisioning, load balancing, scaling, and application health monitoring.

Manual Deployment

The first step in manually deploying your application is to create a zipped archive of your code to upload to EB. As an effort to stay cross-platform, we will do this using git archive (this has the added benefit of automatically ignoring anything in the gitignore).

git archive --format=zip HEAD -o ./target/repo.zip

This will place a zip of our repo into the target folder.

Now we have the zip file, you should following the following steps:

Login to AWS Console
Use the search bar to go to the "Elastic Beanstalk" page, and make sure you are in the "Elastic Beanstalk > Environments page"
Select your desired region using the dropdown menu in the top right
Click the "Create application" button (alternatively, click "Create Environment, and then select "Web server environment")
Give your application a suitable name
Give the application environment (the collection of services that will run your application) the name "production"
Select the following platform options:
Upload your zip file, and then click "Create environment"

If you go back the environments page, you will see that an environment with the name "production" has been created. It should have the status "Pending" - you will now have to wait around ten minutes for your application to build and startup (EB uses the docker-compose.yml file to build the images from scratch on the underlying EC2 instance).

Once the environment health transitions to "Ready", you should be able to go to the provided URL (HTTP only, as setting up HTTPS is beyond the scope of this series).

Creating a Deployment Pipeline

Manually updating your application like this can be time-consuming, so I will now show you how to automatically deploy your code with GitHub Actions.

First, you will need to use the "IAM" service to create a user (so that the workflow can access your AWS account). The user should be called "url-shortener-deploy" and the credential type should be "Programmatic access".

We will be using the Beanstalk Deploy, which requires the AWSElasticBeanstalkWebTier and AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy permissions. Make sure to attach these to the IAM role, and then create the user.

Once you have clicked Create user, you will be shown a table with the access credentials. You will only be shown these once - don't close the window just yet. In a new tab, open your repo on GitHub and add the following secrets (Settings > Secrets > New repository secret):

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY You should set these to the corresponding credentials of your url-shortener-deploy user, and then close the credential view on the AWS console.

Next, create a new workflow called deploy.yml (this should be in the same directory as test.yml). This action will use the secrets we set in the last step to deploy our code to AWS whenever code is pushed to production.

name: Deploy to AWS
on:
  push:
    branches:
      - production
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Latest Repo
        uses: actions/checkout@master
        with:
          ref: "production"

      - name: Generate Deployment Package
        run: mkdir target && git archive --format=zip HEAD -o ./target/repo.zip

      - name: Get timestamp
        uses: gerred/actions/current-time@master
        id: current-time

      - name: Run string replace
        uses: frabert/replace-string-action@master
        id: format-time
        with:
          pattern: '[:\.]+'
          string: "${{ steps.current-time.outputs.time }}"
          replace-with: "-"
          flags: "g"

      - name: Deploy to EB
        uses: einaregilsson/beanstalk-deploy@v18
        with:
          aws_access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws_secret_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          application_name: url-shortener
          environment_name: production
          version_label: "url-shortener-${{ steps.format-time.outputs.replaced }}"
          region: eu-west-2
          deployment_package: target/repo.zip

Note: The string replacement step simply sets the code version to the current time.

Testing Auto-Deployment

Now that we have created the workflow, we need to push to the production branch. Normally we can do this via a PR on the GitHub web interface, but that only works after we have created and pushed the branch. To do this, use the following git commands:

git branch production
git push origin production

If all goes correctly, your workflow should take about 15 minutes to deploy your code to AWS.

Now it's your turn!

Now that you have the barebones structure up and running, perhaps you could try adding the following features:

persistance with an AWS-provided database (like Aurora or DynamoDB)
expiring links, so that a given memory / storage limit isn't exceeded
a fully fleshed-out frontend with common pages (such as an FAQ, contact us)
a custom domain name + https support for EB

That's all for this series! If you have any issues, make sure to check out the part-6 tag of my repo.

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

Disclaimer - I'm a (mostly) self-taught programmer, and I use my blog to share things that I've learnt on my journey to becoming a better developer. Because of this, I apologize in advance for any inaccuracies I might have made - criticism and corrections are welcome!

URL Shortener with Rust, Svelte, & AWS (5/): Frontend

Miles Watson — Wed, 29 Sep 2021 17:04:06 +0000

In the last post, we worked on containerizing the backend of our application, so that we can deploy it with a single command. In this post, we will create a static front-end application with Svelte + Bulma, and then integrate it into our Rust code + Dockerfile. I will be using the Yarn package manager, but feel free to use npm if you prefer.

Cloning the Template

First, run the following command in the root of your repo. This will create a directory called svelte which contains a template that we'll use to build a simple frontend for our API.

npm init svelte@next svelte

Select the Skeleton project template, Yes to Typescript, and then No to all the other questions.

Inside the svelte directory, run yarn install to install dependencies, and then yarn dev to start a dev server. By going to http://localhost:3000/ you should see a simple message:

Adding a Static Adapter

Before we can deploy the site, we need to adapt it to our deployment target. SvelteKit provides a number of different adapters for platforms like Cloudflare Workers, Netlify, and Vercel. In our case, however, we will be using adapter-static to prerender our entire site.

First, we need to install the adapter using Yarn.

yarn add --dev @sveltejs/adapter-static@next

Next, we need to change the svelte.config.js file to configure the adapter. Import the adapter...

import adapter from '@sveltejs/adapter-static';

...then update config.kit.

{
    // hydrate the <div id="svelte"> element in src/app.html
    target: '#svelte',
    // prerender the pages so they can be served statically
    adapter: adapter()
}

By running yarn build, you should see that the /svelte/build directory is populated with an index.html file (among some other files / folders).

Serving with Rocket

Now that we have statically built our application, we can serve it with Rocket. To do so, simply change our launch function to the following:

#[launch]
fn rocket() -> _ {
    rocket::build()
        .manage(DashMap::<u32, String>::new())
        .mount("/", routes![shorten, redirect])
        .mount(
            "/",
            if cfg!(debug_assertions) {
                // debug mode, therefore serve relative to crate root
                FileServer::from(rocket::fs::relative!("/svelte/build"))
            } else {
                // dockerized, therefore serve from absolute path
                FileServer::from("/app/static")
            },
        )
}

The if cfg!(debug... statement will become clearer later on, when we update the Dockerfile.

Fixing Tests

Unfortunately, you may get an email saying that your Github Actions tests have failed. If you look at the logs, it should be apparent that the FileServer failed to mount, because it couldn't find the provided directory. To fix this, we can add steps to install yarn, install dependencies, and then build the static site:

steps:
  - uses: actions/checkout@v2
  - name: Build Rust
    run: cargo build
  - name: Install Yarn
    run: npm install --global yarn
  - name: Install Dependencies
    run: yarn --cwd svelte install
  - name: Build Svelte
    run: yarn --cwd svelte run build
  - name: Run Tests
    run: cargo test

For the sake of completion, I also added a simple test to check that the static site is being served as expected.

#[test]
fn static_site() {
    let client = Client::tracked(rocket()).expect("valid rocket instance");
    let response = client.get("/").dispatch();
    assert_eq!(response.status(), Status::Ok);
}

Tests should now pass as expected.

Updating the Dockerfile

Currently our Dockerfile uses two images - one to build the Rust project, and the second to run the executable. Now we need to add a third one for building the static site.


# ...
RUN cargo install --offline --path .

# use a node image for building the site
FROM node:16 as static

WORKDIR /svelte

COPY ./svelte .

RUN yarn install && yarn build

# use a slim image for actually running the container.
FROM rust:slim

# ...

COPY --from=build /usr/local/cargo/bin/aws-rust-api /usr/local/bin/aws-rust-api
COPY --from=static /svelte/build/ ./static

# ...

You should now be able to start your application using docker-compose --build and see your site when you go to http://127.0.0.1.

Adding Functionality

Although our site is being served statically, we still don't have any functionality! First, we will add support for simple Bulma styling to the app.html <head></head> tags.

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.9.3/css/bulma.min.css">

We will be using SuperAgent for making API requests, so let's add it to our dependency list.
yarn add superagent
Next, we will create a routes/__layout.svelte file, which will wrap around any of the routes in the directory.

<div id="svelte" class="container is-fluid my-5">
    <nav class="navbar is-dark" role="navigation">
        <div class="navbar-brand">
            <div class="navbar-item ml-5 is-dark">
                <img src="/favicon.png" width="32" height="32" alt="logo" />
            </div>
            <h1 class="title is-2 navbar-item">URL Shortener</h1>
        </div>
    </nav>

    <slot />
</div>

This will give us a simple, function-less navbar to go across the top of the screen.
Next, change the index.svelte file to contain the following script.

<script>
    import superagent from "superagent";

    let url = "";
    let request = null;

    function click() {
        request = superagent.post(`/api/shorten?url=${url}`);
    }

    function getUrl(key) {
        return `http://${window.location.host}/${key}`;
    }
</script>

Finally, we can bind to these variables / functions by appending the following to the index.svelte file:

<div class="box">
    {#if request == null}
        <div class="field has-addons">
            <div class="control">
                <input
                    class="input"
                    type="text"
                    bind:value={url}
                    placeholder="URL"
                />
            </div>
            <div class="control">
                <button class="button is-info" on:click={click}>Shorten</button>
            </div>
        </div>
    {:else}
        {#await request}
            <p>Loading...</p>
        {:then response}
            <div class="card">
                <header class="card-header">
                    <p class="card-header-title">Done!</p>
                </header>
                <div class="card-content">
                    <a
                        class="content"
                        href={getUrl(response.text)}
                        target="_blank">{getUrl(response.text)}</a
                    >
                </div>
                <footer class="card-footer">
                    <button
                        class="card-footer-item button"
                        on:click={() => (request = null)}>Back</button
                    >
                    <button
                        class="card-footer-item button is-info"
                        on:click={() =>
                            navigator.clipboard.writeText(
                                getUrl(response.text)
                            )}>Copy</button
                    >
                </footer>
            </div>
        {:catch}
            <p>Something went wrong!</p>
        {/await}
    {/if}
</div>

The bind:value={url} is one of Svelte's special two-way bindings - updating the textbox will update the variable, and vice versa.

When the user clicks the button, the click function will start an asynchronous request to the API, and set the request variable to the uncompleted promise.

This will then cause the page to show Loading... until the request completes, at which point the shortened URL is displayed (with some buttons).

Final Product

If you did everything correctly, you should be able to run docker-compose up --build and use your site at http://127.0.0.1!

That's all for this post! If you have any issues, make sure to check out the part-5 tag of my repo.

In the next post, we will cover the basics of EB, and set up a CD pipeline for automatically deploying your program to the cloud. Make sure to click the "Follow" button if you want to be alerted when the next part is available!

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

URL Shortener with Rust, Svelte, & AWS (4/): Dockerizing

Miles Watson — Sat, 25 Sep 2021 13:50:30 +0000

In the last post, we created some unit tests that will allow us to automatically test our application. In this post, we will cover the process of Dockerizing our application in preparation for deploying with AWS Elastic Beanstalk.

What is Docker?

Explaining fully the features and advantages / disadvantages of Docker would take an entire blog post in itself, so if you aren't familiar with the concept of containers then watch the following video:

Installing Docker

The easiest way to use Docker is with Docker Desktop (mac / windows). This will install both the docker and docker-compose tools.

For Linux users, you have to first install Docker Engine, and then install Docker Compose separately.

Creating the Docker File

First, create an empty file in the root of your repo called Dockerfile (no file extension). This file will contain a list of repeatable instructions that tells Docker how to build our container.

# select a starting image to build off
FROM rust as build

# set our working directory in the container as /repo
WORKDIR /repo

# copy all our files across from our local repo to the /repo directory in the container
COPY Cargo.lock .
COPY Cargo.toml .
COPY src src

# build the release
RUN cargo install --path .

# allow requests to port 8000
EXPOSE 8000

# this command is run when we actually start the container
CMD ["aws-rust-api"]

Building and running the container

We can build the Docker image with the following command (make sure to run it from the root of the repo). This can take a while - we'll work on optimizing build times later.

docker build -t url-shortener .

The -t url-shortener part tags our image so that we can run it in the next step, whilst the . indicates the Dockerfile is in the current directory.

Note: Ensure that the Docker daemon is running, or the Docker commands may fail.

Once the container has been built, we can start it with the following command:

docker run -it --rm --name my-running-app -p 8000:8000 url-shortener

Note: the -p 8000:8000 means that we are binding port 8000 of our local machine to port 8000 of the Docker container.

The application should appear to start as normal - however, you will notice that you cannot connect. This is because our program is listening on 127.0.0.1, but we need it to listen on all available network interfaces (0.0.0.0).

We can fix this using a Rocket.toml configuration file to configure the address and port for Release mode.

[release]
address = "0.0.0.0"
port = 80

The file is read by the Rocket instance on program start up, as long as it is in the directory where the program is run. We can put it in there by adding the following line to the Dockerfile, just before the CMD command.

COPY Rocket.toml .

You should also change the EXPOSE line to expose port 80 (rather than 8000).

We can build the image and run it again, and you should be able to connect to it by going to http://127.0.0.1. Remember to bind to port 80 instead of 8000 in the docker run command this time!

Docker Compose

To simplify the process of creating and running the Docker images, we will create a docker-compose.yml file in the root of the repo.

version: "3.9"
services:
  url-shortener:
    build: .
    ports:
      - "80:80"

Starting our container is now as simple as running docker-compose up, but make sure to append --build if you want to force an image rebuild.

Dependency Caching

If you try and change a small part of your code, you may notice that a tiny change requires downloading and installing every dependency again when building the image. This is because of how Docker's cache works - if a copied file changes at all, then every step after that copy must be recalculated.

We can fix this by splitting our Docker file into two builds - the first build to install dependencies without copying our code, and the second to build the code without reinstalling dependencies.

# ...

# copy all our files across from our local repo to the /repo directory in the container
COPY Cargo.lock .
COPY Cargo.toml .

# cache dependencies by creating an empty
# lib.rs file and building the project
RUN mkdir src
RUN echo "// empty file" > src/lib.rs
RUN cargo build --release

# now copy the code over
COPY src src

# build the release
RUN cargo install --offline --path .

# ...

Note: remember the --offline flag for the second build stage, or dependencies may be fetched again!

This means that we should only have to reinstall dependencies when either of our Cargo.toml or Cargo.lock files change.

Reducing Bloat

This step isn't as noticeable as the previous step, but it can help reduce the bloat of our image. We can do this by using two separate images - one purely for building, and the other for running our application.

# ...

# build the release
RUN cargo install --offline --path .

# use a slim image for actually running the container.
FROM rust:slim

WORKDIR /app

# allow requests to port 80
EXPOSE 80

# install the program onto the current image
COPY --from=build /usr/local/cargo/bin/aws-rust-api /usr/local/bin/aws-rust-api

# copy config file
COPY Rocket.toml .

# this command is run when we actually start the container
CMD ["aws-rust-api"]

Congratulations! Your program is almost ready to be deployed on AWS Elastic Beanstalk. If you have any issues, you compare your code with the part-4 tag of my repo.

In the next post, we will create a statically-served frontend for our application with Svelte and Bulma. Make sure to click the "Follow" button if you want to be alerted when the next part is available!

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

URL Shortener with Rust, Svelte, & AWS (3/): Testing

Miles Watson — Wed, 22 Sep 2021 18:19:47 +0000

In the last post, we created a simple URL-shortener API and tested it manually with curl and a browser. In this post, we will use Rust's integrated unit-testing features to allow testing with a single command, and then automate testing with GitHub Actions.

Creating a test module

In Rust, we can create unit-tests which allow us to test individual parts of our application. To get started, create a module called tests in your main.rs file, and add the cfg(test) macro.

#[cfg(test)]
mod tests {
    // ...
}

Now we have created a tests module, we can add tests inside it with the test macro.

#[test]
fn simple_demo_test() {
    let x = 1 + 1;
    assert_eq!(x, 3)
}

By running cargo test, you should see that the test fails as expected:

running 1 test
test tests::simple_demo_test ... FAILED

By changing the 3 to 2 and rerunning cargo test, you should be able to see it pass as expected.

Valid Request Testing

The Rocket crate includes functionality which allows us to write unit tests for our applications (you can find the full docs here.

In particular, we will use the rocket::local::blocking::Client struct to simulate requests to our API. We will first use it to check that valid requests are accepted correctly.

#[test]
fn valid_requests() {
    // ...
}

First, we create a client that is able to make requests to our Rocket.

let client = Client::tracked(rocket())
    .expect("valid rocket instance");

Next, we make a POST request to shorten a url, and check that the response status is Ok.

let response = client
    .post("/api/shorten?url=https://duck.com")
    .dispatch();
assert_eq!(response.status(), Status::Ok);

We can then attempt to extract the key which was returned from the response body, panicking if the key could not be parsed successfully.

let key: u32 = response
    .into_string()
    .expect("body")
    .parse()
    .expect("valid u32");

Now we can make a GET request to the shortened URL, and check that the response is a redirect to the original URL.

let response = client.get(format!("/{}", key)).dispatch();

assert_eq!(response.status(), Status::SeeOther);

let redirect = response
    .headers()
    .get_one("Location")
    .expect("location header");

assert_eq!(redirect, "https://duck.com")

Running the full test with cargo run should result in a successfull pass.

Invalid Request Testing

In the current edition of our API, there are two ways that a request can fail - we should check these both.

First, we will create a test to check that a missing URL parameter throws an error.

#[test]
fn empty_url() {
    let client = Client::tracked(rocket())
        .expect("valid rocket instance");
    let response = client.post("/api/shorten?url=").dispatch();
    assert_eq!(response.status(), Status::BadRequest);
}

And finally we will create a test to ensure that the server returns Not Found to an invalid shortened URL.

#[test]
fn invalid_url() {
    let client = Client::tracked(rocket())
        .expect("valid rocket instance");
    let response = client.post("/123").dispatch();
    assert_eq!(response.status(), Status::NotFound);
}

Both these tests should pass without issue.

GitHub Actions

Now we have created our tests, we can write a GitHub actions script to automatically run our tests whenever we push to the main branch. Create a file with the path .github/workflows/test.yml, and add the following script:

name: Test
on:
  push:
    branches: [main]
env:
  CARGO_TERM_COLOR: always
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Build
        run: cargo build
      - name: Run tests
        run: cargo test

When you push your code to GitHub, you should be able to see your tests run (and hopefully pass) in the "Actions" tab of your repo.

If you have an issue, you can compare your code to the part-3 tag of my repo.

That's all for this post! In the next one, we will work on containerizing our application with Docker. Make sure to click the "Follow" button if you want to be alerted when the next part is available!

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

URL Shortener with Rust, Svelte, & AWS (2/): Simple HTTP API

Miles Watson — Mon, 20 Sep 2021 20:32:32 +0000

In the first post of the series, I covered the reasons for choosing Rust and AWS, as well as the process for initialising a new Rust project. If you haven't followed the steps in that article, you can find it here.

In this article, we will create a simple URL-shortener API, and serve the endpoint locally. For the web framework, we will be using Rocket to reduce the amount of boilerplate and help us focus on the application logic.

Getting Started with Rocket

Before we can use Rocket, we need to add it to our list of dependencies (cargo.toml). We'll be using the JSON feature

[dependencies]
rocket = "0.5.0-rc.1"

To check everything is working, copy the following code to main.rs.

#[macro_use]
extern crate rocket;

#[get("/")]
fn index() -> &'static str {
    "Hello, world!"
}

#[launch]
fn rocket() -> _ {
    rocket::build().mount("/", routes![index])
}

When you start the program with cargo run, you should be presented with the following message:

🚀 Rocket has launched from http://127.0.0.1:8000

If you open the link in a web browser, you should be presented with the expected "Hello, World!" message.

Concurrent HashMap

To store pairings of shortened URLs to full URLs, we will use a hashmap. However, we need to share this map across threads - those familiar with Rust will know that this can be quite tricky due to the restrictions of the borrow checker.

One approach to solving this would be to simply wrap a HashMap in a Mutex for controlling concurrent accesses, and then use an Arc for referencing counting. However, we can simplify this in two ways.

Firstly, we can use the dashmap crate for a fast, concurrent hashmap (DashMap implements Sync so it can be shared safely across threads). Although perhaps overkill for our use-case, dashmap provides better performance than naively using an RwLock. To install it, add the following dependency to cargo.toml.

dashmap = "4.0.2"

As for sharing this state across threads, we only need to access it from endpoints: therefore we can just let Rocket manage it directly.

#[launch]
fn rocket() -> _ {
    rocket::build()
        .manage(DashMap::<u32, String>::new())
        .mount("/", routes![index])
}

For more information on state in Rocket, you can check the Rocket docs.

Creating the endpoints

Now we need to create the actual endpoints to allow users to create and follow shortened URLs. For random number generation, we will use the rand crate, so add the following to your dependency list.

rand = "0.8.4"

The first endpoint will listen to POST requests of the form /api/shorten?url=___, and then generate a random url to return. It will return an error if the url field is empty, or a key if the URL was added to the hashmap.

Rocket will automatically parse the URL parameter and inject the managed hashmap.

#[post("/api/shorten?<url>")]
fn shorten(url: String, state: &State<DashMap<u32, String>>) -> Result<String, BadRequest<&str>> {
    if url.is_empty() {
        Err(BadRequest(Some("URL is empty!")))
    } else {
        let key: u32 = thread_rng().gen();
        state.insert(key, url);
        Ok(key.to_string())
    }
}

The other endpoint will listen to GET requests of the form /<key> where is a number. If the key exists in the hashmap, then it will redirect the user to the corresponding URL. Otherwise, it will return an error.

#[get("/<key>")]
fn redirect(key: u32, state: &State<DashMap<u32, String>>) -> Result<Redirect, NotFound<&str>> {
    state
        .get(&key)
        .map(|url| Redirect::to(url.clone()))
        .ok_or(NotFound("Invalid or expired link!"))
}

Remember to update your rocket function to mount the new routes!

Manual Testing

To check that your API is working as expected, you can use a tool like Postman or Curl to make POST requests, then enter the link manually in a browser. To use curl, try the following command:

curl -X POST -G --data-urlencode 'url=https://duck.com' http://localhost:8000/api/shorten

Warning: In PowerShell, curl is simply an alias for Invoke-WebRequest, therefore this command may not work without installing it manually.

You should see the endpoint respond with a number - use this in your browser to make the GET request. If your number was 123, for example, you should enter http://127.0.0.1:8000/123 into your address bar. You should be automatically redirected to whatever URL you set in the previous POST request.

If you are having any issues, check out the part-2 tag of my repo.

That's all for this post! In the next post, we will create a simple HTTP API with the Rocket web framework. Make sure to click the "Follow" button if you want to be alerted when the next part is available!

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

URL Shortener with Rust, Svelte, & AWS (1/): Intro + Setup

Miles Watson — Mon, 20 Sep 2021 20:31:53 +0000

In this series of posts, I aim to guide you through the process of creating and deploying a public API. Over the course of the series, you will learn how to use the following technologies:

Rust with Rocket web framework - for handling API requests
Docker and Docker Compose - for containerizing your application
Svelte and Bulma - for creating a simple frontend
Elastic Beanstalk - for hosting your service
IAM and GitHub Actions - for automating testing + deployment

Rust

Explaining fully the reasons for choosing Rust would be a blog post in itself - instead, I recommend you watch the following video by Jon Gjengset:

If you haven't previously used Rust before, then I strongly recommend you spend some time reading through the Rust book before attempting to do any of this yourself.

Svelte

Svelte is a different kind of web framework - instead of being a library like React or Vue, it is actually a compiler under the hood. This enables you to write really clean, concise, and performant code (it does surgical DOM updates instead of using VDOM diffing+reconciliation).

If you want to learn more (or even if you don't), I can highly recommend watching this excellent presentation by the creator of Svelte.

AWS

Amazon Web Services (AWS) is the world's biggest cloud service provider. They provide over 200 services that allow organizations and individuals to rent resources from one of AWS' many data centers.

Cloud computing is becoming increasingly popular, as more and more companies prioritize agility and elasticity (which cloud platforms offer) over control and cost-efficiency (which traditional, managed infrastructure provides).

For beginners, AWS offer a free tier, which should allow you to complete this tutorial without spending any money. However, accidentally spending money is an easy mistake for beginners to make - I recommend following this video to minimize any surprise bills.

Before moving on...

Install the latest stable Rust version by following these instructions
Create a Github repo for storing your code (commit + push whenever you see fit)
Use cargo init --bin to create a new Rust project, and check it works with cargo run
Install Node.js
Install Yarn with npm install --global yarn

If you have any issues, compare your code with the part-1 tag of my repo.

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

Creating Better Applications with MVVM

Miles Watson — Sat, 09 Jan 2021 20:12:48 +0000

In this article, I explain what MVVM is and how it can help you create better web/desktop applications. I use C#/WPF in my examples, although you should be able to follow along with basic knowledge of OOP and HTML.

Context

Until a month ago, my experience of creating graphical user interfaces was limited to plain HTML/CSS websites and some basic python applications using appJar. I always felt intimidated by the concept of having to work with GUIs: as a backend-oriented programmer, my way of thinking never seemed to mesh well with the popular frameworks.

Recently, I have had to started working on a desktop client for my decentralized social media. Most people would agree that this is an intimidating project, especially for an inexperienced frontend developer. Initially, I found making progress hard - until I learnt about MVVM.

What is MVVM?

MVVM stands for Model, View, View-Model. These form the three main parts of an MVVM application.

Models

Models contain core algorithms and data structures to be used by the application - the "business logic". These are often written in a "pure" programming language, without using any special libraries.

View-Models

View-models are special classes to represent the state of a view. For example, they have properties to represent displayed text, user-input textboxes, or even other view models. They should also have functions / commands which represent events (such as when a button is clicked).

View models often have some boilerplate code, so that they can integrate with whatever MVVM framework you are using. However, they should work on their own, with no dependency on the view.

Views

Views contain the code that is used by the UI. They are often written with a markup language (e.g. XAML or HTML). The views outline the structure (and styling) of the view, and their content is set with bindings.

Bindings allow content to be linked to a property of the view-model. Changing the property in the view-model will automatically update the shown content in the view. The binding can also work the other way, using a two-way binding. This is useful for textboxes, as you can have the view-model property update on each keystroke (this can be used for real-time input validation).

A Quick Example

To demonstrate the ideas I've talked about, I will quickly guide you through the process of creating an application using MVVM. The application will allow the user to click through a list of people.

I'll be using WPF (Windows Presentation Framework). If you want to follow along, create a new WPF App (.NET) project in Visual Studio. I've called mine Demo.

Creating a model

First, we'll create a class to represent a person. It will contain two fields - Name and Age.

I've put the class in the Demo.Models namespace, and saved it as Models/PersonModel.cs:



public class PersonModel
{
    public string Name { get; set; }
    public int Age { get; set; }

    public PersonModel(string name, int age)
    {
        Name = name;
        Age = age;
    }
}

Creating the view model

Next, we will create the View Model. There will be four properties - two strings, and two commands. The strings will represent the name and age of the currently displayed person, whilst the buttons will allow us to click through the people.

Whenever we change a property, we have to alert the framework (so that the view can be updated). In WPF, this is achieved by implementing INotifyPropertyChanged and then calling NotifyPropertyChanged whenever a property is set.

I've put the class in the Demo.ViewModels namespace, and saved it as ViewModels/PeopleViewModel.cs:



public class PeopleViewModel : INotifyPropertyChanged
{
    //// Bindable properties

    private string _name;

    public string Name
    {
        get => _name;
        private set
        {
            _name = value;
            NotifyPropertyChanged(nameof(Name));
        }
    }

    private string _age;

    public string Age
    {
        get => _age;
        private set
        {
            _age = value;
            NotifyPropertyChanged(nameof(Age));
        }
    }

    public ICommand Previous { get; init; }

    public ICommand Next { get; init; }

    //// Implementation

    private PersonModel[] people = new PersonModel[] {
        new PersonModel("Alice", 20),
        new PersonModel("Bob", 25),
        new PersonModel("Charlie", 30)
    };

    private int index = 0;

    public PeopleViewModel()
    {
        Name = people[index].Name;
        Age = people[index].Age.ToString();

        Previous = new RelayCommand(() =>
        {
            if (index > 0)
            {
                index--;
                Name = people[index].Name;
                Age = people[index].Age.ToString();
            }
        });

        Next = new RelayCommand(() =>
        {
            if (index < people.Length - 1)
            {
                index++;
                Name = people[index].Name;
                Age = people[index].Age.ToString();
            }
        });
    }

    //// Boilerplate code to satisfy INotifyPropertyChanged

    public event PropertyChangedEventHandler PropertyChanged;

    public void NotifyPropertyChanged(string propertyName)
    {
        if (PropertyChanged is null) return;
        PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
    }
}

In this example, the RelayCommand class is used to wrap functions as an ICommand. An explanation of this (and the code for the class) can be found here.

Why isn't the age an integer?

The age property isn't an integer, because it isn't representing the age of the person: it is representing the displayed content. The view will be displaying a string, therefore the property should be of type string.

Creating the view

Now that the view-model has been created, we will create a UserControl to act as the view. It will have two labels and two buttons - the labels will be bound to the Name and Age strings, and the buttons will bound to the Previous and Next commands.

In WPF, each view has a DataContext. By including the Demo.ViewModels namespace, we can set the data context of the user control to be a PersonViewModel. Then, we can bind to a property X of the UserControl.DataContext by using the "{Binding X}" syntax.

I've saved the user control as Views/PeopleView.xaml:



<UserControl x:Class="Demo.Views.PeopleView"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
             xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
             xmlns:viewmodels="clr-namespace:Demo.ViewModels"
             mc:Ignorable="d"
             d:DesignHeight="450" d:DesignWidth="800">
    <UserControl.DataContext>
        <viewmodels:PeopleViewModel />
    </UserControl.DataContext>

    <StackPanel Background="White">
        <Label Content="{Binding Name}" />
        <Label Content="{Binding Age}" />
        <StackPanel Orientation="Horizontal">
            <Button Content="Previous" Command="{Binding Previous}" />
            <Label />
            <Button Content="Next" Command="{Binding Next}" />
        </StackPanel>
    </StackPanel>
</UserControl>

What about the code-behind?

The code-behind is rarely used in WPF applications that use MVVM. All the UI logic should be contained in the view-model, not the code-behind. This separation of concerns is what makes MVVM such a versatile architecture!

Finishing up

Now that we've created our view, we can display it in MainWindow.xaml.



<Window x:Class="Demo.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:views="clr-namespace:Demo.Views"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">

    <StackPanel>
        <views:PeopleView />
    </StackPanel>
</Window>

Upon running, you should see the application working as expected:

Why all the effort?

You may be wondering why we should go through all this effort. There are three main reasons that MVVM is better than the standard "code-behind" approach.

Testability

Because your view-model is entirely decoupled from your view, you can use a testing framework (e.g. MSTest) to check that your interface logic is correct, without the need to manually test the UI yourself.

Modularity

Now that you have created your model and view-model pair, you can can reuse them as a component across your code. For example, you can add a second instance of the component like so:



<StackPanel>
    <views:PeopleView />
    <views:PeopleView />
</StackPanel>

By nesting components within each other, you can build up complex, modular user interfaces with minimal effort.

You can even create multiple views bound to the same view-model! I've done this by setting the DataContext of the MainWindow to be a new PersonViewModel, and then binding both of the views to it.



<Window x:Class="Demo.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:viewmodels="clr-namespace:Demo.ViewModels"
        xmlns:views="clr-namespace:Demo.Views"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">

    <Window.DataContext>
        <viewmodels:PeopleViewModel />
    </Window.DataContext>

    <StackPanel>
        <views:PeopleView DataContext="{Binding}" />
        <views:PeopleView DataContext="{Binding}" />
    </StackPanel>
</Window>

Note: You would normally bind the DataContext property of a view to a property of the current view-model. However, the main window doesn't have a view-model, and so I've bound it straight to Window.DataContext.

Platform Agnosticism

Splitting your application into the three parts allows you to reuse models and view-models across multiple platforms. This is made easy by using cross-platform frameworks like MvvmCross (which supports Windows, Mac, iOS and Android).

Conclusion

MVVM is an application architecture which allows you to develop testable, modular, and cross-platform applications. It splits code into three parts:

model
view-model
view

The separation of concerns improves code maintainability and readability. Furthermore, the model and view-model can be unit-tested and shared across platforms, allowing for quicker development of cross-platform applications.

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

Disclaimer - I'm a (mostly) self-taught programmer, and I use my blog to share things that I've learnt on my journey to becoming a better developer. I apologise in advance for any inaccuracies I might have made - criticism and corrections are welcome!

Programming is Ruining My Life - What I'm Doing About It

Miles Watson — Sat, 19 Dec 2020 21:06:47 +0000

Okay, the title might have been a slight exaggeration - programming has played a very beneficial role in both my academic and professional development. I have noticed recently, however, that programming has caused a significant decrease to my quality of life.

My Situation

Hi! My name's Miles, and I'm a teenager living in the UK. I've been programming in one way or another for over five years, but have recently been stepping up my workload (mostly due to my school computer science project, among other things). A typical school day over the past month has looked like:

wake up, breakfast
school (5 hours of lessons, most of lunch break programming)
homework / Advent Of Code until dinner
project work after dinner until bed The weekends are fairly similar, except with more homework / programming in the place of lessons.

This schedule has been fairly consistent for a few months now, and I wanted to share some of the issues I've had (as well as some of the methods I'm using to cope).

Issues

Eyesight

Over the past year or so, I have seen a noticeable decrease in my ability to discern details at distances further than a few metres. The impacts were most felt when I started school after lockdown, and had to move closer to the front of the classroom in order to make out questions.

I believe that this was caused long periods working at a computer, rather than simply due to genetics - mostly due to the fact that my eyesight felt quite a bit worse after my more "productive" days.

In addition to having developed short-sightedness, I have recently experienced some of the joys of eye strain. The discomfort persisted for a couple days, forcing me to take a break from computer work. It was after this particular event that I realised that something had to change.

Posture and Back Discomfort

Sitting at a computer has caused a slight discomfort in my lower back and rounded shoulders. Although minor, I assume this discomfort will only become worse as I age.

Lethargy

This one's quite simple - do you lack energy, or feel tired/weak/fatigued after small amounts of exercise? Being sedentary for long periods of time is not healthy, and can lead to both mental and physical illness. I personally noticed myself getting out of breath after flights of stairs, and feeling unfit caused a drop in my self-confidence.

Burnout and Irritability

After a long day of problem solving, I noticed that I felt both mentally and physically exhausted. This exhaustion destroyed my motivation for other hobbies, and made me more irritable (and therefore unsociable). Instead of spending my time reading, playing games, or talking with friends, I instead tried to numb my stress by endlessly scrolling through Reddit and other social media platforms until I fell asleep.

Tilting

Urban Dictionary

Although mostly used to describe a feeling in videogames, I find that the experience translates well to problem solving. Getting frustrated at a bug leads my mental clarity to decrease, causing a feedback loop that ends in me spending hours on a problem (often until late at night, causing irregular sleep patterns).

What I'm Doing About It

Here are some methods that I have found particularly helpful for dealing with and preventing the issues I've talked about:

20-20-20 Rule

I've recently started using the 20-20-20 rule. The rule says that, for every 20 minutes of computer work, you should focus on something 20 feet away for 20 seconds. I've found this to give good (albeit temporary) relief, although I usually do it for longer than 20 seconds.

Regular Breaks

Taking short breaks every hour or so has been very beneficial. During these breaks, I make an effort to move as much as possible - either a short walk outside, a few press-ups/pull-ups, or even just walking around the house.

Posture Exercises

I've been doing daily posture exercises to help fix my posture. So far, I've made noticeable progress.

Here are some good resources:

Gym Routine

You've probably heard this one a lot. There's a reason - it really works. Getting regular exercise at the gym (three days a week) has made me both fitter and happier^{^;)}. It has the combined effect of:

Improving overall health
Providing a way to relax
Clearing my head
Increasing my self confidence

Important note: There is a tendency among gym-goers to focus on "push" actions. These can create a natural tension in the chest, leading to the shoulders being drawn in. Make sure to perform the posture exercises mentioned above, and balance your workout with exercises that target the back - think "pull" actions (deadlifts are also good for general posture, but can be dangerous if performed with bad technique).

Learning How to "Give Up"

Being able to recognise when I am "tilting", and having the self control to force myself to put a problem to the side has been very beneficial. I've found that clearing my head before trying again produces better results with less total time spent when compared to stubbornly grinding away at it.

This is probably the thing which I've found the hardest to do on this list - I have (like many other programmers) a natural tendency to obsess over problems until they are solved. Realising that putting a problem on hold is NOT admitting defeat was a major milestone in getting more consistent sleep and maintaining a better work/life balance.

Getting Kittens

I don't think this one needs much explanation :) Pets can be a great way to destress - just make sure they don't interrupt your Zoom calls (grrr).

Conclusion

As a developer, maintaining your physical and mental health is crucial to staying both productive and happy. Take frequent breaks, regularly exercise, and ensure that you are eating/drinking/sleeping well - your mind will thank you.

Have you been having any issues with burnout, work-life balance, or physical health? How have you been coping? Let me know in the comments below!

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

Disclaimer - I'm a (mostly) self-taught programmer, and I use my blog to share things that I've learnt on my journey to becoming a better developer. Because of this, I apologise in advance for any inaccuracies I might have made - criticism and corrections are welcome!

Why You Should Be Doing Advent of Code (it's not too late to start)

Miles Watson — Wed, 02 Dec 2020 20:18:54 +0000

You may have recently heard some talk about Advent of Code - every developer's favourite advent calendar. In this article, I'm going to try and explain why everyone can get something out of it.

// Detect dark theme var iframe = document.getElementById('tweet-1333646123001634816-432'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1333646123001634816&theme=dark" }

What is Advent of Code?

Advent of Code is a website that sets daily challenges for each day of advent. The context for each challenge tells part of a story that spans the full 25 days.

Each day, you are presented with two problems that are designed to be challenging whilst remaining accessible to those without much programming experience.

Solving a problem rewards a gold star - to complete the challenge, you need to collect all 50 of them. Whilst this sounds simple, the challenges vary in difficulty (with an upwards trend as the event progresses).

What languages can I use?

You can use any language! None of the problems involve uploading your own code - you can solve all of them locally, and then copy/paste the answer into the submission box. You can even solve them with an esolang if you want a challenge!

Why should I be taking part?

Almost everyone has a reason to take part in this competition! However, the most compelling reason depends on where you are in your programming journey.

If you are just starting out

This is an opportunity for you to start using some of the basic skills you may have been learning. If your programming experience is limited to online courses or video tutorials, then AoC is a great way to practice applying your knowledge to solve problems by yourself. The story is a nice touch too!

If have been programming for a bit

Advent of Code is a great way to practice your problem solving skills. It can be used as an introduction to competitive programming - perhaps you might get a hobby out of it! At the very least, you can expect to improve your interview skills.

I would recommend timing yourself to see how quick you can solve the problems, and then try and compete with your friends / colleagues.

If are an experienced programmer

As you might have some competitive programming experience, you can take this challenge a little more seriously. If you are fast enough at completing them from when the challenge launches, you might even be able to make it onto the leaderboard!

Alternatively, you can take AoC as an opportunity to try and get some experience with that new programming language you have been eyeing for a while. I personally have chosen Go, because I want to be familiar with the syntax before experimenting with the various web frameworks that it offers.

Conclusion

Advent of Code is a great idea that can really get you thinking. It is on its sixth year now, and is continuing to grow. In fact, there was so much traffic on the first day this year that it brought down the servers!

You can get started at the following link: https://adventofcode.com/

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

What is Async Programming, and Why Should You Care?

Miles Watson — Fri, 27 Nov 2020 20:05:41 +0000

Async programming is a style of programming that allows you to increase performance when handling events independent of the main program flow. For example, some common tasks that benefit from being asynchronous include:

handling user interface events
communicating over a network
reading / writing to a secondary storage device

What's the problem?

As programmers, we are used to working with synchronous code. Synchronous functions are easy to understand - we call them, they do some work, and they return a value.

There are situations where this leads to very bad performance. Imagine we want to get the contents of every page given by a list of URLs.

public List<string> GetAllPages(List<string> urls)
{
    var pages = new List<string>();
    foreach (var url in urls)
    {
        pages.Add(GetPage(url));
    }
    return pages;
}

This will be very slow for large lists of URLs, as the for loop has to wait for each page request to finish before starting the next one. We can therefore say that each function call is blocking - it blocks the entire thread until the result is returned.

How can we fix it?

Async programming helps fix this issue by separating the function call and the function result. For example, we can start all the requests, and then wait for the results to arrive. Pseudocode might look something like this:

def GetAllPages(urls):
    pages = list()

    for url in urls:
        StartRequest(url)

    for url in urls:
        pages.Append(WaitForResponse(url))

This is rather clunky, as you have to define a start and an end function for each operation you want to make asynchronous. Thankfully, C# has a nice syntax to make this easier.

How does it work in C#?

In C#, asynchronous operations are represented using the Task<T> type. The "start" function returns a Task object, from which the result can be obtained. If the function GetPage() was written using Task Awaitable Programming (TAP), you could write the program as follows:

public List<string> GetAllPages(List<string> urls)
{
    var tasks = new List<Task<string>>();
    foreach (var url in urls)
    {
        // starts the asynchronous operation
        var task = GetPage(url);
        tasks.Add(task);
    }

    var pages = new List<string>();
    foreach (var task in tasks)
    {
        // blocks the thread to wait for the result
        pages.Add(task.Result);
    }

    return pages;
}

Whilst this works, it isn't ideal. Although the individual calls to GetPage aren't blocking the function, any external code that calls GetAllPages will be blocked until all the requests have finished.

To combat this, we can write the function in an asynchronous style. This usually takes three steps:

Add the async keyword
Change the return type to Task<T>
Replace synchronous waits (Task.Wait(), Task.Result) with the await keyword

The previous function would now look as follows:

public async Task<List<string>> GetAllPages(List<string> urls)
{
    var tasks = new List<Task<string>>();
    foreach (var url in urls)
    {
        // starts the asynchronous operation
        var task = GetPage(url);
        tasks.Add(task);
    }

    var pages = new List<string>();
    foreach (var task in tasks)
    {
        // doesn't block the thread
        pages.Add(await task);
    }

    return pages;
}

`await` VS `.Result`

When you call .Result, the system thread remains blocked even when waiting.

On the other hand, using await frees up the thread to allow other tasks to run. For example, this means that a server can use only 4 system threads to handle 100 clients (as opposed to the 100 system threads in a naive approach).

Therefore, await should be used in place of .Result whenever possible.

Note: A function must be marked async for the await keyword to be used.

What's the catch?

Async programming can be very useful in certain situations. As a rule of thumb, it is only increases performance when the program is IO-bound. You shouldn't use async functions to do CPU-bound calculations, as it:

provides almost no useful functionality
makes the code less readable
might decrease performance

One exception to this rule is the Task.Run() function, which allows CPU-bound work to be performed on a background thread.

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!

Why You Should Migrate to .NET 5 (and why you shouldn't)

Miles Watson — Sat, 14 Nov 2020 11:09:10 +0000

.NET 5 is the latest and greatest version of .NET from the folks over at Microsoft, and it's better than ever. Almost everyone should migrate.

What's all the fuss?

Up until now, .NET has been fragmented into 4 different platforms:

Framework
Core
Standard
Xamarin

As you can imagine, this leads to issues with code compatibility, especially when developing cross-platform applications. .NET 5 is an evolution of Core which replaces the need for Standard. It provides a unified platform on which you can build any application.

Why should I care?

There are three main reasons you should switch over:

Unified platform helps with code reuse across multiple platforms
Language updates to C#, F#, and VB
Performance improvements across the board

Is migrating my applications difficult?

In most cases, migrating to .NET 5 can be done in two steps:

Update to the latest version of the SDK (this can be done by updating VS)
Change the target framework in your project file (.csproj for C#).

Make sure to check any breaking issues before updating - here are breaking changes for .NET Core 3.1 to .NET 5.0.

Why shouldn't I migrate?

There are some older technologies that are no longer supported in .NET 5:

Web Forms
Windows Communication Foundation
Windows Workflow

Newer alternatives to these can be found on Microsoft Docs, but they may take time to implement.

Footnote

If you enjoyed reading this, then consider dropping a like or following me:

I'm just starting out, so the support is greatly appreciated!