Forem: lukeojones

1UP your Zsh abilities by autoloading your own functions

lukeojones — Sun, 24 May 2020 12:15:59 +0000

Zsh is quickly becoming the shell of choice for software developers and, if you want to maximise your productivity at the CLI, you should know how to create and autoload your own Zsh functions.

Even novice CLI users will have added the occasional alias in order to avoid repeatedly typing out a verbose command. Whilst aliases are great, they aren’t well suited to the more complicated, logical code snippets that you may want to run on a regular basis.

In this whistle-stop tour, I’ll show you how to create basic Zsh functions and integrate them with Zsh’s autoloading system so they’re available during interactive shell sessions.

Creating our function

After reading this article, I’m sure you’ll be inspired to write lots and lots of functions but, before you get carried away, it’s good practice to create a directory to store them all. I usually create a hidden subdirectory in my home directory called .zshfn

By convention, each function is stored in a file with the same name; for the purposes of keeping this guide snappy, I’m going to create a very simple clock function that just displays the current time and so I’ll create a file called clock (no file extension) to put my function code in.



mkdir ~/.zshfn
touch $_/clock

Now edit the clock file and add the following snippet:



# Initialisation Code
echo "Initialising Clock"

# Display simple clock every second
clock () {
  while [ true ]
  do
    echo "=========="
    date +"%r";
    echo "=========="
    sleep 1;
    clear;
  done
}

# Clear the three lines from the terminal
clear() {
  for i in {1..3};
  do
      tput cuu1
      tput el
  done
}

clock

We’ll come back to the function later — for now just save the clock file.

Configuring our .zshrc file

In order to make our function accessible whenever we startup a new terminal, we need to make a couple of modifications to the .zshrc file.

First we need to ensure we add our .zshfn function directory to the Zsh file search path (a.k.a fpath) so that Zsh knows it should look in our new directory for functions. As you might expect, the fpath is traversed from left-to-right like other unix path variables.

Secondly, we need to tell Zsh about the clock function we just created — this is achieved via the autoload command. The autoload command essentially instructs Zsh to register a function (called clock) using the contents of the clock file that is located in a directory somewhere along the fpath.



# Bottom of .zshrc file
fpath=( ~/.zshfn "${fpath[@]}" )
autoload -Uz clock

As you add more functions, you’ll find it becomes cumbersome to append each one to the end of the autoload command and so (if you want to be super cool) you can choose to autoload all files in your .zshfn folder using the following alternative:



# Bottom of .zshrc file
fpath=( ~/.zshfn "${fpath[@]}" )
autoload -Uz $fpath[1]/*(.:t)

If you’re wondering what the -Uz options do, they simply ensure that autoloading will do the right thing regardless of what other options you may have configured (i.e. enforces disabled alias-expansion and zsh style autoloading).

Running our function

After saving our clock function and the .zshrc file we are now ready to test the function out. In your terminal begin typing clock — you should be able to tab-complete the function and if you have syntax highlighting turned on you should see that kick in too.

Hit return and you should see the following output:

Remember that as far as Zsh is concerned, before running the clock function, the entire contents of the clock file are used as the function definition. This is why the first time you run clock, you see the initialisation code being executed.

Kill the function (Ctrl-c) and run the function again — you should see the following output the second time around:

Notice that in the second execution, the initialisation code does not run— this is because the first time we ran the function, the clock function was redefined to use the actual clock() function defined in our file. This can be a really nice pattern to use if you have some sort of one-off initialisation you need to accomplish.

At this point you might be wondering why the clock() method is even needed? Well actually, it isn’t. You could replace the clock file with the following contents and it would work just fine.



echo "Initialising Clock"

while [ true ]
do
  echo "=========="
  date +"%r";
  echo "=========="
  sleep 1;
  for i in {1..3};
  do
      tput cuu1
      tput el
  done
done

If you now execute this version of the clock function twice in a row you’ll see that, because the clock function is not redefined, the initialisation code runs both times.

Which approach you use is largely down to your use-case and stylistic preferences. I tend to prefer the first option (i.e. encapsulate my clock code in a clock() function block) but in many cases it is completely superfluous.

I hope you learned something from this quick write-up — there is a lot more to Zsh functions than covered here and if you want more detailed information, I’d recommend you check out this resource.

In any case, functions are a great way to optimise your CLI experience and it’s certainly worthwhile knowing how to use them 😎👍

Organisational tools to help make the most of your dev side-projects

lukeojones — Mon, 18 May 2020 17:46:54 +0000

Recently, as part of a migration to Github, I decided to rid my Gitlab account of old repositories. Before starting on that task, if you had asked me how many stale repositories I had in there, I would have said no more than five — I removed over 20!

Working through this list and deleting old projects was a nostalgic walk down memory lane but, as I scanned each project, the pattern that started to emerge was that most of these projects had started out with a real buzz, dedication and enthusiasm only to be abandoned a few weeks (or even days) later in favour of another one (now also deleted).

By the end of this spring cleaning operation, the walk down memory lane had transformed into a depressing jaunt through an abandoned museum — there were some interesting exhibits but no-one was around any more to tell you how they worked or what they were used for. I had little idea about the state of the project and how incomplete it actually was.

Of course the answer to all this was obvious — I needed another new side project! (Dark Kermit strikes again) At this point it was more than clear that I would need to change my approach and ensure I avoided another half-baked project clogging up my Github account.

Break it down

Side projects should be fun, challenging and rewarding but sometimes it’s difficult to distribute those factors over the lifetime of the work.

In particular, when you first devise your new idea, it’s especially fun and rewarding because you’re getting to investigate all the new tools and concepts that might be relevant to the implementation.

After that initial stage, it can often be hard to stay motivated as you start to realise the scale of the project and realise those new tools aren’t as super simple to use as you expected 😫

For me, the best way to deal with this is to break the workload down into small (and hopefully enjoyable) subtasks. We do this in our 9–5 jobs all the time using tools like Jira and Youtrack (OK, maybe it’s not always enjoyable) but it’s easily overlooked when you’re just working on something in your free time and there is no client or customer chasing you for the goods.

Breaking down these ‘awesome ideas’ into smaller tasks has a number of small benefits that compound over time to give you a better result:

Mental Offloading: You don’t need to keep everything in your head since it’s been transferred into pen/pencil/pixels.
Progress: It’s easy to see how much effort is required to hit the next milestone. If you’re just working on one big, nebulous concept it’s hard to keep perspective. It’s also easier to resume working on the project after a small break.
On-boarding: If you decide to team up with mates or colleagues, you’ve got an instant starting point to discuss the remaining work and what has been done to date.
Code Context: Although your code should be the canonical documentation of your project, having a proper space for documentation or issue tracking can be really useful especially when looking back.
Rewarding: Completing tasks is rewarding and after finishing the first few tasks, you start to form positive habits that will keep you on track — just ask Pavlov’s 🐶.

With this in mind, I’m going to quickly describe how I’ve used some free tools to finally get a little side-project off the ground — response.dev — but you should use whatever works for you.

The Trello board

I certainly didn’t want to make my side-project feel exactly like work — where’s the fun in that? I just wanted a lightweight alternative to Jira that provided:

The ability to document tasks quickly and move them between different states without too much config or admin.
A concise way of referring to tasks e.g. a numeric identifier
A way of linking issues to commits, PRs, branches etc.
Everything I needed for free! 💸

Luckily, a tool I’ve used for a number of years — Trello — can be configured to meet all the criteria above. I’ve documented some of the key features I’m using right now but the setup is quite fluid and because this is just for side-project work, I don’t have to get buy-in to change anything 😁

Columns

I’ve experimented with various column setups but this seems to be working quite nicely although YMMV.

Todo: Things that need implementing
Doing: What I’m working on currently. This helps keep me focused so I don’t juggle too many things at once.
Done: Moving things to Done provides some positive feedback and builds a feeling of progress and momentum. It also serves as a knowledge base for previous tasks.
Nice to Have: Things I’d like to implement but aren’t essential.
To Learn: The whole reason I like side-projects is that it gives me a chance to learn and pick up new skills. Sometimes, the list of things I’d like to learn is overwhelming so I add them here and revisit later when I’m commuting or waiting for the kettle to boil etc.

Issue Identifiers

Trello uses Card IDs under the hood but they aren’t visible by default so you only see the description. I like to have an ID so that I can refer to it more easily in conversation, other issues, PRs or commit messages etc.

You can install this Chrome extension to expose the Card IDs without using up a valuable Power-up. If you are using the paid version of Trello and don’t need to worry about Power-up allowance, then I’d recommend using this power-up from Reenhanced.

Github Integration

As a developer working with Trello regularly, it is really worth looking into this fantastic Github Power-up so that you can easily link your Trello cards to your code.

If you’re using the unpaid version of Trello then this is a great way to make the most of your Power-up allowance (just make sure you use the Chrome extension to display the Card IDs in the previous section).

Once you’ve enabled the Power-up, you just need to authenticate Trello using your Github credentials and you’ll be able to add branches, PRs and commits directly to your Cards as shown here:

I generally attach the PR once the code has been written but will also attach specific commits in some cases e.g. when identifying the commit which introduced a bug.

At the board level, it’s really nice to be able to see which issues have been merged in and you can even use badges to show whether certain checks have been passed e.g. Tests, Linting etc.

The association is two-way too so you can also see a link to the relevant Trello Cards from within Github — a huge timesaver.

There is way more you can do but I’ve found that Card IDs and a nice link between Trello and Github is all that I need right now to keep me moving at pace. Making the process as pain free as possible means that I’m actually sticking to the process and I can move between my issues and code really easily.

Summary

I haven’t introduced anything ground-breaking in this article but that’s because it’s probably a concept we’re all already familiar with from our day jobs. Obviously, you don’t need to finish your side projects — they shouldn’t be a chore or source of stress — but if you’re a little bit fed up of constantly abandoning your projects mid-way through then I really encourage you to formalise your approach a bit more and see how it works out.

Host a secure website in S3 using your own custom domain — Part II

lukeojones — Sun, 17 May 2020 08:17:06 +0000

Fact: Studies show that having a custom domain name for your site makes you fitter, funnier and more interesting at dinner parties.

OK, so like last time — I’m sort of making these up. That being said, there are advantages to setting up a custom domain name e.g. better SEO, quicker AdSense approvals and, more generally, higher trust amongst your visitors. If none of those seem tempting then what about having a more unique email address e.g. luke@connectr.com — maybe this could actually make you more interesting at (really lame) dinner parties? P.S. I don't use that email.

If you hadn’t guessed from the ‘Part II’ in the title, this article is the follow-up to Part I which stepped through how to setup an S3 bucket for static site hosting and details on configuring the CloudFront CDN.

Again, like last time, I’m going to use the .tech domain I purchased from GoDaddy — connectr.tech. I have no major plans for this domain so if you’d like to make me a generous offer to buy it, I’m sure we can come to an agreement 🤑

By the end of this article you will have transferred your custom domain to Amazon’s Route 53 service, setup a custom SSL certificate and made your S3 static side available at your custom domain. Let’s get going.

Part IIA — Transferring your domain name to Amazon Route 53

Route 53 is Amazon’s highly available and scalable DNS service — I’ve dealt with other DNS services before and in my opinion the usability of Route 53 is far superior, especially if you’re primarily building out AWS infrastructure (like we are here).

In this guide, I’ll be migrating away from GoDaddy which is great because, quite frankly, using GoDaddy to manage DNS entries is like trying to drive whilst someone throws vinegar in your eyes.

Login to the AWS console and open up Route 53.
Create a new Hosted Zone and complete the panel on the right by filling in your naked domain name (without the www), an identifying comment and a type of Public Hosted Zone.
Click Create and you should see that a number of DNS records have been created — the NS record lists a number of name servers and the SOA record indicates the host name of the primary name server.
Now we need to export the Zone File (in unix format) from the existing provider (in my case GoDaddy) so that we can import these additional records into the Route 53 record set. Your DNS provider should provide documentation on how to do this so I won’t repeat that here.
Ensure that the A record in your Zone File includes a valid IP (typically the IP of your site) before importing. For example if your website is still ‘Parked’ then you won’t be able to use it until you update the value to an IP — we’ll be changing this record later so don’t worry about the exact value too much.
Once you have your valid Zone File you need to import this into Route 53 using the Import Zone File (visible in the image above). Simply copy the contents of your .txt file into the text field and press Import to import the additional DNS entries.
You now need to repoint the name servers on your existing provider (e.g. GoDaddy) to the new name servers provided in Route 53. You’ll want to check your providers documentation on this but essentially you need to add a name server for each of the four lines in the NS record within Route 53 (e.g. ns-1446.awsdns-52.org. and the other three)

Part IIB — Setting up a new SSL certificate for your custom domain

In order to provide secure access over https using your custom domain, we’ll need to create a new SSL certificate and associate it with our domain. The Amazon Certificate Manager (ACM) makes this quite simple.

Open up the ACM service in the AWS console and change to the us-east-1 region — currently, certificates created in other regions cannot be associated with CloudFront distributions correctly.
Press Request a Certificate and then choose to Request a public certificate in step 1 of the wizard.
You will now need to add your custom domain to the certificate — I would recommend adding both the naked domain e.g. connectr.tech and a wildcard entry for all subdomains (e.g. *.connectr.tech) so that www.connectr.tech and api.connectr.tech will be protected.
Click Next and choose DNS validation. Add a tag if you wish, then click Review and finally Confirm and Request.
Before ACM will provision your certificate, you need to prove you own the domain. Fortunately, because we’re now using Route 53 to manage our DNS records, it’s a simple case of expanding each entry in the validation table and clicking the button Create Record in Route 53.
Once you’ve done this for both domains, press Continue and then you’ll need to wait (up to 30 minutes) for the DNS changes to propagate. Once propagated, ACM will know that you are the true overlord of this domain, and allow you to generate the certificate. Drink a beer or maybe attempt the t-shirt handstand challenge and then come back to check the status.
Eventually, your certificates should move to the Issued status 😎

Part IIC — Configuring CloudFront to use your custom domain and cert

Currently, our CloudFront distribution is associated with the S3 provided endpoint and the default CloudFront certificate — we need to amend the distribution settings to sort this out.

Open up the CloudFront service in the AWS Console and select your distribution from the list. Press Distribution Settings to see the current config.
Press Edit so we can start tweaking things.
Under Alternate Domain Names you will want to add any domains that you want your users to be able to use to access your site (e.g. www.connectr.tech and connectr.tech).
Under SSL Certificate you will need to select the Custom SSL Certificate option and then select the certificate you created in Part IIB.
Save your changes and start getting ready for your next attempt at the t-shirt challenge — we’re almost done.

Add DNS aliases to direct visitors to the CloudFront Distro

Now all we need to do is add a couple of DNS records that will ensure visitors to your site will be directed to the CloudFront distribution.

Make a note of the Domain Name of your CloudFront distribution (e.g. d32gg49xh8c4xx.cloudfront.net) as we’ll need this later.
Swap to the Route 53 service and select the Hosted Zone for your site.
Select the A Record for the naked domain from the existing Record Set. In the Edit Record Set pane on the right, change the entry to Alias: Yes so that we can use a domain name rather than an IP. Change the Alias Target to the domain name of your CloudFront distribution and save the record set.
You will almost certainly want to add another A Record to direct the www subdomain to the same CloudFront distribution using an Alias. You can do this by pressing Create Record Set at the top of the page and then following the exact same instructions as you did for the naked domain. In some cases you may first need to remove the CNAME entry with the same name.

Great, only 27 more steps to go…only joking — that’s it. You may need to wait a while for some of the DNS settings to propagate but pretty soon you should be able to visit your new domain name and see your site, served over https using your custom SSL certificate! 😎

Thanks for following along and well done for persevering. If you managed to perfect the t-shirt-handstand challenge whilst reading through this then you have my utmost respect and admiration — if you didn’t then at least you’ve got your custom domain up and running!

Next time, I’ll be looking at how you can use a couple of free tools to setup a CI/CD pipeline for a simple React site 👍

Host a secure website in S3 using your own custom domain — Part I

lukeojones — Sun, 17 May 2020 06:27:47 +0000

Fact: Studies show that developers hosting their sites in S3 are healthier, wealthier and have better hair than their peers.

If you weren’t convinced by the above then at least know that the combination of Amazon S3 and CloudFront offers a simple, cheap and performant way to host static sites and serve them over https. In this two-part series, we’ll spend a little time going over the steps required to get your site live and using a custom domain. We’ll cover the following:

Part I — Configuring an S3 bucket and CloudFront distribution to host and deliver your content.
Part II — Transferring your domain to Amazon Route 53 and configuring an SSL certificate using Amazon Certificate Manager.

Each part will be pretty short — so don’t be put off by the breakdown above — the whole series can be completed in well under an hour.

For the purposes of this guide, I’m going to use a .tech domain I purchased from GoDaddy. I decided to go with connectr.tech as it was cheap, relatively non-specific and sufficiently hipster to be used in a dev.to article 🧐

Part IA— Creating and configuring an S3 bucket to host your site

Login to the AWS console and navigate to the S3 bucket creation page.
Create a new bucket e.g. connectr-site and temporarily remove all public-access blocks as shown below
Select your newly created bucket and then head to Properties : Static Website Hosting to configure the static hosting options.
Configure the index and error pages as index.html and error.html. You’ll also see the public endpoint of your soon-to-be site!
Copy the endpoint and open it up in another tab — you should receive a 403 (forbidden) response because the bucket policy has not yet been configured. To sort this out, go to the Permissions tab (next to Properties) and add the following Bucket Policy using the Bucket Policy Editor.
Save the bucket policy and try hitting your new public endpoint again — this time you should receive a 404 as S3 tries to serve the non-existent index.html and error.html pages. This is what we want at this stage 👍
Create an index.html and error.html file that you are happy to share with the world. For example, I created some ‘awesome’ pages below…
Whilst still in S3, select the Overview tab and then upload the error.html file to your S3 bucket. Make sure you provide public read access on the second page of the upload wizard before pressing Upload.
Once the upload is finished, navigate to your site’s endpoint again and your custom error page should be displayed.
Repeat the process for your index.html — when you navigate to the site again, you should see your lovingly crafted html! (notice the connection is plain http and not secure)
Finally, to improve security and make sure we’re following best practices, we can lock down the bucket policy a little more. Head back to the Permissions tab for your bucket and block public access for ACLs and new bucket policies as shown:
Double check you can still access your index.html page and that’s it — you’re good to go.

Part IB— Configuring a CloudFront distribution for content delivery

To improve how your users access your content, you’ll want to make use of a CDN — in this case CloudFront — to distribute your content using a global network of proxy servers.

In AWS, navigate to the CloudFront service and select Create Distribution to get started. When asked to choose a delivery method, choose Web.
Under Origin Settings you should attach the S3 domain name that was created in Part IA e.g. connectr-site.s3.amazonaws.com
Secondly, you’ll want to redirect all http traffic over https by selecting Redirect HTTP to HTTPS as the Viewer Protocol Policy option
Now set the Default Root Object to index.html (or whatever site root you may have decided to configure).
Now, create the distribution and wait for it to show the status Deployed in the Distributions overview table (this can take 5–10 minutes so find a cat video on Youtube).
Copy the CloudFront domain name e.g. d32gg49xh8c4xx.cloudfront.net (catchy right?) and paste it into a new browser tab. If everything has gone to plan — you should see your lovingly crafted html again — this time served over the CloudFront CDN and transferred using the https protocol!

And that’s it — you’re done for Part I. Hopefully that was relatively painless and you were able to follow along at a decent pace.

In Part II, I’ll go through the process of transferring your custom domain to Route 53, creating an individual SSL certificate and associating these both with the CloudFront distribution we’ve just created here.

Good work! Grab a refreshing drink and smile smugly for a few minutes…