Forem: Shiraaz Moollatjie

A journey with good old base64

Shiraaz Moollatjie — Tue, 06 Oct 2020 11:35:08 +0000

I did some work around base64 in go. I found it pretty interesting and thought that it would make a nice article to write about

What is base64?

Base64 is a binary to text encoding scheme that represents binary data in plain ASCII format. It's designed to give your application a way of transferring binary data across text protocols. The technical specification for base64 can be found at RFC 4648

Common use cases

The most common use cases for base64 are:

Embedding images in html pages. The idea is to reduce the number of server roundtrips made for your website as well as reduce the number of dependencies on your web page
Returning binary data (such as images) in JSON or XML apis
When using HTTP basic authentication, the username/password combination is encoded into base64 format
Saving binary files into your database if a BLOB/CLOB data type is not supported

Leave a comment if you used base64 for anything!

Let's encode in go

You can follow the examples on your local pc using the associated go playground

What can we use?

Go has base64 support available under the encoding/base64 package. It is largely driven by an Encoding type with four default encodings available:

StdEncoding
RawStdEncoding
URLEncoding
RawURLEncoding

For this post, I will not go into the URLEncoding and RawURLEncoding types.

Coding examples - Simple encoding & decoding

We'll start our examples with some base text that we want to encode:

Programs must be written for people to read, and only incidentally for machines to execute.

You can copy & paste this into an online base64 encoder/decoder for external verification. To encode some text, we can simply do the following:

package main

import (
    "encoding/base64"
    "fmt"
    "log"
)

func main() {
    str := "Programs must be written for people to read, and only incidentally for machines to execute."

    encStd := base64.StdEncoding.EncodeToString([]byte(str))
    encRawStd := base64.RawStdEncoding.EncodeToString([]byte(str))

    decStd, err := base64.StdEncoding.DecodeString(encStd)
    if err != nil {
        log.Fatalf("code better for blog posts: %v", err)
    }

    decRawStd, err := base64.RawStdEncoding.DecodeString(encRawStd)
    if err != nil {
        log.Fatalf("code better for blog posts: %v", err)
    }

    fmt.Printf("original string:     %s\n", str)
    fmt.Printf("encoded  string:     %s\n", encStd)
    fmt.Printf("raw encoded  string: %s\n\n", encRawStd)
    fmt.Printf("decoded  string:     %s\n", decRawStd)
    fmt.Printf("raw decoded  string: %s\n", decStd)
}

This gives you the output:

original string:     Programs must be written for people to read, and only incidentally for machines to execute.
encoded  string:     UHJvZ3JhbXMgbXVzdCBiZSB3cml0dGVuIGZvciBwZW9wbGUgdG8gcmVhZCwgYW5kIG9ubHkgaW5jaWRlbnRhbGx5IGZvciBtYWNoaW5lcyB0byBleGVjdXRlLg==
raw encoded  string: UHJvZ3JhbXMgbXVzdCBiZSB3cml0dGVuIGZvciBwZW9wbGUgdG8gcmVhZCwgYW5kIG9ubHkgaW5jaWRlbnRhbGx5IGZvciBtYWNoaW5lcyB0byBleGVjdXRlLg

decoded  string:     Programs must be written for people to read, and only incidentally for machines to execute.
raw decoded  string: Programs must be written for people to read, and only incidentally for machines to execute.

Comments

There is a subtle difference between RawStdEncoding and StdEncoding and that would be the presence of padding characters. The StdEcoding type includes padding characters in its output. The default padding character for base64 is = and it is customizable.

The two encoded formats are also not interchangeable. So adding these snippets will cause errors during decoding (and encoding):

decStd, err = base64.StdEncoding.DecodeString(encRawStd)
    if err != nil {
        fmt.Printf("cannot use a raw encoded base64 string for standard decoding: %v\n", err)
    }

decRawStd, err = base64.RawStdEncoding.DecodeString(encStd)
    if err != nil {
        fmt.Printf("cannot use a standard encoded base64 string for raw decoding: %v\n", err)
    }

Let's do some slightly more complex encoding

In this next example, let us try to base64 encode an image. This program will write the go logo to stdout in base64 format.

package main

import (
    "encoding/base64"
    "io/ioutil"
    "log"
    "net/http"
    "os"
)

func main() {
  // Encode a base64 image. Using a URL because it's simpler than using a file.
    resp, err := http.Get("https://golang.org/lib/godoc/images/go-logo-blue.svg")
    if err != nil {
        log.Fatalf("code better for blog posts: %v", err)
    }

    b, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatalf("code better for blog posts: %v", err)
    }

    encoder := base64.NewEncoder(base64.StdEncoding, os.Stdout)
    defer encoder.Close()

    encoder.Write(b)
}

This yields the following output (viewable on an online base64 decoder)

PHN2ZyBoZWlnaHQ9Ijc4IiB2aWV3Qm94PSIwIDAgMjA3IDc4IiB3aWR0aD0iMjA3IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjxnIGZpbGw9IiMwMGFjZDciIGZpbGwtcnVsZT0iZXZlbm9kZCI+PHBhdGggZD0ibTE2LjIgMjQuMWMtLjQgMC0uNS0uMi0uMy0uNWwyLjEtMi43Yy4yLS4zLjctLjUgMS4xLS41aDM1LjdjLjQgMCAuNS4zLjMuNmwtMS43IDIuNmMtLjIuMy0uNy42LTEgLjZ6Ii8+PHBhdGggZD0ibTEuMSAzMy4zYy0uNCAwLS41LS4yLS4zLS41bDIuMS0yLjdjLjItLjMuNy0uNSAxLjEtLjVoNDUuNmMuNCAwIC42LjMuNS42bC0uOCAyLjRjLS4xLjQtLjUuNi0uOS42eiIvPjxwYXRoIGQ9Im0yNS4zIDQyLjVjLS40IDAtLjUtLjMtLjMtLjZsMS40LTIuNWMuMi0uMy42LS42IDEtLjZoMjBjLjQgMCAuNi4zLjYuN2wtLjIgMi40YzAgLjQtLjQuNy0uNy43eiIvPjxnIHRyYW5zZm9ybT0idHJhbnNsYXRlKDU1KSI+PHBhdGggZD0ibTc0LjEgMjIuM2MtNi4zIDEuNi0xMC42IDIuOC0xNi44IDQuNC0xLjUuNC0xLjYuNS0yLjktMS0xLjUtMS43LTIuNi0yLjgtNC43LTMuOC02LjMtMy4xLTEyLjQtMi4yLTE4LjEgMS41LTYuOCA0LjQtMTAuMyAxMC45LTEwLjIgMTkgLjEgOCA1LjYgMTQuNiAxMy41IDE1LjcgNi44LjkgMTIuNS0xLjUgMTctNi42LjktMS4xIDEuNy0yLjMgMi43LTMuNy0zLjYgMC04LjEgMC0xOS4zIDAtMi4xIDAtMi42LTEuMy0xLjktMyAxLjMtMy4xIDMuNy04LjMgNS4xLTEwLjkuMy0uNiAxLTEuNiAyLjUtMS42aDM2LjRjLS4yIDIuNy0uMiA1LjQtLjYgOC4xLTEuMSA3LjItMy44IDEzLjgtOC4yIDE5LjYtNy4yIDkuNS0xNi42IDE1LjQtMjguNSAxNy05LjggMS4zLTE4LjktLjYtMjYuOS02LjYtNy40LTUuNi0xMS42LTEzLTEyLjctMjIuMi0xLjMtMTAuOSAxLjktMjAuNyA4LjUtMjkuMyA3LjEtOS4zIDE2LjUtMTUuMiAyOC0xNy4zIDkuNC0xLjcgMTguNC0uNiAyNi41IDQuOSA1LjMgMy41IDkuMSA4LjMgMTEuNiAxNC4xLjYuOS4yIDEuNC0xIDEuN3oiLz48cGF0aCBkPSJtMTA3LjIgNzcuNmMtOS4xLS4yLTE3LjQtMi44LTI0LjQtOC44LTUuOS01LjEtOS42LTExLjYtMTAuOC0xOS4zLTEuOC0xMS4zIDEuMy0yMS4zIDguMS0zMC4yIDcuMy05LjYgMTYuMS0xNC42IDI4LTE2LjcgMTAuMi0xLjggMTkuOC0uOCAyOC41IDUuMSA3LjkgNS40IDEyLjggMTIuNyAxNC4xIDIyLjMgMS43IDEzLjUtMi4yIDI0LjUtMTEuNSAzMy45LTYuNiA2LjctMTQuNyAxMC45LTI0IDEyLjgtMi43LjUtNS40LjYtOCAuOXptMjMuOC00MC40Yy0uMS0xLjMtLjEtMi4zLS4zLTMuMy0xLjgtOS45LTEwLjktMTUuNS0yMC40LTEzLjMtOS4zIDIuMS0xNS4zIDgtMTcuNSAxNy40LTEuOCA3LjggMiAxNS43IDkuMiAxOC45IDUuNSAyLjQgMTEgMi4xIDE2LjMtLjYgNy45LTQuMSAxMi4yLTEwLjUgMTIuNy0xOS4xeiIgZmlsbC1ydWxlPSJub256ZXJvIi8+PC9nPjwvZz48L3N2Zz4=

When base64 is not recommended

Base64 is of course not a silver bullet. There are cases where base64 is not recommended.

Do not use base64 for storing passwords

It is highly recommended to NOT use abse64 for storing passwords. It seems easy enough to encode data into a format that's hard to read for humans. However, base64 is birectional, so whatever is encoded can be easily decoded. This highlights an important distinction that base64 is an encoding scheme, not an encryption scheme.

Do not replace all your website images with base64 variants

While it is possible to convert your images to base64, it comes with the following tradeoffs:

The base64 version of your image is larger (heuristically ~30% larger)
You lose the ability to cache images on the client side

Summary

In this post, we wrote some simple go programs to show off its capabilities. These snippets could be used as the base of your own applications.

Base64 is indeed a neat piece of computer science research. However, it is not a silver bullet and there are pros and cons when using the encoding in your application. Always use the best tool for the job.

Gophorem - A go API client for forem

Shiraaz Moollatjie — Mon, 17 Aug 2020 19:39:59 +0000

This is a post about gophorem. Where gophers and forem meet to build communities.

I love to use go and I love the idea of forem. I also noticed that forem
does not have any go client support. So I wrote one and it's called gophorem. It's awesome.

ShiraazMoollatjie / gophorem

Where gophers and forem meet to build communities

🤔 What is gophorem

In reality, it's a REST API client for the entire forem API written in go. I
prefer to call it the place where gophers and forem meet up to build communities.

🤷 What can you do with it?

Many things! Soon I'll be posting about how I used gophorem to do some data
visualization with dev.to data. Essentially, the entire forem API is supported
by gophorem. This means that you can:

Use this on ANY forem
Manage articles
Manage listings
Manage webhooks
List podcasts, video articles

⌨️ How do you use it?

Gophorem is pretty simple to use. There are also examples to help you get started too.
This is an example to retrieve the latest go articles:

func main() {
    cl := gophorem.NewDevtoClient(gophorem.WithAPIKey("MY_API_KEY"))
    ctx := context.Background()

    // Retrieve all the go articles.
    al, err := cl.Articles(ctx, gophorem.Arguments{
        "tag": "go",
    })
    if err != nil {
        log.Fatalf("something went wrong: %+v", err)
    }

    fmt.Printf("All Articles: %+v", al)
}

🌧️ It supports streaming too!

It's also possible to stream articles with this package. This makes it
useful for certain use cases. It's still a work in progress, but the idea is
to support all the listing endpoints as streams.

This is an example (ripped from the examples folder) of how to stream all the recent go articles on dev.to:

func main() {
    cl := gophorem.NewDevtoClient(gophorem.WithAPIKey("MY_API_KEY"))
    ctx := context.Background()

    s := streamer.NewStreamer(cl)

    ch := s.Articles(ctx, gophorem.Arguments{
        "state": "fresh",
    })

    for a := range ch {
        fmt.Printf("Received article ID: %d, Title: %s, Username: %s URL: %s \n", a.ID, a.Title, a.User.Username, a.URL)
    }
}

🤖 Future work and contributions

Since the API is largely complete, I was thinking of adding the following features:

Finalize the streaming features
Some rate-limiting features when querying forem servers
Support oauth authentication
Some showcase articles for what you're able to do. There is a lot of potential in this package

That's all for now, check out the package and have a go at it. All contributions are welcome. Have fun!!

Devtogo update

Shiraaz Moollatjie — Tue, 04 Aug 2020 12:03:25 +0000

devtogo happens to be my first open-source API integration and I have a soft spot for dev.to ♥️.

devtogo is a Go API client library for dev.to. I made a huge update to this in the past week that I would like to share.

But first, let me do my shameless plug. Always accepting contributions!

ShiraazMoollatjie / devtogo

A REST API Wrapper for the dev.to api in go

Fully complete client

The major update is that devtogo is a fully complete and supports 100% of the API features 🎉. This means that you should now be able to:

Do everything with articles
View comments
View tags
Create, update and view listings
View podcast-episodes
View your user profile information
View video articles
View webhooks

This is a simple example that you can use as a base for a dev.to scraper.

package main

import (
    "fmt"
    "log"

    "github.com/ShiraazMoollatjie/devtogo"
)

func main() {
    cl := devtogo.NewClient(devtogo.WithApiKey("YOUR_API_KEY_HERE"))
    al, err := cl.Articles(devtogo.Defaults())
    if err != nil {
        log.Fatalf("something went wrong: %+v", err)
    }

    fmt.Printf("All Articles: %+v", al)
}

Milestones

These are important milestones that happened in this project. It might not be reflective of a successful project, but they did hit and I feel proud of them

🎉 Someone contributed

When I initially did my launch post, I received a much higher response than what I thought I'd get (more than zero LOL). Someone made a PR which is more valuable to me than the project itself.

It felt great to get someone else to contribute. I started this project to learn more about go, so I appreciate it when other gophers tag along.

⭐ Engineers starred it

Again, didn't expect this. Normally I leave my code on GitHub and at best at least one person will star it. This one got 5 stars, not much at all, but surprising to me.

Roadmap ahead

Two themes are going forward that I feel needs to be done for this project.

Fork for forem

The devtogo client needs to align with dev.to's new vision of forem. I think that with a little bit of commits, this project should be able to work on any forem based site. That is super cool!

Use more Go features

devtogo is one of my first Go open source projects, and in some places it shows LOL. The idea is to use more go features such as:

The use of contexts for HTTP calls
Incorporating channels for certain use cases
Maybe add optional metric tracking

Have more examples

There are some pretty interesting things that you can do with the forem/dev.to api. I would want to showcase these in the future.

Shoutout to the dev.to team

A big thanks for the dev.to team for giving me a platform to write about tech. I'm a big lurker, sometimes contributor and the community is so amazing. It always happens that when you think the platform peaked, then there's some innovation that happens which is super cool.

Decent ways of dealing with bad posture during lockdown that's other than getting a good chair?

Shiraaz Moollatjie — Thu, 09 Jul 2020 22:40:30 +0000

Been to the doctor complaining about chest pains. Thought it was coronavirus or worse, but turns out to be inflammation caused by bad posture (Yay and nay!!!).

So the easy mode way to deal with this is to get a decent office chair for home. But to keep the discussion interesting, what are some other good posture tips that are beyond getting a good chair?

Tips for solo coronavirus contributors to help your websites and apps stand out

Shiraaz Moollatjie — Mon, 01 Jun 2020 18:55:01 +0000

Intro

This article has rants and some lessons. If you're facing a similar struggle, please comment on this post. I'm not aware of others going through the same kind of pain.

When the world took the coronavirus pandemic seriously, I wanted to contribute as an engineer. Inspired by https://ncov2019.live/, I released two projects:

Coronawatcher.live

A website that lets you compare coronavirus statistics between countries. It also lets you compare the bitcoin price to the coronavirus statistics.

Aunty Rona

A website for South Africa that makes it easier to interpret the lockdown rules.

Things that work for you, now all of a sudden work against you

For my projects, I targeted a mix of web and mobile. There are some huge obstacles that you encounter along the way.

SEO woes

One of the activities you do when developing a web app is to optimize the content for search engines. Typically, a keyword planner is one of the tools used for SEO optimization so that a web app can rank better on search engines.

The lead time for SEO optimizations to reflect on a new site is around six months. However, I noticed something during keyword planning.
Google does not allow keyword plans for anything related to the coronavirus. This meant that Google was not allowing ads for "coronavirus" keywords.

It may also mean that Google was censoring themed coronavirus web apps in their search engine. The implications for this are:

A new web app will not be able to rank on the first page after six months
Websites/web apps currently on the first page are biased in some way

What annoys me is the possibility of censorship. Web sites in this case will rank because of their timeliness and not because of the quality of content. Even though there are many fake news sites, a website is associated with fake news when it starts. This creates the impression that you are guilty until proven innocent.

Play and app store woes

Mobile app stores will not publish a coronavirus themed application unless it is supported by another organization. As proof, this is an excerpt of one of my appeals for the Aunty Rona mobile app:

For example, your app currently contains references to Covid-19.
If you are one of the following, Please submit valid proof to verify the identity of your organization.

Official government entities or public health organizations who developed this app.

A developer commissioned to build this app by official government entities or public health organizations.

An app supported/acknowledged by official government entities or public health organizations.

The situation looks bleak. Is there a way out?

My situation looked bleak, but there was still hope. Here are a few more ideas that I had to try to get noticed.

If mobile stores are not willing to publish your app, consider turning your native app into a progressive web app (PWA)

A progressive web app is a solution for apps that cannot get published. This allows your app to be installable like a mobile app and depending on your implementation it can behave like a mobile app as well. The disadvantage is that a progressive web app will not benefit from being published on a mobile app store where users can "search" for your app.

Using associative content to take advantage of keywords

One of the ways to take advantage of keywords is to combine your content with other themes. You can't get ranked with coronavirus keywords. However, if you associated the coronavirus statistics with the toilet paper price, could you then rank better with "toilet paper" instead?

We were desperate and we did this to coronawatcher. We started associating the coronavirus statistics to the bitcoin price. I found the results interesting. As a teaser, there is one country that showed an inverse trend between the bitcoin price and coronavirus infections.

While I am not ranking on the first page for bitcoin, website traffic did see an improvement. Additionally, the bitcoin price did increase ever since I implemented the feature. It's most likely a coincidence, but is it?

Hashtags

Instead of relying on keywords for the website, I used hashtags on other social media platforms. These ranked better on search engines for whatever reason.

As an example, I used the "#coronawatcher" hashtag for coronawatcher. These hashtags appear on the first page when you search for "coronawatcher".

Get social media handles early and post on them

Create and own the Instagram, Twitter, and Facebook accounts for your website. These will help in featuring on the first page in search engines. It will also help with occupying more space on the first page.

As an example, the coronawatcher Facebook page features on the first page for "coronawatcher".

There are other platforms that you can use like dev.to and hackernews. I did not focus on these platforms. However, DO NOT bother with posting on Reddit. Reddit does not allow users to advertise their content.

Paying for ads on places other than Google

My coworker on the coronawatcher project also experimented with paid advertising. We ran a campaign against Facebook users. It resulted in a significant traffic spike. I was blown away with the results. I did not know that advertising was that effective.

Conclusion

The projects were not as successful as I wanted it to be, but I learned a lot of things. I kept the ranting to a minimum. Sometimes I feel that too much ranting is petty. So I keep it to a minimum. In the end, I hope you enjoyed this post.

References

Official progressive web app site

What is considered a good career in tech?

Shiraaz Moollatjie — Wed, 19 Feb 2020 23:17:26 +0000

I remembered a discussion with a friend of mine that had us talking about careers. Two questions came up:

What is considered a good career in tech?
How do you want your career to look like when you're "done"?

I feel that it's good to think ahead and reassess this from time to time.

I've seen the definition of a good career change many times on social media. From "working for a big tech company" all the way to "owning my own company" to "being an established writer/speaker/author". It varies from person to person.

So two points up for discussion. Let's see what we think.

One new language every year

Shiraaz Moollatjie — Thu, 02 Jan 2020 19:22:26 +0000

This post talks about a point raised in the pragmatic programmer book. It is to strive to learn one new language each year. We will also take a look at how this can be taken out of context and the anti patterns that comes from this. I think that with a little bit of realism, this is something that all engineers can strive for and can reap the rewards.

This is a short post about using the upcoming year to learn a new language. This definition can be extended a little bit to include learning a new platform, OS or a new framework. For the rest of this post, we will just refer to a new language, but it is interchangeable with platform/OS/framework.

The Intent

The intent behind learning a new language stems from the Pragmatic Programmer. The idea is that our programming language implicitly teaches you to solve a problem in a certain way. So by sticking with one language, we are narrowing our field of vision with regard to solving problems. I believe it's an unintended side effect, but it exists.

Learning a new programming language allows us to relook at existing problems and approach them differently. These lessons can then be brought into our main language and we eventually become more creative and effective at problem solving.

Choosing something new to learn

The way we choose of learning something new is very important. Generally, we shouldn't choose something just because it's similar to our existing language. We do not choose to learn C# if our main language was Java as an example. Nor do we learn Python if our main language was Javascript/Typescript for example. There are edge cases where a switch between similar toolsets makes sense. An example here is a Java engineer that wants to learn Unity has to inevitably learn C#.

In the general case, however, we want to be able to choose something that will force a change in our thinking. A nice way to approach this is to switch to a language with a different programming paradigm. So a switch from an object oriented to a functional programming/scripting language makes sense and so on.

The learning hurdle

This is the area where the least amount of advice is given to engineers. Sure, it's nice to talk about this in theory and market about how it will improve your life. At the same time, this is where engineers run into the most nastiest anti-patterns, so let's talk about success and failure.

What success looks like

Funnily enough, success is subjective and is limited to your availability. Let's be real for a minute, learning a new language is a huge undertaking because of the sacrifices needed to do this. Exponentially so if you're in a situation where you have responsibilities such as raising kids, looking after elderly parents or anything that takes up your time. These responsibilities are always more important.

So realistically, what does success look like? It is probably best to take an incremental approach to success. Starting with the end goal, we work our way down to the next acceptable form of success. So this is one form of a success ladder:

- Have a profit yielding app in production
- Have an app in production
- Have an app that works on my local environment
- I wrote X blog posts about the language
- I can solve interview questions with the language
- I got X points on stackoverflow from answering questions
- I can write small programs with the language
- I can write hello world

Each person will have different forms of success with their new language. The main point is that they must be achievable.

Anti patterns for success

There are two main anti patterns that plague engineers when learning a new language. The one is setting up impossible goals. The other is destroying your work life balance for the goal.

Sometimes engineers set the loftiest, most unachievable goals for themselves. For example, we set a single goal like this:

Learn node.js and use it to build a game server for a multiplayer, 
browser-based game by mid 2020.

There are many problems with this goal. Does the engineer know what game they want to build? Why does it have to be done by mid 2020? Do they have experience in multiplayer game design? How do you know you're done with "learn node.js"?

Always remember that the goal is always to change our way of thinking, not to place extra burden on ourselves. The way out is to redo something you've already done in a new language. This way, you don't have anything extra to learn and you're able to focus just on the language. So reimplement an existing feature instead. Build up towards a small rewrite of your professional work application instead. This is much easier and more approachable than the big bang golden goal approach.

The other anti pattern is actually a side effect of the golden goal. When setting such a huge goal for yourself, sometimes engineers put pressure on themselves and pull off the big all nighters just to get the goal done. This decreases your quality of life and your main priorities suffer. Smaller goals helps you prioritise and backlog your goals. You'll also know ahead of time what's achievable and what isn't. Always remember that this is a hobbyist goal and should not decrease your quality of life nor add extra stress. If you can't finish it all, that's okay!

Conclusion

We looked at how we can use the upcoming year to learn a new programming language. We had a small look into why we want to do this. We also thought about how to choose something new to learn, what success might look like and probably more importantly, the anti patterns involved in learning a new language each year. Hope you enjoyed this. All the best for the upcoming year!

goluhn: A simple package for generating, calculating and verifying luhn numbers

Shiraaz Moollatjie — Thu, 21 Nov 2019 14:09:52 +0000

This short article will introduce you to goluhn, a very simple go package to apply luhn numbers to your projects.

What is a luhn number

A luhn number is a number whose last digit satisfies the luhn checksum formula. It is named after its inventor Hans Peter Luhn. For example, in the luhn number 1111 2222 3333 4444, the last 4 digit is the luhn checksum digit of the number.

Intended use and application

The most popular use of a luhn number is a credit card number. Every credit card number satisfies the luhn algorithm. Luhn numbers are also used for other identification methods such as IMEI numbers, electricity meter numbers, various countries' ID numbers and so on.

The luhn check is mainly designed to protect against accidentally entered numbers, basically typos. It is not designed to protect against malicious checks. Therefore validation of these number is the main use case.

Using `goluhn`

Using goluhn is extremely simple. It exposes three functions Validate, Calculate and Generate.

ShiraazMoollatjie / goluhn

A package that implements the luhn algorithm for validation, calculation and generation

To kick things off, we start with the package import:

import "github.com/ShiraazMoollatjie/goluhn"

Validate

We call the Validate function to validate whether a provided string is a luhn number. Invalid luhn strings return errors.

err := goluhn.Validate("1111222233334444")
if err != nil {
  return err
}

Calculation

This is used when we have a numeric string whose luhn check digit needs to be calculated.

cd, n, err := goluhn.Calculate("111122223333444")
if err != nil {
  return err
}

fmt.Printf("Check digit: %s\n", cd)
fmt.Printf("Luhn number: %s\n", n)

This function will return the check digit as well as the original with the luhn digit appended to it.

Generate

The Generate function will generate a random luhn number of a provided length. This is useful for when you are generating your own numbers in your system. In the example below, we generate luhn numbers of length 16 (effectively a credit card number).

n, err := goluhn.Generate(16)
if err != nil {
  return err
}

fmt.Printf("Luhn number: %s\n", n)

Random Thoughts

This project was quick and fun to implement. I decided to use strings over integers because it is possible to generate luhn numbers for numbers that are greater than the int64 type (this is fun to try out actually). There are no real numeric operations that apply to luhn numbers, so this also motivated me to use strings instead. Strings also allow for zero padded prefixed numbers to be generated.

The Validate function arguably could have returned a bool, error result. I don't really like returning that specific combination of types because either you have an error or the result of the function is free from errors which implies true. Additionally, I think that if you returned a boolean false in this specific case, you're very likely to return an error too.

Conclusion

In this article, we learnt how to use goluhn to generate luhn numbers. We also delved a bit into some of the design decisions. As always, contributions are always welcomed!

References

The wikipedia entry

Representing enums in go

Shiraaz Moollatjie — Wed, 13 Nov 2019 06:50:33 +0000

This article will look at how to represent enums in go. It will also add some practical advice as well.

An enum is a datatype that consists of a set of named values. Many languages use the enum keyword to represent enum types.

Use `iota` to define enums

According to the Go specification, iota is a keyword that represents successive untyped integer constants as long as those constants are in the same constant declaration.

For example, we can define the following set of constants from c0 -> c2:

const (
    c0 = iota  // c0 == 0
    c1 = iota  // c1 == 1
    c2 = iota  // c2 == 2
)

We can reduce the amount of code written by removing the excess iota declarations:

const (
    c0 = iota
    c1
    c2
)

The iota keyword also lets us define specific patterns with your constants, but in this article we are only focusing on enums.

Applying this in practice

So let us apply this in practice. We will use the example of card suites. So we define a suite of cards:

type Suite int

const (
  unknown Suite = iota
  Spades
  Hearts
  Diamond
  Clubs
  sentinel
)

Because our eyes are so sharp, you'd notice that I added two new values to this list: An unknown and a sentinel value. Let us work through their value.

Using `unknown` and `sentinel` values

Within go, generally we want our enum type to be able to support zero values. This makes it easier to write idiomatic go without worrying about processing business logic for zero values.

There are cases where zero values needs to be supported (e.g. for some externally defined protocol or even legacy code), but within our own application code we try to prevent it.

The purpose of the sentinel is to make it easy to iterate and validate our list of enum values. It also adds to the maintainability of our enums.

For example, because Suite is it's own type, we can in theory assign it any value. So let's try to validate that an integer is within our list of enums:

func (s Suite) isValid() bool {
  return  s > unknown && s < sentinel
}

As you can see, our validation code is agnostic of the specific values of our suites. This means that we are able to add values to our enums as long as they appear before our sentinel value.

Deprecating enums?

If your system is small, then this wouldn't really apply. However, suppose you do not want to support certain types for whatever reason. Go actually makes this very easy by using _ values. So let's say that we only want to use the black Suites, but we need to keep the red Suites for historical purposes. We simply do this:

type Suite int

const (
  unknown Suite = iota
  Spades
  _
  _
  Clubs
  sentinel
)

Using this way, we are maintaining the Clubs data throughout the lifetime of our application, and we will be able to catch all the usages of the red Suite values during refactoring.

Conclusion

In this post we looked at how to define enums in go using the iota keyword with our constants. We also recommended the use of an unknown value and a sentinel value to make application programming easier. We even took a look at possibly deprecating enum values in our application.

Efficiently concatenating strings in go

Shiraaz Moollatjie — Tue, 05 Nov 2019 08:00:12 +0000

In this article, we will look at different ways to concatenate strings in Go.

What happens when you concatenate strings with the `+`operator

Using the + is good enough for pieces of code that are not critical or when there are not many strings to join.

However, when we have many strings to join (for whatever reason), then performance starts to degrade.

Using `bytes.Buffer`

Using bytes.Buffer is one of two efficient ways to handle many string concatenations. We define a bytes.Buffer type and append to this type until we are done with concatenation.

A very simple example of bytes.Buffer looks like:

var s bytes.Buffer
for i := 0; i < 100; i++  {
  s.WriteString("somestring")
}
log.Printf("A super long string: %s", s.String())

Using `strings.Builder`

From Go 1.10 a strings.Builder type was added. It's designed for string concatenation and minimizes memory copying. This is the preferred method for implementing efficient string concatenation.

A simple example of strings.Builder looks like:

var s strings.Builder
for i := 0; i < concats; i++  {
  s.WriteString("somestring")
}
log.Printf("A super long string: %s", s.String())

Let's measure some performance!

In order to test the differences between the three methods, we can write a program that can measure some times for a given concatenation count. Thereafter, we'll be able to visualize some performance.

This is not meant to be a very strict performance benchmark. It's more to give an indicator on performance.

Benchmark test

If you happen to be unfamiliar with Go benchmarks, take a look at the testing package godoc for a really nice explanation on how it works.

Our benchmark test looks like this:

package concats

import (
    "bytes"
    "strings"
    "testing"
)

func BenchmarkStringConcatenation(b *testing.B) {
    benchmarks := []struct {
        name   string
        testFn func(int)
        param  int
    }{
        {"Regular 50", regularConcatentation, 50},
        {"Regular 100", regularConcatentation, 100},
        {"bytes.Buffer 50", bytesBuffer, 100},
        {"bytes.Buffer 100", bytesBuffer, 100},
        {"strings.Builder 50", stringBuilder, 100},
        {"strings.Builder 100", stringBuilder, 100},
    }
    for _, bm := range benchmarks {
        b.Run(bm.name, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                bm.testFn(bm.param)
            }
        })
    }
}

func BenchmarkRegularConcatenation40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        regularConcatentation(40)
    }
}

func BenchmarkBytesBuffer40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        bytesBuffer(40)
    }
}

func BenchmarkStringBuilder40(b *testing.B) {
    for i := 0; i < b.N; i++ {
        stringBuilder(40)
    }
}

func bytesBuffer(concats int) {
    var s bytes.Buffer
    for i := 0; i < concats; i++ {
        s.WriteString("somestring")
    }
}

func regularConcatentation(concats int) {
    s := ""
    for i := 0; i < concats; i++ {
        s += "somestring"
    }
}

func stringBuilder(concats int) {
    var s strings.Builder
    for i := 0; i < concats; i++ {
        s.WriteString("somestring")
    }
}

Results

I do think that our benchmark results will be different on our individual machines. After running these benchmarks, we get the following output:

goos: darwin
goarch: amd64
pkg: concats
BenchmarkStringConcatenation/Regular_50-4             300000          4542 ns/op
BenchmarkStringConcatenation/Regular_100-4            100000         15084 ns/op
BenchmarkStringConcatenation/bytes.Buffer_50-4       1000000          1585 ns/op
BenchmarkStringConcatenation/bytes.Buffer_100-4      1000000          1561 ns/op
BenchmarkStringConcatenation/strings.Builder_50-4    2000000           733 ns/op
BenchmarkStringConcatenation/strings.Builder_100-4   2000000           735 ns/op
PASS

Looking at these results, string concatentation is much slower than both bytes.Buffer and strings.Builder. This will be consistent across different machines. When using strings.Builder, it runs at about twice as fast as bytes.Buffer on my machine. I do think that the time differences between these two styles will vary between machines.

Conclusion

In this post, we learnt about the different approaches to mass string concatenation in Go. We also managed to benchmark these techniques and briefly discussed the results.

Introducing devtogo: A wrapper for the dev.to api written in Go

Shiraaz Moollatjie — Mon, 28 Oct 2019 09:57:28 +0000

This post will introduce you to devtogo and will show you some usage examples.

I've been using Go as my main programming language for about six months at work. It's been very enjoyable and I always learn something new every day.

I wrote devtogo as a means for me to learn how to write some good opensource Go code. It's also my very first opensource project that I am publishing. Much of my work is sitting in private repositories, so I thought that I may as well expose this for the world to use.

Try it out and contribute/interact with the Github repo.

ShiraazMoollatjie / devtogo

A REST API Wrapper for the dev.to api in go

devto-go

devto-go is a REST API Wrapper for the dev.to api written in go.

Usage

Import the package into your go file:

import "github.com/ShiraazMoollatjie/devtogo"

Thereafter, create a client and specify your API token:

cl := devtogo.NewClient(devtogo.WithApiKey("MY_API_KEY"))

It is also possible to not use an API key for anonymous operations.

Retrieving articles

To retrieve a list of articles, simply use the Articles function:

articles, err := cl.Articles(devtogo.Defaults())

It is also possible for us to add query parameters. For example, it's useful to retrieve articles for a specific tag. The way to do this would be:

al, err := cl.Articles(devtogo.Arguments{
        "tag": "go",
    })

To retrieve a single article, you need to specify the article id:

article, err := client.GetArticle("167919")

…

View on GitHub

Overview

Devtogo is an rest api wrapper for the dev.to api. It can be used to publish articles or even search for articles of a specific tag. There are many usecases for using the dev.to api.

Creating a client

A client can be created anonymously or via credentials. The only credentials supported at the moment are api keys and one can can obtain them by having a look at the authentication section of the api.

To create a client, we simply use the NewClient function and specify the api key.

cl := devtogo.NewClient(devtogo.WithApiKey("MY_API_KEY"))

Retrieving articles

It's possible to retrieve either a list of articles or a single article. For this post, we are going to retrieve all the articles that are tagged with go

al, err := cl.GetArticles(devtogo.Arguments{
        "tag": "go",
    })

Retrieving articles owned by you

If we are an authenticated user, we can retrieve our own list of published or unpublished articles. This is useful for obtaining a list of our drafts that we want to publish later.

al, err := cl.GetMyUnpublishedArticles(devtogo.Defaults())
    if err != nil {
        panic(err)
    }

Creating and updating articles

To create an article, we use the CreateArticle function. This will create a new article on the dev.to platform.

np, err := cl.CreateArticle(devtogo.CreateArticle{
  Title:        "My new dev.to post",
  Tags:         []string{"go"},
  BodyMarkdown: "my long markdown article that is preferably read from a file",
})

In the above example, we create an article that is in a draft state with a title, some tags and some markdown.

To publish this article on create, we set the Published property to true.

Supposed we want to update this article by publishing it, we simply use the UpdateArticle function.

ua, err := cl.UpdateArticle(np.ID, devtogo.CreateArticle{
        Title:        "My updates dev.to post using the API",
        BodyMarkdown: "my new updated content",
        Published:    true,
    })

Conclusion

In this article, we learnt how to use the devtogo to interact with the dev.to platform using the Go programming language. We also about how to use devtogo.

Full example

This is a full working example that retrieves the latest Go articles.

package main

import (
  "fmt"
  "github.com/ShiraazMoollatjie/devtogo"
  "log"
  "os"
)

func main() {
  apiToken := os.Getenv("DEVTO_TOKEN")
  if apiToken == "" {
    log.Fatalf("Set the DEVTO_TOKEN")
  }
  dtgc := devtogo.NewClient(devtogo.WithApiKey(apiToken))

  al, err := dtgc.GetArticles(devtogo.Arguments{
    "tag": "go",
  })
  if err != nil {
    log.Fatalf("Cannot retrieve articles: %+v", err)
  }

  for _,a :=range al {
    fmt.Printf("Title: %s, Url: %s\n", a.Title, a.URL)
  }
}

References

Dev.to API Documentation

Forem: Shiraaz Moollatjie

A journey with good old base64

What is base64?

Common use cases

Let's encode in go

What can we use?

Coding examples - Simple encoding & decoding

Comments

Let's do some slightly more complex encoding

When base64 is not recommended

Do not use base64 for storing passwords

Do not replace all your website images with base64 variants

Summary

Gophorem - A go API client for forem

ShiraazMoollatjie / gophorem

Where gophers and forem meet to build communities

🤔 What is gophorem

🤷 What can you do with it?

⌨️ How do you use it?

🌧️ It supports streaming too!

🤖 Future work and contributions

Devtogo update

ShiraazMoollatjie / devtogo

A REST API Wrapper for the dev.to api in go

Fully complete client

Milestones

🎉 Someone contributed

⭐ Engineers starred it

Roadmap ahead

Fork for forem

Use more Go features

Have more examples

Shoutout to the dev.to team

Decent ways of dealing with bad posture during lockdown that's other than getting a good chair?

Tips for solo coronavirus contributors to help your websites and apps stand out

Intro

Coronawatcher.live

Aunty Rona

Things that work for you, now all of a sudden work against you

SEO woes

Play and app store woes

The situation looks bleak. Is there a way out?

If mobile stores are not willing to publish your app, consider turning your native app into a progressive web app (PWA)

Using associative content to take advantage of keywords

Hashtags

Get social media handles early and post on them

Paying for ads on places other than Google

Conclusion

References

What is considered a good career in tech?

One new language every year

The Intent

Choosing something new to learn

The learning hurdle

What success looks like

Anti patterns for success

Conclusion

goluhn: A simple package for generating, calculating and verifying luhn numbers

What is a luhn number

Intended use and application

Using goluhn

ShiraazMoollatjie / goluhn

A package that implements the luhn algorithm for validation, calculation and generation

Validate

Calculation

Generate

Random Thoughts

Conclusion

References

Representing enums in go

Use iota to define enums

Applying this in practice

Using unknown and sentinel values

Deprecating enums?

Conclusion

Efficiently concatenating strings in go

What happens when you concatenate strings with the +operator

Using bytes.Buffer

Using strings.Builder

Let's measure some performance!

Using `goluhn`

Use `iota` to define enums

Using `unknown` and `sentinel` values

What happens when you concatenate strings with the `+`operator

Using `bytes.Buffer`

Using `strings.Builder`