Forem: Ardi

Don't learn C

Ardi — Fri, 23 Feb 2024 00:14:57 +0000

There is a murmur in tech twitter lately that you should just learn C in order to learn how the computer really works. Don't do that. Read a book.

It is a well known fact that most programmers have a brittle understanding of their computer. They sit nicely in their ivory tower of abstraction, building very complex systems while completely disregarding all of the tech stack they're sitting on. Thanks to the absolute dominance of programming languages with automatic memory management many of the programmers of this group have never had to think about memory, and then they are surprised when their pretty little nodejs consumes gigabytes of memory.

Recently there has been discussion that to solve this you should just learn C. C is supposed to be low level, you manage your memory yourself, you code for the hardware - you tell C what to do and it translates nicely to machine code that you can understand and be happy in a rainbow world full of computer candy :)

WRONG! You don't code for the hardware - you code for the Abstract Machine. What your program does is most definitely not what the hardware does!

What are you talking about?

What do you think happens when you have a char - an 8 bit signed integer - with the value 127 and you add one to it? If you paid attention in class you might know that with 2's complement it should loop around to -126. And you would be right! That is, if C actually cared about the hardware. It turns out that for historical reasons that is undefined behaviour(UB), which means that it could actually wrap around, optimize your code, or even make demons come out of your nose.

Or if you have an array and you mistakingly access out of bounds, that's also UB. In fact, there are A LOT of behaviours considered undefined in C that you could code in assembly (probably by mistake though).

It turns out that what you tell C to do is not what the CPU will end up doing because C has an optimizing compiler. Most of the time it will end up just tweaking what code it outputs keeping its behavior the same but there are times where it will take some liberties. While you are learning C you have to learn about all of these quirks and footguns, and you're not really learning that much about the computer itself.

There are a lot of web developers that don't understand how memory works and that should be addressed, but telling them to just learn C is not enough. Having a basic understanding of how the computer works can definitely be achieved by some basic reading and yes - maybe write some C, implement memcpy and a toy malloc but C shouldn't be the goal, only the medium.
Read a book that teaches you these concepts, don't just learn C.

The same can be said for operating systems but honestly even Node.js allows you to work with raw file descriptors and buffers. There is indeed value in working with low level APIs, I think all programmers at some point in their life should manually read(2) from a file, at least to have an idea of how it works.

But what do I know? Most programmers don't know and don't care about what goes on when they fetch() and they seem to be doing fine. I always feel like an old man screaming at the sky but I think it's sad that people have such a shallow and brittle understanding of the systems that they use. But when they venture out to the deep waters, to the prohibited knowledge that lies below they should probably be hand held by a good systems book.

SVG is a lie.

Ardi — Sat, 08 Apr 2023 15:55:08 +0000

SVG is a lie.

"What?" You might ask. "It obviously exists, what are you talking about? Are you stupid?"

Yes, SVG exists. What is a lie about it is the mystique it has, the widely spread misconception that SVGs are a small and efficient way to store images. That might have been true at some point in the past, but not anymore, at least most of the time.

"But SVGs are just vector graphics, they should be really small and efficient!"

I would agree, there should be no world where raster images would be smaller than vector graphics but here we are.

It turns out that SVGs are actually XML and plain text. Fortunately we moved as an industry away from XML, but there are still vestiges, such as HTML or SVG. Plain text is not really the most efficient way to describe images, unless you're doing it by hand, and I'm against editing SVG manually. We got tools for that!
Unless you're doing something really simple like an x of a hamburger menu you should probably stay away from it.

You can think about SVGs like having O(1) complexity, but with a high baseline cost. Yes, when you scale them up they will be extremely efficient but when they're small, which is almost always, they will lead to increased bundle sizes and if you have a lot of them it might get a bit slow.

Your network on svg

My findings

The most common use case for SVGs is icons. Most icons get designed and created in vector graphic editors like inkscape or Adobe Illustrator.

Icons tend to be small, ~16-50px

Let's run an experiment. For my personal website I downloaded a bunch of icons for the technologies and programming languages I know. Some of them happened to be SVG. Let's see their sizes:

$ find . -name "*.svg"  -exec ls -lh {} \; | awk '{print $9, $5}' | sort
./css.svg 473
./c.svg 1,4K
./c++.svg 1,6K
./docker.svg 1,4K
./git.svg 703
./go.svg 1,7K
./nodejs.svg 1,7K
./postgres.svg 4,3K
./python.svg 1,1K
./rabbitmq.svg 745
./react.svg 11K

Here we can see that the average icon is ~1KB, except for a few outliers. It's ironic that the biggest one is react, a framework famous for being bloated and slow. :P

Let's try now in webp format! The webp folks have done a great job, claiming 25% size decreases over png and 25/35% over jpeg The icons used in my page are 50px in height, so let's resize them to the correct size.

I'm using imagemagick's mogrify for this.

$ mogrify -resize x50 -format webp *.svg

./css.webp 562
./c.webp 588
./c++.webp 712
./docker.webp 596
./git.webp 502
./go.webp 482
./nodejs.webp 578
./postgres.webp 678
./python.webp 478
./rabbitmq.webp 404
./react.webp 800

WOW! The average here is around ~500B, with some even being 6.8x and 13.8x smaller, while most being 2-3x smaller.

The first time I saw this I couldn't believe my eyes. All my life I thought that SVGs were amazing, small, efficient...
It's great to see that the image compression world has come this far.

These are insane gains, let's see what would happen for favicons, which are 16px

$ mogrify -resize x16 -format webp *
$ find . -name "*.webp"  -exec ls -lh {} \; | awk '{print $9, $5}' | sort

./css.webp 204
./c.webp 196
./c++.webp 226
./docker.webp 190
./git.webp 186
./go.webp 160
./nodejs.webp 178
./postgres.webp 206
./python.webp 192
./rabbitmq.webp 164
./react.webp 184

These are spectacular gains. The react icon was 60 times smaller!

In practice this is not very meaningful. You only have one favicon and 800 bytes are not very significant.

But if it happens that you're sending a large quantity of icons that might start to matter, it's all about the scale.

But Ardi - what if you minify the SVG?

You're right! Let's try to optimize it using svgo

$ svgo .


Processing directory '.':
c.svg:
1.359 KiB - 31.3% = 0.935 KiB

c++.svg:
1.547 KiB - 33.3% = 1.032 KiB

git.svg:
0.687 KiB - 20.6% = 0.545 KiB

go.svg:
1.64 KiB - 16.8% = 1.364 KiB

css.svg:
0.462 KiB - 13.7% = 0.398 KiB

docker.svg:
1.375 KiB - 9.2% = 1.248 KiB

nodejs.svg:
1.606 KiB - 15.4% = 1.359 KiB

postgres.svg:
4.212 KiB - 10% = 3.789 KiB

rabbitmq.svg:
0.728 KiB - 0% = 0.728 KiB

python.svg:
1.081 KiB - 9.7% = 0.977 KiB

react.svg:
10.592 KiB - 16.6% = 8.836 KiB

Well, that's underwhelming. What if we gzip them? Most traffic usually gets gzipped to save bandwith anyways!

(webp of 50px)

./css.svg.gz 276
./css.webp.gz 372
./c.svg.gz 528
./c.webp.gz 400
./docker.svg.gz 668
./docker.webp.gz 389
./git.svg.gz 321
./git.webp.gz 360
./go.svg.gz 564
./go.webp.gz 299
./nodejs.svg.gz 747
./nodejs.webp.gz 367
./postgres.svg.gz 2,0K
./postgres.webp.gz 421
./python.svg.gz 545
./python.webp.gz 397
./rabbitmq.svg.gz 454
./rabbitmq.webp.gz 295
./react.svg.gz 4,1K
./react.webp.gz 406

It's clear that for icons SVGs are bigger than raster images.
However they still have their uses. Mainly CSS animations and larger background images.
If you're doing this SVGs are a fantastically flexible tool.

Conclussion

We are engineers at the end of the day. We should understand the trade-offs of the tools we use.

Most of the time you won't even care about the size of your icons. But if you are, perhaps because you're using a CDN and were considering using a tool such as svgo I hope that at least you consider checking the size of the rastered image.

Confessions from a Golfer Programmer

Ardi — Wed, 05 Apr 2023 15:21:40 +0000

I love code. Not all code, just beautiful code.

One of my favorite activities is to refactor - transmuting a program that works into another that does exactly the same thing, but better: Making it run faster, changing it so that it's shorter or more idiomatic, whatever that means; removing some nasty duplication, etc.
It makes me happy in the inside to see commits that have more removals than additions, which is probably the opposite of what the product manager likes to see.

This is not a good quality to have, however. It's easy for me to get engrossed in the beauty of my own - or someone else's - code, and not get anything done.

"Look!" I conceitedly exclaimed. "I've reduced the size of the function to one third its size"
"Nice, have you added the feature already? we really need it" My coworker inquired
"On it..." Realizing that I spent half an hour and it's still unstarted.

I come from javascript, where every time you try to do something minimally declarative your javascript engine dies a bit on the inside, and the garbage collector wishes you very bad things.

Let's say that our backend has a function that requests IDs. These IDs can be empty (""), and we want to ignore those. It gets the users from a dictionary in memory, and then returns the ones that it has found that are validated.

function getValidatedUsers(userIDs: string[] ): Users[];

There are two ways to achieve this.

The imperative way:

function getValidatedUsers(userIDs: string[]): Users[] {
    const ret = [];
    for i in userIDs {
        if (i === "") continue;
        const user = UsersDictionary[i];
        if (user !== undefined && user.isValidated()) {
            ret.push(user);
        }
    } 
}

The declarative way:

function getValidatedUsers(userIDs: string[]): Users[] {
    return userIDs
        .filter(x => x !== "")
        .map(x => UsersDictionary[x])
        .filter(x => x !== undefined)
        .filter(x => x.isValidated());
}

// Which could even be written more 
// concisely with arrow function syntax.

To me the declarative way more beautiful and clear, the steps look simple and ordered...

...But you would be a fool to use it.

The engine copies the array 4 times which involves a heap allocation, a memory copy and later on a garbage collection pause that is guaranteed to happen at the worst possible time, all of the time. Not to mention the closures generated too. You won't see me writting javascript that looks like that in code that needs to be even minimally performant, especially not in node, which is single threaded and your pretty little declarative function is slowing down the rest of the server.

I love Rust (I promise this is not another blog of Rust propaganda), which probably doesn't come as a surprise given that I'm your stereotypical nerd. Rust is great, but it's also my greatest demise.

My first contact with the expresiveness of Rust blew me away. You can use the declarative way, you can use beautiful declarative code, all without it slowing anything down (it even has the potential to get optimized into being faster!).

Rust is famous for its error handling. It does not support exceptions, instead opting for returning the errors to the caller. The first time I heard of it I thought that it was a step backwards, a return to C and its error code returning. But I was wrong, rust has exceptional tools for working with errors.

Here's an example from selenium-manager, maintained by the selenium project and a contribution I made

This calls three functions that can fail: set_timeout, set_proxy and resolve_driver

If any of them fails the program exits with a return code.

match selenium_manager.set_timeout(cli.timeout) {
    Ok(_) => {}
    Err(err) => {
        selenium_manager.get_logger().error(err);
        flush_and_exit(DATAERR, selenium_manager.get_logger());
    }
}
match selenium_manager.set_proxy(cli.proxy()) {
    Ok(_) => {}
    Err(err) => {
        selenium_manager.get_logger().error(err);
        flush_and_exit(DATAERR, selenium_manager.get_logger());
    }
}

match selenium_manager.resolve_driver() {
    Ok(driver_path) => {
        selenium_manager
            .get_logger()
            .info(driver_path.display().to_string());
        flush_and_exit(0, selenium_manager.get_logger());
    }
    Err(err) => {
        selenium_manager.get_logger().error(err);
        flush_and_exit(DATAERR, selenium_manager.get_logger());
    }
};

This is where my golfer self comes in, I want to make this code as concise as I can.

First, let's use if let Err, which could translate to javascript's if (my_fn() instanceof Error) and extract the error to a function

let err_out = |e| {
    selenium_manager.get_logger().error(e);
    flush_and_exit(DATAERR, selenium_manager.get_logger());
};
if let Err(err) = selenium_manager.set_timeout(cli.timeout) {
    err_out(err)
}
if let Err(err) = selenium_manager.set_proxy(cli.proxy) {
    err_out(err)
}

match selenium_manager.resolve_driver() {
    Ok(path) => {
        selenium_manager
            .get_logger()
            .info(path)
        flush_and_exit(0, selenium_manager.get_logger());
    }
    Err(err) => {
        err_out(err)
    }
};

But Rust has so much more to offer. Notice that there's a happy path (continue) and a gutter, where all of the errors go.

This is a common pattern, and Rust has really nice tools for working with these in a declarative way. We can use all of the expressiveness of rust to golf this down

selenium_manager
    .set_timeout(cli.timeout)
    .and_then(|_| selenium_manager.set_proxy(cli.proxy))
    .and_then(|_| selenium_manager.resolve_driver()
    .and_then(|path| { // If all of them succeed
        let log = selenium_manager.get_logger();
        log.info(path);
        flush_and_exit(OK, &log);
    })
    .unwrap_or_else(|err| { // If any of them fail
        let log = selenium_manager.get_logger();
        log.error(err);
        flush_and_exit(DATAERR, &log);
    });

This is reminiscent of the code we saw earlier in javascript, this time with errors, which in my eyes makes all of it much easier to read and understand, and most importantly: More beautiful. The only metric that matters.

All of these changes are cool and all, but they provide no value. The end user doesn't care, the compiler doesn't care either (they're equivalent), but I care. I'm in for the aesthetics, but don't tell my boss.

When I have to make functional changes they break the perfect aesthetic, but given the time to refactor it will become something even more beautiful than before, complexity that has been tamed into a coherent piece of machinery.

With no time given to making it look beautiful it will become an ugly mess, but an ugly mess that works. There's value in that.

One time when I read the code of a junior coworker of mine, I physically shivered, horrified at some of the things I saw, and I felt an urge to fix it (this is one example of such):

public isDebug(): boolean {
    if (window.localStorage.getItem("debug") == "true") {
        return true
    }
    else {
        return false
    }
}

I'm not kidding, that's real code. I emotionally had the need to rewrite it to

public isDebug(): boolean {
    return localStorage.getItem("debug") === "true";
}

I doesn't make me happy that they save it to localStorage, that's part of a bigger bodge that I don't even want to look at.

But that was a part of the code that didn't affect me in any way, I just tumbled with it and couldn't look away.
It might not be pretty but their code worked, and at the end of the day that's what matters. I have so much to learn from them :)

That was it, I hope you liked my first post.
Remember when I said I wasn't going to make Rust propaganda? I kind of lied a bit there :)