Forem: jD91mZM2

Absorbing Markov Chains, how do they work?

jD91mZM2 — Fri, 21 Dec 2018 05:44:23 +0000

I'm sure a lot of people have heard of Absorbing Markov Chains mainly because of Google Foobar. One of the levels there was to, given an input of a bunch of states and chances to go between the different states, find the percentages of going from one state to a "terminating" state in an unlimited number of steps. When I got to that question, I did not have time to learn this technology. I am very embarrassed to admit that because I can't let things go... I cheated that challenge by hard-coding the answers from somewhere online. I hope there's a button somewhere to redo it - I can't check right now because it's down for maintenance.

I think the least I can do to make up for my mistake is to make the guide I wish I had, so even somebody like me can understand them. Here we go:

Why use Absorbing Markov Chains?

Absorbing Markov Chains can calculate the percentage to go from one state to another, even if such a calculation loops forever.

They can be as simple or complicated as you like.

Understanding

Before I go on, I would like to point out that some of the following images have been remade as SVGs so you can infinitely zoom them without losing quality. Because dev.to's CDN just converts them to PNGs, you can find them all here.

1. Convergence

Initially the biggest question to me was: How can there be a finite answer? Won't there be an infinite number of combinations?
Imagine a box being built by materials that are half the size of the existing material. Since you can divide infinitely, is there really a finite answer?

... Well! Since the box is slowly working towards 1 infinitely, we can just count 1 as the finite answer.
Similarly,

1/2 + 1/4 + 1/8 + 1/16 + ... = 0.9999999... ≈ 1

Notice I used the "almost equal to" sign there. But technically, I didn't even need to. Because in math, there's proof that 0.99999999... IS 1! The secret lies in the fact that infinity has some quirky rules. Here, let me show you something a friend showed me recently:

// As you know:
9x = 10x - x
// Now do this on 0.99999...
x = 0.99999...
0.99999... * 10 = 9.99999...
9.99999... - 0.99999... = 9
// Therefore...
0.99999... * 9 = 9
0.99999... = 1

This works because normally when you multiply by ten, you add one more number before the decimal point and remove one after it. But since removing one from infinity is still infinity, there's still the same amount of numbers after the decimal point, which means they cancel out.

2. Crash course on matrices

Before we continue, I need to give a few crash courses. If you already feel comfortable with any of these subjects you can skip over them if you like. I'll try my best to keep it short and to the point while still being clear!

Matrices are just like 2 dimensional arrays in programming. Doing addition and subtraction on two matrices creates a new matrix where each value in the first matrix is added/subtracted with the corresponding value in the other matrix.

Although, multiplying two matrices is more difficult: Each row/column combination creates a new number in the result matrix that is the sum of each element in the row multiplied with the corresponding element in the column. This means the number of columns in the first matrix must be the same as the number of rows in the second matrix. The result will have the same number of rows as the first matrix and same number of columns as the second.
Yep, this is confusing. Here, let me show you:

Then, in the above image, it'll do the same thing for every row.
This means that unlike normal numbers multiplying two matrices might give different results depending on the order.

Multiplying a normal number by 1 gives itself. The matrix alternative to this is something called the "identity matrix", which is very simply a bunch of ones on the diagonal.

3. Crash course on matrix inversion

Now, as you might know, inverting something in math is written as taking something to the power of minus one, ^-1. For "normal" numbers, this is often 1/n. But inversion also exists for matrices, and is another matrix that meets the following requirements:

the original matrix multiplied by the inverse matrix must be the identity matrix
the inverse matrix multiplied by the original matrix must be the identity matrix

Not all matrices have an inverse, and finding it is a bit tricky! There are special rules for 2x2 and 3x3 matrices, but since we're programmers we need a method for all sizes!

One method of finding an inverse matrix is Gauss-Jordan elimination. It's a cool mathematical technique that's not specific to inverting matrices. It works by applying various "row operations" to each row in order to turn it into the identity matrix, but while doing so also applying the same operation to the result. The result in this case, is the identity matrix.

The row operations are relative operations such as multiplying the row, adding/subtracting any multiple of another row, or swapping two rows.

It's easiest to imagine both the work matrix and result matrix to be both put into one big matrix, but nothing is technically keeping you from having them separate which might be more performant.

Creating an input matrix for Absorbing Markov Chains

Let's create a very very basic example, so we can not only learn how to use this to solve a problem, but also try to see exactly what's going on as we do.

As you can see, we have an Absorbing Markov Chain that has 90% chance of going nowhere, and 10% of going to an absorbing state. We also draw the absorbing state as having 100% chance of going to itself but that honestly doesn't matter in any of the formulas I've seen yet.

We talked about convergence before and because it's an infinitely increasing percentage chance to reach a terminal (absorbing) state, we should expect a 100% chance of reaching state 2. I'll also go as specific as to say we should get an estimated number of 10 steps to reach state 2, as 1/0.1 is 10, how many times 0.1 (10%) is repeated before reaching 1 (100%).

To calculate the result of this, we need to write this as a matrix. Each row will be a state, and each column in that row will be the chance of going to the state with the same row number as that column. I recommend specifying the chance as a fraction, especially if you're making something that calculates this, as fractions never lose precision unlike floats.
Like always, it's easier if I show you:

Creating a fundamental matrix

We have a few things we need to do that applies to all operations you might want to do.

First of all, we need to separate non-absorbing and absorbing states in the matrix. Sort all states so that the absorbing states are at the end (or beginning, if you really really want to. But for this article, I'll assume end). This should allow you to split the input matrix into 4 parts (2 for each dimension):

Secondly, we need a "fundamental matrix". This will allow us to get properties from the chain. Each entry in the fundamental matrix will contain the average number of visits to the state with the same index as the column starting from the state with the same index as the row number. In other words, each entry at position (x, y), must contain the average visits to state x starting from state y.

A cool thing about Markov chains is that you can get a matrix where each entry at position (x, y) contains the probability of transitioning from state y to state x in exactly k steps, simply by taking Q^k (yes, that's an exponent, those work on matrices too, Q*Q*Q*Q etc).

So to get the fundamental matrix, we want to sum together the probability of each number of steps ranging from 0 to infinity. Luckily though, it's been proven that you can get the exact same result using the formula N = (I - Q)^-1.

^Image ^stolen ^from ^Wikipedia

So following the equation N = (I - Q)^-1, where N is the fundamental matrix, Q is the sub-matrix we just established, and I is an identity matrix with the same size as Q (will always be square), we can calculate the fundamental matrix.

Our fundamental matrix is now [10].

Getting the average number of steps to reach an absorbing state

Since each row in the fundamental matrix has columns that each give the average number of times to visit the state with the same index as column, starting from the state with the row's index, we can get the average number of steps to complete by summing together each row, resulting in the average number of visits in total starting from there.

In math, a trick to sum each row in a NxN matrix is to multiply it by a 1xN (1 column, N rows) matrix where each row is a one.
The formula is t = N1, where t is the matrix of average number of steps from each starting state to an absorbing state, and 1 is the 1xN matrix mentioned.

Obviously, since our example matrix is 1x1, multiplying it is really simple...

And, as you can see, the average number of steps to reach the absorbing state is 10, just as we guessed!

Getting the percentage chance to reach each absorbing state

So, this is probably the step you've been wanting to read. After all, getting a percentage for each state sounds more interesting than the average number of steps it takes to reach any (absorbing) state.

The formula for this is B = NR. Remember R from the sub-matrix we cut earlier? Multiplying the matrix containing the average number of visits to each non-absorbing state with the matrix containing the probabilities to transition into an absorbing state in one step, we get the probabilities of entering an absorbing state after any number of steps.

The final result is a matrix where each column is the chance for the state with that index has to reach the index of that row. In this case non-absorbing state 1 has 1 (100%) chance to reach absorbing state 1 (actually state 2). Seems correct!

^(Example output here is the output of https://en.wikipedia.org/wiki/Absorbing_Markov_chain#String_generation)

Final notes

There's a famous quote:

If you can't explain it simply, you don't understand it well enough.

^(Sadly couldn't find who said it, but it's frequently miss-attributed to Einstein.)

And I admit I certainly don't understand this witchery well enough. But I hope that my article may still have given you a little more insight. Wikipedia and other sources I've found just seem to state the facts, but don't try to explain them for 5 year olds. I might not be able to explain them well either, but the least I can do is try.

I have implemented this here: math-types

Programming for Redox OS

jD91mZM2 — Thu, 05 Jul 2018 17:52:55 +0000

If you don't know what Redox OS is, it's an entire operating system written in Rust. This is because Rust has some amazing guarantees about memory and thread safety. To make it even safer, it's a microkernel as opposed to a monolithic kernel. That means that for example drivers and similar run in userspace and cannot crash the entire computer in the same way.

To communicate between processes and the kernel, Redox OS has custom schemes. It follows the tradition of having everything be a URL. Schemes are dead simple and super awesome. You open a file with the path scheme:path. If no scheme is specified it defaults to the one of the current working directory - which is usually going to be file:.

TCP

An example of the scheme is the TCP protocol. Even though networking is supposed to be hard, in redox it's dead simple. You just open("tcp:address", O_RDWR).
Here's an example that curls example.com without any libraries what-so-ever (not an idiomatic use of the http protocol - prefer a library for doing http requests):

let mut file = File::open("tcp:93.184.216.34:80")?; // example.com
file.write(b"GET / HTTP/1.1\nHost: example.com\nConnection: close\n\n")?;

io::copy(&mut file, &mut io::stdout())?;

Program IPC

Another example of a scheme is one I wrote myself - the ipcd daemon. It's a simple scheme for inter-process communication. You call open("chan:name", O_CREAT) to create a server, and dup("listen") to accept a connection. The dup syscall is for duplicating files, but can optionally take a path which can be used from the scheme.
Here is a simple hello world server using ipcd - this time we need the syscall library for redox in order to duplicate the file.

let server = File::create("chan:hello")?;

loop {
    let stream = syscall::dup(server.as_raw_fd(), b"listen")?;
    let mut stream = unsafe { File::from_raw_fd(stream) };

    stream.write(b"Hello World!\n")?;
}

And to read from this ipc channel you just have to cat chan:hello.

Events

This scheme is somewhat special, not because it has any special hardcoded exception to it, but because it's used everywhere, even in other schemes.
The event scheme is kind of like epoll in linux. You write an event to an instance and it registers to listen for events from file (or instance of a scheme), and then you read and it blocks until the first event.
There is an excellent example in the event overhaul RFC.

This scheme works very well with the time: scheme, and can be used for setting a timeout to reading. The time scheme is simple, can read the current time and write a deadline which triggers an event when that time is elapsed.
Here's an example that timeouts a read to 2 seconds.

const TOKEN_TIME: usize = 0;
const TOKEN_FILE: usize = 1;

let mut selector = File::open("event:")?;
let mut some_file = /* anything you want to read from here, like a tcp stream */;

// Deadline
let mut timeout = File::open("time:")?;
let mut time = TimeSpec::default();
timeout.read(&mut time)?;
time.tv_sec += 2; // deadline to 2 seconds in the future
timeout.write(&time)?;

selector.write(Event {
    id: timeout.as_raw_fd(),
    flags: EVENT_READ,
    data: TOKEN_TIME
})?;
selector.write(Event {
    id: some_file,
    flags: EVENT_READ,
    data: TOKEN_FILE
})?;

let mut event = Event::default();
selector.read(&mut event)?;

match event.data {
    TOKEN_TIME => {
        // The timeout of 2 seconds was exceeded
    },
    TOKEN_FILE => {
        // The file can now be read!
        // How much can be read is undefined and depends on the scheme.
        // Most built-in schemes are nowadays edge triggered and you should
        // have the file be non-blocking and read over and over again
        // until you hit EAGAIN.
        // This matches the behavior of linux.
        let mut buf = [0; 16];
        loop {
            match some_file.read(&mut buf)? {
                Ok(0) => break, // EOF, we read everything!
                Ok(n) => {
                    // Handle the read here.
                    io::stdout().write(&buf[..n])?;
                },
                Err(ref e) if e.kind() == ErrorKind::WouldBlock => break, // can't read any more this round.
                Err(e) => return Err(e)
            }
        }
    }
}

How does it work?

Note: Writing a custom scheme requires root

Main Loop

The scheme is registered using the root scheme - which has no name. open(":test", O_CREAT) registers the scheme and returns a file you can read packets from.
How do you handle the packets? That's where the Scheme family of traits come in! They have a function called handle that is already implemented, that reads the packet and calls the correct trait function.
A simple loop may look like this:

let mut handler = Handler; // handler implements SchemeMut
let mut scheme = File::create(":hello")?;
loop {
    let mut packet = Packet::default();
    scheme.read(&mut packet)?;

    handler.handle(&mut packet);
    scheme.write(&packet)?;
}

Sadly most schemes' loops look more difficult because they might need to read from multiple schemes at the same time, or reschedule blocking functions.

Unique IDs

The scheme has functions like these:

fn open(&mut self, path: &[u8], flags: usize, _uid: u32, _gid: u32) -> Result<usize>;

where you return any integer you want!
A cool thing about Redox schemes is that each instance of the scheme (file descriptor) gets mapped to an ID, and multiple file descriptors can have the same ID (so you don't have to implement dups without a path yourself).
What almost all schemes do here is keep an integer that points to the next id, and then on open it adds the id to the an internal map together with some struct with data about the handle, and finally increments the integer.
Later functions are called with the id:

fn read(&mut self, id: usize, buf: &mut [u8]) -> Result<usize>

Blocking

A blocking function like read is usually rescheduled by the scheme when it fails. Usually they used to use the EWOULDBLOCK error internally to add the packet to a list of todos and then later (on either another scheme event or some timer) try to process that packet again. To ease with this, we now have a SchemeBlock trait that can return a None to indicate that it should be rescheduled.
For an example of a really simple blocking loop, check ipcd's main file.

Events

To make your schemes support events, you just need to write a packet to the scheme file with the values of a=SYS_FEVENT,b=id,c=flag,d=count (I don't think count is used anymore, you might just leave it at 1). There is also an fevent scheme function that will let you know when somebody has registered your scheme to an event instance. This can be used to reset variables to keep track of notified state in schemes that send edge triggered events.

Base64 is fun!

jD91mZM2 — Fri, 09 Feb 2018 13:56:11 +0000

Hello! Today I decided to stop feeling like Base64 is black magic, and go really learn it. You can see the result here. I learned a lot of interesting things, so I'll share it with you here.
Because of mixed backgrounds I'll make sections for everything so you can skip anything you already know.

Bitwise operators

To properly learn Base64, you need to learn bitwise operators. I'm sure you've seen stuff like 50 & 1 == 0 somewhere before. What are these weird half-boolean operators doing on the integers?!
As you know, computers work with binary. But the relationship is closer than you think. What bitwise operators do is flip certain bits.
Each binary digit (0 or 1) is called a bit.

Bitwise And: &. This calculates the integer result of two integers where each bit is the same. In other words, it sees which bits number one and number two has. So it's called bitwise and. Examples:

  42 // <- Binary: 101010
& 15 // <- Binary: 001111
= 10 // <- Binary: 001010

  5 // <- Binary: 101
& 6 // <- Binary: 110
= 4 // <- Binary: 100

Bitwise Shift: << and >>. Basically "shifts" all numbers to left or right. These can be compared to multiplying by 10^n in decimal. In fact, it's exactly the same as multiplying by 2^n. When you multiply something by 2, the compiler sometimes optimizes it to bit-shifting by one.

   16 // <- Binary:  10000
<< 1
=  32 // <- Binary: 100000

   42 // <- Binary: 101010
>> 3
=  5  // <- Binary:    101

Byte order

As you may know, an int is a 32-bit integer and a byte is an 8-bit integer.
Since 32 (and all other types of integers) is a multiple of 8, you could technically convert a bigger type of integer to multiple bytes and then back... right?
Like say 666, 0000001010011010, could be stored as 00000010 and 10011010?

Turns out, of course, I wasn't the first one to think of this. In fact, in memory they're represented like this too. In C you could simply cast a int* to a byte* and then access the specific bytes inside the integer.
What you may find (it depends), is that they are in the reverse order of what I just showed (10011010 first, not last). This is called little endian, where as what I showed is big endian. Big endian means the "big end" (the byte that makes the largest impact, just like changing the one in 100 makes a larger impact than changing the one in 10) is first.

Putting it all together

Normal binary data is stored in 8 bits, which means it can go to 256.
64 happened to be a good amount of characters that is a power of 2.
The reason it's a power of two is quite brilliant: It can be converted to by simply reading 6 bits instead of 8.

Say we have the following: "hello".
In bytes, this is 104 101 108 108 111.
In binary: 01101000 01100101 01101100 01101100 01101111.

Now we make this into one integer. Before we do that though, there's a problem. This quickly gets unbelievably large.
We solve this using a convenient fact: 4 6-bit integers has the same bit length as 3 8-bit ones. That's also why it's padded to 4: 6*4 = 8*3. This is really convenient for the programmer when decoding, so he/she doesn't have to bounds check valid base64.

Because of this, we only have to worry about 3 bytes at a time (4 when decoding).

for bytes in input.chunks(3) {
    // Fun fact, you *could* do this, but it's more fun to use bit shifting.
    // let buf = mem::transmute::<&[u8], u32>(bytes).to_be();
    let mut buf = 0;
    buf += (bytes.get(0).cloned().unwrap_or(0) as u32) << 8*2;
    buf += (bytes.get(1).cloned().unwrap_or(0) as u32) << 8*1;
    buf +=  bytes.get(2).cloned().unwrap_or(0) as u32;

The reason for .get(n).cloned().unwrap_or(0) is basically because indexing can crash. The input doesn't have to be dividable by 3.
Now we have

011010000110010101101100

and

011011000110111100000000

.
Note: The last number gets padded out with 0s.

Second step now sounds really simple: Re-split the numbers every 6 bytes and insert = if it runs out before it's a multiple of 4.

    for i in 0..4 {
        let j = (3-i) * 6;
        let buf = ((buf >> j) & 0x3F) as u8;

The first part is actually rather simple but unecessarily verbose. It counts down from 24, 6 steps at a time. It could probably have been written like for j in (0..=24).rev().step_by(6), but I need i later.

But next comes the fun part:
Let's imagine j is 18.

   011010000110010101101100
>> 24
=  011010000110

   011010000110
&  000000111111 // <- 0x3F in binary.
=  000110

See the pattern? In a way, we basically just took a substring... of the integer! The bitwise shift allowed us to fetch a value down, and the bitwise and allowed us to specify which values we cared about.

Now we have

011010 000110 010101 101100

and

011011 000110 111100 000000

Finally, let's insert the value, or = if out of bounds.

        if bytes.len() >= i {
            output.push(char(buf));
        } else {
            output.push('=');
        }
    }
}

Note: I didn't include the definition of char function. It basically just maps a value to a base64 character, like 0 to A.

We now have

a G V s

and

b G 8 =

aGVsbG8=

There are STILL not enough messaging applications

jD91mZM2 — Mon, 08 Jan 2018 09:35:27 +0000

Every single service nowadays has private messaging. Twitter, Facebook, Reddit, Steam, etc. Every single programmer just learning about Tcp wants to make a simple one. Yet there are still not enough.

Alright, since there are so many services I am going to filter them by a simple requirement:

One server can have multiple channels.

If you don't get what I mean by that, I mean basically something like IRC.
Who meets the requirement? IRC, Discord, Slack, Mattermost. Those are the only ones I know of. Now let's check how each individual one falls short:

Discord:

Electron
Client side validation
Huge cache files
Not following the XDG standard (cache files should be in ~/.cache)
Using regex over linkify
Closed source
Stores currently written message in the JS local storage for every single keypress

The main reason Discord has so many points here is because it's what I've been using (and suffering from) the most.

Slack:

Electron
Closed source
Only stores the last 10 000 messages

I have never tried Slack. This is all I know.

Mattermost:

Electron
Channel join/leave system is kind of useless
Can only use one server at a time

IRC:

Difficult
No real message storage
No real authentication system apart from unofficial NickServ
No real bot API

One thing that hurts me a lot here is how many of these are using Electron. Is programming GTK+ really so difficult that you have to steal all the client's CPU? Discord is using 7% CPU idle for me. Sure, Electron is easy and looks good, but it's also the biggest hack ever made. Launching an independent web browser copy because you want to look professional having an EXE file for your webapps... really?

So what I'm trying to say, is that if any of you feel urged to make a chat application but aren't because of how many there already are... think again. We need more alternatives out there.
Myself, I've been writing one that's currently in beta. Although it's been a useful experience for me, it's not very fun at times.

And of course, if any of you know an alternative (even if you made it yourself) that seems to be what I'm looking for, please do link it in the comments!

Arch Linux: Stop recommending people to use makepkg for the AUR

jD91mZM2 — Sat, 06 Jan 2018 11:22:13 +0000

This is relevant to Arch Linux and the Arch User Repository. If you don't know what that is, no need to read this article.

Alright, there seems to be around 3 kinds of people:

"Pacman should be like apt"
"You probably want an AUR helper"
"Real men use makepkg"

2/3 of these people know that adding a custom repository to get a program is HORRIBLE security-wise. That's not worth mentioning.
I will instead be focusing on eliminating the 3rd kind, leaving us with the second (which I'm a part of).

Let me tell you a tale of how I got used to the AUR.
Immediately when I started off, I wanted to be the 3rd kind. I wanted to use only the official tools. "Real men don't need no helper" or whatever. I scanned the wiki on my phone (Arch wasn't completely set up yet), and found out how to install packages. I failed to find out how to update them, so I assumed pacman did that for you. I started cloning all packages to ~/Downloads, building them, and deleting them.
That was my first pitfall. Already, I had made a mistake. I needed to keep them updated. Alright, that's simple enough. I re-downloaded all my packages (TIP: pacman -Qm) to ~/AUR. Then I made a bash script to git pull all the things, and building them if there was anything to update (NOTE: I hadn't thought of -git packages).
This is what I used for a while. And it worked, except it required a lot of interaction. I had to copy the URL, cd, git clone, cd, makepkg. I started avoiding the AUR as much as I could. Alright, simple fix: Just make a bash script to download it? No. This is where I stopped. And I'll tell you why in a second.
But first I want to inject that I never checked any other files than the PKGBUILDs. I never read the wiki carefully enough, so I failed to realize that ALL files could contain viruses. If I had used a helper, this risk would have been avoided all together.

I saved my most important point to last. If you make shell scripts around the manual way of doing it, are you really still using the manual way...
...or have you created a helper?

Are async frameworks really worth it?

jD91mZM2 — Sat, 30 Dec 2017 07:50:15 +0000

This "article" is going to be specifically about Rust and Tokio, but it should apply to more frameworks

This is more of a question than an article. I've been working on Synac for a while, and I've needed to handle multiple connections on one thread.
For the server, I went with tokio. It allowed me to do something like this:

listener.incoming().for_each(|(conn, addr)| {
    ssl.accept_async(conn).map_err(|_| ()).and_then(move |conn| {
        println!("Client connected!");
    });
});

core.run();

The problem with that is that often you can't just make everything async unless you're programming in a language where blocking calls are barely possible (looking at you, JavaScript). Sure, Tokio is easy once you get the hang of it. But usually you want to combine syncs, asyncs, and all kinds of things. This is where Tokio becomes a problem.
For the synac client, I already had experience with a library that only allowed blocking reads, so multithreading wasn't possible. I didn't want to force built-in mutexes upon users, and I didn't want to prevent multithreading, so I came up with my own solution. It gave me code like this:

if let Ok(Some(packet)) = listener.try_read() {
    println!("Packet received: {:?}", packet);
}

(If you wonder how I did this magic sorcery, I simply did something similar to read_exact, but in a struct. Code)
This ended up being a good decision, because I could then do non-blocking reads inside gtk::timeout_add! And it wasn't too difficult to code it either!

Now I want to ask: When should you use an async framework with cores and stuff, and when should you just write a non-blocking listener? I feel like non-blocking listeners are way more flexible, but there has to be a reason tokio exists... Right?

A letter to my past self

jD91mZM2 — Thu, 14 Dec 2017 08:46:47 +0000

A little more than a year ago, I was just about ready to move on. I had my fill of Java, discontinued my shitty "programming language" (which was just a bunch of startsWith("<command> ") statements), and I was making a crappy little chat-bot thing with TCP support. In order to connect to it, I realized I should look into some other language which was easier with this task. So I turned my attention on Go.
So that I was. Finally exploring a new language after a very long time of being addicted to Java Swing. I never liked front-end development, but I thought it was needed. Oh boy I was wrong.
Somewhere there my friend tbodt popped along. He started asking a little about one of my applications, which drove me interested in working more on that application. I don't know when it happened, but after some time it was clear to me this tbodt person was a better, smarter and more knowledgable person than I'll ever be. And I was right, a year later I am a completely changed person thanks to him. Now I want to see if I can be my own tbodt:

Dear past self. In order to improve, here are a few tips.

Font-end is trash. You don't like font-end development. You do it because you know that every user wants a GUI. That is incorrect, you're simply looking at the wrong target users. Start making terminal applications, it's way more fun.
Abandon Windows. What you used to call "Linux" (when you actually meant "GNU/Linux". Linux is just the core of the operating system, the kernel. You still need the GNU operating system to actually make a working system). It isn't what everybody says it is, it's not a pro 1337 h4x0r operating system for edgy people. It's for everyone. It's better than Windows will ever be. Also, don't worry about your C: and D: drive. On GNU/Linux they're for some reason one drive. I still haven't figured out why, but I guess Windows is just drunk. Or maybe they're different partitions (parts of the drive which pretty much act like separate drives). What distribution? Ubuntu. But don't stay there forever. Make a separate /home partition so you can keep all your files when you reinstall your system, and then when you've been on Ubuntu for a while and feel bored with it you can switch to Arch Linux. You won't get bored with Arch. Also look into the i3 window manager. It isn't as empty and scary as it looks when you first start it. You just need to configure it some. If absolutely needed, watch some YouTube video on i3 to really get motivation. By the way, don't bother with trying to get nvidia drivers. They're literally broken. You'll just ruin your computer trying to uninstall them and reinstall the open source ones. Stick with the open source.
Finally learn what the |, &, ^ and ~ operators are!!! I know you've really wanted to know but never been able to learn. Don't worry, it's simple. Just take a look at the binary values. 2 | 1 = 3 because 10 | 1 = 11. 5 | 3 = 7 because 101 | 11 = 111. It basically enables all bits from the second value in the first. ~1 is -2 because 00000001 inverted is 11111110. The reason it's 8 in length is because a byte is 8 bits in memory. And in memory, if the 1st bit is 1 that means it's a negative number, and starts subtracting everything from the max byte, 256. The rest you should be able to experment and learn on your own.
Abandon Java. This is the first time you'll really feel like staying. New languages look new and scary. You think you're confident with Java, and you doubt you'll ever be that confident with another language. You don't wanna throw away all your old skills for nothing. It isn't for nothing. You can fight through this. Stick with Go for a while, but don't stay. Go is just good to know, but it's not the end. Go forward. Get Rust. Rust is everything you've ever wanted. You can code with C-like performance and possibilities while still keeping built-in UTF-8 support, safe memory management. Also threading.
Start with Git and GitHub. Sharing code isn't as bad as you think. Nobody will read and judge your code. Plus, people can already decompile it. Trying to hide your code does nothing. Trying to produce binaries for all systems is a pain (not so much with Java as any other language). You can just give them the source and they'll compile it for themselves. Also versioning is nice for example if you're implementing a new feature but decide you wanna roll the changes back.
I know you think there isn't anything like "true multithreading". You think that computers are just shuffeling instructions. And that is partly correct. But most computers have multiple cores, so there is a chance that "data races" occur when accessing the same variable from different threads. That is where so called "Mutexes" come in. There are also "Atomic" variables for primitive types, which is what mutexes use under the hood. And yeah, you've been misusing Java's "AtomicX", thinking it's just a wrapper in order to edit a final field.
Learn what memory is. It's just a big pile of bytes. If you make the string "hi", that looks like 104, 105 plus a little extra information, depending on programming language. In C it would be [104, 105, 0] because 0 marks the end of a string. In most languages it would be something like [2, 104, 105], where the 2 shows the string length. And that's why pointers have "addresses". Because a pointer is just a number to a memory cell. Also, there are actually 2 different kinds of memory. Heap and stack. The heap is dynamic. That's where you can actually allocate more. Somebody correct me if I'm wrong with the following: The stack is faster to use, because it can never be resized. Once your program is started, you get the heap you ask for and then you can't make more. But yeah, in order to make your program faster, you don't need to do whatever you think you need to do. You just need to make sure to allocate less. Which Rust is awesome at.

How I want my command inputs, but nobody agrees

jD91mZM2 — Sun, 13 Aug 2017 04:38:55 +0000

switch cmd {
case "help":
    search := strings.Join(args, " ")
    printHelp(search)
case "exit":
    closing = true
case "exec":
    if nargs < 1 {
        stdutil.PrintErr("exec <command>", nil)
        return
    }
    ...

This is some source code from my old version of my crappy application DiscordConsole, written in Go. (A rewrite is being made.)
I chose it because it perfectly demonstrates how I want my command systems, and also how languages make this difficult.
Come on, I know you like your OOP, but isn't it damn overkill to

public interface CommandHandler {
    public String[] acceptCommands();
    public int minArg();
    public int maxArg();
    public void handleCommand(String, CommandArgs);
}
public class EchoCommand implements CommandHandler {
    public String[] acceptCommands() {
        return []String{"echo"};
    }
    public int minArg() { return 1; }
    public int maxArg() { return Integer.MAX_VALUE; }
    public void handleCommand(String cmd, CommandArgs args) {
        System.out.println(args.join());
    }
}
...

To me that's just plain ugly. And oh god how inefficient.
Sorry for the mix of programming languages, I thought Java perfectly described OOP, and Go perfectly described how I want my command systems while still using the "switch" keyword and not "match" like Rust does.

But there is this one advantage with your system. It allows splitting commands in multiple files. And now you won't stop nagging on me and how horrible my command system is.
Yes, it's true I can't split mine into multiple files. It's true that's really a problem once you get more than 3 commands. But it's also the OOP way makes unecessary allocations and function calls, and is a pain to set up.

So let's think outside the rules of any programming language. What is a perfect command system to me?

match cmd {
    "echo" => {
        usage_min!(1, "No argument specified");
        println!("{}", args.join(" "));
    },
    "some_long_command" => include!("commands/some_long_command.rs"),
    ...
}

This example was written in Rust. The match keyword is like switch but cooler. It perfectly demonstrates how macros are THE way to make a cheap application look good in code, and yes, that is saying all languages that don't have them suck. I love macros.
It also shows that it very nearly is possible to make a non-OOP command system look good, AND split it into multiple files. The only thing stopping us is that the include! keyword in Rust exists, but it doesn't see local variables and macros.
Still then, a lot more languages should have macros, and a lot more languages should be able to include a file's contents without any hygiene and stuff.

This whole post is essentially me hoping #32379 gets fixed...

Comparing error handling in Java, Go and Rust.

jD91mZM2 — Tue, 04 Jul 2017 08:13:41 +0000

Error checking is everywhere. Sometimes it's more complex than other time. Take file managing for example. You have to remember to close the file every time you return... If it's not automatic.

Let's see how Java does it. First, we'll look at Java 6-.

BufferedReader reader;
try {
    reader = new BufferedReader(new FileInputStream("test.txt"));
    // read file
    if (condition) {
        // closed by finally block
        return;
    }

    // closed by finally block
} catch(Exception e) {
    // do proper error handling
    // closed by finally block
} finally {
    try {
        reader.close();
    } catch(Exception e) {
        // do proper error handling
    }
}

System.out.println("File read");

Oh my God. OK, that is HORRIBLE. Let's see how Java 7+ does.

try(BufferedReader reader = new BufferedReader(new FileInputStream("test.txt"))) {
    // read file
    if (condition) {
        // automatically closed
        return;
    }

    // automatically closed
} catch(Exception e) {
    // do proper error handling
    // automatically closed
}

System.out.println("File read")

That is... very good. Congrats, Java. Let's see how Go does it.

file, err := os.Open("test.txt") // no semicolon :(
                                 // i love semicolons :(
if err != nil {
    // do proper error handling
}
defer file.Close()
// read file
if condition {
    // closed by defer
    return
}

fmt.Println("File read")
// File closed on return, so you probably wanna return ASAP

One thing to note with Go: Independent error handling. You can't handle all errors clumped together. This is GOOD. This let's you specify where exactly it went wrong. Errors in Go are also pretty much strings. You make a custom error with errors.New("oh no"). Good stuff
Now let's take a look at Rust.

{ // Start a new scope. You don't *have* to do this
    let file = match File::open("test.txt") {
        Ok(file) => file,
        Err(err) => {
            // do proper error handling
            return;
        }
    };
    // read file
    if condition {
        // out of scope - automatically closed
        return;
    }

    // out of scope - automatically closed
}

println!("File read");

After writing this, I had to look at the code again, thinking "that's it".
Like you can see, I clearly like Rust's the most. Java 7's is also good. Go's is good, but defer statements only execute on function end, not scope based. You can of course close it manually, but that sucks too.
On the other hand, Go error types are amazing to work with. errors.New("custom error") is amazing. Java's is not too bad, but Rust's is "worst" for us lazy people, but also the most powerful. Plus there are macros that fix this.

Using Travis CI with Discord

jD91mZM2 — Tue, 13 Jun 2017 09:42:06 +0000

It's a little known fact that Travis CI does not support Discord.
A long time it has bothered me, but never enough to do anything about it... Until now.
So I decided to take Techtony96's PHP script, convert it to my Go webserver.

It's simple to use!

Example: https://krake.one/travis-ci/123456789123456789/abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz

Enjoy!

Benefits of Rust

jD91mZM2 — Tue, 23 May 2017 15:49:18 +0000

As you might have noticed, Go seems to be quite the star right now. Strangely enough, I just started with it right before I suddenly heard it everywhere.

And I mean, it's a fantastic language. It was, in fact, my favorite language. Still is one of the top ones. I made a pretty huge application in it actually, which has gotten a lot of love for being from me (I'm pretty damn unknown to be honest).

So Rust then, what is it? Why am I talking about it now? Right from the start actually, my buddy @tbodt seemed fashinated about the language. None of us ever dugged very deep into it, as far as I know. I was busy with Go, pretty much.

Anyways, since I'm not professionally schooled and only a little teen hobbyist, I only just heard about data races. Of course, I hurried to update most of my applications. Including my biggest one. (All goes fine until I suddenly seem to get a goroutine leak for no apparent reason thanks to the new mutexes I put in place.)

Then I was a little bored, I looked into Rust.
My other buddy @nstafie, who I knew from the discordgo community, talked highly about it.

That's when I got into it.

Deciding to actually make a test project, I read the initial tutorial and learned as much as I could and made the tiniest test ideas I could think of.

Go vs Rust????

Remember my good ol' program that is a little buggy? I got the idea of rewriting it in Rust. Of course, I definitely wouldn't want to do that huge thing, but... yeah it was a nice thought.
Until one day, I get convinced by the discordgo community to rewrite my a little crappy code. So would I do it in Go?
Rust.
I chose Rust because it has a few tricks up it's sleeve compared to Go.

Macros. They are probably the most underrated things I've ever seen in programming. Ever wished you were able to return a function from a function? Well, macros can do just that. Plus they trade space for speed. But let's be honest, people have infinite hard drives now-a-days. Still, be careful.
Generics. This might not be something I use myself, but because of generics, it's possible for the standard library to add slightly higher functions like contains and amazing stuff.
Control. Barely any type implements the Copy "trait" (a.k.a. Rust's interface). While this makes you pull your hair out, it also makes you aware of when copies are needed, and you can see if you can avoid them. And even if you can't, which is most of the time, you can just add .clone().

But that wasn't the biggest reason

Compile time errors and warnings

One of the really cool things I saw when using Rust, was when I did something pretty stupid and compared if user input stored in an unsigned int was < 0. Guess what? Rust warned me about useless check!!

One of the best advantages with compiled languages over just-in-time compiled or interpreted ones, just got a million times better. In fact, during my whole rewrite, I haven't gotten a runtime panic one single time until today (when I tried to use a function result in a macro directly... oops)
This goes to show that you suffer for your users.

Of course, I still barely understand the error messages. While they attempt to be helpful, they are still a little big and bulky, and not always really very accurate to what you want.

Go and Rust????

Don't tell me I've switched to Rust. While we both know that's true, I like to see myself as somebody who would not be afraid of switching languages per project.
Go is definitely a cool language, and is still great for a lot of things.

Go is definitely easier to understand.
Go has a lot of things built in, which Rust doesn't. The biggest missing built-in is signal handling but other missing built-ins include lazy static, http support and parsing command line arguments and JSON.

But I loved Go. What could Rust possibly have done that could make me swi-
I didn't.
You might not see it, but Rust and Go are amazingly similar compared to other languages.

Understandable type naming. Seriously, I didn't know what the difference between float and double was until Go and Rust came around and said it's f32/float32 and f64/float64 (where the latter is Go.)
Type AFTER the name. Not the biggest deal, but it would make sense figuring out what type a variable is after figuring out which variable.
Return type last. Same as above.

Conclusion.

Sorry for making you read so much. Jeez!
In short, Rust is C + Go + convenience + (annoyance * 10).
Definitely useful for anything, but not always the right language of choice.

The rewrite I were talking about can be see here

Have fun!

Implementing translations

jD91mZM2 — Fri, 14 Apr 2017 06:18:58 +0000

As a hobby programmer, everything is your responsibility. Front end, back end, all of that.
So when a friend requested translations, I got a lot of work to do.

Let me give you some background. I started working on DiscordConsole some time ago. It's a program to take control over Discord bots, but also works for users and webhooks. It can nearly do every function Discord can, so that's the biggest issue. Every function creates new strings that needs translation.
Even with some heavy vim macros (@a to add current string to a file, @b to add current file entry back to the string), it became extremely tedious.

Now, this post isn't about how to make a simple translation system, or anything.
It's about when to start thinking about translations. C'mmon, don't make my mistake. I sat hours just moving and translating many different files, one being 1500 lines long!
Common signs you need to start thinking about translations are:

Your application is becoming unexpectedly big.
You start to write multiple similar error messages.
Similar messages become different (because you automatically vary them more and more)

Better start early than late...

This was my first post. It was very nervous to write, and I feel it's kinda useless too... So umm... Keep that in mind I guess... in case it was awful...