Forem: mkuts12

Linux simple status bar with mouse controls

mkuts12 — Sat, 23 Oct 2021 13:49:26 +0000

Recently I've grown tired of Manjaro and decided once again delve into ricing world and setup Archlinux from "scratch".
There are things I've never done before like auto-login, setting custom status bar with things I need, auto-configuring screens and sound over HDMI etc. But today I work on the status bar.

I've always been using i3wm, but by my friend's suggestion, decided to try Wayland. Sway is a WM for Wayland that is similar to i3.

It uses a similar tool to i3bar called swaybar and both have a JSON protocol that allows some more advanced configurations. I aim to set something with mouse control.

Basic script

Online I've seen some people creating a script that prints to stdout some string and that would be their status line.
In the Sway's config, in ~/.config/sway/config, we have this section

bar {
    ...
    status_command while date +'%Y-%m-%d %l:%M:%S %p'; do sleep 1; done

The breakdown:
status_command means that it would run a script to get the status line. After that comes the command
while date +'%Y-%m-%d %l:%M:%S %p'; do sleep 1; done is the command, which is a while loop with a sleep of 1 second between executions and the result is the current date, calculated by the command date.

Perfect basic example

Intermediate script

I've created a NodeJS script that bring the Battery status (I have a Thinkpad so 2 batteries for me). The script can get 2 length modes, short and long. The short is taking the percentage of all betteries and calculated the average. The long mode is giving the full breakdown per battery.

const fs = require('fs');
const path = require('path');

let battLogMode = 'short';
const sum = (a, b) => a+b;

function batteryStatus(mode) {
    const batteryPath = '/sys/class/power_supply'
    const batteries = fs.readdirSync(batteryPath).filter(a=>a.indexOf('BAT')>-1);

    if (!batteries.length) {
        return 'no baterries';
    }

    const percentages = batteries.map(batName => {
        const current =  fs.readFileSync(path.join(batteryPath, batName, 'energy_now'));
        const full =  fs.readFileSync(path.join(batteryPath, batName, 'energy_full'));
        return current/full*100;
    })
    if( mode === 'short') {
        return parseInt(percentages.reduce(sum) / percentages.length, 10)
    }
    return batteries.map((batName, i) => batName.concat(': ', parseInt(percentages[i], 10))).join(' ')
}

function isCharging () {
    const status = parseInt(fs.readFileSync('/sys/class/power_supply/AC/online'), 10)
    return status ? 'Charging' : 'Discharging';
}

console.log(batteryStatus(battLogMode), ' ', isCharging())
setInterval(() => {
    console.log(batteryStatus(battLogMode), ' ', isCharging())
}, 1000)

There's some linux wizardry and not so clean code but I can live with that.
Note that the script simply console.log a string

In the config file
We would replace the status_command line with this:

bar {
    ....
    status_command node ~/projects/status_bar/index.js

which results in a simple status bar for battery information

Now we will try to change it to use the JSON protocol instead.

Refactor to use JSON protocol

This one was a bit tricky because of the way of thinking in JS, or maybe I'm a bit slow. The protocol is along these lines:

first message to stdout is a header JSON, basically the version
The body of the communication should be an endless array, in which each element is the whole status line, i.e. it's an array of objects, and each object is a part of the status line.

In practice, you have a stream of the stdout and you send these messages:

send the header JSON
send the opening '[' of the endless array
send a JSON of an array whose elements are objects that have to have 'full_text'.
repeat 3 to update the status line with new data

Here's an example:

console.log(JSON.stringify({version:1})) // header JSON
console.log('[') // the beginning of the endless array
console.log(JSON.stringify([{full_text:'beginning'}, {full_text:'second section'}])) // the first line to be printed

let i = 1;
setInterval(() => {
    console.log(',',JSON.stringify([{full_text:i}, {full_text:'second section'}]))
    i++;
}, 1000)

If you reload the config you will see now that the status line is updating with the counter, yay.

This way we will update it to print the battery status, the script is the same but with the battery information from the beginning:

console.log(JSON.stringify({version:1, click_events: true}))
console.log('[')
console.log(JSON.stringify([{
    name: 'battery',
    full_text: batteryStatus(battLogMode).concat(' ', isCharging())
}]))

setInterval(() => {
    console.log(',',JSON.stringify([{
    name: 'battery',
    full_text: batteryStatus(battLogMode).concat(' ', isCharging())
}]))
}, 1000)

handle mouse events

So the events coming from the sway-bar are the same as the events you are supposed to send, apart from the header object. Therefore you have to skip the '[' message and then skip the comma, ',' at the beginning of each line after the first. Here's the simplest parser of messages

process.stdin.on('data', rawEvent => {
    let e = rawEvent.toString()
    if(e.length <= 2) {
        return;
    }
    if(e[0] !== '{') {
        e = e.slice(1)
    }
    const {name} = JSON.parse(e);
    if(name === 'battery') {
        battLogMode = battLogMode === 'short' ? 'long' : 'short';
    }

})

see that we know the part clicked on by looking at the name in the event and comparing it to the name given to the name property in the event we're sending.

Full script

I've added also a date section and added a console.log to the event handler, so we don't have to wait for a second to pass until a refresh

const fs = require('fs');
const path = require('path');

let battLogMode = 'short';
const sum = (a, b) => a+b;

function batteryStatus(mode) {
    const batteryPath = '/sys/class/power_supply'
    const batteries = fs.readdirSync(batteryPath).filter(a=>a.indexOf('BAT')>-1);

    if (!batteries.length) {
        return 'no baterries';
    }

    const percentages = batteries.map(batName => {
        const current =  fs.readFileSync(path.join(batteryPath, batName, 'energy_now'));
        const full =  fs.readFileSync(path.join(batteryPath, batName, 'energy_full'));
        return current/full*100;
    })
    if( mode === 'short') {
        return '' + parseInt(percentages.reduce(sum) / percentages.length, 10)
    }
    return batteries.map((batName, i) => batName.concat(': ', parseInt(percentages[i], 10))).join(' ')
}

function isCharging () {
    const status = parseInt(fs.readFileSync('/sys/class/power_supply/AC/online'), 10)
    return status ? 'Charging' : 'Discharging';
}

function generateLine() {
    return JSON.stringify([{
        name: 'battery',
        full_text: batteryStatus(battLogMode).concat(' ', isCharging())
    }, {
        name: 'date',
        full_text: (new Date()).toDateString()
    }]);
}

console.log(JSON.stringify({version:1, click_events: true}))
console.log('[')
console.log(generateLine())

setInterval(() => {
    console.log(',', generateLine())
}, 2000)

process.stdin.on('data', rawEvent => {
    let e = rawEvent.toString()
    if(e.length <= 2) {
        return;
    }
    if(e[0] !== '{') {
        e = e.slice(1)
    }
    const {name} = JSON.parse(e);
    if(name === 'battery') {
        battLogMode = battLogMode === 'short' ? 'long' : 'short';
    }
    console.log(',', generateLine())

})

and here's the config in the sway config

bar {

    status_command node ~/projects/status_bar/index.js
    separator_symbol "::"

    colors {
    separator #ffffff
        statusline #ffffff
        background #323232
        inactive_workspace #32323200 #32323200 #5c5c5c
    }
    position bottom
}

And here's the final result

Memory Management in JavaScript

mkuts12 — Sun, 21 Feb 2021 13:31:23 +0000

Note: This is the same post I've posted in Medium while I was working for Walkme.

Creating and modifying objects is something very trivial in JS, and it is continuously done in the runtime of a program. So how do the popular JS engines do it efficiently, and how does it impact your beautiful websites?

For more enjoyment from this article, you should know that the JS engine is the thing that takes JS code as its input and executes it. Chrome’s engine is V8, Firefox’s is SpiderMonkey and Edge’s was Chakra.
Objects in JS are dictionary type data structure (string to data mapping), and they are dynamic, i.e., their structure can be changed on the fly, adding new keys in the dictionary similarly to Hashmaps.

Object Creation

To kickstart the discussion, let’s look at a way to create an object.

function createPoint1(is3d) {
    var point = {};
    if(is3d) {
        point.z = 0;
    } else {
        point.z = undefined;
    }
    point.x = 0;
    point.y = 0; 
    return point;
}

Odd else, right? Not setting a value in JS is logically the same as setting it to undefined.

So let’s omit that else

function createPoint2(is3d) {
    var point = {};
    if(is3d) {
        point.z = 0;
    }
    point.x = 0;
    point.y = 0; 
    return point;
}

Better, right? And you might think that both should be the same,
performance-wise, or even that the second is better since it’s shorter, but the JS engine teams work hard to make it so one of them is better than the other. To understand why, we need to look at how objects are mapped to memory.

Memory?

Computer memory is like a huge array. Every time you create a new something (array, object, function) anywhere in your code, the engine asks the operating system (OS) for a slice of memory to store your creation in.

When you create an array with 3 slots for 3 integer numbers, the OS allocates you the exact space you need, but if you want to add a fourth number to that array, the OS cannot just expand the already given memory, so the engine has to ask for new space for an array with 4 slots and move the data there. If you do this over and over again it takes many priceless CPU cycles.

This creates overhead from 3 main causes:

It takes a while for the OS to find you a big enough chunk (if it can’t, it needs to defrag the memory, that is to reshuffle the allocated blocks to make up some sequential space),
It takes time to copy,
It takes time to clean the old slice the engine does not need anymore (garbage collection).

These are the limitations imposed by handling dynamic data in software. So the engine does its best not do this much, here’s how.

It’s an Array! No! It’s a string! No! It’s an OBJECT!

When creating an object, it still has to be represented in memory as an array of values. There is no other way to represent in memory, since memory is an array-like structure.

So, how does the engine know how to direct a property to its value?

Let’s assume for a moment that we are in a different world, a statically typed (there are types, and they don’t ever change) and compiled world, where we can write something like this:

type Point {
 int x,
 int y,
 int z
};

const p1 = new Point();
p1.x = 12;
p1.z = 1;
console.log(p1.x);

A Point has 3 parameters, ‘x’, ‘y’, and ‘z’. You can create a point and edit the value of its parameters.

With the additional information about the type and the assumption that objects of type Point never change their structure, a compiler can look at this code and change it to this:

const p1 = new Array(3);
p1[0] = 12;
p1[2] = 1;
console.log(p1[0]);

The compiled code changes every creation of a Point to the creation of a new array with 3 slots, and remembers that the first slot is for ‘x,’ the second is for ‘y’ and the third is for ‘z.’

Logically, the two code snippets are the same, but the compiled code allows the engine to map array to memory-array and never have to worry about changes to the structure, etc.

Note: this is similar to what Bucklescript does for ReasonML.

Back to JS Objects

Since in JS we can add properties dynamically, the JS engine tries to “remember” the mapping of the property names to their place in the object array in memory.

So, how does this magic work?

Each object is stored with a mapping, which is a special structure. The first time an object is created, the engine asks for a big slice of memory and saves it with an empty mapping.

p = {}

Note: it asks for a big slice so it can store many properties before expanding. The slice is later shrunk to the right amount.

When you add a property, let’s name it x,

p.x = 1

it saves the value in the first place in the array and changes the mapping of the object properties to a mapping that says:

x -> first

In the memory it might look something like this:

The object’s data array and the mapping array might be stored in different places in the memory, as illustrated in the image. The left array is the mapping, and the first place holds the string “x” in it, indicating that the property with this name is stored in the first place.

So when the engine goes to see what is in the object’s property with the name “x”, it will fetch the mapping, see that “x” is in the first place, then fetch the data array and return the first value.

If you add a property, p.y = 14, it saves the value in the second place and changes the mapping to this:

x -> first
y -> second

And when you ask for “y” in an object, the engine looks at this mapping and returns you the second value in the array.

Since the data and mapping are separate entities, the engine can reuse the mapping for similar objects, if it wants, and the engine sure does want it.

Mapping Reuse

Creating a new mapping is costly, probably just as creating objects. The mapping is also stored somewhere in memory, and each time we add a property, the engine has to create a map object, move the contents of the old mapping to the new and bigger one and then add the new property.
But that’s exactly what we tried to avoid by using mappings!

So engines do 2 things to avert that:

They reuse the old mappings.
They don’t store the whole mapping but only incremental changes with a link to the previous mapping, similar to a linked-list or a tree with 2-way links.

Let’s look at an example,

1: var point = {};
2: point.x = 1;
3: point.y = 14;
4: var vector = {};
5: vector.x = 1;
6: vector.y = 20;

In the first line, the engine creates an empty mapping, M0, just as before. In the second line, it creates a new mapping, M1, that holds the information that x is in the first place in the data array, and changes M0 by adding a special line in the mapping:

M0:
x -> transition to M1M1:
x -> first

Now our object point has the mapping M1 linked to it. So, if you want to read x the engine will go to M1 see that x is in the first place, go to the data array and return the first value. Just like before.

In the third line, it creates a new mapping, M2, and changes M1 like this:

M0:
x -> transition to M1M1:
x -> first
y -> transition to M2M2:
y -> second

So now when reading y on point it knows immediately that it’s in the second place, but when reading x it will see that it’s not in M2 and go upwards to M1 , see that x exists here and return the first place in the value array.

Here’s the example again,

1: var point = {};
2: point.x = 1;
3: point.y = 14;
4: var vector = {};
5: vector.x = 1;
6: vector.y = 20;

When creating vector in line 4, it uses M0 as the initial mapping for vector. On line 5 it checks M0 whether somebody already added x to M0. Since M0 has a line indicating the transition to M1 when adding x, it saves the object with M1 as the mapping. A similar thing happens when adding y to M1. The engine reuses M2 since it's referenced by M1 when adding y.

So now both point and vector have the same mapping. The engine reused both the memory space and saved the time needed for creating these mappings :)
This way the mappings are as small as possible, and we can create the least amount of mappings.

Here’s an illustration of the memory, if we had a small memory of 20 slots.

To simplify graphs in the future, I will show the mappings as a tree:

M0 -> M1 -> M2

And I will use this illustration to show the relation transitions between mappings and describe each mapping separately.

Fool me once, shame on you!

How does it all connect to the example from the beginning?

Let’s look at the mappings tree of the first example when calling it twice.

function createPoint1(is3d) {
    var point = {};
    if(is3d) {
        point.z = 0;
    } else {
        point.z = undefined;
    }
    point.x = 0;
    point.y = 0; 
    return point;
}
var p1 = createPoint1(true);
var p2 = createPoint1(false);

The mapping for both p1 and p2 is the same m3 :

M1:             M2:             M3:
z -> first      x -> second     y -> third

M0 -> M1 -> M2 -> M3

The initialization process is the same no matter how we call it. z is always initialized first to something, then x and then y.

In comparison, let’s look at the mapping of the second example when calling it twice,

function createPoint2(is3d) {
    var point = {};
    if(is3d) {
        point.z = 0;
    }
    point.x = 0;
    point.y = 0; 
    return point;
}
var p1 = createPoint2(true);
var p2 = createPoint2(false);

The mapping for p1 is just as before,

M1:             M2:             M3:
z -> first      x -> second     y -> third

M0 -> M1 -> M2 -> M3

But something different happens after creating p2.

M1:             M2:             M3:
z -> first      x -> second     y -> third

M4:             M5:
x -> first      y -> second

M0 -> M1 -> M2 -> M3
  `-> M4 -> M5

When the function skips the initialization of z on the new object, it goes
from empty to x. Since such a transition does not exist on M0, a new mapping is created and M0 gets a new transition to the mapping.

We almost doubled the mappings because the object created by createPoint2(false) was of a different structure. M1 describes a similar object to M4, both say that x is in the first place in the data array, so we wasted memory real-estate.

Real life implications

To illustrate how important it is to be aware of this, look at these two snippets:

function createSlowly(a,b,c) {
    var vec = {}
    if(a) {
        vec.a = Math.random();
    }
    if(b) {
        vec.b = Math.random();
    }
    if(c) {
        vec.c = Math.random();
    }
    vec.d = Math.random();
    vec.e = Math.random();
}
function createQuickly(a,b,c) {
    var vec = {}
    vec.a = a ? Math.random() : undefined;
    vec.b = b ? Math.random() : undefined;
    vec.c = c ? Math.random() : undefined;
    vec.d = Math.random();
    vec.e = Math.random();
}
for( var i = 0; i < 5000; i ++ )
    createSlowly(i % 3, i % 4, i % 5);
for( var i = 0; i < 5000; i ++ )
    createQuickly(i % 3, i % 4, i % 5);

The function createQuickly runs 50% faster than createSlowly.

The core reason is what we touched here. The engine has different mappings for these objects and different parts of the engine that rely on similar objects to have the same mapping fail to predict the code well and are de-optimized.

Summary

In this article, we saw that we should mind the way we create objects in JS. That JS engines use mappings to deal with the dynamic nature of objects and one optimization that is done to make sure we don’t flood our memory. Lastly, we saw a relatable object initialization method that impacts the performance of our glorious sites.

I hope you understand the reason mappings exist and how they work in state of the art JS engines. Keep in mind that even the smallest things, like removing an “else” clause, might affect your performance.

Thanks to Omer Tellem, Yonatan Behor,Yonatan Kra and Udi Doron for their great help :)

Better Understanding of Vim colors

mkuts12 — Sun, 21 Feb 2021 12:33:50 +0000

I am a programmer with simple needs in regards to Vim, I need it to be Vim with and it should look nice. I've been using only basic plugins, mostly surround and some highlighting, for years now. One of the reasons for that is the fact I came from working on servers, so I don't have my .vimrc with me. Another factor is that I write a lot in Lua/Javascript, so there's not much I can get by autocompletion anyways.

Lately I've been working on some Typescript codebase, and a coworker was reviewing the code with me and we wanted to check the types of some object. It is not so nice without some tsserver to just tell us. So I've decided to go forth into the rabbit hole of Vim configurations once again.

I will skim over the TS configurations, mainly sharing some links and then go into the hightlight

There's a TL;DR at the end to get the bare gist.

God saw that the light was good, and he separated the light from the darkness

I use Mac at work, Yabai as a tiling window manager, Kitty as a terminal emulator, and bash as my shell.

I use terminal a lot, as a file explorer, running scripts and for text editing. I use light themes and I've been looking at my Solarized Light screen for over 3 years now.

I've picked a theme for Kitty,

background            #fdf6e3
foreground            #52676f
cursor                #52676f
selection_background  #e9e2cb
color0 #e4e4e4
color8 #ffffd7
color1 #d70000
color9 #d75f00
color2 #5f8700
color10 #585858
color3 #af8700
color11 #626262
color4 #0087ff
color12 #808080
color5 #af005f
color13 #5f5faf
color6 #00afaf
color14 #8a8a8a
color7 #262626
color15 #1c1c1c
selection_foreground #fcf4dc

This said little to me but I liked what I saw. Sometimes some texts were white, usually when running tests, but I would just select the text and read what it said.

When configuring the TS, I've followed this guide. I've had to install NVim, coc-tssever for that tsserver inategration and autocompletion and added some mappings

nmap <silent> gy <Plug>(coc-type-definition)
nmap <silent> gs :call CocAction("doHover")<CR>
autocmd FileType typescript,javascript,typescriptreact,javascript.jsx nnoremap <buffer> <leader>t :call CocAction("doHover")<CR>

Especially the gs map, that should show me a little hovering window with the type of the object the cursor is on. Here was how it looked.

There was this little issue that the text I wanted to read was unreadble.
I've changed the theme to something else and it was alright but something else was broken. I want that CursorColumn and CursorLine (hightlihgts the row and column) but both were just white.

And here I started reading a lot about Vim highlights and color themes and all that, I will try to pass as much information but be as short as possible.

Off to a great adventure

I've used the ayu light theme and in the project it had a picture that looked like that

and it's generated by this tool

#   Copied from http://tldp.org/HOWTO/Bash-Prompt-HOWTO/x329.html
T='gYw'   # The test text

printf "\n         def     40m     41m     42m     43m     44m     45m     46m     47m\n";

for FGs in '    m' '   1m' '  30m' '1;30m' '  31m' '1;31m' '  32m' \
           '1;32m' '  33m' '1;33m' '  34m' '1;34m' '  35m' '1;35m' \
           '  36m' '1;36m' '  37m' '1;37m' '  38m' '1;38m';

  do FG=${FGs// /}
  printf " $FGs \033[$FG  $T  "

  for BG in 40m 41m 42m 43m 44m 45m 46m 47m 48m;
    do printf "$EINS \033[$FG\033[$BG  $T  \033[0m";
  done
  echo;
done
echo

Basically it shows text colors and backgrounds, according to some 8 base colors. The column and row codes are some Ansi codes so you can use them in your bash customizations ;)

Using this theme I got the part more readable,

But then I wanted the cursorline and cursorcolumn to work well.

X marks the spot

Here how it looks now,

Notice that the column is white, and the line is just an underline.

You can notice in the picture of the theme, that the default is grayish color, and the last column is white.

I recalled that 8 is a number that I've seen here already, and indeed, it was the number of pairs in the theme config.

According to the Kitty configurations, the color scheme comes in pairs, one color for the dark variant, and one for the light variant, and the last pair is the pair for white, dark white and light white.

So I've concluded, and checked the theme that the designer decided to set the "dark white" (gray) as white, #ffffff, same as "light white" (regular white).

So I've changed color7 to #d3d3d3 and now it looks great.

Regarding the line issue. I've read here about changing specific colors of specific things in Vim.

Hit :hi to see the table of all highlight rules,

CursorColumn  ctermbg=7 guibg=Grey90
CursorLine    cterm=underline guibg=Grey90

The image also shows you how it would look <3. So I see that the CursorColumn has 2 values, ctermbg and guibg. Testing this a bit with different values, and reading this thread made me understand this:

guibg is not terminal, it's probably the state that macvim or gvim runs.
there are probably configs for gui similar to cterm
the number in ctermbg is not a magic number, it's the number of the color from the theme file, i.e. the color7 definition
cterm is probable color terminal and it's for terminals that support 256 colors, term is for simpler terminals.

So just to play with it a bit more, I've added 2 more colors in my theme

color16               #aaedbc
color24               #a4e8b6

It's light green gray, 16 is dark and 24 is light, I think.

Then added this to my nvim/init.vim file (I miss just saying vimrc),

hi CursorLine cterm=bold ctermbg=16 guibg=DarkGrey

and now it looks like this

TL;DR

Basically,

Themes are basically setting some env. variables, color1-16, that set the color definitions
they come in pairs, 0+8, 1+9 etc, when the lower is the dark variant and the high is the light variant, e.g. color0+8 are black, 1+9 are red, 2+10 are green. Further reading
you can run the script given above to generate a table of all color configurations for the basic 8, and see if everything is readable and makes sense to you.
Vim uses these colors in it's hightlight definition
Run :hi to see your defintions, you will need to find the configuration you want to change, like CursorLine for me
the ctermbg setting sets the background in color terminals, and the value is the number of color that corresponds to the env. variable, i.e. if you set there 7, it will take the variable color7.