Forem: Abhishek Adhikari

OS Fundamentals 101: Process and Syscalls

Abhishek Adhikari — Fri, 31 May 2024 14:35:02 +0000

Hola! Let's dive into the world of operating systems. Today, we will discuss processes and syscalls.

Process

First, let's quote the bookish answer.

A process is a program in execution.

When we want to execute any program, the OS creates an entity called a process. A process is a kind of container to hold everything needed by the computer to execute it. Each process is given its own address space, a list of memory locations from 0 to some maximum depending on available physical & virtual memory, which the process can read and write to. The address space contains the executable code of the program (mostly assembly), the program's data, and its stack. A process is also associated with allocated resources like registers (for things like program counter, stack counter, etc.), file descriptors, list of related processes, etc. All this information of the process is stored by the OS in the process table.

Address Space

The address space is a map of memory locations from 0 to some max value to actual physical locations on the RAM. When the OS creates a process it allocates an address space where it first stores the location of OS procedures in the beginning, as the program will need them to talk to the Kernel (read-only). Next, it reads the executable code and stores it (read-only), followed by the data segment which contains things like the globals and is both read and writable by the process. Then, the remaining space is divided by heap and stack. The heap is allocated after the data segment whereas the stack is allocated at the end. So the heap grows upwards whereas the stack grows downwards with free space in between.

This address space is also referred to as the core-image.

System Calls

System calls or syscalls is an interface provided by the OS to allow user-space code to talk to the Kernel and instruct it to do specific tasks.

Let's take an example of reading a file. If a process is running in user-mode and it wants to read a file from hard disk which is a system service (takes place in kernel-mode), it issues a read system call. How? To make a system call, the process first sets a few parameters for the system call, then calls the read procedure call which sets the code for read in the register and then issues a trap command, which causes the OS to trap the control to the Kernel. The Kernel then reads the code and figures out the syscall to be executed, and dispatches it to Sys call handler which executes it, and then the control flows back to user-space process.

Learning to draw these kinds of diagrams, pls bear with this one 😜.

Parent/Child Process

Let's take an example where we open a shell and type "htop" and press enter.

Firstly, we only have the sh (shell) process in the process table.

After we type "htop" and press enter, the shell process uses a syscall fork() to create a copy of itself. Fork also copies the registers, fds (file descriptors), address space, etc. This creates a new shell process in the process table which is a child of the shell that used the fork command (the parent).

Fork creates a new process while copying the resources of the parent process. It is to be noted that the address space content is copied and not the locations, i.e., changing any value in the child process won't affect the parent process. Some systems may not copy all the content and rather implement COW (Copy-On-Write) but that's a different topic altogether.

Now, this forked shell process searches the path of this command and if found, it calls the exec() syscall with the user-entered command as the argument. Exec now takes the executable code of "htop" and puts it into the memory replacing the shell code. In a nutshell, it replaces the shell with htop while maintaining the context of the existing process, that is the data remains, address space remains, and it even maintains the PID (Process Identifier).

Now, htop does its thing and after it exists, the entry is removed from the process table, all the resources are released and the parent shell wakes up.

Zombie/Orphan Process

A zombie process is a process that has completed its execution but is still present in the process table. This happens when the parent doesn't "reap" the process. This generally happens when the parent process doesn't call waitpid() syscall to wait for the child to complete its execution, and read the exit status of the child. During the waitpid() call the OS also removes the entry of this child from the process table.

When a child process dies, the parent receives a SIGCHLD signal.

Kill command doesn't work on the zombie process.

An orphan process is a process that is running even when the parent process has terminated. This may happen if the parent process doesn't wait() for the child to finish after fork()ing or system crashes.

A zombie process has completed execution but its parent is still running, whereas in the case of an orphan process, the parent has already exited but the child is still running.

A zombie process doesn't hold any resources except for the data needed to store its entry process table, whereas an orphan process is still running and holds the resources.

Let's end here for today. In the next article let's discuss storage and filesystems.

Please comment below suggesting any changes, asking for any topic, or just hanging out in general. Also, pls reach out to me on my social channels.

[GitHub] [LinkedIn] [Instagram] [YouTube]

Networking 101: Back to School

Abhishek Adhikari — Sat, 25 May 2024 06:34:04 +0000

Hola! Hope your networks are stable and reliable. In this article let's get Back to School (the Basics), in order to later get into advanced networking topics.

What is a network?

A network is interconnection between multiple devices enabling them to share data between each other. An example:

Hmmm... The network doesn't look that impressive. Right?

Maybe this is better.

Still not impressed. You must be thinking where the networking devices like a router or a switch or a bridge are, something should be there. But actually, for the above scenarios these networks will just do fine. There is no use of overengineering stuff and bringing-in extra networking devices which serves no actual purpose but can be a single-point of failure.

Why internetworking? Why not a huge network like this?

Let's now understand why we don't create a huge network of directly/indirectly connected devices, but instead have interconnected smaller networks. And how broadcast and collision domains affect this decision.

Collision

Let's understand what a collision is. In a network when two devices want to send data, they send signals (electromagnetic along the transmission medium). Now, if two devices try to send data at the same time in the same medium, the signals mix up or collide with each other and there is signal loss. When a collision occurs all the devices on the network has to pay attention to it, because nobody knows what the destination on the data signal that was collided. Also, now the data has to be re-transmitted. This increases computation for all devices, increases latency in the network.

So, for any device the smaller the size of collision domain is, lesser is the chance of any collision happening.

Broadcasting

Now, let's understand what is broadcasting. In broadcasting, a device is trying to send some data across the network, so that multiple devices can receive and acknowledge it. On a bigger network, there may be a few devices in this case who doesn't need this data. But still they have to see it, receive it, acknowledge it, check if it needs this data and lastly just drop the data received. For this device, we want to limit the broadcasting domain only to the devices that it actually needs to talk to, while will reduce the computation getting wasted on dropping un-intended data.

Another common problem...

Congestion, it is the thing we as network admins have to always look after and try to reduce. Congestion is similar to traffic jams on the roads. Whenever, collision occurs the packets have to be retransmitted, on unnecessary broadcasts for the unintended devices there are packets travelling on the network, it will have to wait to send its own data. These are the common examples of congestion. Our goal is to somehow reduce this. Now, let's try to find ways of doing. For that we need to introduce few networking devices, so first let's learn about them.

Understanding the devices

Domains. What is a domain? A domain is a region or in this case set of devices.

Types of domains in networking:

Collision Domain
A collision domain for a device is the set of devices who if try to send a signal at the same time as this device, the signals collide.
Broadcast Domain
A broadcast domain for a device is the set of devices who receives the packet broadcasted by this device.

Collision domain is always a subset of the broadcast domain.

Hub

A hub is a dumb network device with several ports used to connect multiple devices or small networks together. In a Hub, a packet received in any port is re-transmitted to the remaining ports. It doesn't read the packet or anything, it just regenerates the signals received on one port to the remaining that's it.

The devices A, B, C are all connected to this hub. Now, let's take an example where A wants to transmit to B. It sends a packet out on the network; it reaches the hub, and the hub retransmits on the remaining ports. So, B receives the packet successfully. Cool right? Not actually, because C will also receive the packet now it will have to do the computation that the packet was not intended for it and drop which was not something cool for device C. Here, both the collision domain and broadcast domain for A was {A, B, C} only.

Hubs are cheap but always lead to congestion.

Bridge

A bridge can be called an intelligent hub. The term "bridging" here simply refers to packets bridging over ports. Here, when a packet arrives in one port like hub, instead of being retransmitted to all the ports, the bridge transmits that to the port which it is destined for. I will not be going to show any bridge diagram below as nowadays we don't buy physical bridges. The bridge technology is rather used in switches and routers which we will talk after this.

Cisco sometimes refers to their switches as multiport bridge, but now you won't get confused why.

Bridges can be used to reduce collisions in a broadcast domain. It increases the number of collision domains and hence provides higher bandwidth for devices.

Switch

A switch is an intelligent device which exactly knows which device or network is connected to each port and smartly routes/bridges/switches the packets between incoming and destination ports. Here each port creates its own isolated collision domain thus reducing congestion in the network. But still as its just switching packets in same network, the broadcast domain remains same.

Let's modify our previous example to make it better.

Now let's analyze the devices.

Device B:
It is directly connected to a port on the switch. So, its collision domain is isolated to the port itself. That is the collision domain for it is {B}. That means collision will never occur hence using maximum available bandwidth to the switch. Can it talk to A? Yes, the packet first reaches the switch, the switch checks the destination of the packet and decides to send it to the port in which the hub is connected. The packet then arrives at the hub, and it transmits it to both A and B. Here, A successfully receives the packet, but B also receives it and then drops it.

Device A, C:
Both of them are connected to the hub, therefore their collision domain is {A, C}. They both can obviously talk with each other and can also talk to B via the reverse of the route we just discussed before.

In this case, the broadcast domain remains same {A, B, C} for all the devices.

Router

A router is a switch only. The difference is that a switch switches packets in a single broadcast domain. But a router is used to switch packets across multiple broadcast domains. Explained simply, a router is used to connect multiple networks (this also means it can connect several smaller networks or sub-networks or subnets within a bigger network. This can also be phrased as it can connect multiple broadcast domains within a wider broadcast domain). How a packet in one subnet would go another subnet, you may ask? Routers do something called NAT (Network Address Translation) to translate the source, destination addresses of the packets from one subnet to another. We will discuss about NAT in a later article.

Let's modify our example before. We had assumed that all the devices are in a same room. Let's modify it in a way A, C remains in one room and B is in another, each room using its own subnet. Also, as we got the chance to modify our network, let's make it better by replacing the hub with a switch.

Let's again analyze the devices.

Device B:
It is connected to a port on the router and therefore has an isolated collision domain {B} eliminating congestion. Can it talk to A or C? Yes, the packet the first goes to router the packet is translated to the subnet of Room 1 and then sent to the switch, the switch then switches the packet only to the intended device. Here broadcast domain is only {B}

Device A, C:
Both of them are connected to the switch. A and C, they have their own isolated collision domains {A} and {C} respectively that they themselves are only present eliminating any collision. Can it talk to B? Yes, again through the reverse of the route discussed before. Broadcast domain for both of them is {A, C}.

Switches create separate collision domains within a single broadcast domain.

Routers provide a separate broadcast domain for each interface. Don’t let this ever confuse
you!

Let's end here for today. In the next article let's discuss network topologies, internetworking models, and a few common protocols.

Please comment below suggesting any changes, asking for any topic, or just hanging out in general. Also, pls reach out to me on my social channels.
[GitHub] [LinkedIn] [Instagram] [YouTube]

Flakify your code as NixOS Packages and Modules

Abhishek Adhikari — Sat, 03 Dec 2022 07:45:22 +0000

Introduction

Recently I had to flakify a Django app so that it outputs a few packages and modules which can be installed and configured in the NixOS System. I needed to figure out how to get started. After a week of searches and reviewing other nix packages, I made it work.

Today, I'll try to explain whatever I have learned in this process. I may be wrong somewhere or maybe there's a better way, so feel free to correct me or share your knowledge in the comments section.

Why Flakes?

When using normal Nix packages the derivation is fetched from the current nix-channel set in the system. So, we may sometimes arrive at a situation when we need to change the nix-channel for fetching a particular package. Now, several such packages from different channels quickly start building a mess. And making sure the nix-channels are in sync becomes hard. Flakes resolves this problem as we specify the nixpkgs version to use by providing its URL in the inputs.

In general, flakes make things a little more declarative and easy to understand and maintain. Also, the dependencies get pinned to a lock file, which makes sure that the flake output is consistently reproducible, the main moto of Nix.

What are we going to do today?

Today we will be writing a script that prints a passed argument along with the current time after every 30 seconds.

Setting up our project

First, let's initialize git.

git init

Now we will set up a basic flake by running the command

nix flake init

It will create a basic flake.nix file.

Now let's create your bash script. First, we will create a folder called "src", the name can be anything but "src" clearly tells that it contains the source files.

Now we will create the derivation for the package. I like to create a folder called "pkgs" and put the derivation there.

Let us understand the derivation here.

pname = "nix-flake-demo";

We are defining the package name here. The package will be available to the path by this name.

outputs = [ "out" ];

We can have more outputs for different purposes, the out is a default one that links to the directory in the nix store for this derivation.

src = ../../src;

The location of the source files.

buildInputs = [ ];

We don't need anything else to build our derivation as this is a simple bash program. But for example, if we had a flask app, then we would have to pass python along with the flask package to the buildInputs.

installPhase = ''
    mkdir $out
    mkdir $out/bin
    cp -rv $src/* $out/bin
    chmod +x $out/bin/nix-flake-demo
'';

We create our output directory and a bin directory inside it. Then we copy our source files there and make the script executable.

One thing to note here is that there should be an executable in the output bin directory of the name same as that of the derivation. If the script is named something else in the source just rename it in the installPhase.

There are several more Phases in mkDerivation. I am not mentioning the rest as I don't have much experience in them.

You can read more about the different phases here.

Now we can edit the flake to output the derivation as a package.

Let's understand the flake code till now.

inputs = {
  nixpkgs.url = "github:nixos/nixpkgs/nixos-22.05";
  utils.url = "github:numtide/flake-utils";
};

We are providing the URL of the nixpkgs to be used in this flake. Also, we are providing flake-utils.

packages =
      let
        system = "x86_64-linux";
        pkgs = import nixpkgs { inherit system; };
      in
      utils.lib.eachDefaultSystemMap (system: {
        nix-flake-demo = pkgs.callPackage ./pkgs/nix-flake-demo { };
      });

We are outputting nix-flake-demo for all types of systems.

defaultPackage.x86_64-linux = self.packages.x86_64-linux.nix-flake-demo;

Then we specify the defaultPackage of this flake. So when we run the command nix run, it will run this package.

Let's test our flake till now. Don't forget to commit the changes first. Then run the command

nix run

The script prints only the time as no argument was passed to it.

Now let's create the module which provides a systemd service to NixOS. I like to keep the code for that in the "modules" directory.

We provide a few options that can be configured in NixOS configuration. And if the enable option is set to true, we add the service to the system.
If you don't know how to add a flake to your NixOS configuration, let me know, and I will try to create a post on that.
I have a boilerplate to set up nix flake. Please visit this repo:

nix-pkg-module-boilerplate

A fork and star will be highly appreciated.

Thanks for reading till the end. Hope I was able to teach something new. Please provide your feedback or request topics in the comment section.

With ❤️ by Abhishek Adhikari.

Get a type-writer for your Portfolio site 😎

Abhishek Adhikari — Mon, 08 Jun 2020 09:23:14 +0000

Wanna get a type-writer effect working on your portfolio to get an awesome impression on new visitors. Then you're at the right post. Today, I will exactly show how to get this working. Ok, so this is what we are trying to achieve today.

To get a live view of this thing working you can visit my portfolio site at https://typewriter.lordvader.me/ .The link to whole source code will be available at the end of this post.

So, let's get started.

First, we need to set-up our html.

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>. . .</title>
    </head>
    <body>
     .
     .
    </body>
</html>

Now, lets a type-writer effect style heading inside the body of html. We'll have our sentences as parameters of div tag and the actual text content of the div empty. We'll then fetch these sentences by javascript and inject it into the empty tag that we'll put there.

<h1>
  <div class="typewrite" data-period="1000" data-type='[ "Hi, I&#39m &#128526; Abhishek Adhikari.", "&#128514; You just copied this code, right?", "No worries.", "👨‍💻 Just go through the code you will get it.", "Jai Hind !!! " ]'>
    <span class="wrap"></span>
  </div>
</h1>

Now, we'll get our javascript code to fetch the sentences from the html and pass it into empty tag.

window.onload = function() {
    var elements = document.getElementsByClassName('typewrite');
    for (var i=0; i<elements.length; i++) {
        var toRotate = elements[i].getAttribute('data-type');
        var period = elements[i].getAttribute('data-period');
        if (toRotate) {
          new TxtType(elements[i], JSON.parse(toRotate), period);
        }
    }
};

Now, we'll create the TxtType function for typing effect.

var TxtType = function(el, toRotate, period) {
    this.toRotate = toRotate;
    this.el = el;
    this.loopNum = 0;
    this.period = parseInt(period, 10) || 2000;
    this.txt = '';
    this.tick();
    this.isDeleting = false;
};

TxtType.prototype.tick = function() {
    var i = this.loopNum % this.toRotate.length;
    var fullTxt = this.toRotate[i];

    if (this.isDeleting) {
    this.txt = fullTxt.substring(0, this.txt.length - 1);
    } else {
    this.txt = fullTxt.substring(0, this.txt.length + 1);
    }

    this.el.innerHTML = '<span class="wrap">'+this.txt+'</span>';

    var that = this;
    var delta = 200 - Math.random() * 100;

    if (this.isDeleting) { delta /= 2; }

    if (!this.isDeleting && this.txt === fullTxt) {
    delta = this.period;
    this.isDeleting = true;
    } else if (this.isDeleting && this.txt === '') {
    this.isDeleting = false;
    this.loopNum++;
    delta = 500;
    }

    setTimeout(function() {
    that.tick();
    }, delta);
};

And to get a cursor we will add a border-right to the tag while injecting the css when the javascript loads.

var css = document.createElement("style");
css.type = "text/css";
css.innerHTML = ".typewrite > .wrap { border-right: 0.10em solid #fff}";
document.body.appendChild(css);

By this time the type-writer is already working. But we'll add little css to get it look better. Putting all this together the code becomes.

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Typewriter-effect</title>
        <style>
            body {
                width: 100%;
                height: 100%;
                background-color: rgb(255, 94, 0);
                overflow: hidden; /* to hide scroll bars nothing to do with type-writer */
                text-align: center;
            }
            h1 {
                padding: 20% 0;
                color: white;
            }
        </style>
    </head>
    <body>
        <h1>
            <div class="typewrite" data-period="1000" data-type='[ "Hi, I&#39m &#128526; Abhishek Adhikari.", "&#128514; You just copied this code, right?", "No worries.", "👨‍💻 Just go through the code you will get it.", "Jai Hind !!! " ]'>
              <span class="wrap"></span>
            </div>
        </h1>
        <script>
var TxtType = function(el, toRotate, period) {
    this.toRotate = toRotate;
    this.el = el;
    this.loopNum = 0;
    this.period = parseInt(period, 10) || 2000;
    this.txt = '';
    this.tick();
    this.isDeleting = false;
};

TxtType.prototype.tick = function() {
    var i = this.loopNum % this.toRotate.length;
    var fullTxt = this.toRotate[i];

    if (this.isDeleting) {
    this.txt = fullTxt.substring(0, this.txt.length - 1);
    } else {
    this.txt = fullTxt.substring(0, this.txt.length + 1);
    }

    this.el.innerHTML = '<span class="wrap">'+this.txt+'</span>'; //the text for each step is inserted inside the <span>

    var that = this;
    var delta = 200 - Math.random() * 100;

    if (this.isDeleting) { delta /= 2; }

    if (!this.isDeleting && this.txt === fullTxt) {
    delta = this.period;
    this.isDeleting = true;
    } else if (this.isDeleting && this.txt === '') {
    this.isDeleting = false;
    this.loopNum++;
    delta = 500;
    }

    setTimeout(function() {
    that.tick();
    }, delta);
};

window.onload = function() {
    var elements = document.getElementsByClassName('typewrite'); //the typewrite tag from html is linked to elements
    for (var i=0; i<elements.length; i++) {
        var toRotate = elements[i].getAttribute('data-type'); //all the sentences are stored here
        var period = elements[i].getAttribute('data-period');
        if (toRotate) {
          new TxtType(elements[i], JSON.parse(toRotate), period);
        }
    }
    //once this function starts working
    //the css for .wrap <span> is injected to the DOM
    var css = document.createElement("style");
    css.type = "text/css";
    css.innerHTML = ".typewrite > .wrap { border-right: 0.10em solid #fff}";
    document.body.appendChild(css);
};
        </script>
    </body>
</html>

Hope, you enjoyed following this post and got your own type-writer effect working. Here's what we've created today
https://typewriter.lordvader.me/ .

Here's the repository to this tutorial.

sith-lord-vader / typewriter-effect

typewriter-effect

Here's the repository. Just clone it to get started right away. 😎

View on GitHub