Forem: Albert Jokelin

Turbocharged C: How to engineer a Sub-50ms Program!⚡🚀

Albert Jokelin — Sun, 02 Feb 2025 17:03:24 +0000

Imagine working on several programs that together must route messages across 5 devices in less than 50ms!

That's what I was working on at my workplace for the last couple of months.

A bit of context here- we work on Backend data transfer technology that routes data across hundreds of kilometres using optical fibre.

There comes a time when those links fail and we must route that data through an alternative path. Because we are dealing with terabytes of data every second, this must be done quickly; within milliseconds.

This is how the flow works:
1) The framing device raises an alarm when it stops receiving data.
2) The switching device processes this alarm and sends relevant information about the connection to the switching device of the alternate path.
3) The alternate switching device starts processing data.

All of these steps have to be completed within 50ms. Needless to say, our solution worked but not within the intended timeframe and that's when we got to debugging.

Since our switching devices communicate using ethernet packets, we had to determine whether the number of packets transmitted made any difference. Surprisingly, it did. We expected the message to be broadcast across the system rather than sent specifically to a single device, but that wasn't the case.

Debugging the system

In my opinion, the best way to debug any system (especially embedded ones) is to break them into the smallest possible units and thoroughly examine each one.

We started by adding timers to the operations essential to the switching mechanism in all the devices. Since we were using C, we used the built-in timer.

This method highlighted two issues- the processor we used had too many operations going on and there was an avoidable time delay when the switch received the data and processed it.

How did we resolve it? Well, the first was fairly straightforward. Since the switching mechanism had different threads, our first approach was to keep the process out of the general scheduler and have a single core work on it. However, our processor had only two cores which meant all the other processes hogged up the first core and lesser urgent but important tasks weren't completed on time.

The next best solution was to increase the thread priority which worked reasonably well.

The second issue was challenging to solve because we had to dive into the switch manufacturer's code and figure out how they sent ethernet packets.

After a week of going through the code, we found out that messages received by the switch were stored in a queue (an elegant solution, realization should have struck earlier IMO).

Now, there are two ways to retrieve messages from the queue- either the processor polls or the queue pushes. By default, our processor was set to polling which explained the time delay between receiving and processing the messages. We switched to an interrupt/push-based mechanism and voila! It worked.

Although the above optimizations did work, what made the program execution faster was using highly optimized algorithms and structures like hashmap. Their effect was noteworthy with times reaching as low as 8ms!

All of Reacts concepts- one article

Albert Jokelin — Sun, 30 Jun 2024 09:30:36 +0000

React, developed by Facebook, has transformed web development with its component-based architecture and efficient rendering. However, mastering React requires a solid grasp of its core concepts.

In this comprehensive article, we'll explore all the essential aspects of React. Whether you're a beginner or an experienced developer, this guide covers everything from components and state management to hooks and lifecycle methods.

By the end, you'll have a thorough understanding of React's ecosystem and be ready to build sophisticated, maintainable applications.

Components

A function that returns markup (JSX). Kind of like legos, basic building blocks of a react app.

function Profile() {
  return (
    <img
      src="https://i.imgur.com/MK3eW3As.jpg"
      alt="Katherine Johnson"
    />
  );
}

export default function Gallery() {
  return (
    <section>
      <h1>Amazing scientists</h1>
      <Profile />
      <Profile />
      <Profile />
    </section>
  );
}

Curly braces

Allows react to be dynamic by allowing values to pass through it.

 <img
      className="avatar"
      src={avatar}
      alt={description}
    />

Fragments

An empty component is usually used as a parent container to return multiple components simultaneously.

<> 
    <ChildComponent />
</>

Props

The parameters passed through containers. Anything can be a prop including other components (as children, known as composition).

A 'key' in props (similar to the primary key in SQL) is used to identify a component. Usually, it is the current index.

// Writing a function that supports props
function Avatar({ person, size }) {
  return (
    <img
      className="avatar"
      src={getImageUrl(person)}
      alt={person.name}
      width={size}
      height={size}
    />
  );
}

// Using props with the component
return (
    <Avatar
      person={{ name: 'Lin Lanying', imageId: '1bX5QH6' }}
      size={100}
    />
  );

Rendering

Works by using a virtual DOM (VDOM). how does it work:

State changed? update VDOM to reflect changes
'Diff'ing is performed: Check for changes between DOM and VDOM.
'Reconciliation' with the DOM is performed.

Event Handling

Handles different events like onClick(), onChange(), and onSubmit().

State

A snapshot of the app at any given point. We use special functions like useState and useReducer.

function Example() {
  // Declare a new state variable, which we'll call "count"
  const [count, setCount] = useState(0);

  return (
    <div>
      <p>You clicked {count} times</p>
      <button onClick={() => setCount(count + 1)}>
        Click me
      </button>
    </div>
  );
}

Controlled Components

Components used by a state to have a more predictable behaviour.

const [value, setValue] = useState('')

We change the state to change the behaviour.

Hooks

It allows us to hook into features like state within function components. There are 5 types of hooks:

State Hooks- useState, useReducer, to manage state
Context Hooks- useContext, use data through context
Ref Hooks- useRef, to reference HTML
Effect Hooks- useEffect, Lets you connect to external systems and APIs.
Performance Hooks- useMemo, useCallback, Boosts performance.

Purity

Describes how react components should work (Like the one in Functional Programming). It follows two rules:

Only return JSX
Don't change stuff that existed before rendering

Strict Mode

A component that detects problems during app development. Here's how to use it:

<StrictMode>
  <App />
</StrictMode>

Effects

Code that reaches outside the react application (like an API). Best done using event handlers or useEffect.

function ChatRoom({ roomId }) {
  const [serverUrl, setServerUrl] = useState('https://localhost:1234');

  useEffect(() => {
    const connection = createConnection(serverUrl, roomId);
    connection.connect();
    return () => {
      connection.disconnect();
    };
  }, [serverUrl, roomId]);
  // ...
}

Refs

Used to reference an element on DOM. Useful for tasks like focusing on an element.

export default function Counter() {
  let ref = useRef(0);

  function handleClick() {
    ref.current = ref.current + 1;
    alert('You clicked ' + ref.current + ' times!');
  }

  return (
    <button onClick={handleClick}>
      Click me!
    </button>
  );
}

Context

Passing data without sending as props.

function Button() {
  const theme = useContext(ThemeContext);
  return <button className={theme} />;
}

Portals

Context for components. Ideal for modals, dropdowns and tooltips.

<div>
  <p>This child is placed in the parent div.</p>
  {createPortal(
    <p>This child is placed in the document body.</p>,
    document.body
  )}
</div>

Suspense

Component to wait for something to load/occur. Provides a fallback UX till the other component is fetched/rendered.

<Suspense fallback={<Loading />}>
  <SomeComponent />
</Suspense>

Error Boundaries

Component to show a fallback component should that app encounter an error (like a 404 page).

export function MessageContainer({ messagePromise }) {
  return (
    <ErrorBoundary fallback={<p>⚠️Something went wrong</p>}>
      <Suspense fallback={<p>⌛Downloading message...</p>}>
        <Message messagePromise={messagePromise} />
      </Suspense>
    </ErrorBoundary>
  );
}

function Message({ messagePromise }) {
  const content = use(messagePromise);
  return <p>Here is the message: {content}</p>;
}

References:
React Dev

Removing redundant libraries from Makefiles

Albert Jokelin — Wed, 26 Jun 2024 06:56:26 +0000

It can be awfully frustrating when your binary is massive, you can’t strip your executable otherwise you may lose vital debugging information and the only way to figure out what libraries are and aren’t needed is by brute-forcing your way through.

There is an alternative way i discovered while working on my C++ program.

Theory: Assume we have two sets S1 and S2 defined as follows,

S1: Set of all libraries included in the makefile. 
S2: Set of all libraries used by the compiler to build the binary.

It is obvious that S1 is the complete domain (i.e. S1 = U) and n(S1) >= n(S2). Now we define a third set, S3 as follows,

S3: Set of all libraries to be removed from the makefile.

The way we obtain S3 is by using the following equation:

S3 = S1 – S2
   = (S1 ∩ S2)’ [The complement of the intersection of S1 and S2]

Add this line to the end of your COMPILER variable in the makefile -Xlinker -Map=output.map. This flag creates a map of all the functions called while linking the binaries.

Now, to make your life easier you can either use the sed command to parse through output.map or you can search for individual libraries and remove the ones not called from your Makefile.

Demystifying Monads: A Primer

Albert Jokelin — Wed, 08 May 2024 17:03:38 +0000

In the realm of functional programming and category theory, there's a concept that often mystifies newcomers and seasoned programmers alike: the monad. Monads are considered difficult to understand, but with the right approach, they can be demystified and appreciated for their elegance and utility.

At its core, a monad is a design pattern used to encapsulate computations in a structured manner. It provides a way to chain operations together while managing side effects, error handling, and state. While this definition might sound abstract, we can break it down into simpler components to gain a better understanding.

The Three Laws of Monads

Before diving deeper, let's first establish the three fundamental laws that govern monads:

Left Identity: return a >>= f is equivalent to f a, where return lifts a value into the monadic context.
Right Identity: m >>= return is equivalent to m, where m is a monadic value.
Associativity: (m >>= f) >>= g is equivalent to m >>= (\x -> f x >>= g).

Understanding Monads through Examples

To truly grasp the essence of monads, let's explore a classic example: the Maybe monad. The Maybe monad is used for computations that may fail, by encapsulating the possibility of a value being absent or present.

Consider a scenario where we want to perform division but need to handle the case where the divisor is zero, which would result in an error. Without monads, we might write code like this:

def divide(x, y):
    if y != 0:
        return x / y
    else:
        return None

result = divide(10, 5)
if result is not None:
    print("Result:", result)
else:
    print("Error: Division by zero")

While this code works, it requires explicit error checking and handling, which can clutter our codebase. Enter the Maybe monad to rescue us from this verbosity:

data Maybe a = Just a | Nothing

instance Monad Maybe where
    return x = Just x
    Nothing >>= _ = Nothing
    Just x >>= f = f x

divide :: Float -> Float -> Maybe Float
divide x 0 = Nothing
divide x y = Just (x / y)

main = do
    let result = do
        x <- divide 10 5
        y <- divide x 2
        divide y 2
    case result of
        Just r -> putStrLn $ "Result: " ++ show r
        Nothing -> putStrLn "Error: Division by zero"

In this example, we define a Maybe type that represents computations that may fail. We implement the Monad type class for Maybe, defining how values are lifted and composed within this monadic context. With the Maybe monad, error handling becomes implicit, and we can chain computations together elegantly using the do notation.

Conclusion

Monads are powerful abstractions that simplify the management of side effects, error handling, and state in functional programming. By following the laws of monads, programmers can write cleaner, more maintainable code.

References

https://blog.ploeh.dk/2022/04/25/the-maybe-monad/
https://builtin.com/software-engineering-perspectives/monads
https://ncatlab.org/nlab/files/KohlSchwaiger-Monads.pdf

Terraform pt-2

Albert Jokelin — Wed, 13 Sep 2023 17:18:00 +0000

Hey Reader-
Just started learning terraform and this articles serves to explain what I have learnt. If you have any thoughts or feel that I may be wrong, do let me know. My plan for this is to make this a series of articles that can explain the tool from my POV.

Cheers
A

Contents

Intro to terraform
Terraform pt-2

In this post, we will discuss how to install and use terraform on your PC.

Installation

First, we head over to the Terraform downloads page and choose the operating system we want to download Terraform for.

If you're downloading for Windows like I did, then there are a couple of steps you must follow:

Download, unzip and place the terraform binary somewhere you wouldn't accidentally delete it.
Add the location to your system PATH variable. Check out this guide for https://stackoverflow.com/questions/1618280/where-can-i-set-path-to-make-exe-on-windows.
Verify the installation by running terraform -help. You should be getting this:

Quick start

Let's try using Terraform with an Nginx server and understand how it works.

Before we go ahead, install Docker on your PC. If you're on windows, you will need WSL 2 to run docker.

We're gonna be referencing the guide here.

Let's start off by making a working directory:

mkdir myTerraformDir
cd myTerraformDir

We're gonna create a file called main.tf and paste the following code into it.

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.1"
    }
  }
}

provider "docker" {
  host    = "npipe:////.//pipe//docker_engine"
}

resource "docker_image" "nginx" {
  name         = "nginx"
  keep_locally = false
}

resource "docker_container" "nginx" {
  image = docker_image.nginx.image_id
  name  = "tutorial"

  ports {
    internal = 80
    external = 8000
  }
}

Once you're done, save the file and run terraform init. This is the output you'll get on Windows.

And then you run terraform plan which give you the following output:

Run terraform apply and head over to localhost:8000, this is what you'll find:

And if you run docker ps, this is what you'll get:

To shut this down, run terraform destroy.

Voila! you've made and destroyed your first server with terraform. In the following guides, we're gonna explore how to use terraform with AWS.

References:

Intro to terraform

Albert Jokelin — Sun, 27 Aug 2023 09:02:35 +0000

Cheers
A

Contents

Intro to terraform
Terraform pt-2

What is terraform?

Terraform is HashiCorp's Infrastructure-as-a-Code (IaaC) Service that allows you to automate provision resources on cloud providers like AWS, GCP and Azure using human-readable configuration files.

IaaCs allow you to automatically provision resources through code in contrast to earlier methods where DevOps teams had to manually provision resources in order to scale up/down systems.

This is awful and reserves unnecessary resources especially if you want to use them for a certain amount of time. Terrafrom on the other hand, observes the initial or current state and the final state (written in the config file) and figures out how to get to that state i.e. it is declarative in nature.

What is it used for?

Terraform can be used:

To manage any infrastructure- From AWS, GCP and Azure to personal servers, Terraform can manage almost any infrastructure you throw at it.
To track your infrastructure- Terraform plans changes to be made to your infrastructure and saves it as state file for you to review it. Terraform uses the state file to determine the changes to be made.
To automate changes- The declarative nature of Terraform's state files enables it to automatically figure out what needs to be provisioned, how and where and does it with minimal involvement from the developer.
Standardize configurations- Terraform supports reusable components known as modules which allows users to have standardized configuration code across the board.
Collaborate- State file can be deployed to a Version Control System (VCS) and upload it to Terraform Cloud or any cloud (AWS S3 + DynamoDB for instance) where anyone on the team can easily make changes and deploy.

It is often used in combination with different tools depending on the usecase:
1) Provisioning (Terraform) + Configuration Management (Ansible- AWS)
2) Provisioning (Terraform) + Server templating (Packer- Hashicorp)
3) Provisioning (Terraform) + Orchestration (Kubernetes)

How does it work?

In a nutshell, Terraform architecture can be divided into three main components:

Terraform creates and manages resources on cloud platforms and other services through their application programming interfaces (APIs). Providers enable Terraform to work with virtually any platform or service with an accessible API.

We start off by defining the infrastructure we need in the configuration file. And then run terraform plan on it.

Plan returns the changes that will be made to the infrastruture. If satisified, run terraform apply to apply the changes.

Microservices

Albert Jokelin — Sun, 20 Nov 2022 11:53:41 +0000

Usually, when we make applications (especially as beginners), we tend to make one application to do everything. This application is said to follow a monolith architecture.

What is a Monolith architecture?

The standard way of writing code here all the components are a part of a single unit/ codebase.
The app must be written with one tech stack.
Teams need to be careful to not affect each other's work.
Every time you update a part of the application, you need to redeploy the entire application.

Challenges with the monolith architecture

Applications become too large and complex.
Parts are more tangled into each other.
You can't scale particular components which leads to higher infrastructure costs.
Difficulties arise if different services require different dependency versions.
Longer release process.
For every change made, the entire application needs to be tested.
A single bug in a relatively insignificant component can bring down the entire application.

Since there are a lot of problems with the standard monolith structure, we resolve them by using microservices.

What are microservices?

We break down the application into several small/micro applications.
What does their architecture look like?
- The splits are based on each component's business functionalities.
- It follows the idea of 'Separation of concerns'- 1 service for 1 specific job.
- They must be self-contained and independent of each other. Each service must be developed, deployed and scaled separately even though they are a part of the same application. (Known as loose coupling)

Communications between microservices

Usually done using API Calls.
- Each service has its own API and they can talk to each other by sending HTTP requests
- It is a synchronous communication
It can also be done asynchronously using a message broker
- Common patterns- Pub/Sub and point-to-point messaging
A third way to communicate is by using a service mesh.
- There's a helper service that takes over the complete communication logic.
- You don't have to code the communication logic into the microservice, it is delegated to the service mesh.

Downsides to using microservices

Added complexity because the microservices application is a distributed system.
- Configuring the communications between services for scenarios where one of the services is down.
- More difficult to monitor with multiple instances of each service distributed across servers. Tools for overcoming these challenges have been built over the years. One of them is Kubernetes.

CI/CD pipelines for microservices

Companies like Meta, Google and Netflix deploy hundreds of microservices every day. The complexity of the pipeline increases with the number of such services deployed.

Monorepo vs Polyrepo

Monorepo means using 1 git repository that contains many projects.
- Each service has its own folder/ directory.
- Makes code management and development easier.
- Clone and work only with 1 repo.
- Changes can be tracked together, tested together and released together.
- Share code and configurations like docker-compose and manifest
Some of the challenges are:
- Tight coupling of projects.
- Easier to break the 'loosely coupled' criterion.
- Cloning, fetching and pushing slowdown due to increased size.
- Makes pipeline code more complex and challenging to write.
- If the main branch is broken, the entire repo is gone.
Used by companies like Google.
Polyrepo means that each service has its repository.
- Code is completely isolated
- Clone and work on them separately.
- In services like GitLab, you can have projects to configure connected projects (Groups).
  - This helps to keep an overview.
  - Create shared secrets, CI/CD variables, runners, etc.
- The CI/CD is more straightforward.
Some downsides are:
- Cross-cutting changes is more difficult.
- Changes spread across projects must be submitted as separate Merge requests instead of a single, atomic MR.
- Switching between projects is tedious
- Searching, testing and debugging is more difficult.
- Sharing resources is relatively more difficult.

I made these notes after watching Techworld with Nana’s video. Let me know what you think of it and if you’ve loved it, drop a like :)

System Design 101

Albert Jokelin — Sun, 03 Jul 2022 20:24:32 +0000

What is system design?

To define the term system design, we need to take a closer look at the term itself, which has two parts- system and design.

System

A system can be loosely defined as an architecture or a collection of technologies that work together to serve a particular purpose. This purpose always has a set of users that have a certain or even specific requirements.

An easier way of understanding this is to relate this to real-world systems like a restaurant.

A restaurant is a system with many different components. We have a kitchen that is responsible for cooking the food, the wait staff is responsible for the customer's experience, the cleaning staff is responsible for supplying the kitchen with clean dishes and maintaining the retaurant, the manager is responsible for the overall functioning of the restaurant.

In layman's terms, that's exactly how a system behaves. A collection of components that work together in harmony to produce some sort of useful output which in the restaurant's case is delicious food with an equally fulfilling dining experience.

Design

Coming back to the example of a restaurant, we can see that we have to setup each component in such a way that it produces the optimal output.

If we're setting up a fast food restaurant, we can get rid of the wait staff for self-service counters and drive-throughs. The kitchens have machines that prepare higher volume of food so that customers won't have to wait long for their meals.

Even though a Michellin-star restaurant and a fast food joint have roughly the same, basic components, the way they have been set up or designed has made all the difference.

Thus, we can define design as the process of understanding the user's requirements and selecting/modifying the software modules and technologies so that they can better serve the need of the user.

What's the point of all this?

By now, you must having a rough idea of what happens in system design. And you might be wondering, "what is the exactly the point of all of this?"

Personally, I'm used to just building projects and publishing them on GitHub, not caring about UI/UX, performance, etc. But we must remember that developers, in a way are engineers.

Sure, we may love programming and building stuff and writing about it. However, at the end of the day we're building products for other people to use and, like any good craftsman, an engineer must know about all the tools he has at his disposal, and knowing which ones to use when and where makes all the difference.

Stay tuned for more articles about system design!

Improve your programming skills with 2 beers!

Albert Jokelin — Thu, 22 Jul 2021 16:03:21 +0000

Alas! Imposter syndrome is real… One day, it hit me so hard I said, “Enough is enough. I have to do something about it.” So, I started going through Reddit, Youtube and every other platform that’s ever existed, looking for ways to be a ‘better programmer’. Hours of scrolling and all I could find was Leetcode, practice and other stuff you’d find everywhere. I wanted to give up until I came across the Ballmer Peak.

The theory that computer programmers obtain quasi-magical, superhuman coding ability when they have a blood alcohol concentration percentage between 0.129% and 0.138%.

[Photo by thom masat on Unsplash]
That’s what Urban Dictionary has to say about this. In layman’s terms, drink two beers and becoming a literal programming god. Honestly, I don’t believe in this. Any sane person would know that you’d find programmers laying drunk on-campus of many of the tech giants the moment they set up a bar next to the cafe.
A little bit of googling may prove otherwise. Some articles have even compared booze to Popeye’s spinach! Steve Ballmer, is an incredibly energetic guy (not kidding check out his retirement speech). You’d think that with all this energy he’d have to spend a lot of time letting it out of him but instead he used to sit around, observing people (and himself). This constant observation led him to see the effects of this graph:

source: http://ballmerpeak.web.elte.hu
As we can infer from the graph given, moderate alcohol consumption does yield a notable increase in creative output. If we remove the typing out of it, programming is just creativity with computer software. The language is your paint, the functions you use, your style and the billions of devices capable of running the code, your canvas.
In my opinion, whenever people drink alcohol their brains tend to be unable to focus on things at hand which makes them carefree. If a person has a carefree attitude, they may not worry about the future and tend to take bold steps like trying out the wildest solution to fix the code.
This is a pretty cool phenomenon, would I try it before coding rounds? Certainly not. But if you wanted an excuse to drink beer you’ve got it :). Additionally, if you’re interested in maintaining the blood alcohol levels check this out: https://github.com/alexcrist/ballmer-peak.