Forem: Shanker

Understanding ASCII / Unicode character encoding format

Shanker — Wed, 29 Dec 2021 05:16:41 +0000

Computer stores everything in binary format. The process of converting a value into binary is called Encoding and the the process of converting value from binary is called Decoding.

value --> binary :: Encoding

binary --> value :: Decoding

for example:

A number 12 is stored in its binary format as below.

        12 ---> 00001100

but how are the characters are stored? If we want to store a character 'a' what formatting should be used ? It easiest way is to map the characters to number and store the binary format of that number.

So, the character 'a' is represented as number 97 is stored as 1100001.

ASCII:

In ASCII encoding format, 128 unique characters are identified and mapped with numbers (mostly English). One byte is used to represent all the characters. It uses 7 bits to represent numeric value 0 to 127 (total 128 characters). The most significant bit is a vacant bit.

The vacant bit can be used for other characters not represented in ASCII table. Different regional languages used the vacant to represent their characters which caused ambiguity when documents are shared between both the parties.

for example:

Value 129 used to represent 'Va' character in devnagiri in India while same value to represent 'Xi' in Mandarin.

ASCII Table. Dec = Decimal Value. Char = Character

To deal with these issue a new encoding scheme was invented to map all the characters in the world. This set of mapping is known as unicode.

Unicode:

In unicode every character is represented in codepoint. Most used formats utf-32, utf-16, utf-8. UTF stands for Unicode Transformation Format.

** UTF-32:**

In UTF-32, 32 bits are used to represent a codepoints. The 32 bits are fixed length. So a character 'a' that uses 1 byte in ASCII, now uses 4 bytes in UTF-32 format. Hence an ASCII document of 1KB would take 4KB storage space if stored in UTF-32 format, leading to more storage requirement.

UTF-16:

In UTF-16, either 16 bits or 32 bits are used to represent the codepoints. The length can be varying between 16 or 32 bits. So a character 'a` that took 4 bytes in UTF-32 format now takes 2 bytes if stored in UTF-16 format.

If a character codepoint requires more than 16 bits then that character codepoint is stored using 4 bytes.

UTF-8 is a variable-width character encoding used for transferring data over the network. It uses one to four one-byte or 8-bit of code points. The code points represent a character. The first 128 characters of Unicode, which also corresponds to one-to-one with ASCII is represented using 1 byte in UTF-8 format.

UTF-8 has variable width, that is it can use 1 byte or upto 4 bytes to encode a character. It can encoding all 1112064 valid code points character in Unicode.

For 1 byte encoding, it uses 7 bits. The most significant bit is set to 0.

For 2 bytes encoding, it uses 11 bits. 3 bits of MSB of leading byte is set to 110 followed by MSB of continuation byte is set to 10.

For 3 bytes encoding, it uses 15 bits. 4 bits of MSB of leading byte is set to 1110 followed by MSB of continuation byte-1 is set to 110 and MSB of continuation byte-2 to 10

For 4 bytes encoding, it uses 18 bits. 5 bits of MSB of leading byte is set to 11110 followed by MSB of continuation byte-1 is set to 1110, MSB of continuation byte-2 to 110, MSB of continuation byte-3 to 10.

Max length for utf-8 can go upto 6 bytes in future.

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->

Java 8 for beginners - Part 1

Shanker — Wed, 01 Dec 2021 07:50:59 +0000

Java 8 for beginners - Part 1

Introduction

JAVA 8 is a major feature release of Java programming language. It was initially released on 18 March 2014. Java 8 provides support for functional programming, new JavaScript engine, new APIs for date time manipulation, new streaming API, etc.

Below are some of the new features in Java 8 :

1) Lambda expression − Adds functional processing capability to Java.

2) Method references − Referencing functions by their names instead of invoking them directly. Using functions as parameter.

3) Default method − Interface to have default method implementation.

4) New tools − New compiler tools and utilities are added like ‘jdeps’ to figure out dependencies.

5) Stream API − New stream API to facilitate pipeline processing.

6) Date Time API − Improved date time API.

7) Optional − Emphasis on best practices to handle null values properly.

8) Nashorn, JavaScript Engine − A Java-based engine to execute JavaScript code.

Check out below program to sort names in Java 7 vs Java 8 to understand how lambda help us write a clean and elegant code in java.


import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

public class Main {

    public static void main(String[] args) {
        List<String> names1 = new ArrayList<String>();
        names1.add("Sam ");
        names1.add("James ");
        names1.add("Wilson ");
        names1.add("Edward ");
        names1.add("Ramesh ");

        List<String> names2 = new ArrayList<String>();
        names2.add("Sam ");
        names2.add("James ");
        names2.add("Wilson ");
        names2.add("Edward ");
        names2.add("Ramesh ");

        Main tester = new Main();
        System.out.println("Sort using Java 7 syntax: ");

        tester.sortByJava7(names1);
        System.out.println(names1);
        System.out.println("Sort using Java 8 syntax: ");

        tester.sortByJava8(names2);
        System.out.println(names2);
    }

    //sort by java 7
    private void sortByJava7(List<String> names) {
        Collections.sort(names, new Comparator<String>() {
            @Override
            public int compare(String s1, String s2) {
                return s1.compareTo(s2);
            }
        });
    }

    //sort by java 8
    private void sortByJava8(List<String> names) {
        Collections.sort(names, (s1, s2) -> s1.compareTo(s2));
    }
}

Output :

Sort using Java 7 syntax: 
[Edward , James , Ramesh , Sam , Wilson ]
Sort using Java 8 syntax: 
[Edward , James , Ramesh , Sam , Wilson ]

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->

Ruby gem explained

Shanker — Sat, 30 Oct 2021 18:00:22 +0000

What is a Ruby Gem?

Gems are open source libraries just like jar packages in java or modules in Go, the gem contains contain Ruby code. This helps in modularization of the code. A gem can be used by a developer in their own program, without writing that piece of code.

You can check the different gems at https://rubygems.org/. Below are some of the popular gems:

Bundler — Provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed.

RSpec — A testing framework that supports BDD for Ruby.

Devise — Devise works with authentication. Website that needs to user log-in’s, Devise handles sign in, sign up, reset password, etc.

JSON — Provides an API for parsing JSON from text.

Nokogiri — Provides HTML, XML, SAX, and Reader parsers with XPath and CSS selector support.

Rails — Rails is a fullstack web application development framework.

Install Gems

To Install gems locally you can run the command:

gem install [gem_name]

The gem install command fetches the code, downloads it to your computer, and installs the gem and any necessary dependencies and builds documentation for the installed gems.

To see all gems installed locally run the command

gem list

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->

Common handy git commands

Shanker — Fri, 09 Jul 2021 09:55:37 +0000

In this article I'll explain the commonly used git commands every developer, DevOps engineer should know.

git config

Git config commands gives you the information about your git configurations. It can be used to setup initial configs when you install git.

Usage:

list all the configuration for the git

$ git config -l

Setup global username

$ git config --global user.name "Your name"

Setup global email

$ git config --global user.email "Your email"

git version

Displays the version of the git you are working on.

Usage:

$ git version

git init

This command use to initialise git in a new project. Once the git is initialised for a project you can add source code to track the commits.

Usage:

$ git init

Use init with repository name will create a new project with git initialized for it.

$ git init <repo name>

git clone

The git clone command will use an existing remote repository to copy.

The only difference between the git init and git clone is, Git clone makes a copy on an existing remote repository. The git clone command internally uses the git init command first and then checks out all its contents.

Usage:

$ git clone <your project URL>

git add

Git add command adds all the new code files or modified files into your repository's staging area.

Usage:

$ git add file_name (adds a single file to the staging area)

$ git add * (adds all the modified and new files to the staging area)

git commit

Git commit command adds your changes from staging area to your local repository.

Usage:

$ git commit -m “commit message”

git status

Status command allows you to see the current branch and how many files are added/modified.

Usage:

$ git status

git branch

When you work on multiple branches in your Git repository. Git branch command allows you to manage your branches. Git branch lists all the branches in your local repository.

Usage:

list all branches

$ git branch

create a new branch

$ git branch <branch_name>

delete a branch:

$ git branch -D <branch_name>

git checkout

Checkout command is used to switch between branches.

Usage:

$ git checkout <branch_name>

you can also create and checkout to a new branch in a single command

$ git checkout -b <your_new_branch_name>

git log

Git log command helps to see the history of previous commits.

Usage:

$ git log

To see git logs in one line

$ git log --oneline

git remote

Git remote command is used to connect your local repository with the remote repository.

Usage:

$ git remote add <url>

git push

Push command is used to push your local changes to the remote repository.

Usage:

$ git push origin <your_branch_name>

Git push should have origin and upstream set up before you push code to remote repository.

Usage:

$ git push --set-upstream origin <branch_name>

git fetch

Fetch command downloads all information for commits, refs, etc. You can review before applying those changes in your local repository.

Usage:

$ git fetch

git pull

Git pull command updates your local repository with the latest content from remote repository.

Usage:

$ git pull <remote_url>

git stash

Stash command temporarily stores your modified files.

Usage:

$ git stash

You can view all of your stashes

$ git stash list

if you need a apply a stash to a branch. Below apply stash{3}

$ git stash{3} apply

git shortlog

The shortlog command gives you a short summary from the Git log command. It displays who worked on what by grouping authors with their commits.

Usage:

$ git shortlog

git show

Show command git show display details about a specific commit.

Usage:

$ git show <your_commit_hash>

git rm

rm command deletes files from your code, it deletes tracked files from the index and the working directory.

Usage:

$ git rm <your_file_name>

git merge

merge helps to integrate changes from different branches into the current branch.

Usage:

$ git merge <diff_branch_name>

git rebase

Git rebase similar to the git merge command. It integrates changes from different branches into the current branch.

The only difference is git rebase command rewrites the commit history.

Usage:

$ git rebase <branch_name>

git bisect

bisect command helps to find bad commits.

Usage:

to start the git bisect

$ git bisect start

let git bisect know about a good commit

$ git bisect good a123

let git bisect know about a bad commit

$ git bisect bad z123

git cherry-pick

Git cherry-pick command allows to pick any commit from any branch and apply it to any other branch. It doesn’t modify the commit history of a repository, new commits are applied to the commit history.

Usage:

$ git cherry-pick <commit-hash>

git archive

Git archive command combines multiple files into a single file. It is useful to create a zip of a repository.

Usage:

$ git archive --format zip HEAD > archive-HEAD.zip

git pull --rebase

When you need to do a rebase instead of merge when using git pull command.

Usage:

$ git pull --rebase

git blame

blame is used to examine a file line by line and to determine who made the changes to a file.

Usage:

$ git blame <your_file_name>

git tag

tag command is used to manage the release.

Usage;

$ git tag -a v1.0.0

git verify-commit

git verify-commit command checks the gpg signature. GPG or “GNU Privacy Guard” is the tool used in sign files and contains their signatures.

Usage:

$ git verify-commit <commit>

git verify-tag

Just like verify-commit, you can confirm a tag.

Usage:

$ git verify-tag <tag>

git diff

diff command is used to compare two git files or branches before you commit or push.

Usage:

to compare the working directory with the local repo:

$ git diff HEAD <filename>

to compare two branches:

$ git diff <source branch> <target branch>

git citool

Git citool is a graphic tool alternative of the Git commit.

Usage:

$ git citool

git mv

mv command is used to rename a git file.

Usage:

$ git mv <old-file-name> <new-file-name>

git clean

clean command is used to remove/delete all the untracked files from your working directory. To remove tracked files you can use the git reset command.

Usage:

$ git clean

git help

To get more information or help on any of the git command, you can use git help.

Usage:

$ git help <git_command>

git whatchanged

git whatchanged command is same as git log but in a raw format.

Usage:

$ git whatchanged

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->

Five untold skills of a software engineer

Shanker — Wed, 24 Mar 2021 12:02:50 +0000

Today there are multiple programming languages, frameworks, libraries and innovative technologies. They either get popular or fade away. But today I’m going to talk about few of the top untold skills in tech industry that don’t fade away with time.

1) Communication skills — Communication is by far one of the important skill to possess. It is very important for you to articulate your problems/ideas/solutions when you work in teams. It allows others to visualize the subject you are see. You ability to communicate the problems/ideas/solutions can lead to better understanding amongst others and the overall productivity of the team.

2) Ability to find solutions — As a software engineer we often spend time on stackoverflow looking solutions to our problem. Your ability to reference similar type problem on stackoverflow and connect it to your problem to get your solution is very important. I have often seen junior developers unable to connect the dots to their problem with similar related problems online. Eventually end up impacting the delivery of the product.

3) Being humble — Be humble to very one around you. Your team, house cleaning staff, support staff everyone has a role to play and cumulatively you are trying to achieve your deliverable goals. It is important to be humble.

4) Asking help — In tech if you are asking for some help, is often perceived as incompetency and can lead to imposter syndrome. On the other side don’t be a leech and ask for help without even trying once. That is way more bad. Striking a perfect balance is important, knowing when you really need help and when you just need more time to figure out.

5) Documentation — Last but not the least, have the ability to document well. Sometimes we have see great technologies being sidelined because of poor documentation. Poor documentation can make a good technology difficult understand. New adopters, hobbyists, enthusiasts may start to avoid the technology for the same reason.

Above are five skills I feel every software engineer should know. Do let me know in the comments section which other skills you feel are important but less spoken skills needed as a software engineer.

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->

Telepresence

Shanker — Wed, 20 Jan 2021 10:43:31 +0000

What is it?

Allows you to run a service on your local machine while also connecting other services on the remote Kubernetes cluster.

Why is it?

It is difficult to test and debug a service in a complex microservice architected applications from you local machine.

For a developer with a machine 16 Gigs of RAM, it is difficult to spin up docker compose or minikube to run all the applications on a local machine. You might just end up heating 🔥 the system.

Often, to complete feature testing we need good amount of traffic hence we deploy the application on a cluster environment with good amount of traffic, named as staging cluster.

From the time you commit the code to it be available on the cluster, it goes through below phases :

1) Compile - to compile and run all the test cases it takes on an average 1-2 mins

2) Build - to build the image it takes on an average 1 min

3) Push - to delete the existing pod and create a new pod with fresh code takes on an average 2-3 mins

So by the time the code it available on cluster, the developer slips a good 4-6 mins on every change made to the code which is to be tested on cluster.

How does it?

Fundamentally, It replaces your pod running in the cluster with a two way network proxy. Essentially swapping out the pod in the cluster with your local development.

Any traffic routed to the service will now be proxied from the kubernetes cluster to the local machine.

Which means your machine is now part of the cluster and is serving the live traffic from the cluster environment.

Telepresence proxies the environment variables and volumes and TCP connections to your machine. It intercepts DNS so DNS calls and TCP traffic are routed to the cluster through the proxy from your local machine.

Advantage?

Saves individual developer time by quick allowing them to testing their changes with the cluster services.

Helps the developers to locally test/debug and change the service.

It allows to make your local deployment be part of the cluster easily.

Installing telepresence

OS X

brew cask install osxfuse
brew install datawire/blackbird/telepresence

Ubuntu 16.04 or later

curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh | sudo bash
sudo apt install --no-install-recommends telepresence

Windows

Install Windows Subsystem for Linux
Start the BASH.exe program
Install Telepresence by following the Ubuntu instructions above

To verify you can simply open the terminal and run the telepresence command :

And btw ... its not a penguin ... its a bird. 😉

link for reference -> https://www.telepresence.io

Thank you folks. Please do check my other articles on https://softwareengineeringcrunch.blogspot.com

If you like my article, you can support me by buying me a coffee ->