Forem: Sainath Ramanathan

Empty phone number field validation using yup-phone

Sainath Ramanathan — Sat, 08 Oct 2022 08:42:19 +0000

I recently encountered an issue while using the library yup-phone

yup-phone is a react-library that can be used to validate phone numbers entered in the form field. It uses google-libphonenumber to validate any given phone number in the input field.

Sample usage of yup-phone is given below,

const validationSchema = Yup.object().shape({
    mobile: Yup.string().phone()
})

The above schema validates if the given number is a valid phone number format or not. You can read in detail about its usage on the official page.

The problem with the above snippet is it makes the form submission mandatory to provide a phone number even if it is not required. If the field is left empty the validation is made against the empty field. Yup methods optional and notRequired didn't work either. ( Note: This error is not resolved by yup-phone at the time of writing this post. Please check the official page for tracking the issue. )

Here is a workaround to fix the empty field validation.

const validationSchema = Yup.object().shape({
        mobile: Yup.string().when('mobile', {
            is: (value) => value?.length > 0,
            then: Yup.string().phone(),
            otherwise: Yup.string(),
        }),
    },
    [
        ['mobile', 'mobile']
    ]
)

Note that the mobile property is written twice in the list of dependencies. This is to avoid cyclic dependency errors as explained in the StackOverflow post.

Below is the code snippet for multiple phone number fields.

const validationSchema = Yup.object().shape({
        mobile: Yup.string().when('mobile', {
            is: (value) => value?.length > 0,
            then: Yup.string().phone(),
            otherwise: Yup.string(),
        }),
        phone: Yup.string().when('phone', {
            is: (value) => value?.length > 0,
            then: Yup.string().phone(),
            otherwise: Yup.string(),
        }),
    },
    [
        ['mobile', 'mobile'],
        ['phone', 'phone']
    ]
)

Official page for yup-phone: https://github.com/abhisekp/yup-phone
Issue: https://github.com/abhisekp/yup-phone/issues/313

The Great Merge

Sainath Ramanathan — Fri, 16 Sep 2022 11:42:17 +0000

Introduction

Ethereum is the largest blockchain network next to Bitcoin. It is evolving a lot and equally competing with Bitcoin. Ethereum has paved the way for so many blockchains like Solana, Polygon, and Cardano that try to solve the problems and limitations of Ethereum. The following diagram shows the size of the Bitcoin vs Ethereum blockchain.

A Quick History

Ethereum was created with the idea of a permissionless blockchain back in 2015 with Proof-of-Work (PoW) as its consensus mechanism. Until then people have known blockchain only as a cryptocurrency but Ethereum changed it with a plethora of use cases like digital assets, supply chain tracking, etc by leveraging smart contracts and ETH tokens which was famously called a 'gas fee' to validate a transaction which was then paid to the validators in the network for Proof-of-Work done.

Consensus Mechanism

Ethereum has many networks for development, testing, and production use cases. 'Ethereum Mainnet' was the primary public Ethereum production blockchain based on the consensus mechanism of Proof-of-Work (PoW).

Proof-of-Work(PoW) is hardware intensive and not eco-friendly as several miners participate using expensive resources in validating the block.
Proof-of-Stake (PoS) lets a validator participate in the Ethereum network by staking the ETH which is more scalable, energy-efficient, secure, and has a lower barrier to entry than PoW.

The Evolution of Beacon Chain

In recent years, Web3 gave birth to a lot of products like Decentralized Finance(DeFi), Non-Fungible Tokens (NFT), etc. Networks have to come up with speed and energy efficiency to serve the purpose and maximize the adoption of blockchain. Ethereum launched Beacon Chain based on PoS in late 2020 parallel to the existing Mainnet to slowly replace it with a single consensus i.e the PoS. The Ethereum team knew that PoW won't be good enough for the long run and decided to merge it with the Mainnet.

The Merger

Ethereum Mainnet merged with Beacon Chain on September 15, 2022. As Mainnet merged with the Beacon Chain, it also merged the entire transactional history of Ethereum. Mining is no longer the means of producing valid blocks in Ethereum. Going forward, the Beacon Chain is the engine of block production. The merger led the Beacon Chain to accept transactions from the original Ethereum chain, bundle them into blocks, and then organize them into a blockchain using a proof-of-stake-based consensus mechanism.

Terminology change

The term Eth2 has been deprecated. There is nothing called Ethereum 2.0. The new terminologies for Ethereum are as followed,

Eth1 - Execution Layer
Eth2 - Consensus Layer

Execution Layer + Consensus Layer -> ETHEREUM

Conclusion

ETH remains ETH. You don't need to do anything from your side owing to this change. All you need to know is that Ethereum now has a new engine based on PoS which is eco-friendly, slightly faster, scalable, and more secure.

Note: This article is more about the technical details of the Ethereum blockchain. Anything related to cryptocurrency is not in the scope of this discussion.

References

Node Package Manager (NPM)

Sainath Ramanathan — Wed, 20 Apr 2022 15:02:43 +0000

Need for NPM

Node.js has got built-in modules for accessing the file system, networking, and other I/O operations. The problem is it offers only essential packages as core modules. We often want to use some external packages to re-use code and avoid complexity. Node has a way to cater to these packages through the node package manager (npm). NPM offers a fleet of packages that are highly used in production-grade apps. Express is one such example that has seen a wider adoption. We will learn more about express.js in an upcoming post, stay tuned to this Node.js series. Also, there are many notable packages like react, react-dom, underscore, vue, axios, body-parser, etc.

Initiate NPM

To start with npm one can straightaway begin to use it. It is by default installed during the node.js installation. However, if you want to check the version, you can do it in the terminal.

npm --version

The first thing one has to do to import npm packages is to create a folder and initiate npm in it. To initiate npm, navigate to your project folder in the terminal (in the case of windows, use the command prompt) and run npm init. You will get a list of questionnaires to which you can input your preferences. In the interactive terminal, if nothing is filled it will populate a default value for certain fields like package name, version, and leaves it blank for description, author, etc. Press enter to move on to the next item. In the end, a JSON is created with a list of key-value items that you gave as input.

A short way to quickly do the initiation is by running npm init --yes. The yes flag will populate default values and skips the interactive questions to the user.

Installing a package

The package.json created by npm can be found inside the project folder. This file is a snapshot of the list of dependencies for the project. It is necessary and be cautious to not delete this file.

The packages can be installed using npm commands.

npm install <package-name> --save

This command will add the package to your local project and the scope remains local i.e. you can't access the package outside of this folder.

Let's go ahead and install a package for our project. I am going to install chalk which helps in rendering colorful output in the console and this will be useful when you design CLI-based applications.

npm install chalk --save

If everything goes well, the package gets added to the project. If you are getting errors while downloading, check your installation and head to the official project page to see if there are any issues reported.

You can find the package name and version in the package.json under dependencies.

node_modules

You can see an additional folder named node_modules got added to our project. It consists of files related to the dependencies that were installed. The size of node_modules is huge and it is downloaded to your machine. You can often find the node_modules folder ignored while pushing your code to the repository. It can also be deleted and a fresh installation of packages can be triggered. Once you have package.json in place, it is no longer necessary to mention the package name to reinstall. All the details of the package and its version are available in the package.json and the installation will pick it up from there.

npm install

Dependencies

There is an option to segregate dependencies based on usage. Not all packages are necessary to be installed, some are meant for development purposes like the nodemon package. Nodemon keeps a watch on your file changes and automatically restarts the node application. This need not be added to your production environment as a dependency as you won't be making any file changes in production. To save a dependency for development purposes, add a flag --save-dev instead of --save. In contrast to local, there is an option to download a package globally to be used by any project by appending the flag -g or --global to the installation package. It is available globally but limited to your machine where it is installed. It is not necessary to install a global package in a project as is it a one-time installation and it will be available to use when imported.

npm install nodemon --save-dev
npm install nodemon -g

Using the installed package

To use a package it should be imported. Different module formats have their unique way of importing. The most known formats are CommonJS(CJS) and ES2015 Modules(ESM). There are also other formats like Asynchronous Module Definition (AMD) and Universal Module Definition (UMD) which you can explore in detail. Packages that use the ESM format insist to import the format using import and the CJS format needs to use require.

#ESM 
import chalk from 'chalk'

To use chalk as CJS format, the documentation specifies to downgrade the version as the latest version supports EJS format.

Chalk 5 is ESM. If you want to use Chalk with TypeScript or a build tool, you will probably want to use Chalk 4 for now.

#CJS
const chalk = require('chalk')

In our example, I have used the ESM format to import the chalk module. An additional entry of type: "module" has to be provided in package.json for using ESM format.

Create an index.js file and import the 'chalk' module. Chalk can now be used to log your output colorfully. Check more about chalk and try it yourself.

Hope this post helped you in understanding the usage of npm. This post only caters to the essentials, you can go deep and learn about the detailed usage of npm on their official page.

References

Introduction to Node.js

Sainath Ramanathan — Tue, 19 Apr 2022 03:57:02 +0000

Introduction

Welcome to the Node.js learning series. This series will walk you through from basics to advanced concepts of Node.js. I will try my best to explain the concepts more simpler. Without further ado let us jump into the sea of Node.js.

Node.js is a very popular backend javascript runtime environment. Back in time, there were only a few languages like Java, Python, Php, etc. that helped build a backend to do all complex tasks. The developers needed to learn multiple tech stacks to build the front and backend of an application. This became hard and the developer groups were segregated into frontend engineers and backend engineers. The rise of Node.js was the first big step toward full-stack web development. It gave rise to a plethora of full-stack developers who can code both frontend and backend in one single language i.e Javascript.

Javascript

Javascript is the blood of Node.js. Javascript knowledge must be required even before continuing with Node.js. If you are a newcomer, bookmark this post for future reference and continue learning javascript. Some important concepts that are needed to better understand Node.js.

Objects
Functions
Classes
Asynchronous
Callbacks
Arrow Functions
Promises
Async/Await

Hop on to the next topic if you are good with the topics listed above.

Node.js Basics

Node.js can be installed on any platform (Windows, macOS, Linux). The steps to install and different download formats are available on the official page of node.js. Make sure you have installed the setup correctly and always install the latest stable version of the node.js. NPM gets installed along with node.js installation. We can discuss NPM in upcoming topics. Check your node and npm versions after the installations are complete.

node --version
npm --version

You are now ready to hack and let's start with the scripting. It is very boring to learn the concepts if you don't connect the dots. Hence we will learn the concepts by building a simple app. Our mission is to build a library application.

Open a suitable IDE of your choice. I prefer Visual Studio Code and I have been using it as my default editor for web development. Open a folder and create a javascript file and name it of your choice filename.js. It can be of any name but in general, the standard practice for naming the main file would be app.js, server.js, and index.js. I am naming it server.js. Open your file and start with writing the first line.

Any programming language will have the first line with import statements. Similar is the case with node.js. It is not necessary that you should always have imports but commonly to use the inbuilt standard packages you may need to require the libraries to use them. For our online library, we need a server to get requests from clients, solve their queries and respond. Feel free to know more about client-server architecture and how the web works. In some frameworks like Php one may need to spin a separate web server and attach it to the main program but in the case of node.js everything is offered in one plate.

Built-in Modules

Node.js has got this built-in HTTP library that handles server creation. To import a module or package, use require and pass the module/package name. Since we need the HTTP package, we will import with require('http'). The imported package can then be assigned to a constant for local use. It can also be assigned to a variable but it is prudent that we are not going to alter the package and that's why it is assigned to a constant.

const http = require('http')

Built-in modules help us to interact with the operating system, file system, networking, I/O operation, and a bunch of other kinds of stuff. There are different built-in modules apart from HTTP. You can read more about it on the official documentation page. Here is an example of writing content to a file.

const fs = require('fs')

fs.writeFileSync('file.txt', 'Hello, welcome to online library')

console.log('Completed writing to file')

The above snippet shows how to import the file system module. It has different methods out of which we use writeFileSync to write content to a file and create it on run-time. The word sync in writeFileSync has to be kept in mind as we move further we will get to know about synchronous and asynchronous operations in node.js.

Create Server

Let us now create a server in your local machine that can be accessed through localhost. We have already imported the HTTP module and now we will use one of its methods createServer which accepts requestListener as an input argument and returns a server. requestListener is a function that accepts two arguments 'request' and 'response' denoted as 'req' and 'res'. The requestListener function will be called whenever there is an HTTP request. The request can be received and served by making the server listen. The listen method will facilitate this by getting the inputs like hostname, port, etc. The server can also respond with headers like HTTP status codes which denote if the request has succeeded or not. Also, one can return an HTML output as a response when a specific request is made in the URL. For example, whenever there is a request to the root http://localhost:4000/, it can be pointed to a landing page/ home page with a welcome message. Once the response has been served to the client, it has to be ended, else it will keep on staying in the event loop waiting for other requests. In short, you will see the page keep on loading.

#server.js

const http = require('http')

const server = http.createServer((req, res) => {
  res.statusCode = 200
  res.write('<h1>Welcome to Online Library</h1>')
  res.end()
})

server.listen(4000)

Routing Requests

Requests can be served depending on the type of request received. Imagine a post office segregating the mail based on addresses. Likewise, a request can be routed based on the URL. Let us redo our server.js file to make it less clumsy and add a new file routes.js to define the routes. It is simple to have a single route defined for a test application but in reality, even a simple application would serve multiple requests. We will separate the requestListener from server.js and define all the routes in routes.js file.

In the routes.js, we are going to define two routes for our online library. The root path / will act as a landing page and the /books will show the list of books in our library. We will abstract the requestListener function in server.js and define it in routes.js. The requestListener function will be exported from routes.js and this module can be imported like any other module in server.js. We will assign the function to reqHandler and export the module using module.exports. Inside the requestListener function let us define the routes to be handled. We will create a simple if condition to switch between routes based on the request URL. The request returns a pile of data as a JSON object. You can have a look by printing the request in the console console.log(req). We are more interested in categorizing based on the URL. The request can be parsed by getting the value of URL. req.url will return the URL (http://localhost:4000/) that was requested by the client. We will grab the URL and compare it with our logic to route it. In the case of the root path, it is straightforward where we just return a response with an HTTP status code of 200 and a welcome message in the HTML body. If the URL has /books then we need to populate it with a list of books which is done using the <li> tag in our HTML response.

#routes.js

const reqHandler = (req, res) => {
  const url = req.url
  const method = req.method
  if (url === '/') {
    res.statusCode = 200
    res.write('<html><head><title>Library</title></head><body><h1>Welcome to Online Library</h1></body></html>')
    return res.end()
  } else if (url === '/books') {
    res.statusCode = 200
    res.write(
      "<html><head><title>List of Books</title></head><body><ul><li>'So Good They Can't Ignore You'</li><li>'Deep Work'</li><li>'Digital Minimalism'</li><li>'Atomic Habits'</li><li>'Thinking, fast and slow'</li></ul></body></html>"
    )
    return res.end()
  }
}

module.exports = reqHandler

We will now import the requestListener function in the server.js file by importing the routes.js. The imported routes module is now passed as an input to the createServer function which is nothing but the requestListener function that we created on the routes.js file.

#server.js

const http = require('http')
const routes = require('./routes')
const server = http.createServer(routes)
server.listen(4000)

Once everything is connected, you will see the following output on your browser.

http://localhost:4000/

http://localhost:4000/books

I hope this post will help you quickly set up a node server. There are other concepts like blocking/non-blocking, asynchronous, callbacks, etc which are more important when you write I/O intensive operations. Node.js primarily does asynchronous execution to avoid waiting for a single request to resolve. It will serve whenever a new request is received and resumes back to the previous request using a callback to resolve whenever it is ready. Get a good hold of these concepts by reading through the official documentation of node.js. You can find below some useful links to delve more into these topics.

Cognitive and Learning Disabilities Accessibility Task Force (aka) COGA

Sainath Ramanathan — Sun, 23 May 2021 00:00:00 +0000

Accessibility is a well-known practice and most product developers and organizations give importance to it to ensure their content/application caters to everyones needs. I am not going to dwell on accessibility as there are lots of posts concerning that. Here is a short explanation of accessibility.

Accessibility

Accessibility is an approach to enable differently-abled users to access the application or user interface by introducing supporting features to assist them. Provisions like reading out the content, enlarged-view, talkback, etc help the challenged users access the content without hassle. To incorporate this as a guideline W3C (Worldwide Web Consortium) introduced a separate guideline called WCAG (Web Content Accessibility Guideline) to enhance the user experience. Also, they keep publishing new changes year on year to improve the features. One such draft that was recently published on Making Content Usable for People with Cognitive and Learning Disabilities, shortly named COGA and it is going to be the primary focus of this post.

Source: MKBHD

Even if you think of it as a feature to help the disabled community, it has wide applicability beyond that.

Sundar Pichai during an interview with MKBHD

During Google I/O 2021, Sundar Pichai was interviewed by Marques Brownlee in which Sundar mentioned the importance of accessibility. Accessibility is not a feature that is designed only to help the differently-abled but to everyone during their temporary disability at any part of their life. It might be an accident that could have made their arm to be not usable for a certain period. It could be a senior citizen whose vision and hearing would decrease due to their age. It could be a feeding mother who always needs to use her mobile in one hand. So, accessibility is better to be said as a general guideline to enable users to access content using enhanced features to support them.

Source: Google Developers

Cognitive and Learning Disabilities Accessibility Task Force

COGA was initiated by WCAG to attend to the needs of the users who find difficulty in understanding the content on the web.

https://twitter.com/w3c/status/1390295033987751937?s=20

The draft emphasizes simplifying the content to the challenged users by providing them assistance and features to understand better. Learning and cognitive disability could occur to anyone owing to age or other mishaps. Let us understand better how we could improve this.

Numbers

https://weather.com/

Numbers play an important role in statistical websites or financial or data-intensive applications. Historical facts, weather reports, stock market prices, and the list can go on. A normal user without any domain-specific knowledge would be interested in knowing the outcome of these data rather than the number itself. Say if a weather report says 40 degrees or wet clouds then it is prudent to convey the user to wear a hat or take an umbrella along with the numerical data. A stock price thats soaring high or falling low can be indicated by arrow symbols and emojis to quickly give a glimpse of the situation. Any data that is represented raw will not add value to the story being conveyed.

Date & Time

https://support.microsoft.com

Across the globe, there are different time zones and different formats of date conventions that are being followed. Achieving unique formats is difficult as everyone would have got accustomed to specific formats. I am good with the format DD/MM/YYYY or DD/MM/YY, other formats might confuse me where I would pause to understand what month it is. This is because our minds are trained by the continuous usage of it. We cant label this as a disability rather a preferred way would be apt to tell. If an application is built for a global audience, all formats of date and time must be presented to them. Another example would be related to time. If there is an event that is happening in a specific time zone and the users participating would be interested in knowing the local time of the event rather than the actual time zone.

Visuals

Photo by Robo Wunderkind on Unsplash

Visual learners are the ones who need assistive diagrams, illustrations, etc to understand the context. Reading a story or novel will give you a mental image where you feel like you are in the story whereas advanced concepts, theorems, axioms, or scientific principles on the web are likely to be completely text and numbers. To assist visual learners it might be helpful if the explanations are coupled with illustrations and graphics. The combinations of text and illustration or an image could give a better grasp rather than a misconception.

Text

Photo by Luke Chesser on Unsplash

We often encounter websites having long texts, the best example is Wikipedia. Those are informational websites that tend to be full of texts as it is necessary to read an entire page to get a complete understanding (similar to my current post 😁). To get the context, one has to run through the entire content which is time-consuming whereas if we could provide a video summary, a gist, or an infographic would help them understand the concept and read through to get the complete essence of the content. Also, the font sizes should be compatible to let the users change as and when required.

Audio and Video

Photo by Kelly Sikkema on Unsplash

There are smart devices that require inputs from users to enact based on voice commands. It can also be a recording device that indicates to the user that voice/sound is being recorded. Similarly Google Assistant, Alexa, and others respond to users that they are listening using a light or sound indication that it is listening else the user would never know if the input is actually received.

Forms

Photo by Kelly Sikkema on Unsplash

Forms are a set of questions that need to be answered. Filling the form can be a text box or a multiple choice field. Questions like Do you agree with this? will have options like I Agree, I Strongly Agree, and Do Not Agree. The options I Agree and I Strongly Agree are relatively the same and this creates unnecessary confusion and it is difficult to measure the agreeability. Hence binary options will always be straightforward and can give a better outcome.

Product Study

Photo by Campaign Creators on Unsplash

A product has to be well researched before shipping to the market. Never design multiple copies of the same product to cater to accessibility needs. It shows that you are discriminating against people with accessibility needs and as mentioned at the beginning of the topic, accessibility is not only for people with special needs but for everyone at a different point in time. Give out the application or product for feedback and observe the reaction from peoples faces. If they are confused and blinking, it means they dont understand your product. If they feel happy and relaxed, you are on the way to AAA accessibility score.

Tech Tools

Considering the need for accessibility, technology companies have come up with tool kits and software to support accessibility. Commonly, people often use mobile devices as their primary tool so the major players have defined their own accessibility tool kits to help their users.

Googles Android offers a range of accessibility services viz. screen reader, display controls, interaction controls, braille display, audio, and on-screen text. Check out more on Android accessibility.

Apples iOS has classified accessibility into vision, physical, and motor. Check out more on iOS accessibility features.

Players involved in accessibility support: Microsoft, Apple, Google, Facebook, Twitter.

References

Minnesota IT Services Blog

COGA Guidelines

WCAG 2.2

SQL: Basics to Advanced in a Nutshell

Sainath Ramanathan — Wed, 10 Mar 2021 10:51:49 +0000

Cover Photo by Christina Morillo from Pexels.

Databases are the containers/repository of data. They hold the data in the form of a spreadsheet(not the same but for the sake of imagination). There are many databases viz. Relational and Non-Relational. Let us not dwell on the latter part but stick with the relational databases. Watch this video to know more about databases.

This post will focus primarily on writing SQL queries. SQL queries are common across all the databases but there are some flavors and they may vary according to different databases. There are various vendors like IBM, MySQL, PostgreSQL, Oracle, etc. Every vendor has its own style of SQL to communicate with its database. The one I would suggest is PostgreSQL which is open source and has got wider adoption.

The primary way to communicate with a relational database is through SQL. Structured Query Language (SQL) is a language to communicate with the database. Without further ado, let us jump into learning SQL.

Image Source: The Data Labs

A table looks like the above spreadsheet with rows and columns. Columns are the headers for the category of the values that we input. A column that is defined as a name will have only names in it and a column defined as a currency would certainly have numbers in the entire column. So, a column defines what kind of values it should comprise. Rows are the list of entries made in different columns. Rows define the number of entries in a spreadsheet.

Now, to get all the columns from the customer table, we use the keyword SELECT and specify the column names that we are interested in. We can use the * symbol to fetch all the columns from the table. To connect to a table, the keyword FROM is used which comes after the SELECT keyword.

SELECT * FROM customer;

To get specific columns from the customer table.

SELECT first_name, last_name FROM customer;

In spreadsheets, we would have used filters similar to that in SQL there are filters to query. For instance, if you want to look only for emails that have the first name as 'Sai', the keyword that should be used is WHERE. WHERE should be used after the FROM command.

SELECT email FROM customer WHERE first_name='Sai';

To get all the emails of the customer whose first name is 'Sai' and the last name is 'Kumar', we use a keyword called AND which is a logical operator similar to OR, and NOT. You can learn more about logical operators in SQL.

SELECT email FROM customer WHERE first_name='Sai' AND last_name='Kumar';

To get all the emails of the customer whose first name is 'Sai' or the last name is 'Kumar'.

SELECT email FROM customer WHERE first_name='Sai' OR last_name='Kumar';

To arrange the data in ascending or descending order use the ORDER BY keyword followed by the column name based on which the entire table would be ordered. To arrange is ascending use ASC and for descending DESC. By default if you don't specify the order, it will sort the column in ascending order. ORDER BY always comes at the tail end of the query.

To get the list of customer ids and arrange them in ascending order of their transaction date.

SELECT customer_id FROM customer ORDER BY transaction_date ASC;

To get the list of customer ids and arrange them in descending order of their transaction date.

SELECT customer_id FROM customer ORDER BY transaction_date DESC;

We can also limit the number of records displayed by using LIMIT with a positive integer number to specify the limit.

To get the list of the first 10 customer ids.

SELECT customer_id FROM customer ORDER BY transaction_date ASC LIMIT 10;

COUNT is a type of function that is used in columns and it should come after the SELECT. COUNT returns the number of data in the specified column. In general, passing * as an argument to the COUNT will also return the same result as the column name.

To get the count of some customers whose age is not more than 50.

SELECT COUNT(*) FROM customer WHERE age<=50;

The use of regular expressions is ubiquitous in all programming languages similarly we use regex in SQL to match the strings. LIKE is used in conjunction with WHERE and followed by the regex.

%String - grabs all the words that end with String
String% - grabs all the words that start with String
%String% - grabs all the words that comprise the String in between them Note: LIKE is case sensitive and looks only for the string that is mentioned. ILIKE is an alternative that is case insensitive and can match across various cases of the string.

To get all the names of the customer whose first letter in their first name starts with 'P'

SELECT first_name FROM customer WHERE first_name LIKE 'P%';

To get all the customers whose last name has 'siva' in between and should be case insensitive.

SELECT * FROM customer WHERE last_name ILIKE '%siva%';

At times the entries in the table might have duplicate data. To filter out unique data, DISTINCT is used. DISTINCT applies only to the column and comes next to SELECT and even as an argument to the COUNT function.

To get the count of distinct areas where the customers are from.

SELECT COUNT(DISTINCT area) FROM customer;

To get distinct areas where the customers are from.

SELECT DISTINCT area FROM customer;

IN is used to specify the list of items against which a column is compared. For example, we can check if a column consists of (Orange, Mango, Banana) and the function filters and display only the entries with orange, mango, and banana.

To get the count of customers who are located in the area with Pincode '82' between the age group 60 to 70.

SELECT COUNT(*) FROM customer WHERE pincode IN(82) and age BETWEEN 60 AND 70;

AGGREGATION

Aggregation means a cluster of things that are brought together. Likewise aggregation in SQL clusters the data into a homogeneous category and groups it. For example,

To get the total number of transactions made by all customers.

SELECT customer_id, COUNT(transaction_date) FROM customer GROUP BY customer_id;

Here COUNT is the aggregation function that has been performed on the transaction_date column concerning the customer_id. So a group is formed for every customer_id where you will get a matching count of the number of transactions. A normal COUNT would have resulted in the total number of transactions whereas in aggregation with customer_id by using GROUP BY, it produces a table with individual customer ids and their corresponding count of transactions.

GROUP BY should always have the column names that are mentioned in the SELECT criteria except for the aggregation function.

To get an average amount that a customer has spent on purchases.

SELECT customer_id, AVG(amount) from customer GROUP BY customer_id;

To get a rounded value of the average calculated.

SELECT customer_id, round(AVG(amount),2) from customer GROUP BY customer_id;

To arrange the SUM(amount) in descending order and display only 10 rows.

SELECT customer_id, SUM(amount) FROM customer GROUP BY customer_id ORDER BY SUM(amount) DESC LIMIT 10;

It is not possible to use the WHERE condition on the aggregate function and that is where we use HAVING after the GROUP BY and specify the condition.

To display the customers who have spent more than 100 INR.

SELECT customer_id, COUNT(amount) FROM customer GROUP BY customer_id HAVING COUNT(amount)>=100;

To display the customers who were attended by the staff with id '2' and their total spending is more than 1000 INR.

SELECT customer_id, staff_id, SUM(amount) FROM customer WHERE staff_id=2  GROUP BY customer_id, staff_id HAVING SUM(amount)>1000;

JOINS

JOINS play an important role while analyzing data by combining multiple tables. There are four types of JOINS

INNER JOIN
LEFT OUTER JOIN
RIGHT OUTER JOIN
FULL OUTER JOIN

INNER JOIN - Compares between two tables on the specified columns and returns the data that matches in both the columns.

LEFT OUTER JOIN - Compares between two tables on the specified columns and returns the data that matches in the left table. If no value is present in the right table matching the left, then it returns the data as null.

RIGHT OUTER JOIN - Compares between two tables on the specified columns and returns the data that matches in the right table. If no value is present in the left table matching the right, then it returns the data as null.

FULL OUTER JOIN - Compares between two tables on the specified columns and returns the data that matches in both the columns along with mismatches with a value as null.

You can read more about JOINS here

INNER JOIN to merge two tables according to matching values of customer_id.

SELECT payment_id, customer.customer_id, first_name, last_name FROM customer INNER JOIN payment ON customer.customer_id = payment.customer_id;

FULL OUTER JOIN to merge two tables according to matching values of customer_id.

SELECT payment_id, customer.customer_id, first_name, last_name FROM customer FULL OUTER JOIN payment ON customer.customer_id = payment.customer_id WHERE payment.customer_id IS null OR customer.customer_id IS null;

LEFT OUTER JOIN to merge two tables according to matching values of customer_id based on the left table.

SELECT payment_id, customer.customer_id FROM payment LEFT OUTER JOIN customer on payment.customer_id = customer.customer_id WHERE payment.customer_id IS NULL;

RIGHT OUTER JOIN to merge two tables according to matching values of customer_id based on the right table.

SELECT payment.customer_id,customer_id FROM customer RIGHT OUTER JOIN payment on customer.customer_id = payment.customer_id WHERE payment.customer_id IS NULL;

You can also use nested joins to query from multiple tables.

SELECT customer_id, first_name, last_name FROM customer INNER JOIN payment ON customer.customer_id = payment.customer_id INNER JOIN billing ON payment.customer_id = billing.customer_id WHERE first_name='Sai' AND last_name='Kumar';

Other SQL Commands

To display the timezone of your current location.

SHOW TIMEZONE;

To get today's timestamp.

SELECT NOW();

To get the current time of the day.

SELECT TIMEOFDAY();

To extract month, day, year from the timestamp use the function EXTRACT which will return a numeric month, date, or year.

SELECT EXTRACT(MONTH FROM transaction_date) FROM customer;

To convert a timestamp into a specific string format like 'DD/MM/YYYY', the TO_CHAR function is used where you specify the timestamp column name as the first argument and followed by the format to be returned.

SELECT DISTINCT (TO_CHAR(payment_date,'DD/MM/YYYY')) FROM payment;

A sub query returns either a single data or a list of data which can be asserted with its parent query.
-- SUB QUERY

SELECT first_name || ' ' || last_name AS Full_Name FROM customer WHERE address_id IN (SELECT address_id FROM address WHERE district = 'Chennai');

Similar to conditionals in programming languages, SQL has a CASE statement that runs a check against the selected column and performs specified operations on it.

SELECT 
SUM(CASE citizen
    WHEN 'INDIAN' THEN 1
    ELSE 0
    END)as IN,
SUM(CASE rating
    WHEN 'NRI' THEN 1
    ELSE 0
    END)as NRI
FROM customer;

That will be all and there are few more concepts like views, procedures etc. which is much more advanced and I will cover those is my future posts.

Extent Reports made easy using Cucumber Adapter

Sainath Ramanathan — Tue, 09 Jun 2020 15:30:38 +0000

Report! Report! and yes we all need reports of whatever we do. This applies to test automation as well and we have a lot of options to do that. Cucumber gives out reporting feature which is readily available inside the cucumber package but it looks pretty old and not insightful. To make our lives easy we have a readymade solution for it and that's called Extent Reports.

Before getting into the details there are two ways to achieve this as mentioned below.

Long Route - Using the Extent Reports with customizable configuration
Short Route - Using the Extent Reports Adapter corresponding to the Cucumber version

The Long Route

If you are building a Maven-based project, add the following dependencies to your pom.xml

<!-- https://mvnrepository.com/artifact/com.vimalselvam/cucumber-extentsreport -->
<dependency>
    <groupId>com.vimalselvam</groupId>
    <artifactId>cucumber-extentsreport</artifactId>
    <version>3.1.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
<dependency>
    <groupId>com.aventstack</groupId>
    <artifactId>extentreports</artifactId>
    <version>4.1.5</version>
</dependency>

Note: If you are using Java 7 and below you don't need to add the second dependency but for Java 8 and above, you should add both the dependencies.

The next step is to create a configuration XML file. In your folder structure go to src/test/ and create a new folder called 'resources' (src/test/resources). Inside 'resources', create a new file called extent-config.xml (or any other name that you prefer) and add the following code in it. This XML defines how to display your report in the HTML page.

<?xml version="1.0" encoding="UTF-8"?>
<extentreports>
  <configuration>
    <!-- report theme --> <!-- standard, dark -->
    <theme>standard</theme>

    <!-- document encoding -->  <!-- defaults to UTF-8 -->
    <encoding>UTF-8</encoding>

    <!-- protocol for script and stylesheets -->   <!-- defaults to https -->
    <protocol>https</protocol>

    <!-- title of the document -->
    <documentTitle>BDD Cucumber Framework Reports</documentTitle>

    <!-- report name - displayed at top-nav -->
    <reportName>Android - Cucumber Report</reportName>

    <!-- global date format override -->  <!-- defaults to yyyy-MM-dd -->
    <dateFormat>yyyy-MM-dd</dateFormat>

    <!-- global time format override -->   <!-- defaults to HH:mm:ss -->
    <timeFormat>HH:mm:ss</timeFormat>

    <!-- custom javascript -->
    <scripts>
      <![CDATA[
        $(document).ready(function() {
        });
      ]]>
    </scripts>

    <!-- custom styles -->
    <styles>
      <![CDATA[
      ]]>
    </styles>
  </configuration>
</extentreports>

Now you have to configure the Runner class to map with Extent Reports.

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(
        features= "src/test/resources",
        tags = {"@RegressionTest"},
        monochrome = true,
        glue= {"stepDef"},
        plugin = {"com.cucumber.listener.ExtentCucumberFormatter:target/html/ExtentReport.html"})

public class RunnerTest {

}

Having set the plugin, the next step is to create an 'AfterClass' method to define the reporting parameters and to showcase the test results in the dashboard.

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(
        features= "src/test/resources",
        tags = {"@RegressionTest"},
        monochrome = true,
        glue= {"stepDef"},
        plugin = {"com.cucumber.listener.ExtentCucumberFormatter:target/html/ExtentReport.html"})

public class RunnerTest {
Reporter.loadXMLConfig(new File("src/test/resources/extent-config.xml"));
Reporter.setSystemInfo("AUT Name", "Make My Trip");
Reporter.setSystemInfo("Environment", "Production");
Reporter.setTestRunnerOutput("BDD Cucumber Framework Reports");
}

That's it! you have set up the reporter and when you execute the cases you can find the results under target/html/ExtentReport.html.

The Short Route

The Aventstack has always made our lives easy and to achieve this they have been releasing adapters for Cucumber versions 1, 2, 3, 4, and now for 5. Mind that 5 is in the test phase as of now since Cucumber is also new to 5, it is better to stick with version 4 of Cucumber at this point. You gotta pick the relevant adapter version corresponding to your Cucumber. I am using Cucumber 4 and I will be using the corresponding adapter which is 'extentreports-cucumber4-adapter'. Also for some, you may need to add the exclusions of 'cucumber-java' and 'cucumber-core' just to avoid the adapter from pulling its dependencies of Cucumber.

<dependency>
            <groupId>com.aventstack</groupId>
            <artifactId>extentreports-cucumber4-adapter</artifactId>
            <version>1.0.8</version>
            <exclusions>
                <exclusion>
                    <groupId>io.cucumber</groupId>
                    <artifactId>cucumber-java</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.cucumber</groupId>
                    <artifactId>cucumber-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

As mentioned in the above setup (in The Long Route) you may need to create a resources folder in 'src/test/' and initiate a file called extent.properties. This property file is similar to the configuration file mentioned previously but this is alight weighted file and you need to write just 3 steps to spin your report.

Step 1 - Tells the reporter to start the reporting
Step 2 - Leave the config field empty as it is pre-defined in the adapter
Step 3 - Specify where the report should be placed

extent.reporter.html.start=true
extent.reporter.html.config=
extent.reporter.html.out=test-output/HtmlReport/ExtentHtml.html

You're almost done. Just go to your Runner class and add the following plugin inside the CucumberOptions.

import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;

@RunWith(Cucumber.class)
@CucumberOptions(
        features= "src/test/resources",
        tags = {"@RegressionTest"},
        monochrome = true,
        glue= {"stepDef"},
        plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"})

public class RunnerTest {

}

This is so simple and you don't need to remember a lot of steps to set up the report. I prefer the short route to be employed and of course, if you need the customization, go for the long route. I have also added an example in my GitHub repository.

rksainath / MMT-CukeTest

This is a cucumber test framework with added reporting feature using Extent Report

Auto-Heal tests with recheck-web

Sainath Ramanathan — Fri, 05 Jun 2020 11:15:15 +0000

This post is going to talk about auto-healing and the first thing that comes to your mind is Artificial Intelligence (AI). I do agree with that and it has been a cliche these days to make everything intelligent. The problem with AI is that you need to pre-process data, build a feature, pick an algorithm, and start training the system which may eventually take some time to improvise and get the desired outcome. Long story short, we don't have enough time for that in some cases. I am not offensive to AI, I evangelize about it all the time but still, it can't be applied for everything as "one shoe doesn't fit for all".

Auto-Healing

Auto-Healing in terms of computer science refers to the computer program that can correct itself whenever it encounters a blocker or a new change that has been incorporated in the system. In a normal scenario, the script would fail to proceed due to a lack of knowledge. Auto-healing makes sure that the system keeps up and running and does not break during its course.

The kind of auto-healing that we are going to see here is specific to UI test automation where most of the automation engineers would agree that most of their time has been spent on fixing the code due to constant changes in the front-end. This is prevalent now as we have element attributes in HTML that keep changing dynamically because of the recent front-end frameworks that are being used. To overcome this problem without further ado let me introduce a framework called recheck-web which is from a company called retest.

recheck-web

recheck-web works based on 'Difference Testing' where a Golden Master is created during the first run and it is considered as the reference. The Golden Master is a combination of screenshot and XML file which will have all the attributes related to elements on a web page. The following is a brief introduction to recheck-web from the company's official GitHub page.

You can explicitly or implicitly create Golden Masters (essentially a copy of the rendered website) and semantically compare against these. Irrelevant changes are easy to ignore and the Golden Masters are effortless to update. In case of critical changes that would otherwise break your tests, recheck-web can now peek into the Golden Master, find the element there, and (based on additional attributes) still identify the changed element on the current website.

In the above image, the overall concept has been simplified into a diagram. It is prudent that the UI is always subjected to changes, having that as a problem recheck-web does the following actions,

A usual selenium test will break if it finds a locator is changed in the UI and if we have not updated the test script and throws NoSuchElementException
A selenium test wrapped with recheck-web will have a Golden Master where all the attributes of the elements are captured for the entire page (only if you pass a driver but if a web element is passed, it would only capture the attributes of that particular element)
If a test script is run against the new UI whilst having the old locators in place, it would not break rather the test will execute as it would refer to the Golden Master and compares the present state of the locator with previous history a.k.a the Golden Master
The test would successfully execute all the functional validations but eventually will finish it with a warning and in a result, it shows what is expected and what has changed
This not only applies to the locator changes but also this validates the CSS and any Style changes as well which benefits the visual validation
The last step would be the decision making step where the user can accept all the changes or ignore it to update the Golden Master

It might seem to be complex to grasp but I will debrief it in a while with a working example.

Setup

You might need to ensure that Java and Maven are installed in your machine/server (Java 8 is the minimum version to be used)
Create a maven project and add the following dependencies to your pom.xml

        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.6.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>de.retest</groupId>
            <artifactId>recheck-web</artifactId>
            <version>1.9.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
            <scope>test</scope>
        </dependency>

The next thing you would need is an URL or a demo application against which you can test the script. I have created a sample login application which we will consider for this example.

Selenium Test

We can write a simple selenium test for the login app. Also, make a note that you should specify whichever driver you are using and the following driver should be made available to execute. In this case, I am using a Chrome driver.

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import java.nio.file.Paths;

public class SeleniumTest {
    public WebDriver driver;

    @BeforeEach
    public void setupDriver() {
        System.setProperty("webdriver.chrome.driver","src/test/resources/drivers/chromedriver");
        driver = new ChromeDriver();
    }

    @Test
    public void breakableTest() throws Exception {
        String url = Paths.get("src/test/resources/loginpage.html").toUri().toURL().toString();
        driver.get(url);
        driver.findElement(By.id("username-field")).sendKeys("user");
        driver.findElement(By.id("password-field")).sendKeys("dev");
        driver.findElement(By.id("login-form-submit")).click();

        String result = driver.findElement(By.id("login-success-msg")).getText().equals("You have successfully logged in") ? "Login Pass" : "Login Fail";
        System.out.println(result);
    }

    @AfterEach
    public void windUp(){
        driver.quit();
    }
}

You could see from the above scenario that we need to explicitly make an assertion to find if the successful login message is displayed. Run this test and you will see that the test will pass. What if there are some UI changes that are made in the new release of your app. Consider the below example where you have the ID attributes 'username-field' and 'password-field'. It is now changed to 'username' and 'password' in the new release of the app.

<form id="login-form">
        <input type="text" name="username" id="username" class="login-form-field" placeholder="Username">
        <input type="password" name="password" id="password" class="login-form-field" placeholder="Password">
        <input type="submit" value="Login" id="login-form-submit">
</form>

But you could see that the selenium test script carries the id as 'username-field' and 'password-field' rather than 'username-field' and 'password-field'. This test would fail with NoSuchElementExeption.

org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#username"}

Unbreakable Selenium Test

recheck-web truly stands to its name 'Unbreakable' where it creates a Golden Master with which it compares the current state. To create a recheck-web test, simply call the RecheckDriver and pass the driver argument to it as how you would do a RemoteWebDriver invocation. It is so simple and you don't need to add anything else. Of course, there are methods like 'startTest', 'cap', 'check, 'capTest' which can be called using RecheckImplementation class but you don't need to worry about that as all these are encompassed in RecheckDriver class.

import de.retest.web.selenium.RecheckDriver;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.chrome.ChromeDriver;

import java.nio.file.Paths;
public class UnbreakableSeleniumTest {
    RecheckDriver driver;

    @BeforeEach
    public void setupRecheck() {
        System.setProperty("webdriver.chrome.driver","src/test/resources/drivers/chromedriver");
        driver = new RecheckDriver(new ChromeDriver());
    }

    @Test
    public void unBreakableTest() throws Exception {
        String url = Paths.get("src/test/resources/loginpage.html").toUri().toURL().toString();
        driver.get(url);
        driver.findElement(By.id("username-field")).sendKeys("user");
        driver.findElement(By.id("password-field")).sendKeys("dev");
        driver.findElement(By.id("login-form-submit")).click();
    }

    @AfterEach
    public void windUp(){
        driver.quit();
    }
}

Consider the below login HTML.

<form id="login-form">
        <input type="text" name="username" id="username-field" class="login-form-field" placeholder="Username">
        <input type="password" name="password" id="password-field" class="login-form-field" placeholder="Password">
        <input type="submit" value="Login" id="login-form-submit">
</form>

Now if you run the test it will fail for the first time as you don't have a Golden Master created yet.

Run the test again and you will see that the test will succeed but with some errors as mentioned below in the picture.

This assures that our present state is compared against the Golden Master. The errors that you see are some metadata and attributes that are compared against the Golden Master which can be ignored by writing them in the recheck.ignore file generated in .retest folder. Add attribute=grid-template-rows and metadata=window.height in the recheck.ignore file.

attribute=grid-template-rows
metadata=window.height

Re-run the test and it will succeed without any errors.

Let us now change the login HTML and tweak the ID to 'username' and 'password'.

<form id="login-form">
        <input type="text" name="username" id="username" class="login-form-field" placeholder="Username">
        <input type="password" name="password" id="password" class="login-form-field" placeholder="Password">
        <input type="submit" value="Login" id="login-form-submit">
</form>

If you run the test against this new change, the execution will not break but it will refer to the Golden Master that is available since the first run and it will compare the similar attributes that were present earlier and make a one-to-one connection, thereby finds the new element without even throwing any exception. Obviously, the test will report with errors (i.e. the locator change) that were found during the execution.

Now you can see the difference that the expected was 'username-field' but the actual ID attribute was 'username'. The same applies to the ID attribute 'password'. It is not a rule that Golden Master always refers to the outcome that was derived from the first run. We need to update the Golden Master to the latest stable UI changes by liaising with the development community. So we could clearly see that there are often changes happening in and out of the script. This might look like Déjà vu to you as recheck-web has a CLI called recheck.cli which is similar to the operations of GIT. It is imperative that every time you cannot manually update the recheck.ignore file to filter the result and to commit or accept the new UI changes to the Golden Master. This is where recheck.cli would help us in speeding up the process.

recheck.cli

In order to install and setup the CLI, refer to this documentation. To ensure if the recheck is working, restart your CMD/Terminal after the installation and type recheck. It should return with list of options that are available.

$ recheck

Usage: recheck [--help] [--version] [COMMAND]
Command-line interface for recheck.
      --help      Display this help message.
      --version   Display version info.
Commands:
  account     Allows to log into and out of your account and show your API key.
  help        Displays help information about the specified command
  commit      Accept specified differences of given test report.
  completion  Generate and display an auto completion script.
  diff        Compare two Golden Masters.
  ignore      Ignore specified differences of given test report.
  show        Display differences of given test report.
  version     Display version info.

Every recheck test will generate a report in target/test-classes/retest/recheck/YOURTESTNAME.report. You could run this report using the recheck.cli by hitting the following command by navigating to the report folder.

$ recheck show YOURTESTNAME.report

To accept all the recent changes in your test related to ID attributes, you can simply commit all at once by issuing the below command,

$ recheck commit --all YOURTESTNAME.report

If you go back and check your retest.xml the attributes would have changed from 'username-field' to 'username' and 'password-field' to 'password'. Now that you have updated the Golden Master, you have to update the corresponding test scripts as well else the test would fail because we don't have a history of 'username-field' or 'password-field' anymore. In order to auto-heal, retest has a GUI called review which is an insightful dashboard that can automatically make the relevant changes in Golden Master and in the test script.

Also, recheck-web has an important feature called virtual ID. It creates the virtual ID for all the states viz. username, password, and login button. If you want your test to be completely unbreakable, using virtual IDs will make your life worry-free. The retestId has to be imported from the package de.retest.web.selenium.By.

import de.retest.web.selenium.RecheckDriver;
import de.retest.web.selenium.By;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.chrome.ChromeDriver;
import java.nio.file.Paths;

public class UnbreakableSeleniumTest {
    RecheckDriver driver;

    @BeforeEach
    public void setupRecheck() {
        System.setProperty("webdriver.chrome.driver","src/test/resources/drivers/chromedriver");
        driver = new RecheckDriver(new ChromeDriver());
    }

    @Test
    public void unBreakableTest() throws Exception {
        recheck.startTest();
        String url = Paths.get("src/test/resources/loginpage.html").toUri().toURL().toString();
        driver.get(url);
        driver.findElement(By.retestId("usernamefield")).sendKeys("user");
        driver.findElement(By.retestId("passwordfield")).sendKeys("dev");
        driver.findElement(By.retestId("loginformsubmit")).click();
    }

    @AfterEach
    public void windUp(){
        driver.quit();
    }
}

Check the recheck-web documentation for more insights and you can also pull my sample implementation from GitHub.

Hyperledger Fabric (v1.1) Setup on Ubuntu 18.04 — Part III

Sainath Ramanathan — Tue, 17 Dec 2019 00:00:00 +0000

In the previous section Part - I we saw about the necessary pre-requisites for running the Hyperledger Fabric and in this final section, we will finally install the essential command-line tools to run the fabric development environment.

Install Composer-CLI using npm which is the essential aspect of fabric development using composer.

npm i -g composer-cli</span>

We will need Composer Rest Server in order to expose our business network as RESTful APIs.

npm i -g composer-rest-server</span>

In the previous post, you would have seen about Yeoman which is a scaffolding tool that we will be using here in order to pack the application assets related to fabric.

npm i -g generator-hyperledger-composer</span>

There is an online app called Playground for developing business network applications using hyperledger fabric. If you would like to have a local copy of it just install the package and use it from your local machine.

npm i -g composer-playground</span>

Create a folder called fabric-dev-servers and navigate to the folder.

mkdir ~/fabric-dev-servers cd ~/fabric-dev-servers</span>

Using CURL command fetch fabric-dev-servers tar file and unpack it using tar -xvf by passing the file to it.

curl -O https://raw.githubusercontent.com/hyperledger/composer-tools/master/packages/fabric-dev-servers/fabric-dev-servers.tar.gz tar -xvf fabric-dev-servers.tar.gz</span>

Unzipping the folder will give you a list of files out of which you have to run downloadFabric.sh and prior to this kindly set the environment for the desired fabric version as mentioned in the code below.

cd ~/fabric-dev-servers export FABRIC_VERSION=hlfv12 ./downloadFabric.sh</span>

The downloadFabric.sh would install a list of docker images required for running the fabric. You can check the list of images using the command docker images. Ensure that you have all the images in the list as mentioned below in the screenshot.

Thats it and you are done. Before finishing there are few things to be noted. If you are using Ubuntu as a cloud instance or connecting a remote Ubuntu server, ensure that you dont do the fabric installation as a root user. Kindly create a new user and change your profile to that new user that you have created. At times if you dont have the right to perform or run docker commands, use sudo and execute it.

Hyperledger Fabric (v1.1) Setup on Ubuntu 18.04 — Part II

Sainath Ramanathan — Wed, 11 Dec 2019 00:00:00 +0000

This part will talk about the pre-requisites for installing Hyperledger Composer. If you need assistance on the choice of platform, refer to Part - I for more details.

Pre-requisites

NodeJS is essential for hyperledger composer as the development will be based on javascript. Install version 8.x as fabric relies on it and don't install the latest version of the node. To make it more switchable between the latest and version 8.x, one could install Node Version Manager so that you dont need to stick to 8.x all the time and use only when the fabric demands it.

Open up your terminal and run the below-mentioned command to install NVM. Note: Ensure that you have CURL installed prior to running this command.

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash</span>

Once you are done with the installation, run the following command to install node 8.x. If you dont mention the version, the latest stable version will be installed. Once the node is installed, tell nvm to use 8 as the default node environment. Finally, run the last line to check if you have installed version 8.x. For more details refer here.

nvm install 8 nvm use 8 node -v</span>

Python is by default available in all major GNU/Linux distros. If your choice of the distro is not listed or new then check here on the steps to install python in it. It is essential that you install Python 2.7.x and not Python 3.x.

Git is one of the most widely used source control management feature and it can be installed by running the command below. For more details on git, refer here.

sudo apt-get install git</span>

Yeoman is a client-side web scaffolding tool and it is primarily used by fabric to render hyperledger composer generator that is scaffolded using yeoman. Check the docs here.

npm install -g yo</span>

Docker is a container that relieves one from worrying about the dependencies for the application and wraps up all the essentials that are needed to run your application as one docker image. Docker CE (Community Edition) is required to be installed and there are a series of steps involved while installing Docker for Ubuntu. Check the documentation here for more clarity and follow the instructions.

Visual Studio Code is the most recommended IDE for fabric development as we have the hyperledger composer plugin for VS Code. Once you install the editor, go to extensions and search for hyperledger composer plugin and install. You can also find the plugin in Visual Studio Marketplace.

sudo snap install --classic code</span>

In the next Part - III, we shall see about the installation of Hyperledger Fabric tools.

Hyperledger Fabric (v1.1) Setup on Ubuntu 18.04 — Part I

Sainath Ramanathan — Mon, 02 Dec 2019 00:00:00 +0000

This post is going to talk about the installation of Hyperledgers tool called Hyperledger Composer which is used to create business networks and enables business analysts and developers to create smart contracts and blockchain applications.

Before we jump into the installation of the composer tool, I would like to give you clarity on the choice of the platform that best fits for the development.

Platform Support

Windows

If you own a Windows machine then I would recommend that you install a virtual machine like VirtualBox / VMWare and then spin an Ubuntu image in it. Since the fabric heavily relies on the Unix environment and docker it is suggested that we use a Linux variant for installing the fabric. Native Windows support is not there yet for Hyperledger Fabric as of (April 2019)

MacOS

If you own a Mac OS then it is not going to be a problem as the environment itself is based on Unix. The only aspect that you have to ensure is sufficient disk space

Ubuntu

If you are on Ubuntu then it is a cake-walk for you who enjoys the perks of being a Linux user.

Cloud Infrastructure

Cloud platform comes to the rescue irrespective of what operating system you run. It is so handy that you can just ssh to the instance that you have created in the cloud. There are many providers viz. Amazon Web Services, Microsoft Azure, Google Cloud Platform, and so on. You can refer to this page for detailed support by cloud providers for Ubuntu. I am not going to guide you through in choosing the cloud provider nor in creating an instance but I can show you a sample on AWS as a reference.

Kindly visit the link to check the AWS marketplace for subscribing to Ubuntu 18.04 LTS Bionic image.

Choose the T2-medium tier as the option while choosing the image.

T2 medium is required as the Fabric needs at least 4GB of memory. Kindly note that charges for t2.medium may vary from other models. Please ensure that you check the cost per hour before spinning as the running EC2-instance may incur charges. You can refer to the EC2 documentation here.

It is important to note that in the sequence of subscribing to the t2.medium ec2 instance you will be landed on a specific page where you will be asked to define the security group for the instance to communicate with the outside world. By default, SSH will be open, adding to this you need to open up all the TCP ports for your IP (NOTE: It is a risk to open TCP ports to all IPs and be specific on which IP you are going to use). Kindly ensure that you refer the documentation/experts in order to define the security group settings.

Now that we are done with the choice of platform, in the following series we can start installing the pre-requisites.

Blockchain vs Distributed Database

Sainath Ramanathan — Sun, 10 Mar 2019 00:00:00 +0000

This debate has been prevalent since the inception of blockchain. Always there is a counter-argument whenever people talk about the way we handle data either it is distributed or centralized. I am not going to judge between the two but give you an overview of how it will operate with different use cases.

Distributed Database

Consider you have a system where you need to pass around the document/digital asset from one entity to another. It should be a system where all the entities should be known to each other. This is where distributed databases come in to play. Some of the features a system must possess in order to achieve the distribution of data,

Periodic synchronization of data
Limits the access to all the entities and defines the access privileges
Universal update of data across the system
Ability to handle multiple users and provide the same data as served to others
Depends on an Admin to monitor and operate the system

So, based on the above it is clear that distributed data is apt for the scenarios where you share the data among known-groups. For example, if IBM has a group of companies that it owns, it is imperative that they form a distributed database among them. This is because you know all the entities participating and there is less chance that the data may get tampered (unless there is a DDOS) and it is governed by a database admin who has the strong privileges to handle the data. This is where blockchain is unnecessary in terms of the trust but in terms of data tamper and security, it is worthy that you have it implemented. Another example would be the CRUD application which will never need blockchain unless there are multiple players involved in it.

Blockchain

The word Blockchain is a cliche now. Every company and its experts are constantly exploring ways to implement blockchain in their business. It is a buzzword that is being used in every hackathon, seminars, workshops, and even in tea-shops. Thanks to Satoshi Nakamoto that he/she is anonymous else their mailbox would be bombarded with queries. Blockchain is more like a suit. It is not fit for all but tailored for specific needs. One cant compare an ocean to a glass of water similarly blockchain is not a solution to all the cases. One has to use it worthy for the real challenge that needs to be addressed among the heterogeneous group.

Why we focus blockchain for heterogeneous groups?

Blockchains verdict runs on transparency, tamper-proof and autonomous systems. When I mean heterogeneous there involves multiple players who dont trust each other. You could have financial institutions, educational, internet, media, and so on. This cross-contamination and sharing of data among them can be achieved by blockchain. Blockchain will set up a node for every participant and group them to specific channels where they can transfer data. The data here being a unique ciphered content where non-trusting parties can check the authenticity of the data received by verifying with the hash of original data. This not only introduces transparency and tamper-proof ingredients but also a ledger that will be maintained to maintain the audit trail of all the transactions that have happened. There is no need for any middleman/admins to control the data as we have contracts (software code) that are written as business logic which will take care of the transfer.

The inference is, one cant vote which is best simply by putting the technology in front but the use case is the one that will decide which is better.