Forem: Modgil

Breaking Down Effect TS : Part 2

Modgil — Sun, 09 Mar 2025 19:31:40 +0000

In the previous article, we introduced Effect-TS, a library designed to tackle common challenges in application development, such as handling complex side effects, managing concurrency, and error handling. Effect-TS is ideal for developers who seek robust applications with a focus on consistency, type safety, and advanced concurrency, all within a functional programming paradigm. We briefly discussed the basics of effects, including their type and how to create and execute them.

In this post, we will delve deeper into creating and executing effects, providing an example to build your confidence in using Effect-TS.

Unlike standard TypeScript functions, effects in Effect-TS are neither inherently synchronous nor asynchronous; they simply describe computations that can either succeed or result in errors.

Let's first start with

`Effects with Synchronous computations`:

Synchronous computations execute sequentially, blocking further execution until they complete, producing immediate and deterministic results.
In Effect-TS, you can represent synchronous computations using Effect.sync.

const generateRandomNumber = Effect.sync(() => Math.random());

Effect.sync always returns a success value. If it throws an error, that indicates a defect rather than a typical effect failure. Therefore, use Effect.sync only when you are certain that your operation will not fail

Executing generateRandomNumber directly will not yield the desired result, as effects are lazy and defer execution until required. To execute the effect and obtain the computation's result, use:

Effect.runSync(generateRandomNumber)

This will execute the sync effect immediately.

When you know that your operation might fail, use Effect.try. This structure allows you to handle exceptions functionally:

const parseJson = (jsonString: string): Effect.Effect<unknown, Error, unknown> =>
  Effect.try({
    try: () => JSON.parse(jsonString),
    catch: (error) => new Error(`Failed to parse JSON: ${(error as Error).message}`),
  });

To execute this effect, run it with Effect.runSync:

const inputString ='{"name": "Alice", "age": 30}';
try {
Effect.runSync(parseJson(inputString)); 
} catch (error) {
  console.error(error);
}

This executes the effect synchronously, catching and logging any parsing errors.

`Effects with Asynchronous computations`:

Asynchronous computations involve tasks that execute independently of the main program flow, allowing other operations to proceed while waiting for a result.

In Effect-TS, asynchronous computations are managed using the Effect.tryPromise function. This function wraps an asynchronous operation, such as a Promise, into an Effect, allowing you to handle asynchronous processes in a functional, type-safe manner.

const fetchData = (url: string): Effect.Effect<unknown, Error, Response> =>
  Effect.tryPromise({
    try: () => fetch(url), 
    catch: (error) => new Error(`Failed to fetch data: ${(error as Error).message}`),
  });

To execute an asynchronous effect, you use Effect.runPromise, which handles the Promise-based nature of the operation:

const url = "https://api.example.com/data";
Effect.runPromise(fetchData(url))
  .then(data => console.log(data))
  .catch(error => console.error(error));

`Avoid Executing Effects Within Other Effects`:

Always aim to execute effects at the boundaries of your program, such as in main functions. Running effects inside other effects negates the benefits of using an effect system, as it blurs the clear separation between effect description and execution. By maintaining this separation, you ensure that your program remains modular, predictable, and easier to test and maintain.

`Combinators: Combining, Transforming, and Managing Effects`

Combinators provide a framework for creating expressive and modular effect descriptions without executing them. By avoiding the nesting of effects and using combinators instead, you leverage the full power of effect systems, aligning with functional programming principles.

Instead of executing effects within other effects:

const initialEffect = Effect.succeed(4);
const transformedEffect = Effect.sync(() => {
  const result = Effect.runSync(initialEffect);
  return result * 2;
});

You can use combinators to maintain separation for e.g.

Effect.map: This takes an effect and lets you change the result it produces using a simple function.

const initialEffect = Effect.succeed(4);
const transformedEffect = Effect.map(initialEffect, x => x * 2);

This transformation does not execute the effect; it only modifies the outcome once the effect is run.

Effect.flatMap: It takes the output of an effect, which produces another effect, and flattens the result. This is commonly used when need to perform sequential computations that result in an effect, allowing the next computation to depend on the value produced by the first effect.

const initialEffect = Effect.succeed(10);
const chainedEffect = Effect.flatMap(initialEffect, x => Effect.succeed(x * 2));

pipe: It allows chaining functions in a left-to-right manner, increasing readability

const pipeline = pipe(
  Effect.succeed(10),
  Effect.map(x => x + 5),
  Effect.map(x => x * 2)
);

and there are many more available to help you handle different scenarios effectively....

Let's build an application called dataTransformer to download data from an S3 bucket, transform it, and then re-upload it back to S3. This example will provide a practical understanding of Effect-TS in action.

First, we create an S3 client and define the functions needed for our tasks:

import { S3Client, GetObjectCommand, PutObjectCommand } from "@aws-sdk/client-s3";
import { Effect, pipe } from 'effect';

const s3Client = 
  new S3Client({
    region: "your-region",
    forcePathStyle: true,
  });

const downloadData = (bucket: string, key: string): Effect.Effect<string, Error> =>
  Effect.tryPromise({
    try: async () => {
      const response = await s3Client.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
      return response.Body?.transformToString() ?? "";
    },
    catch: (error) => new Error(`Failed to download data: ${error.message}`),
  });


const transformData = (data: string) =>
  Effect.sync(() => {
    return data.toUpperCase();
  });


const uploadData = (bucket: string, key: string, data: string) =>
  Effect.tryPromise({
    try: () =>
      s3Client.send(
        new PutObjectCommand({
          Bucket: bucket,
          Key: key,
          Body: data,
        })
      ),
    catch: (error) => new Error(`Failed to upload data: ${error.message}`),
  });

Next, we compose the transformer function using Effect-TS combinators:

const dataTransformer = (sourceBucket: string, sourceKey: string, targetBucket: string, targetKey: string) =>
  pipe(
    downloadData(sourceBucket, sourceKey),
    Effect.flatMap(transformData),
    Effect.andThen((transformedData) => uploadData(targetBucket, targetKey, transformedData))
  );

Finally, we define the main function to execute the effectful computation:

Effect.runPromise(dataTransformer("bucket-name", "key", "target-bucket-name", "target-key"))
  .then(() => console.log("Data processed successfully"))
  .catch(console.error);

I hope we are now clear on the creation and execution of effects. Let's continue in the next part, where we'll explore efficient error handling, concurrency management, effect data types, resource management, and much more. Stay tuned!

Breaking Down Effect TS : Part 1

Modgil — Thu, 06 Mar 2025 14:56:11 +0000

If you're looking to develop applications that are type-safe, handle errors efficiently, manage concurrency effectively, and tackle complexity with improved code quality, all while applying functional programming principles, then Effect-TS is the ideal tool for you.

Effect-TS is a powerful TypeScript library designed to bring the principles and advantages of functional programming to the forefront of TypeScript development. It focuses on enabling developers to manage effects—such as state, errors, and asynchronous operations—in a functional, declarative, and type-safe manner.
Effect is inspired by ZIO (a Scala library)

Before diving into Effect-TS, let's explore what Functional Programming is and the problems it solves.

Functional Programming is a programming paradigm where functions are treated as first-class citizens. In this context, functions always yield the same result given the same input, highlighting the concept of immutability. Functional Programming enhances predictability and simplifies debugging through pure functions and immutability, ensuring consistent behaviour. It promotes higher modularity and reusability by encouraging the development of small, composable functions, which can be easily reused across various application modules. This leads to more robust, maintainable, and scalable software solutions compared to other paradigms.

Key Characteristics of Functional Programming:

- Immutability of Data/State:
This refers to the principle that data or state should not be modified after its creation, promoting stability and predictability in your code.

- Pure Functions:
These functions always produce the same output for the same input and have no side effects. They do not rely on any external state, ensuring deterministic behaviour.

Benefits of Pure Functions:

Predictability and Composability: Pure functions are predictable and can be easily composed to build complex logic.
Concurrency: Since pure functions do not modify external states, they are inherently thread-safe.

- Functions as First-Class Citizens:
In functional programming, functions are treated like any other value, meaning they can be assigned to variables, passed as arguments, returned from other functions, and stored in data structures. This enables the use of higher-order functions, which accept functions as arguments and return functions as output. For example, array.map() is a common higher-order function.

- Preference for Recursion Over Loops:
Functional programming prefers recursion to loops because recursion is more declarative and avoids mutable state by relying on the call stack to manage state. This approach avoids mutation by repeatedly calling the function with new arguments.

- Referential Transparency:
An expression is referentially transparent if it can be replaced with its corresponding return value without altering the program's behaviour. This improves code readability and reasoning.

How Effect-TS is aligned with Functional Programming Principles?
Effect-TS aligns with functional programming principles by emphasizing purity and immutability, managing side effects explicitly and predictably. It leverages TypeScript's type system to ensure type safety, reducing errors. By supporting effect composition and maintaining referential transparency, Effect-TS enables the creation of modular, reusable, and robust applications.

Apart from this Effect main feature includes(as per Effect-TS documentation):

Let's understand how to use Effect:

In Effect-TS, we use effects to manage a function's side effects. Normally, a function may either succeed or throw an exception, and handling exceptions can become challenging when managing numerous functions. Effect-TS provides a structured way to handle these scenarios by defining the return type of a function as an Effect.

An Effect is represented as Effect<Success, Error, Requirements>,
where:

Success is the type of value returned when the function succeeds.

Error is the type of error returned if the function fails.

Requirements represents the dependencies required by the function.

Effect<A, E, R> abstracts complex computations, allowing you to express operations that might fail, return a result, and need certain inputs. By handling both synchronous and asynchronous computations through this abstraction, Effect-TS streamlines functional programming in TypeScript, ensuring side effects are managed safely and predictably.

Effect functions return values of the effect type, enabling lazy execution. To execute and resolve the values, you need to explicitly run the effect after invoking the function.
This is different from Promises, which are initiated immediately as they are defined.

Let's start with the Basics:

The key advantage of using effects in Effect-TS is that they define computations without executing them immediately. This means you can set up your workflow and only execute it when you are fully prepared and satisfied with its setup. Think of it as mapping out a plan and only putting it into action when you're ready.

Creating Effects:
When we say:

export const succeed = Effect.succeed(5);

Here, succeed creates an Effect that encapsulates its argument 5 in the success channel (A in Effect<A, E, R>).

Type of succeed is Effect.Effect<number, never, never>

This indicates it will produce a success of type number and cannot fail or require any environment.

export const fail = Effect.fail(3);

fail creates an Effect that encapsulates its argument 3 in the failure channel (E in Effect<A, E, R>).

Type of fail is Effect.Effect<never, number, never>;

This denotes that it results in a failure of type number and does not produce a success or require any environment.

There are multiple ways to create effects in Effect-TS, whether for synchronous or asynchronous operations. To illustrate the creation of an asynchronous effect, let's consider the example of uploading a file to an S3 bucket.

export const uploadFileToS3 = (
  s3Client: S3Client,
  bucketName: string,
  key: string,
  body: Buffer | Uint8Array | Blob | string | ReadableStream
) =>
  Effect.tryPromise({
    try: () =>
      s3Client.send(
        new PutObjectCommand({
          Bucket: bucketName,
          Key: key,
          Body: body,
        })
      ),
    catch: (error) =>
      new Error(`Error uploading file to S3 bucket - ${bucketName}: ${error.message}`),
  });

The uploadFileToS3 function creates an effect that attempts to upload a file to an S3 bucket using the provided S3Client, handling success with the upload command execution and capturing errors with a descriptive error message. It uses Effect.tryPromise to wrap the asynchronous operation, ensuring structured error handling and functional program flow.

Executing Effects:
As discussed earlier, up to this point, we have only constructed Effect values, and none of the computations we defined have been executed. Now, to execute the effects, just as there are various ways to create effects, there are also different ways to execute them. One such example of executing the above-created effect is:

Effect.runPromise(uploadFileToS3(s3Client, bucketName, key, fileContent));

This executes the uploadFileToS3 effect using the provided inputs, initiating the file upload process to the specified S3 bucket.

I hope this introduction to Effect-TS has sparked your interest! There’s so much more to explore.

Part 2 continued

LocalStack - Your Local Cloud Partner ☁️🤝

Modgil — Mon, 10 Jun 2024 08:41:27 +0000

Do you want to test your AWS resources locally before deploying to AWS? Do you aim to produce high-quality code and write integration tests for AWS services? Do you need to test your resources in a CI/CD pipeline to avoid mistakes in applications? Well, the solution for you is LocalStack.

LocalStack provides a platform to create AWS resources on your local machine. It's a cloud service emulator that runs in a single container and allows you to simulate AWS services locally and in CI environments.

Why Use LocalStack? 🤔

Local Development: Simulate AWS services on your local machine, enabling faster and safer development without the risk of incurring costs or affecting live environments.
Quality Code: Test your code against AWS APIs locally, ensuring it meets high standards before deployment.
Integration Testing: Write and run integration tests for AWS services, ensuring all components work seamlessly together.
CI/CD Pipelines: Test your infrastructure in CI/CD pipelines to catch errors early and avoid costly mistakes in production.

Features and Limitations 🚀

LocalStack supports a wide range of AWS services but does come with some limitations. It does not provide the functionality to use and create all AWS resources. Additionally, not all features are available for free.

Community Version: Provides access to core AWS services such as S3, SQS, DynamoDB, Lambda, etc., at no cost.
Pro Version: Offers access to additional AWS services and enhanced features.

Here is a list of community and pro version resources supported by LocalStack, along with information on the level of support compared to actual AWS resources.

Getting Started with LocalStack 🛠️

Before you start, ensure you have a functional Docker environment installed on your computer.

Installation. 📥

There are several ways to get started with LocalStack:

LocalStack CLI: The quickest way to start. You can create AWS resources through the terminal.

Install via brew.
Download the pre-built LocalStack CLI binary directly.
Install using pip python3 -m pip install localstack

Alternatives:
Other methods include LocalStack Desktop, LocalStack Docker Extension, Docker-Compose, and Docker. You can find more details on these alternatives.

For this guide, we will use Docker-Compose to create DynamoDB and perform various actions on it.

Interacting with LocalStack:

To interact with LocalStack, you can use the AWS CLI or the LocalStack AWS CLI in the command line interface.

Creating DynamoDB with Docker-Compose 🗄️

Set Up Docker-Compose:
Create a docker-compose.yml file with the following content

version: "3.8"

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}"
    image: localstack/localstack
    ports:
      - "127.0.0.1:4566:4566"            # LocalStack Gateway
      - "127.0.0.1:4510-4559:4510-4559"  # external services port range
    environment:
      # LocalStack configuration: https://docs.localstack.cloud/references/configuration/
      - DEBUG=${DEBUG:-0}
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"

The docker-compose.yml file specifies LocalStack to run as a service using the localstack/localstack image, exposing ports 4566 for AWS service simulations. The volumes configuration maps a local directory to LocalStack's temporary storage.

Start LocalStack:
Ensure LocalStack is running with the docker-compose up command.

Create DynamoDB Table:
Open a terminal and use the AWS CLI to create a DynamoDB table:

aws --endpoint-url=http://localhost:4566 dynamodb create-table --table-name my-table \
  --attribute-definitions AttributeName=ID,AttributeType=S \
  --key-schema AttributeName=ID,KeyType=HASH \
  --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

Verify Table Creation:

List the tables to verify that your table has been created:

aws --endpoint-url=http://localhost:4566 dynamodb list-tables

Automating Resource Creation with Scripts:

You can also write scripts to create different resources and mount the script folder to LocalStack volumes. This approach allows you to easily create all the resources in one go when starting LocalStack with Docker-Compose. This method is recommended for setting up a local development environment since you can't always write commands manually in the terminal.

Start using LocalStack today to streamline your AWS development and testing processes.

🌟 Stay connected for more insights on using LocalStack with various cloud development tools. 🌟

Let us Understand D3

Modgil — Wed, 28 Sep 2022 20:24:16 +0000

Example 1: Create Rectangle using D3
First of all, define a container in html code

<div id="chart"></div>

Now we have container with id as chart
We will select this container and will try to manipulate this DOM element using d3.

DOM selection

d3.select(selectorName)

It will select the first element with the provided name.

d3.selectAll(selectorName)

It will select all elements with the provided name.
So in the example , it will select the chart container.

d3.select("#chart")

DOM Manipulation

After selecting DOM, we need to modify it.
In given example we are adding an element inside the selected element, using "append" method.

append("svg")

To get and set the attribute value, we need to specify it in "attr" method

.attr("width", 600).attr("height", 600)

It will set width and height of svg element as 600 and 600 respectively.

We have several other DOM manipulation methods also.

After selecting the chart container and appending the svg, we will append "rect" , in our svg. "rect" is predefined in d3, similar kind of shapes are also available, where we need to specify the attributes value to get the shape created.

.append('rect')

It will append the rectangle element

.attr('x', 100)
.attr('y', 120)

It will get and set the coordinates of our shape for its positioning in our DOM element.

.attr('width', 100)
.attr('height', 60)

It will get and set the width and height of our rectangle created.

.attr('fill', 'pink')

It will fill the color to our rectangle.

The whole code to create rectangle in our chart container will look like this

d3.select("#chart").append("svg").attr("width", 600).attr("height", 600)
.append('rect')
  .attr('x', 100)
  .attr('y', 120)
  .attr('width', 100)
  .attr('height', 60)
  .attr('fill', 'pink');

Result:

Now let's move to some actual graph example

Example 2: Create pie chart using given data

data = [
{name: "A", score: 18},
{name: "B", score: 16},
{name: "C", score: 14},
{name: "D", score: 22},
{name: "E", score: 5},
{name: "F", score: 14}
];

First of all, define a container in html code

<div id="chart"></div>

Now select the DOM and append svg

var svg = d3.select("#chart")
.append('svg')
.attr('class', 'pie')
.attr('width', width)
.attr('height', height);

Now append "g" element to svg, g element is used to group svg shapes.

var g = svg.append('g')
.attr('transform', 'translate(' + (width/2) + ',' + (height/2) + ')');

transform attribute will help us to transform our element, we are using translate function here so it will move our shape to given x and y value.
There are many other transform functions available like rotate, scale etc.

var arc = d3.arc()
.innerRadius(0)
.outerRadius(100);

d3.arc() will return arc generator. innerRadius and outerRadius will set inner and outer radius value of each arc.

var pie = d3.pie()
.value(function(d) { return d.score; })

d3.pie() function will return pie generator.

var path = g.selectAll('path')
  .data(pie(data))
  .enter()
  .append("g")  
  .append('path')
  .attr('d', arc)
  .attr('fill', (d) => color(d.data.score))
  .style('opacity', opacity)
  .style('stroke', 'white')

Now we will append path to our g element, when we need to create complex figures other than simple ones like rectangle and lines we use path element. Path provides d attribute that defines its shape and position in addition to other attributes which are provided by other elements.

The whole code to create pie chart :

var data = [
  {name: "A", score: 18},
  {name: "B", score: 16},
  {name: "C", score: 14},
  {name: "D", score: 22},
  {name: "E", score: 5},
  {name: "F", score: 14}
];

var width = 400;
var height = 400;
var opacity = .8;

var radius = 95
var color = d3.scaleOrdinal(d3.schemeCategory10);

var svg = d3.select("#chart")
.append('svg')
.attr('class', 'pie')
.attr('width', width)
.attr('height', height);

var g = svg.append('g')
.attr('transform', 'translate(' + (width/2) + ',' + (height/2) + ')');

var arc = d3.arc()
.innerRadius(0)
.outerRadius(100);

var pie = d3.pie()
.value(function(d) { return d.score; })

var path = g.selectAll('path')
  .data(pie(data))
  .enter()
  .append("g")  
  .append('path')
  .attr('d', arc)
  .attr('fill', (d) => color(d.data.score))
  .style('opacity', opacity)
  .style('stroke', 'white')

Result:

In next blog we will discuss some advanced charts.
Stay tuned !!!!!

D3, why D3?

Modgil — Wed, 28 Sep 2022 12:52:51 +0000

What if you want to create a chart that no one has ever used and only you know its functionality ,only you know it looks and you don't know its name, it is completely a new dish which you want to make. In this case what will you do? You can consider using d3.

What is D3?
Data Driven Documents, if you want to make charts in your application that are using dynamic data then d3 is the right choice. D3 serves you a lots of chart options to visualise data, you can customise your charts, can remove or add any functionality from your chart. D3 provides more flexibility and control as it directly manipulates the DOM.

Why D3? Benefits of using d3

Currently D3 is famous and has large community of users as compared to other libraries.

D3 is more than a normal visualisation library, it can help to create animations, diagrams and many more as it works with html, css and svg.

-> D3 helps us to create customised charts, unlike many other available charts libraries where we just use the given code template to create chart, these can be easy to use but comes with lots of restrictions as we'll not be able to modify the charts according to our need, we are just left with their chart design, as a result chart look and experience will be too common whereas, d3 does not provides us such templates, it provides us helper functions, these helper functions works at basic level like DOM selection, creating axis. Moreover d3 provides us the flexibility to create our own helper functions.
In short, only we define each and every thing in our graph while using d3, which helps us to make more customised charts.

-> In other chart libraries, while using a specific type of chart, the structure of the input data is already defined, so we are restricted to provide same structure of input data, whereas in d3 we can customise it as well.

->D3 is open source, large number of chart options are available, many examples are also available, large community and users as compared to others.

-> D3 can be easily used with other JS based frameworks like react, angular.

Problems we can face while using d3:

-> If you are new to d3 then you might find the syntax a bit confusing, but we have a solution for that too.
Start by practicing the simplest chart or may be start by creating a simple square diagram using d3. Notice how d3 is manipulating DOM< how helper functions are working.
In next blog we'll try to understand working of d3 in simplest way.

Other chart frameworks are easy to use but as you know they come with lot of restrictions.

-> No doubt that d3 is compatible with JS based frameworks but sometimes you can face problems while using d3, just look if you are using the right version of d3 that is compatible with your JS framework.

Let me know your thoughts about d3 in comments and what library do you use for making charts in your applications?

Designing REST APIs

Modgil — Fri, 17 Jun 2022 06:45:18 +0000

What are API's and why do we need them?

API (Application Programming Interface) comes into picture when you want to interact with a system to retrieve some information or perform a task. API's act as mediator between the client and server. It serves the information to the client from the server whenever called.

There are many advantages of API's like API's make data accessible, API's help in making product seamless from start to finish.

What are REST API's?

Representational State Transfer, a way of accessing web services in a simple way. In simple words it is an API which follows REST Design Principles. REST API's provide high flexibility and easiness to developers as REST API's can be developed using any programming language and supports a variety of data formats. Only requirement is that it should follow the REST Principle.
A Restful system includes a client who requests for the information and a server who has the information.It is important to create REST API according to industry standards which results in ease of development and easy client adoption.
Architectural Constraints of RESTful API:

Uniform Interface
Stateless
Cacheable
Client-Server
Layered System
Code on Demand We will discuss these REST API’s constraints in detail, we should consider these points before designing an API.

-> Client-Server : API should follow client server architecture. A server having many endpoints for different functionalities and client (Application) consuming them using HTTP requests. There should not be any dependency between client and server applications. The only information required for a client should be URI (Uniform Resource Identifier) of the requested resource.

-> Uniform Interface : Deciding resource URI’s, these URI’s are nothing but endpoints for API’s. URI should be the same for similar kinds of requests. For e.g. :
Information for user profiles for different users should come from the same URI. URI for the same can look like -> /profile , /profile{id} .

-> Stateless : Communication between client and server should be stateless.Stateless means there will be no information from past transactions, each request is separate and independent of each other.

-> Cacheable : Whenever required, or we need to make an application more scalable we can cache the response on the client side. For e.g.: If we are making the same request again and again, we can simply cache the response . It will lead to a more scalable product as less requests will be made.

-> Layered System : REST API’s use a layered system. A layered system means a system with different units of functionality. These layers serve different purposes like providing security, authentication, load balancing or any other functionality according to our requirements. We can decide the layers of our API according to the business requirements while designing it.

-> Code on Demand : Servers can send some kind of executable scripts to clients to increase or change functionality on the client side. It's an optional constraint.
If a service violates any other constraint, it cannot strictly be referred to as RESTful.

Working of REST API’s

REST API’s communicate via HTTP requests to perform CRUD operations on DB.
Request headers and parameters are also important in REST API calls as they include information such as authorisation, URI, caching and more.
Request headers and parameters along with HTTP status codes are used for designing REST API’s.
Request API implements many HTTP request methods and before designing the API we should consider them, we need to decide which kind of request we are going to make to perform the required task.

HTTP Request	Meaning
GET	Retrieve information
POST	Send information
PUT/PATCH	Update information
DELETE	Delete information

Status Codes :
Status codes are the codes which a website's server sends to the browser to indicate whether a request can be fulfilled or not . Different response codes indicate different messages. We know that response can be either successful or failed but in order to know the reason for failure or success we can use these status codes. Understanding of status codes and their meaning is very important while designing the REST APIs.

Success Codes : 200 - OK , 201 - Created , 204 - No Content

Error Codes :

Error from client Side: 400 - Bad Request, 401 - Unauthorised, 403 - Forbidden, 404 - Not Found

Server Error : 500 - Internal Error , 501 - Not Implemented , 502 - Bad Gateway, 503 - Service Unavailable, 504 - Gateway Timeout

Basic step in image classification model - Loading Image Dataset (Part-1)

Modgil — Wed, 30 Jun 2021 14:17:57 +0000

It's expected that you know the problem and must have the dataset on which you want to work on.

In image classification/detection tasks the basic step is to load the data in your notebook.

It is very important to arrange the data in proper order before using it.
The image dataset must follow a particular hierarchy depending upon your problem.
image-data/ image-data/class-1 image-data/class-2 . . . image-data/class-n

The dataset folder contains file having sorted images of different classes.

Now we will be loading the dataset
Using Keras

1. Using Keras

We'll use image_dataset_from_directory function to load dataset.

 image_size = (n, n)  #depending 
 #upon the model requirement
 batch_size = 32  

 train_ds = 
 tf.keras.preprocessing.
 image_dataset_from_directory(  
 Directory,
 validation_split=0.2,
 labels="inferred",
 subset="training",
 class_names=["Class1","Class2"],
 seed=123,
 image_size=image_size,
 batch_size=batch_size)

The function will automatically split the dataset into 2 parts, train and validation set depending upon the value provided in validation_split parameter.

val_ds = 
tf.keras.preprocessing.
image_dataset_from_directory(
Directory,
validation_split=0.2,
labels="inferred",
subset="validation",
class_names=['Class1','Class2'],
seed=123,
image_size=image_size,
batch_size=batch_size)

Hence, this way we can load our image data and can use for further training process.
For more info about parameters used in this function refer to keras documentation .

In next article we will learn how to load image data using other techniques.

Evolution of Object Detection Techniques - RCNN

Modgil — Fri, 05 Feb 2021 22:32:01 +0000

RCNN ( Region Based Convolution Neural Network ) is used for object detection and Localization (Identification of Objects in image with Boundaries). In this, Selective Search method (Edge Boxes and Other Region Proposal Methods) are used to determine the region proposal. This technique overcomes the problems faced using Sliding Window technique.

Around 2000 Region Proposals per Image are extracted using this technique and all these regions are Cropped and warped into fixed size (depending on the dimension which the used algorithm in the network for Classification expects) and then these regions are fed to the CNN network one by one and on the last layer of the network SVM is used for Classification.

But,

RCNN is very slow as every single region requires forward pass of CNN and as a result we end up with too many inputs to localization network.

Simple Linear Regression

Modgil — Fri, 09 Oct 2020 16:57:38 +0000

Linear Regression helps us to predict value of dependent variable(y) based on given independent variables(x1,x2,........).

We prefer Linear Regression Algorithm when there is a Linear Relationship between independent and dependent variable.

If there is a linear relation between dependent variable(y) and independent variables then, equation for linear regression will be,

y=m1.x1 + m2.x2 + m3.x3 + -------- 
                       ---+mn.xn+c

If only one independent variable is given then equation will be,

              y=mx+c

So, when dependent variable is dependent linearly only on one independent variable then we will use Simple Linear Regression.

Given 2 features YearsExperience and Salary and we have to predict the Salary using YearsExperience Feature.

Lets first visualize our data,

As we can see that there is a linear Relation between both the features.

Now we have to find the best fit line that is, the Regression Line.
So, to get the best fit line, we have to find best suitable 'm' and 'c' values.
‘m’ is the values of slope and ‘c’ is the intercept.

The line will be best fit if it has minimum error difference between predicted and actual value.
For e.g.
Mean Squared Error (M.S.E)=
1/n(Σ(predicted_y - actual_y)**2)
So our main aim is to minimize the value of Error in order to get best possible fit that is, our regression line.

Implementation of Simple Linear Regression using different techniques like Least Square Method , Sklearn Library and Gradient Descent is given here