Forem: Badhon Nandi

Types of Machine Learning you must know!

Badhon Nandi — Sun, 08 Sep 2024 12:41:09 +0000

There are 4 Major types of Machine Learning

Supervised Learning
- Regression
- Classification
Unsupervised Learning
- Clustering
- Dimensionality Reduction
- Anomaly Detection
- Association
Semi Supervised Learning
Reinforcement Learning

Lets Explain one by one for a clear idea that what exactly they are about!

Supervised Learning

If we have a dataset with both input and output, our job is to understand the relationship between them. Then, we can use that understanding to predict the output for new input. This type of learning is called supervised machine learning.

Example: Lets take 1000 Students Data.

Now the ML model will create a mathematical link between the input and output. So, in the future, if I give it details about a new student, it will be able to tell me if that student is placed or not.

Now, the model can give the output on its own by using the pattern it learned from the previous input and output data. This is how supervised learning works.

Now if we talk about data types, there are 2 types of Data.

Numberical
- Age
- Weight
- CGPA
- IQ and etc.
Category
- Gender
- Nation
- Place and etc.

Supervised Learning has 2 Parts:

Regression: If the output data is numerical then this is called regression.

Classification: Non-Numerical Datatypes are called classification.

this was all about supervised learning.

Unsupervised Learning

We only have input, with no output. So, making predictions is impossible because we don't know what to predict. So, what's the purpose of it?

Unsupervised Learning has 4 Parts:

Clustering
Dimensionality Reduction
Anomaly Detection
Association Rule Learning

Clustering
Clustering in unsupervised learning is when we group similar data points together without knowing the labels or categories beforehand. The goal is to find patterns or structures in the data on its own.

Imagine on the X-axis we have IQ and on the Y-axis we have CGPA. Clustering helps us see which students should be grouped together based on these factors. We can also use clustering for our own analysis.

For example, in my e-commerce website, I can figure out what type of customers I have and how to group them. And, clustering algorithms can group data with many dimensions that we can't see with our eyes.

This makes clustering very useful.

Dimensionality Reduction
In supervised learning, we often have many input columns, like in image or text data, sometimes over 1000. When there's too much data, the algorithm runs slower. After a certain point, adding more columns doesn't improve the result, so those extra columns are unnecessary. Removing them helps reduce the data size and makes the algorithm faster and more efficient. This process is called dimension reduction.

A simple example of this is called feature extraction. It's also well-known as the PCA algorithm (Principal Component Analysis).

We can not plot any data if we have 4 or more columns of data. Human brain can only draw till 3D coordinate system. So, here reducing the dimensionality from 1000 to 2/3D coordinate system helps us to plot those points. Now we can easily study and visualise those data. So, dimension reduction plays here an important role.

Anomaly Detection
It is a method to find patterns or data points that are unusual or different from the norm. It helps to spot things that don't fit the regular trend or pattern.

Association Rule Based Learning
It is a method used to find relationships between different items in data. It helps to understand which items are often bought or occur together.

Example: If you run a big store, you can use past sales data to see which products are often bought together. For example, if you find that when people buy milk, they often buy eggs too, you should keep milk and eggs next to each other in the store.

So sells will be improved. In this way we do data mining and then draw conclusion using Association Rule Based Machine Learning.

Semi-Supervised Learning

Semi-supervised learning is a method where a computer learns from both labeled data (with known answers) and unlabelled data (without known answers). That means partially Supervised and partially unsupervised learning.

When we add labels to data, it usually takes a lot of time because it's done manually by people.but human though I will only add 1-2 label and except that all the data/label get cover it self. This is the core-idea behind semi-sup learning.

Example: Google Photos.
We store many of our pictures in Google Photos. One day, Google Photos asked, "Who is this?" I said, "This is my brother." After that, Google Photos grouped all the pictures of my brother under the same label, making a cluster of them.

Reinforcement Learning

Reinforcement Learning is a type of machine learning where a computer learns to make decisions by trying different actions and getting rewards or penalties. It learns over time to choose actions that give the best rewards.

When I go for my university after leaving my own home, I don't know what should I do. So I make mistakes and learn from those.

Imagine teaching a dog to sit. Every time the dog sits when you say "sit", you give it a treat as a reward. If the dog doesn't sit, it doesn't get a treat. Over time, the dog learns that sitting when you say "sit" will give it a reward. This is like reinforcement learning, where the dog is learning through rewards (treats) to do the right action.

Self Driving cars can also be a great example of it.

These are all types of Machine Learning. Thank you for your time.

Understanding the Differences: AI vs. ML vs. DL

Badhon Nandi — Sat, 07 Sep 2024 01:39:13 +0000

Artificial Intelligence

Which is focused on creating systems or machines that can perform tasks typically requiring human intelligence.These tasks include reasoning, learning, problem solving, perception, language understanding and decision-making. Main goal is Making machines intelligence.

Human intelligence encompasses a wide range of abilities, including pattern recognition, coding, problem-solving, emotional understanding, and more. It is a complex and multifaceted capability that allows us to navigate and interact with the world in diverse ways.But, Artificial Intelligence is a specialised subset of human intelligence, primarily focused on tasks like pattern recognition, decision-making and data processing.

Symbolic AI, also called Good Old-Fashioned AI (GOFAI), is a method of artificial intelligence that uses clear rules and logic to make decisions. It relies on a lot of if-else conditions, where specific situations lead to specific actions. This approach is often used in expert systems, which are programs designed to mimic human decision-making. For example, Symbolic AI can be used to create a chess-playing program by following predefined rules for moving pieces and evaluating positions.

But there are some limitations of it. Symbolic AI struggles with tasks involving fuzzy logic or situations that can’t be defined by clear rules. For example, creating a system to recognize different dog breeds would require an infinite number of if-else conditions which is quite impossible. To solve this problem ML comes here.

Machine Learning

In ML instead of using explicit programming, statistical methods are employed to identify patterns within the given data. We start with known inputs and outputs and through training, the model learns the underlying relationships and logic needed to solve the problem. Once trained, the model can make predictions on new, unseen data.

Unlike Symbolic AI, which requires predefined rules and logic, ML systems automatically generate their own logic based on the data they are fed. By providing large datasets, the system learns to recognize patterns, enabling it to deliver accurate outputs when presented with new inputs. This ability to learn and adapt from data is what sets ML apart from traditional rule-based AI systems.

When training a ML model, it is crucial to specify which features to focus on in the data. These features guide the model in identifying relevant patterns. For example, if the task is to recognize a dog, features such as the number of ears and eyes would be important. It will follow this way to draw patterns. But what if, we don't know the features? and to solve this problem DL was invented.

Deep Learning

It is a type of machine learning that is inspired by how the human brain works. It uses math to create artificial neurons, which are similar to the neurons in our brain, though not exactly the same.

Deep Learning does not require predefined features, it automatically creates or detects features from the given data. This is especially useful when working with fuzzy logic, where it's unclear what features might exist. For example, if I want to predict which candidate is likely to get a job based on their CV, the deep learning model can identify features like the number of certifications or good grades on its own, without needing manual input.

The more layers of neurons we add to a deep learning model, the more accurate its classification and predictions become. Also increasing the amount of input data helps improve the model’s performance. But in ML models it their might be some limitations of performance.

So we can see a clear picture that Deep Learning is a sub-domain of Machine Learning. And Machine learning is a sub-domain of Artificial Intelligence.

ML and DL are both rapidly advancing fields. You might wonder why DL isn’t always used if it’s so powerful. The reason is that ML tends to perform better with smaller datasets, while DL excels with large amounts of data, like those used by companies such as Meta or YouTube.

Thank you for reading.

Introductions to ML

Badhon Nandi — Fri, 06 Sep 2024 08:43:34 +0000

What is Machine Learning?

Machine Learning is a field of Computer Science that uses statical technologies to give computer systems the ability to 'Learn' with data, without being explicitly programmed.

That means, "ML is all about Learning from Data"

Explicit Programming means, writing codes for each scenario, to handle that situation.

In machine learning, instead of writing explicit code for each scenario, we train models to learn patterns from data, allowing them to make predictions or decisions for unseen situations.

So, We give input and output, but don't write any code for each and every case. ML Algorithms automatically handle them.

An simple example can use:

Summation Function:

In explicit programming, to add 2 numbers, we write specific code that works only for that case. This code won’t work for adding 5 or N numbers without modification.

In contrast, with ML, we can provide an Excel file where each row contains different numbers and their sum. As the ML algorithm trains on this dataset, it learns the pattern of addition. In the future, when given 2, 10, or N numbers, it can perform the addition based on the learned pattern, without needing specific code for each scenario.

Where we are using ML?

Email Spam Classifier:

In explicit programming, I wrote multiple if-else conditions, such as: “If a keyword appears 3 or more times, it will be flagged as spam.” For example, if the word “Huge” is used 3 times, it’s marked as spam.

Now, imagine an advertising company realize there’s an algorithm like this to detect their spam. So instead of repeating “Huge” 3 times, they use synonyms like “Huge,” “Massive,” and “Big.” In this case, the original rule wouldn’t work. What would be the solution? Should I again change my previous algorithms? How many time I will able to do that?

In ML, the model learns from the data provided and automatically creates algorithms based on that data. If the data changes, the algorithm adjusts accordingly. There’s no need to manually change the algorithm, it will update itself as needed based on the new data.

Image Classification:

In explicit programming for image classification, we would need to manually write rules to identify features of a dog, like its shape, size, fur color, or tail. These rules would only work for specific images and would not generalize well to all dog breeds. If we encountered new breeds or variations, we would need to add new rules for each one.

In ML, instead of writing specific rules, we provide the model with a large dataset of dog images labeled by breed. The model then learns patterns from the data, such as the common characteristics of different breeds, and uses that learned knowledge to classify new dog images, even if it hasn't seen those exact breeds before. The algorithm automatically adapts to variations in the data.

also, there are thousand of uses of ML. You might wonder,
why wasn’t machine learning as popular before 2010?

Limited storage capacity made it difficult to store large amounts of data due to the shortage of hard drives.
There wasn’t enough available data to effectively train machine learning models.
Hardware limitations, such as less powerful GPUs and processors, restricted the ability to run complex algorithms efficiently.

Nowadays, we are generating millions of data points every day. Using this vast amount of data, ML models are now becoming more accurate, efficient, and capable of solving complex problems. They can learn patterns, make predictions, and automate tasks across various fields such as healthcare, finance, and technology, improving decision-making and driving innovation.

Thank you for taking the time to read through this.

Data Structure & Algorithms

Badhon Nandi — Tue, 15 Aug 2023 11:32:56 +0000

What is Data Structure?
Data Structures are different way of organising data on your computer , that can be used effectively.

So if we want to find any specific colour scale we can find that very easily from the right side box.Because it is well organised.

For software application performance point of view , the efficiency and the performance of the software depends on how the data is stored, organised and grouped together during the program execution.

What is Algorithm?
An algorithm is just a set of instruction to perform a task or solve a problem.

Example:
We need a margin in our A4 size paper.So:

Choosing the scale from different colour scale
Place the scale to my A4 size paper
use a pencil or pen and draw the margin

Moreover, Algorithms in our daily life:

To drink a coffee:

1.go to Coffee Shop
2.pay money
3.take coffee

This set of steps referees to algorithms.

Algorithms in computer science:
Set of rules for a computer program to accomplish a task.

What makes a good algorithm?

Correctness
Efficiency

As a software or a computer, our responsibility is to perform operation on the data.First we give some input, then we process it and then we give back processed data as output.Input of data can be in any form.

Example: Google Map

Input(Starting and ending) --> Process(DSA) --> Output(Shortest Path)

Other Example: Library (Well Organised)

Think , if I need algorithms book I first go to computer science section, then Algorithms section.I will able to find my desired book.Because it is well organised Which referrers to Data Structure.

Types of Data Structures

In python, we can devide in 2 sections:

Primitive
Non Primitive

Primitive:

basic data types that can not be broken into simpler data type.
fixed size and usually smaller in size than non primitive.
used for simple operation.
represented in memory as a simple values.

Non Primitive:

more complex and can be broken down into smaller data types.
can be large in size and grow or shrink dynamically.
used for complex operations such as data manipulation,sorting and searching.
represented in memory as pointers to other memory locations.

Topic in Data Structure:

These are the data structures we have.There are more but these are the most important one.

Linear data structures are used to represent a sequence of data where the order of elements are important.
Non Linear's are not represented sequentially.Its used to represent a hierarchical relationship between data elements where each element is connected to one or more other elements in a specific way.

Some types of Algorithms:

Sorting (to sort data in assending or desending order)
Searching (to find specific value in a data set)
Graph (to work with data that can be represented as a graph)
Recursive
Devide and Conquer

and Many more...

Why should you learn C++ in 2023, when AI is ruling?

Badhon Nandi — Mon, 08 May 2023 11:59:08 +0000

Well, this is 2023, and we all can see the rise of chatbots and AI tools all over the world, and everybody is thinking they should learn Python for sure as the industry is going down that path. But did you ever wonder how the world was before there was AI around us? People used to make their own tools by hand. In our daily lives, we use the Chrome browser, video players, play games, and do tons of other things using software. Most of them are made using languages like C++. It's true that work is getting easier compared to previous days, and I don't think AI can match the creativity of humans! So today we will talk about whether you should learn C++ in 2023.😀

In 1642, the first mechanical calculator was invented in France by mathematician and philosopher Blaise Pascal. So after 1642, has the creativity stopped for the new mathematicians at all, so that they have failed to invent any new things for society? Not at all, right? We know the names Leonhard Euler, Sir Isaac Newton, Carl Friedrich Gaius, and many more. A calculator is a tool that makes work easier. So I think Ai will do the same as well. So people who know C++ will be able to work creatively using their own brains and make the best use of AI. 100% of developers cannot work only with AI, as there are a lot of sectors like cloud,cyber security, game development, API,etc. Therefore, inverting time for C++ in 2023 will not be a loss for you at all.

🐱‍🏍Let's talk about what sectors are there so that you can work with C++ .To begin with,C++ is frequently used because of its flexibility and popular industry application.
These are some sectors where we use c++ now a days.

Game Development
Operating Systems
Financial Systems
Networking and Telecommunications
Scientific and Engineering Applications

Game Development : C++ is the basis for popular game development systems like Unreal Engine and Unity. Learning C++ is highly recommended if you are interested in game development. Unreal Engine's primary programming language for creating games and apps is C++. Unreal Engine developers may build immersive and high-performance experiences by using the power of C++.If you are passionate about that, you are welcome.

Operating Systems: C++ plays a vital role in the creation of operating systems. Its low-level capabilities, efficient memory management, and direct hardware access are used in kernel development.It is also used to create system libraries, which enable functions such as file system operations, network connectivity, process management, and inter-process communication.The development of system tools and utilities for system management, debugging, and performance analysis within operating systems is also done in C++.

Financial Systems:C++ is frequently used in the financial industry to create algorithmic trading systems, risk management software, and high-frequency trading platforms. Its speed and control over system resources are critical for real-time processing of massive amounts of financial data.

Networking and Telecommunications: C++ is used in the creation of networking protocols, network management systems, and telecom software. Its performance and low-level control make network operations and data transfer more efficient.

Scientific and Engineering Applications: C++ is used in a wide range of scientific and technical disciplines, including computational physics, aeronautical engineering, bioinformatics, and others. It is useful in these disciplines because of its capacity to perform complicated computations, optimize algorithms, and interface with specialized hardware.

Many of us love to do competitive programming, and C++ is the best language for competitive programming. Many people use Python to do that, but Python is slow to do that. C++ is a very close language to computers and its very first at the same time. C++ is an object-oriented programming language. We know many competitive contests from Google or some platforms have already been turned off, So if you are not passionate about this, I will suggest you not do competitive programming and focus on other sectors of it.

In conclusion, if you feel interested in game development, software development, networking, or competitive programming, then C++ will be the best choice for you.