Forem: Rohan Raj

Generative vs Discriminative modeling

Rohan Raj — Sun, 21 Aug 2022 20:13:17 +0000

What is Generative Modeling?

Let’s understand this with an example. Suppose that we have a dataset of some rare species of animals on the planet. Since those are rare species the dataset won’t contain enough pictures for our model to learn from and this might lead to underfitting of the model.

To tackle this, we wish to build a model that can generate a new image of the species that never existed but still looks real because the model has learned the general rules that govern the appearance of a species.

In short, generative modeling is generating something new in terms of a probabilistic model. By sampling from this model, we are able to generate new data.

It is our goal to build a model that can generate new sets of features that look as if they have been created using the same rules as the original data. As easy as it may sound, for image generation this is an incredibly difficult task, considering the number of pixels associated with an image and the probability associated with them.

A generative model must also be probabilistic rather than deterministic. Let’s suppose we govern our model with a rule to generate images based on the average value of each pixels in the dataset. This is not the characteristic of a generative model. Rather, the model must include a stochastic element (random noise) which influences the individual samples generated by the model.

Generative versus Discriminative modelling

To truly understand what a generative model tries to achieve, we must compare it to it’s counterpart, Discriminative modelling.

So what is discriminative modeling ? A simpler answer would be, a model which discriminates between two or more objects and identifies which is what.
In a cats vs dogs model, the main aim is to identify which image is of cat and which one is of a dog. The model learns about the several details on how to differentiate between a cat and a dog and draws a rule based on which it can perform inference.

The dataset consists of two parts, an observation and a label. For a binary classification problem for cats vs dogs, the image of cat would be labeled as 0 and the image of a dog would be labeled as 1.

Discriminative model is somewhat similar to supervised learning but not exactly. Generative modeling is usually performed with an unlabeled dataset (form of unsupervised learning), though it can also be applied to labeled dataset as well.

In mathematical notations, we can represent Generative and Discriminative models as follows:

Generative modeling estimates p(x) —- the porbability of observing an observation x.
If the data is labeled then it is represented as p(x | y).

Discriminative modeling estimates p(y | x) —- the probability of a model y given observation x.

In general, discriminative modeling aims to estimate the probability that an observation x belongs to category y whereas generative modeling aims to estimate the probability of seeing the observation at all.

Hope you had a good read! :)
Please do give it a like.

Introduction to MLops

Rohan Raj — Mon, 31 Jan 2022 16:27:19 +0000

Let us suppose you have trained an accurate machine learning model and might be wondering what to do next?
Well, after having developed such a good model it's time to celebrate first and next is to use it in production so that we have a real-time software making actual use of the model.

So, what is MLops?

As the name suggests, it's ML + DEV + OPS

It's a whole machine learning project life cycle in which we start by scoping our project, to data, to modelling through deployment all the way from start to finish.

So, is developing a Machine Learning system in production a single step process? Well, it isn't! There are several aspects and challenges that come into picture when the deployment has been done. Having a system up and running is halfway through the process. The other half is keeping that system consistent and dependable and handling issues such as data drift, where there is gradual drift in terms of the data, which we used to train our model and the data we are using to perform inferences.

D. Sculley et. al. NIPS 2015: Hidden Technical Debt in Machine Learning Systems

There are several tools and methods which are involved in a complete machine learning system. The ML model is a small yet essential component in the infrastructure.

But no model lasts forever. Even if the data quality is fine, the model itself can start degrading. What does this mean in practice?
The world is dynamic, and our model needs to be aware of the changes happening to be accurate and dependable.

In short, having a machine learning model in production is an iterative process where we constantly evolve and improve our model.
MLops is the way to use several tools and methods to automate this process and make this process of evolving easier.

So, are you ready to dive in?

Software Engineering issues in MLops;

Let’s suppose we have a system which takes in “x” as an input and predicts “y” as the output, then there are some aspects which you need to keep in mind:

Realtime or Batch: Does the model make real-time predictions as in case of speech recognition models or is the model working on a batch of data whose output may or may not be instant.
Cloud vs. Edge/Browser: Is your model deployed in the browser or at an edge device such as mobiles which do not require internet access? Let’s suppose there is a critical system running in clouds and suddenly the internet goes down. In such cases, edge device-based models would be more reliable.
Compute resources: Choosing the right software architecture also depends on the hardware limitations we have. A model trained on an exceedingly high GPU or TPU may not work with same efficacy on a CPU.
Latency, throughput (QPS): The architecture also depends on the number of queries being handled by our model. It can be measured in Queries Per Second.
Security and privacy: For different applications, the level of security and privacy would be different based on how sensitive the data is.

Deployment patterns

Common deployment cases:

New product/capability: If you are offering a new service such as voice commands, then the best way to implement it is to start with a small batch of service and then slowly ramp it up.
Automate/assist with manual task: There are some tasks being done by humans and we want to use the advantage of ML models to automate or assist the task.
Replace previous ML system: You now want to replace the already running ML system with a new one.

For a system, we start with a small traffic and gradually increase its uses.

There are several deployment patterns which can be used:

Shadow mode: In this scenario, the ML system shadows the human and runs in parallel. The output of the ML system is not used for any decision during this phase. This can also be thought of as the monitoring phase in which we measure how accurate our model is.
Canary deployment: In this method we pass a small part of the incoming traffic to the ML system to make real time predictions. Here we keep monitoring the system and ramp up the traffic gradually.
Blue green deployment: In this type of deployment, we have two instances of system running. One instance is the previous version (Blue) and the other one is the updated version (Green). The router switches the traffic from blue to green instantly to use the new system. And if there is any issue, the traffic can be routed back to the previous version.

So, in general, there is a spectrum of the degree of automation we want to achieve. On one end of the spectrum are humans and on the other are machines. We can have a pollination of both and decide the degree to which we want to involve humans in the loop.

Monitoring ML systems:

The best way to monitor any system is through dashboards. We can monitor several metrics of our system such as server load, latency etc. through these dashboards.
Using these metrics, we can brainstorm through them to understand if our learning algorithm is performing well.
Just as ML modelling is iterative, so is deployment. After deployment, we keep monitoring the metrics and do performance analysis to improve it. This metric can also be used to update the model and tune it for more accuracy.

Hope you have a brief idea about MLops in general.

If you enjoyed reading it, please do give it a like :)

Why TensorFlow.js?

Rohan Raj — Sat, 04 Dec 2021 18:00:50 +0000

In today's world almost everyone has come across the buzz around Artificial Intelligence (AI). We have come a long way in this domain of computer science and the amount of development happening in this field is tremendous. Several unsolved problems have been solved by harnessing the power of AI.
So, is AI magic?

Well, it isn't. It's just science, the science of getting computers to act without being explicitly programmed.

The know how's and the basics.

Before we get deep into this dazzling world of predictions and learning, we need to have our basics strong. Phrases like AI, machine learning, neural networks, and deep learning mean related but different things.

Artificial Intelligence
As the Venn diagram above depicts, AI is a broad field. It encompasses machine learning, neural networks, and deep learning, but it also includes many approaches distinct from machine learning. A crisp definition of the field would be: the effort to automate intellectual tasks normally performed by humans.

Machine Learning
Machine Learning, as a subfield of AI distinct from symbolic AI arises from a question that could a computer learn on its own without being explicitly programmed for a specific task?

Machine Learning seeks to avoid the hard-coding way of doing things. But the question is how a machine would learn if it were not explicitly instructed on how to perform a task. A simple answer to this question is from the examples in data.

This opened the doors to a new programming paradigm.

In the classical programming paradigm, we input the data and the rules to get the answers.

Whereas, in the machine learning paradigm we put in the data and the answers and get a set of rules which can be used on other similar data to get the answers.

Let's take the example of identifying human faces in an image.

We, as humans can very well classify objects based on their characteristics and features. But how do we train a machine to do so? It is hard for any programmer, no matter how smart and experienced, to write an explicit set of rules in a programming language to accurately decide whether an image contains a human face.

The hypothetical search space without any constraint is infinite and it's impossible to search for explicit rules to define a task in a limited amount of time.

Any heuristic we produce is likely to fall short when facing the myriad variations that faces can present in real-life images, such as differences in size, shape, and details of the face; expression; hairstyle; color; the background of the image and many more.

There are two important phases in machine learning.

The first is the training phase.

This phase takes the data and answers, together referred to as the training data. Each pair of input data (instances) and the desired answer (labels) is called an example. With the help of the examples, the training process produces the automatically discovered rules.

Although the rules are discovered automatically, they are not discovered entirely from scratch. In other words, although the machine is intelligent, but not enough to produce the rules.

A human engineer provides a blueprint for the rules at the outset of training. The use of labeled data and human guidance in producing the rules is also known as Supervised Learning.

It's just like a child learning to walk and requires support in their initial stages.

The blueprint is encapsulated in a model, which forms a hypothesis space for the rules the machine may possibly learn. Without this hypothesis space, there is a completely unconstrained and infinite space of rules to search in, which is not conducive to finding good rules in a limited amount of time.

In the second phase of the machine learning paradigm, we use these generated rules to perform inferences on new data.

Neural Networks and deep learning
Neural networks are a subfield of machine learning which is inspired by the neurons present in the human and animal brains. The idea here is to replicate the way a brain learns things as it perceives. We build a net of interconnected neurons each responsible for memorizing certain aspects of a given task to perform.

The data is passed through multiple separable stages also known as layers. These layers are usually stacked on top of each other, and these types of models are also known as sequential models.

These neural networks apply a mathematical function over the input data to produce an output value. These neural networks are generally stateful, i.e. they hold internal memory.

Each layer's memory is captured in its weights.

Why TensoFlow.js?

As it's known, JavaScript is scripting language traditionally devoted to creating the web pages and back-end business logic.
Someone who primarily works with JavaScript may feel left out by the deep-learning revolution which seems to be an exclusive territory of languages such as Python, R, C++. TensorFlow, is a primary tool for building deep-learning models.

TensorFlow.js is the product of cross-pollination between JavaScript and the world of deep learning. It is suitable for folks who are good in JavaScript and want to explore the world of deep learning and for folks who have basic mathematical understanding of the deep learning world and are looking for a place to dive deep into this field. With deep learning, JavaScript developers can make their web apps more intelligent.
TensorFlow.js is created and maintained by Google, so it's worth noting that some of the best brains in the world have come together to make it happen.
Provides a no-install experience in the world of machine learning. Generally, the AI in a website is locked in an AI and the performance varies with the bandwidth of the connection. TensorFlow.js provides us with the ability to run deep learning models directly in the browsers without any installation of other dependencies.
JavaScript based applications can run anywhere. These codes can be added to progressive web-apps or React application and then these applications can run without being connected to the internet.
It also provides a great deal of privacy as the data never leaves a user's system.
It can also be used in IoT based devices such as RaspberryPi.

Conclusion, mastery of TensorFlow.js can help us in building cross platform intelligent application with great efficacy and security.

And a huge yes to the picture above XD
Hope you enjoyed reading the blog!
Thank you :)