Forem: Ayush Goel

Data And Analysis

Ayush Goel — Fri, 17 Sep 2021 13:49:50 +0000

I am excited to try this new format, this time I was thinking of doing this in the form of ques/ans. This is something I am trying for the first time I hope you like it so fingers crossed!!!!

Q.1 Over the last decade, data has transformed the way the world works. Describe an area where you think data will change the world in the next five years. Which lesson(s) from the evolution of data in recent years would you draw on to help make sure data changes the world in positive ways going forward?

It is very exciting to see how data is changing the pace and transformative potential of today’s innovative technologies and how much we can do with it. Data is just facts and statistics collected together but how we use it can determine the future and how we apply it can solve the world’s most pressing problems, such as feeding a global and growing population; improving access to and quality of healthcare; and significantly reducing carbon emissions to arrest the negative effects of climate change. The next five years will see profound improvements in addressing these challenges.
While the COVID-19 pandemic has provided a difficult lesson in just how susceptible our world is today to human and economic turmoil, it has also - perhaps for the first time in history - necessitated global collaboration, data transparency, and speed at the highest levels of government in order to minimize an immediate threat to human life. The data which was collected all over the world during this difficult phase can help us to fight such situations in the future.
I think in the next five years the area which will be most affected by data will be industries, over the next five years, carbon-heavy industries will use machine learning and AI technology to dramatically reduce their carbon footprint. Traditionally, industries like manufacturing and oil and gas have been slow to implement decarbonization efforts as they struggle to maintain productivity and profitability while doing so. However, climate change, as well as regulatory pressure and market volatility, are pushing these industries to adjust.
For example, oil and gas and industrial manufacturing organizations are feeling the pinch of regulators, who want them to significantly reduce CO2 emissions within the next few years. Technology-enabled initiatives were vital to boosting decarbonizing efforts in sectors like transportation and buildings - and heavy industries will follow a similar approach.
Indeed, as a result of increasing digital transformation, carbon-heavy sectors will be able to utilize advanced technologies, like AI and machine learning, using real-time, high-fidelity data from billions of connected devices to efficiently and proactively reduce harmful emissions and decrease carbon footprints.
It will be us who will decide how we use DATA in the future and how we use it to change the world in a positive way going forward. One of the major lessons that I learned from the evolution of data in recent years is that we are at a stage where data is used in every industry and how much efficient decision making is now because they all are mostly data-driven which makes the growth of industries much faster but data can also be easily abused which can harm people but at the end, it is us who decides how to use it to make the world a better place to live.

In 2 paragraphs, please explain why you’re passionate about Data Science.

Technology is something that has always fascinated me even when I was a kid. When I was in high school that’s when I decided to pursue my bachelor's in technology and when I got into university my first instinct was to pursue software engineering because all the people around me were also pursuing the same then I continued my journey to learn software development skills but there was something I wasn't that engaged in it, I was not excited to work and that was the time when I started looking for different things and got introduced to AI, Machine Learning, Data Analysis, Data science etc and that was something which excited me to work on and than I started to explore these new streams which helped me to get my passion back.
Who likes to argue? Data analysis provides objective answers that can put an end to an argument. Data and analytics allow us to make informed decisions – and to stop guessing. I was never fond of making decisions based on gut feeling, perhaps because the gut says one thing one day, and something quite different the following day. It’s exciting and really interesting . It satisfies curiosity. It’s mysterious. It can be applied to many different domains. Businesses need to make trade-offs. Data and analytics can have real influence on the decisions a business takes, and on the outcome.

We want to better understand your experiences and interests relevant to data and analytics. Describe a previous project you’ve worked on that involved data. Please be sure to share: (1) Which question(s) were you trying to answer? (2) What aspects of the project were most interesting? (3) Which technical aspects of the project were most challenging and how did you work through challenges? (4) What was the conclusion/outcome of the project? (5) Beyond any analysis results/findings, what did you learn from working on the project?

I have worked on many basic projects to find what I was most interested in. What's crucial for me when starting one is to get very clear on the goals right at the start and then create a plan with milestones. I also like dealing with the most difficult parts of the projects early on—that way in case there are any significant issues, I'll still have a nice amount of time to complete before the deadline. I also typically break down large tasks into smaller chunks, so that it is easier to know where to start. Detailed planning is very important to ensure an important project goes smoothly.
For example, one of my biggest projects was when I challenged myself to work on a dataset provided by quantium, in which I divided my work into three basic tasks

Data preparation and customer analytics Conducted analysis on client's transaction dataset and identify customer purchasing behaviors to generate insights and provide commercial recommendations.
Experimentation and uplift testing Extended my analysis from Task 1 to identify benchmark stores that allowed me to test the impact of the trial store layouts on customer sales.
Analytics and commercial application Used my analytics and insights from Task 1 and 2 to prepare a report for my client, the Category Manager. The second task was really exciting extended analysis from Task 1 to help identify benchmark stores to test the impact of the trial store layouts on customer sales was interesting while performing this task I got to know many new things and aspects of how correct data visualization can help us to get businesses to grow much faster.   One of the main problems that I struggled with was when I was getting started the amount of data was something that I was scared of because it was such a big dataset, getting started with it was something that I struggled with but once I started it day by day I was getting into a groove which made this project such great for me. The project was completed on time, but looking back, I realize some problems could have been avoided but I learned from them and kept moving forward which is something I am proud of. I enjoyed how this project made such a big difference in me, it helped me to gain confidence that I too can work on big projects, I learned more about myself, whenever I was stuck on something I communicated with many wonderful people from different tech communities which helped my communication skills and it was a wonderful experience and I feel great about it.

I hope you like it. link for the project above mentioned-I'm an inline link

Starting With Git

Ayush Goel — Mon, 28 Jun 2021 09:35:18 +0000

INTRODUCTION

In this blog we are going to learn a little bit about git and important git commands that one must know to make their first commit.

Let's get started

To understand Git first let's see what a Version Control System(VCS) is - A VCS or version control system as it sounds is simply a tracker of content(usually code) that tracks changes to code-base and helps developers to simultaneously work on the same project by managing repositories.

So now what is Git?

Git is a distributed version control system(DVCS) which simply means that Git stores code-base on a repository in a Server and simultaneously distribute it to the local repository (usually in Developer's Computer) of each developer. It also helps developers manage their code efficiently by keeping a track record of the changes committed to the code-base by time.

If you are a new or experienced developer, you have to use source control. And good chances are you are using Git to manage your source code.

And to use Git to its full potential, you need to know Git commands. Here you will learn the most helpful Git commands that will take you from one level to another.

Basic Git Commands

1) git config [ options ]

This command helps you set the username and email address to be used with your commits respectively.
usage: git config --global user.name "Ayush Goel"
usage: git config --global user.email "1234.@gmail.com"

2) git init [ repository name ]

This command initializes a repository as git repository. To perform git operations you need to initialize the target repository as a git repository.
usage: git init /home/Documents/git

3) git status

This command is used to see the status of the working tree. It shows you the list of unstaged files and list of files that needs commit. If there is nothing do then it will simply show that the branch is clean.
usage: git status
Note: You need to be in a git repository to know its status.

4) git add [ File Name ]

This command puts the desired file to the staging area for operations to performed on it.
usage: git add index.html
Note: You can add multiple files to the staging area by separating files with a space - git add index.html main.css main.js

5) git commit [ options ]

This command records changes to the git repository by saving a log message together with a commit id of the changes made.
usage: git commit -m "Write your message here"
Alternatively you can use git commit -a to commit changes to the files added using git add without specifying a message
Note: You should always commit with a message.

6) git log

This command is used to show the log of commits made so far to the current branch.
usage: git log
usage: git log --follow John_Doe to see the log of commits made together with renaming of files of the specified file.
Note: You can use git log to see all the commit ids

7) git rm

this command is used to remove a file from the staging area.
usage: git rm--cached[file name]

8) git push [ options ] [ variable name ] [ branch ]

This command is used to push the contents of your local repository to the added remote repository. This sends the committed changes of your master branch to the added remote repository.
usage: git push -u origin master
Note: -u depicts upstream here.
Alternatively you can use -f instead of -u to forcefully push the contents of your repository.

9) git pull [ URL ]

This command is used to fetch and integrate the contents of the remote repository to your local repository.
usage: git pull https://www.github.com/example.git

10) git branch [ options ] [ branch name ]

This command is used to perform operations over the specified branch
usage: git branch -d branch_1 to delete the specified branch.
Note: You can use -D to forcefully delete a branch
usage: git branch -m old_name new_name to rename the branch
usage: git branch -c branch_1 /home/Documents to copy the branch and corresponding reflog
usage: git branch -C branch_1 /home/Documents to forcefully copy the branch
usage: git branch -M branch_1 /home/Documents to forcefully move the branch
usage: git branch --list to list all the branches

11) git checkout [ branch name ]

This command is used to move from one branch to another
usage: git checkout 'branch_1'

12) git checkout [ options ] [ branch name ]

usage: git checkout -b branch_2 is used to create specified branch and is simultaneously switches to it.

13) git merge [ branch name ]

This command is used to merge the history of the specified branch into the current branch.
usage: git merge branch_3

14) git show[commit id]

This command is used to list the metadata for the specified commit
usage: git show 521747298a3790fde1710f3aa2d03b55020575aa
Note: All the commits have unique ids.

I hope this was useful for you
Thanks For Reading ;)

INTRODUCTION TO MACHINE LEARNING

Ayush Goel — Sun, 16 May 2021 18:19:49 +0000

Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it.Many researchers also think it is the best way to make progress towards human-level AI.

We see daily news stories that herald new breakthroughs in facial recognition technology, self driving cars or computers that can have a conversation just like a real person. Machine Learning technology is set to revolutionise almost any area of human life and work, and so will affect all our lives, and so you are likely to want to find out more about it.

Model is nothing but just a mathematical equation for eg y=mx+c where y=output, x=input and m&c are parameters and training the model just means generating the parameters for the equation.

We have to identify for what type of data, which algo we have to use to get the ML model, we can also combine many algos to get the best model for us.
Train:to understand how to calculate Test:to check if the machine can calculate.

TYPES OF MACHINE LEARNING?

Machine Learning can be classified into 3 types of algorithms.

1. Supervised Learning

2. Unsupervised Learning

3. Reinforcement Learning

SUPERVISED LEARNING:

It is defined by its use of labeled datasets to train algorithms that to classify data or predict outcomes accurately. As input data is fed into the model, it adjusts its weights until the model has been fitted appropriately, which occurs as part of the cross validation process. Supervised learning helps organisations solve for a variety of real-world problems at scale, such as classifying spam in a separate folder from your inbox.
Supervised learning uses a training set to teach models to yield the desired output. This training dataset includes inputs and correct outputs, which allow the model to learn over time. The algorithm measures its accuracy through the loss function, adjusting until the error has been sufficiently minimised.

Supervised learning can be separated into two types of problems when data mining -- classification and regression:

Classification:

It uses an algorithm to accurately assign test data into specific categories. It recognises specific entities within the dataset and attempts to draw some conclusions on how those entities should be labeled or defined. Common classification algorithms are linear classifiers, support vector machines (SVM), decision trees, k-nearest neighbour, and random forest, which are described in more detail below.

Regression:

It is used to understand the relationship between dependent and independent variables. It is commonly used to make projections, such as for sales revenue for a given business. Linear regression, logistical regression, and polynomial regression are popular regression algorithms.

UNSUPERVISED LEARNING:

It uses machine learning algorithms to analyse and cluster unlabelled datasets. These algorithms discover hidden patterns or data groupings without the need for human intervention.

Its ability to discover similarities and differences in information make it the ideal solution for exploratory data analysis, cross-selling strategies, customer segmentation, and image recognition.

Unsupervised learning models are utilised for three main tasks—clustering, association, and dimensionality reduction.

Clustering:

It is a data mining technique which groups unlabelled data based on their similarities or differences. Clustering algorithms are used to process raw, unclassified data objects into groups represented by structures or patterns in the information. Clustering algorithms can be categorised into a few types, specifically exclusive, overlapping, hierarchical, and probabilistic.

Association rule:

It is a rule-based method for finding relationships between variables in a given dataset.These methods are frequently used for market basket analysis, allowing companies to better understand relationships between different products. Understanding consumption habits of customers enables businesses to develop better cross-selling strategies and recommendation engines. Examples of this can be seen in Amazon’s “Customers Who Bought This Item Also Bought” or Spotify’s "Discover Weekly" playlist.

Dimensionality reduction:

While more data generally yields more accurate results, it can also impact the performance of machine learning algorithms (e.g. overfitting) and it can also make it difficult to visualise datasets. Dimensionality reduction is a technique used when the number of features, or dimensions, in a given dataset is too high. It reduces the number of data inputs to a manageable size while also preserving the integrity of the dataset as much as possible. It is commonly used in the preprocessing data stage.

REINFORCEMENT LEARNING:

In reinforcement learning, an artificial intelligence faces a game-like situation. The computer employs trial and error to come up with a solution to the problem. To get the machine to do what the programmer wants, the artificial intelligence gets either rewards or penalties for the actions it performs. Its goal is to maximise the total reward.

Although the designer sets the reward policy–that is, the rules of the game–he gives the model no hints or suggestions for how to solve the game. It’s up to the model to figure out how to perform the task to maximise the reward, starting from totally random trials and finishing with sophisticated tactics and superhuman skills. By leveraging the power of search and many trials, reinforcement learning is currently the most effective way to hint machine’s creativity. In contrast to human beings, artificial intelligence can gather experience from thousands of parallel gameplays if a reinforcement learning algorithm is run on a sufficiently powerful computer infrastructure.

CONCLUSION:

In this blog, I have presented you with the basic concepts of Machine Learning and I hope this blog was helpful and would have motivated you enough to get interested in the topic.