Forem: Purva Masurkar

Day 5-7 of 100 Days of ML Code: Data Manipulation with Pandas

Purva Masurkar — Fri, 17 Mar 2023 17:20:43 +0000

"Embrace the struggle, trust the process, and persist with effort."

To start using the Pandas library, we need to import it into our Python environment. For this example, I downloaded a Shop Customer Data dataset from Kaggle that provides a detailed analysis of an imaginary shop's ideal customers. To read the CSV file into our Pandas DataFrame, we can use the pd.read_csv() function.

Indexing

Indexing in Pandas refers to the process of selecting specific rows and/or columns from a Pandas DataFrame or Series. There are several ways to perform indexing in Pandas:

iloc(): This method allows you to select rows and columns by integer location.

loc(): This method allows you to select rows and columns by label or boolean mask.

Boolean Indexing: This method allows you to filter a DataFrame by a boolean condition.

Filtering

Filtering in Pandas refers to the process of selecting a subset of data from a DataFrame based on certain conditions.

Updating Rows and Columns

Updating rows and columns in a Pandas DataFrame involves changing the values of specific cells, rows, or columns.

Updating a specific cell:

Updating a specific row:

Updating a specific column:

In Pandas, there are two methods along with filter that are commonly used for transforming data:

apply: applies a function to a DataFrame or a Series along an axis.

map: applies a function to each element of a Series.

Adding/ Removing Rows and Columns

append(): You can use this method to add one or more rows to an existing DataFrame.

drop(): You can use this method to remove one or more rows from a DataFrame.

Sorting Data

Sorting data is an important operation in data analysis. In Pandas, you can sort data using the sort_values() method.

Grouping and Aggregating Data

Grouping and aggregating data is a common operation in data analysis. In Pandas, you can group data using the groupby() method and aggregate it using the agg() method.

Day 4 of 100 Days of ML Code: Pandas Library

Purva Masurkar — Mon, 13 Mar 2023 18:35:25 +0000

"Programs must be written for people to read, and only incidentally for machines to execute." - Harold Abelson and Gerald Jay Sussman, Structure and Interpretation of Computer Programs

Introduction to Pandas

Pandas is an open-source Python library that is widely used for data manipulation and analysis.

Summarizes the data.
Read and write different formats of file like CSV, JSON, EXCEL, HTML etc.
We can filter and modify the data based on multiple conditions.
We can merge multiple files.

Difference between Attributes and Methods

Attributes are used to represent properties or state of an object, while methods are used to represent behaviors or operations on its data. Attributes are accessed using the dot notation without parentheses, while methods are called using the dot notation with parentheses and optional arguments.

Importing Pandas

To use the Pandas library in Python, we first need to import it into our code. There are different ways to import Pandas, but the most common one is using the import statement

This statement imports the entire Pandas library, and we can access its functions and classes using the pd namespace.

Reading and Viewing the csv file

To work with real-world data, I have selected the Stack Overflow Annual Developer Survey file, which is a widely used dataset for data analysis and machine learning. This dataset contains information about the demographics, education, employment, and technology preferences of software developers from different parts of the world. The survey is conducted annually by Stack Overflow, a popular Q&A website for programmers.

To read a CSV file using Pandas, we use the pd.read_csv() function.

df.head(n): Displays the first n rows of the DataFrame (by default, n=5).
df.tail(n): Displays the last n rows of the DataFrame (by default, n=5).
df.shape: Returns a tuple containing the number of rows and columns in the DataFrame.
df.columns: Returns a list of column names in the DataFrame.
df.dtypes: Returns the data type of each column in the DataFrame.

To check null values in data we use. This function counts the total number of missing data from columns and sums them up.

To give summary of the data we use. It only includes columns that are numerical and not strings.

Gives all information of column such as number of rows, missing value, data types.

We are not allowed to see all columns so we use this function

DataFrame

In Pandas, a DataFrame is a two-dimensional table-like data structure that consists of rows and columns. Once created, a DataFrame can be manipulated, transformed, and analyzed using various Pandas functions and methods.

iloc and loc are two methods in Pandas that allows to select subsets of rows and columns from a DataFrame based on their index or label values. iloc is used for integer-based indexing, while loc is used for label-based indexing.

Conclusion
I am interested in continuing my exploration of the Pandas library because there is a lot to learn from it that can be helpful for my future applications. I will continue listing my daily progress and try to remain consistent. Please do share your feedback on how I can my 100daysofcode challenge more productive. I'll see you tomorrow for my daily update.

Day 3 of 100 Days of ML Code: NumPy

Purva Masurkar — Sun, 12 Mar 2023 17:38:02 +0000

"Success is not final, failure is not fatal: it is the courage to continue that counts." - Winston Churchill

On the third day of my #100daysofcode challenge, I learned NumPy (Numerical Python) a Python library that is used for scientific computing and data analysis. I referred various resources for this Baiscs of NumPy and NumPy in One hour by freeCodeCamp.

Introduction to NumPy

NumPy is the fundamental package for scientific computing with python. It contains many other things:

N-dimensional array object
Broadcasting Functions
Useful for linear algebra, Fourier transform and random number capabilities.

Broadcasting means when you do any operation on the array that operation will get implemented on each element of the array

Difference between numpy array and list

Data type in NumPy should be same whereas it can be different in lists
Faster to read less bytes of memory
Contiguous memory
No type checking when iterating through objects.

List is built in int-type so it consists of four different things it consists Size, Refernce count, Object Type, Object Value.
This makes NumPy faster than Lists.

Importing and Creating NumPy array

Broadcasting and Creating matrix

Creating Random arrays

Array Concatenation

Accessing/ Changing specific elements, rows, columns

Conclusion

During my recent learning experience, I acquired knowledge on statistics, array manipulation such as reorganizing and stacking, as well as loading data from files. Additionally, I attended a Women Coders Meetup where I gained valuable insights, expanded my knowledge, and networked with individuals who share my interests. As a result, I took a break from my studies to attend this event.
I will continue listing my daily progress and try to remain consistent. Please do share your feedback on how I can my 100daysofcode challenge more productive. I'll see you tomorrow for my daily update.

Follow me on Twitter: @purvamasurkar12
Follow me on LinkedIn: @purvamasurkar12

Day 2 of 100 Days of ML Code: Python Basics

Purva Masurkar — Thu, 09 Mar 2023 09:03:05 +0000

"Python is an experiment in how much freedom programmers need. Too much freedom and nobody can read another's code; too little and expressiveness is endangered." - Guido van Rossum, creator of Python

On the second day of my #100daysofcode challenge, I have successfully completed the fundamental concepts of Python, such as conditional statements, loops, data structures, string manipulation, functions, modules, packages, and text file handling. I reviewed these concepts from ML course for beginners. I am now excited to explore the Python Libraries for Data science.

I learned a lot of concepts today, but it was too many to list here. I will continue listing my daily progress starting from tomorrow. I'll see you tomorrow for my daily update.

Follow me on Twitter: @purvamasurkar12
Follow me on LinkedIn: @purvamasurkar12

My 100 Days of Machine Learning: A Journey Towards Consistent Programming

Purva Masurkar — Wed, 08 Mar 2023 04:24:12 +0000

Introduction

Hi everyone! I'm Purva Masurkar, currently a third-year student pursuing BE in Information Technology. Recently, I've made a decision to publicly commit to 100 days of learning Machine Learning and I'm excited to document my journey here on this blog.

As a student, I've struggled to consistently build my programming skills, often finding myself starting from the basics repeatedly and eventually giving up mid-way. Through public commitment and sharing my progress, I hope to build the habit of coding daily and improve my skills in Python, Machine Learning, and Deep Learning.

Additionally, this commitment to public sharing will allow me to improve my English through writing and interacting with like-minded individuals. By documenting my learning journey, I aim to not only track my progress but also convince others to embark on their own journey towards consistent programming and personal growth.

## Day 1

As I kick off my 100 Days of Machine Learning journey, I decided to start off by taking things slow and steady. Day 1 involved getting acquainted with some basic terminologies of Machine Learning. I didn't want to dive too deep into the complexity right away, so I'm taking my time to build a strong foundation. After all, Rome wasn't built in a day, and neither will my ML expertise!

Machine Learning

ML is a modern software development technique that enables computers to solve problems by using examples of real world data. It's a subset of Artificial Intelligence that includes three major categories:

Supervised Learning : In this every data has a corresponding label.
Unsupervised Learning: There are no labels for the training data.
Reinforcement learning: In this it tells us which action to take in a situation to maximize the reward.

Components of ML

ML model :It is an extremely generic program or block of code that can be modified to solve different but related problems.
Model training algorithm: The current model iteration is analyzed to determine what changes can be made to get closer to the goal.
Model Inference: Here the trained model is used to generate predictions.

Steps in ML process

Define the problem
Build the Dataset
Train the model
Evaluate the model
Use the model

We first collect the data and inspect it for outliers, missing values, data reformatting, and data visualisations. We also randomly split the data, which allows us to keep data hidden during training and use it to test the model before releasing it into production.

The model training algorithm iteratively updates a model's parameters to minimize some loss function.

The loss function is a measure of how well the model is performing on a given task. It measures the difference between the predicted output and the true output for a given set of input data.
Whereas Model's parameter are setting or configuration that the training algorithm can update to change how model behaves.

Hyperparameters are setting that are not changes during training but can affect how quickly or how reliably the model trains such as number of clusters the model should identify.

Closing Remarks

I understand that all of the above terms are basic, but I want to brush up on my previous knowledge for the first two days before diving deeper into it. Please leave comments on how I may better my learnings and blogs. I will take all of your suggestions into consideration.