Forem: Icheka Ozuru

Data Visualization Using Python

Icheka Ozuru — Tue, 16 Mar 2021 19:20:49 +0000

Intended Audience

If you're looking to improve your skills as a Pythoneer/Pythonista, or as a Data Scientist, congratulations! This article is for you!
This article is not for complete beginners to either Python or data analysis.

What Is Data Visualization?

Data visualization is the process of representing data visually in order to identify important patterns in the data that can be used as the basis for decision-making or inference. Visually representing data can reveal trends and patterns that would be nigh-impossible to perceive ordinarily.

Data visualization is a precursor to many important decision-making processes in the Data Science, Business Intelligence and Artificial Intelligence fields. It's usefulness cannot be over-emphasized. Undoubtedly, data visualization is one of the most important skillsets in the 21st century.

In this article, you will learn various data visualization techniques using Python, specifically using the NumPy, Matplotlib and Pandas libraries.

Data Visualization Using NumPy and Matplotlib

NumPy, which stands for Numerical Python, is arguably the most popular library for numerical computing in the world. Its applications include data engineering, statistics, business analysis, computational modeling and machine learning, across an ever-widening range of industries such as Engineering, Education, Health and Politics. What makes NumPy so popular is its ease of use and comfortable learning curve. While not strictly a data visualization library, we will use NumPy to generate some test data that our Matplotlib methods will act on.

Matplotlib is another library that is popular as a go-to for data visualization. It offers a nice interface that we will leverage to create graphical presentations of our test data.

I will assume that you already have Anaconda installed (head over to https://anaconda.com/distr and grab a copy if you don't).

The Line Plot

The line plot is the first plot that we are going to learn in this article. The line plot is the easiest of all the Matplotlib plots.
The line plot is normally used to plot the relationship between two numerical sets of values that have a one-to-one relationship between them. A good example is the relationship between Humidity and Time of Day in a region. Let us create test datasets and visualize this relationship in our data.

To create a uniform distribution of values for Humidity, we will use the linspace method of the NumPy library.

import numpy as np
from matplotlib import pyplot as plt
# another way to import pyplot is:
# import matplotlib.pyplot as plt

humidity = np.linspace(0, 100, 24) # generate 24 uniformly-distributed numbers from 0 to 100

Running print(humidity) should give you:

[  0.           4.34782609   8.69565217  13.04347826  17.39130435
  21.73913043  26.08695652  30.43478261  34.7826087   39.13043478
  43.47826087  47.82608696  52.17391304  56.52173913  60.86956522
  65.2173913   69.56521739  73.91304348  78.26086957  82.60869565
  86.95652174  91.30434783  95.65217391 100.        ]

Now we will create our Time-of-Day column:

import numpy as np
from matplotlib import pyplot as plt

humidity = np.linspace(0,100,24)
hours = np.arange(1,25) # generate 24 numbers from 1 (inclusive) to 25 (exclusive): this gives us our 24 hours

We can now plot our line plot using the plot method of pyplot:

import numpy as np
from matplotlib import pyplot as plt

humidity = np.linspace(0,100,24)
hours = np.arange(1,25)

plt.pyplot(humidity, hours) #plot a linear graph with humidity on the x-axis and hours on the y-axis

At this point you must be grinning to yourself. Pretty cool, eh?

Now, let's see what information we can extract from this plot. After all, inference is the end-goal of all data visualization.

The first thing you will notice about our plot is that it lacks labels: it does not tell us which dataset is Humidity or Hours. We will soon remedy this.

The second thing you will notice is that the line of our plot is 'straight' and runs unbroken from the lower-left vertex ([0,0]) to the upper-right vertex ([100, 24]). This tells us that the relationship between our datasets is linear, one-to-one and directly-proportional. An increase in the value at an instance in one dataset corresponds to an increase in the other. While this does not resemble real-life weather information, it is sufficient as an introduction to the line plot.

Labeling Our Data and Resizing Our Graph

Our line plot works, but it is more useful to label the axes of our plot. This way, we (and other people that we work with) will always know what data our visualization represents.

We can label our plot using the xlabel and ylabel methods.

import numpy as np
from matplotlib import pyplot as plt

humidity = np.linspace(0,100,24)
hours = np.arange(1,25)

plt.xlabel("Hours")
plt.ylabel("Humidity")
plt.plot(hours, humidity)

We now have a plot with Hours on the x-axis and Humidity on the y-axis.

Resizing Our Plot

We can resize our plot using the rcParams property of pyplot.
For this exercise, we will generate new data. We will plot the relationship between numbers and their square roots.

import numpy as np
from matplotlib import pyplot as plt
import math
# the math module provides specialized math functions 
# like the sqrt function that we will use

numbers = np.arange(0, 100, 10)
# generate 10 numbers from 0 to 100
squareroots = [math.sqrt(number) for number in numbers]
# generate a list of square roots using the 
# list-comprehension syntax

plt.xlabel("Numbers")
plt.ylabel("Square roots")
plt.title("Relationship between numbers and their square roots")

plt.rcParams['figure.figsize'] = [12,10]
# set the size of the plot on the x- and y-axis

plt.plot(numbers, squareroots)

The plot is noticeably larger. We have also set a title using the title method.

Making Our Plot More Readable Using Legends and Colors

We can improve the readability of our plot using legends and colors. Legends are a great way to provide important information about our data in a way that it can be understood at a glance!

A color for the line of our line plot can be specified by simply passing shorthand notation for the color name to the plot function.

We can pass 'r' for red, 'g' for green, and so on.

plt.plot(numbers, squareroots, 'r')

To create a legend for our plot, all we need to do is give our plot a label (this is not the same as labeling our axes) and set the location for our legend.

import numpy as np
from matplotlib import pyplot as plt
import math

numbers = np.arange(0, 100, 10)
print(numbers)
squareroots = [math.sqrt(number) for number in numbers]

plt.xlabel("Numbers")
plt.ylabel("Square roots")
plt.title("Relationship between numbers and their square roots")

plt.rcParams['figure.figsize'] = [12,10]

plt.plot(numbers, squareroots, 'r', label="Square Root")
plt.legend(loc="upper center") # set the location (or locus) of our legend

The Scatter Plot

A Scatter plot is used to plot the relationship between two numeric columns in the form of scattered points. It is normally used when for each value in the x-axis, there exists multiple values in the y-axis. This kind of relationship is called many-to-one.

A prime example is the relationship between the ages of children in a school and their heights. We are going to read the data from a CSV file using the read_csv method of the `pandas' library.

First, head over to https://raw.githubusercontent.com/hadley/r4ds/master/data/heights.csv to download the CSV file. Save the file in the same directory as your Jupyter notebook.

import numpy as np
from matplotlib import pyplot as plt
import math
import pandas as pd

data = pd.read_csv("heights-ages.csv") # use the name you saved your CSV file with here

plt.scatter(data['age'], data['height'])

import numpy as np
from matplotlib import pyplot as plt
import math
import pandas as pd

data = pd.read_csv("heights-ages.csv") # use the name you saved your CSV file with here

plt.scatter(data['age'], data['height'])

We can make our plot more readable my setting the color, labeling the axes and setting a legend. This will make it easier for us to infer information from our dataset.

import numpy as np
from matplotlib import pyplot as plt
import math
import pandas as pd

data = pd.read_csv("heights-ages.csv") # use the name you saved your CSV file with here

plt.scatter(data['age'], data['height'])

From this, we can infer that:

The average age of our 'students' is somewhere between 25 and 40. In fact, the average age is 38. We can see this by running:

import numpy as np
from matplotlib import pyplot as plt
import math
import pandas as pd

data = pd.read_csv("heights-ages.csv")

ages = np.array(data['age'])
print(np.median(ages))

The average height lies between 62.5 inches and 70.0 inches. This is, in fact, 66.45 inches, as we can confirm by running:

#...
heights = np.array(data['height'])
print(np.median(heights))

`
These are just examples of the kinds of information a scatter plot can give us.

The Bar Plot (Finally)

The bar plot is used to plot the relationship between unique values in two categorical column, often grouped by an aggregate function such as sum, mean or median.

We are going to reuse the CSV file from our scatter plot.

from matplotlib import pyplot as plt
import pandas as pd

data = pd.read_csv("./heights-ages.csv")

plt.xlabel("Sex")
plt.ylabel("Earnings")

plt.title("Plot of sex against income of inhabitants of planet Venus")

plt.bar(data['sex'], data['earn'])

The bar plot shows us clearly that there is a great disparity between the incomes of men and women on planet Venus!

Cheers and thank you for reading! I do hope you enjoyed reading as much as I did writing this. I'd be delighted to answer any questions you have on this or any other topic.

Thank you.

Practical Object-Oriented JavaScript

Icheka Ozuru — Mon, 15 Feb 2021 08:12:50 +0000

Object-Oriented: What Does It Mean?

As human beings, our lives depend on objects within our immediate and extended environments. We see, hear, feel, and manipulate various objects as we go about our daily business. The bunch of keys (and each individual key), the door knob, the bus, the iPhone, are all objects that we interact with in diverse and often complicated ways. Our perception of the world that surrounds us is based entirely on our experiences and memories of past interactions with objects. Interacting with objects requires us to assess them using metrics that are either distinct and measurable, or intuitive and immeasurable. For example, we often describe objects in terms of their size, their weight, their color and their value. In fact, these four qualities are the most intrinsic we can find for most objects.

Object-Oriented Programming recognizes the place of objects as central to our experience of life as humans and extends this into our programming. Object-Oriented (or Orientation Toward Objects) reminds the programmer that in OOP code, objects (and their attributes) are superlative. Prior to the development of Object-Oriented Programming languages and practices, programmers could only struggle to express physical structures (like rocks) and their attributes (like the color grey) using "primitive" data structures like the linked list and the array. This practice has proven to be quite inadequate for providing the kind of flexibility and seamlessness that today's applications demand. For example, it would be quite difficult to express a soccer player in a video game as an array or a hash table.

// soccer player as an array
// [name, nationality, occupation]
const Messi = ['Lionel Messi', 'Argentine', 'Soccer player']

Arrays are one on the most handy data structures we can use, but they are just unwieldy for this kind of description. To describe Lionel Messi better, in a way that we can easily manipulate, we need another data structure: objects.

JavaScript Objects

Objects in JavaScript are often more flexible to work with:

// soccer player as an object
const Messi = {
    'name': 'Lionel Messi',
    'nationality': 'Argentine',
    'occupation': 'Soccer player'
}

In JavaScript, objects are often created by simply declaring a variable (using the let, const or var keywords) and assigning data to it using the object curly brace syntax.

Curly brace syntax: key-value pairs of data, separated from each other by single commas, and enclosed within a pair of opening and closing curly braces.

/*
{
    key: value
}
*/
... {
    'name': 'Lionel Messi',
    ...
}

Objects like this are handy because we can easily refer to their attributes using their keys, like so:

// two methods of referring to object attributes
// 1. using the 'dot' syntax
console.log(Messi.name);
// prints "Lionel Messi"

// 2. using square brackets
console.log(Messi['name'])
// prints "Lionel Messi"

Most people prefer to use the 'dot' syntax to refer to traverse their objects.

Creating Re-usable Objects

We have our 'Messi' object and we have defined a name, nationality and occupation for our object. We can easily extend the object's properties by adding attributes as key: value pairs. Like so:

// Messi object, extended
const Messi = {
    'name': 'Lionel Messi',
    'nationality': 'Argentine',
    'occupation': 'Soccer player',
    'team': 'FC Barcelona',
    'age': 33
}

console.log(Messi.age, Messi.team)
// prints 33 "FC Barcelona"

But what if we wanted to create more soccer players for our video game? Would we define and initialize objects like this for the 11-player FC Barcelona team? Plus another 11 for Manchester United's? This might work, but what if we need to extend our player objects? For example, if we wanted to add a 'height' property to our players? Would we make this change in 22 places? Or in 990, if we have 90 teams in our video game?

Luckily for us, the variable declaration method isn't all that we have to work with. More robust options exist, of which about the most robust is to create a prototype so that we can simply copy or clone similar objects from one source. My favorite way of doing this is using the class keyword.

The class keyword is special in a lot of languages, particularly Object-Oriented ones. It is widely used to declare a Class, or a blueprint or schema for cloning objects. The class keyword is used across several popular languages, such as Python, C++, Java, C#, PHP and CSS. It almost always means: "here comes a blueprint for a specific kind of object". After creating our classes, we can easily derive as many objects as we need from them. Each object is a child that will inherit all of its parent's attributes/properties.

// create a class called Player
class Player {
    setName(name) {
      this.name = name
    }
}

// clone three objects from it
let player_1 = new Player();
let player_2 = new Player();
let player_3 = new Player();

// set player_1's name
player_1.setName('Lionel Messi')
// print player_1's name
console.log(player_1.name)
// prints "Lionel Messi"

// set player_2's name
player_2.setName('Diego Maradona')
// print player_2's name
console.log(player_2.name)
// prints "Diego Maradona"

We started by creating a class, using the class keyword:

class Player {
}

Next, we added a function (also called a member function) for setting the Player object's name. Note that calling console.log(player_1.name) before setting the name will print undefined. In JavaScript, member functions of prototypes (i.e blueprints) defined using the class keyword are not defined with the function keyword:

class Player {
    // right
    setName(name) {
        this.name = name
    }

    // wrong
    function setName(name) {
        this.name = name
    }
}

The setName member function sets the name property of the object it is called from.

...
    this.name = name
...

Finally, we print the name property of our objects:

...
console.log(person_2.name)
...

Attributes of Objects: Properties and Member Functions

Objects in programming, in general, and in JavaScript, in particular, have attributes that are inherent and intrinsic to their nature, first as objects, and second as specific objects. Specificity refers to objects that are derived from the same prototype (or class). These attributes also exist in the real world. They are not some abstract idea that is far removed from the realm of the average developer. These attributes are divided into two groups:

Object properties
Object member functions

Understanding these attributes is easy: a soccer player like Lionel Messi has a name, measurable weight, height, complexion and favorite color. These are all object properties.

class Player {
    setProps(name, age, weight, height) {
        this.name = name
        this.age = age
        this.weight = weight
        this.height = height    
    }
}

let Messi = new Player()
Messi.setProps("Lionel Messi", 33, "200Ibs", "1.7m")

console.log(Messi)
/*
prints --
Object {
  age: 33,
  height: "1.7m",
  name: "Lionel Messi",
  weight: "200Ibs"
}
*/

In real life, Lionel Messi knows how to sprint, tackle, kick, jump, and shoot. These are member functions.

class Player {
    setProps(name, age, weight, height) {
        this.name = name
        this.age = age
        this.weight = weight
        this.height = height    
    }

    tackle() {
        console.log(this.name + " executed a tackle!")
    }

    sprint() {
        console.log(this.name + " is running!")
    }

    shoot() {
        console.log(this.name + " kicked the ball really hard this time!")
    }
}

let Messi = new Player()
Messi.setProps("Lionel Messi", 33, "200Ibs", "1.7m")
Messi.sprint()
Messi.tackle()
Messi.shoot()

Performing Time-Of-Instantiation Tasks With Constructors

You'll often find that there are certain tasks that you would like your object to perform as soon as it is created (properly called instantiation). Such tasks may include starting an event loop, making an API call, or simply setting a few key properties (e.g name, age and height). In our code above, we can notice that Messi.setProps() needs to be called as soon as possible after the Messi object is instantiated. To 'instatiate' means to create a new object from a prototype using the new keyword, just as we have been doing. The name, age, weight and height properties need to be set before any call to the action member functions (sprint, shoot, tackle) may be called. Calling Messi.tackle() before Messi.setProps(), for example, will result in a slight error since the name property of the object has not been initialized yet. This way, it is quite easy for a programmer, even a first-rate one, to make this mistake and attempt to have the object tackle before it is given a name.

class Player {
    setProps(name, age, weight, height) {
        this.name = name
        this.age = age
        this.weight = weight
        this.height = height    
    }

    tackle() {
        console.log(this.name + " executed a tackle!")
    }
}

let Messi = new Player()
// 1
Messi.tackle()
// prints "undefined executed a tackle"

Messi.setProps("Lionel Messi", 33, "200Ibs", "1.7m")

// 2
Messi.tackle()
// prints "Lionel Messi executed a tackle"

The task of initializing key properties is usually handled by a special kind of function called a constructor. The constructor is a special function that the programming language calls as soon as the object is instantiated. This nice feature allows the programmer to have his application handle important tasks during the object instantiation process.

In our case, we want our player objects to have a name, age, weight and height, and we would like to set up all of these as soon as the object is instantiated.

In JavaScript, the special constructor function is called simply constructor. Using the special constructor function, we can create Player objects and assign unique names, ages, weights and heights like so:

class Player {
    constructor(name, age, weight, height) {
        this.name = name
        this.age = age
        this.weight = weight
        this.height = height    
    }

    tackle() {
        console.log(this.name + " executed a tackle!")
    }
}

// create a new Player object and assign properties at instantiation-time
let Messi = new Player("Lionel Messi", 33, "200Ibs", "1.7m")

console.log(Messi.name)
// prints "Lionel Messi"
Messi.tackle()
// prints "Lionel Messi executed a tackle!"

When I work, I usually prefer to use the constructor to initialize properties like so:

class Player {
    constructor(props) {
        Object.keys(props).forEach(prop => this[prop] = props[prop])
    }

    tackle() {
        console.log(this.name + " executed a tackle!")
    }
}

// create a new Player object and assign properties at instantiation-time
let props = {
    name: "Lionel Messi",
    age: 33,
    weight: "200Ibs",
    height: "1.7m"
}
let Messi = new Player(props)

console.log(Messi.name)
// prints "Lionel Messi"
Messi.tackle()
// prints "Lionel Messi executed a tackle!"

This way, I can easily create several similar objects, without having to write the properties each time I create an object. I can just pass the same props object (remember, objects are key: value pairs of data: props is an object). If I need to create 5 more Messi objects:

...
let Messi2 = new Player(props)
let Messi3 = new Player(props)
let Messi4 = new Player(props)
let Messi5 = new Player(props)
let Messi6 = new Player(props)

And if I need to effect a change in their properties later in my program, I can make that change once in the props object and all my six Messi Player objects will be updated:

...
// change the 'weight' property of the props object
props.weight = "400Ibs" // same as writing props['weight'] = "400Ibs"

...
let Messi6 = new Player(props)

console.log(Messi6.weight)
// prints "400Ibs"

There you have it! You can go on to add more properties and methods (member functions) to your Player class, and even use it to do really nice things, like write a soccer game. You can also create other classes and objects by following the general process we used here.
I hope you enjoyed reading this as much as I did writing it.

May the Code be with us all.
Cheers.

Learn Object-Oriented Programming In C++

Icheka Ozuru — Sat, 06 Feb 2021 12:47:00 +0000

Learn Object-Oriented Programming In C++

Throughout our lives, we interact with objects. We see, we feel, we touch and manipulate. We pick things up and we drop them. We even describe our experiences with objects to other persons: we talk about attributes of objects we have interacted with in terms of their size, shape, color, weight or usefulness.

The idea of objects, their form, properties and value, is central to our existence as human beings. For example, we know that a car is red because we can identify the color red as one of its attributes. Just as we know that a rock is heavy because we can identify its weight as one of its properties.

This idea is the basis of Object-Oriented Programming practices. That is, data can be represented as objects that have attributes (which we call properties or props) and methods (which we also call member functions). Intrinsically, objects (in the real world and in the world of code) have values that can be deduced, depended upon and evaluated. We will see this momentarily.

For many students, the idea of objects and Object-Oriented Programming is a difficult one to grasp, often because there exists a "leap of faith" between the practice of procedural, logical, programming that they are used to and the "new" practice of objects and object-abstraction that OOP requires. Students are often scared to make this jump: "What if I don't get it quickly enough?" because the idea of representing data in their code as objects that can be interacted with (by the user of the system, the programmer, and even other objects) is strange.

This article aims to eradicate this fear, not entirely, because fear makes the journey of learning exciting, but just enough to help the reader make this leap. The concepts discussed here are simple enough for anybody to follow in any Object-Oriented language (not just C++). I will publish future versions of this article that will introduce the concepts again in other popular high-level languages like JavaScript, Java and Python.

The practice of Object-Oriented Programming grew out of the need for more effective ways of expressing the solution to many programming problems in terms of natural, real-world structures and processes. This new perspective makes problem-solving simpler, easier and less prone to bugs because problems, solutions and data can now be expressed in ways that every programmer is familiar with. Instead of talking about data in terms of bits and memory blocks, OOP encourages programmers to discuss their code in terms of real-life objects like fruits and stones. This is known as abstraction.

Objects As Structures

One of the first concepts I would like the reader to come to terms with is the idea of objects as structures. Structures are concrete matter. They have a noun by which they can be referred to (stone). This is their first attribute or property. They have mass (their second attribute or property) and they occupy space (their third property). A proper example is a stone as a structure. A stone has mass and can be weighed. It also occupies some space, which we call volume. This property can be more easily observed when a person drops a bit of stone in a bucket filled to the brim with a liquid, like water. Some liquid is displaced, and the volume of the displaced liquid is equal to the volume of the stone.
In order to properly represent a stone as data in a program, we will need to:

Give our data structure a name.
Give our data structure some mass.
Give our data structure some volume.

We create objects in C++ (and in many other languages like JavaScript, Python, Java and PHP) using something called a class.
A class is simply a document where we write the properties and functions of our objects.

#include <string>
using namespace std;

class Stone {
    public:
        string mass;
        string volume;
};

int main() {
   return 0;
}

It is the convention to name classes in camel-case.

We now have a class (which is just a place where we write the properties and functions of our objects) named "Stone" (first property) and having both mass and volume (second and third properties). For now, we have not assigned values to the mass and volume properties (i.e we haven't given the mass a value, like 10kg), but this is normal and we'll often have classes that just describe our objects but don't actually assign values to them. We often assign our values when we create objects of the class. This process is known as instantiation and we will see how in a bit.

Why 'public'?

You must have noticed the public keyword in our Stone class. What does it mean?
This particular keyword is an important part of creating classes. It allows us to define properties and functions that are accessible by functions or objects outside the class. Remember when I said objects can interact with other objects? They do this by accessing these public properties and functions. It is called public because the attributes it encapsulates are publicly-accessible. But we will see a bit of this soon.

Extending Our Stone Class

Our Stone class has two properties (albeit three attributes: its class name, and its two public properties). We can add a third and a fourth public property: color and texture.

#include <string>
using namespace std;

class Stone {
    public:
        string mass;
        string volume;
        string color;
        string texture;
};

int main() {
    return 0;
}

Now we can describe any Stone object we instantiate from this class in four terms: mass, volume, color and texture. The good thing about this is that we can now have several stones in our program, each with its own unique properties. We can have a heavy stone and a light stone, a blue stone and a purple stone, a smooth stone and a rough stone. We can even add another property to our class (value_in_dollars) and have expensive stones (like sapphires, for example) and cheap stones (like gravel).

#include <string>
using namespace std;

class Stone {
    public:
        string mass;
        string volume;
        string color;
        string texture;
        int value_in_dollars;
};

int main() {
    return 0;
}

Instantiation: making objects from classes

Instantiation is the process of declaring a new object based on the properties and methods (functions) of a class.

Instantiation is the final step in creating objects and we use it every time we perform operations in programming. The truth, which I feel very guilt-stricken about keeping from you for this long, is that data types (like integers and strings and vectors) are all classes that we instantiate from standard classes that come pre-packaged with the language.
Let's see how this works by, first, looking at constructors.

Object Constructors

An object constructor is a special function that is executed each time instantiation occurs. In plain English, this compiles to: constructors are special functions that are run whenever an object is created from a class. They are useful for two major reasons:

They make it possible for the programmer to perform certain operations on objects or on data in the program for every object that is instantiated.
They make it easier to individualize objects so that, although two objects were instantiated from the same class, they have different properties and can behave in different, albeit predictable, ways.

In real life, every human has several properties that make him or her unique. A person can be 1.6 meters tall, light-skinned, red-haired, and have a large nose. These are all properties of this person. Her friend can be 1.7 meters tall, dark-skinned, dark-haired, and have a small nose. Despite their differences, these two persons have the same characteristics. They have:
1. Height
2. Complexion
3. Hair color
4. Nose size

So how do we express this difference in code, especially in OOP code?

Instantiation

The process of instantiation is a pretty straightforward one. We instantiate using the new keyword. Like so:

#include <string>
using namespace std;

class Person {
    public:
        string name;
        float height;
        string complexion;
        string hair_color;
        string nose_size;
};

int main() {
    // instantiation
    Person person_1;
    Person Anna;
    Person Bella;

    return 0;
}

You might notice how I use Person just like I would use int or string or float. Remember how I 'confessed' earlier that I had been hiding something from you? This is it: the classes that you make are data types. Object-Oriented Programming is really about expressing real-world objects as data structures. Object-Oriented Programming gives you the freedom and the power to define your own data types. You are not limited by types int, float and char. As a matter of fact, one fundamental difference between C++ and C is the addition of a new data type: string. The class that makes this possible (string) is written right there in the Standard C++ Library. If this isn't neat, I don't know what is.

The classes that you make are data types.

Individualism Using Constructors

We have learned how to instantiate new objects from our classes using the new keyword. OOP takes this freedom further by giving us the ability to individualize our objects. In real life, we have nearly 8 billion individual Persons, and each person is unique. One property that makes a person unique is his (/her) name. This name is often assigned at, or soon after, birth. In the case of our code, the constructor function allows us to access the events that take place at the birth (or construction) of our objects. We can give our persons names using the constructor function so that each person is 'born' with a unique name.

#include <string>
#include <iostream>
using namespace std;

class Person {
    public:
        string name;
        float height;
        string complexion;
        string hair_color;
        string nose_size;
        Person(string birth_name) {
            name = birth_name; // makes the person's name the name he is given at birth
        }
};

int main() {
    Person Anne("Anne");
    cout "Anne's name is " << Anne.name << endl;
    // Anne's name is Anne
}

In C++ we create constructors by naming them after their classes.

#include <string>
#include <iostream>
using namespace std;

class Stone {
    public:
        string kind;
        float weight;
        float value_in_dollars;
        // constructor below
        Stone(string kind, float value) {
            kind = kind;
            value_in_dollars = value;
        }
};

int main() {
    Stone my_sapphire("sapphire", 3120.00);
    cout << my_sapphire.value_in_dollars << endl;
    // 3120.00
    cout << my_sapphire.kind << endl;
    // sapphire
}

Final Section: Member Functions

The final lesson we'll see in this article is member functions and how they relate with the concept of objects.

A member function (also called a "method") is a function defined within a class. In clearer terms, a member function is an operation that objects of a class have access to. They can call these functions, get return values from them, and use them to manipulate the data within that instance of the class (i.e within the object itself).

Member functions are important because they allow programmers to express actions and processes after real-life counterparts. For example, people speak, hear and think. We can define these verbs in our Person class like so:

#include <string>
#include <iostream>
using namespace std;

class Person {
    public:
        string name;
        float height;
        string complexion;
        string hair_color;
        string nose_size;
        Person(string birth_name) {
            name = birth_name; // makes the person's name the name he is given at birth
        }

        void speak(string speech) {
            cout << this.name << " says: " << speech << endl;
        }

        void listen(string speech) {
            cout << "You said '" << speech << "'" << endl;
        }
};

int main() {
    Person Anne("Anne");
    Anne.speak("The weather is hot today, don't you agree?");
    // Anne says: The weather is hot today, don't you agree?
    Anne.listen("What will you do this weekend?");
    // You said 'What will you do this weekend?'
}

Member functions can also be used to change the data within the object (this data is called state).

#include <string>
#include <iostream>
using namespace std;

class Person {
    public:
        string name;
        float height;
        string complexion;
        string hair_color;
        string nose_size;
        Person(string birth_name) {
            name = birth_name; // makes the person's name the name he is given at birth
        }

        void changeName(string new_name) {
            this->name = new_name;
        }

        void listen(string speech) {
            cout << "You said '" << speech << "'" << endl;
        }
};

int main() {
    Person Anne("Anne");
    Anne.speak("The weather is hot today, don't you agree?");
    // Anne says: The weather is hot today, don't you agree?
    Anne.listen("What will you do this weekend?");
    // You said 'What will you do this weekend?'
}

There you go! I hope that you have seen how easy and useful Object-Oriented Programming is, and that you have made your leap of faith. This isn't all there is, but it is enough to help you write applications for simple problems by yourself. If you really, really want to master Object-Oriented Programming, I recommend that you take some time to read about the subject. In particular, W3Schools has a nice tutorial on the subject. It's very simple and concise. You can find it here.
Also, Bruce Eckel's Thinking In C++ is an excellent book about the language. I found it a bit verbose, and it does seem targeted at the mid-level to advanced programmer (but not necessarily a C++ programmer), but there is so much insight to gain and Bruce shares his vast wealth of experience liberally.

Don't hesitate to reach out to me if you need any help at all. I can be reached via email at rhemafortune@gmail.com. I'll email you back as soon as possible.

May the Code be with us all.
Cheers.

A Gentle Introduction To Computer Algorithms: Implementing the Bubble Sort algorithm in six languages.

Icheka Ozuru — Fri, 22 Jan 2021 00:10:06 +0000

A Gentle Introduction To Computer Algorithms: Implementing the Bubble Sort algorithm in six languages (Python, JavaScript, C++, C, Java, PHP)

Computer algorithms form one of the two core pillars of computer science (the other being data structures) and a comfortable understanding of algorithms is necessary for the programmer/developer who is intent on standing out in the field. In this article we will explore a succint overview of computer algorithms, the necessity of their study, their applications, as well learn to write one all by ourselves. The code (which comes towards the end of this article) is written in five of the most popular languages so that almost anybody can follow. All one has to do is read along with the language one is most familiar with.

The code in this article is available on GitHub here. Click on the appropriate file to view/save the source code for the algorithm in your language (C, C++, Java, JavaScript, PHP, Python).

A Gentle Dive

Computer science developed from humankind's innate need for tooling: our primal drive to find easier, faster, more efficient ways of solving problems. In the case of the first computer, the abacus, humankind wanted a way to perform basic arithmetic under the hot and scorching sun of their open-air market stalls. Think about it: if you lived 4400 years ago (in a world without iPhones and the Internet, mind you), would you:

Want to tally your pebbles by hand every time?

Carry around a wooden box capable of keeping track of your profit for you?

Yeah. I knew you'd choose (2).

So, the abacus is humankind's earliest recognized attempt at teaching a machine (aka, a tool) to accomplish a task. Prior to that point in time, we had tried to teach the animals, but you know how poorly horses count!

So, the most basic lesson in this article is this: "Don't be afraid of algorithms. An algorithm is only a textbook for machines (and machines are merely tools, unless you're like me and you call them "pals") to 'read' and 'learn' from. Algorithms are nothing to be afraid of."

An algorithm is only a textbook for machines to "read" and "learn" from.

I make this point here, before you read any further, because for many people, the word "algorithm" is like a scare-crow in the barn at night. Many programmers, especially inexperienced ones, tend to avoid the subject entirely. Until they need to prepare for a interview. This world-wide fear of the second pillar of computer science is probably due to three factors, which I outline here:

Algorithms often (but not always) involve some kind of math.
Algorithms are often taught at universities by old, gray-haired professors in an old, gray-haired language like C. Many programmers who were formally trained in the craft graduate with an ingrained fear of the subject.
At least 50% of programmers alive** today have not received any formal instruction in computer science. They are self-taught, and usually one finds that the self-taught programmer skipped the basics (data structures and algorithms) in a hurry to start the application development part (web development, for example).

To every self-taught programmer who reads this: you're one of the most inspiring people in the world. I appreciate and admire you. Keep up the good work!

So, as I said earlier, we find many programmers who fear algorithms: the purpose of this article is to remedy that in some small, humble way.

What, Exactly, Is An Algorithm?

I won't bore you with the etymology of the word (which you can find by doing a web search). An algorithm is a process, often logical, that you pass to a computer (aka "machine", double aka "just a tool") to "teach" it to solve a problem. The website Britannica.com explains:

An algorithm is a specific procedure for solving a well-defined computational problem.
Algorithms and Complexity, Britannica.com

Britannica continues: "It requires an understanding of the alternatives available for solving a computational problem, including the hardware, networking, programming language, and performance constraints that accompany any particular solution."

Algorithms are used everywhere. When you pick up your smartphone and place a call to a friend, several algorithms make the connection, convert your voice (sound) to signals (electricity), convert these signals to radio waves, and "push" the waves far enough to a cell tower where other algorithms perform really complicated operations on the radio waves that make it possible for your friend's phone to ring. In fact, algorithms are possibly the foundation of life itself (think: one cell joins with another cell in a very specific way, and nine months later: a collection of hundreds of millions of cells is born).

You use algorithms each time you cross a street.

Yes. A breakdown of the process might help you realize this:
Step 1: You decide to cross the street. In making this decision, you have already used several algorithms (for example: "if I leave now will I be home in time for dinner? Yes. So I will.") to act on data (for example: "the time for dinner is set at 6.30pm and it's 4.30pm already, but it takes an hour and fifty minutes to get home from here") and the result of your computations spurs you into action.
Step 2: You stop at the curb/sidewalk. This small act is the result of many, many (think: hundreds or even thousands) algorithms that you have trained your brain to remember and execute over the years. Two-year olds lack these algorithms, which is why you often see them trying to play "hugs" in the middle of a busy street. Again, your brain is acting on data it receives in real-time from your eyes, ears, skin.
Step 3: You look left.
Step 4: You look right.
Step 5: You look left again.
Step 6: The road is clear so you walk briskly across.
Step 7: A horrible person speeds by while you are still halfway across.
Step 8: You jump a little. Land on the curb. Wince and give her the finger.

Sound familiar?

In reality, it often requires many more steps (for example, I have to navigate a very busy bridge twice daily: in the mornings on my way to work, in the evenings on my way back) but your brain has the right algorithms so you survive.

Do you see how simple working with algorithms is now?

Analyzing The Bubble Sort Algorithm
The Bubble Sort algorithm is a very special algorithm. It is the simplest algorithm in the world for sorting a collection of items, but this isn't what makes it so special.
The Bubble Sort algorithm is special because it can be used to teach any and every class of programmer: from newbies who are still studying data structures to mid-level developers who want to learn to optimize their code to more experienced programmers who want to gain deeper insight into the inner workings of computer systems.
Why 'Bubble'?
The special algorithm is called 'Bubble Sort' because it uses a simple method to sort collections in a manner resembling 'bubbling'. The GIF below shows this bubbling at work.

Observe how the number 8 has 'bubbled' to the end of the collection.

Basically, the algorithm sorts collections by comparing each consecutive item in the collection with its neighbor (usually the neighbor to the right). If the item is larger than the neighbor to the right (or less than the neighbor to the left), the algorithm swaps the two items (i.e the current item and the neighbor), arranging the collection in the process.
The Bubble Sort algorithm can be used to sort a collection in either ascending or descending order. In this article we will work on learning to sort in ascending order, but I will leave assignments for sorting in descending order at the end so that you can practice and master your new knowledge.

Breaking Down The Bubble Sort Algorithm
Let's imagine a collection (either a list or an array will do). This collection will look like one of these, depending on your language:

// C/C++

int arr[] = {5,3,1,4,8,7,9}; //array

// Java

int arr[] = {5,3,1,4,8,7,9}; //array

// JavaScript

let arr = [5,3,1,4,8,7,9]; //array

// PHP

$arr = array(5,3,1,4,8,7,9); //array

# Python

arr = [5,3,1,4,8,7,9] #list: Python does not have arrays

Notice how the items of each array/list are randomly distributed.

Our implementation of the Bubble Sort algorithm will start at the beginning of the collection (index 0) and compare each item with its near-right neighbor. If the item is less than its neighbor, the algorithm will leave the item in position and move on to compare the item's near-right neighbor with its own near-right neighbor.
5 (at index 0) is larger than 3 (its near-right neighbor) so our algorithm will swap their positions: index 0 becomes 3 while index 1 becomes 5*. At this point the collection will look like:

[3,5,1,4,8,7,9] #python

[3,5,1,4,8,7,9]; //JavaScript

{3,5,1,4,8,7,9}; #C/C++/Java

The algorithm will do the same thing for index 1 and index 2 (5 and 1, respectively). At this point the collection will look like:

[3,1,5,4,8,7,9] #python

{3,1,5,4,8,7,9}; #C/C++/Java/JavaScript

The algorithm will continue to do this until it gets to index 5 (7) and index 6 (9). Since 7 is less than 9, the algorithm will not swap these items.

Easy, right?

The Code: Implementing The Bubble Sort Algorithm
We have analyzed the Bubble Sort algorithm and written some pseudocode (which is what programmers call a step-by-step analysis of an algorithm written in a format that slightly resembles a real programming language. We did this in the section above this.). We are now equipped enough to implement the algorithm in our favorite programming language.

If you are not very comfortable in any of these five languages, you can still benefit maximally from this article. By following the pseudocode you can write an implementation in your language. If you get stuck, a quick web search might be of help. In any case, please, do not hesitate to reach out to me via email via rhemafortune@gmail.com. I'd be delighted to assist you.

Feel free to skip to your language in this section.

C

// C

#include <stdio.h>

void bubbleSort(int arr[], int n)
{
    int i, j, temp;
    for(i = 0; i < n; i++)
    {
        for(j = 0; j < n-i-1; j++)
        {
            if( arr[j] > arr[j+1])
            {
                // swap the elements
                temp = arr[j];
                arr[j] = arr[j+1];
                arr[j+1] = temp;
            }
        }
    }

    // print the sorted array
    printf("Sorted Array: ");
    for(i = 0; i < n; i++)
    {
        printf("%d  ", arr[i]);
    }
}

int main()
{
    int arr[100], i, n, step, temp;
    // ask user for number of elements to be sorted
    printf("Enter the number of elements to be sorted: ");
    scanf("%d", &n);
    // input elements if the array
    for(i = 0; i < n; i++)
    {
        printf("Enter element no. %d: ", i+1);
        scanf("%d", &arr[i]);
    }
    // call the function bubbleSort
    bubbleSort(arr, n);

    return 0;
}

C file available on GitHub: here

C++

// C++

#include <stdio.h>

#define MAXSIZE 10 
// change the value of MAXSIZE to the number of items in your array when you test

int main() {
    int array[MAXSIZE];
    int i, j, num, temp;

    printf("Enter the count of elements \n");
    scanf("%d", &num);
    printf("Enter the elements one by one \n");
    for (i = 0; i < num; i++) {
        printf("Enter element %d of %d\n", (i+1), num);
        scanf("%d", &array[i]);
    }

    for (i = 0; i < num; i++) {
        for (j = 0; j < (num - i - 1); j++) {
            if (array[j] > array[j + 1]) {
                temp = array[j];
                array[j] = array[j + 1];
                array[j + 1] = temp;
            }
        }
    }

    printf("Sorted array is...\n");
    for (i = 0; i < num; i++) {
        printf("%d\n", array[i]);
    }
}

C++ file available on GitHub: here

Java

// Java

class Sort { 
    void bubbleSort(int arr[]) { 
        int n = arr.length; 
        for (int i = 0; i < n-1; i++) 
            for (int j = 0; j < n-i-1; j++) 
                if (arr[j] > arr[j+1]) 
                { 
                    // swap arr[j+1] and arr[j] 
                    int temp = arr[j]; 
                    arr[j] = arr[j+1]; 
                    arr[j+1] = temp; 
                } 
    } 

    /* Output the array */
    void printArray(int arr[]) { 
        int n = arr.length; 
        System.out.print("["):
        for (int i = 0; i < n; ++i) System.out.print(arr[i] + ", "); 
        System.out.print("["):
        System.out.println();
    } 
}

    public static void main(String args[]) { 
        Sort sort = new Sort(); 
        int arr[] = {5,3,1,4,8,7,9};
        sort.bubbleSort(arr); 
        System.out.println("Sorted array"); 
        sort.printArray(arr); 
    }

Java file available on GitHub: here

JavaScript

class Sort {
  swap(arr, idx1, idx2) {
    if (arr[idx1] > arr[idx2]) {
      let tmp = arr[idx1];
      arr[idx1] = arr[idx2];
      arr[idx2] = tmp;
    }
    return arr;
  }

  bubbleSort(arr) {
    for (let i = 0; i < arr.length - 1; i++) {
      for (let j = 0; j < arr.length - 1; j++) {
        arr = this.swap(arr, j, j + 1);
      }
    }

    return arr;
  }
}

function main() {
    const arr = [3,5,1,4,8,7,9];
    const Sort = new Sort();
    return Sort.bubbleSort(arr);
}

console.log(main());

JavaScript code available on CodeSandbox: here

Also available on GitHub: here

PHP

// PHP

class Sort {
    protected sort(array $arr, int $idx1, int $idx2) {
        if ($arr[idx1] > $arr[idx2]) {
            $tmp = $arr[idx1];
            $arr[idx1] = $arr[idx2];
            $arr[idx2] = $tmp;
        }

        return arr;
    }

    public function bubbleSort(array $arr) {
        for ($i = 0; $i < count(arr) - 1; $i++) { //perform the swap operation on each item in the array
            for ($j = 0; $j < count(arr) - 1; $j++) { //get each item and its near-right neighbor
                $arr = $this->swap($arr, $j, $j + 1); //perform the swap 
            }
        }

        return $arr;
    }

}

function main() {
    $arr = array(3,5,1,4,8,7,9);
    $sort = new Sort();
    print_r($sort->bubbleSort($arr));   
}

main();

PHP code available on GitHub: here

Python

# Python

class Sort:
    def _swap(self, arr, idx1, idx2):
        if (arr[idx1] > arr[idx2]):
            tmp = arr[idx1]
            arr[idx1] = arr[idx2]
            arr[idx2] = tmp

        return arr

    def bubbleSort(self, arr):
        for i in range(0, len(arr) - 1):
            for idx in range(0, len(arr) - 1):
                arr = self._swap(arr, idx, idx + 1)

        return arr


def main():
    arr = [3,5,1,-3,4,8,7,9]
    sort = Sort()
    sorted_arr = sort.bubbleSort(arr)
    return sorted_arr

print("Sorted list: {}".format(main()))

Python code available on GitHub: here

I hope you learned a lot. I look forward to hearing from you.
May the Code be with us all.
Cheers!

Coming Soon:

Articles and essays planned for the next couple of weeks:

Introduction To Web Scraping With Python: Part 3
**Introduction To Statistical Analysis Using NumPy and Python
Bits, Bytes and Everything In-between: How Computers Really Work
The T-shaped Developer
Five Algorithms That The World Cannot Do Without

Recently Published:

Introduction To Web Scraping With Python: Part 2

Introduction To Web Scraping With Python: Part 2

Icheka Ozuru — Fri, 15 Jan 2021 23:46:27 +0000

“If programming is magic, then web scraping is surely a form of wizardry.”

Web Scraping With Python; Ryan Mitchell.

This is article 2 of 4. If you have not read article 1, or if you do not have foundational knowledge web scraping with Python, please, read the first article.

In the first article of the series, I gave an introduction to the meaning, applications and basic principles of web scraping. I also introduced the Python programming language as a very robust general-purpose language that is very well suited for data retrieval/handling tasks like web scraping. I briefly discussed the BeautifulSoup library, and I shared a few illustrative scrips that display the basic syntax of the BeautifulSoup library. In addition to these, I explained the basic mechanics of networking -- creating requests and receiving responses, as well as the foundational principles behind server design.

In this article, I am going to take things up a notch or several. I will provide more in-depth analysis of the urllib and BeautifulSoup libraries, their syntax, methods and properties; errors and exceptions that can arise during the data-retrieval process, and robust error-handling approaches to designing and implementing scrapers. I will go through easy-to-follow steps too creating reliable web scrapers, as well as how to store the data they retrieve for analysis in future. Throughout the article, I will share code that has been written for implementing real-world solutions to practical problems. At the end of this article, assuming that you have read and completely understood the previous one (read article 1 here), you will be able to tackle real-world problems with simple scrapers that you can depend on to retrieve and store your data for you.

The First Rule Of Web Scraping

Always look for patterns and target specific attributes.

The First Rule, and possibly the most important, is to always inspect the data (e.g web document) for patterns that you can target using attributes in the markup, and write scrapers that target these attributes.

The first URL we will scrape for data is a simple online store. The web document contains information about all available products (product name, price in USD, description, and rating). We will begin by learning to quickly extract information about patterns in markup. Then we will learn to design our scraper to extract the needed information based on the attributes our patterns offer.

A quick and easy way to view the markup for a page is to request the document, parse it using BeautifulSoup, and use BeautifulSoup's prettify() method to indent it and make it easily-understandable by humans.

# 1.py
from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen('https://webscraper.io/test-sites/e-commerce/allinone')
bsoup = BeautifulSoup(html.read(), 'html.parser')
print(bsoup.prettify())

This script should give output similar to:

<!DOCTYPE html>
<html lang="en">
 <head>
  <!-- Anti-flicker snippet (recommended)  -->
  <style>
   .async-hide {
        opacity: 0 !important
    }
  </style>
  <script>
   (function (a, s, y, n, c, h, i, d, e) {
        s.className += ' ' + y;
        h.start = 1 * new Date;
        h.end = i = function () {
            s.className = s.className.replace(RegExp(' ?' + y), '')
        };
        (a[n] = a[n] || []).hide = h;
        setTimeout(function () {
            i();
            h.end = null
        }, c);
        h.timeout = c;
    })(window, document.documentElement, 'async-hide', 'dataLayer', 4000,
        {'GTM-NVFPDWB': true});
  </script>
  <!-- Google Tag Manager -->
  <script>
   (function (w, d, s, l, i) {
        w[l] = w[l] || [];
        w[l].push({
            'gtm.start':
                new Date().getTime(), event: 'gtm.js'
        });
        var f = d.getElementsByTagName(s)[0],
            j = d.createElement(s), dl = l != 'dataLayer' ? '&l=' + l : '';
        j.async = true;
        j.src =
            'https://www.googletagmanager.com/gtm.js?id=' + i + dl;
        f.parentNode.insertBefore(j, f);
    })(window, document, 'script', 'dataLayer', 'GTM-NVFPDWB');
  </script>
  <!-- End Google Tag Manager -->
  <title>
   Web Scraper Test Sites
  </title>
  <meta charset="utf-8"/>
  <meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/>
  <meta content="web scraping,Web Scraper,Chrome extension,Crawling,Cross platform scraper" name="keywords">
   <meta content="The most popular web scraping extension. Start scraping in minutes. Automate your tasks with our Cloud Scraper. No software to download, no coding needed." name="description">
    <link href="/favicon.png" rel="icon" sizes="128x128"/>
    <meta content="width=device-width, initial-scale=1.0" name="viewport"/>
    <link href="/css/app.css?id=e4de8db16b64e604046e" rel="stylesheet"/>
    <link href="https://webscraper.io/test-sites/e-commerce/allinone" rel="canonical"/>
    <link href="/img/logo-icon.png" rel="apple-touch-icon"/>
    <script defer="" src="/js/app.js?id=e64f07a3ce1d466cf04c">
    </script>
   </meta>
  </meta>
 </head>
<!--- content truncated due to length -->

This might not make much sense, unless you come from a web development background or understand HTML, however, we do not need to concern ourselves with this section of the webpage just yet. The juicy parts come much later.
Notice how prettify() 'prettifies' the HTML, indenting it nicely and making it much easier to follow, understand and/or find patterns in snippets. Without the prettify() function, we would have a mishmash of tags and words that might not make much sense at all to us:

b'<!DOCTYPE html>\n<html lang="en">\n<head>\n\n\t\t\t<!-- Anti-flicker snippet (recommended)  -->\n<style>.async-hide {\n\t\topacity: 0 !important\n\t} </style>\n<script>(function (a, s, y, n, c, h, i, d, e) {\n\t\ts.className += \' \' + y;\n\t\th.start = 1 * new Date;\n\t\th.end = i = function () {\n\t\t\ts.className = s.className.replace(RegExp(\' ?\' + y), \'\')\n\t\t};\n\t\t(a[n] = a[n] || []).hide = h;\n\t\tsetTimeout(function () {\n\t\t\ti();\n\t\t\th.end = null\n\t\t}, c);\n\t\th.timeout = c;\n\t})(window, document.documentElement, \'async-hide\', \'dataLayer\', 4000,\n\t\t{\'GTM-NVFPDWB\': true});</script>\n\t\n\t<!-- Google Tag Manager -->\n<script>(function (w, d, s, l, i) {\n\t\tw[l] = w[l] || [];\n\t\tw[l].push({\n\t\t\t\'gtm.start\':\n\t\t\t\tnew Date().getTime(), event: \'gtm.js\'\n\t\t});\n\t\tvar f = d.getElementsByTagName(s)[0],\n\t\t\tj = d.createElement(s), dl = l != \'dataLayer\' ? \'&l=\' + l : \'\';\n\t\tj.async = true;\n\t\tj.src =\n\t\t\t\'https://www.googletagmanager.com/gtm.js?id=\' + i + dl;\n\t\tf.parentNode.insertBefore(j, f);\n\t})(window, document, \'script\', \'dataLayer\', \'GTM-NVFPDWB\');</script>\n<!-- End Google Tag Manager -->\n\t<title>Web Scraper Test Sites</title>\n\t<meta charset="utf-8">\n\t<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">\n\n\t<meta name="keywords"\n\t      content="web scraping,Web Scraper,Chrome extension,Crawling,Cross platform scraper"/>\n\t<meta name="description"\n\t      content="The most popular web scraping extension. Start scraping in minutes. Automate your tasks with our Cloud Scraper. No software to download, no coding needed."/>\n\t<link rel="icon" sizes="128x128" href="/favicon.png">\n\n\t<meta name="viewport" content="width=device-width, initial-scale=1.0">\n\t\n\t<link rel="stylesheet" href="/css/app.css?id=e4de8db16b64e604046e">\n\n\t<link rel="canonical" href="https://webscraper.io/test-sites/e-commerce/allinone">\n\t<link rel="apple-touch-icon" href="/img/logo-icon.png">\n\n\t\t<script defer src="/js/app.js?id=e64f07a3ce1d466cf04c"></script>\n\n\t\n</head>\n<body>\n<!-- Google Tag Manager (noscript) -->\n<noscript>\n\t<iframe src="https://www.googletagmanager.com/ns.html?id=GTM-NVFPDWB"\n\t        height="0" width="0" style="display:none;visibility:hidden"></iframe>\n</noscript>\n<!-- End Google Tag Manager (noscript) -->\n<header role="banner" class="navbar navbar-fixed-top navbar-static">\n\t<div class="container">\n\n\t\t<div class="navbar-header">\n\n\t\t\t<a data-toggle="collapse-side" data-target=".side-collapse" data-target-2=".side-collapse-container">\n\t\t\t\t<button type="button" class="navbar-toggle pull-right collapsed" data-toggle="collapse"\n\t\t\t\t        data-target="#navbar" data-target-2=".side-collapse-container" data-target-3=".side-collapse"\n\t\t\t\t        aria-expanded="false" aria-controls="navbar">\n\n\t\t\t\t\t<span class="sr-only">Toggle navigation</span>\n\t\t\t\t\t<span class="icon-bar top-bar"></span>\n\t\t\t\t\t<span class="icon-bar middle-bar"></span>\n\t\t\t\t\t<span class="icon-bar bottom-bar"></span>\n\n\t\t\t\t</button>\n\t\t\t</a>\n\t\t\t<div class="navbar-brand">\n\t\t\t\t<a href="/"><img src="/img/logo_white.svg" alt="Web Scraper"></a>\n\t\t\t</div>\n\t\t</div>\n\n\t\t<div class="side-collapse in">\n\t\t\t<nav id="navbar" role="navigation" class="navbar-collapse collapse">\n\t\t\t\t<ul class="nav navbar-nav navbar-right">\n\t\t\t\t\t<li class="hidden">\n\t\t\t\t\t\t<a href="#page-top"></a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/" class="menuitm">\n\t\t\t\t\t\t\t<p>Web Scraper</p>\n\t\t\t\t\t\t\t<div class="crta"></div>\n\t\t\t\t\t\t</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/cloud-scraper" class="menuitm">\n\t\t\t\t\t\t\t<p>Cloud Scraper</p>\n\t\t\t\t\t\t\t<div class="crta"></div>\n\t\t\t\t\t\t</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/pricing" class="menuitm">\n\t\t\t\t\t\t\t<p>Pricing</p>\n\t\t\t\t\t\t\t<div class="crta"></div>\n\t\t\t\t\t\t</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t<li class="dropdown">\n\t\t\t\t\t\t<a href="#section3" class="menuitm dropdown-toggle" data-toggle="dropdown">\n\t\t\t\t\t\t\t<p>Learn</p>\n\t\t\t\t\t\t\t<div class="crta"></div>\n\t\t\t\t\t\t</a>\n\t\t\t\t\t\t<ul class="dropdown-menu">\n\t\t\t\t\t\t\t<li>\n\t\t\t\t\t\t\t\t<a href="/documentation">Documentation</a>\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t<li>\n\t\t\t\t\t\t\t\t<a href="/tutorials">Video Tutorials</a>\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t<li>\n\t\t\t\t\t\t\t\t<a href="/how-to-videos">How to</a>\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t<li>\n\t\t\t\t\t\t\t\t<a href="/test-sites">Test Sites</a>\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t\t<li>\n\t\t\t\t\t\t\t\t<a href="https://forum.webscraper.io/" target="_blank" rel="noopener">Forum</a>\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t</ul>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a target="_blank" href="https://chrome.google.com/webstore/detail/web-scraper/jnhgnonknehpejjnehehllkliplmbmhn?hl=en" class="btn-menu1 install-extension" rel="noopener">Install</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="https://cloud.webscraper.io/" class="btn-menu2">Login</a>\n\t\t\t\t\t</li>\n\t\t\t\t</ul>\n\t\t\t</nav>\n\t\t</div>\n\t</div>\n</header>\n\n<div class="wrapper">\n\t\t<div class="formenu-here container-fluid">\n\n\t</div>\n\t<div class="container-fluid blog-hero">\n\t\t<div class="container">\n\t\t\t<div class="row">\n\t\t\t\t<div class="col-md-12">\n\t\t\t\t\t<h1>Test Sites</h1>\n\t\t\t\t</div>\n\t\t\t</div>\n\t\t</div>\n\t</div>\n\n\t<div class="container test-site">\n\t\t<div class="row">\n\t\t\t<div class="col-md-3 sidebar">\n\t\t\t\t\t<div class="navbar-default sidebar" role="navigation">\n\t<div class="sidebar-nav navbar-collapse">\n\t\t<ul class="nav" id="side-menu">\n\n\t\t\t<li  class="active" >\n\t\t\t\t<a href="/test-sites/e-commerce/allinone">Home</a>\n\t\t\t</li>\n\n\t\t\t\t\t\t<li >\n\t\t\t\t<a href="/test-sites/e-commerce/allinone/computers" class="category-link ">\n\t\t\t\t\tComputers\n\t\t\t\t\t<span class="fa arrow"></span>\n\t\t\t\t</a>\n\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t\t<li >\n\t\t\t\t<a href="/test-sites/e-commerce/allinone/phones" class="category-link ">\n\t\t\t\t\tPhones\n\t\t\t\t\t<span class="fa arrow"></span>\n\t\t\t\t</a>\n\n\t\t\t\t\t\t\t</li>\n\t\t\t\t\t</ul>\n\t</div>\n</div>\n\n\t\t\t</div>\n\t\t\t<div class="col-md-9">\n\t\t\t\t\n\t<div class="jumbotron">\n\t\t<h1>E-commerce training site</h1>\n\t\t<p>\n\t\t\tWelcome to WebScraper e-commerce site. You can use this site for training\n\t\t\tto learn how to use the Web Scraper. Items listed here are not for sale.\n\t\t</p>\n\t</div>\n\n\t<h2>Top items being scraped right now</h2>\n\n\t<div class="row">\n\t\t\t<div class="col-sm-4 col-lg-4 col-md-4">\n\t<div class="thumbnail">\n\t\t<img class="img-responsive" alt="item"\n\t\t     src="/images/test-sites/e-commerce/items/cart2.png">\n\t\t<div class="caption">\n\t\t\t<h4 class="pull-right price">$809.00</h4>\n\t\t\t<h4>\n\t\t\t\t\t\t\t\t\t<a href="/test-sites/e-commerce/allinone/product/537" class="title" title="Acer Nitro 5 AN515-51">Acer Nitro 5 AN5...</a>\n\t\t\t\t\t\t\t</h4>\n\t\t\t<p class="description">Acer Nitro 5 AN515-51, 15.6&quot; FHD IPS, Core i5-7300HQ, 8GB, 1TB, GeForce GTX 1050 2GB, Windows 10 Home</p>\n\n\t\t</div>\n\t\t<div class="ratings">\n\t\t\t<p class="pull-right">0 reviews</p>\n\t\t\t<p data-rating="1">\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t</p>\n\t\t</div>\n\t</div>\n</div>\n\t\t\t<div class="col-sm-4 col-lg-4 col-md-4">\n\t<div class="thumbnail">\n\t\t<img class="img-responsive" alt="item"\n\t\t     src="/images/test-sites/e-commerce/items/cart2.png">\n\t\t<div class="caption">\n\t\t\t<h4 class="pull-right price">$1102.66</h4>\n\t\t\t<h4>\n\t\t\t\t\t\t\t\t\t<a href="/test-sites/e-commerce/allinone/product/584" class="title" title="Dell Latitude 5280">Dell Latitude 52...</a>\n\t\t\t\t\t\t\t</h4>\n\t\t\t<p class="description">Dell Latitude 5280, 12.5&quot; HD, Core i5-7300U, 8GB, 256GB SSD, Windows 10 Pro</p>\n\n\t\t</div>\n\t\t<div class="ratings">\n\t\t\t<p class="pull-right">8 reviews</p>\n\t\t\t<p data-rating="1">\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t</p>\n\t\t</div>\n\t</div>\n</div>\n\t\t\t<div class="col-sm-4 col-lg-4 col-md-4">\n\t<div class="thumbnail">\n\t\t<img class="img-responsive" alt="item"\n\t\t     src="/images/test-sites/e-commerce/items/cart2.png">\n\t\t<div class="caption">\n\t\t\t<h4 class="pull-right price">$399.99</h4>\n\t\t\t<h4>\n\t\t\t\t\t\t\t\t\t<a href="/test-sites/e-commerce/allinone/product/556" class="title" title="Asus VivoBook E502NA-GO022T Dark Blue">Asus VivoBook E5...</a>\n\t\t\t\t\t\t\t</h4>\n\t\t\t<p class="description">Asus VivoBook E502NA-GO022T Dark Blue, 15.6&quot; HD, Pentium N4200 1.1GHz, 4GB, 128GB SSD, Windows 10 Home, En/Ru kbd</p>\n\n\t\t</div>\n\t\t<div class="ratings">\n\t\t\t<p class="pull-right">3 reviews</p>\n\t\t\t<p data-rating="4">\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t\t<span class="glyphicon glyphicon-star"></span>\n\t\t\t\t\t\t\t</p>\n\t\t</div>\n\t</div>\n</div>\n\t</div>\n\n\n\t\t\t</div>\n\t\t</div>\n\t</div>\n\t<div class="clearfix"></div>\n\t<div class="push"></div>\n</div>\n\n<div class="container-fluid footer" id="layout-footer">\n\t<div class="container">\n\t\t<div class="row">\n\t\t\t<div class="col-md-3">\n\t\t\t\t<ul>\n\t\t\t\t\t<li><p>Products</p></li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/">Web Scraper browser extension</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/pricing">Web Scraper Cloud</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t</ul>\n\t\t\t</div>\n\t\t\t<div class="col-md-3">\n\t\t\t\t<ul>\n\t\t\t\t\t<li><p>Company</p></li>\n\t\t\t\t\t\n\t\t\t\t\t<li><a href="/contact">Contact</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/privacy-policy">Website Privacy Policy</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/extension-privacy-policy">Browser Extension Privacy Policy</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="http://webscraperio.us-east-1.elasticbeanstalk.com/downloads/Web_Scraper_Media_Kit.zip">Media kit</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\t\t\t\t\t<li><a href="/jobs">Jobs</a></li>\n\t\t\t\t</ul>\n\t\t\t</div>\n\t\t\t<div class="col-md-3">\n\t\t\t\t<ul>\n\t\t\t\t\t<li><p>Resources</p></li>\n\t\t\t\t\t<li><a href="/blog">Blog</a></li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/documentation">Documentation</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/tutorials">Video Tutorials</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/screenshots">Screenshots</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="/test-sites">Test Sites</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a target="_blank" href="https://forum.webscraper.io/" rel="noopener">Forum</a>\n\t\t\t\t\t</li>\n\t\t\t\t</ul>\n\t\t\t</div>\n\t\t\t<div class="col-md-3">\n\t\t\t\t<ul>\n\t\t\t\t\t<li><p>CONTACT US</p></li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="mailto:info@webscraper.io">info@webscraper.io</a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\t\t\t\t\t<li>Rupniecibas iela 30,<br> Riga, Latvia, LV-1045</li>\n\t\t\t\t</ul>\n\t\t\t\t<ul class="smedia">\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="https://www.facebook.com/webscraperio/" target="_blank" rel="noopener"><img src="/img/fbicon.png" alt="Web Scraper on Facebook"></a>\n\t\t\t\t\t</li>\n\t\t\t\t\t<li>\n\t\t\t\t\t\t<a href="https://twitter.com/webscraperio" target="_blank" rel="noopener"><img src="/img/twicon.png" alt="Web Scraper on Twitter"></a>\n\t\t\t\t\t</li>\n\t\t\t\t\t\n\n\t\t\t\t</ul>\n\t\t\t</div>\n\t\t</div>\n\t\t<div class="row">\n\t\t\t<div class="col-md-12">\n\t\t\t\t<p class="copyright">Copyright &copy 2021\n\t\t\t\t\t<a href="#">Web Scraper</a> | All rights\n\t\t\t\t\treserved | Made by zoom59</p>\n\t\t\t</div>\n\t\t</div>\n\t</div>\n</div>\n\n\n</body>\n</html>\n'
>>>

If you run the first script (and you should follow the exercises as you read!) you will be able to scroll down and see something similar to:

      <h2>
       Top items being scraped right now
      </h2>
       <div class="col-sm-4 col-lg-4 col-md-4">
        <div class="thumbnail">
         <img alt="item" class="img-responsive" src="/images/test-sites/e-commerce/items/cart2.png"/>
         <div class="caption">
          <h4 class="pull-right price">
           $1260.13
          </h4>
          <h4>
           <a class="title" href="/test-sites/e-commerce/allinone/product/616" title='Apple MacBook Air 13"'>
            Apple MacBook Ai...
           </a>
          </h4>
          <p class="description">
           Apple MacBook Air 13", i5 1.8GHz, 8GB, 256GB SSD, Intel HD 6000, ENG
          </p>
         </div>
         <div class="ratings">
          <p class="pull-right">
           8 reviews
          </p>
          <p data-rating="4">
           <span class="glyphicon glyphicon-star">
           </span>
           <span class="glyphicon glyphicon-star">
           </span>
           <span class="glyphicon glyphicon-star">
           </span>
           <span class="glyphicon glyphicon-star">
           </span>
          </p>
         </div>
        </div>

The header for this section of the webpage (designated "h2") clearly tells us: "Top items being scraped right now". This is obviously a play on "Top items being searched right now" or something related. What is the takeaway here? Websites and web documents often have "pointers" or cues that can sometimes tell us where to find the data we are looking for. Seeing "Top items being ordered right now" in an online store is a cue as good as any that tells us: "Here come the products!".

Another Serving of BeautifulSoup
So far we have gotten by with very limited vocabulary, and that's fine. However, the BeautifulSoup object provides a whole suite of methods and properties that make sifting through markup as easy and tasting soup. I will gradually introduce the most important of these attributes.

How might we extract the name of every character in http://www.pythonscraping.com/pages/warandpeace.html?
"Oh. I wouldn't know", you might say.
And you wouldn't be wrong. You might not always know right away how to extract the data you need, but you can certainly analyze the markup for the HTML web document and find a reliable pattern. While this is entirely possible using the first script above, there is another, faster, easier way that comes without the strain that staring intently at the default white screen of the Python IDLE editor. This method is by using the developer console that comes pre-packaged with most modern browsers. Most likely, in your the browser you are reading this article with, this developer markup inspector console can be accessed by right-clicking the text or media that you want to inspect, and clicking "Inspect". (Most browsers also allow use of the keyboard shortcut CTRL+SHIFT+I or CMD+SHIFT+I on Macs).

So, there are two ways for you to inspect the markup of the War and Peace webpage:

Using BeautifulSoup's prettify() method.
Using the "Inspect" option of your browser's Developer Tools console.

<h1>
   War and Peace
  </h1>
  <h2>
   Chapter 1
  </h2>
  <div id="text">
   "
   <span class="red">
    Well, Prince, so Genoa and Lucca are now just family estates of the
Buonapartes. But I warn you, if you don't tell me that this means war,
if you still try to defend the infamies and horrors perpetrated by
that Antichrist- I really believe he is Antichrist- I will have
nothing more to do with you and you are no longer my friend, no longer
my 'faithful slave,' as you call yourself! But how do you do? I see
I have frightened you- sit down and tell me all the news.
   </span>
   "
   <p>
   </p>
   It was in July, 1805, and the speaker was the well-known
   <span class="green">
    Anna
Pavlovna Scherer
   </span>
   , maid of honor and favorite of the
   <span class="green">
    Empress Marya
Fedorovna
   </span>

Carefully analyzing the War and Peace markup might reveal an interesting pattern:

Quoted text is always found within a pair of span tags with a class attribute of "red".
Character name are always found within a pair of span tags with a class attribute of "green".

Didn't notice it? Look again.

In fact, if you simply navigate to the URL with your browser, you will be met with a screen similar to:

You will be able to easy spot the pattern: quoted text in red, character names in green.

Now that you have spotted a pattern in your data, it is time to design an algorithm for extracting the information you want.

Algorithms

What Are Algorithms?

And, more importantly, can we avoid using them?

Because they are a cross-breed of unfriendly Mathematics and hardcore computer programming, right?
Right?
Wrong.

While many students of programming fear algorithms, it is quite impossible to do any form of impactful programming without an implementation of some algorithm.

Why?
The reason lies in the fact that, underneath all the hype computer systems get from the media (such as the hype around self-driving cars), computer programs (and computers generally) are really dumb things! Sometime in the future, our programs might evolve to the point where they are able to parallel human intelligence, but that future is still so far away. We can catch glimpses of such a future in movies like The Matrix, but this is about all we can do.
In order for a computer program to solve a problem, it has to follow a series of steps and logical thought-flows. These steps or flows of thought that lead a computer program from the beginning of its execution to its arrival at a solution (even the wrong solution) is known as algorithms.

Algorithms: Where Computer Science meets Mathematics?
No. Algorithms aren't the product of a collision between Mathematics and any form of Computer Science. That is not how it works and the misconceptions that exist around the A-word are largely to blame for the fear many students of computer programming have for algorithms.
As I tried to explain earlier, an algorithm is a logical sequence of steps to take (aka subtasks to complete) in order to arrive at the solution to a problem.
You may think of algorithms as a "trail" of rewards that you lay on the ground in order to coax a pet to follow you.

If you need a more extensive introduction to algorithms, I recommend this article at HowStuffWorks.
For professional-level in-depth study of computer algorithms, I recommend the following books, in increasing order of difficulty:

Fundamentals of Computer Algorithms, Horowitz and Sahani.
Computational Thinking - First Algorithms, Then Code, * Fabrizio Luccio and Paolo Ferragina*.
Graphs, Networks and Algorithms (Algorithms and Computation In Mathematics), Dieter Jungnickel.
The Art Of Computer Programming, Donald Knuth.

Why Are Algorithms Important To Productive Web Scraping?
First, programming software is a tool for problem-solving,, nore than anything else. And web scrapig happens to solve a very ubiquitous, very troublesome problem: how to extract structured data in an automated and timely manner from otherwise inaccessible data sources.
The experienced programmer understands that "coding" is only about a fifth of his job. Programming is a skill, a tool, for solving problems. There is no other purpose. This reason is sufficient.
In the case of War and Peace, our problem is extracting the name of every character in the data source. Our problem is compounded by the fact that there exist several different forms of data in the webpage. How does one extract data from data? By finding patterns that enable us to define a structure, an expectation of sanity, amidst the larger data.
This, by itself, is one step of the algorithm.

We do not need to delve into the intricacies of computational algorithm design to solve our problem.
We can "break" down our task into subtasks based on the pattern we have found for the information we need:

Request the webpage and store the response.
Parse the response and create a BeautifulSoup object of it.
Find all occurrences of text between span tags that have the class="green" attribute and store them.
Iterate through the occurrences, extracting the text for each span occurrence and printing it.

Congratulations! You have just written your first(?) web scraping algorithm!!

Implementing The Algorithm
Now that we have a thorough understanding of our problem, its peculiarities and requirements, as well as an algorithm for arriving at a solution, we are finally ready to write some code:

# 2.py
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = "http://www.pythonscraping.com/pages/warandpeace.html"

html = urlopen(url)
bsoup = BeautifulSoup(html.read(), 'html.parser')

characters = bsoup.findAll('span', {'class': 'green'})
for character in characters:
    print(character.get_text())

Executing this script will give output similar to:

Anna
Pavlovna Scherer
Empress Marya
Fedorovna
Prince Vasili Kuragin
Anna Pavlovna
St. Petersburg
the prince
Anna Pavlovna

Let's work through our scraper and analyze what each line does.
After importing urlopen and BeautifulSoup from their modules (a module is like a package that contains function that perform related operations), we assign the string-value of the War and Peace URL to a variable (url) (line 3). We do this because we want to be able to simply swap URLs in future, modify the scraper a little bit and re-use it for a similar scraping task.
In line 5, we create our bsoup object and parse the network response from the URL we made our request to in line 3.
line 6 is where things start to get interesting. The findAll() method locates all occurrences of tags matching our specifications (tag name: span, class attribute: green) and appends each occurence to a list (characters).
Having located all of our targets, we simply iterate through each element of our characters list and extract the text between the span tags using the get_text() method.

Don't let all this talk of methods confuse you. Method is just another name for function, and a function is just a bit of code (called snippet in programming jargon) that performs a specific action.

Easy as pie, right?

In fact, we can take our scraper a bit further and instruct it to print serial numbers for the character names, as well as a count of character names. Modifying our algorithm shouldn't be too much hassle.

Request the webpage and store the response.
Parse the response and create a BeautifulSoup object of it.
Find all occurrences of text between span tags that have the class="green" attribute and store them.
Iterate through the occurrences, extracting the text for each span occurrence and printing it with a serial number attached.
Print a count of all occurrences of character names.

We can now modify our scraper thus:

# 3.py

from urllib.request import urlopen
from bs4 import BeautifulSoup

url = "http://www.pythonscraping.com/pages/warandpeace.html"

html = urlopen(url)
bsoup = BeautifulSoup(html.read(), 'html.parser')
characters = bsoup.findAll('span', {'class': 'green'})

for i in range(len(characters)):
    print("{}: {}".format(i + 1, characters[i].get_text()))

print("========")
print("The count is: {}".format(len(characters)))

(If you have problems understanding the complicated print statement above, you might need to study Python's basic syntax and data structures. There are recommendations for publications at the end of this article.)

The find() and findAll() Methods: Maximizing Your Ability To Extract Information
In your work scraping the web for data, the two functions you are likely to use again and again are the find() and findAll() BeautifulSoup methods. Therefore, a thorough understanding of the methods, their parameters and usage is important to maximizing your ability to extract the information you need from web sources.

The findAll() method
The findAll() method is a very special method because it's one you are likely to use in all of your scrapers.
The findAll() function takes six (6) arguments (you'll see what arguments are in a bit, so don't worry if you don't understand the word):

bsoup.findAll(tag, attributes, recursive, text, keywords)

A brief aside: Arguments

arguments are simply values that you pass to functions when you call them so that the function can use your values to do something. In the case of our web scraper, here is a (non-encompassing) list of functions that we called with arguments:

# 1
urlopen(url)
# 2
BeautifulSoup(html.read(), 'html.parser')
# 3
findAll('span', {'class': 'green'})
# 4
len(characters)
# 5
format(i + 1, characters[i].get_text())
# 6
format(len(characters))
# 7
print("========")
# 8
print(f"The count is: {len(characters)}")

And here's a list of functions we called without any arguments:

# 1
read()
# 2
get_text()

You must have noticed that the functions we called with arguments have values passed to them, within their parentheses. these values are separated from each other by commas. On the other hand, the functions that we called without arguments do not have any values between their parantheses.

Detailed information about arguments can be found in the recommended publications for learning Python at the end of this article.

The first and second arguments of the findAll() method are the ones you are most likely to use. They are the tag and attribute arguments. In our scraper (3.py) we used them thus:

...
characters = bsoup.findAll('span', {'class': 'green'})
...

This means that we asked BeautifulSoup to find all all occurrences of span tags that have a class attribute of green.
This matches all tags that look like:

<span class="green">...</span>

Simple, right?
But, you might wonder, 'what other cool stuff can I do with this findAll() function? Besides targeting tags based on their name and attributes?'

More Cool Stuff With findAll()
The format of arguments to findAll() is:

findAll(tag, attributes, recursive, text, limit, keywords)

I'll discuss each argument and show you examples of what you can do with each.

tag: You have already seen how you can pass a tag's name as a string as this argument.

# find all images in a webpage
findAll('img')
# find all sections in a webpage
findAll('section')
# find all videos in a webpage
findAll('video')

But what happens if you need to find multiple tags that have different tag names (and possibly share the same attribute)?

# passing multiple tag names in a list will find them all:
# this line will find all images <img>, videos <video> and paragraphs <p> in a webpage
findAll(['img', 'video', 'p'])

What if we only want our scraper to find images and videos that have a class attribute of 'multimedia'?

# this line uses the first two arguments to findAll(): 'tag' and 'attribute'
findAll(['img', 'video'], {'class': 'multimedia'})

attribute: The attribute argument is the second argument to findAll(). HTML elements can have several different attributes, so it is important to be certain that the attributes you are targeting will not return unwanted elements. We will see how easily this can happen. Given the HTML:

<div class="product featured" data-featured-product="pr/123456">
    <img src="some_long_path_to_an_image_of_a_cup" alt="product:123456" />
    <div class="product-name">Gold-plated Cups!</div>
</div>
<div class="product last-featured">
    <img src="some_long_path_to_an_image_of_a_spoon" alt="product:123457" />
    <div class="product-name">Gold-plated Spoons</div>
</div>
<div class="product">
    <img src="some_long_path_to_an_image_of_a_doorknob" alt="product:123456" />
    <div class="product-name">Gold-plated Doorknobs</div>
</div>

This is an extract from a (fictional) e-commerce site. What we want to do is retrieve every item for sale and store in a database. The algorithm for the retrieval part of our scraper might look like:

Request URL and save response.
Find all div tags that have a class attribute of 'product-name'.
Iterate through each ta and get its text. Short, simple and effective! Let's get to the programming bit:

# 4.py
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = "our_fictional_url"

html = urlopen(url)
bsoup = BeautifulSoup(html.read(), 'html.parser')
products = bsoup.findAll('div', {'class': 'product-name'})

for product in products:
    product_name = product.get_text()
    # save product_name to database
    # we'll learn how to save data to databases soon

This is good and fine. But what if we want to get the name of the featured product for the day, and possibly have our script start itself every morning to get the name of the featured product of the day?
You might not have noticed it, but the three products in our HTML are not exactly alike. The first product has an extra class value: 'featured'. The second has another class: 'last-featured'. The third only has one class.
Bases on the extra information suddenly available to you, you can modify your scraper to get the featured product and print it (or save it to a file, which is what we will now do). Since we are dealing with a fictional

# 5.py
from urllib.request import urlopen
from bs4 import BeautifulSoup
# we'll use the datetime module to get today's date
from datetime import date

url = "our_fictional_url"

html = urlopen(url)

bsoup = BeautifulSoup(html.read(), 'html.parser')
products = bsoup.findAll('div', {'class': {'product-name', 'featured'}})

for product in products:
    product_name = product.get_text()
    # save product to a file
    with open('featured-products-file.txt', 'a+') as f:
        msg = "{}: The featured product is {}\n".format(date.today(), product_name)
        f.write(msg)
print("File saved.")

(If you try to run this scraper, you'll get a ValueError. This is because our_fictional_url is not a valid URL! Although this script will not run, you can easily understand the logic behind it, as well as how effective it would be if the URL pointed to a valid resource.)

Practice Exercise 1:
Write a scraper for retrieving the last-featured product, and storing it to a file.

recursive: The recursive argument is pretty straightforward. If it is set to true, and it is by default, the findAll() function retrieves all tags that match the specification, whether they are nested or not. You'll likely always want to set this to true, and since it's true by default, you can forget the argument even exists for the remainder of this article. If set to false, the findAll() function only retrieves top-level tags. No nested tags will be returned.

If you do not have a background in web development, I recommend a quick primer course in HTML. John Duckett's 'HTML And CSS' is the best book I know for learning the language.

text: The text argument is used to target elements that contain the specified text. It's a handy argument to have.

<div>
<span>English</span>
<span>French</span>
<span>Latin</span>
<span>English</span>
<span>German</span>
</div>

Find the number of occurrences of the word 'English' in the document.
One thing you will notice about this markup is that there are six elements, all <span> tags, with no attributes. We cannot count on classes here.

But we can count on text.

...
occurrences = findAll(text="English")
print(f"The number of times the word 'English' occurs: {len(occurrences)}")
...

This script is the exact same thing as:

...
occurrences = findAll('span', text="English")
print(f"The number of times the word 'English' occurs: {len(occurrences)}")
...

Can you figure out why?

limit: The limit argument is simple: it limits the number of tags the findAll() function to the value specified.

# 6.py
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = 'http://www.pythonscraping.com/pages/warandpeace.html'
html = urlopen(url)
bsoup = BeautifulSoup(html.read(), 'html.parser')

characters = bsoup.findAll('span')
for character in characters:
    print(character.get_text())

This prints text from the first two <span> tags (there are 75 <span> tags). We can verify that we are actually printing the first two <span> tags by calling BeautifulSoup's prettify() method.

# 7.py
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = 'http://www.pythonscraping.com/pages/warandpeace.html'
html = urlopen(url)
bsoup = BeautifulSoup(html.read(), 'html.parser')

characters = bsoup.findAll('span')
# prettify
print(characters.prettify())

The find() method is the same as calling the findAll() method with limit set to 1

# retrieve the first <span> element
...
bsoup = BeautifulSoup(html.read(), 'html.parser')

# 1
characters = findAll('span', limit=1)
# is the same as:
# 2
characters = find('span')

Introduction To Web Scraping With Python: Part 1

Icheka Ozuru — Sun, 10 Jan 2021 21:52:03 +0000

INTRODUCTION TO WEB SCRAPING WITH PYTHON: A four-week beginner-to-advanced-level course in web scraping using the Python programming language.

“If programming is magic, then web scraping is surely a form of wizardry.”

Web Scraping With Python; Ryan Mitchell.

To those who have not developed the skill, computer programming can seem like a kind of magic. If programming is magic, web scraping is wizardry: the application of magic for particularly impressive and useful – yet surprisingly effortless – feats.

What is Web Scraping?
In the past twenty years, there has been an explosion of data – volume and availability – that has been matched only by the explosion that the Internet itself has experienced. Data Scientists estimate that in the last three years, more data has been generated, stored and computed, than has been in the rest of history. In fact, in 2013 IBM (a company that made some of the earliest personal computers) estimated that 90% of data available on the Internet had been generated within the last two years. This surely gives us a surprisingly accurate perspective on the exponential growth curve of data available over the Internet.
This vast amount of data is what makes it possible to have such internet applications as payment gateways and social media, that power most emerging businesses (i.e organisations formed within the last thirty years) and even many established ones (businesses birthed more than thirty years ago). Really, it can be said that data powers the web and drives the Internet.

Where does all the data go?

Data that has been collected has to be stored somewhere so that it can be accessed again, retrieved, compared, updated, manipulated and shared. For example, an automobile company might collect some personal data about their customers for generating mailing lists that will be used to share information about their future self-driving cars. This data can come from a variety of sources: a HTML form on their official website, detachable cards on their brochures, forms that their customers fill during purchases, or even data from advertising agencies!
This data needs to be stored somewhere so that it can be retrieved at a later date and used or updated.
Most data collected over the web (the web is short for World Wide Web, which is a vast network of computers and computerized devices that are connected over the Internet) is stored on special computers called servers and servers aren’t difficult to understand. Basically, all a server does is wait until a request is made for a file, then it locates that file, if it exists, and sends it to whomever made the request.
At the heart of data handling is the need to retrieve data. This is where web scrapers come in.

So, what are web scrapers?

A web scraper is a program that is designed to load a web document (remember that data is stored on servers in the form of documents; when these documents are accessed over the web, they are called web documents), parse it (a fancy word that means “to understand”), and extract the needed information from it. Before the end of this article, you would have written several web scrapers.

Why learn to scrape the web?

There are numerous ways to access data over the web and, if we’re being fair, web scraping is only one of them. Data can be retrieved from APIs (Application Programming Interfaces, which I’ll write about soon), manipulated with SDKs (Software Development Kits), visualized with macros, and generally accessed (and assessed) in at least ten different ways.
So why web scrapers?
Firstly, all of the special software listed above (APIs, SDKs and macros) are higher-level programs that require special programming skills to develop adequately, and they may not always be available. For example, a college student who starts a free WordPress blog might not have the finances required for hiring a software developer to develop APIs for her audience. This is where web scrapers step in: that gap that so often exists between where data is stored and the people who need to access it. Generally, almost every data or document that can be viewed in a browser can be retrieved with a scraper. In fact, web scrapers can be designed to log-in to your social media page and retrieve data (like how many likes you have or who just DM-ed you), even while you are asleep! Really advanced web scrapers are used by social media agencies to keep track of their clients’ social accounts and automate many actions like “liking”, commenting or replying messages.

Industries where web scraping knowledge can be applied:
What makes knowledge of web scraping such an in-demand skill is how quickly it can be used to solve a wide range of data problems in a wide range of industries, even in industries where software isn't the Number One tool.
Some industries that benefit from web scraping are:

Data Science/Data Engineering: Web scraping is often used to mine very useful data that can be analyzed for patterns and used to build much larger applications or to solve problems.
Information Security: With the vast amounts of information available over the web, and the attendant need to restrict access to private information from the public, Information Security is a fast-growing industry. It has its foundations in cyber security and data engineering, and web scraping is definitely an important skill in any company in this industry.
Web Development: Designing and developing software for the web is known as web development. Websites, web apps and server-side web software are often developed around a central store of data that can be retrieved from external sources using web scrapers. For example, quite recently, I had to develop a real estate listing web app for a client. I was given three websites to poll (that is, repeatedly assess) and cull data from. The data I culled would be saved to my client's database and used to furnish his own website using APIs. While this must have been a difficult task for my competitors to accomplish (we had to pitch to be awarded the contract), I immediately saw the opportunity for three scrapers that would get the job done. In fact, using Selenium one of my scrapers was able to automatically (without my help) sign in to a website, navigate to a listings page, search for the required search text in the search field, extract the data and finally save the data to the database! Pretty amazing work for a program, right?!
Advertising and Media: Web scrapers help advertising/media agencies to obtain data that can be run through data analysis, during which important information can be extracted. Information gotten by this method is often necessary for the critical decisions that must be taken in such companies.
Security Firms, Militaries and Intelligence: By this point you can probably list a few applications of web scrapers in companies that specialize in personal/properties security, militaries and intelligence organisations.

Your first web scraper.

Now that you know so much about scrapers, what they are and why knowledge of how to write one is important, it is time to write your first scraper.
Step 1: Install Python and set up your programming environment.
The programming language we will be using to write our web scrapers is called Python. Python is one of the most popular, most versatile, programming languages in the world. It has been used to develop hundreds of thousands of computer applications, and today we will add yours to this figure!
To install Python on Windows or Mac:

Go to https://downloads.python.org
Choose the write installation file for your computer (i.e Windows or Mac). Be sure to download the latest version (at the time of writing this article, the latest version is version 3.9.1).
Twiddle your thumbs and congratulate yourself in advance for writing your first scraper while the installation file is being downloaded.
Once the download is completed, double-click on it to run the installation. This shouldn’t take more than ten minutes.
You are now ready to start programming in Python.

To install Python on Linux:
If you run Ubuntu or a similar Linux distro, you can install the latest version of Python from the Ubuntu Software Center (also called Synaptics on some distros).

In your app drawer, search for ‘Software Center’ or ‘Synaptics’ and run the app.
In the search field, type ‘Python’ and press the Enter key. Your computer must be connected to the Internet.
Click the Python software package and install it.
After Python is installed, you should see the Python icon added to the desktop or the app drawer. Python is now installed and ready to be used.
In the Ubuntu Software Center (or Synaptics), search for, and install, ‘Python IDLE’. Windows and Mac users will have the IDLE installed automatically when they run their installation files.
You are now ready to continue.

Step 2: Start the Python IDLE, click the 'File' button in the menu tab and select 'New File'.

from urllib.request import urlopen
html = urlopen("https://google.com")
print(html.read())

Save this file (CTRL + S, or CMD + S on Macs) as firstscraper.py (you can even save your program in a folder and build a collection of similar scripts by saving them all to the same folder).

Step 3: In the menu tab (of the Python IDLE where you wrote your scraper), click 'Run', then 'Run Module'. Your computer must be connected to the Internet for this awesome step to work. Wait a few seconds (how long depends on your network speed) and... you should see a load of gibberish miraculously fill your screen.

Congratulations! You have successfully written your first scraper!!!

Now we will analyze our code and understand exactly what we have written and what all that gibberish means. Feel free to pause here, stretch, yawn, and sip some soda water because, soon, you will become a badass Python programmer!

Understanding network requests: what goes on under the hood of our scrapers?
At the heart of every scraper is the network request. An understanding of networks and network requests is important for understanding web scrapers. Without this knowledge, we will not be able to write really practical web scrapers or similar advanced software (like a social media bot). Fortunately, networks and network requests aren't difficult to understand at all.

A computer network is like a spider web. It is simply a connection between two or more computer devices. The Internet is the largest and most popular of them all. It is an enormous, gigantic, connection of millions (maybe billions) of computer devices that range in size and purpose from tiny, invisible-to-the-naked-eye nano-computers used in medical research, to IoT (IoT stands for Internet of Things. Go figure!) devices like the special computers that opens the doors at the supermarket when a person steps near them to more common computer devices like smartphones and personal computers. When you ran (that is, executed) your scraper, your program became a part of this global, massive network for a few seconds! Impressive, isn't it?
Other examples of computer networks are WiFi's, paired bluetooth devices, and the infrared connection between your TV and its remote.

So, what is a network request?
Given that data is stored on servers and data can be accessed as web documents, there is only one small left to know about the data-retrieval process: network requests.
A network request is simply a request for a web document that you or the programs you write make to the server for the data it stores.

The second line of our scraper script:

html = urlopen("https://google.com")

makes a network request to Google's homepage and stores the response in the 'html' variable (a variable is a temporary storage location in the computer's memory; you can find out more on variables on the Internet).

Anatomy of a network request

A computer (or browser or program) called the 'user agent' makes a request to the server for a document.
The server checks to see if it has the document in storage.
If it does, it returns the document to the user agent. This is called a network response. (So a network response is the server's reply to a network request.)
If the file does not exist on the server, a '404 HTTP status code' is returned as the network response. We will learn more about status codes in general, and the 404 code in particular, in this series.

Understanding the rest of our scraper
We have seen the meaning of our second line in the context of network requests and network responses. The first and second lines are much easier to understand, and are integral to almost every scraper that we will write:

from urllib.request import urlopen

This line simply imports 'urlopen' method of the 'request' object of the 'urllib' module. This might not make sense now, and if it doesn't all you need to do is take a course in foundational Python programming. Some really amazing courses can be found online on Udemy, W3Schools and YouTube. Moreover, you can read some textbooks or join a Python programming bootcamp (my personal recommendation).

print(html.read())

The third line merely 'reads' the network response (Google's homepage) as a web document (this particular web document was saved as a HTML file) and 'prints' it by displaying its contents.

There's all there is to our first web scraper. Next, we will introduce the amazing BeautifulSoup library and begin to improve our web scraping abilities. Do feel free to take five, stretch your muscles, eat some cereals and generally congratulate yourself for a job well done!

Introducing BeautifulSoup

Beautiful Soup, so rich and green,
Waiting in a hot tureen!
Who for such dainties would not stoop?
Soup of the evening, beautiful Soup!

The BeautifulSoup library was named after a poem by Lewis Caroll in Alice's Adventures In Wonderland.
Like its name, the BeautifulSoup library converts a mish-mash of ingredients (web documents and incompatible file formats, encodings, and sources) into something edible, even appreciable. Yum yum!

Installing BeautifulSoup
The BeautifulSoup library is not packaged by default with Python, unlike the urllib library, so it must be installed before it can be imported and used to make awesome scrapers. For most computers, we will need to install pip, a special Python software that we will finally use to install that most interesting of Soups.
For Windows systems
Versions of Python later than 3.4 usually install pip by default during the installation. You might already have pip installed, especially if you followed the Python installation steps at the beginning of this article.

First, verify that you don't have pip installed. Open Command Prompt and enter:

pip --help

If the output begins with "Usage: pip [options], you have pip installed.
Else,

Enter the following:

python get-pip.py

This should install pip for you.

For Macs
If pip isn't already installed (which you can verify by running:

pip --help

in the Terminal), running

sudo easy_install pip

should get it done.
For Linux
Nothing for you here. Skip to the next section.

Finally, to install BeautifulSoup
On Windows and Macs
In the Terminal/Command Prompt, run:

pip install beautifulsoup4

This will take a few seconds to download and install BeautifulSoup.

On Linux
In the Terminal, run:

sudo apt-get install python-bs4

There. You are now ready to explore BeautifulSoup and become a master web scraper. Wait. That came out wrong. Web scrapist? If the software that scrapes the web is called a 'web scraper', what is the person who writes the web scraper called?

(It is highly likely that you will run into problems installing pip or BeautifulSoup. If errors occur or issues arise, please don't hesitate to reach out to me via email @ rhemafortune@gmail.com).

A More Advanced Web Scraper Using BeautifulSoup

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://pythonscraping.com/pages/page1.html")
bsoup = BeautifulSoup(html.read(), 'html.parser')
print(bsoup.h1)

Write this in a new file (remember: Click 'File' > 'New File') and save it. Then run it by clicking 'Run' > 'Run Module'. After a few moments (during which the program will be making a network request for the document) you should see output similar to:

<h1>An Interesting Title</h1>

What this advanced scraper does is to make a network request, save the network response (to the html variable), "parse" it into an object format (which is what Python can really understand), and print the first h1 tag it finds.
You can view the entire web document in the network response by modifying your code:

from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen("http://pythonscraping.com/pages/page1.html")
bsoup = BeautifulSoup(html.read(), 'html.parser')
print(bsoup)

Notice the absence of h1 in the print statement. This will print the entire BeautifulSoup object (stored in bsoup) and you should see output similar to:

<html>
    <head>
        <title>A Useful Page</title>
    </head>
    <body>
        <h1>An Interesting Title</h1>
        <div>
            Lorem ipsum dolor sit amet, consectetur adipisicing             
            elit, sed do eiusmod tempor incididunt ut labore et 
            dolore magna aliqua. Ut enim ad minim veniam, quis 
            nostrud exercitation ullamco laboris nisi ut aliquip 
            ex ea commodo consequat. Duis aute irure dolor in 
            reprehenderit in voluptate velit esse cillum dolore eu 
            fugiat nulla pariatur. Excepteur sint occaecat 
            cupidatat non proident, sunt in culpa qui officia 
            deserunt mollit anim id est laborum.
         </div>
    </body>
</html>

And changing the last line to print(bsoup.div) should print:

<div>
            Lorem ipsum dolor sit amet, consectetur adipisicing             
            elit, sed do eiusmod tempor incididunt ut labore et 
            dolore magna aliqua. Ut enim ad minim veniam, quis 
            nostrud exercitation ullamco laboris nisi ut aliquip 
            ex ea commodo consequat. Duis aute irure dolor in 
            reprehenderit in voluptate velit esse cillum dolore eu 
            fugiat nulla pariatur. Excepteur sint occaecat 
            cupidatat non proident, sunt in culpa qui officia 
            deserunt mollit anim id est laborum.
</div>

The BeautifulSoup method which creates the BeautifulSoup object bsoup takes two "arguments" in order to cook up our steamy soup. They are:

A document to be parsed.
A parser that should be used to parse the document. Your choice of the parser will always depend on the document format of the web document you want to scrape. For example, for scraping HTML web documents, the html.parser parser is used. In fact, 90% of the scrapers you write will use this parser, since HTML is the most ubiquitous document format on the web. You can play around with your scraper, make requests to different URLs (Uniform Resource Locator; aka website address), print all of your documents or just small section sections of them. In the next article we will more fully explore the BeautifulSoup library and write some really practical web scrapers that will extract practical information (like apartments available for sale in Lagos, Nigeria). We will learn the full capabilities of BeautifulSoup, its methods and arguments and the correct way to handle network requests that might arise when our script is trying to extract information. Finally, I will leave several juicy exercises that you can attempt and strengthen your knowledge of the wizardry that is web scraping.

Cheers!

Forem: Icheka Ozuru

Data Visualization Using Python

Intended Audience

What Is Data Visualization?

Data Visualization Using NumPy and Matplotlib

The Line Plot

Labeling Our Data and Resizing Our Graph

Resizing Our Plot

Making Our Plot More Readable Using Legends and Colors

The Scatter Plot

The Bar Plot (Finally)

Practical Object-Oriented JavaScript

Object-Oriented: What Does It Mean?

JavaScript Objects

Creating Re-usable Objects

Attributes of Objects: Properties and Member Functions

Performing Time-Of-Instantiation Tasks With Constructors

Learn Object-Oriented Programming In C++

Learn Object-Oriented Programming In C++

Objects As Structures

Why 'public'?

Extending Our Stone Class

Instantiation: making objects from classes

Object Constructors

Instantiation

Individualism Using Constructors

Final Section: Member Functions

A Gentle Introduction To Computer Algorithms: Implementing the Bubble Sort algorithm in six languages.

A Gentle Introduction To Computer Algorithms: Implementing the Bubble Sort algorithm in six languages (Python, JavaScript, C++, C, Java, PHP)

A Gentle Dive

To every self-taught programmer who reads this: you're one of the most inspiring people in the world. I appreciate and admire you. Keep up the good work!

What, Exactly, Is An Algorithm?

C

C file available on GitHub: here

C++

C++ file available on GitHub: here

Java

Java file available on GitHub: here

JavaScript

JavaScript code available on CodeSandbox: here

Also available on GitHub: here

PHP

PHP code available on GitHub: here

Python

Python code available on GitHub: here

Recommended Reading:

Coming Soon:

Recently Published:

Introduction To Web Scraping With Python: Part 2

This is article 2 of 4. If you have not read article 1, or if you do not have foundational knowledge web scraping with Python, please, read the first article.

What Are Algorithms?

And, more importantly, can we avoid using them?

A brief aside: Arguments

Introduction To Web Scraping With Python: Part 1