Forem: Dennis O'Keeffe

Recapping Kubernetes Basics

Dennis O'Keeffe — Fri, 01 Oct 2021 10:15:09 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

import { K8sControlPlaneQuiz } from "@quiz/index";

This post will recap some of the basics of Kubernetes to lay the foundations for upcoming posts.

The recap will include:

What does the Control Plane comprise of?
What is within a Node?
What is a Pod?
What is a Deployment
What is a Service?

Prerequisites

Basic familiarity with running Kubernetes on your local machine. This post is meant to recap some of basic concepts.

The Control Plane

The Control Plane manages, monitors, plans and schedules nodes on a Kubernetes cluster.

It is made up of the following:

App	Does
`etcd`	Key-value store for critical cluster info
Kube scheduler	Puts containers to proper nodes
`kubectl` manager	Ensures proper state of cluster components
Kube API Server	An exposed server that lets you communicate with the cluster

It is the Control Plane that brings the cluster to a desired state.

Anytime you want to make a change to the cluster, you need to make changes to the Control Plane. This is done by interacting with the master node via the Kubernetes API that the Kube API Server exposes. We can interact with this with kubectl.

What is within a Node?

You can use the acronym NPC to remember how the order of encapsulation works for a Node.

Letter	Meaning
N	Node
P	Pod
C	Container

A Node consists of a number of Pods which in turn consist of a number of Containers.

The Node needs the Container Runtime Engine (Docker, Containerd, CRI-O, frankti). It is the software required to run the images.

Something needs to run in the node to communicate with the Control Plane - that is the kubelet that runs in each node of the cluster.

As for node-to-node communication, this is done using kube proxy. The kube proxy is a network proxy that runs on each node in the cluster. It maintains network rules on nodes. These rules allow communication to your nodes from inside or outside your cluster.

What is a Pod?

Containers that we build are deployed to a Kubernetes object named a Pod. We cannot directly deploy a container to the cluster itself.

Pods are the smallest unit of execution in Kubernetes. They are composed of one or more containers.

Pods themselves are ephemeral by design. If a pod fails, the scheduler can schedule the creation of a new replica of that pod to continue operations.

Each Pod has a unique IP address that other Pods within the cluster can communicate with.

Apps can be scaled by spinning up more pods with another app container within it.

You can read more about Pods here.

What is a Deployment?

Deployments are a way to describe the desired state of your application.

The Deployment Controller changes the actual state to the desired state at a controlled rate.

You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.

Some use cases:

Create a Deployment to rollout a ReplicaSet. The ReplicaSet creates Pods in the background. Check the status of the rollout to see if it succeeds or not.
Declare the new state of the Pods by updating the PodTemplateSpec of the Deployment. A new ReplicaSet is created and the Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates the revision of the Deployment.
Rollback to an earlier Deployment revision if the current state of the Deployment is not stable. Each rollback updates the revision of the Deployment.
Scale up the Deployment to facilitate more load.
Pause the Deployment to apply multiple fixes to its PodTemplateSpec and then resume it to start a new rollout.
Use the status of the Deployment as an indicator that a rollout has stuck.
Clean up older ReplicaSets that you don't need anymore.

An example of a deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80

This deploy will create 3 replicated Pods containing the nginx container (as denoted by .spec.replicas).

The container itself is denoted at .spec.template.spec.containers[0].

The .spec.selector field defines how the Deployment finds which Pods to manage. In this specification, we select a label that is defined in the Pod template (app: nginx).

We could create this deployment on our cluster by running:

$ kubectl apply -f https://k8s.io/examples/controllers/nginx-deployment.yaml
$ kubectl get deployments
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   0/3     0            0           1s

A more sophisticated example:

# The deployment object
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  # Manages pods with label `app: nginx` and manages replicaset defined within
  labels:
    app: nginx
# Creates a replicaset of 3
spec:
  replicas: 3
  selector:
    # This replicaset will manage pods with label `app: nginx`
    matchLabels:
      app: nginx
    # This takes care of rolling updates
    minReadySeconds: 10
    strategy:
      rollingUpdate:
        maxSurge: 1
        # Update must be done in a way that at least 3 pods are running
        maxUnavailable: 0
  template:
    metadata:
      # Where the label is denoted for the app container
      labels:
        app: nginx
    # Creates a pod with app container nginx:1.14.2
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80

The comment throughout the YAML configuration explain how this configuration works.

What is a Service?

In an example where a pod goes down within a node and a ReplicaSet brings it back up, we know that the new pod will have a new IP address. How does our cluster know what is happening?

Services are a way to expose a set of Pods to the outside world by connecting the Pods using IP and connecting to a Service that will distribute traffic.

Each service has a name. In the example given, there might be a service that helps the web server pods connect to database pods. At the same time, there might be a frontend service that connects users to the web servers.

The service helps to discover new pods and helps with the distribution when there are changes in state.

How does it work? We tell it to manage any pods using a selector (for example app: frontend).

Service can be a few different types:

Type	Description
`ClusterIP`	Default kind of service and only accessible from within a cluster. This means if you are outside of the cluster and want to access a service, you cannot do it.
`NodePort`	Accessible from outside of the cluster and creates a cluster-wide port. It allows us to pick a port number and the combination of the node ID and node port can access the pods from outside of the cluster.
`LoadBalancer`	Cloud-specific implementation (AWS vs Google Cloud vs Azure)

In production, the LoadaBalancer type is the most common and NodePort is barely used in practice to export applications.

The LoadBalancer is access from outside the cluster, has a DNS name and also includes features such as SSL termination, Web Application Firewall (WAF) integration, access logs, health check etc.

Summary

Today's post recapped some of the important definitions and terms in Kubernetes. We will be referencing these all the time over the upcoming posts as we dive more into Kubernetes and EKS.

Resources and further reading

Photo credit: virussinside

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Oil Paint Effect With OpenCV Python

Dennis O'Keeffe — Mon, 27 Sep 2021 10:15:09 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 25 of the #100DaysOfPython challenge.

This post will use the OpenCV Python library to apply an oil painting effect to an image.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
An image to use with the OpenCV library.

Getting started

Let's create the oil-paint-effect-with-open-cv-python directory and install Pillow.

# Make the `oil-paint-effect-with-open-cv-python` directory
$ mkdir oil-paint-effect-with-open-cv-python
$ cd oil-paint-effect-with-open-cv-python
$ touch main.py

# Init the virtual environment
$ pipenv --three
$ pipenv install opencv-python opencv-contrib-python


# if you have issues with a hanging lockfile, try add the --skip-lock option

At this stage, you will need to add an image to the root of your directory. In my case, I will add base_img.jpg to the directory (which will be an image from Unsplash).

We are now ready to start coding!

Applying the oil painting effect

This section simply loads the image in var base_img (assuming you are following the directory structure where the notebook is in the docs folder).

import cv2
img = cv2.imread('./base_img.jpg')

Once that is complete, we can apply the oil paiting effect with one liner of code:

res = cv2.xphoto.oilPainting(img, 7, 1)

We can now compare by displaying the images:

cv2.imshow("original", img)
cv2.imshow("res", res)
cv2.waitKey(0)
cv2.destroyAllWindows()

This will display the images in a window.

The original image:

After applying the effect:

When you are finished with viewing, hit escape to exit.

Summary

Today's post demonstrated how to use the OpenCV package to programmatically apply an oil painting effect to an image.

Resources and further reading

Photo credit: dancristianp

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Generating Fake CSV Data With Python

Dennis O'Keeffe — Sat, 25 Sep 2021 10:15:12 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 24 of the #100DaysOfPython challenge.

This post will use the Faker library to generate fake data and export it to a CSV file.

We wil be emulating some of the free datasets from Kaggle, in particular the Netflix original films IMDB score to generate something similar.

The final code can be found here.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with JupyterLab. See here for my post on JupyterLab.

Getting started

Let's create the generating-fake-csv-data-with-python directory and install Pillow.

# Make the `generating-fake-csv-data-with-python` directory
$ mkdir generating-fake-csv-data-with-python
$ cd generating-fake-csv-data-with-python
# Create a folder to place your icons
$ mkdir docs

# Init the virtual environment
$ pipenv --three
$ pipenv install faker
$ pipenv install --dev jupyterlab

At this stage, we have the packages that we

Now we can start up the notebook server.

# Startup the notebook server
$ pipenv run jupyter-lab
# ... Server is now running on http://localhost:8888/lab

The server will now be up and running.

Creating the notebook

Once on http://localhost:8888/lab, select to create a new Python 3 notebook from the launcher.

Ensure that this notebook is saved in generating-fake-csv-data-with-python/docs/generating-fake-data.ipynb.

We will create four cells to handle four parts of this mini project:

Importing Faker and generating data.
Importing the CSV module and exporting the data to a CSV file.

Before generating our data, we need to look at what we are trying to emulate.

Emulating The Netflix Original Movies IMDB Scores Dataset

Looking at the preview for our dataset, we can see that it contains the following columns and example rows:

Title	Genre	Premiere	Runtime	IMDB Score	Language
Enter the Anime	Documentary	August 5, 2019	58	2.5	English/Japanese
Dark Forces	Thriller	August 21, 2020	81	2.6	Spanish

We only have two rows for example, but from here we can make a few assumptions about how we want to emulate it.

In our langauges, we will stick to a single language (unlike the example English/Japanese).
IMDB scores are between 1 and 5. We won't be too harsh on any movies and go from 0.
Runtimes should emulate a real movie - we can set it to be between 50 and 150 minutes.
Genres may be something we need to write our own Faker provider for.
We are going to be okay with non-sense data, so we can just use a string generator for the names.

With this said, let's look at how we can fake this.

Emulating a value for each column

We will create seven cells - one to import Faker and one for each column.

For the first cell, we will import Faker.

from faker import Faker

fake = Faker()

Secondard, we will fake a movie name with words:

def capitalize(str):
    return str.capitalize()
words = fake.words()
capitalized_words = list(map(capitalize, words))
movie_name = ' '.join(capitalized_words)
print(movie_name) # Serve Fear Consider

Third, we will generate a date this decate and use the same format as the example:

from datetime import datetime

date = datetime.strftime(fake.date_time_this_decade(), "%B %d, %Y")
print(date) # April 30, 2020

Fourth, we will create our own fake data geneartor for the genre:

# creating a provider for genre
from faker.providers import BaseProvider
import random

# create new provider class
class GenereProvider(BaseProvider):
    def movie_genre(self):
        return random.choice(['Documentary', 'Thriller', 'Mystery', 'Horror', 'Action', 'Comedy', 'Drama', 'Romance'])

# then add new provider to faker instance
fake.add_provider(GenereProvider)

# now you can use:
movie_genre = fake.movie_genre()
print(movie_genre) # Horror

Fifth, we will do the same for a language:

# creating a provider for genre
from faker.providers import BaseProvider
import random

# create new provider class
class LanguageProvider(BaseProvider):
    def language(self):
        return random.choice(['English', 'Chinese', 'Italian', 'Spanish', 'Hindi', 'Japanese'])

# then add new provider to faker instance
fake.add_provider(LanguageProvider)

# now you can use:
language = fake.language()
print(language) # Spanish

Sixth we need to generate a runtime:

# Getting random movie length
movie_len = random.randrange(50, 150)
print(movie_len) # 143

Lastly, we need a rating with one decimal point between 1.0 and 5.0:

# Movie rating
random_rating = round(random.uniform(1.0, 5.0), 1)
print(random_rating) # 2.2

Now that we have all our information together, it is time to generate a CSV with 100 entries.

Generating the CSV

We can place everything we know into a last cell to generate some data:

from faker import Faker
from faker.providers import BaseProvider
import random
import csv

class GenereProvider(BaseProvider):
    def movie_genre(self):
        return random.choice(['Documentary', 'Thriller', 'Mystery', 'Horror', 'Action', 'Comedy', 'Drama', 'Romance'])

class LanguageProvider(BaseProvider):
    def language(self):
        return random.choice(['English', 'Chinese', 'Italian', 'Spanish', 'Hindi', 'Japanese'])

fake = Faker()

fake.add_provider(GenereProvider)
fake.add_provider(LanguageProvider)

# Some of this is a bit verbose now, but doing so for the sake of completion

def get_movie_name():
    words = fake.words()
    capitalized_words = list(map(capitalize, words))
    return ' '.join(capitalized_words)

def get_movie_date():
    return datetime.strftime(fake.date_time_this_decade(), "%B %d, %Y")

def get_movie_len():
    return random.randrange(50, 150)

def get_movie_rating():
    return round(random.uniform(1.0, 5.0), 1)

def generate_movie():
    return [get_movie_name(), fake.movie_genre(), get_movie_date(), get_movie_len(), get_movie_rating(), fake.language()]

with open('movie_data.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['Title', 'Genre', 'Premiere', 'Runtime', 'IMDB Score', 'Language'])
    for n in range(1, 100):
        writer.writerow(generate_movie())

Running the cell will output the CSV file movie_data.csv in our root that looks like this:

Title,Genre,Premiere,Runtime,IMDB Score,Language
Discuss According Model,Horror,"February 09, 2020",107,2.6,Japanese
People Conference Be,Comedy,"April 25, 2020",84,1.8,Chinese
Forget Great Kind,Drama,"May 22, 2021",128,3.3,Chinese
Trial Employee Cover,Drama,"February 24, 2020",90,3.6,Spanish
Choose System We,Drama,"June 29, 2020",102,3.3,Spanish
Range Laugh Reach,Comedy,"August 09, 2021",92,3.9,Spanish
Increase Fire Popular,Romance,"May 03, 2020",107,4.1,Japanese
Show Job Believe,Thriller,"March 13, 2021",62,1.6,English
Or Power Century,Comedy,"February 29, 2020",146,2.3,Spanish
Ago Ability Within,Drama,"July 23, 2020",120,4.8,Italian
Foreign Always Sing,Mystery,"May 16, 2021",112,1.9,English
Once Movie Artist,Documentary,"February 09, 2020",79,4.1,Hindi
Near Explain Process,Action,"July 17, 2021",134,2.0,Spanish
Big Information Grow,Romance,"February 25, 2020",64,4.4,Spanish
Wind Project Heavy,Drama,"February 20, 2021",128,4.8,English
Child Form Theory,Mystery,"January 12, 2021",91,3.0,Spanish
Bring Sport Present,Drama,"March 02, 2021",87,2.7,Hindi
Themselves That Activity,Action,"August 20, 2020",148,3.0,Spanish
City Threat Almost,Thriller,"February 16, 2020",107,3.9,Spanish
See Main Student,Drama,"January 17, 2020",125,1.4,Chinese
Population Impact Season,Action,"March 19, 2020",109,2.3,Italian
Manager Thank Truth,Documentary,"February 12, 2021",124,4.1,Hindi
Child South Believe,Thriller,"April 18, 2020",65,3.9,Italian
Present Main Themselves,Romance,"September 08, 2020",89,3.8,Hindi
Maintain Order Old,Drama,"December 14, 2020",110,1.8,Hindi
Difficult Town Hair,Documentary,"October 12, 2020",51,4.9,Japanese
Page Hold Discussion,Drama,"November 01, 2020",139,1.9,Chinese
Style True Car,Comedy,"July 03, 2021",84,5.0,Japanese
Care Item Sing,Comedy,"November 16, 2020",100,4.9,Japanese
Do Car Organization,Romance,"February 28, 2021",129,1.1,Japanese
Learn Service Figure,Documentary,"March 04, 2020",50,2.0,Italian
Forget Situation Fact,Comedy,"January 22, 2020",52,3.9,English
Order International Report,Documentary,"December 17, 2020",101,2.2,Chinese
Another Black Teach,Mystery,"December 08, 2020",96,4.2,Italian
Professor Watch Throughout,Action,"September 15, 2020",111,4.0,English
Which Quickly Son,Documentary,"July 02, 2021",98,2.4,Chinese
Change East Article,Comedy,"March 28, 2020",61,2.4,English
Partner Individual Local,Romance,"May 07, 2020",149,5.0,English
Instead Watch Particular,Horror,"May 04, 2020",115,2.3,Hindi
Democratic Someone Available,Romance,"July 26, 2021",98,1.4,Italian
Place Would Mind,Drama,"May 09, 2021",141,2.4,Italian
Likely Economy Weight,Mystery,"February 03, 2021",106,3.1,Hindi
Could Certain More,Drama,"January 31, 2021",137,4.9,Hindi
Source Operation Sure,Action,"March 03, 2020",81,3.3,Hindi
Really Share Treat,Documentary,"August 05, 2020",99,2.2,English
Edge When Data,Drama,"July 27, 2020",115,1.6,Italian
Huge Imagine Federal,Romance,"August 08, 2021",141,3.0,Chinese
Tend Often Collection,Documentary,"June 25, 2020",73,3.2,Chinese
Wait Major Move,Action,"June 17, 2021",120,2.5,Spanish
Firm Reason With,Thriller,"July 16, 2021",67,2.6,Spanish
Significant Fall Travel,Romance,"March 14, 2021",123,2.0,Hindi
Send Size Eye,Comedy,"June 18, 2021",74,3.5,Spanish
Describe Hospital She,Drama,"March 14, 2021",90,1.4,Spanish
Give Drive Better,Mystery,"March 15, 2020",106,1.2,Spanish
Their Measure Choose,Action,"April 28, 2021",86,2.8,Italian
Resource Sell Agent,Thriller,"February 08, 2020",50,3.1,Hindi
Next Plan Soon,Action,"May 16, 2021",93,3.7,Hindi
Land Allow Simply,Mystery,"May 23, 2021",144,1.0,Hindi
Friend Total Few,Mystery,"June 12, 2021",93,4.1,Italian
Role Might Bad,Drama,"December 08, 2020",100,3.5,Japanese
Opportunity Public Certainly,Horror,"August 07, 2020",76,2.0,Italian
Else Play Politics,Drama,"August 01, 2021",145,2.5,Italian
Staff Main West,Documentary,"May 09, 2021",76,2.5,Japanese
Ready Treat Everything,Drama,"July 24, 2021",121,1.6,Hindi
Ahead Yourself Crime,Horror,"February 09, 2021",80,4.9,Italian
Next These Night,Comedy,"February 20, 2020",65,3.4,Hindi
Line Else Along,Comedy,"February 05, 2020",83,1.8,Hindi
Degree Continue Green,Documentary,"March 10, 2020",73,3.8,Hindi
Marriage Until Cover,Thriller,"November 26, 2020",147,4.8,English
Republican Way Mission,Drama,"April 04, 2021",57,2.9,Chinese
Prepare Rich Street,Romance,"February 26, 2021",94,2.6,Japanese
Term Five On,Horror,"September 06, 2020",62,2.7,English
Sister Manage Relate,Documentary,"August 17, 2020",76,4.4,Hindi
Scientist Beat Wonder,Horror,"June 23, 2021",137,1.5,Chinese
Fast Staff If,Romance,"February 05, 2021",148,2.7,Hindi
Ready Campaign Field,Comedy,"October 25, 2020",147,2.7,Chinese
Worker State Every,Mystery,"May 17, 2021",104,1.7,English
Bar Wind Story,Action,"January 28, 2021",108,3.2,Hindi
At Total Half,Thriller,"December 03, 2020",79,4.4,Spanish
One Something Focus,Thriller,"June 29, 2020",59,1.2,Japanese
Play We Impact,Comedy,"March 19, 2020",88,1.3,Hindi
Message After Again,Comedy,"May 28, 2021",75,4.1,Chinese
Such Something Information,Comedy,"June 01, 2021",145,2.2,Spanish
Power Organization Myself,Action,"January 29, 2021",119,1.4,Hindi
Apply Boy Success,Documentary,"August 06, 2020",93,1.4,Italian
Evening Production Bar,Romance,"April 13, 2020",102,2.5,Chinese
Work For Form,Drama,"September 19, 2020",80,4.4,Hindi
Occur Billion Cover,Documentary,"December 03, 2020",56,3.7,Chinese
Budget Wall Tv,Horror,"January 02, 2021",135,1.0,English
Share Beyond Loss,Action,"January 23, 2021",55,1.5,Italian
Professional Source Make,Horror,"December 08, 2020",107,4.1,Japanese
To Protect Improve,Mystery,"July 30, 2020",100,3.6,Japanese
Democratic Hundred Appear,Horror,"August 18, 2020",84,4.3,Hindi
Face Central Summer,Documentary,"November 25, 2020",63,1.8,Spanish
Involve Clearly At,Documentary,"November 25, 2020",56,1.5,Italian
Fall Term Drug,Horror,"April 05, 2020",52,2.2,Chinese
Fly Language Where,Romance,"May 18, 2021",102,4.4,Chinese
Service Local Door,Drama,"August 04, 2020",63,1.9,Italian
Son Avoid Himself,Drama,"July 30, 2020",53,1.8,Hindi

Success!

Summary

Today's post demonstrated how to use the Faker package to generate fake data and the CSV library to export that data to file.

In future, we may use this data to make our data sets to work with and some some data science around.

Kaggle and Open Data are great resources for data and data visualization for any use you may also have when not generating your own data.

This "100 Days in Python" series will move towards data science and machine learning from here on out.

Resources and further reading

Photo credit: pawel_czerwinski

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Semantic Versioning In Python With Git Hooks

Dennis O'Keeffe — Wed, 22 Sep 2021 10:15:14 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 22 of the #100DaysOfPython challenge.

This post will use the pre-commit and commitizen packages to automate our semantic versioning for Python based on our commits.

The post will build off the work done in the post "Your First Pip Package With Python".

The final code can be found here.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.

Getting started

We are going to clone and work from the okeeffed/your-first-pip-package-in-python repository and install the required packages.

$ git clone https://github.com/okeeffed/your-first-pip-package-in-python semantic-versioning-in-python-with-git-hooks
$ cd semantic-versioning-in-python-with-git-hooks

# Init the virtual environment
$ pipenv install --dev pre-commit Commitizen toml

# Create some required files
$ touch .pre-commit-config.yaml

At this stage, we are ready to configure our pre-commit hook.

Pre-commit configuration

We need to add the following to .pre-commit-config.yaml:

---
repos:
  - repo: https://github.com/commitizen-tools/commitizen
    rev: master
    hooks:
      - id: commitizen
        stages: [commit-msg]

Once done, we can setup our Commitizen configuration.

Setting up Commitizen

We can set this up by running pipenv run cz init:

$ pipenv run cz init
? Please choose a supported config file: (default: pyproject.toml) pyproject.toml
? Please choose a cz (commit rule): (default: cz_conventional_commits) cz_conventional_commits
No Existing Tag. Set tag to v0.0.1
? Please enter the correct version format: (default: "$version")
? Do you want to install pre-commit hook? Yes
commitizen already in pre-commit config
commitizen pre-commit hook is now installed in your '.git'

You can bump the version and create changelog running:

cz bump --changelog
The configuration are all set.

We are now at a stage where our commits can affect our versioning! A file is created for us pyproject.toml:

[tool]
[tool.commitizen]
name = "cz_conventional_commits"
version = "0.0.1"
tag_format = "$version"

This will contain the version information that is maintained by Commitizen for us.

Automated semantic versioning at work

First of all, we will need to create a tag for version 0.0.1:

$ git tag -a 0.0.1 -m "Init version"

Our version will change based on the naming that we give our commits. The installed Git hook will enforce that our commits follow the Conventional Commits naming conventions.

Let's see this in action. First, let's add a new function to the file demo_pip_math/math.py:

def divide(x: int, y: int) -> float:
    """divide one number by another

    Args:
                    x (int): first number in the division
                    y (int): second number in the division

    Returns:
                    int: division of x and y
    """
    return x / y

We want to add and commit this code. Let's see what happens when we set an invalid version:

$ git add demo_pip_math/math.py
$ git commit -m "added new feature division"
commitizen check.........................................................Failed
- hook id: commitizen
- exit code: 14

commit validation: failed!
please enter a commit message in the commitizen format.
commit "": "not a valid commit name"
pattern: (build|ci|docs|feat|fix|perf|refactor|style|test|chore|revert|bump)(\(\S+\))?!?:(\s.*)

This tells use that we did not match the expected pattern.

Given that we are adding a new feature, let's try with the feat: prefix:

$ git commit -m "feat: added new divide functionality"
commitizen check.........................................................Passed
[main 333d291] feat: added new divide functionality
 1 file changed, 13 insertions(+)

We've passed!

Now that we've made a change, we can use Commitizen to help us update our version and create a Changelog and update the version:

$ pipenv run cz bump
bump: version 0.0.1 → 0.1.0
tag to create: 0.1.0
increment detected: MINOR

Done!

If we now check pyproject.toml, we can see that change reflected for us:

[tool]
[tool.commitizen]
name = "cz_conventional_commits"
version = "0.1.0"
tag_format = "$version"

Creating the Changelog

To create a Changelog, run pipenv run cz changelog:

$ pipenv run cz changelog
# ... no output

This will create a CHANGELOG.md file in our project root directory, now with the following information:

## 0.1.0 (2021-08-10)

### Feat

- added new divide functionality

## 0.0.1 (2021-08-10)

### Feat

- add in division function
- init commit

Success!

Keeping setup.py in sync

Finally, we want to make sure that our setup.py is in sync with what is reflected in pyproject.toml.

We can do this with the toml package.

Inside of setup.py, change the file to be the following:

import setuptools
import toml
from os.path import join, dirname, abspath

pyproject_path = join(dirname(abspath("__file__")),
                      '../pyproject.toml')
file = open(pyproject_path, "r")
toml_str = file.read()

parsed_toml = toml.loads(toml_str)

with open("README.md", "r") as fh:
    long_description = fh.read()

setuptools.setup(
    name="demo_pip_math",
    version=parsed_toml['tool']['commitizen']['version'],
    author="Dennis O'Keeffe",
    author_email="hello@dennisokeeffe.com",
    description="Demo your first Pip package.",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/okeeffed/your-first-pip-package-in-python",
    packages=setuptools.find_packages(),
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
    keywords='pip-demo math',
    project_urls={
        'Homepage': 'https://github.com/okeeffed/your-first-pip-package-in-python',
    },

)

We are reading in the TOML and finding the reference to the current version.

We can check that this still works as expected with a build script that was already added in the repo:

$ pipenv run build
running sdist
running egg_info
creating demo_pip_math.egg-info
writing demo_pip_math.egg-info/PKG-INFO
writing dependency_links to demo_pip_math.egg-info/dependency_links.txt
writing top-level names to demo_pip_math.egg-info/top_level.txt
writing manifest file 'demo_pip_math.egg-info/SOURCES.txt'
reading manifest file 'demo_pip_math.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'demo_pip_math.egg-info/SOURCES.txt'
running check
creating demo_pip_math-0.1.0
creating demo_pip_math-0.1.0/demo_pip_math
creating demo_pip_math-0.1.0/demo_pip_math.egg-info
creating demo_pip_math-0.1.0/tests
copying files to demo_pip_math-0.1.0...
copying LICENSE -> demo_pip_math-0.1.0
copying MANIFEST.in -> demo_pip_math-0.1.0
copying README.md -> demo_pip_math-0.1.0
copying pyproject.toml -> demo_pip_math-0.1.0
copying setup.py -> demo_pip_math-0.1.0
copying demo_pip_math/__init__.py -> demo_pip_math-0.1.0/demo_pip_math
copying demo_pip_math/math.py -> demo_pip_math-0.1.0/demo_pip_math
copying demo_pip_math.egg-info/PKG-INFO -> demo_pip_math-0.1.0/demo_pip_math.egg-info
copying demo_pip_math.egg-info/SOURCES.txt -> demo_pip_math-0.1.0/demo_pip_math.egg-info
copying demo_pip_math.egg-info/dependency_links.txt -> demo_pip_math-0.1.0/demo_pip_math.egg-info
copying demo_pip_math.egg-info/top_level.txt -> demo_pip_math-0.1.0/demo_pip_math.egg-info
copying tests/__init__.py -> demo_pip_math-0.1.0/tests
copying tests/test_math.py -> demo_pip_math-0.1.0/tests
Writing demo_pip_math-0.1.0/setup.cfg
creating dist
Creating tar archive
removing 'demo_pip_math-0.1.0' (and everything under it)
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/demo_pip_math
copying demo_pip_math/__init__.py -> build/lib/demo_pip_math
copying demo_pip_math/math.py -> build/lib/demo_pip_math
creating build/lib/tests
copying tests/__init__.py -> build/lib/tests
copying tests/test_math.py -> build/lib/tests
warning: build_py: byte-compiling is disabled, skipping.

installing to build/bdist.macosx-11-x86_64/wheel
running install
running install_lib
creating build/bdist.macosx-11-x86_64
creating build/bdist.macosx-11-x86_64/wheel
creating build/bdist.macosx-11-x86_64/wheel/demo_pip_math
copying build/lib/demo_pip_math/__init__.py -> build/bdist.macosx-11-x86_64/wheel/demo_pip_math
copying build/lib/demo_pip_math/math.py -> build/bdist.macosx-11-x86_64/wheel/demo_pip_math
creating build/bdist.macosx-11-x86_64/wheel/tests
copying build/lib/tests/__init__.py -> build/bdist.macosx-11-x86_64/wheel/tests
copying build/lib/tests/test_math.py -> build/bdist.macosx-11-x86_64/wheel/tests
warning: install_lib: byte-compiling is disabled, skipping.

running install_egg_info
Copying demo_pip_math.egg-info to build/bdist.macosx-11-x86_64/wheel/demo_pip_math-0.1.0-py3.9.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.macosx-11-x86_64/wheel/demo_pip_math-0.1.0.dist-info/WHEEL
creating 'dist/demo_pip_math-0.1.0-py3-none-any.whl' and adding 'build/bdist.macosx-11-x86_64/wheel' to it
adding 'demo_pip_math/__init__.py'
adding 'demo_pip_math/math.py'
adding 'tests/__init__.py'
adding 'tests/test_math.py'
adding 'demo_pip_math-0.1.0.dist-info/LICENSE'
adding 'demo_pip_math-0.1.0.dist-info/METADATA'
adding 'demo_pip_math-0.1.0.dist-info/WHEEL'
adding 'demo_pip_math-0.1.0.dist-info/top_level.txt'
adding 'demo_pip_math-0.1.0.dist-info/RECORD'
removing build/bdist.macosx-11-x86_64/wheel

We can see that version 0.1.0 is used in this build.

Summary

Today's post demonstrated how to use the commitizen, pre-commit and toml packages to help automate our versioning process based on Conventional Commits.

This helps larger teams keep their packages semantically correct with little effort added to their work.

Resources and further reading

Photo credit: snapsbyclark

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

PyTest With GitHub Actions

Dennis O'Keeffe — Fri, 17 Sep 2021 10:15:11 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 20 of the #100DaysOfPython challenge.

This post will build off previous work done in the Python Unit Testing With PyTest post to add a GitHub Action for running test runner jobs on a push event to the repo.

The final project code can be found on my GitHub repo.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Read Python Unit Testing With PyTest if you are unfamiliar with Python.

Getting started

Let's clone the original PyTest repo in hello-pytest-github-actions.

$ git clone https://github.com/okeeffed/hello-pytest.git hello-pytest-github-actions
$ cd hello-pytest-github-actions

# Install the deps
$ pipenv install

# Prepare the file for the GitHub action
$ mkdir -p .github/workflows
$ touch .github/workflows/pytest.yml

At this stage, we can test that our repo is working correctly with PyTest locally:

$ pipenv run pytest
============================== test session starts ==============================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /path/to/code/blog-projects/hello-pytest
collected 3 items

tests/test_math.py ...                                                    [100%]

=============================== 3 passed in 0.01s ===============================

At this stage, we are ready to set things up for our GitHub action!

Adding the test script

In of the Pipfile, we want to add a [scripts] section to add a test script:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]
pytest = "*"

[requires]
python_version = "3.9"

[scripts]
test = "pytest"

That test script will simply call pytest. We will use this script within our GitHub action.

In order to also prepare our GitHub action to give us more vebose information, we will also be passing a -v flag to our call to pytest.

We can test things are working as expected by running pipenv run test -v from our terminal:

$ pipenv run test -v
========================== test session starts ===========================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/dennisokeeffe/code/blog-projects/hello-pytest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/dennisokeeffe/code/blog-projects/hello-pytest-github-actions
collected 3 items

tests/test_math.py::test_add PASSED                                [ 33%]
tests/test_math.py::test_subtract PASSED                           [ 66%]
tests/test_math.py::test_multiply PASSED                           [100%]

=========================== 3 passed in 0.02s ============================

The verbose flag gives us more information about which test ran and passed. This can be helpful for debugging in CI.

Adding the GitHub action

We are now ready to add the GitHub action. Within the .github/workflows/pytest.yml file that we created earlier, add the following:

# .github/workflows/app.yaml
name: PyTest
on: push

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10

    steps:
      - name: Check out repository code
        uses: actions/checkout@v2

      # Setup Python (faster than using Python container)
      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.x"

      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel
      - id: cache-pipenv
        uses: actions/cache@v1
        with:
          path: ~/.local/share/virtualenvs
          key: ${{ runner.os }}-pipenv-${{ hashFiles('**/Pipfile.lock') }}

      - name: Install dependencies
        if: steps.cache-pipenv.outputs.cache-hit != 'true'
        run: |
          pipenv install --deploy --dev
      - name: Run test suite
        run: |
          pipenv run test -v

Here we are doing a couple of things:

Creating a job call PyTest.
Running this job on a push event to the repository.
Running the job on ubuntu-latest.
Setting a custom timeout of 10 minutes (albeit this is overkill, feel free to omit).
Setting up the Python environment for the latest in version 3.x.
Install pipenv and wheel.
Install dependencies with a cache set to be the hash of the lockfile.
Running the test suite that we setup the command and tested before.

That is all that we need for this repo to be working!

Be sure at this stage that you have set up your own origin remote for the repo.

At this stage, all we need to do is commit the code and push the repo and the job will be available under the actions tab in the GitHub UI.

Summary

Today's post demonstrated how to use GitHub actions to test Python code on a push to the remote repository. We used the pytest testing framework to test our code.

Resources and further reading

Photo credit: amseaman

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

How PyDoc Helps Your Python Development

Dennis O'Keeffe — Mon, 13 Sep 2021 10:15:14 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 19 of the #100DaysOfPython challenge.

This post will use the pydoc module from the standard library to to explore how we can navigate documentation as well as create our own.

The code can be found on my GitHub repo.

Getting started

Let's create the hello-pydoc directory and set up the required files.

# Make the `hello-pydoc` directory
$ mkdir hello-pydoc
$ cd hello-pydoc
# Create a folder to place our example files
$ mkdir src
# Create the required files
$ touch src/math.py main.py src/__init__.py

At this stage, we have our files sorted and we can start to write some code.

Exploring PyDoc from the command line

PyDoc is a tool that allows us to explore the documentation of a module.

As a simple starter, we can use PyDoc to explore the documentation of the datetime module from the command line.

To do so, we can run python -m pydoc [module] and we should see the output based on the doc strings for each of the functions in the module.

Let's explore this for the datetime module. From the command line, we can run the following command:

$ python -m pydoc datetime
Help on module datetime:

NAME
    datetime - Fast implementation of the datetime type.

FILE
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/datetime.so

MODULE DOCS
    https://docs.python.org/library/datetime

CLASSES
    __builtin__.object
        date
            datetime
        time
        timedelta
        tzinfo

    class date(__builtin__.object)
     |  date(year, month, day) --> date object
     |
     |  Methods defined here:
     |
     |  __add__(...)
    x.__add__(y) <==> x+y
     |
     |  __eq__(...)
# ... reset omitted for brevity ...

When you run the command, it opens up the documentation for the datetime module that can be traversed in the command-line.

It gives us information on the following:

Heading	Description
`NAME`	The name of the module.
`FILE`	The file that the module is defined in.
`MODULE DOCS`	The link to the documentation for the module.
`CLASSES`	The classes defined in the module.

We can search within the docs by using the / or ? operator followed by a search term.

For example, in my previous blog post Datetime in Python, we heavily relied upon the datetime.datetime.now function.

To search for that documentation, I can run / followed by now and one of the search results will return the following:

now(tz=None) from builtins.type
    Returns new datetime object representing current time local to tz.

      tz
        Timezone object.

    If no tz is specified, uses local timezone.

Tip: after searching, you can use the n key to navigate to the next result.

The above documentation tells us that the now function returns a datetime object with the timezone's local time and date (with an optional argument of the timezone).

Given this, we know that we could can the function both with and without an argument:

from datetime import datetime, timezone

datetime.now() # datetime.datetime(2021, 8, 7, 7, 22, 37, 216899)
datetime.now(timezone.utc) # datetime.datetime(2021, 8, 6, 21, 22, 30, 841500, tzinfo=datetime.timezone.utc)

If we go searching for the synonymous function utcnow, we can see the following:

utcnow(...) from builtins.type
    Return a new datetime representing UTC day and time.

This tells us that there are no arguments but the returned value is a datetime object with the timezone of UTC.

from datetime import datetime

datetime.utcnow() # datetime.datetime(2021, 8, 6, 21, 24, 2, 865402)

Now that we know the basics of searching PyDoc, let's now document our math module.

Writing our own documented code

We can write and search for our own docstrings.

To demonstrate, add the following code to src/math.py:

def add(x, y):
    """add together two numbers

    Args:
        x (int): First number to add
        y (int): Second number to add

    Returns:
        int: sum of x and y
    """
    return x + y

The docstring is a string that is used to document the function and is denoted as the comment between lines 2 and 10 of the code above.

Tip: if you use VSCode, check out the docstring generator

Once we have that code in place, it is enough to view our own documentation for the module!

Exploring our documented code

We can run python -m pydoc src.math and we should see the output of the docstring:

Help on module src.math in src:

NAME
    src.math

FUNCTIONS
    add(x, y)
        add together two numbers

        Args:
            x (int): First number to add
            y (int): Second number to add

        Returns:
            int: sum of x and y

FILE
    /path/to/code/blog-projects/hello-pydoc/src/math.py

How easy was that!

Viewing PyDoc in the browser

If we run pydoc -h, we see the following:

-h
pydoc - the Python documentation tool

pydoc <name> ...
    Show text documentation on something.  <name> may be the name of a
    Python keyword, topic, function, module, or package, or a dotted
    reference to a class or function within a module or module in a
    package.  If <name> contains a '/', it is used as the path to a
    Python source file to document. If name is 'keywords', 'topics',
    or 'modules', a listing of these things is displayed.

pydoc -k <keyword>
    Search for a keyword in the synopsis lines of all available modules.

pydoc -p <port>
    Start an HTTP server on the given port on the local machine.  Port
    number 0 can be used to get an arbitrary unused port.

pydoc -g
    Pop up a graphical interface for finding and serving documentation.

pydoc -w <name> ...
    Write out the HTML documentation for a module to a file in the current
    directory.  If <name> contains a '/', it is treated as a filename; if
    it names a directory, documentation is written for all the contents.

This gives us a list of the options that we can use to view our own documentation.

First of all, we will use the pydoc -p 4000 option to start a server on port 4000.

From here, you can open up http://localhost:4000 in your browser and explore the documentation. The interface isn't pretty, but the job gets done.

Outputting PyDoc as HTML

Finally, we can output the documentation as HTML by passing the -w option to pydoc.

Running pydoc -w src.math outputs HTML into src.math.html which contains the following:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
  <head>
    <title>Python: module src.math</title>

  </head>
  <body bgcolor="#f0f0f8">
    <table
      width="100%"
      cellspacing="0"
      cellpadding="2"
      border="0"
      summary="heading"
    >
      <tr bgcolor="#7799ee">
        <td valign="bottom">
          &nbsp;
          <font color="#ffffff" face="helvetica, arial"
            >&nbsp;<big
              ><big
                ><strong
                  ><a href="src.html"><font color="#ffffff">src</font></a
                  >.math</strong
                ></big
              ></big
            ></font
          >
        </td>
        <td align="right" valign="bottom">
          <font color="#ffffff" face="helvetica, arial"
            ><a
              href="file:/path/to/code/blog-projects/hello-pydoc/src/math.py"
              >/path/to/code/blog-projects/hello-pydoc/src/math.py</a
            ></font
          >
        </td>
      </tr>
    </table>
    <p></p>
    <p></p>
    <table
      width="100%"
      cellspacing="0"
      cellpadding="2"
      border="0"
      summary="section"
    >
      <tr bgcolor="#eeaa77">
        <td colspan="3" valign="bottom">
          &nbsp;
          <font color="#ffffff" face="helvetica, arial"
            ><big><strong>Functions</strong></big></font
          >
        </td>
      </tr>

      <tr>
        <td bgcolor="#eeaa77"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td>
        <td>&nbsp;</td>
        <td width="100%">
          <dl>
            <dt>
              <a name="-add"><strong>add</strong></a
              >(x, y)
            </dt>
            <dd>
              <tt
                >add&nbsp;together&nbsp;two&nbsp;numbers
                &nbsp;
                Args:
                &nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;(int):&nbsp;First&nbsp;number&nbsp;to&nbsp;add
                &nbsp;&nbsp;&nbsp;&nbsp;y&nbsp;(int):&nbsp;Second&nbsp;number&nbsp;to&nbsp;add
                &nbsp;
                Returns:
                &nbsp;&nbsp;&nbsp;&nbsp;int:&nbsp;sum&nbsp;of&nbsp;x&nbsp;and&nbsp;y</tt
              >
            </dd>
          </dl>
        </td>
      </tr>
    </table>
  </body>
</html>

Summary

Today's post demonstrated how to use the pydoc package to explore built-in and custom documentation.

Albeit a shallow exploration, the best way to improve on your searching capability is to use the command line in practice.

Resources and further reading

Photo credit: milada_vigerova

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Time Functionality In Python

Dennis O'Keeffe — Sun, 12 Sep 2021 10:15:10 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 18 of the #100DaysOfPython challenge.

This post will use the time module from Python's standard library to explore how we can work with different time capabilities such as getting the local time and sleeping within a program.

All of the code used in this post can be found in my GitHub repository.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with iPython.
...

Getting started

Let's create the hello-python-time-module directory and install Pillow.

# Make the `hello-python-time-module` directory
$ mkdir hello-python-time-module
$ cd hello-python-time-module

# Init the virtual environment
$ pipenv --three
$ pipenv install --dev ipython

At this stage, we are ready to explore using the time module using iPython.

To do so, run pipenv run ipython from the command line to open up the REPL.

Importing the module

First off, we will want to import the time module. We can do so from within the REPL with the following:

import time

Once imported, we can check that we have access to the time module by checking the __name__ attribute and playing around with a few methods.

time.__name__ # 'time'
time.time() # 1628200068.664737

The method time that we called on the module returns the time in seconds since the epoch as a floating point number.

On most systems, the epoch is January 1st, 1970 at midnight. This is currently referred to as Unix time.

Getting the local time

We can also use the time module to get the local or GM time, as well as format the time to a more readable format.

time.localtime() # time.struct_time(tm_year=2021, tm_mon=8, tm_mday=6, tm_hour=7, tm_min=46, tm_sec=20, tm_wday=4, tm_yday=218, tm_isdst=0)
time.gmtime() # time.struct_time(tm_year=2021, tm_mon=8, tm_mday=5, tm_hour=21, tm_min=46, tm_sec=32, tm_wday=3, tm_yday=217, tm_isdst=0)
time.asctime(time.localtime()) # 'Fri Aug  6 07:51:53 2021'

Formatting and parsing

We have control over how we format and parse time with the strftime and strptime functions.

time.strptime("30 Nov 00", "%d %b %y") # time.struct_time(tm_year=2000, tm_mon=11, tm_mday=30, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=335, tm_isdst=-1)
time.strftime("%m/%d/%Y, %H:%M:%S") # '08/06/2021, 07:56:44'
time.strftime("%d %b %y", time.strptime("30 Nov 00", "%d %b %y")) # '30 Nov 00'

Sleep

We can use the time module to set a timer and sleep for a certain amount of time.

time.sleep(5) # notice that the REPL does not return until 5 seconds have passed

This can be useful when looping through intervals based on conditional logic.

Comparing times

We can use the datetime module to compare times by converting time objects to datetime objects.

We need to do so with the datetime.datetime.fromtimestamp method.

import datetime

# Check time now is less than 1 second later
datetime.datetime.fromtimestamp(time.time()) < datetime.datetime.now() + datetime.timedelta(seconds=1) # True
# Check time now is after 1 second before
datetime.datetime.fromtimestamp(time.time()) < datetime.datetime.now() - datetime.timedelta(seconds=1) # False

For more examples of datetime comparisons, see my post on Datetime In Python and more from my series Working with dates and times in Python.

Summary

Today's post demonstrates some usages of the time module from Python's standard library.

We covered a number of the standard methods and finished with an example on how to compare using the datetime module.

Resources and further reading

Photo credit: pawel_czerwinski

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Mocking Python datetime In Tests With FreezeGun

Dennis O'Keeffe — Tue, 07 Sep 2021 10:15:13 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 17 of the #100DaysOfPython challenge.

This post will use the FreezeGun library to demonstrate how to mock the datetime.datetime.now return value to set up consistent test environments.

The repo code can be found on GitHub at okeeffed/hello-python-datetime

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with JupyterLab. See here for my post on JupyterLab.
My previous blog post Datetime In Python
Familiarity with PyTest. See my blog post Python Unit Testing With PyTest for a start post.

Getting started

The previous code will built on top of the code from Datetime In Python. The final repo is at okeeffed/hello-python-datetime.

For the sake of simplicity, we will operate as if we are building a brand new repo:

Let's create the hello-python-datetime directory and install the required dependencies.

# Make the `hello-python-datetimes` directory
$ mkdir hello-python-datetimes
$ cd hello-python-datetimes

# Init the virtual environment
$ pipenv --three
$ pipenv install --dev ipython freezegun types-freezegun

# Create a folder to place files
$ mkdir src tests
# Create the required files
$ touch src/datetimes.py src/__init__.py tests/datetimes_test.py tests/__init__.py main.py

At this stage, we are now ready to update our main.py file and src/datetimes.py to be up to par with what we need for testing.

Add the following to src/datetimes.py:

from datetime import date


def is_date_before_today(date_str: str):
    """Check if date is before today

    Args:
        date_str (str): String of a date to pass

    Returns:
        bool: Value of if date is before today
    """
    try:
        date_obj = date.fromisoformat(date_str)
        return date_obj < date.today()
    except Exception:
        return False

Add the following to main.py:

from src.datetimes import is_date_before_today
from datetime import datetime, timedelta

print(is_date_before_today("2019-01-01"))
print(is_date_before_today("2022-01-01"))
print(is_date_before_today("2021-08-03"))
print(is_date_before_today("2021-08-04"))

now = datetime.now()
now_str = now.strftime('%Y-%m-%d')
print(now_str)
print(is_date_before_today(now_str))

now_subtract_one_day = now - timedelta(days=2)

now_subtract_one_day_str = now_subtract_one_day.strftime('%Y-%m-%d')
print(now_subtract_one_day_str)
print(is_date_before_today(now_subtract_one_day_str))

now_add_one_day = now + timedelta(days=1)

now_add_one_day_str = now_add_one_day.strftime('%Y-%m-%d')
print(now_add_one_day_str)
print(is_date_before_today(now_add_one_day_str))

Running python main.py should bring us up to par with the following:

$ python main.py
True
False
True
True
2021-08-05
False
2021-08-03
True
2021-08-06
False

The output matches up to us printing values out from main.py. We are at a stage now where we are up to par and able to start writing tests.

Note: If you are using a virtual environment, you will need to run pipenv shell to enter the virtual environment.

Exploring FreezeGun with PyTest

FreezeGun is a library that helps with mocking out the datetime.datetime.now function. It is a very useful tool for testing code that uses the datetime library.

We can use the library with a decorator for the test or creating a with block.

To demonstrate, add the following code to tests/datetimes_test.py:

import datetime
from freezegun import freeze_time
from src.datetimes import is_date_before_today


def test_freeze_time():
    assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)
    # Mocking the time to be 2012-01-14
    with freeze_time("2012-01-14"):
        assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
    # Without the mock, the time should be back to normal
    assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)


@freeze_time("2012-01-14")
def test_freeze_time_with_decorator():
    # Testing with a decorator that mocks throughout the test
    assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)

The first test demonstrates the with block will the second test demonstrates usage with a decorator.

Running pipenv run pytest will now run the tests and display the results.

$ pipenv run pytest
pipenv run pytest
================================== test session starts ===================================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /path/to/hello-python-datetimes
collected 2 items

tests/test_datetimes.py ..                                                          [100%]

=================================== 2 passed in 0.21s ====================================

Now we are ready to test our is_date_before_today function in a manner similar to how our main.py invokes the functions.

Testing the `is_date_before_today` function

Let's update our tests/datetimes_test.py file to test the is_date_before_today function.

import datetime
from freezegun import freeze_time
from src.datetimes import is_date_before_today


def test_freeze_time():
    assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)
    # Mocking the time to be 2012-01-14
    with freeze_time("2012-01-14"):
        assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
    # Without the mock, the time should be back to normal
    assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)


@freeze_time("2012-01-14")
def test_freeze_time_with_decorator():
    # Testing with a decorator that mocks throughout the test
    assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)

# Converting the output we expected from main.py into a set of tests.
# Mocking time unnecessary, but done for the sake of completion.


@freeze_time("2021-08-05")
def test_is_date_before_today():
    """Should return False"""
    now = datetime.datetime.now()
    now_str = now.strftime('%Y-%m-%d')
    assert is_date_before_today(now_str) is False


@freeze_time("2021-08-05")
def test_is_one_day_ago_before_today():
    """Should return True"""
    now_subtract_one_day = datetime.datetime.now() - datetime.timedelta(days=1)
    now_subtract_one_day_str = now_subtract_one_day.strftime('%Y-%m-%d')
    assert is_date_before_today(now_subtract_one_day_str) is True


@freeze_time("2021-08-05")
def test_is_one_day_ahead_before_today():
    """Should return False"""
    now_add_one_day = datetime.datetime.now() + datetime.timedelta(days=1)
    now_add_one_day_str = now_add_one_day.strftime('%Y-%m-%d')
    assert is_date_before_today(now_add_one_day_str) is False

In our test, we are freezing time (using the decorator) to the date of this blog post 2021-08-05 checking the following scenarios:

is_date_before_today when compared to today should be False.
is_date_before_today when compared to one day ago should be True.
is_date_before_today when compared to one day ahead should be False.

We can confirm this to be true by once again running pipenv run pytest:

$ pipenv run pytest
================================== test session starts ===================================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/dennisokeeffe/code/blog-projects/hello-python-datetimes
collected 5 items

tests/test_datetimes.py .....                                                      [100%]

=================================== 5 passed in 0.21s ====================================

Summary

Today's post demonstrated how to use the FreezeGun package to mock the date for testing with PyTest.

Resources and further reading

Photo credit: pawel_czerwinski

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Exceptions In Python

Dennis O'Keeffe — Mon, 06 Sep 2021 10:15:12 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 15 of the #100DaysOfPython challenge.

This post will look at home exceptions are rescued and will demonstrate how to handle different exceptions raised.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with JupyterLab. See here for my post on JupyterLab.

Getting started

Let's create the hello-python-exceptions directory and install Pillow.

# Make the `hello-python-exceptions` directory
$ mkdir hello-python-exceptions
$ cd hello-python-exceptions

# Init the virtual environment
$ pipenv --three
$ pipenv install --dev jupyterlab

Now we can start up the notebook server.

# Startup the notebook server
$ pipenv run jupyter-lab
# ... Server is now running on http://localhost:8888/lab

The server will now be up and running.

Creating the notebook

Once on http://localhost:8888/lab, select to create a new Python 3 notebook from the launcher.

Ensure that this notebook is saved in hello-python-exceptions/docs/<your-file-name>.

We will create four cells to handle four parts of this mini project:

A demonstration on how try/except works.
Looking at raising an error from a function.
Creating and using custom errors.

Try/Except

We can use the try/except statement to handle errors. You raise an Exception whenever you want to throw an error from a code block.

The Exception is self can then be delegated higher up the code chain to be handled.

For example, if we execute this in our first JupyterLab code block:

try:
  raise Exception('You shall not pass')
  print('Success')
except:
  print('There was an issue')

We notice the output is There was an issue. We do not make it passed the raised exception.

We can also capture exceptions by their type. For example, if we execute this in our second JupyterLab code block:

try:
  raise Exception('You shall not pass')
  print('Success')
except Exception:
  print('Did not make it to final catch-all block')
except:
  print('There was an issue')

You will notice that 'Did not make it to final catch-all block' was captured and printed, where as the final except code block is used a capture all.

Raising an error from a function

Defining a function that raises an error will traverse up the code block to the top level.

def example_exception():
  raise Exception('You shall not pass')

def example_fn():
  example_exception()

try:
  example_fn()
except:
  print('There was an issue')

This also prints out 'There was an issue'.

Creating and using custom errors

We can create custom errors simply by defining a class that extends Exception.

class CustomError(Exception):
    """Custom error"""
    pass

We can see this in action working randomly when we use the randrange function as a helper to determine which error to raise:

from random import randrange

# define Python user-defined exceptions
class CustomError(Exception):
    """Raised when using a custom error"""
    pass

def example_exception():
  val = randrange(10)
  if val <= 5:
    raise CustomError('You shall not pass')
  else:
    raise Exception("There was an error")

try:
  example_exception()
except CustomError as e:
  print(e)
except Exception as e:
  print(e)

Running this code in a block will print out 'You shall not pass' or 'There was an error' based on the random value.

That means that our CustomError is being handled in the except CustomError as e block.

Summary

Today's post demonstrated how to create custom errors by extending Exception and demonstrating how errors are raised in a simple manner.

Managing errors in Python is a necessity when working with more complex code that requires more fine-grained control over the possible error outcomes.

Resources and further reading

Photo credit: freestocks

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Datetime In Python

Dennis O'Keeffe — Sun, 05 Sep 2021 10:15:11 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 16 of the #100DaysOfPython challenge.

This post will use the standard datetime library to work with dates. The final project code can be found here.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with iPython or interactive shells.

Getting started

Let's create the hello-python-datetimes directory and install the required dependencies.

# Make the `hello-python-datetimes` directory
$ mkdir hello-python-datetimes
$ cd hello-python-datetimes

# Init the virtual environment
$ pipenv --three
$ pipenv install --dev ipython

# Create a folder to place files
$ mkdir src
# Create the required files
$ touch src/datetimes.py

We are now ready to begin exploring the standard datetime library.

Playing around with dates and times

Start the shell with pipenv shell ipython. This will activate the virtual shell and start an iPython interactive shell.

Now we are free to from datetime import date and play around with it:

from datetime import date

date(2021, 8, 4).isocalendar()
# datetime.IsoCalendarDate(year=2021, week=31, weekday=3)

date(2021, 8, 4).isoformat()
# '2021-08-04'

date(2021, 8, 4).ctime()
# 'Wed Aug  4 00:00:00 2021'

date(2021, 8, 4).strftime('%A')
# Wednesday

today = date.today()
# datetime.date(2021, 8, 4)

The above displays some of the common methods of the date class. More information about the date class can be found here.

A lot of what we want to do today is look at how to take some of these methods and get to a point where we can compare dates.

Comparing dates

We want to write a helper method to check if a given date in the format YYYY-MM-DD is before today. Exit the iPython kernel and add the following code to our src/datetimes.py file:

from datetime import date


def is_date_before_today(date_str: str):
    """Check if date is before today

    Args:
        date_str (str): String of a date to pass

    Returns:
        bool: Value of if date is before today
    """
    try:
        date_obj = date.fromisoformat(date_str)
        return date_obj < date.today()
    except Exception:
        return False


print(is_date_before_today("2019-01-01"))
print(is_date_before_today("2022-01-01"))
print(is_date_before_today("2021-08-03"))
print(is_date_before_today("2021-08-04"))

We can use the > and < operators to compare dates.

If we now run python src/datetimes.py we should see the following output:

$ python src/datetimes.py
True # "2019-01-01"
False # "2022-01-01"
True # "2021-08-03"
False # "2021-08-04"

We can see the output reflects a date string that is before today (with today returning False).

Working with now

The datetime module from the datetime library also has a now method that returns the current date and time.

We can use this in combination with strftime to get the current date and time in a string format. Update the code to the following:

from datetime import date, datetime


def is_date_before_today(date_str: str):
    """Check if date is before today

    Args:
        date_str (str): String of a date to pass

    Returns:
        bool: Value of if date is before today
    """
    try:
        date_obj = date.fromisoformat(date_str)
        return date_obj < date.today()
    except Exception:
        return False


now = datetime.now()
now_str = now.strftime('%Y-%m-%d')
print(now_str)
print(is_date_before_today(now_str))

Running python src/datetimes.py will output 2021-08-04 and False respectively.

To learn more about strftime, there is a great blog post here.

Adding and subtracting dates

We can use datetime.timedelta to add and subtract dates.

Update our code to the following:

from datetime import date, datetime, timedelta


def is_date_before_today(date_str: str):
    """Check if date is before today

    Args:
        date_str (str): String of a date to pass

    Returns:
        bool: Value of if date is before today
    """
    try:
        date_obj = date.fromisoformat(date_str)
        return date_obj < date.today()
    except Exception:
        return False


now = datetime.now()
now_str = now.strftime('%Y-%m-%d')
print(now_str)
print(is_date_before_today(now_str))

now_subtract_one_day = now - timedelta(days=2)

now_subtract_one_day_str = now_subtract_one_day.strftime('%Y-%m-%d')
print(now_subtract_one_day_str)
print(is_date_before_today(now_subtract_one_day_str))

now_add_one_day = now + timedelta(days=1)

now_add_one_day_str = now_add_one_day.strftime('%Y-%m-%d')
print(now_add_one_day_str)
print(is_date_before_today(now_add_one_day_str))

Once again we will python src/datetimes.py which will output the following:

$ python src/datetimes.py
2021-08-04
False
2021-08-02
True
2021-08-05
False

Awesome, now we know how to add and subtract from dates!

Summary

Today's post demonstrated how to use the datetime module to compare times and determine the current date.

Dates and times are incredibly important to any temporal applications, and the following posts will spend some time looking further into this and how to write tests for this form of functionality.

Resources and further reading

Photo credit: pawel_czerwinski

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Working With Regex Expressions In Python

Dennis O'Keeffe — Sat, 04 Sep 2021 10:15:12 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 14 of the #100DaysOfPython challenge

This post will use the regular expressions module from the standard library to ... .

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.
Familiarity with JupyterLab. See here for my post on JupyterLab.
Familiarity with Regular Expressions

Getting started

Let's create the hello-python-regex directory and install Pillow.

# Make the `hello-python-regex` directory
$ mkdir hello-python-regex
$ cd hello-python-regex
# Create a folder to place your icons
$ mkdir icons

# Init the virtual environment
$ pipenv --three
$ pipenv install --dev jupyterlab

At this stage, we can start up the notebook server.

# Startup the notebook server
$ pipenv run jupyter-lab
# ... Server is now running on http://localhost:8888/lab

The server will now be up and running.

Creating the notebook

Once on http://localhost:8888/lab, select to create a new Python 3 notebook from the launcher.

Ensure that this notebook is saved in hello-python-regex/docs/regex.ipynb.

We will explore the following in each cell of the notebook:

Importing the Regex module.
A basic usage of the Regex module.
String replacement with the Regex module.

Importing the module

This imports the regex module from the standard library.

import re

m = re.search("Hello, (.+)", "Hello, world!")
m.group(1)
# 'world!'

Basic usage of the module

There are a number of useful module methods that we can use that we will demonstrate:

Searching strings.
Matching strings.
Usage without compile.
Splitting a string into a list.
Replacing matches.

Searching strings

Scan through string looking for the first location where this regular expression produces a match, and return a corresponding match object. Return None if no position in the string matches the pattern; note that this is different from finding a zero-length match at some point in the string.

import re

pattern = re.compile("ello, (.+)")
m = pattern.search("Hello, world!")
m.group(1)
print(m) # <re.Match object; span=(0, 13), match='Hello, world!'>
print(m.group(1)) # world!

n = pattern.search("Hello, world!", 0)
print(n) # <re.Match object; span=(0, 13), match='Hello, world!'>
print(n.group(1)) # world!

Matching strings

If zero or more characters at the beginning of string match this regular expression, return a corresponding match object. Return None if the string does not match the pattern; note that this is different from a zero-length match.

pattern = re.compile("ello, (.+)")
m = pattern.match("Hello, world!")
# No match as "e" is the 2nd character the "Hello, world!".
print(m) # None

pattern = re.compile("Hello, (.+)")
# Does match
n = pattern.match("Hello, world!")
print(n) # <re.Match object; span=(0, 13), match='Hello, world!'>

Usage without compile

When you use re.match and re.search as a static method, you can pass the Regex as the first argument:

m = re.match("Hello, (.+)", "Hello, world!")
print(m) # <re.Match object; span=(0, 13), match='Hello, world!'>

n = re.match("Hello, (.+)", "Hello, world!")
print(n) # <re.Match object; span=(0, 13), match='Hello, world!'>

Splitting a string into a list

m = re.split(",", "Hello, world!")

print(m) # ['Hello', ' world!']

n = re.split("\s", "Hello beautiful world!")

print(n) # ['Hello', 'beautiful', 'world!']

Replacing matches

We can make use of the search and sub methods to replace matches.

# Simple example
target = "Photo credit by [@thomas](https://site.com/@thomas)"
m = re.search(r"Photo credit by \[@(.+)\]\(https://site.com/@(.+)\)", target)
res = re.sub(m.group(1), "dennis", target)
print(res) # Photo credit by [@dennis](https://site.com/@dennis)

# By iterating for multiple matches
target = "Photo credit by [@thomas](https://site.com/@user)"
m = re.search(r"Photo credit by \[@(.+)\]\(https://site.com/@(.+)\)", target)
res = target

for idx, val in enumerate(m.groups()):
        res = re.sub(val, "dennis", res)

print(res) # Photo credit by [@dennis](https://site.com/@dennis)

For a more specific replacement (particularly in a large set of text), we can be more explicit with the string to replace:

target = """
Other words thomas and user we don't want to replace.
Photo credit by [@thomas](https://site.com/@user)
"""
new_name = "dennis"

pattern = re.compile(r"Photo credit by \[@(.+)\]\(https://site.com/@(.+)\)")
res = pattern.sub(f"Photo credit by [@{new_name}](https://site.com/@{new_name})", target)
# Other words thomas and user we don't want to replace.
# Photo credit by [@dennis](https://site.com/@dennis)

Summary

Today's post demonstrated how to use the re module from the standard library to search, match, split and replace text in Python strings.

This can be unbelievably useful when working with text files.

Resources and further reading

Photo credit: pawel_czerwinski

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Annotating Video Clips With MoviePy

Dennis O'Keeffe — Fri, 03 Sep 2021 10:15:10 +0000

I write content for AWS, Kubernetes, Python, JavaScript and more. To view all the latest content, be sure to visit my blog and subscribe to my newsletter. Follow me on Twitter.

This is Day 12 of the #100DaysOfPython challenge.

This post will use the MoviePy library to annotate some captions onto a short video I recorded about my site workingoutloud.dev.

Prerequisites

Familiarity with Pipenv. See here for my post on Pipenv.

Getting started

Let's create the hello-moviepy directory and install Pillow.

# Make the `hello-moviepy` directory
$ mkdir hello-moviepy
$ cd hello-moviepy
# We will write everything in main.py
$ touch main.py
# Add this stage, you will need to add you video clip - I added mine to the `hello-moviepy` directory
$ cp path/to/wol-dev.mp4 .

# Init the virtual environment
$ pipenv --three
$ pipenv install moviepy

At this stage, we can write our script to annotate the video.

Annotating the video

As an example, here is a Gif of my video that I am editing:

The video is a short clip of my workingoutloud.dev site. It is around 9 seconds long and I want to add some text to it.

From watching the video, there are three things that are essentially in the demo:

Viewing tasks and subtasks.
Sorting subtasks.
Opening the current goals.

These occur at the 0:00, 0:03 and 0:06 second mark respectively.

We can add in some captions for each of these easily thanks to MoviePy.

In main.py, add the following:

from moviepy.editor import VideoFileClip, TextClip, CompositeVideoClip

video = VideoFileClip("wol-dev.mp4")

In the above code, we simply import the required modules and then we create a VideoFileClip object from the video file (relative to the project root directory).

We now want to create a TextClip object for each of the three points in the video.

According to the MoviePy TextClip docs, we can see the TextClip constructor can take some relevant arguments around the text, font, color and font size.

The TextClip instance itself then has access to methods to set the position, duration and start time. We will abstract our sensible defaults into a function and implement this:

def text_clip(text: str, duration: int, start_time: int = 0):
    """Return a description string on the bottom-left of the video

    Args:
                text (str): Text to show
                duration (int): Duration of clip
                start_time (int, optional): Time in video to start at (in seconds). Defaults to 0.

    Returns:
                moviepy.editor.TextClip: A instance of a TextClip
    """
    return (TextClip(text, font="Arial", fontsize=24, color='black')
            .set_position((20, video.h - 44))
            .set_duration(duration)
            .set_start(start_time))


# Make the text. Many more options are available.
text_clip_one = text_clip("Create tasks and subtasks", 3)
text_clip_two = text_clip("Sort subtasks by table header", 3, 3)
text_clip_three = text_clip("View goals", 3, 6)

After writing out the helper method, we are creating three TextClip instances for each of our different annotations that we wish to add to the video.

Finally, we can compose them all together into a CompositeVideoClip object and write out the video file.

I am also writing out a .gif file to use as a preview of the video on this post.

# Overlay text on video
result = CompositeVideoClip(
    [video, text_clip_one, text_clip_two, text_clip_three])
result.write_videofile("wol_dev_edited.mp4", fps=25)
result.write_gif("wol_dev_edited.gif", fps=8)

We can now run our script with python main.py. There will be some progress bars to indicate the progress.

After the script has completed, we can now see wol_dev_edited.mp4 and wol_dev_edited.gif in the hello-moviepy directory.

The outcome looks like so:

Summary

Today's post demonstrated how to use the MoviePy package to add some text captions to a video programmatically.

I will be hoping to make use of this to help annotate some of my videos that I am working through to save some time in future.

The final code looks like so:

from moviepy.editor import VideoFileClip, TextClip, CompositeVideoClip

video = VideoFileClip("wol-dev.mp4")


def text_clip(text: str, duration: int, start_time: int = 0):
    """Return a description string on the bottom-left of the video

    Args:
                text (str): Text to show
                duration (int): Duration of clip
                start_time (int, optional): Time in video to start at (in seconds). Defaults to 0.

    Returns:
                moviepy.editor.TextClip: A instance of a TextClip
    """
    return (TextClip(text, font="Arial", fontsize=24, color='black')
            .set_position((20, video.h - 44))
            .set_duration(duration)
            .set_start(start_time))


# Make the text. Many more options are available.
text_clip_one = text_clip("Create tasks and subtasks", 3)
text_clip_two = text_clip("Sort subtasks by table header", 3, 3)
text_clip_three = text_clip("View goals", 3, 6)

# Overlay text on video
result = CompositeVideoClip(
    [video, text_clip_one, text_clip_two, text_clip_three])
result.write_videofile("wol_dev_edited.mp4", fps=25)
result.write_gif("wol_dev_edited.gif", fps=8)

Resources and further reading

Photo credit: element5digital

Originally posted on my blog. To see new posts without delay, read the posts there and subscribe to my newsletter.

Forem: Dennis O'Keeffe

Recapping Kubernetes Basics

Prerequisites

The Control Plane

What is within a Node?

What is a Pod?

What is a Deployment?

What is a Service?

Summary

Resources and further reading

Oil Paint Effect With OpenCV Python

Prerequisites

Getting started

Applying the oil painting effect

Summary

Resources and further reading

Generating Fake CSV Data With Python

Prerequisites

Getting started

Creating the notebook

Emulating The Netflix Original Movies IMDB Scores Dataset

Emulating a value for each column

Generating the CSV

Summary

Resources and further reading

Semantic Versioning In Python With Git Hooks

Prerequisites

Getting started

Pre-commit configuration

Setting up Commitizen

Automated semantic versioning at work

Creating the Changelog

Keeping setup.py in sync

Summary

Resources and further reading

PyTest With GitHub Actions

Prerequisites

Getting started

Adding the test script

Adding the GitHub action

Summary

Resources and further reading

How PyDoc Helps Your Python Development

Getting started

Exploring PyDoc from the command line

Writing our own documented code

Exploring our documented code

Viewing PyDoc in the browser

Outputting PyDoc as HTML

Summary

Resources and further reading

Time Functionality In Python

Prerequisites

Getting started

Importing the module

Getting the local time

Formatting and parsing

Sleep

Comparing times

Summary

Resources and further reading

Mocking Python datetime In Tests With FreezeGun

Prerequisites

Getting started

Exploring FreezeGun with PyTest

Testing the is_date_before_today function

Summary

Resources and further reading

Exceptions In Python

Prerequisites

Getting started

Creating the notebook

Try/Except

Raising an error from a function

Creating and using custom errors

Summary

Resources and further reading

Datetime In Python

Prerequisites

Getting started

Testing the `is_date_before_today` function