Forem: Ian Cleary (he/him)

Test and Publish a Python Package to PyPi using Poetry and GitHub Actions

Ian Cleary (he/him) — Sat, 29 Aug 2020 19:46:51 +0000

My Workflow

I created a python package cookiecutter showcasing two GitHub actions workflow files I've seen and used successfully to test and publish a Python package to PyPi using GitHub actions.

The detailed explanation and python cookiecutter wrapper are the novel additions. 🎉 I found it difficult to just inspect open source projects work and put the pieces together.

So I hope this helps you (no matter your previous experience).

Please let me know what isn't clear!

Getting started with this template

There are two workflow files, each with their named purpose:

Test
Publish

If you have an existing python project or are starting a new one, you can use your GitHub username and your project name as inputs into my Cookiecutter template!

First, what is cookiecutter?

Cookiecutter

A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.

Documentation: https://cookiecutter.readthedocs.io
GitHub: https://github.com/cookiecutter/cookiecutter
PyPI: https://pypi.org/project/cookiecutter/
Free and open source software: BSD license

We are proud to be an open source sponsor of PyCon 2016.

Features

Did someone say features?

Cross-platform: Windows, Mac, and Linux are officially supported.
You don't have to know/write Python code to use Cookiecutter.
Works with Python 3.6, 3.7, 3.8, 3.9 and PyPy3.
Project templates can be in any programming language or markup format Python, JavaScript, Ruby, CoffeeScript, RST, Markdown, CSS, HTML, you name it You can use multiple languages in the same project template.
Simple command line usage:

# Create project from the cookiecutter-pypackage.git repo template
# You'll be prompted to enter values.
# Then it'll create your Python package in the current working directory,
# based on those values.
$

…

View on GitHub

Make sure you have Python3 installed!

Instructions to use iancleary/pypackage

python3 -m pip install cookiecutter
# you can add the flag `--user` if you'd like

$ cookiecutter https://github.com/iancleary/pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
$ cookiecutter gh:iancleary/pypackage

Features

Poetry (virtual environment and publish to PyPi, all with one tool)
black (linting/formatter)
autoflake (removing unused packages)
isort (dependency organization)
mypy (static type checking)
pytest (including test coverage)
pre-commit (hooks on commit)
GitHub Actions for CI/CD
mkdocs for documentation (with material theme)

Only Python 3.6+ is supported as required by the black, pydantic packages

Submission Category: Maintainer Must-Haves

Yaml File or Link to Code

ARCHIVED, pypackage

I no longer with to maintain this cookiecutter, please see br3ndonland/template-python's template. While not a cookiecutter at the time of this writing, it has the tooling I was going for :)

Rapid python package setup with modern venv, dependencies, testing, docs, and CI

The Basic Idea

This is a template module collecting many utilities I have liked from other projects, to serve as a personal reference.

Getting started with this template

pip install --user cookiecutter
$ cookiecutter https://github.com/iancleary/pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
$ cookiecutter gh:iancleary/pypackage

Features

Poetry (virtual environment and publish to PyPi, all with one tool)
black (linting/formatter)
autoflake (removing unused packages)
isort (dependency organization)
mypy (static type checking)
pytest (including test coverage)
pre-commit (hooks on commit)
GitHub Actions for CI/CD
mkdocs for documentation (with material theme)

Only Python 3.6+ is supported as…

View on GitHub

Test

name: Test

on:
  push:
  pull_request:
    types: [opened, synchronize]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8]
      fail-fast: false

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python-version }}
      - name: Get full python version
        id: full-python-version
        run: echo ::set-output name=version::$(python -c "import sys; print('-'.join(str(v) for v in sys.version_info))")
      - name: Install poetry
        run: |
          curl -fsS -o get-poetry.py https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
          python get-poetry.py -y
          echo "::set-env name=PATH::$HOME/.poetry/bin:$PATH"
      - name: Configure poetry
        run: poetry config virtualenvs.in-project true
      - name: Set up cache
        uses: actions/cache@v1
        id: cache
        with:
          path: .venv
          key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
      - name: Ensure cache is healthy
        if: steps.cache.outputs.cache-hit == 'true'
        run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
      - name: Install Dependencies
        run: poetry install
      - name: Test
        run: poetry run bash scripts/test.sh
      - name: Upload coverage
        uses: codecov/codecov-action@v1

Publish

Repo Secrets

Go to your repo settings and add a PYPI_TOKEN environment variable:

name: Publish

on:
  release:
    types:
      - created

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: "3.7"
      - name: Install poetry
        run: |
          curl -fsS -o get-poetry.py https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
          python get-poetry.py -y
          echo "::set-env name=PATH::$HOME/.poetry/bin:$PATH"
      - name: Configure poetry
        run: poetry config virtualenvs.in-project true
      - name: Set up cache
        uses: actions/cache@v1
        id: cache
        with:
          path: .venv
          key: venv-${{ runner.os }}-${{ steps.full-python-version.outputs.version }}-${{ hashFiles('**/poetry.lock') }}
      - name: Ensure cache is healthy
        if: steps.cache.outputs.cache-hit == 'true'
        run: poetry run pip --version >/dev/null 2>&1 || rm -rf .venv
      - name: Install Dependencies
        run: poetry install
      - name: Publish
        env:
          PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
        run: |
          poetry config pypi-token.pypi $PYPI_TOKEN
          bash scripts/publish.sh

That's it!

When you make a release on GitHub, the publish workflow will run and deploy to PyPi! 🚀🎉😎

Additional Resources / Info

I have used the cookiecutter successfully in my FastRF project

fastrf

FastRF is an application to create and track key system metrics

The Basic Idea

Fastrf is a web server that makes it easy to create radio frequency (RF) gain line ups 📡, link budgets 🌎📡🛰️, and simulation models 🧪🧮 to caputre the performance of a RF system.

This project will stand on the shoulders of some giants:

Installing fastrf

Install the latest release:

pip install fastrf

Documentation

Documentation is hosted at fastrf.org.

Dependencies

The pyproject.toml file is used by poetry to install dependencies into a virtual environment.

Poetry Command Line Interface Documnetation: https://python-poetry.org/docs/cli/

Only Python 3.6+ is supported as required by the black, fastapi, pydantic packages

This package was created with the iancleary/pypackage cookiecutter.

View on GitHub

Source Link for Banner Image: https://medium.com/swlh/automate-python-testing-with-github-actions-7926b5d8a865

Automating your desktop with Ansible

Ian Cleary (he/him) — Sun, 03 May 2020 00:45:17 +0000

Hello there,

First off, I hope you are doing well 😊.

When it comes to my laptop set up, I care about not having to think too much about my setup between machines. Especially when I want to get right to work, gaming, etc.

For that reason, it is very comfortable for me to have the same base configuration on each machine (look, feel, keyboard shortcuts, core software, etc.).

When I learned about infrastructure as code concept, I thought it would be a fun learning experience to try it on my desktop.

Here is my story of the ways I found to help configure and maintain my desktop configuration.

Ansible Newbie: A true underdog story

I found a cool bash script on the internet (I didn't understand most of it) 🤷
I modified cool bash script for my needs and it worked! 🚀
I learned about Makefiles. 😄
I proceeded to do everything with Makefiles. 🤩🤩🤩
I got tired of running successive Make targets over and again 😐.
I used Ansible at work and decided to take a course to learn more about it. 🤓
I put off actually taking the plunge to use Ansible. 🕒🕕🕘🕛
I finally took the plunge and decided to use Ansible for my configuration. ✔️

Supported Ubuntu LTS Versions

Ubuntu 18.04
Ubuntu 20.04

There are no plans to ensure non-LTS versions are supported.
Software support is LTS version dependent.

Why script this?

While these changes don't take long to do, we could have a script check and assert that they are set the way you want, and change them if they are different than your desired state.

Lets get into an example to see why I went this route.

GNOME Settings and configuration

If you have ever install GNOME Tweak Tools, you've been exposed to some of these settings. The Settings application exposes some of them as well.

The settings are organized in a hierarchical manner



- name: GNOME Preferences - Shell - Attach Modal Dialogues
  dconf:
    key: "/org/gnome/shell/overrides/attach-modal-dialogs"
    value: "false"
    state: present

- name: GNOME Preferences - Nautilus - Clock Format
  dconf:
    key: "/org/gnome/desktop/interface/clock-format"
    value: "'24h'"
    state: present

The two tasks above allow modal dialogues to be moved independently of the application (so you can see what is behind the dialog) and change the clock to 24 hour time.

The underlying tools that manage these settings are dconf and gsettings. The dconf-editor is a graphical application for editing these items. gsettings is a way to get, set, and query the schema of entries on the command line.

The two Ansible tasks use dconf to set the value for each key. The state: present flag tells ansible that we want the key/value pair to be present on our system.

What else can be configured and source controlled

Core Software

This isn't exactly the list of Ansible roles,
but below is a list of what the software installs.

ansible,
cherrytree (like OneNote for Linux)
Breaktimer,
Visual Studio Code,
kite AI Autocompletion and Docs for Python, runs locally
VSCodium,
carbon-cli for beautiful images of my code - right inside my terminal,
Chromium,
docker,
docker-compose,
Flameshot,
Flameshot keybinding to the Print Screen key using gsettings,
Firefox,
Flatpak,
gh GitHub CLI
Nordvpn,
nodejs,
Peek animated GIF screen recorder,
Postman for API testing,
Protonmail-bridge https://protonmail.com/bridge/install,
python3.6,
python3.7,
python3.8,
Signal Desktop,
Slack,
Snap,
Spotify,
Stacer https://github.com/oguzhaninan/Stacer,
Standard Notes,
Steam,
Sublime Text,
Telegram,
TickTick using nativefier,
Timeshift https://github.com/teejee2008/timeshift,
Tresorit,
yarn,
zsh

Makefile help out

I use Makefiles to help run existing Ansible roles or scripts to make editing/updating my configuration easier.

More details

To give you a sense of more details around what some of the make targets do, below is my group_vars/all.yml file at the time of this article.

This file sets global variables for all hosts I want to run the playbook on. In my case, this generally is just the local machine.



---
docker_compose_version: 1.25.4
flameshot_version: 0.6.0
flatpak_applications:
  - "com.uploadedlobster.peek"
  # - "com.github.alainm23.planner" # not used
  - "com.valvesoftware.Steam"
  - "org.gnome.Evolution"
  - "ch.protonmail.protonmail-bridge"
  - "org.libreoffice.LibreOffice"
gh_version: 0.7.0
jetbrains_mono_version: 1.0.3
protonmail_bridge_version: 1.2.6-1
nodejs_version: "12.x"
nodejs_yarn_global_packages:
  - name: "@vue/cli"
  - name: "@gridsome/cli"
  - name: "nativefier"
  - name: "markdownlint-cli"
  - name: "carbon-now-cli"
snaps_chat:
  - "signal-desktop"
  - "telegram-desktop"
snaps_chat_classic:
  - "slack"
snaps_common:
  - "snap-store"
  - "breaktimer"
  - "cherrytree"
  - "drawio"
  # https://github.com/jgraph/drawio-desktop/issues/138,
  # snap 12.6.5 > flatpak 12.4 (as of 2020-02-28)
  - "spotify"
  - "ncspot"
  - "standard-notes"
snaps_development:
  - "postman"
snaps_development_classic:
  - "code"
  # - "codium" # Code without telemetry
  - "sublime-text"
snaps_web_browsers:
  - "chromium"
  - "firefox"
snaps_web_browsers_plugs:
  - plug: "home"
    app: "chromium"
code_extensions:
  ## Ansible
  - vscoss.vscode-ansible

  ## Docker and Remote Development
  - ms-azuretools.vscode-docker
  - ms-vscode-remote.remote-containers
  - ms-vscode-remote.remote-ssh
  - ms-vscode-remote.remote-ssh-edit
  # - ms-vscode-remote.remote-wsl
  - ms-vscode-remote.vscode-remote-extensionpack

  ## General Development
  - christian-kohler.path-intellisense
  - vscode-icons-team.vscode-icons
  - riccardoNovaglia.missinglineendoffile
  - shardulm94.trailing-spaces
  # - Shan.code-settings-sync

  ## Git Utilities
  - eamodio.gitlens
  - donjayamanne.githistory

  ## Markdown Linting
  - DavidAnson.vscode-markdownlint

  ## Python Development
  - ms-python.python
  - himanoa.Python-autopep8
  - njpwerner.autodocstring
  - wholroyd.jinja
  # Pyright Attribution to tiangolo and florimondmanca
  # https://twitter.com/tiangolo/status/1252891149708275713?s=21
  # > Disabled type checking in the extension settings as I rely on mypy,
  # > but autoimport and unused import colouring alone make it a
  # > 100% worthwhile companion on VSCode
  - ms-pyright.pyright

  ## Spellchecking
  - streetsidesoftware.code-spell-checker

  ## Travis-CI
  # - felixrieseberg.vsc-travis-ci-status # has issues with status resolution

  ## Vue.js Development
  - octref.vetur
  - dbaeumer.vscode-eslint
  - pranaygp.vscode-css-peek

  ## YAML
  - redhat.vscode-yaml # doesn't allow hostname in .travis.yml
stacer_version: 1.1.0
zsh_theme: robbyrussell
zsh_plugins:
  - ansible
  - aws
  - docker
  - docker-compose
  - git
  - pipenv

Existing Machine

For a walk through of my process on existing machine, please visit
https://ansible-desktop.iancleary.me/makefile.

New Machine Setup

For a new machine, I run the following command
to set up my computer:

I like to do this during the initial configuration of the machine.



wget -qO- \
https://github.com/iancleary/ansible-desktop/raw/main/run.sh | \
bash

This will prompt you for your sudo password
for the bash script and then once later for
ansible's "BECOME PASSWORD" prompt.

Voila! 🎉🎉🎉

Customization

If you want to edit the variables, hit CTRL + C at ansible's "BECOME PASSWORD" prompt.

The script created two extra files that ensure that the ansible-playbook will on your host name.

Note: Both files are intentionally not version controlled.
This allows hostname specific group_vars relative to the playbook.

a private .inventory file



#.inventory
[$(hostname)]
127.0.0.1

a group_vars file



# group_vars/$(hostname)/all.yml
---
# You can copy and modify variables over from ../all.yml

Then run make all

Voila (with your edits)! 🚀🚀🚀

Documentation

Detailed documentation is hosted by Netlify at https://ansible-desktop.iancleary.me.

Thanks

I hope you enjoyed reading about my desktop configuration and feel free to use/modify/share.

iancleary/ansible-desktop/ is on GitHub and licensed GPL-3.0.

If you like it give it a star ⭐ and let me know if you have any questions!

Especially if you a suggestion for cool application or piece of software you use 😎😎😎.

Cheers,
Ian

Edit: Changed default branch from master to main

An introduction to CI/CD with Travis CI and Python

Ian Cleary (he/him) — Sun, 08 Dec 2019 22:54:16 +0000

Automate the boring stuff with Continuous Integration (CI), Continuous Deployment (CD)

I first learned about Travis-CI through venturing out into GitHub to checkout several open source projects. I've seen how they have continuous integration workflows set up test their code on every commit, branch, and pull request.

That seemed so convenient so I went off to try it myself.
I've documented what I've learned below to help get you started!

Choosing a continuous integration service

What to consider when creating a Python3 CI/CD pipeline

Python Community on GitHub - support for all python3 🐍 versions.
Choose your testing framework - Use it with any popular testing framework: pytest, unittest, etc.
Open vs. closed source - I recommend whenever possible to choose an open source option. The below providers generally offer zero cost options for new open source projects with reasonable build 🛠 constraints. If you go closed source, you’re going to needs to fork over some 💵.

Common providers

Travis-CI - my first and has served me well
Circle CI - seems to have a little saner syntax and finer configuration relative to Travis-CI
Gitlab - leans open source where the others are closed source
GitHub Actions - newer, native to GitHub, and offers Linux, Windows, and MacOS builds.

My learning started with Travis-CI and am using it since I am familiar with it. I haven't done a detailed comparison of the tools and it fits the needs of simpler python CI/CD projects. I want to look into GitHub actions, now that it’s coming out of Beta.

You are free to use what you prefer, feel is right, or would like to start with. This article is not a comparison, but rather an introduction into what the process looks like with one CI service.

Start small and build up what you learn. You will make progress, run into challenges, figure some out. Remember to sleep!!!!

Photo credit: 4geeksacademy.co/feelings-learning-coding/

It's an exhilirating, rewarding, and sometimes stressful day to day. 😬

There isn't a wrong option on your path to learning more about these tools and process. Where you start doesn't have to be where you are in the future. My recommendation is that you pick one, don't over think it, and get it up and working!

Put your work out there and connect with others. You don't have to create something from scratch. You can help your favorite existing projects set up or improve their CI/CD pipeline.
It's better when we build together. 🌎🌈🌍🚀🌏

Knowledge compounds and you'll marvel at where you're at soon. 🗺️ 🚀 💻

Prerequisites

You should have basic knowledge about Github, Python, testing frameworks, and how to use the Terminal. Knowledge of Pipenv (Pipfiles) and pytest is a plus, but not required. CI/CD tools are a great way to learn both. We will set up a Travis-CI account as well.

YAML Basics

Most CI services interact with your repositories using a YAML configuration file.

In case you need a tune up on YAML, here is a recommended link:

YAML Syntax, from Red Hat Ansible

Note: Ansible uses the YAML format, and is is another useful tool. A more detailed guide is an encouraged topic for another time.

Travis-CI YAML Example

Here is a full example .travis.yml file from one of my projects. We will break down each section.

# .travis.yml

dist: xenial

language: python

cache: pip

python:
    - "3.6"
    - "3.7"
    - "3.8"
    - "nightly"

matrix:
    allow_failures:
        - python: "nightly"

install:
    - pip install pipenv --upgrade-strategy=only-if-needed
    - pipenv install --dev

script:
    - bash scripts/test.sh

after_script:
    - bash <(curl -s https://codecov.io/bash)

Let's explain each section

dist: xenial

dist is where the Ubuntu Release codename is specified. xenial is Ubuntu 16.04 LTS codename - see: http://releases.ubuntu.com/ for a full list.

This specifies the base operating system used for the rest of workflow.

language: python

language is the programming language used. This tutorial uses python, but it very well could be cpp, go, rust, etc.

cache: pip

cache allows for python package versions to be stored between runs, to speed up sequential builds. Cache can apply to more than just python packages.

python:
    - "3.6"
    - "3.7"
    - "3.8"
    - "nightly"

python, given the above language specification, is a key for a sequence of python versions to perform builds against. Generally most CI tools use the latest bug release version for each minor version. The build logs will tell you the specific versions.

matrix:
    allow_failures:
        - python: "nightly"

matrix allows for modifications in the above build sequence. In this case, the allow failures key is specifies a reference to the python sequence above, and has the value of "nightly", meaning that that version is allowed to fail. Think of it as early detection and future python development. *Note: there are also development versions of each python version, such as "3.8-dev". These are used when python version are in release candidate or beta stages.

install:
    - pip install pipenv --upgrade-strategy=only-if-needed
    - pipenv install --dev

install is the first section that is performed inside each of the build targets (virtual environments for each of the python versions above).

Each item in the sequence is executed in order!

Pipenv and Pipfiles

In this example, we are using pipenv and a Pipfile.

Most Pipfiles have production dependencies and development dependencies. You will often find the packages required to run test cases or build documentation under the dev-packages section. Below is an example:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
mypy = "*"
black = "*"
jupyter = "*"
better-exceptions = "*"
pytest = "*"
pytest-cov = "*"
isort = "*"
flit = "*"
autoflake = "*"
mkdocs = "*"
mkdocs-material = "*"

[packages]
pydantic = "==0.32.2"

The command pip install --dev installs all packages under [packages] and [dev-packages]. pip install would only install the packages under [packages].

Requirement.txt files

Note: It is also common to use pip with a requirements.txt file.

pip install -r requirements.txt

Python.org now recommends pipenv as part of its general tool recommendations.

Optional aside about different virtual environment options

Pipenv has the advantage of providing a mechanism to install your packages into a virtual environment without another package. There are many alternative - one simple alternaive is venv.

A newer alternative is poetry (https://poetry.eustace.io/). poetry aims to facilitate what both requirements.txt and Pipfiles do not bundle, and that is released dependencies when deploying to PyPi. You will typically see flit and a pyproject.toml file, or a setup.py file with redundant depedency information. That redundancy is what poetry is aiming to improve upon.

Now lets help you get your CI/CD setup

1. Get setup to install my python package project template

Ensure you have a compatible python 🐍 environment on your computer.


python3 --version
python3 -m pip --version
python3 -m pytest --version

If you are missing any of the above, you can add the packages to your main python install.

Recommended reading on Python main install vs. altinstall.
If you only have or want one python3 version on your machine, there is no harm to run the following commands. There are nuances between Ubuntu versions (16.04 vs. 18.04 vs. latest) on which python3 version is the default install.


sudo apt-get update
sudo apt-get install python3-pip
sudo apt-get install python3-pytest

Now that we have confirmed your base python setup, let’s go ahead and ensure the pipenv and cookiecutter python packages are installed.


python3 -m pip install --user pipenv
python3 -m pip install --user cookiecutter

2. Create a Python project and setup the Pipenv virtual environment

# make sure your path finds --user installs

## add `export PATH="$HOME/.local/bin:$PATH"` 
## to your ~/.bashrc, ~/.zshrc file on linux

cookiecutter https://github.com/iancleary/pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
cookiecutter gh:iancleary/pypackage

Fill out the wizard!

Use cd new-directory to change into the new directory you just created (replace new-directory)

Run pipenv to display it's options. Take a look at what’s offered.

Next, run pipenv install --dev to install the production and development dependecies specified in the Pipfile.

Run pipenv shell to load the virtual environment.

3. Run the tests locally and make sure they pass

I recommend you open up VS code or your text editor of choice to view the folder structure

Execute ./scripts/tests.sh from within the directory's Pipenv.

This command executes a bash script that does several things:

runs pytest to check test cases and check test coverage
checks formatting with the black package
runs static type checking against your code base with mypy
checks import sorting from standard lib, your application, and custom packages

These are all things that should be relegated to scripts to allow you to free your mind from simple formatting and other draining tasks. Let it do that work for you!

4. Push the directory to a remote git repo

I recommend you follow the guide on the Github Help page:

Adding an existing project to GitHub using the command line

5. Connect Travis-CI to the repo

Again, the documentation of both GitHub and Travis-CI are great to follow for this step.

Head over to Travis-CI and sign in with your GitHub account:

https://help.github.com/en/enterprise/2.16/admin/developer-workflow/continuous-integration-using-travis-ci

Once you login to Travis-CI and Enable GitHub Access, you'll want to configure Travis from your Github Settings page.

6. Now we are configured, let's start a build

Make a simple change to the repo, either a commit, or open a new branch to start a build!

Grab your 🍿 and let's watch the CI process start and complete! 😎

It’s very cool to watch the process.
The key to remember is the repeatability of the test suite.

Once complete, you will have your new project added to the Travis-CI dashboard. The three red boxes below are:

The connected repo that is currently selected.
The action that started a build (commit or pull request)
The build matrix visualized, with links to view more details.

7. Happy coding with CI/CD pipelines 🎉🙌

Your setup should give you and others confidence that the code will work under those conditions. If it’s an open source repo, there is no taking your word for it. The logs are right there!

You will not want to go back to your life before it!

It is such a quality improvement!

Where to go from here?

Start offloading and automating your testing, deployments, and other activities. There is so many opportunites to tie in different hooks or features, this is just the beginning.

Some examples of what else can you automate

Pypi publishing

I haven't done much in this space yet, but I've seen other projects successfully use the flit python package.

If that is your next step, checkout flit. I know there are some other good tutorials out there to get started inside and outside of a CI/CD pipeline.

Jupyter Notebook image deployment with CI/CD to Docker Hub

I've been using Jupyter notebooks and Docker more and more recently. I significantly improved the maintenance model of my personal image by setting up an automated testing and deployment pipeline using Travis CI. Check out my personal Jupyter notebook repository!

Checkout the Docker image is iancleary/personal-notebook.

bash docker pull iancleary/personal-notebook

Amazon EC2 instance deployments

I also am in the process of learning how to leverage continuous deployment onto Amazon EC2 instances.

You can find my notes on connecting AWS CodeDeploy and Travis CI using a Flask project.

This example is a proof of concept, and I'm excited to use this workflow more in the future.

References

I learned a lot from https://github.com/tiangolo/fastapi and credit them as authors within my cookicutter pypackage AUTHORS.md. This is not only nice to do, but is required by many open source licenses.

Forem: Ian Cleary (he/him)

Test and Publish a Python Package to PyPi using Poetry and GitHub Actions

My Workflow

Getting started with this template

Cookiecutter

cookiecutter / cookiecutter

A command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, VueJS projects.

Cookiecutter

Features

Instructions to use iancleary/pypackage

Features

Submission Category: Maintainer Must-Haves

Yaml File or Link to Code

iancleary / pypackage

Cookiecutter python package using Poetry, mypy, black, isort, autoflake, pytest, mkdocs, and GitHub Actions

ARCHIVED, pypackage

The Basic Idea

Getting started with this template

Features

Test

Publish

Repo Secrets

Additional Resources / Info

iancleary / fastrf

RF Design Server

fastrf

The Basic Idea

Installing fastrf

Documentation

Dependencies

Automating your desktop with Ansible

Ansible Newbie: A true underdog story

Supported Ubuntu LTS Versions

Why script this?

GNOME Settings and configuration

What else can be configured and source controlled

Core Software

Makefile help out

More details

Existing Machine

New Machine Setup

Customization

Documentation

Thanks

An introduction to CI/CD with Travis CI and Python

Automate the boring stuff with Continuous Integration (CI), Continuous Deployment (CD)

Choosing a continuous integration service

What to consider when creating a Python3 CI/CD pipeline

Common providers

Prerequisites

YAML Basics

Travis-CI YAML Example

Let's explain each section

Pipenv and Pipfiles

Requirement.txt files

Optional aside about different virtual environment options

Now lets help you get your CI/CD setup

1. Get setup to install my python package project template

2. Create a Python project and setup the Pipenv virtual environment

3. Run the tests locally and make sure they pass

4. Push the directory to a remote git repo

5. Connect Travis-CI to the repo

6. Now we are configured, let's start a build

7. Happy coding with CI/CD pipelines 🎉🙌

Where to go from here?

Some examples of what else can you automate

Pypi publishing

Jupyter Notebook image deployment with CI/CD to Docker Hub

Amazon EC2 instance deployments

References