Forem: Andrea Grandi

Python: it is now() time to migrate from utcnow()

Andrea Grandi — Wed, 17 Jul 2024 16:02:52 +0000

This morning I randomly found this post from Miguel Grinberg: It's Time For A Change: datetime.utcnow() Is Now Deprecated.

The main point is that Python utcnow() method is not timezone aware, and Python 3.12 is deprecating it. Therefore, you should start migrating your code to use now() instead.

Current state until Python 3.11

Until Python 3.11, the utcnow() method returns a datetime object, and you would use it like this:

>>> from datetime import datetime
>>> datetime.utcnow()
datetime.datetime(2024, 7, 17, 12, 17, 9, 835551)

The issue with this method is that it doesn't include timezone information, so you can't be certain if the time is in UTC or not.

(the post continues on my blog https://www.andreagrandi.it/posts/python-now-time-to-migrate-from-utcnow/ )

Cryptomator: end-to-end encrypt files in any cloud

Andrea Grandi — Sat, 13 Jul 2024 15:35:26 +0000

If you are using a cloud storage service like Google Drive, Dropbox, OneDrive, etc., you should be aware that your files are not encrypted by default. This means that the cloud provider is able to access and, in some cases, even share them with third parties. In case your cloud provider gets hacked, your files could also be exposed to the public.

To prevent this from happening, you can use a tool like Cryptomator to automatically encrypt your files before uploading them to the cloud.

How does it work?

Cryptomator is a free and open-source software that allows you to create an encrypted vault on your computer where you can store your files. Once you have added your files to the vault, Cryptomator will encrypt them using strong encryption algorithms (256 bit AES).

To be more clear, Cryptomator doesn't upload your files. Your cloud provider application does:

You write files to the Cryptomator vault -> Cryptomator encrypts them and writes them in a folder which is synced up to your cloud provider -> Your cloud provider application uploads the encrypted files to the cloud.

When you want to access your files, you simply access them from your mounted vault and Cryptomator will automatically decrypt them for you...

You can find the rest of this post here: https://www.andreagrandi.it/posts/cryptomator-end-to-end-encrypt-files-in-cloud/

Using pyenv to install Python and create a virtual environment

Andrea Grandi — Sun, 11 Oct 2020 10:21:26 +0000

A few days ago Python 3.9.0 has been released and I really wanted to test ist latest features (maybe I will do a separate post to talk about them) without messing my system with another Python version.

To manage my Python versions I've been using pyenv for a while and once configured, it's very easy to install a new Python version.

Make sure your pyenv is updated

You should have at least pyenv 1.2.21 if you want to test Python 3.9.0

In case you haven't updated it and you are using MacOS, you can do it with this command:

brew update && brew upgrade pyenv

once installed you should see the latest version:

pyenv --version
pyenv 1.2.21

Install Python 3.9.0

To install Python 3.9.0 you only need pyenv install 3.9.0:

pyenv install 3.9.0
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.9.0.tar.xz...
-> https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz
    Installing Python-3.9.0...
    python-build: use readline from homebrew
    python-build: use zlib from xcode sdk
    Installed Python-3.9.0 to /Users/andrea/.pyenv/versions/3.9.0

Set Python 3.9.0 as the local version

Now that the version you want has been installed, you need to tell pyenv you want to use it:

pyenv local 3.9.0
pyenv which python
/Users/andrea/.pyenv/versions/3.9.0/bin/python

You can find the rest of the tutorial on my blog https://www.andreagrandi.it/posts/install-python-with-pyenv-create-virtual-environment-with-specific-python-version/

How to deploy a static website to Github Pages using CircleCI

Andrea Grandi — Sun, 24 Feb 2019 21:11:31 +0000

Since I created my blog with a static pages generator, I've been using TravisCI to automate the pages build and deployment.
My desire to learn something new (we are using CircleCI at work, but I never configured it from scratch) and the recent news about
TravisCI acquisition and employees layoff, led me to think about moving to a different service.

Github Pages

Every account on Github can use a special repository to publish static pages. In my case, since I have github.com/andreagrandi, my special repository is
named github.com/andreagrandi.github.io. Once I publish my pages there, they will be accessible from https://andreagrandi.github.io.

You will need to use the master branch of the special repository directly and not the gh-pages branch which is available to each repository.

CircleCI

CircleCI is a very flexible and powerful continuous integration tool, which is also free for open source projects. As long as your static website is located on a public repository on Github, you won't have to pay anything to use it. In my case, the surce code of this website is available at https://github.com/andreagrandi/andreagrandi.it

Configuration

You can find the complete configuration at this address.
The only value you won't find is GH_TOKEN. You need to generate this token on Github, at this address: https://github.com/settings/tokens. Give it a nice description like "CircleCI deployment token", select repo scope and finally click Generate token button. This token will be used to git push...
your pages once they are built. Please remember to keep this token secret and not to publish it anywhere.

In my configuration you may notice that I'm using Pelican static websites generator, but apart from a few changes, the structure of the configuration should be very similar even if you use Jekill, Hugo etc... it doesn't really matter how you generate the pages, the deployment phase will be the same.

Deployment script

You will notice that there is a complete bash script embedded in the CircleCI configuration. This script configures git, fetches the existing andreagrandi.github.io repository,
and sync the built pages with the existing ones (this avoid creating a commit which contains all the pages so it will contain just the added content). Once the commit is made, the script will finally push the changes to the repository.

Please note: regardless of CircleCI settings, the deployment will only happens if we are pushing (or merging a pull request) to master (if [ "${CIRCLE_BRANCH}" = "master" ]; then) and it will actually commit and push pages only if there is something new to commit (if git commit -m "CircleCI build $CIRCLE_BUILD_NUM pushed to Github Pages" ; then). For example if I'm just updating something in the CircleCI configuration, which doesn't change anything in the content, the pages won't be deployed again.

Conclusion

My first impression of CircleCI is that is faster than TravisCI and this means that I can publish my content more quickly. The possibility of using Docker containers as base image is really powerful and in more complex scenarios we can reproduce the building environment locally on our machine. If you have any advices about how to improve my build script, feel free to leave a comment here.

Note: this post first appeared on my personal blog at https://www.andreagrandi.it/posts/how-to-deploy-static-website-github-pages-circleci/

PyCon X Italy: submit a talk for the Italian Python conference (Florence May 2nd - 5th)

Andrea Grandi — Thu, 03 Jan 2019 13:44:30 +0000

Even this year, the Italian Python conference will be organised in Florence and will take place between May 2nd and May 5th included.

It's an amazing opportunity to meet a lot of Python developers, listen to very interesting talks and visit one of the best cities in the world (I may be slightly biased on this last point :P )!

Subjects range from pure Python, PyData, Machine Learning, web, DevOps, security and community related talks and there will be a good number (usually they are the majority) of talks available in english language.

This event wouldn't be possible without our wonderful speakers: not just experts but especially first time speakers are warmly encouraged to submit a proposal for the conference.

If you are interested, you can find all the information on the official website https://www.pycon.it/en/call-for-proposals/

Please hurry up because the deadline is January 6th!

Installing Python and virtualenv on OSX

Andrea Grandi — Wed, 19 Dec 2018 15:53:18 +0000

Every time I need to install Python on OSX or whenever a colleague asks for help, I have to search fo the most updated instructions on Google, and every time I find different ways of doing the exact same thing.

Tired of this, I decided to write down my own notes. Please note that I don't claim this to be the best way of installing Python on OSX. It works fine for me so use it at your own risk.

Requirements

To follow these instructions you need to at least have installed brew on OSX. Please follow the instructions on the official website: https://brew.sh

Installing Python 3.7.x and Python 2.7.x

Even if I strongly suggest to start every new project with Python 3 (since Python 2 will only be supported until the end of 2019), there may be use cases when version 2 is still required, so I will give you the instructions to install both.

Installing Python 3.7.x

brew install python

This will install Python 3 by default.

....

You can find the rest of the post on my blog: https://www.andreagrandi.it/posts/installing-python-and-virtualenv-on-osx/

Why I mentor on Exercism.io

Andrea Grandi — Sun, 04 Nov 2018 14:00:00 +0000

Exercism (https://exercism.io) is a platform that has been created to help people improving their coding skills, thanks to the volunteers mentoring the students. There are tracks for almost all the popular languages and each track has coding test that the students can download, solve offline using their preferred editor, and test the solution against the provided unit tests. Once the solution is ready to be reviewed (or even if it's not complete but the student needs help), it can be submitted to the website and it will go in a queue where the first available mentor will pick it and start mentoring.

The service is free to use for all the students and the mentors are all volunteers (this doesn't mean that the platform doesn't have any costs. If you are curious about the resources needed to keep the platform alive, you can give a look at this answer on Reddit.

When I found out about the platform, I decided to use it (as student) to improve my Go coding skills. I must say that I've been learning a lot from the mentors and some of them are putting a lot of effort to give you all the possible advices to improve your coding style. In a single exercise once, I learnt at least five things about Go I didn't know before!

I've been a Python developer (professionally) for the last 5 years, but I've never considered myself an "expert". I decided to give it a try with mentoring, because I felt I wanted to give something back to the community, so I registered as mentor too and started mentoring in the Python track.

The first surprise has been that mentoring other students, I was probably learning more than how much I was teaching. First of all, once you already know how to solve a problem, it's always interesting to look at other possible solutions. I've found sometimes that students were providing better (more concise and readable) solutions than mine. Last but not least, before advising someone about conding style or a more idiomatic solution, I always double check things from different sources. There is nothing wrong making mistakes, especially if you are learning... but it would be damaging for the student if I was teaching them something wrong, so I need to be sure about what I say. This of course makes me study, even the basic things, again and again and at the end of the day, my skills are better too.

Once you join the mentors group, you are invited to a private Slack where you can count on the help of other mentors (we have channels for each track/language) or ask questions. So, if you are not sure about something, you can always ask around.

If my story and experience convinced you, Exercism is looking for more mentors! The more we have available, the less time the students have to wait in a queue to be mentored. You can find all the instructions at this address https://mentoring.exercism.io

Note: this post appeared first on my blog at https://www.andreagrandi.it/posts/why-i-mentor-on-exercism/

Creating a production ready API with Python and Django Rest Framework - part 1

Andrea Grandi — Tue, 01 May 2018 13:09:08 +0000

The aim if this tutorial is to show how to create a production ready solution for a REST API, using Python and Django Rest Framework. I will show you how to first create a very basic API, how to handle the authentication and permissions and I will cover deployment and hosting of images. The full source code of the tutorial is available at: https://github.com/andreagrandi/drf-tutorial

Summary of the complete tutorial

Create the basic structure for the API
Add Authentication and POST methods
Handling details and changes to existing data
Testing the API
Switching from Sqlite to PostgreSQL
Hosting the API on Heroku
Add an Image field and save images to S3

Create the basic structure for the API

For this tutorial I will assume you have correctly installed at least Python (I will use Python 2.7.x), virtualenv and virtualenvwrapper on your system and I will explain how to create everything else step by step.

Note: at the time of writing, the tutorial has been based on Django 1.10.1 and Django Rest Framework 3.4.7

Creating the main project structure

mkdir drf-tutorial
mkvirtualenv drf-tutorial
cd drf-tutorial
pip install django djangorestframework
django-admin.py startproject drftutorial .
cd drftutorial
django-admin.py startapp catalog

Data Model

We will create the API for a generic products catalog, using a very simple structure (to keep things simple). Edit the file catalog/models.py adding these lines:

from __future__ import unicode_literals
from django.db import models


class Product(models.Model):
    name = models.CharField(max_length=255)
    description = models.TextField()
    price = models.DecimalField(decimal_places=2, max_digits=20)

You can find the rest of the tutorial on my blog https://www.andreagrandi.it/2016/09/28/creating-production-ready-api-python-django-rest-framework-part-1/

Using Python ipdb from Jupyter

Andrea Grandi — Wed, 25 Apr 2018 09:40:08 +0000

(this blog post originally appeared on my blog https://www.andreagrandi.it/posts/using-python-ipdb-from-jupyter/ )

If we try to use the usual ipdb commands from a Jupyter (IPython notebook)

import ipdb; ipdb.set_trace()

we will get a similar error:

--------------------------------------------------------------------------
MultipleInstanceError                     Traceback (most recent call last)
<ipython-input-1-f2b356251c56> in <module>()
    1 a=4
----> 2 import ipdb; ipdb.set_trace()
    3 b=5
    4 print a
    5 print b

/home/nnn/anaconda/lib/python2.7/site-packages/ipdb/__init__.py in <module>()
    14 # You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
    15 
---> 16 from ipdb.__main__ import set_trace, post_mortem, pm, run, runcall, runeval, launch_ipdb_on_exception
    17 
    18 pm                       # please pyflakes

/home/nnn/anaconda/lib/python2.7/site-packages/ipdb/__main__.py in <module>()
    71         # the instance method will create a new one without loading the config.
    72         # i.e: if we are in an embed instance we do not want to load the config.
---> 73         ipapp = TerminalIPythonApp.instance()
    74         shell = get_ipython()
    75         def_colors = shell.colors

/home/nnn/anaconda/lib/python2.7/site-packages/traitlets/config/configurable.pyc in instance(cls, *args, **kwargs)
    413             raise MultipleInstanceError(
    414                 'Multiple incompatible subclass instances of '
--> 415                 '%s are being created.' % cls.__name__
    416             )
    417

MultipleInstanceError: Multiple incompatible subclass instances of TerminalIPythonApp are being created.

The solution is to use Tracer instead:

from IPython.core.debugger import Tracer
Tracer()()

Source: http://stackoverflow.com/questions/35613249/using-ipdb-to-debug-python-code-in-one-cell-jupyter-or-ipython

Getting latest Ubuntu AMI with Terraform

Andrea Grandi — Tue, 24 Apr 2018 15:21:05 +0000

This post was originally published on my own blog: https://www.andreagrandi.it/posts/getting-latest-ubuntu-ami-with-terraform/

When we need to create an EC2 resource on AWS using Terraform, we need to specify the AMI id to get the correct image.

The id is not easy to memorise and it changes depending on the zone we are working one. On every new release the id changes again.

So, how can we be sure to get the correct ID for our region, of the latest image available for a given Linux distribution?

Getting latest Ubuntu AMI id

In this example I will show how to get the ID for the latest version of Ubuntu 16.04 server, for the London region and create an EC2 instance using that ID.

variable "aws_region" { default = "eu-west-2" } # London

provider "aws" {
    region = "${var.aws_region}"
    access_key = "youraccesskey"
    secret_key = "yoursecretkey"
}

data "aws_ami" "ubuntu" {
    most_recent = true

    filter {
        name   = "name"
        values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*"]
    }

    filter {
        name   = "virtualization-type"
        values = ["hvm"]
    }

    owners = ["099720109477"] # Canonical
}

resource "aws_instance" "web" {
    ami           = "${data.aws_ami.ubuntu.id}"
    instance_type = "t2.micro"

    tags {
        Name = "HelloUbuntu"
    }
}

output "image_id" {
    value = "${data.aws_ami.ubuntu.id}"
}

After we have initialised our script using terraform init if we run it, we will get the AMI id and the instance will be created:

➜  example1$: terraform apply
data.aws_ami.ubuntu: Refreshing state...
aws_instance.web: Creating...
    ami:                          "" => "ami-03998867"
    associate_public_ip_address:  "" => "<computed>"
    availability_zone:            "" => "<computed>"
    ebs_block_device.#:           "" => "<computed>"
    ephemeral_block_device.#:     "" => "<computed>"
    instance_state:               "" => "<computed>"
    instance_type:                "" => "t2.micro"
    ipv6_address_count:           "" => "<computed>"
    ipv6_addresses.#:             "" => "<computed>"
    key_name:                     "" => "<computed>"
    network_interface.#:          "" => "<computed>"
    network_interface_id:         "" => "<computed>"
    placement_group:              "" => "<computed>"
    primary_network_interface_id: "" => "<computed>"
    private_dns:                  "" => "<computed>"
    private_ip:                   "" => "<computed>"
    public_dns:                   "" => "<computed>"
    public_ip:                    "" => "<computed>"
    root_block_device.#:          "" => "<computed>"
    security_groups.#:            "" => "<computed>"
    source_dest_check:            "" => "true"
    subnet_id:                    "" => "<computed>"
    tags.%:                       "" => "1"
    tags.Name:                    "" => "HelloUbuntu"
    tenancy:                      "" => "<computed>"
    volume_tags.%:                "" => "<computed>"
    vpc_security_group_ids.#:     "" => "<computed>"
aws_instance.web: Still creating... (10s elapsed)
aws_instance.web: Still creating... (20s elapsed)
aws_instance.web: Still creating... (30s elapsed)
aws_instance.web: Creation complete (ID: i-0f58f8bd55b3a7e38)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

image_id = ami-03998867

That's all we need to spin up an EC2 instance on AWS using latest Ubuntu image available.