Forem: Alvaro Cavalcanti

Setting Up a Python Remote Interpreter Using Docker

Alvaro Cavalcanti — Fri, 08 Nov 2019 13:28:19 +0000

Why a Remote Interpreter instead of a Virtual Environment?

A well-known pattern in Python (and many other languages) is to rely on virtual environment tools (virtualenv, pyenv, etc) to avoid the SnowflakeServer anti-pattern. These tools create an isolated environment to install all dependencies for any given project.

But as of today there's an improvement to that pattern, which is to use Docker containers instead. Such containers provide much more flexibility than virtual environment, because they are not limited to a single platform/language, instead they offer a fully-fledged virtual machine. Not to mention the docker-compose tool where one can have several containers interacting with each other.

This article will guide the reader on how to set up the two most used Python IDEs for using Docker containers as remote interpreters.

Pre-requisites

A running Docker container with:

A volume mounted to your source code (henceforth, /code)
SSH setup
SSH enabled for the root:password creds and the root user allowed to login

Refer to this gist for the necessary Docker files.

PyCharm Professional Edition

Preferences (CMD + ,) > Project Settings > Project Interpreter
Click on the gear icon next to the "Project Interpreter" dropdown > Add
Select "SSH Interpreter" > Host: localhost, Port: 9922, Username: root > Password: password > Interpreter: /usr/local/bin/python, Sync folders: Project Root -> /code, Disable "Automatically upload..."
Confirm the changes and wait for PyCharm to update the indexes

Visual Studio Code

Install the Python extension
Install the Remote - Containers extension
Open the Command Pallette and type Remote-Containers, then select the Attach to Running Container... and selecet the running docker container
VS Code will restart and reload
On the Explorer sidebar, click the open a folder button and then enter /code (this will be loaded from the remote container)
On the Extensions sidebar, select the Python extension and install it on the container
When prompet on which interppreter to use, select /usr/local/bin/python
Open the Command Pallette and type Python: Configure Tests, then select the unittest framework

Expected Results

Code completion works
Code navigation works
Organize imports works
Import suggestions/discovery works
(VS Code) Tests (either classes or methods) will have a new line above their definitions, containing two actions: Run Test | Debug Test, and will be executed upon clicking on them
(PyCharm) Tests (either classes or methods) can be executed by placing the cursor on them and then using Ctrl+Shift+R

Bonus: TDD Enablement

One of the key aspects of the Test-Driven Development is to provide a short feedback on each iteration (write a failing test, fix the test, refactor). And a lot of times a project's tooling might work against this principle, as it's fairly common for a project to have a way of executing its test suite, but it is also common that this task will run the entire suite, not just a single test.

But if you have your IDE of choice able to execute just a single test in a matter of seconds, you will feel way more comfortable on given TDD a try.

Readability for Real - Using Meaningful Names for Database Queries

Alvaro Cavalcanti — Tue, 16 Oct 2018 19:33:22 +0000

Readability matters

Anyone that has programmed for a while and had to deal with someone else's code knows it (especially when this someone is yourself, six months ago). It can be frustrating to try to understand a seemingly innocent bunch of lines of code, but end up thinking "what was this person thinking back then?".

I have come across a scenario similar to the one below, which I managed to have its obscure business rules exposed into the code without the use of comments. You see, I am strongly against using comments. For me, they should be used as a last resort, with maintainability as the main argument for this. We all know how easily comments can get out of synch with the code.

Scenario Description

Imagine you have a subscription-based business, where your customers sign up to different plans, which can be billed monthly or annually. This system has been running for a couple of years now and you need to perform some batch process to subscriptions six hours before they get renewed. But this should not be for all plans, you need to exclude the subscriptions to "Partner Plus" plans, both annually and monthly recurrences.

At first, it may seem simple, but the reality is that during the past years things evolved rather freely and you don't have a simple ID for each plan. In fact, the whole concept of what is a "Partner Plus" plan depends on different things. The plan's name should have "partner" on it, but it's not limited to that alone: "partner agile expo 15", "partner plus", "new partner plus", etc. Also, the price is variable, but at least is always greater than 249.00 for monthly and 2490.00 for annually.

Writing the Code

With that in mind, you might end up with a code similar to this (it's a Python code for a Django application):

    subscriptions = Subscription.objects.filter(
        period_end__lte=datetime.now(timezone.utc) + timedelta(hours=6),
        active=True,
    ).exclude(
        "plan__plan_id__contains": "partner",
        "plan__recurrence": "MONTHLY",
        "plan__price__gt": Decimal('249.00')
    ).exclude(
        "plan__plan_id__contains": "partner",
        "plan__recurrence": "ANNUALLY",
        "plan__price__gt": Decimal('2490.00')
    )

Well, based on what I described earlier this code does the job beautifully. It looks ahead six hours, and excludes the plans that match the Partner Plus definition, right?

But, what if six months from now a new team member is tasked to change this code. Will she be able to understand why that exclusion is being done? And what kind of objects are being excluded? Most probably she would need to ask around a bit.

But now have a look on this refactored code:

    next_6_hours = datetime.now(timezone.utc) + timedelta(hours=6)

    partner_plus_monthly = {
        "plan__plan_id__contains": "partner",
        "plan__recurrence": "MONTHLY",
        "plan__price__gt": Decimal('249.00')
    }
    partner_plus_annually = {
        "plan__plan_id__contains": "partner",
        "plan__recurrence": "ANNUALLY",
        "plan__price__gt": Decimal('2490.00')
    }

    subscriptions = Subscription.objects.filter(
        period_end__lte=next_6_hours,
        active=True,
    ).exclude(
        **partner_plus_monthly
    ).exclude(
        **partner_plus_annually
    )

Now the variables bear the context, and even if the new team member does not get it completely, at least she can be more informed when asking around, she can start off with "why do we need to exclude partner plus plans from this query?".

The thing with readability-driven refactorings is that they tend to be overlooked due to the seemingly "low impact", because, you know, "it's almost the same code", "everyone in the team knows what it means", "a comment line would do it", etc.

Well, to that I say: challenge yourself! Writing is about communication. Either writing prose or code!

It's Dangerous to go Alone, Take This

Before you go, I'm gonna leave the following link here (and in all my future readability articles) because it was the one article which opened my mind on the matter and I keep coming back to it every now and then: FunctionLength, by Martin Fowler

Practical Python - Duck-typing to the rescue!

Alvaro Cavalcanti — Thu, 04 Oct 2018 14:46:07 +0000

Duck-typing?

If it walks like a duck, quacks like a duck and swims like a duck: it's a duck.

That above is the simple definition of duck-typing, which is a great side-effect feature of non-static typed languages, like Python.

Alright, how can I use it?

Let's consider the following object, a Django model (simplified for this example):

class User(models.Model):
    username = models.CharField()
    full_name = models.CharField()
    address_street = models.CharField()
    address_zip = models.CharField()
    subscription = models.ForeignKey()
    pets = ArrayField(Pet())  # Consider a one-to-many relationship

Now, you are tasked with creating a change tracker for the user, but that change should only care about the username and pets collection (because reasons). You then create a new model similar to this:

class UserChange(models.Model):
    before_username = models.CharField()
    before_pets = ArrayField(Pet())
    after_username = models.CharField()
    after_pets = ArrayField(Pet())

All is well, you have implemented almost every scenario, until you get to the removal of a single Pet from a User. The issue is, since the collection is loaded from the database, and you should only track the change once it is completed, it gets tricky to load the previous state. Well, you can grab the initial collection before it gets modified, that's fine, but where to store it? You can also do a copy.deepcopy of it, but that might sound like using a cannon to kill an ant. Then again, you only need to store the username and pets collection.

Duck-typing to the rescue!

You can use a light-weighted and elegant solution as:

class Duck:
    def __init__(self, **kwds):
        self.__dict__.update(kwds)

The above class takes whatever keyword arguments you pass in and adds them as proprieties to the class and can be used as simple as:

user_duck_before = Duck(username="foo", pets=["dog", "cat"])
user_duck_after = Duck(username="bar", pets=["cat"])

That's it. Now you can use these "ducks" on the deletion flow without much memory overhead.

This approach is similar to using unites.mock but with production code instead of testing code.

And to be fair, even though I already had the idea in my mind, I didn't came up with the code, I found it on this SO post which pointed me to the code recipe by Alex Martelli. Thanks, Alex!

Edit

I realized that something was not clear, but I didn't want to modify it in place, thus this section. What really motivated me in searching for a duck-type approach was the one-to-many collection. In my real world scenario this collection was a RelatedManager, which prevented me from loading the previous state after the item have been removed, thus I needed to store that info in a way that I would neither break my contract nor implement an almost identical method just for this new data structure.

Ever wished for a pull request dependency check? You've got it.

Alvaro Cavalcanti — Tue, 06 Mar 2018 19:05:56 +0000

Pull Requests are a thing (like it or not)

Those of us that work with GitHub, either contributing to some open-source projects or working on private repositories, are familiar with pull requests. Even if your team is adept of trunk based development you should have heard about PRs as "evil things".

But if you're on the first group, as myself, and deal with a couple of different repositories that have any kind of dependency between them you might have come around with merging a PR that depended on another repo's PR that wasn't merged yet. I know I have!

The Episode®

It was a RoR frontend that depended on a Django-Rest-Framework backend. We merged the former before the latter, thus some production bugs cropped up and we had to deal with them.

After that episode, I started looking for tools that could prevent that from happening again. And it would be awesome to have some tool that would publish against the PR status/check, the same way CI services as Jenkins and CircleCI and some GitHub features do.

To be honest, I did find a couple of tools/services that seemed to offer this feature. What I didn't like, tho, was that they had a truckload of features. And I didn't want that, I wanted a simple checker.

If you don't like, do it yourself

From that point I changed my research. Instead of looking for a tool, I started looking at GitHub API's documentation. I decided to create a tool myself and it felt quite achievable after my initial research.

As I felt really comfortable and productive with Python, it was my platform of choice. But instead of going with Django-Rest-Framework I chose Flask for two reasons, 1) a Flask app is way slimmer than a DRF one, and slim and simple is better, and 2) I wanted to try Flask out. :)

There you have it: Pierre DeCheck

After roughly 2 months, and a complete rewrite, I had a working version that was ready for real-world testing. Here I present to you: Pierre DeCheck

Pierre's concept is simple, it should be configured as a webhook for a given repository, then you add the dependencies to a PR using keywords like "Depends on #2" for same-repo issues/PRs, or "Depends on owner/repo#2" for external ones and then it will publish a check to your PR status, telling you if the dependencies are met or not:

The installation and usage details can be found on Pierre's GitHub page and the only caveat you should be aware of is that, for now, the dependency check is synchronous. Meaning that when GitHub sends an event to the webhook, the check and status update will be made during that request. Which isn't optimal at all, I know, but it's a start.

Edit: Pierre will perform checks only upon PR creation and Comment Activity (added/removed). In every case it will fetch all the PR's bodies (the PR body itself and from all its comments), extract the dependencies and perform the checks. Thus, it does not observe the dependencies themselves and re-run the checks if their status change. But who knows what future holds? Feel free to create an issue/feature request! :)

Help Wanted

So, if you think this tool could help your team, it would be great if you could test it, create issues for the bugs you find and even contribute!

Thanks a lot!

To Create or To Fix?

Alvaro Cavalcanti — Thu, 08 Feb 2018 22:18:15 +0000

The first thrill we feel upon writing the first line of code in our life is when we make the computer do something. Write our name, say hi to the world or even print a computed sum. It's the joy of creation.

Of course that feeling does not last for too long, because the first bug is never too far away. Then the back-and-forth begins. Writing features and fixing bugs.

It's the circle of life.

Further down the road we end up with a lot of created features, and a pretty good amount of crushed bugs. Which ones did you enjoy the most?

If I must pick one...

It will be crushing bugs!

To be fair, it's really exciting to talk to the P.O. trying to understand better the requirements for a given story, to do either a kickoff or a desk-check session with the users and see the value of that feature in their eyes. And to deliver that, obviously.

But in the end it feels more rewarding, to me, to get a bug and crush it. To read through the logs, to understand better the business, the code and investigate. The investigation is thrilling. Unfortunately, the bugs usually come bundled into a pressure pan, with anxious people who want it to be gone and that's not really pleasant.

But even then, I do enjoy the investigation and the fixing.

What about you peeps?

Hi, I'm Alvaro Cavalcanti

Alvaro Cavalcanti — Fri, 24 Mar 2017 18:10:35 +0000

I have been coding for almost 20 years.

You can find me on Twitter as @alvarocavalcant

I live in Recife (Pernambuco, Brasil).

I work for ThoughtWorks

I mostly program in these languages: Swift, Python and Java.

I am currently learning more about iOS and Mobile Architectures.

Nice to meet you.