Forem: Juan Benitez

Best Python resources from beginner to advanced

Juan Benitez — Mon, 03 Jan 2022 17:23:09 +0000

Hey there, I made this compilation of websites, courses, youtube videos for learning Python.

1. Courses

Couple of free courses to learn Python from scratch

2. Algorithms and Data structures.

Videos, texts, repositories about algorithms and data structures,

3. Websites

Websites all about Python

4. Web frameworks.

Some of the best Python web frameworks

5. Books

For those who likes to read

6. Advanced topics.

Some advanced topics, from metaclasses to machine learning

I hope you find helpful this list and can start your Python journey.

You can follow me on Twitter and GitHub to be up to date with all my projects and content.

Trending Python Projects of the week

Juan Benitez — Thu, 30 Dec 2021 21:30:24 +0000

I wanted to make a compilation of some of the nicest Python projects that have gained popularity on GitHub this week.

SQLModel

As his creator defined, SQLModel is a Python library for interacting with SQL databases. It is based on Python type hints, and it uses Pydantic and SQLAlchemy internally. We can argue that SQLModel is just a layer on top of Pydantic and SQLAlchemy.

SQLModel was created by the same person who created FastAPI, so you can expect a pleasant and flawless integration between both.

Although SQLModel was launched no more than 4 months ago (at the time of writing this), it already offers many features, and some others are already being developed, I encourage you to give it a try.

Python Fire

Python Fire is a library designed by Google for automatically generating command-line interfaces (CLIs) from absolutely any Python object, so basically you can create a Python method and generate a CLI from it.

Here are some examples,

import fire

def greeting(name="World"):
  return "Hello %s!" % name

if __name__ == '__main__':
  fire.Fire(greeting)

After import fire, we can use it to generate a CLI from greeting in this case.

python greeting.py  # Hello World!
python greeting.py --name=Juan  # Hello Juan!
python greeting.py --help  # Shows usage information.

I think this library is really helpful for debugging, developing some Python scripts.

If you like to play with CLIs, you should try Typer, another great tool for building CLIs with Python.

Sherlock

This is simple, a Python script for searching usernames across several social networks, how cool is that huh?

I tried with my username and there are some sites that I don’t know about, I should look into that…

Github Repository

changedetection.io

This is a tool for monitoring websites and get notifications when they change, so if you want to be on top of new information on the sites you like the most, give it a try.

Some use cases for this:

Governmental department updates (changes are often only on their websites)
Products and services have a change in pricing
New software releases, security advisories when you're not on their mailing list.
Festivals with changes
Real estate listing changes
COVID-19 related news from government websites

The best part about changedetection is that it is open source and you can self-host it.

Github Repository

I hope you like this list and leave a comment if you have used some of these tools or if you know about other Python projects gaining popularity.

You can follow me on Twitter and GitHub to be up to date with all my projects and content.

How to apply filters to images with Python

Juan Benitez — Wed, 22 Dec 2021 00:01:44 +0000

A few weeks ago, we received a new ticket: “Users want to be able to apply filters to their pictures”, yes, something like Instagram does. We immediately thought about the Lightroom API, but after some research, we came to the conclusion that it doesn’t quite meet our needs. We were searching for something like a library, where we can just load the image, apply the filters and then save the new image with the filters applied, since we couldn’t find it we decided to build one.

FImage

FImage is a Python module to apply and create multiple filters to images, it exposes an API that you can use for applying the different color transformations to the images. It works by converting the image to an RGB matrix and applying different math formulas to it. We used NumPy for all the matrix operations since it is faster and optimized, and Pillow for handling the loading and saving of the images.

How to use it?

First, we need to install it, for this you need to be using Python 3.6 or greater to be able to use FImage.

pip install fimage

And for these examples, I’m gonna use this picture to apply it filters:

Applying a simple filter

Create a file app.py with:

from fimage import FImage
from fimage.filters import Sepia


def main():
    # replace 'my_picture.jpg' with the path to your image
    image = FImage('my_picture.jpg')

    # apply the Sepia filter to the image
    image.apply(Sepia(90))

    # save the image with the applied filter
    image.save('my_picture_sepia.jpg')


if __name__ == "__main__":
    main()

Now, just run it :

python app.py

And this is how the new image my_picture_sepia.jpg looks like after the filter was applied.

Applying multiple filters

FImage offers more filters besides the Sepia one, even you can combine multiples filters to give a better look to your picture.

Modify the file app.py to import more filters from FImage

from fimage import FImage
from fimage.filters import Contrast, Brightness, Saturation


def main():
    image = FImage('my_picture.jpg')

    # apply the mutiple filters to the image
    image.apply(
        Saturation(20),
        Contrast(25),
        Brightness(15)
    )

    # save the image with the applied filter
    image.save('my_picture_mixed.jpg')


if __name__ == "__main__":
    main()

We run it by

python app.py

And our new my_picture_mixed.jpg looks like

The order in which the filters are passed to the apply function matters, this is because the filters are applied sequentially, so the next filter will be applied over the resultant image from the previous one.

Presets

Presets are just the combinations of multiple filters with already defined adjustment values.

Let’s change our app.py one more time to use the Presets

from fimage import FImage
from fimage.presets import SinCity


def main():
    # replace 'my_picture.jpg' with the path to your image
    image = FImage('my_picture.jpg')

    # apply the SinCity preset to the image
    image.apply(SinCity())

    # save the image with the applied preset
    image.save('my_picture_sincity.jpg')


if __name__ == "__main__":
    main()

After we run it, we get our new my_picture_sincity.jpg

Custom Presets

If you like the look your picture got after testing different filters and want to store this combination for applying it to more pictures, you can create your own Preset by just extending the Preset Class and specifying these filters and their adjust values in it.

In our app.py let’s do

from fimage import FImage
from fimage.presets import Preset
from fimage.filters import Contrast, Brightness, Saturation


# Create my custom preset and specify the filters to apply
class MyOwnPreset(Preset):
    transformations = [
        Contrast(30),
        Saturation(50),
        Brightness(10),
    ]


def main():
    # replace 'my_picture.jpg' with the path to your image
    image = FImage('my_picture.jpg')

    # apply MyOwnPreset to the image
    image.apply(MyOwnPreset())

    # save the image with the applied preset
    image.save('my_picture_custom.jpg')


if __name__ == "__main__":
    main()

The new my_picture_custom.jpg

This is basic usage of FImage, we are still developing it, and it would be really great any feedback or contribution you have.

GitHub Repo

You can follow me on Twitter and GitHub to be up to date with all my projects and content.

How to return DateTime in different time zones from Django Rest Framework

Juan Benitez — Fri, 03 Sep 2021 01:09:59 +0000

This is how we can return DateTimes with dynamic time zones from DRF.

So, early this week I got a new ticket where I had to create new endpoints for our API, a few endpoints with DRF were not a big deal I said, this was until I realized that the DateTime values in the response from these endpoints needed to be converted from UTC to other time zones.

We always store and return our DateTime values in UTC, and we convert them to the time zone of the user at display time, this can be in the browser using JS. But this time was different, the API needed to return the values in a specific time zone.

We have something like this.


class Venue(models.Model):
    time_zone = models.CharField(max_length=128, default='UTC')
    ...

class Event(models.Model):
    start_time = models.DateTimeField()
    end_time = models.DateTimeField()
    venue = models.ForeignKey(Venue, on_delete=models.CASCADE)
    ...

Each Event is happening in a Venue, this venue is placed in some city, so we need to store the time zone used for that venue. Since we want to sell tickets or just promote some events we need to return the DateTime values in the venue time zone so we don't confuse the customers and other resellers.

The first solution I thought of was the serializer field DateTimeField, it accepts an attribute called default_timezone, this is a pytz.timezone representing the timezone. If not specified and the USE_TZ setting is enabled, this defaults to the current timezone. If USE_TZ is disabled, then datetime objects will be naive.

class EventSerializer(serializers.ModelSerializer):
    start_time = serializers.DateTimeField(default_timezone=pytz.timezone('America/Bogota'))
    end_time = serializers.DateTimeField(default_timezone=pytz.timezone('America/Bogota'))
    ...

This works, if the start_time in UTC is 2021-09-02T10:00:00 in the resultant JSON will be displayed as 2021-09-02T05:00:00-05:00. But it didn't just work for me because I don't know the time zones beforehand and also they will be different, so I needed to do this at runtime, my solution was to override the to_representation() method from the serializer.

class EventSerializer(serializers.ModelSerializer):
    def to_representation(self, instance):
        self.fields['start_time'] = serializers.DateTimeField(default_timezone=pytz.timezone(instance.venue.time_zone))
        self.fields['end_time'] = serializers.DateTimeField(default_timezone=pytz.timezone(instance.venue.time_zone))
        return super().to_representation(instance)
    ...

to_representation is run for each instance that is going to be returned by the serializer, so basically what we are doing is overriding the start_time and end_time declaration for each instance, this is for specifying that we want to use a DateTimeField and we want to pass it a default_timezone which is going to be taken from the venue that is linked to the instance.

This helped me to do what I needed and I hope that this can help you as well.

You can follow me on Twitter and GitHub to be up to date with all my projects and content.

How to throttle your API with Django Rest Framework

Juan Benitez — Sat, 28 Aug 2021 03:02:38 +0000

Control the rate of requests that clients can make to your API.

But what is throttling...?

As DRF says, throttling is

Throttling is similar to permissions, in that it determines if a request should be authorized. Throttles indicate a temporary state, and are used to control the rate of requests that clients can make to an API

In other words, we can say that throttling is a mechanism for limiting how many requests my API can accept in a period of time, we can specify this limit per user, IP address, etc. This is similar to how some APIs limit the number of requests you can make in a day or an hour.

How throttling is handle in DRF

As with permissions and authentication, throttling in DRF is always defined as a list of classes.

Before running the main body of the view each throttle in the list is checked. If any throttle check fails an exceptions.Throttled exception will be raised, and the main body of the view will not run.

And the same as permissions we can set these throttles globally and per views.

Global Throttles

We can set a global default throttling policy using the DEFAULT_THROTTLE_CLASSES and DEFAULT_THROTTLE_RATES settings.

REST_FRAMEWORK = {
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.AnonRateThrottle',
        'rest_framework.throttling.UserRateThrottle'
    ],
    'DEFAULT_THROTTLE_RATES': {
        'anon': '20/day',
        'user': '50/day'
    }
}

The rate descriptions used in DEFAULT_THROTTLE_RATES may include second, minute, hour, or day as the throttle period.

With these settings,

Unauthenticated users will be able to only make 20 requests per day to our API, the IP address of the incoming request is used to generate a unique key to throttle against.
Authenticated users will be able to make 50 requests per day to our API, for these the id of the user is going to be used to generate the unique key.

When they reached the maximum of requests our API will respond with the error code 429 - Too Many Requests

Since this is the global configuration this will apply to all views, but we can still override these settings per view.

Throttles Per View

It's always better to have control over what views we want to limit, for this DRF offers us the possibility of setting these throttles classes per view.

We do this by passing a list of throttles classes to the throttle_classes attribute on the APIView class-based views.

from rest_framework.response import Response
from rest_framework.throttling import UserRateThrottle
from rest_framework.views import APIView

class ExampleView(APIView):
    throttle_classes = [UserRateThrottle]

    def get(self, request, format=None):
        content = {
            'status': 'request was permitted'
        }
        return Response(content)

In this way, ExampleView will use the UserRateThrottle class to limit the number of requests that this view can receive, the rates, in this case, are still defined on the DEFAULT_THROTTLE_CLASSES settings key.

But what if we want to specify a different rate for a specific view, we can do this by extending the UserRateThrottle class and specifying a new rate.

from rest_framework.response import Response
from rest_framework.throttling import UserRateThrottle
from rest_framework.views import APIView

class CustomUserRateThrottle(UserRateThrottle):
    rate= '5/day'

class VeryLimitedView(APIView):
    throttle_classes = [CustomUserRateThrottle]

    def get(self, request, format=None):
        content = {
            'status': 'request was permitted'
        }
        return Response(content)

Now only for this view the authenticated users have 5 requests per day, even though the global settings say that the users have 50 requests per day.

ScopedRateThrottles

The ScopedRateThrottle class can be used to restrict access to specific parts of the API. This throttle will only be applied if the view that is being accessed includes a .throttle_scope attribute.

The allowed request rate is determined by the DEFAULT_THROTTLE_RATES setting using a key from the request "scope".

For example, given the following views...

class ContactListView(APIView):
    throttle_scope = 'contacts'
    ...

class ContactDetailView(APIView):
    throttle_scope = 'contacts'
    ...

class UploadView(APIView):
    throttle_scope = 'uploads'
    ...

...and the following settings.

REST_FRAMEWORK = {
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.ScopedRateThrottle',
    ],
    'DEFAULT_THROTTLE_RATES': {
        'contacts': '100/day',
        'uploads': '50/day'
    }
}

User requests to either ContactListView or ContactDetailView would be restricted to a total of 100 requests per day. User requests to UploadView would be restricted to 50 requests per day.

So, what's next...

As we saw, throttles is a powerful and really helpful feature that we can implement on our API if we need to impose different constraints on different parts of the API, due to some services being particularly resource-intensive. Also, it's worth mentioning that DRF provides a BaseThrottle class which you can override to create custom throttles with custom implementations.

If you plan to use this method as a security feature just consider that the DRF throttling isn't intended as a security feature, it has some weaknesses and you shouldn't rely only upon the throttling.

Yet, it's a cool feature and you should definitely try it on your API.

You can follow me on Twitter and GitHub to be up to date with all my projects and content.

My First FastAPI Experience

Juan Benitez — Thu, 26 Aug 2021 17:52:29 +0000

A few weeks ago I started my journey with FastAPI a modern, fast web framework, this is how his creator defines it.

I heard about FastAPI a long time ago when some coworker told me that this framework was built by a Colombian, I'm from Colombia as well so I was really curious about it, anyway I didn't pay too much attention to it at first because I was really focused on learning Django.
But now when I feel like I have a certain level of expertise in Django I wanted to give it a try.

Where I started?

I started at the docs as usual and I thought at first, well this is pretty much Flask but with async support. But obviously, it wasn't just that.

While I was reading the docs I was always trying to make comparisons with Django Rest Framework, even though these are completely different web frameworks, this helped me to better understand what I was reading and why I should use one over the other.

Django and FastAPI

The first comparison I made was serializers and Pydantic, these both can be used to validate the data you are passing in the request and to serialize your objects but the big difference comes when Pydantic benefits from the Python type hints, so you don't learn extra syntax, and this type hints fit perfectly with the IDEs, so you have autocompletion and all those nice features.

The second thing that I wanted to check was the views, these guys that are in charge of taking the request and returning a response. DRF provides many options when it comes to views, it has generic classes, viewsets, mixins, etc. FastApi on the other hand has something they call path operation function, this is a function that is related to a path and it's going to be executed when this path is requested. First, I was confused because it can be hard to understand how this functions works, for example, if you pass a parameter to this function this is going to be treated as a query parameter but you also define the path parameter there, and also the body is passed as a parameter, so you need to define this parameter with special types provided by FastAPI so it knows how to treat every parameter.
Please, I know this sounds confusing but when you understand it, it is pretty simple. I have to be honest and I kind of like the DRF Class views more, it just seems more organized and simple at least for me. Yet, you can organize and share code along with the application, they have something called Dependencies that is used for this, it's worth taking a look at it because it's a really powerful feature.

The last thing I want to mention and that is the async support, this is just great, it comes with the framework so we don't need to configure or install something else. And the same goes for automatic documentation, every time you create a new endpoint the docs for this endpoint are created automatically using all the different things you defined for that endpoint, query parameters, path parameters, body requests, responses, authentication, etc.

What is next?

I'm really excited about this journey, I'm already doing some projects with FastApi and I hope I can tell you soon about it.

Also, I'm contributing to translate the docs to Spanish, this has helped me a lot as well because in this way I can read really carefully all the things that the framework provides.

I hope this can help you as well, and if you have any questions please don't hesitate, and of course check FastApi, it's really powerful and nice to learn.

Simple tweet locator using Python, Flask SocketIO and Tweepy

Juan Benitez — Sat, 14 Mar 2020 20:27:56 +0000

Simple application to locate tweets on a map based on the location from where they were sent.

This application was written in Python 3.7, and uses Flask Socket IO which gives Flask applications access to low latency bi-directional communications between the clients and the server, and Tweepy a Python-based client to interact with the Twitter REST APIs.

In order to have access to Twitter data programmatically it's necessary to have some access tokens, you can create an app and get your secret tokens from here.

Installation

The full source code can be found on Github.

git clone https://github.com/JuanBenitez97/Tweet-Locator.git
cd Tweet-Locator
pip install -r requirements.txt

After this proccess is done you need to create a .env file to store all your secret tokens and being able to use the Twitter APIs, this file should be in the root folder.

CONSUMER_KEY = 'Your API KEY'
CONSUMER_SECRET = 'Your API SECRET KEY'
ACCESS_TOKEN = 'Your Access Token'
ACCESS_SECRET = 'Your Access Secret Token'

Run

python app.py

If everything is correct you will see the server is running and which host and port is running on.
Just go to the url and wait for the tweets appear on the map.

If you want to filter tweets based on other words you can change the tags in the app.py file

if __name__ == '__main__':
    """
    tags = ["#python", "programming", "flask"]   
    """

Enjoy it!!