Forem: Ori Roza

The project that will make you enjoy writing tests for your Django app

Ori Roza — Wed, 03 Jan 2024 15:35:44 +0000

Hi all! I’m proud to share my new first open-source project drf-api-action, and I’d be glad to receive your feedback!

https://github.com/Ori-Roza/drf-api-action

This project was built as a side project at work in which we had to tweak DRF for our own needs, this project was successful back then so I wanted to publish it to everyone

The drf-api-action Python package is designed to elevate your testing experience for Django Rest Framework (DRF) REST endpoints by treating REST endpoints as regular functions!

Features:

Simplified Testing: Testing DRF REST endpoints using the api-action decorator, treating them like regular functions.
Seamless Integration: Replacing DRF's action decorator with api-action in your WebViewSet seamlessly or using it as a pytest fixture.
Clear Traceback: Instead of getting a response with error code, get the real traceback that led to the error.
Pagination Support: Paginating easily through pages by a single kwarg.

Here's a snippet with new pytest fixture addition:

from .models import DummyModel
from rest_framework import status
from rest_framework.response import Response
from rest_framework.viewsets import ModelViewSet
from rest_framework.serializers import IntegerField, ModelSerializer, Serializer


class DummySerializer(ModelSerializer):
    dummy_int = IntegerField()

    class Meta:
        model = DummyModel
        fields = "__all__"

class DummyViewSetFixture(ModelViewSet):
    queryset = DummyModel.objects.all()
    serializer_class = DummySerializer

    @action(detail=True, methods=["get"], serializer_class=DummySerializer)
    def api_dummy(self, request, **kwargs):
        serializer = self.get_serializer(instance=self.get_object())
        return Response(data=serializer.data, status=status.HTTP_200_OK)

import pytest
from tests.test_app.models import DummyModel
from tests.test_app.views import DummyViewSetFixture
from drf_api_action.fixtures import action_api

@pytest.mark.action_api(view_set_class=DummyViewSetFixture)
def test_call_as_api_fixture(db, action_api):
    dummy_model = DummyModel()
    dummy_model.dummy_int = 1
    dummy_model.save()
    res = action_api.api_dummy(pk=1)
    assert res["dummy_int"] == 1

Please let me know what you think, Your feedback means a lot to me!

Simplifying django-rest-framework Testing with drf-api-action

Ori Roza — Sun, 17 Dec 2023 15:00:23 +0000

Testing Django Rest Framework #DRF endpoints can sometimes be a complex and time-consuming process. However, the #drf-api-action Python package aims to simplify and enhance the testing experience by introducing a custom decorator, api-action.
In this article, we will explore how to leverage #drf-api-action to streamline testing in DRF, making the development process more time-efficient and enjoyable.

Installation:

Let's start by installing the #drf-api-action package using pip:

pip install drf-api-action

Usage:

Now that we have the package installed, let's dive into how
#drf-api-action can significantly improve testing workflows in #DRF.
Add the package to an existing project:
we have a User model which exposes 2 endpoints:

/api/users/{id} : A GET request that returns details on a specific user.
/api/users/ : A POST request that creates a new user.

from rest_framework import mixins, viewsets, status
from rest_framework.decorators import action
from rest_framework.response import Response

from drf_api_action_example.users.models import User
from drf_api_action_example.users import serializers

class UsersViewSet(mixins.RetrieveModelMixin, viewsets.GenericViewSet):

    @action(detail=True,
            methods=['get'],
            url_path='/',
            url_name='users/',
            serializer_class=serializers.GetUserDetailsSerializer)
    def get_user_details(self, request, **kwargs):
        """
        returns user details, pk expected
        """
        serializer = self.get_serializer(instance=self.get_object())
        return Response(data=serializer.data, status=status.HTTP_200_OK)

    @action(detail=False,
            methods=['post'],
            url_path='/',
            url_name='users/',
            serializer_class=serializers.AddUserSerializer)
    def add_user(self, request, **kwargs):
        """
        adds a new user
        """
        serializer = self.get_serializer(data=request.data)
        serializer.is_valid(raise_exception=True)
        serializer.save()
        return Response(data=serializer.data, status=status.HTTP_201_CREATED)

Now, to apply the benefits of #drf-api-action we will change the following:

from rest_framework import mixins, viewsets, status
#  instead of from rest_framework.decorators import action do:
from drf_api_action.mixins import APIRestMixin
from drf_api_action.decorators import action_api
from rest_framework.response import Response

from drf_api_action_example.users.models import User
from drf_api_action_example.users import serializers

#  inheriting APIRestMixin
class UsersViewSet(APIRestMixin, mixins.RetrieveModelMixin, viewsets.GenericViewSet):

    #  replacing @action decorator
    @action_api(detail=True,
                methods=['get'],
                url_path='/',
                url_name='users/',
                serializer_class=serializers.GetUserDetailsSerializer)
    def get_user_details(self, request, **kwargs):
        """
        returns user details, pk expected
        """
        serializer = self.get_serializer(instance=self.get_object())
        return Response(data=serializer.data, status=status.HTTP_200_OK)

    #  replacing @action decorator
    @action_api(detail=False,
                methods=['post'],
                url_path='/',
                url_name='users/',
                serializer_class=serializers.AddUserSerializer)
    def add_user(self, request, **kwargs):
        """
        adds a new user
        """
        serializer = self.get_serializer(data=request.data)
        serializer.is_valid(raise_exception=True)
        serializer.save()
        return Response(data=serializer.data, status=status.HTTP_201_CREATED)

Fun Part - Writing tests!

First, import:

import pytest
from users.models import User
from users.views import UsersViewSet
from rest_framework.exceptions import ValidationError

The first test test_get_user_detais simply tests /api/users/{id} GET endpoint:

import pytest
from users.models import User
from users.views import UsersViewSet
from rest_framework.exceptions import ValidationError

users_api = UsersViewSet()

def test_get_user_details(users_api):
    user = User(first_name='shosho', last_name='bobo', age=30)
    user.save()

    user_details = users_api.get_user_details(pk=user.id)
    assert user_details['first_name'] == user.first_name

The second test test_add_user simply tests /api/users POST endpoint:

def test_add_user(users_api):
    output = users_api.add_user(first_name='bar', last_name='baz', age=30)
    assert output['id'] is not None

The third test test_add_user_exception_on_age simply tests /api/users POST endpoint when age is not in the valid range:

def test_add_user_exception_on_age(users_api):
    with pytest.raises(ValidationError) as error:
        users_api.add_user(first_name='bar', last_name='baz', age=150)
    assert "Ensure this value is less than or equal to 120" in str(error.value)

Unlike traditional #testing methods, #drf-api-action provides clear tracebacks, making it easier to identify and fix errors in your code.

Demo

Conclusion

The #drf-api-action package is a game-changer for testing #DRF/#Django endpoints in #Python. By simplifying the testing process and providing clear tracebacks and pagination support, it allows developers to focus on writing robust and time-efficient code.
With #drf-api-action, testing in #DRF becomes more intuitive and enjoyable, contributing to an overall improved development experience.
Start leveraging this powerful package in your #DRF projects today!

P.S if you liked it, star it on #GitHub 💫

Resources

full example project
drf-api-action source code
django-rest-framework documentation

Creating API Actions in Django Rest Framework

Ori Roza — Wed, 29 Nov 2023 13:49:56 +0000

A few weeks back, my R&D manager asked me to create a Python package that would eventually evolve into a standalone web service. We use Django, a widely used web framework in Python, particularly known for its rest framework.

Photo by Faisal on Unsplash

Django has an impressive track record, with over 70,000 stars on GitHub and being utilized by big names like Instagram, National Geographic, Mozilla, Spotify, and Pinterest.

Approaches

To achieve our goal, I had a couple of approaches in mind:

Creating a Python Package: This involves building a Python package with Django apps and later developing a separate web service utilizing this package.
Custom Action Decorator: Another option was to craft the action decorator, allowing API functions to be explicitly called from the ViewSet. I opted for the second choice to avoid managing two separate projects, and surprisingly, it brought additional benefits.

Action Decorator

Understanding the Action Decorator

Let’s start with a quick overview of the action decorator.

According to the official documentation:

"If you have ad-hoc methods that should be routable, you can mark them as such with the @action decorator."

For example:

class DummyAPIViewSet(ModelViewSet):
    # ... (previous code)

    @action(detail=True, methods=["get"], serializer_class=DummySerializer)
    def dummy(self, request, **kwargs):
        serializer = self.get_serializer(instance=self.get_object())
        return Response(data=serializer.data, status=status.HTTP_200_OK)

    @action(detail=False, methods=["post"], serializer_class=GetDummyByIntSerializer)
    def by_dummy_int(self, request, **kwargs):
        # ... (rest of the code)

This decorator is very straightforward. It defines a route in a ViewSet and usually requires the following arguments:

detail: whether the call expects to get PK or not.
methods: which methods the call supports.
serializer_class: the serializer that handles the specific view.

Custom Action Decorator

I chose to customize this decorator because it is frequently used and easy to replace with a new one.

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(func)

Now, we want to call by_dummy_int explicitly from the ViewSet:

>>> api = DummyAPIViewSet()
>>> api.dummy(request=None, pk=1)
>>> {'id': 1, 'dummy_int': 1}

In order to do that, we need to:

Create our own decorator.
Inject our function arguments into the method.
Inject our serializer into the view.

Create Our Own Decorator

In the action implementation, we see that the following decorator gets the following arguments:

action(methods=None, detail=None, url_path=None, url_name=None, **kwargs)

So we will create the same:

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(func)

Now we want to distinguish between an API call and a REST call. We can achieve that by indicating whether the func argument request is None or not:

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        def route_call(self, request, **params):
            if request:
                return func(self, request, **params)
            else:
                pass

        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(route_call)

The route_call mocks func method with the same arguments:

self
request
kwargs

If you run this app as a server, it will behave the same.

Inject Function Arguments into the Request

As mentioned before, a ViewSet method has 3 arguments:

self
request
kwargs

self we get from the instance, so we need to mock a request object. We want to pass the function arguments into the request data/query_params.

According to the DRF code, the Request object has two relevant properties:

data: POST payload
query_params: GET query parameters

class CustomRequest:
    """
    Mock for a custom request
    """

    def __init__(self, data, query_params):
        self.data = data
        self.query_params = query_params

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        def route_call(self, request, **params):
            if request:
                return func(self, request, **params)
            else:
                # injecting our custom request
                request = CustomRequest(params, params)
                return func(self, request, **params)

        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(route_call)

Let’s try it out!

>>> res = api.dummy(request=None, pk=1)
ret = func(self, request, **kw)
# (output)

It means that the ViewSet also has a request attribute! Let’s add it:

class CustomRequest:
    """
    Mock for a custom request
    """

    def __init__(self, data, query_params):
        self.data = data
        self.query_params = query_params

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        def route_call(self, request, **params):
            if request:
                return func(self, request, **params)
            else:
                # injecting our custom request
                request = CustomRequest(params, params)
                self.request = request
                return func(self, request, **params)

        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(route_call)

And when trying again:

>>> res = api.dummy(request=None, pk=1)
>>> {'id': 1, 'dummy_int': 1}

We made it! but it’s enough, we want each method to know the serializer_class and stop using the general one.

Inject Our Serializer into the View

We want to inject each method with its corresponding serializer. To achieve it we need to do

two things that are connected:

Inject serializer_class to the ViewSet kwargs:

class CustomRequest:
    """
    Mock for a custom request
    """

    def __init__(self, data, query_params):
        self.data = data
        self.query_params = query_params

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    # popping `serializer_class` out
    serializer_class = kwargs.pop("serializer_class")
    def decorator(func):
        def route_call(self, request, **params):
            if request:
                return func(self, request, **params)
            else:
                request = CustomRequest(params, params)
                self.request = request
                # add `serializer_class` to ViewSet `kwargs`
                params.update({"serializer_class": serializer_class})
                self.kwargs = params
                return func(self, request, **params)

        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(route_call)

Override get_serializer method of GenericViewSet:

class APIRestMixin(viewsets.GenericViewSet):
    """
    creates our custom get_serializer in order to use our serializer injection
    """

    def get_serializer(self, request_or_query_set=None, *args, **kwargs):
        serializer_class = self.kwargs.get("serializer_class", self.serializer_class)
        return serializer_class(*args, **kwargs)

Let’s connect everything together:

class CustomRequest:
    """
    Mock for a custom request
    """

    def __init__(self, data, query_params):
        self.data = data
        self.query_params = query_params

def action_api(methods=None, detail=None, url_path=None, url_name=None, **kwargs):
    def decorator(func):
        def route_call(self, request, **params):
            if request:
                return func(self, request, **params)
            else:
                # injecting our custom request
                request = CustomRequest(params, params)
                self.request = request
                # add `serializer_class` to ViewSet `kwargs`
                params.update({"serializer_class": serializer_class})
                self.kwargs = params
                return func(self, request, **params)

        return action(
            methods=methods,
            detail=detail,
            url_path=url_path,
            url_name=url_name,
            serializer_class=serializer_class,
            **kwargs
        )(route_call)


class APIRestMixin(viewsets.GenericViewSet):
    """
    creates our custom get_serializer in order to use our serializer injection
    """

    def get_serializer(self, request_or_query_set=None, *args, **kwargs):
        serializer_class = self.kwargs.get("serializer_class", self.serializer_class)
        return serializer_class(*args, **kwargs)


class DummyAPIViewSet(APIRestMixin, ModelViewSet):
    queryset = DummyModel.objects.all()
    serializer_class = DummySerializer

    def get_serializer_context(self):
        return {
            "request": self.request,
            "view": self,
            "pk": self.kwargs.get("pk"),
        }

    @action_api(detail=True, methods=["get"], serializer_class=DummySerializer)
    def dummy(self, request, **kwargs):
        serializer = self.get_serializer(instance=self.get_object())
        return Response(data=serializer.data, status=status.HTTP_200_OK)

    @action_api(detail=False, methods=["post"], serializer_class=GetDummyByIntSerializer)
    def by_dummy_int(self, request, **kwargs):
        self.get_serializer(data=request.data).is_valid(raise_exception=True)
        queryset = DummyModel.objects.filter(dummy_int=request.data["dummy_int"]).order_by("id")
        serializer = self.get_serializer(queryset, many=True)
        return Response(data=serializer.data, status=status.HTTP_200_OK)

Conclusion

In this article, we explored using the Django rest framework for API functions, delving into its internals, and adapting it to our needs. Leveraging rest frameworks offers various advantages such as:

Clear separation between function signature and business logic
Makes Django DB models accessible in other libraries/web services
Arguments validation provided by serializers
Internal pagination mechanism
And many more!

I hope you found this article insightful and feel motivated to implement these practices in your projects! The full code and drf-api-action project are available here.

Don’t forget to give it a star if you find it helpful! 🌟