Forem: Constantine

Ordering by Choice Label in Django

Constantine — Thu, 12 Aug 2021 15:55:15 +0000

Let's say you have a Choice field on your model. And you want to order by that choice labels. It's a no-brainer if values and labels are the same. Because values are stored in the database and it's the one in charge of ordering.

from django.db.models import TextChoices


class State(TextChoices):
    STARTED = 'STARTED', 'Started'
    WORKING = 'WORKING', 'Working'
    FINISHED = 'FINISHED', 'Finished'

This will work out of the box:

from django_filters import (
    FilterSet,
    ...,
    OrderingFilter,
)

class MyFilter(FilterSet):
    ...some fields filtering declaration...

    ordering = OrderingFilter(
        fields: {
            'state': 'state',
        }
    )

    class Meta:
        model = MyModel

But if labels differ from values it just won't work. Because now the database doesn't know anything about labels.

from django.db.models import TextChoices


class State(TextChoices):
    STARTED = 'STARTED', 'Initiated'
    WORKING = 'WORKING', 'Operating'
    FINISHED = 'FINISHED', 'Stopped'

What you'd need to do now is to sort State labels and then create Case > When > Then chains. This way the database will know how you want to order results (but it's still oblivious about the whole "labels" thing).

class MyOrderingFilter(OrderingFilter):
    def filter(self, qs, value):
        state = None
        if value:
            for idx, param in enumerate(value):
                if param.strip('-') == 'state':
                    state = value.pop(idx)

        qs = super(MyOrderingFilter, self).filter(qs, value)
        if state:
            sorted_states = {
                val[0]: idx
                for idx, val in enumerate(
                    sorted(
                        State.choices,
                        key=lambda x: x[1],
                        reverse='-' in state,
                    )
                )
            }
            qs = qs.order_by(
                Case(
                    *[
                        When(state=key, then=Value(val))
                        for key, val in sorted_states.items()
                    ]
                )
            )

        return qs

And then use MyOrderingFilter instead of OrderingFilter in MyFilter from the snippet above. The magic happens inside sorted_states = {...} dict comprehension. We tell our DB that we want our values with label "Initiated" to be first (or rather zero) in order, values with "Operating" to be second and so on. And when we're building qs = qs.order_by(Case(...)) we're just constructing the query itself.

So now your ordering will work according to the choice labels and not it's values.

Cleaning Up Space after Docker

Constantine — Fri, 19 Mar 2021 15:24:24 +0000

Photo by Wil Stewart on Unsplash

You know how I love Docker and try to move everything inside containers. But a couple days ago I got a system warning that I have about 1 Gb left on the root partition. That was a surprise. At the time I was in a Zoom call. And the first thing that came to my mind was "eh, must be Zoom eating up space". But in the span of 30 minutes 1 Gb turned to 100 Mb. So I panicked a little, turned everything off, rebooted and got angry that it was still less than 100 Mb on /.

Imagine my amazement when I discovered that /var/lib/docker/ was eating around 20 Gb! What could it be? Turns out it was.. logs! Tons of logs from all the containers I have. And I only have around 10 running daily. Plus 5 more that I spin up when I need them. Such as Elastic, Kafka and Logstash.

Thankfully, the solution was quite easy:

sudo sh -c "truncate -s 0 /var/lib/docker/containers/*/*-json.log"

This will remove all the logs. That alone freed me 16 Gb! My-my...

But I don't want to run this command with Crontab. Or, even worse, run it manually from time to time. There must be a way to set Docker logging policy, I thought. And there sure is. Official documentation says how to configure logging and set the max number of files with their size. So, now my /etc/docker/daemon.json looks like this:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "1",
    "env": "os,customer"
  }
}

On a MacOS it's even easier. Go to Docker Desktop -> Preferences -> Docker Engine and edit the configuration.

Meaning I'll have only one file under 100Mb. Which I think should be suffice for the logs.

But, this will not affect already created containers! You'll need to recreate every single one in order for them to respect the new logging config.

Hope this will save you some time and nerves if/when you'll encounter a similar issue. So you can just focus on writing quality code 😊

Black with Docker

Constantine — Sat, 13 Mar 2021 06:12:06 +0000

Photo by Jan Kopřiva on Unsplash

Fromatting Python code remotely with PyCharm

I'm sure you've heard about Black, a formatter for Python. It's opinionated, but really good. One thing I disagree with - is double quotes everywhere. I prefer single. But that is disable-able.

So, what's Docker doing here? Well, you can run Black as a service. Meaning you can skip installation on a local machine and deploy straight with containers.

Dockerfile

I've published a final image so you can just pull that from dockerhub. Or build it yourself. Here we go with a multi-stage build to make our container slim.

FROM python:3.9-slim AS builder
RUN apt update && apt install -y git build-essential \
    && pip install --upgrade pip setuptools wheel black[d]

FROM python:3.9-slim
COPY --from=builder /usr/local/ /usr/local/
EXPOSE 45484
ENTRYPOINT ["blackd", "--bind-host", "0.0.0.0", "--bind-port", "45484"]

First we build a builder image, installing dependencies and Black itself. Then we just copy what we need from builder to the final image. That way we'll have around 160 Mb in size.

Now you can build with:

docker build . -t blackd:latest

Container

Let's go to "Containers" side-menu in Portainer and smash the "Add container" button. Set:

name: blackd
image: blackd:latest OR ceeveeya/blackd:latest
publish port: 45484:45484

A handy screenshot:

And press "Deploy".

Usage

Now when we have working Black service all is left - to use that.

Pycharm

Add BlackConnect plugin and set config as on the screenshot below.

VSCode

Unfortunately, I couldn't find a similar extension for this IDE. But I'm not using VSCode for python development so.. ¯\_(ツ)_/¯ Let's hope somebody will make something like BlackConnect one day 😅

Now your code should be formatted every time you save it!

Setting Up Kafka in Docker

Constantine — Thu, 04 Mar 2021 17:23:44 +0000

"Here we go, another article on setting up some thing in Docker 😒", you might think. This time it's Kafka. And I wouldn't write about it if it was that trivial.

Thanks to wurstmeister, we have separate docker images for Kafka and Zookeeper. But setting this up my way without docker compose is a task in and of itself.

Zookeper

We'll need two volumes for this guy. You know how it goes, go to "Volumes" side-menu and smash the "Add volume" button. Name the first one zookeeper_data, and the second one - zookeeper_conf.

Next go to the "Containers" side menu, click the "Add container" button. Set:

name: zookeeper
image: wurstmeister/zookeeper:latest
publish port: 2181:2181
add volumes:
- /opt/zookeeper-3.4.13/data to zookeeper_data
- /opt/zookeeper-3.4.13/conf to zookeeper_conf

As always, a handy screenshot for you:

Now destroy that "Deploy the container" button and wait a little.

Kafka

This guy will need just one volume. Name it kafka_volume. Now pay attention when creating a container. Set:

name: kafka
image: wurstmeister/kafka:latest
publish port: 9092:9092
add volumes:
- /kafka to kafka_data
- bind /var/run/docker.sock to /var/run/docker.sock
env:
- KAFKA_ADVERTISED_HOST_NAME: 172.17.0.1
- KAFKA_ZOOKEEPER_CONNECT: 172.17.0.1:2181
- ZK: 172.17.0.1:2181
- HOST: 172.17.0.1:9092
- KAFKA_ADVERTISED_PORT: 9092

ZK and HOST envs are just for shortening commands later on.

Screenshot for sanity check:

And deploy.

Communicate

There is already a guide on how to use this stuff, but there is a mistake in the consuming part.

Create topic

But let's start with creating a topic. Go to ">_ Console" of the Kafka container. Or execute command below in your terminal session

docker exec -it kafka /bin/bash

Once inside container create topic just as in above mentioned guide:

kafka-topics.sh --create --topic topic \
--partitions 4 --zookeeper $ZK --replication-factor 1
kafka-topics.sh --describe --topic topic --zookeeper $ZK

You'll see messages about topic creation and it's description:

Created topic topic.

Topic: topic PartitionCount: 4 ReplicationFactor: 1 Configs:
Topic: topic Partition: 0 Leader: 1002 Replicas: 1002 Isr: 1002
Topic: topic Partition: 1 Leader: 1002 Replicas: 1002 Isr: 1002
Topic: topic Partition: 2 Leader: 1002 Replicas: 1002 Isr: 1002
Topic: topic Partition: 3 Leader: 1002 Replicas: 1002 Isr: 1002

Consumer

The guide says to run this:

kafka-console-consumer.sh --topic=topic --zookeeper=$ZK

But if you try to do that you'll get an error about zookeeper is not a recognized option.

So the correct command is this:

kafka-console-consumer.sh --topic=topic --bootstrap-server=$HOST

It didn't freeze, it just waits for messages to come.

Producer

Open another console or terminal session and connect to the kafka container. Execute:

kafka-console-producer.sh --topic=topic \
--broker-list=`broker-list.sh`

Now you can type some messages here and it'll show in another session where we have a consumer 🚀

Bonus

Inside kafka_app in the project repo, you'll find two short files with producer and consumer. This is just to show how we can use Kafka in Python:

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python consumer.py

In another terminal session:

source venv/bin/activate
python producer.py

Now look at the terminal with our python consumer and you'll see a topic, message key and value.

Also, if you didn't close previous sessions with the console consumer, you'll see the same message there. And publishing from the console producer will deliver a message to both console and python consumers.

Now you have one more tool in your toolbox! Go use this stuff 💪

Setting Up Local ELK Stack

Constantine — Wed, 17 Feb 2021 14:46:27 +0000

Photo by Chandler Cruttenden on Unsplash

As I promised in the previous post, here we'll take a look at how to spin up an ELK. Let's go! 🚀

Elastic

Straight to the point: open Portainer. Go to "Containers" and smash the "Add container" button. Then look at the screenshot below and fill values for Name, Image, Ports and Volumes. Maybe change Restart Policy to Unless stopped. One mild difference is that you need to click on "Advanced mode" in order to be able to pull an image from Elastic hub. In the screenshot it says "Simple mode" since I already chose advanced.

Then hit the "Deploy the container" button and wait a little.

Almost nothing new, if you've read my first article about dockerizing stuff you need. You did read that, right? 🤔

Here I'm not creating any volumes just because. But you most definitely can. Just map it to /usr/share/elasticsearch/data.

Kibana

First create a volume, for example kibana_data. Then create container as on the screenshot below:

Logstash

Man, am I tired to repeat this 😅 Create volume and compare the stuff you're typing with a screenshot:

Deploy and wait.

Next, configuring.

sudo nano /var/lib/docker/volumes/logstash_data/_data/config/logstash.yml

Don't know what is this, but it's important:

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://172.17.0.1:9200" ]

Now pipeline. I have zero explanation why it was necessary to separate main and pipeline configs, but when I was first figuring it out it took me 3 hours of swear, frustration and eye strain. Maybe it's somewhere in documentation but it's buried so good I couldn't find it.

sudo nano /var/lib/docker/volumes/logstash_data/_data/pipeline/logstash.conf

I went with this config below since I don't need beat or w/e it is called. I just want to POST logs to Logstash. And you can always configure that as you wish by referencing the official documentation.

input {
  http {
    port => 5044
  }
}

output {
  stdout {
    codec => json
  }
  elasticsearch {
    hosts => ["http://172.17.0.1:9200"]
    index => "logstash-%{+YYYY}"
  }
}

Now we try this! Simple curl -XPUT 'http://127.0.0.1:5044/' -d 'log' will suffice. Now navigate to http://0.0.0.0:5601/app/management. It's under "Management" / "Stack management" inside the toast menu. Choose "Index Management" below "Data" from the side menu. You can see our "logstash-2021" index. Or whatever year you live in. That means it has data!

Go to http://0.0.0.0:5601/app/discover. It's "Kibana" / "Discover" inside the toast menu. And what we'll see? Right, one hit we've just sent!

Here you can filter everything as you'd like 🐧

Hope you've learned something new from this article. And now you're a master of local deployment with Docker and a help from Portainer! Cheers and happy coding!

Monitoring Apps with Docker Containers

Constantine — Sun, 07 Feb 2021 16:16:26 +0000

Photo by Ibrahim Boran on Unsplash

I already wrote about my flow when I deploy one container per service and use it for all of my projects. So it's easier to get a development process up to speed.

This time I'll tell you about connecting containerized services so they can talk to each other. And we'll start with somewhat easy things like Prometheus and Grafana. And after that it will be a no-brainer for us to spin up an ELK stack. Let's go! 🚀

Side Note

All code examples and configs are available on my GitHub repo.

Metrics

If you didn't know what Prometheus is used for - it collects various metrics from a service. And a little side thing we need to do before setting up our God of Fire is to create a service we'll be monitoring. I chose it to be a simple Flask app because it requires almost zero efforts.

Flask App

Create a flask_app directory and put next two files there:

# flask_app/app.py
from flask import Flask
from prometheus_flask_exporter import PrometheusMetrics

app = Flask(__name__)
metrics = PrometheusMetrics(app)

# flask_app/wsgi.py
from app import app

Also create requirements.txt right next to previous two:

click==7.1.2
Flask==1.1.2
itsdangerous==1.1.0
Jinja2==2.11.3
MarkupSafe==1.1.1
prometheus-client==0.9.0
prometheus-flask-exporter==0.18.1
Werkzeug==1.0.1

Then create Dockerfile, same place:

FROM python:3.9

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

EXPOSE 8000

RUN apt-get update \
  && apt-get install -y build-essential \
  && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
  && apt-get clean -y && rm -rf /var/lib/apt/lists/* \
  && pip install --upgrade pip

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

ENTRYPOINT ["gunicorn", "wsgi:app", "--bind", "0.0.0.0:8000"]

That's quite mouthful but all it does is:

creating an image from python 3.9;
opens up port 8000;
updates everything;
and installs our great app.

Now we build!

docker build . -t flask_app

And now we run!!

docker run -d -p 8000:8000 flask_app --name flask_app

And on http://0.0.0.0:8000/metrics we can see a bunch of stuff!!1 💪

Prometheus

It's time to start metering. Open my our favorite tool to manage containers - Portainer. And first, create a volume. Actually, two volumes. If you've read my previous article you know how it goes. But in case you didn't (why didn't you tho?) - go to "Volumes" tab and hit that blue "Add volume" button. First volume will be used for Prometheus data. So let's name it prometheus_data. Second will be used for Prometheus config, therefore the name prometheus_config_data.

"Why data for config?" you might ask. Well, because apparently you can't access the Prometheus container via the Portainer console button. Or I'm just stupid 🤷‍♂️ But we need a way to edit config files without rebuilding container every time. Thus volume for config files.

Creating container, finally. Go to "Containers" and smash the "Add container" button. Then look at the screenshot below and fill values for Name, Image, Ports and Volumes. Maybe change Restart Policy to Unless stopped.

Then hit the "Deploy the container" button and wait a little.

Configure Prometheus

Since we ~~don't know how to~~ can't connect to a Prometheus container we will edit it's config file via terminal session. If you're using nano like I do then just execute next line and make it look the same as below:

sudo nano /var/lib/docker/volumes/prometheus_config_data/_data/prometheus.yml

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

  - job_name: 'flask_app'
    metrics_path: '/metrics'
    static_configs:
    - targets: ['172.17.0.1:8000']

It's the same default config but with additional job_name at the end. I decided to be explicit with metrics_path here, showing that we can change that if we need to. But the most important part is targets! You can just believe me that on Linux your host machine's IP address will be 172.17.0.1 for your docker containers. Or you can visit a "Network" tab in Portainer and take a look at "IPV4 IPAM Gateway" for a bridge interface.

Now restart Prometheus container and visit http://0.0.0.0:9090/targets. You should see "State" UP for both targets.

If we refresh our Great Flask App that serves us 404 on http://0.0.0.0:8000/ and then go to graph for total requests, we should see.. well.. a graph for total requests.

Cool, eh? Let's make it beautiful! 🐹

Grafana

Again, create a volume, for example grafana_data. Then create container as on screenshot below

Hit deploy, wait and go to http://0.0.0.0:3000/. Default login/password is admin/admin. You can change that after login.

Configure data source on http://0.0.0.0:3000/datasources. Or locate "Data Sources" tab hovering on ⚙️ sign on the left. Then click the blue "Add data source" button, select "Prometheus". Very important, as before, "URL" parameter inside the "HTTP" section should be 172.17.0.1:9090. Because Grafana can't access Prometheus container by host name. So we tell it (her?) to connect to our host machine IP address on the port where Prometheus is running. Then smash "Save & Test". As always, you can try to set "URL" to prometheus_container_name:port and after saving observe "HTTP Error Bad Gateway".

Now we can hover over ➕ sign and click on "Dashboard" under "Create". Or just visit http://0.0.0.0:3000/dashboard/new. Then smash "Add new panel". On the right side change "Title" to whatever your heart desires. I chose simple "Flask Total Requests". Below graph set "Metrics" to flask_http_request_total. Here is a handy screenshot for you:

Then click "Apply" in the top right corner. And now you have Grafana which makes beautiful graphs from Prometheus data which monitors your application!

In the next article we will set up ELK. But I guess you already know how to do that 😉

Dockerizing stuff you need

Constantine — Fri, 25 Dec 2020 15:29:51 +0000

Photo by Dominik Lückmann on Unsplash

When we develop something we all need to use some services such as PostgreSQL, Redis, RabbitMQ and many-many others. But I don't like the idea of installing each and every one of them on my local machine. So the idea of dockerizing everything looks very appealing.

In this article I'll show how to deploy three mentioned services locally with docker. And, as a bonus, easy way of configuring and monitoring our containers park via Portainer.

Bonus first

Portainer is such a great and powerful tool that I just love! Let's set it up and ease our next steps.

You need to create a volume with a simple terminal command

$ docker volume create portainer_data

and deploy Portainer

$ docker run -d -p 9443:9443 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce

All of this you can find in deploy section on the official documentation.

Then finish the initial setup. Again, everything is described in the docs

After it's done you can navigate to localhost:9443 and see the Portainer dashboard.

Now you can monitor your containers, attach to them, check logs, see and remove unused images and volumes, and do everything you could do from the terminal. But you don't need to google or remember those lengthy commands.

Allright, let's go!

PostgreSQL

As we need to have a persistent storage for our DB let's create a volume. Go to "Volumes" tab in Portainer and push that "Add volume" button. Give it a name, e.g. pg_data and hit "Create the volume".

Then go to the "Containers" tab and smash the "Add container" button.
Set Name: pg, Image: postgres:latest, on "Manual network port publishing" click plus sign and enter 5432 for both "host" and "container".

Next in the "Advanced container settings" section leave everything as is on "Command and logging tab". Move to the "Volumes" tab, hit plus sign and map /var/lib/postgresql/data to our pg_data volume. After that to the "Env" tab. Set POSTGRES_PASSWORD to, well, root password for the database. Let's say it will be password. I also like to set unless stopped on the "Restart policy" tab. Now we are ready to smash that "Deploy the container" button above our advanced settings section.

Here is an example screenshot:

Where do I get those variables and values? Docker Hub! Just open the service page and it's all there.

Also you can do everything by hand, but I don't remember how to create a volume with a custom name and map that to a container.

And we can now connect to Postgres via localhost:5432 as postgres user and password password!

Redis

If Postgres was quite a handful, Redis is much easier. The steps are similar.

Create volume
Create container
- from redis:latest
- with port binding 6379:6379;
- map created volume to /data;
- and, optionally, set restart policy.

Boom, done!

RabbitMQ

Next in line is Rabbit. It's so easy I don't even want to repeat 😀

Create volume
Create container
- from rabbitmq:3-management
- with port bindings 5672:5672 and 15672:15672 for management;
- map created volume to /var/lib/rabbitmq;
- and, if you want, set restart policy.

And again, done! 🎉

Summary

When using docker you can free yourself from the need to install every service you need. Not to mention if you need different versions of the same thing.

Hope this will ease your struggles with containers (if any) and you'll be able to focus on writing quality code!

About dualboot Arch + Win

Constantine — Sun, 08 Nov 2020 07:44:05 +0000

Photo by Avel Chuklanov on Unsplash

Below are just things to remember and do when installing two OS'es. Because I'm always forgetting something and then thinking "I should've written this down as a step-by-step guide" 😌.

Order matters

Always install Windows first, and then Arch. That way Win will create a boot partition and other stuff it needs. So when installing Arch you just need to create partitions for / and /home.

Windows stuff

Separate drives

Yes, drives C: and D: is a salvation. That way you can keep games, downloads, documents, etc. when reinstalling Win. So just do that and install Windows on drive C:. And D: will be the "storage".

UTC time

Just do this to spare yourself from thinking "Why the Windows time is X hours behind/ahead?".

run regedit
navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation
create new DWORD (32-bit) value named RealTimeIsUniversal
set this value to 1

Disable fastboot

It just messes with linux for some reason. So Settings > Power > Advanced (or w/e) params > Power button actions > Click on Change params > Uncheck Enable fast boot (recommended).

The case for reinstalling Windows

Before installing do all of the above, obviously. But when you reformat drive C: and try to install in that partition it will say something like "Not enough space on main partition". So you need to reformat boot partition too. Yes, it will wipe Arch bootloader but that is easy to restore.

Restoring Arch bootloader [rEFInd]

Boot with a live cd/usb and mount / to /mnt and /boot to /mnt/boot. Then arch-chroot in /mnt and install linux package. Then just refind-install, and edit /boot/EFI/refind.conf. Delete unnecessary lines and change ro to rw.

Maybe, after booting change refind theme.

Arch stuff

Partitioning

Set EFI partition created by Win as /boot when creating new table. And create separate partition for /home, just like drives C: and D: for Win. That's pretty much it.

`D:` aka Storage

When generating fstab don't forget to add Windows' drive D:, for example as a /storage.

Installation

Just follow the official wiki and you're good 😉

Simple React app to track time

Constantine — Wed, 05 Aug 2020 13:40:01 +0000

Photo by Aron Visuals on Unsplash

Idea

Every day I track time on tasks I'm working on. My typical flow is write down a task number, description and time spent in a text document right after I'm done with that task. So at the end of the day I can just copy-paste that info in task tracker. But sometimes I sit there calculating how many hours of work I have left. Did I overwork? Or maybe I have couple of hours left to start another task? And that's when I had an idea to write an app that would track all of it for me.

After I had a grasp at React and Redux I thought "that would be a great side project!". And after spending around 48 hours I can say it was worth it! I learned and once more repeated a lot of things. Here goes a brief description of my work process. And you can try the app here. It works without any backend (that's intentional). So everything is stored in your browser. Hence, if you change devices/browsers the tasks won't be there.

Work Process

I started with an idea to just display and edit my working hours (or total time) and to add tasks. So I'd need a couple of inputs and a button. That was easy enough. Then I needed to store those tasks in a local storage and display them. It didn't took long. But afterwards I realized that I need to edit the tasks. As a side note, I didn't want to use Redux here. And was hoping to lean on state. Unfortunately, that didn't fly. As I soon bumped into problems of updating the total time when the tasks are changing.

So came the Redux. At first I went with updating total time when the task added or edited. But when the task is edited hours and minutes can be both added or subtracted. So the logic became too complicated, meaning there was something wrong with this approach. Next attempt was to just sum all of the hours and minutes of every task on task add/edit and subtract that value from the total time. This time my code was much cleaner.

Also there were couple moments when I was like "oh, fck, I also need to %edit/add new feature%". But that taught me to think a little deeper (but not too deep) about a feature I'm about to implement.

TL:DR

If you're doing something mundane every day - think of ways you can automate that. And then how to actually implement your idea.

It's okay to think "I might need a library/plugin/technology X" (in my case Redux) but try to work it out without that. And add that library later when you sure your code/logic get's too complicated.

Try to think about a feature you're about to implement a little bit longer, give it some time. Maybe you'll see things you didn't notice at first.

Thank you for reading! If you've tried this little app and it was helpful - I'm really glad (and proud 🐺). If you've noticed any issues or bugs, please submit them on the project's page. If you want to see any features.. well, submit them too but I can't promise I'll implement them.

** or arguments unpacking in Python

Constantine — Sat, 04 Jul 2020 09:51:30 +0000

How you could use arguments unpacking

Photo by Markus Spiske on Unsplash

We all know *args and **kwargs in function parameters. It says that args are positional arguments however many you want to provide. And kwargs are keyword arguments, however many as well. We aren't limited to name them args and kwargs, it's just everyone do it and it's convenient to name them this way.

def foo(a, b, *args, **kwargs):
    print(a)
    print(b)
    print([arg for arg in args])
    print([f'{k}: {v}' for k, v in kwargs.items()])

foo(1, 2, 3, 4, f=5, g=6)
1
2
[3, 4]
['f: 5', 'g: 6']

1 and 2 are our required arguments. 3 and 4 would be our args. f=5 and g=6 are kwargs.

But the cool thing I started to implement not so long ago is arguments unpacking. Basically, you tell your function to get a list or dictionary of arguments and use values (or keys and values) as parameters.

foo(*[1, 2])
1
2
[]
[]

foo(1, 2, **{'f': 3})
1
2
[]
['f: 3']

In some weird scenario you might need a bunch of arguments in your function. So instead of declaring all those parameters you could use **kwargs. And instead of writing every parameter on function call you could use arguments unpacking.

Also, you can use unpacking as a dictionary update.

pets = {'cat': 1, 'dog': 1}
{'cat': 1, 'dog': 1}

animals = {'pig': 2, 'cow': 1, **pets}
{'pig': 2, 'cow': 1, 'cat': 1, 'dog': 1}

My use case for this was filtering a Django query. I had a bunch of filters but they all had a few common ones. So my code looked something like this:

finished_filter = {
    'cakes__status': CakeStatus.FINISHED, #  much more like this
}
Store.objects.annotate(
    finished=Count('cakes', filter=Q(**finished_filter),
    ready_to_ship=Count('cakes', filter=Q(**finished_filter, cakes__ready_to_ship=True)
)

You get the idea.

I think it's a really powerful tool but as always, don't abuse that power.

Django + React, Redux and JWT

Constantine — Wed, 06 May 2020 17:42:44 +0000

Preface

For the past 4 years I've been coding in Python and when it comes to web I always go with Django + Bootstrap and jQuery. I also know how to style a page with css but nothing super fancy. And I always stayed far away from modern SPAs and their frameworks/libraries such as React, Angular or Vue. I tried it once and the experience was horrible. I knew nothing about babel, webpack, yarn and how it all glues together. Not to mention arrow functions and destructuring in JS. But eventually I decided to try it one more time and spent I-don't-know-how-many hours watching tutorials on React. So now this is my attempt to make Django backend work with React frontend.

The goal of this article is to have a minimal backend with JSON Web Token authentication and simple frontend with login/logout functionality as well as a protected page for logged-in users only. And this is mostly for me to try to make it all work. So if and when I'll need to reproduce the steps in the future I could just review what I did and repeat the steps. And for that reason I decided to:

leave the default SQLite database so it could be replaced with what's needed
not to use any UI frameworks or any styles, because that would be opinionated and not suitable for every project

One more thing to note. I won't get in much details about the code listed here. There are tons of useful information out there if you want to really understand things. I'll list every resource that helped me on my way. This is just a how-to guide. And the complete code is available on my github, as well as gitlab.

With all that out of the way, embrace yourself for a long read! And I hope it will be of any use to you 😊

Prerequisites

You'll need to have following packages installed on your system: python (version 3, no legacy code here 😎), pip, node, npm, yarn. I'm using the Arch linux, so commands listed should be the same or similar to any other Unix-like system.

Let's start with creating a project directory, mkdir djact and cd into it. Then create virtual environment with python -m venv venv and activate it - source venv/bin/activate.

Creating Django project

Install Django, REST Framework and JWT handling with pip install django djangorestframework djangorestframework-simplejwt django-cors-headers. The last package is necessary to allow our development react server to interact with Django app. And let's save our dependencies after we install something: pip freeze > requirements.txt. Now start a new project django-admin startproject djact .. Note the . in the end, that's telling Django to create project in current directory.

Apps

I like all my apps in a separate directory as well as settings. So let's make it: mkdir djact/{apps, settings}. And move setting.py in a newly created settings directory. To make settings a package touch djact/settings/__init__.py and insert following lines in there:

# djact/settings/__init__.py
from .settings import *

Here and in every file listing first line will be a comment with a relative path to the file. Just so you know.

This way we won't need to override the DJANGO_SETTINGS_MODULE variable.

Core

Now create a directory for the core app mkdir djact/apps/core and the app itself python manage.py startapp core djact/apps/core. Inside this newly created directory mkdir {templates,templatetags}.
Create an empty __init__.py and react loader templatetag load_react.py inside templatetags dir:

# djact/apps/core/templatetags/load_react.py
from django import template
from django.conf import settings
from django.utils.safestring import mark_safe

register = template.Library()


@register.simple_tag
def load_react():
    css = load_css()
    js = load_js()
    return mark_safe(''.join(css + js))


def load_css():
    return [
        f'<link rel="stylesheet" href="/static/{asset}"/>'
        for asset in load_files('.css')
    ]


def load_js():
    return [
        f'<script type="text/javascript" src="/static/{asset}"></script>'
        for asset in load_files('.js')
    ]


def load_files(extension: str):
    files = []
    for path in settings.STATICFILES_DIRS:
        for file_name in path.iterdir():
            if file_name.name.endswith(extension):
                files.append(file_name.name)

    return files

I know there is a django-webpack-loader but I prefer a simpler approach like the above.

Next create index.html with the following content inside templates dir:

{# djact/apps/core/templates/index.html #}
{% load static %}
{% load load_react %}
<!doctype html>
<html lang="en">
    <head>
        <meta charset="utf-8"/>
        <title>Djact</title>
        <link rel="icon" href="{% static 'favicon.ico' %}">
    </head>
    <body>
        <div id="app"></div>
        {% load_react %}
    </body>
</html>

Authentication

Next we need an app for authentication, so mkdir djact/apps/authentication and python manage.py startapp authentication djact/apps/authentication. Inside this directory edit the models.py file:

# djact/apps/authentication/models.py
from django.contrib.auth.models import AbstractUser


class User(AbstractUser):
    class Meta:
        verbose_name = 'User'
        verbose_name_plural = 'Users'

    def __str__(self):
        return f'<{self.id}> {self.username}'

Next we need a serializer for users to sign up djact/apps/authentication/serializers.py:

# djact/apps/authentication/serializers.py
from rest_framework import serializers

from .models import User


class UserSerializer(serializers.ModelSerializer):
    email = serializers.EmailField(required=True)
    username = serializers.CharField()
    password = serializers.CharField(min_length=8, write_only=True)

    class Meta:
        model = User
        fields = ('email', 'username', 'password')
        extra_kwargs = {'password': {'write_only': True}}

    def create(self, validated_data):
        password = validated_data.pop('password', None)
        instance = self.Meta.model(**validated_data)
        if password is not None:
            instance.set_password(password)

        instance.save()

        return instance

Then the view djact/apps/authentication/views.py:

# djact/apps/authentication/views.py
from rest_framework import permissions
from rest_framework.generics import CreateAPIView
from rest_framework.response import Response
from rest_framework.views import APIView

from .serializers import UserSerializer


class UserCreate(CreateAPIView):
    permission_classes = (permissions.AllowAny,)
    authentication_classes = ()
    serializer_class = UserSerializer


user_create = UserCreate.as_view()


class Protected(APIView):
    def get(self, request):
        return Response(data={'type': 'protected'})


protected = Protected.as_view()

The Protected view is to check that we can access the page only after logging in.

And for the urls we'll have paths to our two views and also to obtain and refresh JWT:

# djact/apps/authentication/urls.py
from django.urls import path
from rest_framework_simplejwt import views as jwt_views

from . import views

app_name = 'authentication'
urlpatterns = [
    path(
        'users/create/',
        views.user_create,
        name='sign-up'
    ),
    path(
        'token/obtain/',
        jwt_views.TokenObtainPairView.as_view(),
        name='token-create'
    ),
    path(
        'token/refresh/',
        jwt_views.TokenRefreshView.as_view(),
        name='token-refresh'
    ),
    path(
        'protected/',
        views.protected,
        name='protected'
    )
]

Update main urls.py at djact:

# djact/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include('djact.apps.authentication.urls')),
]

Settings

I love the new Pathlib module so lets rewrite everything using this instead of os. I'm using django-environ to handle environment variables so let's install that pip install django-environ && pip freeze > requirements.txt. Copy DJANGO_SECRET_KEY from existing config so you won't need to generate a new one (although it's easy). We'll put that in a .env file.

# djact/settings/settings.py
import pathlib
from datetime import timedelta

import environ

BASE_DIR = pathlib.Path(__file__).parent.parent
PROJECT_ROOT = BASE_DIR.parent

env = environ.Env()

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.0/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = env.str('DJANGO_SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env.bool('DJANGO_DEBUG', False)

ALLOWED_HOSTS = env.list('DJANGO_ALLOWED_HOSTS', default=list())

# Application definition

INSTALLED_APPS = [
    'djact.apps.authentication',
    'djact.apps.core',

    'rest_framework',

    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'djact.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'djact.wsgi.application'

# Database
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': str(BASE_DIR.joinpath('db.sqlite3')),
    }
}

# Password validation
# https://docs.djangoproject.com/en/3.0/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

AUTH_USER_MODEL = 'authentication.User'

REST_FRAMEWORK = {
    'DEFAULT_PERMISSION_CLASSES': (
        'rest_framework.permissions.IsAuthenticated',
    ),
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    ),  #
}
SIMPLE_JWT = {
    'ACCESS_TOKEN_LIFETIME': timedelta(minutes=30),
    'REFRESH_TOKEN_LIFETIME': timedelta(days=30),
    'ROTATE_REFRESH_TOKENS': True,
    'BLACKLIST_AFTER_ROTATION': False,
    'ALGORITHM': 'HS256',
    'SIGNING_KEY': SECRET_KEY,
    'VERIFYING_KEY': None,
    'AUTH_HEADER_TYPES': ('JWT',),
    'USER_ID_FIELD': 'id',
    'USER_ID_CLAIM': 'user_id',
    'AUTH_TOKEN_CLASSES': ('rest_framework_simplejwt.tokens.AccessToken',),
    'TOKEN_TYPE_CLAIM': 'token_type',
}

LOGIN_URL = '/login'
LOGIN_REDIRECT_URL = '/'
LOGOUT_REDIRECT_URL = '/login'

# Internationalization
# https://docs.djangoproject.com/en/3.0/topics/i18n/

LANGUAGE_CODE = 'ru'

TIME_ZONE = 'Europe/Moscow'

USE_I18N = True

USE_L10N = True

USE_TZ = True

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.0/howto/static-files/

STATIC_URL = '/static/'
STATICFILES_DIRS = [
    PROJECT_ROOT.joinpath('static'),
]

STATIC_ROOT = PROJECT_ROOT / 'public' / 'static'
pathlib.Path(STATIC_ROOT).mkdir(exist_ok=True, parents=True)

MEDIA_URL = '/media/'
MEDIA_ROOT = PROJECT_ROOT / 'public' / 'media'
pathlib.Path(MEDIA_ROOT).mkdir(exist_ok=True, parents=True)

# Logging

LOG_DIR = PROJECT_ROOT / 'log'
LOG_DIR.mkdir(exist_ok=True)

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'console': {
            'format': '%(levelname)-8s %(name)-12s %(module)s:%(lineno)s\n'
                      '%(message)s'
        },
        'file': {
            'format': '%(asctime)s %(levelname)-8s %(name)-12s '
                      '%(module)s:%(lineno)s\n%(message)s'
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'console',
        },
        'file': {
            'class': 'logging.handlers.RotatingFileHandler',
            'formatter': 'file',
            'filename': LOG_DIR / 'django.log',
            'backupCount': 10,  # keep at most 10 files
            'maxBytes': 5 * 1024 * 1024  # 5MB
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['console', 'file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

LOGGING['loggers'].update(
    {app: {
        'handlers': ['console', 'file'],
        'level': 'DEBUG',
        'propagate': True,
    } for app in INSTALLED_APPS}
)

# Load dev config

if DEBUG:
    try:
        from .dev import *
    except ModuleNotFoundError:
        print('Dev config not found')

We can override some settings or add something related only to dev environment in djact/settings/dev.py that's why we need last 5 lines. My dev.py is looking like this:

# djact/settings/dev.py
from .settings import LOGGING, INSTALLED_APPS, MIDDLEWARE

LOGGING['handlers']['file']['backupCount'] = 1

INSTALLED_APPS += ['corsheaders']
CORS_ORIGIN_ALLOW_ALL = True
MIDDLEWARE.insert(2, 'corsheaders.middleware.CorsMiddleware')

Here we tell Django to allow interacting with our react dev server, which will be running on different port and therefore considered as cross origin.

Our .env.example file is looking like this:

<!-- .env.example -->
PYTHONDONTWRITEBYTECODE=1

DJANGO_SECRET_KEY=random long string
DJANGO_DEBUG=True for dev environment|False or omit completely for production
DJANGO_ALLOWED_HOSTS=localhost,127.0.0.1:8000,0.0.0.0:8000

So, create a .env file with those variables.

Now create urls.py inside djact/apps/core/ directory containing:

# djact/apps/core/urls.py
from django.urls import re_path
from django.views.generic import TemplateView

app_name = 'core'
urlpatterns = [
    re_path(r'^.*$', TemplateView.as_view(template_name='index.html'), name='index'),
]

And update main urls file:

# djact/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include('djact.apps.authentication.urls')),
    path('', include('djact.apps.core.urls')),
]

Then run python manage.py makemigrations and python manage.py migrate.

Our directory structure should look like this:

.
├── djact
│   ├── apps
│   │   ├── authentication
│   │   │   ├── admin.py
│   │   │   ├── apps.py
│   │   │   ├── __init__.py
│   │   │   ├── migrations
│   │   │   │   ├── 0001_initial.py
│   │   │   │   └── __init__.py
│   │   │   ├── models.py
│   │   │   ├── serializers.py
│   │   │   ├── urls.py
│   │   │   └── views.py
│   │   └── core
│   │       ├── admin.py
│   │       ├── apps.py
│   │       ├── __init__.py
│   │       ├── migrations
│   │       │   └── __init__.py
│   │       ├── templates
│   │       │   └── index.html
│   │       ├── templatetags
│   │       │   ├── __init__.py
│   │       │   └── load_react.py
│   │       └── urls.py
│   ├── asgi.py
│   ├── __init__.py
│   ├── settings
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── settings.py
│   ├── urls.py
│   └── wsgi.py
├── .env
├── .env.example
├── manage.py
└── requirements.txt

Creating React application

Let's mkdir for our React frontend and dive into it - mkdir frontend && cd frontend.

First initialize the frontend project with yarn init and answer the questions. Here is my example:

$ yarn init
yarn init v1.22.4
question name (frontend): djact
question version (1.0.0):
question description: Django + React
question entry point (index.js):
question repository url:
question author: Constantine
question license (MIT):
question private:
success Saved package.json
Done in 34.53s.

Now we can add dependencies with yarn add react react-dom axios react-redux redux redux-thunk reselect. And our dev dependencies with yarn add -D eslint babel-eslint babel-polyfill eslint-plugin-import eslint-plugin-react eslint-plugin-react-hooks eslint-loader style-loader css-loader postcss-loader webpack-dev-server mini-css-extract-plugin cssnano html-webpack-plugin npm-run-all rimraf redux-immutable-state-invariant webpack webpack-cli babel-loader @babel/core @babel/node @babel/preset-env @babel/preset-react.

Configuring

Create .eslintrc.js in current directory with following content:

// frontend/.eslintrc.js
module.exports = {
  parser: "babel-eslint",
  env: {
    browser: true,
    commonjs: true,
    es6: true,
    node: true,
    jest: true,
  },
  parserOptions: {
    ecmaVersion: 2020,
    ecmaFeatures: {
      impliedStrict: true,
      jsx: true,
    },
    sourceType: "module",
  },
  plugins: ["react", "react-hooks"],
  extends: [
    "eslint:recommended",
    "plugin:react/recommended",
    "plugin:react-hooks/recommended",
  ],
  settings: {
    react: {
      version: "detect",
    },
  },
  rules: {
    "no-debugger": "off",
    "no-console": "off",
    "no-unused-vars": "warn",
    "react/prop-types": "warn",
  },
};

Babel config is stored in babel.config.js:

// frontend/babel.config.js
module.exports = {
  presets: ["@babel/preset-env", "@babel/preset-react"],
};

Webpack config for dev environment stored in webpack.config.dev.js:

// frontend/webpack.config.dev.js
const webpack = require("webpack");
const path = require("path");
const HtmlWebpackPlugin = require("html-webpack-plugin");

process.env.NODE_ENV = "development";

module.exports = {
  mode: "development",
  target: "web",
  devtool: "cheap-module-source-map",
  entry: ["babel-polyfill", "./src/index"],
  output: {
    path: path.resolve(__dirname),
    publicPath: "/",
    filename: "bundle.js",
  },
  devServer: {
    historyApiFallback: true,
    headers: { "Access-Control-Allow-Origin": "*" },
    https: false,
  },
  plugins: [
    new webpack.DefinePlugin({
      "process.env.API_URL": JSON.stringify("http://localhost:8000/api/"),
    }),
    new HtmlWebpackPlugin({
      template: "./src/index.html",
      favicon: "./src/favicon.ico",
    }),
  ],
  module: {
    rules: [
      {
        test: /\.(js|jsx)$/,
        exclude: /node_modules/,
        use: [
          {
            loader: "babel-loader",
          },
          "eslint-loader",
        ],
      },
      {
        test: /(\.css)$/,
        use: ["style-loader", "css-loader"],
      },
    ],
  },
};

And edit package.json scripts section to make it look like this:

// frontend/package.json
{
  "name": "djact",
  "version": "1.0.0",
  "description": "Django + React",
  "scripts": {
    "start:dev": "webpack-dev-server --config webpack.config.dev.js --port 3000",
    "clean:build": "rimraf ../static && mkdir ../static",
    "prebuild": "run-p clean:build",
    "build": "webpack --config webpack.config.prod.js",
    "postbuild": "rimraf ../static/index.html"
  },
  "main": "index.js",
  "author": "Constantine",
  "license": "MIT",
  "dependencies": {
    ...
  },
  "devDependencies": {
    ...
  }
}

Now let's add a directory for the frontend sources: mkdir -p src/components. Also create entry point for React - touch src/index.js, with the following content:

// frontend/src/index.js
import React from "react";
import { render } from "react-dom";
import { BrowserRouter as Router } from "react-router-dom";
import App from "./components/App";

render(
  <Router>
    <App />
  </Router>,
  document.getElementById("app")
);

Create html template - touch src/index.html:

<!-- frontend/src/index.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Djact</title>
  </head>

  <body>
    <div id="app"></div>
  </body>
</html>

You can add a favicon inside src directory if you're fancy.

Then create the App component - touch src/components/App.js. Make it return something simple:

// frontend/src/components/App.js
import React from "react";

function App() {
  return <h1>Hello from React!</h1>;
}

export default App;

We can now test that our app is working with yarn start:dev. After navigating to http://localhost:3000 we should see a "Hello from React!" greeting!

And here is a production webpack.config.prod.js:

// frontend/webpack.config.prod.js
const webpack = require("webpack");
const path = require("path");
const HtmlWebpackPlugin = require("html-webpack-plugin");
const MiniCssExtractPlugin = require("mini-css-extract-plugin");

process.env.NODE_ENV = "production";

module.exports = {
  mode: "production",
  target: "web",
  devtool: "source-map",
  entry: {
    vendor: ["react", "react-dom", "prop-types"],
    bundle: ["babel-polyfill", "./src/index"],
  },
  output: {
    path: path.resolve(__dirname, "../static"),
    publicPath: "/",
    filename: "[name].[contenthash].js",
  },
  plugins: [
    new MiniCssExtractPlugin({
      filename: "[name].[contenthash].css",
    }),
    new webpack.DefinePlugin({
      // This global makes sure React is built in prod mode.
      "process.env.NODE_ENV": JSON.stringify(process.env.NODE_ENV),
      "process.env.API_URL": JSON.stringify("http://localhost:8000/api/"),
    }),
    new HtmlWebpackPlugin({
      template: "src/index.html",
      favicon: "./src/favicon.ico",
      minify: {
        // see https://github.com/kangax/html-minifier#options-quick-reference
        removeComments: true,
        collapseWhitespace: true,
        removeRedundantAttributes: true,
        useShortDoctype: true,
        removeEmptyAttributes: true,
        removeStyleLinkTypeAttributes: true,
        keepClosingSlash: true,
        minifyJS: true,
        minifyCSS: true,
        minifyURLs: true,
      },
    }),
  ],
  module: {
    rules: [
      {
        test: /\.(js|jsx)$/,
        exclude: /node_modules/,
        use: [
          {
            loader: "babel-loader",
          },
          "eslint-loader",
        ],
      },
      {
        test: /(\.css)$/,
        use: [
          MiniCssExtractPlugin.loader,
          {
            loader: "css-loader",
            options: {
              sourceMap: true,
            },
          },
          {
            loader: "postcss-loader",
            options: {
              plugins: () => [require("cssnano")],
              sourceMap: true,
            },
          },
        ],
      },
    ],
  },
};

Now we can yarn build and see our bundled file in static directory. And if we start our Django app via python manage.py runserver 0.0.0.0:8000 we would see exactly the same thing but running in production mode.

Our project directory should look like this:

.
├── djact
│   ├── apps
│   │   ├── authentication
│   │   │   ├── admin.py
│   │   │   ├── apps.py
│   │   │   ├── __init__.py
│   │   │   ├── migrations
│   │   │   │   ├── 0001_initial.py
│   │   │   │   └── __init__.py
│   │   │   ├── models.py
│   │   │   ├── serializers.py
│   │   │   ├── urls.py
│   │   │   └── views.py
│   │   └── core
│   │       ├── admin.py
│   │       ├── apps.py
│   │       ├── __init__.py
│   │       ├── migrations
│   │       │   └── __init__.py
│   │       ├── templates
│   │       │   └── index.html
│   │       ├── templatetags
│   │       │   ├── __init__.py
│   │       │   └── load_react.py
│   │       └── urls.py
│   ├── asgi.py
│   ├── db.sqlite3
│   ├── __init__.py
│   ├── settings
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── settings.py
│   ├── urls.py
│   └── wsgi.py
├── .env
├── .env.example
├── frontend
│   ├── babel.config.js
│   ├── package.json
│   ├── src
│   │   ├── components
│   │   │   └── App.js
│   │   ├── favicon.ico
│   │   ├── index.html
│   │   └── index.js
│   ├── webpack.config.dev.js
│   ├── webpack.config.prod.js
│   └── yarn.lock
├── log
│   └── django.log
├── manage.py
├── public
│   ├── media
│   └── static
├── requirements.txt
└── static
    ├── bundle.76ba356d74f1017eda2f.js
    ├── bundle.76ba356d74f1017eda2f.js.map
    ├── favicon.ico
    ├── vendor.9245c714f84f4bbf6bdc.js
    └── vendor.9245c714f84f4bbf6bdc.js.map

API service

Inside components directory create axiosApi.js:

// frontend/src/components/api/axiosApi.js
import axios from "axios";

const baseURL = process.env.API_URL;
const accessToken = localStorage.getItem("access_token");

const axiosAPI = axios.create({
  baseURL: baseURL,
  timeout: 5000,
  headers: {
    Authorization: accessToken ? "JWT " + accessToken : null,
    "Content-Type": "application/json",
    accept: "application/json",
  },
});

axiosAPI.interceptors.response.use(
  (response) => response,
  async (error) => {
    const originalRequest = error.config;

    // Prevent infinite loops
    if (
      error.response.status === 401 &&
      originalRequest.url === baseURL + "token/refresh/"
    ) {
      window.location.href = "/login/";
      return Promise.reject(error);
    }

    if (
      error.response.status === 401 &&
      error.response.statusText === "Unauthorized"
    ) {
      const refresh = localStorage.getItem("refresh_token");

      if (refresh) {
        const tokenParts = JSON.parse(atob(refresh.split(".")[1]));

        // exp date in token is expressed in seconds, while now() returns milliseconds:
        const now = Math.ceil(Date.now() / 1000);

        if (tokenParts.exp > now) {
          try {
            const response = await axiosAPI.post("/token/refresh/", {
              refresh,
            });
            setNewHeaders(response);
            originalRequest.headers["Authorization"] =
              "JWT " + response.data.access;
            return axiosAPI(originalRequest);
          } catch (error) {
            console.log(error);
          }
        } else {
          console.log("Refresh token is expired", tokenParts.exp, now);
          window.location.href = "/login/";
        }
      } else {
        console.log("Refresh token not available.");
        window.location.href = "/login/";
      }
    }

    // specific error handling done elsewhere
    return Promise.reject(error);
  }
);

export function setNewHeaders(response) {
  axiosAPI.defaults.headers["Authorization"] = "JWT " + response.data.access;
  localStorage.setItem("access_token", response.data.access);
  localStorage.setItem("refresh_token", response.data.refresh);
}

export default axiosAPI;

And authenticationApi.js:

// frontend/src/components/api/authenticationApi.js
import axiosAPI, { setNewHeaders } from "./axiosApi";

export async function signUp(email, username, password) {
  const response = await axiosAPI.post("users/create/", {
    email,
    username,
    password,
  });
  localStorage.setItem("user", response.data);
  return response;
}

export async function obtainToken(username, password) {
  const response = await axiosAPI.post("token/obtain/", {
    username,
    password,
  });
  setNewHeaders(response);
  return response;
}

export async function refreshToken(refresh) {
  const response = await axiosAPI.post("token/refresh/", {
    refresh,
  });
  setNewHeaders(response);
  return response;
}

// eslint-disable-next-line
export async function logout(accessToken) {
  localStorage.removeItem("access_token");
  localStorage.removeItem("refresh_token");
  // TODO: invalidate token on backend
}

export const isAuthenticated = () => {
  const token = localStorage.getItem("access_token");
  return !!token;
};

Redux

First create redux directory under djact/frontend/src/ and put following files there:

// frontend/src/redux/configureStore.dev.js
import { createStore, applyMiddleware, compose } from "redux";
import rootReducer from "./reducers";
import reduxImmutableStateInvariant from "redux-immutable-state-invariant";
import thunk from "redux-thunk";

export default function configureStore(initialState) {
  const composeEnhancers =
    window.__REDUX_DEVTOOLS_EXTENSION_COMPOSE__ || compose; // add support for Redux dev tools

  return createStore(
    rootReducer,
    initialState,
    composeEnhancers(applyMiddleware(thunk, reduxImmutableStateInvariant()))
  );
}

// frontend/src/redux/configureStore.prod.js
import { createStore, applyMiddleware } from "redux";
import rootReducer from "./reducers";
import thunk from "redux-thunk";

export default function configureStore(initialState) {
  return createStore(rootReducer, initialState, applyMiddleware(thunk));
}

// frontend/src/redux/configureStore.js
// Use CommonJS require below so we can dynamically import during build-time.
if (process.env.NODE_ENV === "production") {
  module.exports = require("./configureStore.prod");
} else {
  module.exports = require("./configureStore.dev");
}

Store is configured, now to actions! Create actions directory inside redux with following files:

// frontend/src/redux/actions/types.js
export const LOGIN_USER_SUCCESS = "LOGIN_USER_SUCCESS";
export const LOGOUT_USER = "LOGOUT_USER";

// frontend/src/redux/actions/auth.js
import { LOGIN_USER_SUCCESS, LOGOUT_USER } from "./types";
import { obtainToken, logout } from "../../components/api/authenticationApi";

export function loginUserSuccess(token) {
  return { type: LOGIN_USER_SUCCESS, token };
}

export function loginUser(username, password) {
  return async function (dispatch) {
    try {
      const response = await obtainToken(username, password);
      dispatch(loginUserSuccess(response.data.access));
    } catch (error) {
      console.log("Error obtaining token. " + error);
    }
  };
}

export function logoutUserSuccess() {
  return { type: LOGOUT_USER };
}

export function logoutUser() {
  return async function (dispatch) {
    await logout();
    dispatch(logoutUserSuccess());
  };
}

And the final step for redux is reducers themself, inside frontend/src/redux/reducers directory.

// frontend/src/redux/reducers/initialState.js
export default {
  accessToken: localStorage.getItem("access_token"),
};

// frontend/src/redux/reducers/auth.js
import * as types from "../actions/types";
import initialState from "./initialState";

export default function authReducer(state = initialState.accessToken, action) {
  switch (action.type) {
    case types.LOGIN_USER_SUCCESS:
      return action.token;
    case types.LOGOUT_USER:
      return "";
    default:
      return state;
  }
}

// frontend/src/redux/reducers/index.js
import { combineReducers } from "redux";
import auth from "./auth";

const rootReducer = combineReducers({
  auth,
});

export default rootReducer;

Now we need to register everything in index.js:

// frontend/src/index.js
import React from "react";
import {render} from "react-dom";
import {BrowserRouter as Router} from "react-router-dom";
import {Provider as ReduxProvider} from "react-redux";

import App from "./components/App";
import configureStore from "./redux/configureStore";

const store = configureStore();

render(
  <ReduxProvider store={store}>
    <Router>
      <App/>
    </Router>
  </ReduxProvider>,
  document.getElementById("app")
);

Components

Authentication

We have our reducers ready and now we need to put them to use. So let's create authentication dir inside frontend/src/components and put the next three files there.

This is will be our wrapper for private routes:

// frontend/src/components/authentication/PrivateRoute.js
import React from "react";
import { Redirect, Route } from "react-router-dom";
import PropTypes from "prop-types";
import { isAuthenticated } from "../api/authenticationApi";

const PrivateRoute = ({ component: Component, ...rest }) => (
  <Route
    {...rest}
    render={(props) =>
      isAuthenticated() ? (
        <Component {...props} />
      ) : (
        <Redirect
          to={{ pathname: "/login", state: { from: props.location } }}
        />
      )
    }
  />
);

PrivateRoute.propTypes = {
  component: PropTypes.func.isRequired,
  location: PropTypes.object,
};

export default PrivateRoute;

// frontend/src/components/authentication/LoginPage.js
import React, { useState } from "react";
import { connect } from "react-redux";
import PropTypes from "prop-types";
import { loginUser } from "../../redux/actions/auth";

const LoginPage = ({ loginUser, history }) => {
  const [state, setState] = useState({
    username: "",
    password: "",
  });

  const handleChange = (event) => {
    const { name, value } = event.target;
    setState({ ...state, [name]: value });
  };

  const login = async (event) => {
    event.preventDefault();
    const { username, password } = state;

    await loginUser(username, password);
    history.push("/");
  };

  return (
    <div>
      <h1>Login page</h1>
      <form onSubmit={login}>
        <label>
          Username:
          <input
            name="username"
            type="text"
            value={state.username}
            onChange={handleChange}
          />
        </label>
        <label>
          Password:
          <input
            name="password"
            type="password"
            value={state.password}
            onChange={handleChange}
          />
        </label>
        <input type="submit" value="Submit" />
      </form>
    </div>
  );
};

LoginPage.propTypes = {
  loginUser: PropTypes.func.isRequired,
  history: PropTypes.object.isRequired,
};

const mapDispatchToProps = {
  loginUser,
};

export default connect(null, mapDispatchToProps)(LoginPage);

And the Sign Up component will be simple because I was lazy to implement this but it should be easy enough:

// frontend/src/components/authentication/SignUpPage.js
import React from "react";
import { useHistory } from "react-router-dom";

const SignUpPage = () => {
  const history = useHistory();

  const handleClick = () => {
    history.push("/");
  };

  return (
    <div>
      <h1>Sign Up page</h1>
      <button onClick={handleClick}>sign up</button>
    </div>
  );
};

export default SignUpPage;

Common

Common components will contain only Header. But in theory there could live everything.. you know.. common.

// frontend/src/components/common/Header.js
import React from "react";
import { connect } from "react-redux";
import PropTypes from "prop-types";
import { NavLink, useHistory } from "react-router-dom";
import { logoutUser } from "../../redux/actions/auth";

const Header = ({ accessToken, logoutUser }) => {
  const history = useHistory();

  const handleLogout = async () => {
    await logoutUser();
    history.push("login/");
  };

  return (
    <nav>
      {accessToken ? (
        <>
          <NavLink to="/">Profile</NavLink>
          {" | "}
          <NavLink to="/logout" onClick={handleLogout}>
            Logout
          </NavLink>
        </>
      ) : (
        <>
          <NavLink to="/login">Login</NavLink>
          {" | "}
          <NavLink to="/sign-up">SignUp</NavLink>
        </>
      )}
    </nav>
  );
};

Header.propTypes = {
  accessToken: PropTypes.string,
  logoutUser: PropTypes.func.isRequired,
};

function mapStateToProps(state) {
  return {
    accessToken: state.auth,
  };
}

const mapDispatchToProps = {
  logoutUser,
};

export default connect(mapStateToProps, mapDispatchToProps)(Header);

Core

And the final piece is core components with application logic. Here we'll have our protected page:

// frontend/src/components/core/ProfilePage.js
import React from "react";
import axiosAPI from "../api/axiosApi";

const ProfilePage = () => {
  const handleClick = async () => {
    const response = await axiosAPI.get("protected/");
    alert(JSON.stringify(response.data));
  };
  return (
    <div>
      <h1>Profile page</h1>
      <p>Only logged in users should see this</p>
      <button onClick={handleClick}>GET protected</button>
    </div>
  );
};

export default ProfilePage;

Last thing to do is to update our App.js:

// frontend/src/components/App.js
import React from "react";
import {Route, Switch} from "react-router-dom";

import PageNotFound from "./PageNotFound";
import Header from "./common/Header";
import ProfilePage from "./core/ProfilePage";
import PrivateRoute from "./authentication/PrivateRoute";
import LoginPage from "./authentication/LoginPage";
import SignUpPage from "./authentication/SignUpPage";

function App() {
  return (
    <>
      <Header/>
      <Switch>
        <PrivateRoute exact path="/" component={ProfilePage}/>
        <Route path="/login" component={LoginPage}/>
        <Route path="/sign-up" component={SignUpPage}/>
        <Route component={PageNotFound}/>
      </Switch>
    </>
  );
}

export default App;

Our final project structure should look like this:

.
├── blogpost.md
├── djact
│   ├── apps
│   │   ├── authentication
│   │   │   ├── admin.py
│   │   │   ├── apps.py
│   │   │   ├── __init__.py
│   │   │   ├── migrations
│   │   │   │   ├── 0001_initial.py
│   │   │   │   └── __init__.py
│   │   │   ├── models.py
│   │   │   ├── serializers.py
│   │   │   ├── urls.py
│   │   │   └── views.py
│   │   └── core
│   │       ├── admin.py
│   │       ├── apps.py
│   │       ├── __init__.py
│   │       ├── migrations
│   │       │   └── __init__.py
│   │       ├── templates
│   │       │   └── index.html
│   │       ├── templatetags
│   │       │   ├── __init__.py
│   │       │   └── load_react.py
│   │       └── urls.py
│   ├── asgi.py
│   ├── db.sqlite3
│   ├── __init__.py
│   ├── settings
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── settings.py
│   ├── urls.py
│   └── wsgi.py
├── .env
├── .env.example
├── frontend
│   ├── babel.config.js
│   ├── package.json
│   ├── src
│   │   ├── components
│   │   │   ├── api
│   │   │   │   ├── authenticationApi.js
│   │   │   │   └── axiosApi.js
│   │   │   ├── App.js
│   │   │   ├── authentication
│   │   │   │   ├── LoginPage.js
│   │   │   │   ├── PrivateRoute.js
│   │   │   │   └── SignUpPage.js
│   │   │   ├── common
│   │   │   │   └── Header.js
│   │   │   ├── core
│   │   │   │   └── ProfilePage.js
│   │   │   └── PageNotFound.js
│   │   ├── favicon.ico
│   │   ├── index.html
│   │   ├── index.js
│   │   └── redux
│   │       ├── actions
│   │       │   ├── auth.js
│   │       │   └── types.js
│   │       ├── configureStore.dev.js
│   │       ├── configureStore.js
│   │       ├── configureStore.prod.js
│   │       └── reducers
│   │           ├── auth.js
│   │           ├── index.js
│   │           └── initialState.js
│   ├── webpack.config.dev.js
│   ├── webpack.config.prod.js
│   ├── yarn-error.log
│   └── yarn.lock
├── log
│   └── django.log
├── manage.py
├── public
│   ├── media
│   └── static
├── requirements.txt
└── static
    ├── bundle.c86ace9a42dd5bd70a59.js
    ├── bundle.c86ace9a42dd5bd70a59.js.map
    ├── favicon.ico
    ├── vendor.0d40e04c29796a70dc89.js
    └── vendor.0d40e04c29796a70dc89.js.map

Running

Now, set environment variables export $(cat .env | xargs). Build the frontend part cd frontend && yarn:build. Create superuser for testing with cd ../ && python manage.py createsuperuser and follow instructions. Run Django app python manage.py runserver and navigate to http://localhost:8000. We should see our login page. Enter credentials you provided when created superuser and we'll get to a protected Profile page. If we click on a GET protected button we would see the alert with response from the server.

And that's it! If you're came all the way down here.. wow! And if you've actually implemented all of this.. WOW!! Outstanding job, my friend! Hope you've learned new things or solved a problem of yours 🚀

Thank you and happy coding!

Resources

As I promised at the beginning of this article, here is a list of every resource that helped me to build this whole thing:

PluralSight courses:

Articles:

Why ForeignKey reference in Django model's str method is a bad idea

Constantine — Sat, 28 Mar 2020 07:22:45 +0000

Cutting database queries by not referencing Foreign Keys. Or fixing bad behavior.

Photo by israel palacio on Unsplash

So you have a model with foreign key:

from django.db import models
from .models import Author

class Book(models.Model):
    name = models.CharField(...)
    author = models.ForeignKey(to=Author,...)

Now you want to see an author in your admin panel. Easy, right? Just add:

def __str__(self):
    return f'{self.id} {self.author.name}'

What this will do is hit our database for an author name for every row present on a page. Imagine there are 100 rows. That's 100 more hits. This problem is known as the n+1 query.

The solution is quite simple. If that's a ForeignKey, you need to use select_related on your queryset. If that's a ManyToMany field or reverse FK, then you need a prefetch_related method. And, by the way, that's the difference between those two methods. It's a common interview question.

But in our case it's not a simple query. If you really want to fix that you would need to dig deeper into how Django admin works and provide custom queryset to the view.

When I encountered this problem the solution that has been accepted was just removing the reference to ForeingKey from the __str__ method ¯\_(ツ)_/¯

A little ToDo for myself: find a way to solve this via select_related.

UPDATE after half a year

For some time I didn't face this problem but not long ago I did. So, as I said, the solution to this is quite simple. All we need to do is to redefine get_queryset method on admin model. For example:

@admin.register(models.Book)
class BookAdmin(admin.ModelAdmin):
    def get_queryset(self, request):
        qs = self.model._default_manager.select_related(
            'author',
        )
        # This stuff is marked as TODO in Django 3.1
        # so it might change in the future
        ordering = self.get_ordering(request)
        if ordering:
            qs = qs.order_by(*ordering)

        return qs

Now there will be just one query per record, containing JOIN clause, hitting our database.

Optimization FTW 😏

Forem: Constantine

Ordering by Choice Label in Django

Cleaning Up Space after Docker

Black with Docker

Fromatting Python code remotely with PyCharm

Dockerfile

Container

Usage

Pycharm

VSCode

Setting Up Kafka in Docker

Zookeper

Kafka

Communicate

Create topic

Consumer

Producer

Bonus

Setting Up Local ELK Stack

Elastic

Kibana

Logstash

Monitoring Apps with Docker Containers

Side Note

Metrics

Flask App

Prometheus

Configure Prometheus

Grafana

Dockerizing stuff you need

Bonus first

PostgreSQL

Redis

RabbitMQ

Summary

About dualboot Arch + Win

Order matters

Windows stuff

Separate drives

UTC time

Disable fastboot

The case for reinstalling Windows

Restoring Arch bootloader [rEFInd]

Arch stuff

Partitioning

D: aka Storage

Installation

Simple React app to track time

Idea

Work Process

TL:DR

** or arguments unpacking in Python

How you could use arguments unpacking

Django + React, Redux and JWT

Preface

Prerequisites

Creating Django project

Apps

Core

Authentication

Settings

Creating React application

Configuring

API service

Redux

Components

Authentication

Common

Core

Running

Resources

Why ForeignKey reference in Django model's __str__ method is a bad idea

Cutting database queries by not referencing Foreign Keys. Or fixing bad behavior.

UPDATE after half a year

`D:` aka Storage

Why ForeignKey reference in Django model's str method is a bad idea