Forem: myk_okoth_ogodo

Lets Build a Digital /E-Wallets API

myk_okoth_ogodo — Sun, 05 Nov 2023 13:39:18 +0000

Project Code:

All the code contained herein can be found at: E_WalletZ. Please feel free to clone, pull,star or use it as you like. Also, don't forget to follow me for more projects on React, React-Native, Python(Django) and Go-lang. Happy Learning.

Disclaimer !!.

Please note that code contained in this project is nowhere near close to being production grade.This project is easy enough for mid-level developers, for beginners you will need to make chat-GPT your friend(i Assume you have a pretty good understanding of python as a language and Django as a framework),if you don't understand a block of code just paste it on chat-GPT and it will give you an in-depth explanation. I hope it will be fun project to build and learn with.

Prelude

In today's digital world, finance in its essence has entirely gone online. By the essence of finance, i mean the holding, moving and exchange of money for goods and services . Based on these macro-events, we have seen a push by countless startups to bank the unbanked, and the easiest way to do this is to use the smartphones that have found their way into the hands of most of earths population. Simply ask the people to register for an application account and then they can use this account to send, receive money, purchase goods and services, access overdraft based on expenditures, access insurance etc.

In this article i want us to build simple wallets to hold cash or tokens,The wallets will allow for depositing of cash, withdrawal of cash and transfer of cash between wallets of different persons.

Technologies Used:

Django Framework

This is a high-level python framework that can be used to build almost any type of website. We will be using it to build our API.

Graphql

This is an open source API query language, it majorly describes how a client should ask for information through an API. We will be using graphl to present our back-end data for clients. I might have as well chosen to use the REST framework, but to avoid over-fetching and under-fetching and make it easier for front-end integration i chose graphql. BTW, let me know in the comments if you would like me to make a follow up tutorial building a React-Native app to consume the API.

AWS S3(Simple Storage Service)

Amazon web services is one of the most popular cloud computing provider. We will be using one of it most popular services called s3. s3 stands for simple storage service and its an object storage service. We will use it to store the static and media files for Django and Graphql.

JWT(Json Web Token)

We will be using this for user authentication. Still a lightweight authentication method but it will do for this case. Please remember that for production level projects the authentication procedures have to be more stringent, JWT wont do. JWT object will carry our users information.

Docker Containers

We will containerize our service and will also run the PostgreSQL database in a container. A docker container image is a lightweight, standalone executable package of software that includes everything needed to run our application.

Postgres Database

This is a relational database. It supports both relational(SQL) and non-relational(JSON) functions. Postgres is better than MySQL in terms of read-write operations, dealing with massive datasets and executing complicated queries.

Pre-requisites

Setting up the developer environment

Operating system
I am assuming, you will be using a UNIX based OS for development, in my case i am using Ubuntu 22.04.3 LTS. If you are using windows please install WSL(Windows Subsystem for Linux), then install Ubuntu.Follow the tutorial here.

virtualenv
To set up our environment we are going to create a virtual environment, what this does is it creates a separate environment from the systems' environment, where you can add all you packages and libraries without mixing those up with the rest of the systems environment installed packages, it creates sort of a "sandbox" environment, so lets go ahead and create one:

check if you have virtualenv installed on your system.



virtualenv --version

"cd" into your folder where you do your development.
Make a directory called WalletZ, and then "cd" into this new directory.Now in this WalletZ directory, lets create a virtual environment as shown below:



mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ virtualenv venv
created virtual environment CPython3.10.12.final.0-64 in 889ms
  creator CPython3Posix(dest=/home/mykmyk/data/Development/Django/EcommerceProjects/WalletZ/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mykmyk/.local/share/virtualenv)
    added seed packages: pip==23.0.1, setuptools==67.6.0, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ ls -la
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 21:46 .
drwxrwxr-x 5 mykmyk mykmyk 4096 Okt 22 20:58 ..
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 21:46 venv
mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$

lets then activate that environment as shown below:



mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ source venv/bin/activate
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ ls -la
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 21:46 .
drwxrwxr-x 5 mykmyk mykmyk 4096 Okt 22 20:58 ..
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 21:46 venv
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$

Now that we have activated our virtualenv, lets install the Django framework version 3.1.5.We are using pip the package manager for python to do this.



(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ pip install Django==3.1.5

We then create out Django Project "E-WalletZ" to house our apps, as shown in the code below:



(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ django-admin startproject E_WalletZ
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ$ ls
E_WalletZ  venv

Enter into the new E_WalletZ directory, we want to create our apps:



(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ django-admin startapp Wallet
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ ls -la
total 20
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 22:54 .
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 22:23 ..
drwxrwxr-x 2 mykmyk mykmyk 4096 Okt 22 22:23 E_WalletZ
-rwxrwxr-x 1 mykmyk mykmyk  665 Okt 22 22:23 manage.py
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:54 Wallet
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ django-admin startapp Transanctions
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ ls
E_WalletZ  manage.py  Transanctions  Wallet
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ ls -la
total 24
drwxrwxr-x 5 mykmyk mykmyk 4096 Okt 22 22:54 .
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 22:23 ..
drwxrwxr-x 2 mykmyk mykmyk 4096 Okt 22 22:23 E_WalletZ
-rwxrwxr-x 1 mykmyk mykmyk  665 Okt 22 22:23 manage.py
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:55 Transanctions
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:54 Wallet
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ django-admin startapp user_controller
(venv) mykmyk@mykmyk-Lenovo-G50-30:~/data/Development/Django/EcommerceProjects/WalletZ/E_WalletZ$ ls -la
total 28
drwxrwxr-x 6 mykmyk mykmyk 4096 Okt 22 22:55 .
drwxrwxr-x 4 mykmyk mykmyk 4096 Okt 22 22:23 ..
drwxrwxr-x 2 mykmyk mykmyk 4096 Okt 22 22:23 E_WalletZ
-rwxrwxr-x 1 mykmyk mykmyk  665 Okt 22 22:23 manage.py
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:55 Transanctions
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:55 user_controller
drwxrwxr-x 3 mykmyk mykmyk 4096 Okt 22 22:54 Wallet

Move into the E_Walletz directory that houses the files "asgi.py", "init.py", "settings.py","urls.py", "wsgi.py".

A little explanation of the files:

asgi.py
This is the asynchronous server gateway interface,it defines an interface between asynchronous python web servers and applications and also supports all features provided by WSGI(web server gateway interface).

wsgi.py
This file defines a communication interface between a web server and a python web application. Since its synchronous its less suitable for handling long-lived connections.

settings.py
This file houses all the configurations to the project we are working on. It contains database configurations, middlewares(for Django and Graphql), security settings. It provides with a centralized project configuration.

urls.py
Its in this file that we sew together specific urls to their respective views or handlers, what is known as URL routing.In this file we define how incoming request are mapped to their respective views.

Now open the "settings.py" file, in that we import the OS package and if you look at the "INSTALLED_APPS" list we add all the apps we just created "user_controller", "Transanctions", "Wallet".

"settings.py"



from pathlib import Path
import os

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-n18use^q@nwz&)s#os&zxlhy=3d57ys1wubfwi036e4h)7rol8'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []


# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'user_controller',
    'Transanctions',
    'Wallet'
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'E_WalletZ.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'E_WalletZ.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = 'static/'

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

AWS CONNECTION

Storing our Static Files In An AWS s3 Bucket

We now want to push static files related to our Django app to the AWS S3 bucket, lets first head to AWS , if you have an account that is perfect, if not then here is a link to help you through the process of creating one here. After you have created an account ,generate access keys for that account(this will be used by Django to access the AWS resources, in this case the s3 bucket). Here is a link to guide you in creating access keys.

Creating an s3 Bucket:

On the AWS console search for s3, which stand for simple storage service, its a service that enables for object storage of virtually any size. We want to create a new s3 Bucket called "e-walletz" as shown below:

choose the region as "Europe(London) eu-west-2", on the Object Ownership choose "ACLs enabled " as shown below :

Leave all other settings as they are and proceed to create the "e-walletz" bucket, after it has completed creating enter into the bucket and choose "create a folder" button, create a folder named "static" as shown below:

One last step ......

On the "e-walletz" page, click on "permissions" tab and scroll down to bucket policy, click on edit :

Then Click on Policy generator :

We want to allow access to our Bucket and its children, allowing essentially all actions for our principal .(PS: to be safe you can restrict the actions to uploads and downloads only)

generate the access policy and then copy paste it onto the text field that is supposed to hold our Bucket policy.

Phew !.... back to Django now..

Create a ".env" file at the project root level as below:

The contents of this .env file should be as follows, remember this file holds our environment variables ,and we will plug those into the "settings.py" using decouple tool.

".env"



DEBUG=True

DB_NAME=E_WalletZ_user
DB_USER=E_WalletZ_user
DB_PASSWORD=E_WalletZ_password
DB_HOST= 'here you will put the ip address of your host machine'          
DB_PORT=5432


AWS_STORAGE_BUCKET_NAME='e-walletz'
AWS_S3_ACCESS_KEY_ID='input the access key ID we generated earlier'                    
AWS_S3_SECRET_ACCESS_KEY='input the secret access key generated earlier'                                        
AWS_HOST_REGION='eu-west-2'
S3_BUCKET_URL='https://e-walletz.s3.eu-west-2.amazonaws.com/static/'

You will notice that i have indicated that for "DB_HOST" you will put your host machine IP address, this is a functionality that will be scripted when doing automated deployment(CI/CD). To find machine IP use the following command:



ifconfig | grep "172."

Now head to the "settings.py" file, located here :

Install decouple package using the command:
The decouple package helps us strictly separate the settings parameters from the source code. As you go on you will notice i haven't strictly applied myself to this.



pip install python-decouple==3.4
pip freeze > requirements.txt

The contents of the file should look like below, notice we import config from decouple at the top, then we use it to plug in all the environment variables we just set in our ".env" file :

"settings.py"



from pathlib import Path
import os
from decouple import config

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-n18use^q@nwz&)s#os&zxlhy=3d57ys1wubfwi036e4h)7rol8'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ["*"]



# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'user_controller',
    'Transanctions',
    'Wallet'
]



MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'E_WalletZ.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'E_WalletZ.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = 'static/'

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field



#AWS CONFIG
# AWS CONFIG
AWS_STATIC_LOCATION = 'static'
S3_BUCKET_URL       = config('S3_BUCKET_URL')
STATIC_ROOT         = 'staticfiles'

AWS_ACCESS_KEY_ID     =  config('AWS_S3_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY =  config('AWS_S3_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')
AWS_HOST_REGION         = config('AWS_HOST_REGION')
AWS_S3_CUSTOM_DOMAIN    = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_DEFAULT_ACL         = None

AWS_LOCATION            = 'static'
MEDIA_URL               = 'media/'
AWS_QUERYSTRING_AUTH    = False


STATICFILES_DIRS        = [
    os.path.join(BASE_DIR, 'static'),
        ]
STATIC_URL              = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'


DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

Notice all the AWS Configuration we have done above, these will help us "talk" to the s3 bucket we just created.

Install s3transfer(a python library for managing AWS s3 transfers) and django-storages(provides a variety of storage backends in a single library) packages as shown below:



pip install s3transfer==0.3.7
pip install django-storages==1.11.1
pip freeze > requirements.txt

head back to the folder that held the "settings.py" file and create a new file "storage_backends.py"

Add the following contents into the file



from storages.backends.s3boto3 import S3Boto3Storage
from django.conf import settings

class MediaStorage(S3Boto3Storage):
    location = settings.AWS_STATIC_LOCATION
    default_acl = 'public-read'
    file_overwrite = False

The code above in its totality allow us to use AWS s3 object store to store and serve media files for our application.So we want to store and serve static and media files from AWS s3 not locally to do this we are sub-classing S3BotoStorage class that we have imported from storages.backends.s3boto3, we then define our custom class MediaStorage and in this we specify settings for location of storage, default access control list, and file overwrite settings.

Now open the "settings.py" and reference the local Media storage we just created : the file should now look as below with the new additions;

"settings.py"



from pathlib import Path
import os
from decouple import config

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-n18use^q@nwz&)s#os&zxlhy=3d57ys1wubfwi036e4h)7rol8'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ["*"]



# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'user_controller',
    'Transanctions',
    'Wallet'
]


MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'E_WalletZ.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'E_WalletZ.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases




DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'db.sqlite3',
    }
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = 'static/'

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field



#AWS CONFIG
# AWS CONFIG
AWS_STATIC_LOCATION = 'static'
S3_BUCKET_URL       = config('S3_BUCKET_URL')
STATIC_ROOT         = 'staticfiles'

AWS_ACCESS_KEY_ID     =  config('AWS_S3_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY =  config('AWS_S3_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')
AWS_HOST_REGION         = config('AWS_HOST_REGION')
AWS_S3_CUSTOM_DOMAIN    = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_DEFAULT_ACL         = None

AWS_LOCATION            = 'static'
MEDIA_URL               = 'media/'
AWS_QUERYSTRING_AUTH    = False


STATICFILES_DIRS        = [
    os.path.join(BASE_DIR, 'static'),
        ]
STATIC_URL              = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
DEFAULT_FILE_STORAGE    = 'E_WalletZ.storage_backends.MediaStorage'
STATICFILES_STORAGE     = 'E_WalletZ.storage_backends.MediaStorage'
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}




DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

Setting Up Our Docker Containers.

We want our service the E_WalletZ API and the Postgres Database that we will be using to persist our data to be run inside docker containers;

In our project root directory ; create a file "Dockerfile":

The contents of that file should look like:

"Dockerfile"



FROM python:3.7

ENV PYTHONDONTWRITEBYTECODE  1
ENV PYTHONUNBUFFERED 1

RUN mkdir /E_WalletZ

WORKDIR /E_WalletZ

COPY .  /E_WalletZ/

RUN pip install -r requirements.txt

This file is used to build the image from which we will be launching our docker container,as evident above we are using python 3.7 as the base layer, and in the next step we will be using docker compose, first before that lets set up our postgres database configuration



pip install psycopg2==2.9.5
pip install psycopg2-binary==2.9.5
pip freeze > requirements.txt

Go to the project "settings.py" file and add the following database configurations:

"settings.py"



from pathlib import Path
import os
from decouple import config

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-n18use^q@nwz&)s#os&zxlhy=3d57ys1wubfwi036e4h)7rol8'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ["*"]



# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'user_controller',
    'Transanctions',
    'Wallet'
]


MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'E_WalletZ.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'E_WalletZ.wsgi.application'


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases

DB_NAME = config("DB_NAME")
DB_USER = config("DB_USER")
DB_PASSWORD = config("DB_PASSWORD")
DB_HOST = config("DB_HOST")
DB_PORT = config("DB_PORT")


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': DB_NAME,
        'USER': DB_USER,
        'PASSWORD': DB_PASSWORD,
        'HOST' : DB_HOST,
        'PORT':  DB_PORT,
    }
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = 'static/'

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field



#AWS CONFIG
# AWS CONFIG
AWS_STATIC_LOCATION = 'static'
S3_BUCKET_URL       = config('S3_BUCKET_URL')
STATIC_ROOT         = 'staticfiles'

AWS_ACCESS_KEY_ID     =  config('AWS_S3_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY =  config('AWS_S3_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET_NAME')
AWS_HOST_REGION         = config('AWS_HOST_REGION')
AWS_S3_CUSTOM_DOMAIN    = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_DEFAULT_ACL         = None

AWS_LOCATION            = 'static'
MEDIA_URL               = 'media/'
AWS_QUERYSTRING_AUTH    = False


STATICFILES_DIRS        = [
    os.path.join(BASE_DIR, 'static'),
        ]
STATIC_URL              = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
DEFAULT_FILE_STORAGE    = 'E_WalletZ.storage_backends.MediaStorage'
STATICFILES_STORAGE     = 'E_WalletZ.storage_backends.MediaStorage'
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}




DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

Now lets head back to our project root folder and add our docker-compose file "docker-compose.yml", create a file with the same name i.e

Inside that file add the following: (This file helps us define and manage multi-container Docker applications, in it as you will see we define the services, networks, volumes to be used and other configurations for our containers in a declarative format).

"docker-compose.yml"



version: "3"
services: 
  web:
    build: .
    command: bash -c "python manage.py runserver 0.0.0.0:8080"
    container_name: E_WalletZ_API
    restart: always
    volumes:
      - .:/E_WalletZ
    ports:
      - "8080:8080"
    depends_on:
      postgres:
        condition: service_healthy

    networks:
      - E_WalletZ_net

  postgres:
    container_name: E_WalletZ_DB
    image: postgres
    restart: always
    environment:
      POSTGRES_USER: 'E_WalletZ_user'
      POSTGRES_PASSWORD: 'E_WalletZ_password'
      PGDATE: /data/E_WalletZ_postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U E_WalletZ_user"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - postgres:/data/E_WalletZ_postgres
    ports:
      - '5432:5432'
    networks:
      - E_WalletZ_net


networks:
  E_WalletZ_net:
    driver: bridge

volumes:
  postgres:

Wallet Users

Now before we spin up the containers for tests, lets code the user model, open "user_controller" folder

inside it lets open the "models.py" file and add the following code :

In the code we are using UserManager to control user creation in the project through the function create_user(), i.e in the case below we make sure user supplies email address, we also define how a superuser should be created. We then define our User class and its attributes, ImageUpload, UserProfile and UserAddress. (PS: please make use of chatGPT if you dont understand any of the code, the reason for this is if i go into deep explanation the article will grow longer than it already is)

"models.py"



from django.db import models
from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin, BaseUserManager



class UserManager(BaseUserManager):
    def create_user(self, email, password, **extra_fields):
        if not email:
            raise ValueError("Email is required")

        email = self.normalize_email(email)
        user  = self.model(email = email, **extra_fields)
        user.set_password(password)

        user.save()
        return user


    def create_superuser(self, email, password, **extra_fields):
        extra_fields.setdefault('is_staff', True)
        extra_fields.setdefault('is_superuser', True)
        extra_fields.setdefault('is_active',True)

        extra_fields.setdefault('first_name', 'admin')
        extra_fields.setdefault('last_name', 'admin')


        if not extra_fields.get('is_staff', False):
            raise ValueError('SuperUser must have is_staff=True')

        if not extra_fiels.get('is_superuser', False):
            raise ValueError('Superuser must have is_superuser=True')
        return self.create_user(email, password, **extra_fields)


class User(AbstractBaseUser, PermissionsMixin):
    email = models.EmailField(unique=True)
    first_name = models.CharField(max_length=100)
    last_name  = models.CharField(max_length=100)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    is_staff  = models.BooleanField(default=False)
    is_active = models.BooleanField(default=True)

    USERNAME_FIELD = "email"
    objects = UserManager()

    def __str__(self) -> str:
        return self.email

    def save(self, *args, **kwargs):
        super().full_clean()
        super().save(*args, **kwargs)


class ImageUpload(models.Model):
    image = models.ImageField(upload_to="images")

    def __str__(self):
        return str(self.image)


class UserProfile(models.Model):
    user = models.OneToOneField(User, related_name="user_profile", on_delete=models.CASCADE)
    profile_picture = models.ForeignKey(ImageUpload, related_name="user_images", on_delete=models.SET_NULL, null=True)
    dob   = models.DateField()
    phone = models.PositiveIntegerField()
    country_code = models.CharField(default="+254", max_length=5)
    created_at   = models.DateTimeField(auto_now_add=True)
    updated_at   = models.DateTimeField(auto_now=True)
    def __str__(self):
        return self.user.email


class UserAddress(models.Model):
    user_profile = models.ForeignKey(UserProfile, related_name="user_addresses", on_delete=models.SET_NULL, null=True)
    street = models.TextField()
    city   = models.CharField(max_length=100)
    state  = models.CharField(max_length=100)
    country = models.CharField(max_length=100, default="Kenya")
    is_default = models.BooleanField(default=False)

    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    def __str__(self):
        return self.user_profile.user.email

Now back to the project root folder(the one with Dockerfile and docker-compose.yml files). let run the following command :



docker-compose up

The output of this command will look something like:

It will build the image as defined in the Dockerfile and it will then create and launch the containers for the API and the Postgres Database using the compose file.

now lets open a new terminal and execute the following command to enter into the shell(bash) of the containers running our services.

first



docker ps

The command above will list the containers on your system, the result will be something like:

Now take the container ID to our API(E_WalletZ_API), from the command above and plug it into the command below:



docker exec -it <container_ID> /bin/bash

The result will look something like below, also notice on the last line i am executing the "collectstatic", the subsequent result of such command is shown below:

The result of collectstatic:

Now, after you have answered yes to the prompt, if you head to AWS Console and look into the bucket we just created, you will see the following:

Now, while still on this shell(bin/bash) for the API container, lets execute the following commands :



python manage.py makemigrations
python manage.py migrate

This is to test out our connection to the Postgres database, you should see something like the one below:

To confirm these migrations lets check into our database and check if the table has been created:

Exit out of your current E_WalletZ_API shell by executing the following command repeatedly till you end back to the terminal:



exit

from our terminal, we now want to enter into the shell for the postgres container, remember to do this we need the container ID for postgres container and we then plug it into the command below:



docker exec -it  <container_id> /bin/bash

We will enter into the shell, while in the same shell execute the following command to link to the postgres database instance



psql -U E_WalletZ_user

Now if you are keen, u will remember in our .env file we had two variables with the value "E_WalletZ_user"

to list databases, the command is

\l

to connect to our database , the command is:



\c E_WalletZ_user

After, we connect to our, E_WalletZ_user database, we can list the tables as shown below:

\dt

JWT AUTHENTICATION

Lets work on an Authentication module for requests to resources

lets first install pyJWT as below :



PyJWT==2.0.0
pip freeze > requirements.txt

lets head into the folder that houses our "settings.py" and "urls.py" and here we will create a new file called "authentication.py"

"authentication.py"



from datetime import datetime
import jwt
from django.conf import settings

class TokenManager:
    @staticmethod
    def get_token(exp: int, payload, token_type="access"):
        exp = datetime.now().timestamp() + (exp * 60)

        return jwt.encode(
            {"exp": exp, "type": token_type, **payload},
            settings.SECRET_KEY,
            algorithm = "HS256"
        )


    @staticmethod
    def decode_token(token):
        try:
            decoded = jwt.decode(token, key=settings.SECRET_KEY, algorithms="HS256")
        except jwt.DecodeError as e:
            print("Cannot decode token because : ", e)
            return None
        if datetime.now().timestamp() > decoded["exp"]:
            return None

        return decoded

    @staticmethod
    def get_access(payload):
        return TokenManager.get_token(100, payload)

    @staticmethod
    def get_refresh(payload):
        return TokenManager.get_token(14*24*60, payload, "refresh")


class Authentication:
    def __init__ (self, request):
        self.request = request

    def authenticate(self):
        data = self.validate_request()

        if not data:
            return None

        return self.get_user(data["user_id"])

    def validate_request(self):
        authorization = self.request.headers.get("Authorization", None)

        if not authorization:
            return None

        token = authorization[4:]
        decoded_data = TokenManager.decode_token(token)

        if not decoded_data:
            return None
        return decoded_data


    @staticmethod
    def get_user(user_id):
        from user_controller.models import User

        try:
            user = User.objects.get(id = user_id)
            return user
        except User.DoesNotExist:
            return None

After the user authentication implemented above using JWT,below we implement pagination and querying formating below in a file called "permissions.py"

Create a permissions.py file in the same folder that holds our "urls.py", "wsgi.py" , "settings.py" i.e

Open the file and add the following code :

"permissions.py"



import graphene
from django.conf import settings
from django.core.paginator import EmptyPage, PageNotAnInteger, Paginator
import re
from django.db.models import Q


def is_authenticated(func):
    def wrapper(cls, info, **kwargs):
        if not info.context.user:
            raise Exception("You are not authorized to perform this operation")
        return func(cls, info, **kwargs)

    return wrapper


def paginate(model_type):
    structure = {
        "total"       :   graphene.Int(),
        "size"        :   graphene.Int(),
        "current_page":   graphene.Int(),
        "has_next"    :   graphene.Boolean(),
        "has_prev"    :   graphene.Boolean(),
        "results"     :   graphene.List(model_type)
    }

    return type(f"{model_type}Paginated", (graphene.ObjectType,), structure)


def resolve_paginated(query_data, info, page_info):
    def get_paginated_data(qs, paginated_type, page):
        page_size = settings.GRAPHENE.get("PAGE_SIZE", 10)


        try:
            qs.count()
        except:
            raise Exception(qs)

        p = Paginator(qs, page_size)


        try:
            page_obj = p.page(page)
        except PageNotAnInteger:
            page_obj = p.page(1)
        except EmptyPage:
            page_obj = p.page(p.num_pages)


        result = paginated_type.graphene_type(
            total = p.num_pages,
            size  = qs.count(),
            current_page = page_obj.number,
            has_next = page_obj.has_next(),
            has_prev = page_obj.has_previous(),
            results  = page_obj.object_list
        )
        return result

    return get_paginated_data(query_data, info.return_type, page_info)

def normalize_query(query_string, findterms=re.compile(r'"([^"]+)"|(\S+)').findall, normspace=re.compile(r'\s{2,}').sub):
    return [normspace(' ', (t[0] or t[1]).strip()) for t in findterms(query_string)]


def get_query(query_string, search_fields):
    query = None  #Query to search for every search term

    terms = normalize_query(query_string)
    for term in terms:
        or_query = None #Query to search for a given term in each field
        for field_name in search_fields:
            q = Q(**{"%s__icontains" % field_name: term})
            if or_query is None:
                or_query = q
            else:
                or_query = or_query | q
        if query is None:
            query = or_query
        else:
            query = query & or_query
    return query

GRAPHQL

Now lets plug in our graphql, schema..............

First things first, let install graphql packages to be used:



pip install graphene==2.1.8
pip install graphene-django==2.15.0
pip install graphene-file-upload==1.2.2
pip install graphql-core==2.3.2
pip install graphql-relay==2.0.1
pip freeze > requirements.txt

Head to the settings.py file, and add graphene_django as one of the installed apps, i.e :



INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'graphene_django',
    'user_controller',
    'Transanctions',
    'Wallet'
]

Let head into our user_controller app , in this folder we will create a new file "schema.py" to hold our graphql code as shown below:

Open the schema.py file and add the following code :

"schema.py"



import graphene
from .models import User, ImageUpload, UserProfile, UserAddress
from graphene_django import DjangoObjectType
from django.contrib.auth import authenticate
from datetime import datetime
from E_WalletZ.authentication import TokenManager
from E_WalletZ.permissions  import paginate, is_authenticated
from django.utils import timezone
from graphene_file_upload.scalars import Upload
from django.conf import settings

class UserType(DjangoObjectType):
    class Meta:
        model = User


class ImageUploadType(DjangoObjectType):
    image = graphene.String()
    class Meta:
        model = ImageUpload


    def resolve_image(self, info):
        if (self.image):
            pass


class UserProfileType(DjangoObjectType):
    class Meta:
        model = UserProfile

class UserAddressType(DjangoObjectType):
    class Meta:
        model = UserAddress


class RegisterUser(graphene.Mutation):
    status = graphene.Boolean()
    message = graphene.String()


    class Arguments:
        email = graphene.String(required=True)
        password = graphene.String(required=True)
        first_name = graphene.String(required=True)
        last_name  = graphene.String(required=True)

    def mutate(self, info, email, password, **kwargs):
        User.objects.create_user(email, password, **kwargs)

        return RegisterUser(
            status = True,
            message = "User has been Registered Succesfully"
        )

class LoginUser(graphene.Mutation):
    access  = graphene.String()
    refresh = graphene.String()
    user    = graphene.Field(UserType)

    class Arguments:
        email = graphene.String(required=True)
        password = graphene.String(required=True)

    def mutate(self, info, email, password):
        user = authenticate(username=email, password=password)
        if not user:
            raise Exception("Invalid User credentials entered")

        user.last_login = datetime.now(tz=timezone.utc)
        user.save()

        access  = TokenManager.get_access({"user_id" : user.id })
        refresh = TokenManager.get_refresh({"user_id" : user.id })

        return LoginUser(
            access = access,
            refresh = refresh,
            user = user
        )

class GetAccess(graphene.Mutation):
    access = graphene.String()

    class Arguments:
        refresh = graphene.String(required = True)

    def mutate(self, info, refresh):
        token = TokenManager.decode_token(refresh)

        if not token or token["type"] != "refresh":
            raise Exception("Invalid token or has expired, re-apply for fresh token")

        access = TokenManager.get_access({ "user_id" : token["user_id"]})

        return GetAccess(
            access = access
        )


class ImageUploadMain(graphene.Mutation):
    image = graphene.Field(ImageUploadType)

    class Arguments:
        image = Upload(required=True)

    def mutate(self, info, image):
        image = ImageUpload.objects.create(image=image)

        return ImageUploadMain(
            image = image
        )


class UserProfileInput(graphene.InputObjectType):
    profile_picture = graphene.String()
    country_code    = graphene.String()



class CreateUserProfile(graphene.Mutation):
    user_profile   = graphene.Field(UserProfileType)

    class Arguments:
        profile_data = UserProfileInput()
        dob          = graphene.Date(required=True)
        phone        = graphene.Int(required=True)

    @is_authenticated
    def mutate(self, info, profile_data, **kwargs):
        try:
            info.context.user.user_profile
        except Exception:
            raise Exception("You don't hava a profile to update")

        UserProfile.objects.filter(user_id = info.context.user.id).update(**profile_data, **kwargs)

        return CreateUserProfile (
            user_profile = info.context.user.user_profile
        )



class UpdateUserProfile(graphene.Mutation):
    user_profile = graphene.Field(UserProfileType)

    class Arguments:
        profile_data = UserProfileInput()
        dob          = graphene.Date()
        phone        = graphene.Int()

    @is_authenticated
    def mutate(self, info, profile_data, **kwargs):
        try:
            info.context.user.user_profile
        except Exception:
            raise Exception("You do not have a profile to update")

        UserProfile.objects.filter(user_id = info.context.user.id).update(**profile_data, **kwargs)

        return UpdateUserProfile(
            user_profile = info.context.user.user_profile
        )



class AddressInput(graphene.InputObjectType):
    street  = graphene.String()
    city    = graphene.String()
    state   = graphene.String()
    country = graphene.String()


class CreateUserAddress(graphene.Mutation):
    address = graphene.Field(UserAddressType)

    class Arguments:
        address_data = AddressInput(required=True)
        is_default   = graphene.Boolean()


    @is_authenticated
    def mutate(self, info, address_data, is_default=False):
        try:
            user_profile_id = info.context.user.user_profile.id
        except Exception:
            raise Exception("You need a profile to create an address")
        existing_addresses = UserAddress.objects.filter(user_profile_id=user_profile_id)
        if is_default:
            existing_addresses.update(is_default=False)

        address = UserAddress.objects.create(
            user_profile_id = user_profile_id,
            is_default = is_default,
            **address_data
        )
        return CreateUserAddress(
            address = address
        )

class UpdateUserAddress(graphene.Mutation):
    address = graphene.Field(UserAddressType)

    class Arguments:
        address_data = AddressInput()
        is_default   = graphene.Boolean()
        address_id   = graphene.ID(required=True)


    @is_authenticated
    def mutate(self, info, address_data, address_id, **kwargs):
        profile_id = info.context.user.user_profile.id
        address    = UserAddress.objects.filter(
            user_profile = profile_id,
            id = address_id,
        ).update(is_default = is_default, **address_data)

        if is_default:
            UserAddress.objects.filter(user_profile_id=profile_id).exclude(id = address_id).update(is_default = False)

        return UpdateUserAddress(
                address = UserAddress.objects.get(id = address_id)
        )

class DeleteUserAddress(graphene.Mutation):
    status = graphene.Boolean()

    class Arguments:
        address_id = graphene.ID(required=True)

    @is_authenticated
    def mutate(self, info, address_id):
        UserAddress.object.filter(
            user_profile_id = profile_id,
            id = address_id
        ).delete()

        return DeleteUserAddress(
            status = True
        )


class Query(graphene.ObjectType):
    users  = graphene.Field(paginate(UserType), page=graphene.Int())
    images = graphene.Field(paginate(ImageUploadType), page=graphene.Int())
    me     = graphene.Field(UserType)


    def resolve_users(self, info, **kwargs):
        print(User.objects.filter(**kwargs))
        return User.objects.filter(**kwargs)

    def resolve_images(self, info, **kwargs):
        return ImageUpload.objects.filter(**kwargs)



    def resolve_me(self, info):
        return info.context.user


class Mutation(graphene.ObjectType):
    register_user = RegisterUser.Field()
    login_user    = LoginUser.Field()
    get_access    = GetAccess.Field()
    image_upload  = ImageUploadMain.Field()

    create_user_profile = CreateUserProfile.Field()
    update_user_profile = UpdateUserProfile.Field()
    create_user_address = CreateUserAddress.Field()
    update_user_address = UpdateUserAddress.Field()
    delete_user_address = DeleteUserAddress.Field()


schema = graphene.Schema(query=Query, mutation=Mutation)

Now, we need to unify the schemas from different apps into a project level schemas file :

In the folder that contains the "setting.py", "urls.py", "asgi.py" , add a file "schema.py" : i.e

Open the file "schema.py" and add the following code :

"schema.py"




import graphene

from user_controller.schema import schema as user_schema


class Query(user_schema.Query, graphene.ObjectType):
    pass


class Mutation(user_schema.Mutation, graphene.ObjectType):
    pass


schema = graphene.Schema(query=Query, mutation=Mutation)

We need to include setting for our graphene integration, head to the settings.py and add the GRAPHENE setting below:

"settings.py"



GRAPHENE = {
    'SCHEMA': 'E_WalletZ.schema.schema',
    'MIDDLEWARE': [
        ],
        'PAGE_SIZE': 5
}

CORS_ALLOW_ALL_ORIGINS = True

still on this you will notice that our 'MIDDLEWARE' attributes is empty, this are middle-wares specific to graphql:

just next to the settings.py file, add a file named "middlewares.py"

open that file and add the following code:



from .permissions import resolve_paginated

class CustomAuthMiddleware(object):
    #the resolve function can be used for resolving URL paths to the corresponding view functions, the signature is resolve(path, urlconf=None    ), path is the URL path you want to resolve. The return type is a ResolverMatch object that allows one to access various metadata about th    e ressolved URl.

    def resolve(self, next, root, info, **kwargs):
        info.context.user = self.authorize_user(info)
        return next(root, info, **kwargs)

    @staticmethod
    def authorize_user(info):
        from .authentication import Authentication

        auth = Authentication(info.context)
        return auth.authenticate()


class CustomPaginationMiddleware(object):
    def resolve(self, next, root, info, **kwargs):
        try:
            is_paginated = info.return_type.name[-9:]
            is_paginated = is_paginated == "Paginated"
        except Exception:
            is_paginated = False

        if is_paginated:
            page = kwargs.pop("page", 1)
            return resolve_paginated(next(root, info, **kwargs).value, info, page)
        return next(root, info, **kwargs)

Now back to the "settings.py" open it up and add the two custom Middlewares we just created:



GRAPHENE = {
    'SCHEMA': 'E_WalletZ.schema.schema',
    'MIDDLEWARE': [
        'E_WalletZ.middlewares.CustomAuthMiddleware',       
        'E_WalletZ.middlewares.CustomPaginationMiddleware'       
        ],
        'PAGE_SIZE': 5
}

CORS_ALLOW_ALL_ORIGINS = True

With the graphene integration completed, we now need to push our graphene_django static files to AWS S3, remember what we did in the initial step

List the containers using the command below :



docker ps

Pick the container id of the container that is running the API

Use that "container id" in the command below to enter into the shell



docker exec -it <container_id> /bin/bash

Now after we have succesfully enter into the bash of the container, we execute the commands below to push the files to AWS



 python manage.py collectstatic

After this, if you check your AWS bucket it now has another new file called "graphene_django" :

Now lets head to our "urls.py" file and add the following code :

"urls.py"



from django.contrib import admin
from django.urls import path
from django.conf import settings
from django.conf.urls.static import static
from graphene_django.views import GraphQLView
from django.views.decorators.csrf import csrf_exempt
from graphene_file_upload.django import FileUploadGraphQLView


urlpatterns = [
    path('admin/', admin.site.urls),
    path('graphview/', csrf_exempt(FileUploadGraphQLView.as_view(graphiql=True)))
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

On your container logs you should now see the following:

if u copy and paste the url on your browser(in my case it is http://0.0.0.0:8000) .you should now see the graphql UI as shown below:

WALLETS MODEL AND SCHEMA

The Wallets app will be the object that holds the token or cash or representation of such that will be attached to a user object that we have already created above.

Open up the "models.py" file, here we want to create a model object to represent a wallet:

"models.py"



from django.db import models
from django.contrib.auth import get_user_model
from djmoney.models.fields import MoneyField
from decimal import Decimal
from djmoney.models.validators import MaxMoneyValidator, MinMoneyValidator
#from Transanctions.models import  Transanction


User = get_user_model()

# Create your models here.

class Wallet(models.Model):
    user = models.OneToOneField(User, related_name="user_wallet", on_delete=models.PROTECT)
    """ amount_available = MoneyField(
            max_digits=11, decimal_places=2, default=0, default_currency='KES',
            validators=[
                MinMoneyValidator(Decimal(0.00)), MaxMoneyValidator(Decimal(999999999.99)),
                ]
            )
    """
    amount_available =  models.DecimalField(max_digits=11, decimal_places=2, default=0.00)
    created_on = models.DateTimeField(auto_now_add=True)
    updated_on = models.DateTimeField(auto_now=True)


    def __str__(self):
        return f"{self.user.firstName}-{self.user.lastName}-wallet"




class WalletProfile(models.Model):

    TYPES = {
            (0, "Mini"),
            (1, "Regular"),
            (2, "Super"),
            }

    LIMITS = {
            (0, 100000),
            (1, 1000000),
            (2, 100000000),
            (3, 9999999999.99)
            }
    name  = models.PositiveSmallIntegerField(default=1, choices=TYPES)
    limit = models.PositiveSmallIntegerField(default=1, choices=LIMITS)
    wallet = models.OneToOneField(Wallet, related_name="wallet_profile", on_delete=models.CASCADE)
    created_at = models.DateTimeField(auto_now_add=True)


    def __str__(self):
        return f"{self.name}-WithLimit-{self.limit}"

Next we want to pipe the model and its attributes through grapqhl, sort of build a wrapper around it.

create a file called schema.py and add the following code:

"schema.py"



import graphene
from .models import Wallet, WalletProfile
from graphene_django import DjangoObjectType
from django.contrib.auth import authenticate
from datetime import datetime
from E_WalletZ.authentication import TokenManager
from E_WalletZ.permissions import paginate, is_authenticated
#from django.utils import timezone
#from django.conf import settings
from decimal import Decimal
from graphql.language import ast
from django.contrib.auth import get_user_model


User = get_user_model()



class WalletType(DjangoObjectType):
    class Meta:
        model = Wallet


class WalletProfileType(DjangoObjectType):
    class Meta:
        model = WalletProfile



class RegisterWallet(graphene.Mutation):
    status  = graphene.Boolean()
    message = graphene.String()

    @is_authenticated
    def mutate(self, info, **kwargs):

        user = User.objects.get(id=info.context.user.id)
        Wallet.objects.create(user=user)

        return RegisterWallet(
            status= True,
            message = "Wallet Created Succesfully"
        )


class CreditWallet(graphene.Mutation):
    status = graphene.Boolean()
    message = graphene.String()

    class Arguments:
        credit_amount = graphene.Float()

    @is_authenticated
    def mutate(self, info, credit_amount, **kwargs):
        #credit_amount = graphene.Decimal(credit_amount)
        wallet = Wallet.objects.get(user_id=info.context.user.id)
        wallet.amount_available +=  Decimal(str(credit_amount))
        wallet.save()

        return CreditWallet(
            status  = True,
            message = f"Your Wallet has been successfully credit with the amount {credit_amount}"
        )



class DeleteUserWallet(graphene.Mutation):
    status = graphene.Boolean()

    @is_authenticated
    def mutate(self, info, **kwargs):
        Wallet.objects.filter(user_id=info.context.user.id).delete()
        return DeleteUserWallet(
            status = True
        )


class Query(graphene.ObjectType):
    wallet = graphene.Field(paginate(WalletType), page = graphene.Int())

    @is_authenticated
    def resolve_wallet(self, info):
        return Wallet.objects.filter(user_id=info.context.user.id)



class Mutation(graphene.ObjectType):
    register_wallet   = RegisterWallet.Field()
    credit_wallet     = CreditWallet.Field()
    delete_user_wallet = DeleteUserWallet.Field()



schema = graphene.Schema(query=Query, mutation=Mutation)

Now that we have the wallet schema built, we have to head to the project level schema file and plug it there:

So head to the folder that houses "urls.py", "settings.py" i.e :

open the schema.py file and add the following code:

"schema.py"



import graphene

from user_controller.schema import schema as user_schema
from Wallet.schema import schema as wallet_schema

class Query(user_schema.Query, wallet_schema.Query, graphene.ObjectType):
    pass


class Mutation(user_schema.Mutation, wallet_schema.Mutation, graphene.ObjectType):
    pass


schema = graphene.Schema(query=Query, mutation=Mutation)

Now with all changes let migrate the model wallets to the postgres database:

remember what we did before:
list containers using the command :



docker ps

Note the container id for the container that is running the API, i.e E_WalletZ_API and plug it into the following command:



docker exec -it <container_id> /bin/bash

Now in the shell lets make the migrations; to do this we use the commands below:



python manage.py makemigrations
python manage.py migrate

The result should look something like:

Now lets head to the postgres database shell to confirm the wallets table has been created:

do the same thing we did above, now in this case enter the postgres' container id in the command:



docker exec -it <container_id> /bin/bash

Once in the container shell, lets use the command below to log into our database:



psql -U E_WalletZ_user

Then execute the commands shown below to see if wallets table has been created:

Now to our graphiql API UI, let see if the changes have reflected; you should ideally see the following:

wallah ! , we are done with Wallets app ,now lets head to Transanctions app

Transanctions Model and Schema

Open the folder to the Transanctions app, and add the following code to the models.py file

"models.py"



from django.db import models
from django.contrib.auth import get_user_model
from user_controller.models import User
from djmoney.models.validators import MaxMoneyValidator, MinMoneyValidator
from decimal import Decimal
from djmoney.models.fields import MoneyField
from Wallet.models import Wallet


transactee = get_user_model()

class Transanction(models.Model):
    wallet = models.ForeignKey(Wallet, related_name="transanction_wallet", on_delete=models.PROTECT)
    sender   = models.OneToOneField(transactee, related_name="sender_transanction", on_delete=models.PROTECT)
    receiver = models.OneToOneField(transactee, related_name="receiver_transanction", on_delete=models.PROTECT)
    """amount  =  MoneyField(
                max_digits=7, decimal_places=2, default=0, default_currency='KES',
                validators=[
                    MinMoneyValidator(Decimal(0.00)), MaxMoneyValidator(Decimal(99999.99))
                ]
            )
    """
    amount = models.DecimalField(max_digits=7, decimal_places=2, default=0.00)
    created_on = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f"{self.sender.firstName}_{self.sender.lastName}-to-{self.receiver.firstName}_{self.receiver.lastName}={self.amount}-{self.created_on}"



class TransanctionProfile(models.Model):
    TYPE = {
    (0, "AFRICA"),
    (1, "KENYA"),
    (2, "WORLD")
    }

   CATEGORY = {
    (0, "0 - 100,000"),
    (1, "100,000 - 1,000,000"),
    (2, "1,000,000 - 100,000,000"),
    (3, "100,000,000 - 500,000,000"),
    (4, "> 500,000,000")
    }


    transanction = models.OneToOneField(Transanction, related_name="transanction_profile", on_delete=models.CASCADE)
    transanction_category = models.PositiveSmallIntegerField(default=0, choices=CATEGORY)
    transanction_type     = models.PositiveSmallIntegerField(default=1, choices=TYPE)


    def __str__(self):
        return f"{self.transanction_category}-{self.transanction_type}"

Now lets wrap around this model using , a graphql schema; create a file "schema.py" in it add the following code:

before that lets create a movemoney.py module to execute the logic of moving money from one wallet to another. You will notice that the movemoney module is imported by the "schema.py"

"movemoney.py"



class MovingMoney:

    @staticmethod
    def movemoney(info,sender,receiver,wallet,amount):
        from Wallet.models import Wallet
        import decimal

        print(f"Cash transfer of {amount} from you to {receiver.first_name}-{receiver.last_name} starting")

        try:
            sender_wallet = Wallet.objects.get(user_id=sender.id)
            sender_wallet.amount_available   = (sender_wallet.amount_available - decimal.Decimal(str(amount)).quantize(decimal.Decimal('.01')))
            sender_wallet.save()

        except Exception:
            return Exception("Updating of the senders amount_available failed")


        try:
            receiver_wallet = Wallet.objects.get(user_id = receiver.id)
            receiver_wallet.amount_available = (receiver_wallet.amount_available + decimal.Decimal(str(amount)).quantize(decimal.Decimal('.01')))
            receiver_wallet.save()

        except Exception:
            sender_wallet.amount_available = (sender_wallet.amount_available + decimal.Decimal(str(amount)).quantize(decimal.Decimal('.01')))
            return Exception("Money transfer to receiver has failed")


        print ("Money Transfer is now complete!")

"schema.py"



import graphene
from .models import Transanction, TransanctionProfile
from graphene_django import DjangoObjectType
from django.contrib.auth import authenticate
from datetime import datetime
from E_WalletZ.authentication import TokenManager
from E_WalletZ.permissions import paginate, is_authenticated
from user_controller.models import User
from Wallet.models import Wallet
from .movemoney import MovingMoney


CONST_TRANSANCTION_FEE_PERCENTAGE = 0.25

class TransanctionType(DjangoObjectType):
    class Meta:
        model = Transanction


class TransanctionProfile(DjangoObjectType):
    class Meta:
        model = TransanctionProfile


class MakeTransanction(graphene.Mutation):
    status  = graphene.Boolean()
    message = graphene.String()

    class Arguments:
        amount_to_send = graphene.Float()
        email = graphene.String()

    @is_authenticated
    def mutate(self, info, email, amount_to_send):
        receiver = User.objects.get(email=email)
        if (receiver.id == info.context.user.id):
            return Exception("Cant send Money to yourself")
        user = User.objects.get(id = info.context.user.id)
        receiver_wallet = Wallet.objects.get(user_id=receiver.id)
        sender_wallet   = Wallet.objects.get(user_id=info.context.user.id)
        if sender_wallet.amount_available <= ((amount_to_send*CONST_TRANSANCTION_FEE_PERCENTAGE)+amount_to_send):
            return Exception(f"You have insufficient funds '{Wallet_user.amount_available}' to complete the cash transfer and pay charges, top up to continue")
        #amount_to_send = graphene.Decimal(amount_to_send)
        try:
            Transanction.objects.create(wallet=sender_wallet,   sender=user, receiver=receiver, amount=amount_to_send)
            Transanction.objects.create(wallet=receiver_wallet, sender=user, receiver=receiver, amount=amount_to_send)
            #MovingMoney.movemoney(info=info, sender=user, receiver=receiver,wallet=Wallet ,amount=amount_to_send)
        except Exception:
            return Exception("Transanction failed please try again")
        MovingMoney.movemoney(info=info, sender=user, receiver=receiver,wallet=Wallet,amount=amount_to_send)
        return MakeTransanction(
            status= True,
            message = f"You have succesfully transfered {amount_to_send} from your wallet to {receiver.email}"
        )



class DeleteTransanction(graphene.Mutation):
    status = graphene.Boolean()
    message = graphene.String()

    class Arguments:
        transanction_id = graphene.ID(required=True)


    @is_authenticated
    def mutate(self, info, transanction_id):
        Wallet = Wallet.objects.get(user_id=info.context.user.id)
        Transanction.object.filter(
            wallet=Wallet,
            id = transanction_id
        ).delete()
        return DeleteTransanction(
            status = True,
            message = f"you have succesfully deleted the transanction {transanction_id}"
        )



class Query(graphene.ObjectType):
    transanctions = graphene.Field(paginate(TransanctionType), page=graphene.Int())

    @is_authenticated
    def resolve_transanctions(self,info):
        wallet = Wallet.objects.get(user_id=info.context.user.id)
        print("Wallet :", wallet)
        return Transanction.objects.filter(wallet_id=wallet.id)



class Mutation(graphene.ObjectType):
    make_transanction = MakeTransanction.Field()

    delete_transanction  = DeleteTransanction.Field()



schema = graphene.Schema(query=Query, mutation=Mutation)

Similar to how we did with the wallets app, lets do the same with the Transanctions app, we first head to the project level schema.py file to plug the transanctions model schema i.e

Go to the folder that holds the "settings.py" and "urls.py"

"schema.py"



import graphene

from user_controller.schema import schema as user_schema
from Transanctions.schema import schema as transanction_schema
from Wallet.schema  import schema as wallet_schema


class Query(user_schema.Query, transanction_schema.Query, wallet_schema.Query, graphene.ObjectType):
    pass


class Mutation(user_schema.Mutation, transanction_schema.Mutation, wallet_schema.Mutation, graphene.ObjectType):
    pass


schema = graphene.Schema(query=Query, mutation=Mutation)

Next, we make migrations to update our database tables, with the transanctions model:

Follow the process i mentioned above to enter into the container shell, your results should look like :

confirming in the postgres database :

When we check on graphiQL:

Time for a well earned break, on the next article we work on the testing the API by creating users and wallets, we then simulate the deposit of cash, the transfer of cash between wallet all from Graphiql and also build a jenkins CI/CD pipeline for automated deployment on AWS EC2 instances.Goodbye , see you then.

If you get the error: ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/init.py) : find the solution here

Go-lang Gorilla API + MongoDB in 30 minutes

myk_okoth_ogodo — Sun, 24 Jul 2022 23:32:22 +0000

Now hold your horses, before you start the thirty minute timer there is some house keeping we need to do first.One, we need to talk about No-SQL database family that MongoDB belongs to,two, we also need to touch on how to install MongoDB in our system (i am assuming you are using one of the Linux distros), then we will integrate this MongoDB database with a simple Gorilla-Mux API.

The code to this article can be found here.

No-SQL(Non-Relational Databases)

These are document-oriented databases that have a different data storing format to SQL database such as MySQL or PostgreSQL.There are various flavors of non-relational databases based on their data model. Main types are document databases like MongoDB, key-value databases like Redis, wide-column databases like Apache and Cassandra and finally graph databases like Orient-DB, Dgraph.

No-SQL databases give developers the freedom to store huge amounts of unstructured data. This is due to the fact that No-SQL databases are less structured or confined in format thus allowing for greater flexibility and adaptability in the type of data stored.

A popular example of an application that uses a No-SQL database would be Facebook's Messenger app that uses an Apache HBase to stream data to Hadoop clusters. Apache HBase is a wide-column database modeled on by Google's Big Table.

Characteristics of No-SQL(non-Relational) Databases:

Simple to integrate and use with apps for developers
Allows for Horizontal scaling or Scaling out.
Allows for Fast Querying
Have Flexible Schema.

Dive deeper into this topic at this article here.

Installing MongoDB

We first update our apt and install the MongoDB as shown below:


mykmyk@skynet:~$ sudo apt update

then;

mykmyk@skynet:~$ sudo apt-get install mongodb

Now, to confirm if your MongoBD database has installed
correctly, check that by doing the following:

mykmyk@skynet:~$ mongo
MongoDB shell version v3.6.3
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.3
Server has startup warnings: 
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] 
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2022-07-23T22:58:19.824+0300 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] 
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2022-07-23T22:58:30.835+0300 I CONTROL  [initandlisten] 
>

If you ran into errors while you were installing MongoDB, i refer you to this well written article, it should guide sufficiently enough to get the database safely installed on your system.

To check the databases that are avaible in our MongoDB DMS:

> show databases;
admin   0.000GB
appdb   0.000GB
config  0.000GB
local   0.000GB
test    0.000GB

The above command "show databases" lists all available databases , by default,admin,test, and local are the three that should be available before you create any new database;
We are going to create a database called, "booksdb", so while still in your MongoDB shell:

> use booksdb;
switched to db booksdb

The command "use booksdb" above switches from the current database to the booksdb database that we just created. Its interesting to note that the "booksdb" wont show up if we query all the databases available using the "show databases" command. Go ahead and try it.

> use booksdb;
switched to db booksdb
> show databases;
admin   0.000GB
appdb   0.000GB
config  0.000GB
local   0.000GB
test    0.000GB

The above phenomenon results from the internal implementation of MongoDB,a database is only created in MongoDB when actual initial data is inserted into it , so lets go ahead and do just that.

still on your MongoDB shell type in:

> db.books.insertOne({_id:5, title: 'A Song Of Ice and Fire',authors:['George R.R Martin','Phyllis Eisenstein'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{ "acknowledged" : true, "insertedId" : 5 }

The json we just inserted has an ID called _id, we could either provide that id while inserting our books object or we could leave it to MongoDB to insert it for us. Lets try and insert a book without specifying an id below and see what MongoDB will do:

> db.books.insertOne({title: 'A Feast Of Crows',authors:['George R.R Martin'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{
    "acknowledged" : true,
    "insertedId" : ObjectId("62dda3f1457193537fd4d245")
}
>

As you can see above in the insert aknowledgment JSON response, insertId value has now changed to a long hash value that is generated internally by MongoDB, in our case it is "62dda3f1457193537fd4d245"

To search for books we use the find() keyword as shown below:

> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
>

The find() without any supplied arguments returns all the
documents in our "booksdb" collection. To return a single document we use the findOne(), if empty it returns latest document i.e:

> db.books.findOne()
{
    "_id" : 5,
    "title" : "A Song Of Ice and Fire",
    "authors" : [
        "George R.R Martin",
        "Phyllis Eisenstein"
    ],
    "publishdate" : "August 1,1996",
    "characters" : [
        "Jon Snow",
        "Daenerys Targaryen",
        "Eddard Star",
        "Sansa Stark",
        "Jammie Lannister",
        "Rob Stark",
        "Cerser Lannister"
    ],
    "publisher" : {
        "name" : "Bantam Books",
        "country" : "United States",
        "website" : "www.randomhousebooks.com"
    }
}
>

We can supply filtering parameters to our find() collection queries as shown below:

> db.books.find({title:{$eq: "A Feast Of Crows"}})
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }

We can delete a document from a given collection using the deleteOne and deleteMany functions:

> db.books.deleteOne({"_id": ObjectId("62dda3f1457193537fd4d245")})
{ "acknowledged" : true, "deletedCount" : 1 }

Next stage we concentrate on building up our very simple GorillaMux API:

Gorilla-Mux API

First things first,to allow our API to talk to the MongoDB database we will be using a database driver called mgo.

mgo- A Rich MongoDB driver for Go

This is MongoDB driver for go-lang that enables developers to build applications like in our case and have that API interface directly with the MongoDB database without having to go through the Mongo interactive shell. This diver acts as a wrapper around the MongoDB API.

Check out the documentation on this driver here, for a deeper understanding on its implementation and application.

mgo installation:

First lets create a folder for our application, create a folder called MongoGorilla:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/GoMongo$ mkdir MongoGorilla

then run go mod init, that create go.mod file to track the dependencies that we will use to build our API application like mgo driver above:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go mod init
go: creating new go.mod: module github.com/myk4040okothogodo/MongoGorilla
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ ll
total 12
drwxrwxr-x  2 mykmyk mykmyk 4096 Jul 23 23:24 ./
drwxrwxr-x 20 mykmyk mykmyk 4096 Jul 23 23:09 ../
-rw-rw-r--  1 mykmyk mykmyk   58 Jul 23 23:24 go.mod

Now, let install our driver "mgo" as shown below:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get gopkg.in/mgo.v2
go: downloading gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
go: added gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22

Gorilla-Mux

Next we should add the gorilla/mux package , that will help use implement a very simple router in our API, to help match incoming request to respective handlers.
Dive deeper into the package and how it implements its dispatching features here.

To install the package and have it accessible in our project, while in our project root folder that houses the go.mod file, type in the following:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get -u github.com/gorilla/mux
go: downloading github.com/gorilla/mux v1.8.0
go: added github.com/gorilla/mux v1.8.0
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$

API Building

Now lets write a simple program that talks to the MongoDB database using our "mgo" driver and inserts a book " A song of Ice and Fire" book record.

Create a file called "main.go" open it and add the following code shown below:

 1 package main                                                                                                                                                        
  2 
  3 import (
  4   "os"
  5   "os/signal"
  6   "context"
  7   "time"
  8   "net/http"
  9   "io/ioutil"
 10   "encoding/json"
 11   "github.com/hashicorp/go-hclog"
 12   "log"
 13   "github.com/gorilla/mux"
 14   mgo "gopkg.in/mgo.v2"
 15   "gopkg.in/mgo.v2/bson"
 16 
 17 )
 18 
 19 
 20 // The struct below holds our database session information
 21 type DBSession struct {
 22     session      *mgo.Session
 23     collection   *mgo.Collection                                                                                                                                    
 24 }                                                                                                                                                                   
 25                                                                                                                                                                     
 26                                                                                                                                                                     
 27 //Book struct holds book data                                                                                                                                       
 28 type Book struct {                                                                                                                                                  
 29     ID            bson.ObjectId    `json: "id"          bson:"_id,omitempty"`                                                                                       
 30     Title         string           `json: "title"       bson:"title"`                                                                                               
 31     Authors       []string         `json: "authors"     bson: "authors"`                                                                                            
 32     Genre         []string         `json: "genre"       bson: "genre"`                                                                                              
 33     PublishDate   string           `json: "publishdate" bson: "publishdate"`                                                                                        
 34     Characters    []string         `json: "characters"  bson: "characters"`                                                                                         
 35     Publisher     Publisher        `json: "publisher"   bson:"publisher"`                                                                                           
 36 }                                                                                                                                                                   
 37                                                                                                                                                                     
 38  
   39 // Publisher is nested in Movie
 40 type Publisher struct {
 41     Name     string   `json: "budget"    bson:"name"`
 42     Country  string   `json: "country"   bson:"country"`
 43     website  string   `json: "website"   bson:"website"`
 44 }
 45 
 46 
 47 // GetBook fetches a book with a given ID
 48 func (db *DBSession) GetBook (w http.ResponseWriter, r *http.Request){
 49     vars := mux.Vars(r)
 50 
 51     w.WriteHeader(http.StatusOK)
 52     var book Book
 53     err := db.collection.Find(bson.M{"_id": bson.ObjectIdHex(vars["id"])}).One(&book)
 54     if err != nil {
 55         w.Write([]byte(err.Error()))
 56     } else {
 57         w.Header().Set("Content-Type", "application/json")
 58         response, _ := json.Marshal(book)
 59         w.Write(response)
 60     }
 61 }
 62 
 63 
  64 //PostBook adds a new book to our MongoDB collection
 65 func (db *DBSession) PostBook (w http.ResponseWriter, r *http.Request){
 66     var book Book
 67     postBody, _ := ioutil.ReadAll(r.Body)
 68     json.Unmarshal(postBody, &book)
 69 
 70     //Create a Hash ID to insert
 71     book.ID = bson.NewObjectId()
 72     err := db.collection.Insert(book)
 73     if err != nil {
 74         w.Write([]byte(err.Error()))
 75     } else {
 76         w.Header().Set("Content-Type","application/json")
 77         response, _ := json.Marshal(book)
 78         w.Write(response)
 79     }
 80 }
 81 
 82 
 83 
 84 
 85 //UpdateBook modifies the data of an existing book resource
 86 func (db *DBSession) UpdateBook(w http.ResponseWriter, r *http.Request){
 87     vars := mux.Vars(r)
 88     var book Book
 89 
 90     putBody, _ := ioutil.ReadAll(r.Body)
 91     json.Unmarshal(putBody, &book)
 92     err := db.collection.Update(bson.M{"_id":                                                                                                                       
 93         bson.ObjectIdHex(vars["id"])}, bson.M{"$set": &book})
 94 
 95     if err != nil {
 96         w.WriteHeader(http.StatusOK)
 97         w.Write([]byte(err.Error()))
 98     } else {
 99         w.Header().Set("Content-Type","text")
100         w.Write([]byte("Update succesfully"))
101     }
102 }
103 
104 
105 //DeleteBook removes the data from the db                                                                                                                           
106 func (db *DBSession) DeleteBook (w http.ResponseWriter, r *http.Request){
107      vars := mux.Vars(r)
108      err := db.collection.Remove(bson.M{"_id":
109          bson.ObjectIdHex(vars["id"])})
110      if err != nil {
111          w.WriteHeader(http.StatusOK)
112          w.Write([]byte(err.Error()))
113      } else {
114          w.Header().Set("Content-Type", "text")
115          w.Write([]byte("Delete Succesfully"))
116      }
117 }
118 
119 func main() {
120     l := hclog.Default()
121     session, err := mgo.Dial("127.0.0.1")
122     c := session.DB("booksdb").C("books")
123     db := &DBSession{session: session, collection:c}
124     addr := "127.0.0.1:8000"
125     if err != nil {
126         panic(err)
127     }
128     defer session.Close()
129     
130     //logger := log.New(os.Stdout, "", log.Ldate | log.Ltime)
131     // Create a new router
132     r := mux.NewRouter()
133 
134     //Attach an elegant path with handler
135     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.GetBook).Methods("GET")
136     r.HandleFunc("/api/books", db.PostBook).Methods("POST")
137     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.UpdateBook).Methods("PUT")
138     r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.DeleteBook).Methods("DELETE")
139 
140     srv := &http.Server{
141         Handler:       r,
142         Addr:          addr,
143         ErrorLog:      l.StandardLogger(&hclog.StandardLoggerOptions{}),
144         IdleTimeout:   time.Minute,
145         WriteTimeout:  15 * time.Second,
146         ReadTimeout:   15 * time.Second,
147     }                                                                                                                                                               
148 
149     //start the server
150     go func() {
151       l.Info("Starting server on port 8000 ")
152       err := srv.ListenAndServe()
153       if err != nil {
154           l.Error("Error starting the server :", "error", err)
155           os.Exit(1)
156       }        
157     }()
158 
159 
160     //Trap sigterm or interupt and gracefully shutdown the server
161     ch := make(chan os.Signal, 1)
162     signal.Notify(ch, os.Interrupt)
163     signal.Notify(ch, os.Kill)
164     
165     //Block untill a signal is received
166     sig := <- ch
167     log.Println("Got signal :", sig)
168 
169 
170     //gracefully shutdown the server, waiting max 30 for current operations to complete
171     ctx, _ := context.WithTimeout(context.Background(), 30 * time.Second)
172     srv.Shutdown(ctx)
173 
174 }

Remember to run the command "go mod tidy", to have all the direct and indirect dependecies brought in.

In the above code we are first creating two structs , DBSession and Book struct;

DBSession structure will be used to hold our database session and collection properties of the mgo driver. Book structure will hold our book data, also note that the Book struct has a nested Publisher struct.

Next we define a receiver Pointer Method for our Book struct that we intend to have implement Handler interface.In this case we first define the method GetBook, that fetches a book from the database using an id that has been supplied as part of the request.It then returns this method as a response. We then define the second method, called PostBook that does just what its title suggests, it takes a book sent in as part of the Request body and then attempts to insert this book into our database. We the created another handler method called UpdateBook, that takes the updated values of a book that already exists in our database and tries to alter one of its values. Our Final Handler is the DeleteBook Method that is used the data pertaining to a book that we have stored in our database. We identify this particular book using an id that is sent in as part of the request.

We then create our main function, in this we instantiate a session with our database that we will use for all our subsequent queries and updates, we then create a new router using the NewRouter() method of gorilla/mux, we then attach our handlers to this router to enable dispatching of the various request to their subsequent handlers. We then launch our server as a go-routine , we then create a channel to listen and respond to server terminating signals.

To add a new book to our database and to test our POST endpoint lets make a post request to our API, open your terminal from any location and make the following curl request:

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X POST \
> http://localhost:8000/api/books \
> -H 'cache-control: no-cache' \
> -H 'content-type: application/json' \
> -H 'postman-token: 6ef9507e-65b3-c3dd-4748-3a2a3e055c9c' \
> -d '{"title" :"A Dance with Dragons","authors":[ "George R.R Martin", "Phyllis Eisenstein"],"publishdate":"August 6,1998","characters":[ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}'

You will receive a reply from the API server that will look like:

{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":null,"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}

Go to your database and confirm that there us a new entry, whose id has been auto-generated by MongoDB for us, it should look like:

{ "_id" : ObjectId("62ddc5e87071de3d88044140"), "id" : ObjectId("62ddc5e89c99ea1313ae2568"), "title" : "A Dance with Dragons", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "genre" : [ ], "publishdate" : "August 6,1998", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}

Lets test our GetBook endpoint, we are go supply the id that was returned when we added our book using the PostBook endpoint above, i.e in my case "62ddc5e89c99ea1313ae2568":

mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X GET \
> http://localhost:8000/api/books/62ddc5e89c99ea1313ae2568 \
> -H 'cache-control: no-cache' \
> -H 'postman-token: 00282916-e7f8-5977-ea34-d8f89aeb43e2'

It should return:

{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":[],"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}

To test the delete endpoint:

curl -X DELETE http://localhost:8000/api/books/62ddc5e87071de3d88044140

it should return:

Delete Successfully

Now if we check our database using the find(), command we find that, the specific book "A Dance with Dragons" has been removed i.e:

> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }

To test the Put endpoint, that we will use to alter specific values of of book data:

I want to change the publisher to the book "A Feast of Crows" from "Bantam Books" to "Voyager" and i want to remove one of the authors "Phyllis Eisenstein"

the books currently looks like:

{ "_id" : 62dda3f1457193537fd4d245, "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }

Can you figure out how to alter the data, hint:

 curl -X PUT http://localhost:8000/api/books/.................

If you get stuck put it down in the comments and i will be sure to help.

Conclusion
In this article we have mainly looked at MongoDB, how to install and add data to it using its interactive shell, then we talked about building a simple gorilla-mux api and having it interface with our database to store, delete and alter the details of books send using HTTP request methods like PUT, POST, GET and DELETE.

Goodbye,see you soon.

Insertion Sort Algorithm : Go-lang and Python implementation.

myk_okoth_ogodo — Sun, 03 Jul 2022 00:01:56 +0000

The Insert-Sort Algorithm

In the case of an array based sequence. We start with the first element in the array.An array that only has a single
element by itself is already sorted by default.

Lets now add an arbitrary second element of the same type(let use integers in this case), if the second element that we add to our array is smaller than the first element we had previously added , we swap them such that the smaller second element goes to first position while the larger first element is relegated to the second position.

Lets now add a third element that we want to add, To settle this third element in its right position in our array that is already populated by two element, then this third element will have to be compared to the two elements already populating our array:

It will go like this;

1 - Is this third element smaller than the second element, if it is we move it to second position and the element that had been previously occupying that 2nd position is relegated to third position. If not smaller than the second position element(i.e if its the largest of the three array elements then we let if occupy the third position)

2 - If the third element was smaller than the second element, it now occupies the second position after displacing the originally second element. We now compare it to the first element, if its smaller than the first element, it goes into the first position as the smallest element, displacing the originally smallest element into second position.

3 - The above steps are repeated for each new addition to the array in-order to find the correct position for that element.

We can express the above describe algorithm as pseudo-code below


Algorithm InsertSort(Arr):
    input: an Array Arr of n comparable elements
    Output: The array Arr with elements rearranged in nondecreasing order
    for k from 1 to n - 1 do
        Insert Arr[k] at its proper location within Arr[0],Arr[1],...Arr[k]

The implementation of this Algorithm in python is shown below:


def InsertSort(Arr):
    for k in range(1, len(Arr)):
        currval = Arr[k]
        j = k
        while j > 0 and Arr[j-1] > currval:
            Arr[j] = Arr[j-1]
            j -= 1
        Arr[j] = currval    
    return Arr

In the above code we use an outer loop to consider each element in turn, and an inner loop that moves a newly considered element to its proper location relative to the
(sorted) sub-array of elements that are to its left.

The above implementation in python when translated line for line, logic for logic to Go-lang results in the code below,
Also note the input into the go-lang function is a Slice.In Go-lang a slice is a segment of dynamic arrays that can grow and shrink as you see fit.Like arrays,slices are index-able and have a length.Slices have a capacity and length property.

func InsertSort(Arr []int) []int{
    var x = len(items)
    for n := 1;n < x;n++ {
        v := n
        for v > 0 {
            if Arr[v-1] > Arr[v] {
               Arr[v-1], Arr[v] = Arr[v], Arr[v-1]
            }
         v = v - 1
        }
    }
   return Arr
}

Goodbye, see you soon.

Linked Lists : Go-lang and Python implementations.

myk_okoth_ogodo — Sat, 02 Jul 2022 22:19:43 +0000

Prelude

Linked lists provide an alternative to array-based sequences i.e Lists . Both array based sequences and linked lists keep elements in a specific order, but the implementation of this feature varies widely between the two.

Arrays provide a more centralized representation with one large chunk of memory capable of accommodating references to many elements. A linked list on the other hand relies of a more distributed architecture in which a lightweight object known as a node is used to represent each individual element of the linked list.

Each node in a linked list maintains a reference to its constituent element, and one or more references to neighboring nodes depending on whether its a single-linked list implementation or a double-linked list implementation. This representation then allows us to collectively represent the linear order of our sequence.

SINGLY-LINKED LISTS

Below we are going to give implementations of singly-linked list in both python and Go-lang as Queue, now this can be extended to Stacks and also trees for doubly linked lists.

Memory use

In comparison with doubly linked list, a singly linked list occupies less memory because it needs only one pointer to its successor. Each pointer variable holds the address to an element, and the pointer occupies 4 bytes; therefore the memory space occupied by the pointer variable in the singly linked list is 4 bytes. So memory also expands as per O(n) with n representing the number of elements and the 4 bytes being held constant for each pointer.

Time complexity

A singly linked list has an access time of 0(1), a search time of O(n), since we have to traverse the whole list from from first to last in one direction, it has an insertion time of O(n) and deletion time of O(n), in worst case and average case time.

Golang Singly-linked list Queue Implementation

package main

import "fmt"

type Node struct {
    element int
    next    *Node
}
type GoLinkedQue struct {
    head *Node
    tail *Node
    size int
}

func (l *GoLinkedQue) Len() int {
    return l.size
}

func (l *GoLinkedQue) is_empty() bool {
    if l.size == 0 {
        return true
    }
    return false
}

func (l GoLinkedQue) First() (int, error) {
    if l.head == nil {
        return 0, fmt.Errorf("The Queue is empty !")
    }
    return l.head.element, nil
}

func (l GoLinkedQue) Last() (int, error) {
    if l.head == nil {
        return 0, fmt.Errorf("The Queue is empty !")
    }
    if l.size == 1 {
        return l.head.element, nil
    }
    return l.tail.element, nil
}

func (l *GoLinkedQue) dequeue() int {
    if l.is_empty() {
        fmt.Errorf("The Queue is empty !")
    }
    answer := l.head.element
    l.head = l.head.next
    l.size--
    if l.is_empty() {
        l.tail = nil
    }
    return answer

}
func (l *GoLinkedQue) enqueue(e int) *Node {
    newest := &Node{
        element: e,
        next:    nil,
    }
    if l.is_empty() {
        l.head = newest
    } else {
        l.tail.next = newest
    }
    l.tail = newest
    l.size++
    return newest
}

func main() {
    queue := GoLinkedQue{}
    queue.enqueue(100)
    queue.enqueue(200)
    queue.enqueue(300)
    queue.enqueue(400)
    queue.enqueue(500)
    queue.enqueue(600)
    firstval, _ := queue.First()
    lastval, _ := queue.Last()
    fmt.Println("Length = ", queue.Len())
    fmt.Println("First Element :", firstval)
    fmt.Println("Last Element  :", lastval)
    fmt.Println("Is Queue empty:", queue.is_empty())
    queue.dequeue()
    queue.dequeue()
    queue.dequeue()
    queue.dequeue()
    queue.dequeue()
    queue.dequeue()
        fmt.Println("Length =", queue.Len())
        fmt.Println("Is Queue empty:", queue.is_empty())    
}

If you run the above code in the golang-playground you get the results below

output

Length =  6
First Element : 100
Last Element  : 600
Is Queue empty: false
Program exited.

single linked list implementation of a Queue structure in Python below

  1 class PythonLinkedQueue:                                                                                                                                            
  2     """ A python implementation of a queue data structure using a 'singly linked-list' for storage."""
  3 
  4     class _Node:
  5         """"Lightweight, non-public class for storing  a single Nodes attributes."""
  6         __slots__ = "_element", "_next"
  7 
  8         
  9         def __init__(self, element, next):
 10             """"Instantiate a single Node object."""
 11             self._element = element
 12             self._next    = next
 13 
 14     
 15     def __init__(self):
 16         """Create an empty queue."""
 17         self._head = None
 18         self._tail = None
 19         self._size = 0
 20 
 21 
 22     def __len__(self):
 23         """Return the number of elements in the queue."""
 24         return self._size
 25 
 26     def is_empty(self):
 27         """Return True if the queue is empty. """
 28         return self._size == 0
 29 
 30     def first(self):
 31         """Return but do not remove the element that sits at the front of the queue."""
 32         if self.is_empty():
 33             raise Empty("The Queue is Empty!")
 34         return self._head._element
 35 
 36     def last(self):
 37         """ Return butt do not remove the element that sits at the back of  the queue."""
  38         if self.is_empty():
 39             raise Empty("The Queue is Empty!")
 40         elif self._size == 1:
 41             return self._head._element
 42         else:
 43             return  self._tail._element
 44 
 45     def dequeue(self):
 46         """Remove and return the first element of the queue(FIFO)"""
 47         if self.is_empty():
 48             raise Empty('Queue is empty')
 49         result =  self._head._element
 50         self._head = self._head._next
 51         self._size -= 1
 52         if self.is_empty():
 53             self._tail = None
 54         return result
 55 
 56     def enqueue(self, e):
 57         """Add an element to the back of the queue. """
 58         newest = self._Node(e, None)
 59         if self.is_empty():
 60             self._head = newest
 61         else:
 62             self._tail._next = newest
 63         self._tail = newest
 64         self._size += 1

DOUBLY LINKED-LISTS

In a singly liked list each node maintains a reference to the node that is immediately after it. This asymmetry of the singly linked list comes with glaring limitations. Such as being unable to efficiently delete a node from the tail of the list or even we cannot perform an arbitrary node from the interior position of the if only given a reference to that node, because we cannot determine the node that immediately precedes the node we want to delete.

This is where a doubly-linked list enters the scene. To provide greater symmetry we define a linked list in which each node keeps an explicit reference to the node before it and a reference to the node after it.

Memory usage

This list implementation holds two addresses in a single node, one address points to the previous Node and the other address to the next Node. Therefore the space occupied by the two pointer variable is 8 bytes. The space is also bound to expand as per O(n), with n standing for the number of Nodes the linked list is currently holding, with each node eating up a constant of 8 bytes per Node.

Time Complexities

The doubly linked list has an insertion time complexity of O(1), a deletion time complexity of O(1), a search time complexity of O(n) and a access time complexity of O(n) both in average time and worst case time.

we will be implementing the deque data structure using doubly-linked lists. Deque is a queue-like data structure that supports insertion and deletion at both the front and the back of the queue.

Header and Trailer Sentinels

In order to avoid some special cases when operating near the boundaries of a doubly linked list, it helps to add special nodes at both ends of the list: a header node at the beginning of the list, and a trailer node at the end of the list. These “dummy” nodes are known as sentinels (or guards), and they do not store elements of the primary
sequence.

Implementation In python

  1 class _DoublyLinkedBase:
  2     """"A base class providing a doubly linked list representation."""                                                                                              
  3 
  4     class _Node:
  5         """A lightweight, non public class for storing a doubly linked node"""
  6         __slots__ = "_element","_next","_prev"
  7 
  8         def __init__(self, elment, prev, next):
  9             self._element = element
 10             self._prev    = prev
 11             self._next    = next
 12 
 13     def __init__(self):
 14         """Create ane empty list."""
 15 
 16         self._header  = self._Node(None, None, None)
 17         self._trailer = self._Node(None, None, None)
 18         self._header._next = self._trailer
 19         self._header._prev = self._header
 20         self._size = 0
 21 
 22     def  __len__(self):
 23         """Return the number of elements in the list."""
 24         return self._size
 25 
 26 
 27     def is_empty(self):
 28         """Return True if list is empty."""
 29         return self._size == 0
 30 
 31     def _insert_between(self, e, predecessor, successor):
 32         """Add element e between two existing nodes and return new node."""
 33         newest = self._Node(e, predecessor, successor)
 34         predecessor._next = newest
 35         successor._prev = newest
 36         self._size += 1                                                                                                                                             
 37         return newest 
 38 
 39     def _delete_node(self, node):
 40         """ Delete nonsentinel node from the list and return its element."""
 41         predecessor = node._prev
 42         successor = node._next
 43         predecessor._next = successor
 44         successor._prev = predecessor
 45         self._size -= 1
 46         element = node._element
 47         node._prev = node._next = node._element = None
 48         return element

The above base class is inherited by the following class below:

  1 class LinkedDeque(_DoubleLinkedBase):
  2     """ Double-ended queue implemented based on a doubly linked list."""                                                                                            
  3 
  4     def first(self):
  5         """Return but do not remmove the element at the front of th deque."""
  6         if self.is_empty():
  7             raise Empty("Deque is empty")
  8         return self._header._next._element
  9 
 10 
 11     def last(self):
 12         """Return but do not remove the element at the back of the deque."""
 13         if self.is_empty():
 14             raise Empty("Deque is empty")
 15 
 16         return self._trailer._prev._element
 17 
 18 
 19     def insert_first(self, e):
 20         """Add an element to the front of the deque."""
 21         self._insert_between(e, self._header, self._header._next)
 22 
 23     def insert_last(self, e):
 24         """Add an element to the backk of the deque."""
 25         self._insert_between(e, self._trailer._prev, self._trailer)
 26 
 27     def delete_first(self):
 28         """Remove and return the lement from the front of the deque."""
 29         if self.is_empty():
 30             raise Empty("Deque is empty")
 31         return self._delete_node(self._header._next)
 32 
 33     def delete_last(self):
 34         """Remove and return the element from the back of the deque."""
 35         
 36         if self.is_empty():                                                                                                                                         
 37             raise Empty("Deque is empty") 
 38         return self._delete_node(self._trailer._prev)
 39

To implement the deque data structure using doubly linked lists in Go-lang we implement it as below

Implementation in Golang

package main

import "fmt"

type Node struct {
    element int
    prev    *Node
    next    *Node
}

type GoDoublyLinkedQue struct {
    header  *Node
    trailer *Node
    size int
}

func (l *GoDoublyLinkedQue) Len() int {
    return l.size
}

func (l *GoDoublyLinkedQue) is_empty() bool {
    if l.size == 0 {
        return true
    }
    return false
}

func (l GoDoublyLinkedQue) First() (int, error) {
    if l.header.next == l.trailer {
        return 0, fmt.Errorf("The Queue is empty !")
    }
    return l.header.next.element, nil
}

func (l GoDoublyLinkedQue) Last() (int, error) {
    if l.trailer.prev == l.header {
        return 0, fmt.Errorf("The Queue is empty !")
    }
    return l.trailer.prev.element, nil
}

func (l *GoDoublyLinkedQue) insert_between(e int, predecessor, sucessor *Node) *Node {
    newest := &Node{
        element: e,
        prev:    predecessor,
        next:    sucessor,
    }
    predecessor.next = newest
    sucessor.prev = newest
    l.size++
    return newest
}

func (l *GoDoublyLinkedQue) delete_node(node *Node) int {
    predecessor := node.prev
    sucessor := node.next
    predecessor.next = sucessor
    sucessor.prev = predecessor
    l.size--
    velement := node.element
    node.prev = nil
    node.next = nil
    node.element = 0
    return velement
}

func (l *GoDoublyLinkedQue) insert_first(e int) *Node {
    node := l.insert_between(e, l.header, l.header.next)
    return node

}

func (l *GoDoublyLinkedQue) insert_last(e int) *Node {
    node := l.insert_between(e, l.trailer.prev, l.trailer)
    return node
}

func (l *GoDoublyLinkedQue) delete_first() int {
    if l.is_empty() {
        fmt.Errorf("The Deque is empty")
    }
    element := l.delete_node(l.header.next)
    return element
}

func (l *GoDoublyLinkedQue) delete_last() int {
    if l.is_empty() {
        fmt.Errorf("The deque is empty")
    }
    element := l.delete_node(l.trailer.prev)
    return element
}

func main() {
    deque := GoDoublyLinkedQue{}
    deque.header = &Node{
        element: 0,
        prev:    nil,
        next:    nil,
    }
    deque.trailer = &Node{
        element: 0,
        prev:    nil,
        next:    nil,
    }
    deque.header.next = deque.trailer
    deque.trailer.prev = deque.header
    deque.insert_first(100)
    deque.insert_last(500)
    firstVal, _ := deque.First()
    lastVal, _ := deque.Last()
    fmt.Println("Length = ", deque.Len())
    fmt.Println("Is deque empty: ", deque.is_empty())
    fmt.Println("First Element :", firstVal)
    fmt.Println("Last Element  :", lastVal)
    deque.delete_last()
    deque.delete_first()
    fmt.Println("Is deque empty: ", deque.is_empty())
}

The above code results in

Length =  2
Is deque empty:  false
First Element : 100
Last Element  : 500
Is deque empty:  true

Program exited.

Goodbye, see you soon.

Lets Build An E-commerce Django REST API & GraphQL on the same App .

myk_okoth_ogodo — Mon, 20 Jun 2022 12:09:29 +0000

In my previous tutorial i wrote about building Go-lang micro-services,in this tutorial i want to us to build a single application to serve GraphQL and REST at the same time. It will be an e-commerce API and depending on the reception we might get to building a front-end store based on React or React-Native(vote below in the comments).

We will be using the python framework called Django and the tool DRF(Django Rest Framework). To authenticate our API we will be implementing one scheme that comes out of the box with Django-rest and also one more third-party scheme just for demonstration purposes.
In the front-end i intend to use react-native, to build a simple mobile front-end store or react for a simple we-app, it will entirely depend on what the readers will prefer.

I intend to break this application into three small parts:

Build a simple Django REST API
Build a simple Django Graphql API based on the models we define for the REST API above.
Build a simple front-end app using React-Native/ React

You can refer to this respository if you get stuck.
Get yourself a warm cup of coffee and Lets begin.....

REST API BACK-END.

setting up our development Environment.
You either have Pipenv or Virtualenv installed in your system(if not i advice you to get those installed because they help us create an isolated development environment for our application).
if you have virtualenv installed in your system,open your terminal create a directory graphRestAPI and inside that directory create a development environment as below.

mykmyk@skynet:~/graphRestAPI$ virtualenv venv
Using base prefix '/usr'
New python executable in /home/mykmyk/graphRestAPI/venv/bin/python
Installing setuptools, pip, wheel...done.

To activate our newly created environment:

mykmyk@skynet:~/graphRestAPI$ source venv/bin/activate
(venv) mykmyk@skynet:~/graphRestAPI$

Now in this environment we install all the packages that we will be using in this application, we start with our framework Django and the django-rest tool.

(venv) mykmyk@skynet:~/graphRestAPI$ pip install Django djangorestframework

we then create a project using Django admin to house our apps, this structure while being a bit more code verbose, allows for better scaling of our applications.
We type in the following commands in our terminal.

(venv) mykmyk@skynet:~/graphRestAPI$ django-admin startproject CoreAPI
(venv) mykmyk@skynet:~/graphRestAPI$ ls
CoreAPI  venv

enter into the newly created 'CoreAPI' project folder and create an app that will hold all the project specific apps.

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ django-admin startapp coreGraphRestAPI
(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ ls
CoreAPI  coreGraphRestAPI  manage.py

move into the newly created 'coreGraphRestAPI' and delete the files , admin.py, models.py, tests.py,views.py

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI/coreGraphRestAPI$ ll
total 32
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun 19 09:55 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 19 09:55 ../
-rw-rw-r-- 1 mykmyk mykmyk   63 Jun 19 09:55 admin.py
-rw-rw-r-- 1 mykmyk mykmyk  164 Jun 19 09:55 apps.py
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun 19 09:55 __init__.py
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 19 09:55 migrations/
-rw-rw-r-- 1 mykmyk mykmyk   57 Jun 19 09:55 models.py
-rw-rw-r-- 1 mykmyk mykmyk   60 Jun 19 09:55 tests.py
-rw-rw-r-- 1 mykmyk mykmyk   63 Jun 19 09:55 views.py
(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI/coreGraphRestAPI$ rm admin.py models.py tests.py views.py

We then move into the 'CoreAPI' folder containing our 'settings.py' that houses our Django configurations, we then add our main app and our django-rest to the INSTALLED_APPS:

 #CoreAPI/settings.py

 33 INSTALLED_APPS = [
 34     'django.contrib.admin',
 35     ....
 39     'django.contrib.staticfiles',
 40     'rest_framework',                                                                                                                                               
 41     'coreGraphRestAPI'
 42 ]

Create User Application:

We will now create a user application for our project, To create a user application

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ ls
CoreAPI  coreGraphRestAPI  manage.py
(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ cd coreGraphRestAPI/ && python ../manage.py startapp Users

now if we move into the 'coreGraphRestAPI' folder we see:

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI/coreGraphRestAPI$ ll
total 24
drwxrwxr-x 5 mykmyk mykmyk 4096 Jun 19 10:24 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 19 09:55 ../
-rw-rw-r-- 1 mykmyk mykmyk  164 Jun 19 09:55 apps.py
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun 19 09:55 __init__.py
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 19 09:55 migrations/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 19 10:24 __pycache__/
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun 19 10:24 Users/

we now have to add this application to our settings.py

INSTALLED_APPS = [
 34     'django.contrib.admin',
        ...
 40     'rest_framework',
 41     'coreGraphRestAPI',
 42     'coreGraphRestAPI.Users',                                               
 43 ]

we then move into the file 'coreGraphRestAPI/Users' and open the file apps.py and add in the following

 1 from django.apps import AppConfig
  2 
  3 
  4 class UsersConfig(AppConfig):
  5     default_auto_field = 'django.db.models.BigAutoField'
  6     name  = 'coreGraphRestAPI.Users'                                        
  7     label = 'coreGraphRestAPI_Users'

And then open the file "init.py " in the same folder and add the following:

 1 default_app_config = 'coreGrapRestAPI.Users.apps.UsersConfig'

Sewing Our User Logic Together.
Django Auth System
Django comes with a built-in user authentication system. It handles user accounts, groups, permissions and cookie-based sessions. Django's authentication system handles both authentication and authorization.
However, we usually find that this out of the box solution doesn't cover all our use cases and we need to extend it to fit our specific project needs

User Model
In this case we will be creating a Custom user model by wrapping the Django AbstractBaseUser class. Our custom user will extend AbstractBaseUser and we will rewrite the UserManager to customize creation of a user in our database.

Lets dive into it.....

open the file "graphRestAPI/CoreAPI/coreGraphRestAPI/Users/models.py" and add the following code to it.

1 from django.db import models                                                                                                                                        
  2 from django.contrib.auth.models import AbstractBaseUser, PermissionsMixin, BaseUserManager
  3 
  4 
  5 class UserManager(BaseUserManager):
  6     def create_user(self, email, password, **extra_fields):
  7         if not email:
  8             raise ValueError("Email is required")
  9 
 10         email = self.normalize_email(email)
 11         user = self.model(email=email, **extra_fields)
 12         user.set_password(password)
 13         user.save()
 14         return user
 15 
 16     def create_superuser(self, email, password, **extra_fields):
 17         extra_fields.setdefault('is_staff', True)
 18         extra_fields.setdefault('is_superuser', True)
 19         extra_fields.setdefault('is_active', True)
 20         extra_fields.setdefault('first_name', "admin")
 21         extra_fields.setdefault('last_name', "admin")
 22 
 23         if not extra_fields.get("is_staff", False):
 24             raise ValueError('Superuser must have is_staff=True.')
 25 
 26         if not extra_fields.get("is_superuser", False):
 27             raise ValueError('Superuser must have is_staff=True.')
 28 
 29         return self.create_user(email, password, **extra_fields)
 30 
 31 
 32 
 33 class ImageUpload(models.Model):
 34     image = models.ImageField(upload_to="images")
 35 
 36     def __str__(self):
 37         return str(self.image)
 38 
 39 
 40 class UserProfile(models.Model):
 41     #user = models.OneToOneField(User, related_name="user_profile", on_delete=models.CASCADE)
 42     profile_picture = models.ForeignKey(ImageUpload, related_name="user_images", on_delete=models.SET_NULL, null=True)
 43     dob = models.DateField()
 44     phone = models.PositiveIntegerField()
 45     country_code = models.CharField(default="+234", max_length=5)
 46     created_at = models.DateTimeField(auto_now_add=True)
 47     updated_at = models.DateTimeField(auto_now=True)
 48 
 49     def __str__(self):
 50         return self.user.email
 51 
 52 
 53 class UserAddress(models.Model):
 54     user_profile = models.ForeignKey(UserProfile, related_name="user_addresses", on_delete=models.CASCADE)
 55     street = models.TextField()
 56     city = models.CharField(max_length=100)
 57     state = models.CharField(max_length=100)
 58     country = models.CharField(max_length=100, default="Nigeria")
 59     is_default = models.BooleanField(default=False)
 60 
 61     created_at = models.DateTimeField(auto_now_add=True)
 62     updated_at = models.DateTimeField(auto_now=True)
 63 
 64     def __str__(self):
 65         return self.user_profile.user.email
 66                                                                                                                                                                     
 67 
 68 
 69 class User(AbstractBaseUser, PermissionsMixin):
 70     email = models.EmailField(unique=True)
 71     first_name = models.CharField(max_length=100, default="yourfirstname")
 72     last_name = models.CharField(max_length=100, default="yourlastname")
 73     user_address = models.OneToOneField(UserAddress, related_name="user_address",blank=True, null=True, on_delete=models.CASCADE)
 74     created_at = models.DateTimeField(auto_now_add=True)
 75     updated_at = models.DateTimeField(auto_now=True)
 76 
 77     is_staff = models.BooleanField(default=False)
 78     is_active = models.BooleanField(default=True)
 79 
 80     USERNAME_FIELD = "email"
 81     objects = UserManager()
 82 
 83     def __str__(self):
 84         return self.email
 85 
 86     def save(self, *args, **kwargs):
 87         super().full_clean()
 88         super().save(*args, **kwargs)

Then head to your "settings.py" file in the folder "graphRestAPI/CoreAPI/CoreAPI" , in hat file add the following line , what it does is it now uses our extension of the Django User as the application user for things like authentication, cookie-based session management etc.

*settings.py *

AUTH_USER_MODEL = 'coreGraphRestAPI_Users.User'

The next logical step is serialization of our data, i.e our recently defined models.Serialization refers to the process of converting a data object (e.g., Python objects, Tensorflow models) into a format that allows us to store or transmit the data and then recreate the object when needed using the reverse process of de-serialization. In this case we will be serializing our data from python native objects to JSON for transmission and then de-serializing from JSON format into python native data types for storage.

create a file "serializer.py" in the folder "graphRestAPI/CoreAPI/coreGraphRestAPI/Users", open that file and add in the following code:

serializer.py

  1 from django.contrib.auth import get_user_model
  2 from rest_framework import serializers                                                                                                                              
  3 from rest_framework.reverse import reverse
  4 from .models import UserProfile, UserAddress, ImageUpload
  5 from drf_writable_nested import WritableNestedModelSerializer
  6 
  7 
  8 
  9 User = get_user_model()
 10 
 11 
 12 class ImageUploadSerializer(serializers.ModelSerializer):
 13     class Meta:
 14         model = ImageUpload
 15         fields = ('image',)
 16 
 17 
 18 class UserProfileSerializer(WritableNestedModelSerializer, serializers.ModelSerializer):
 19     profile_picture = ImageUploadSerializer(read_only = False)
 20     class Meta:
 21         model = UserProfile
 22         fields = ('profile_picture','dob','phone','country_code')
 23 
 24 
 25 class UserAddressSerializer(WritableNestedModelSerializer, serializers.ModelSerializer):
 26     user_profile = UserProfileSerializer(read_only = False)
 27 
 28     class Meta:
 29         model = UserAddress
 30         fields = ('user_profile','street','city','state','country','is_default')
 31 
 32 
 33 
 34 class UserSerializer(WritableNestedModelSerializer, serializers.ModelSerializer):
 35     links = serializers.SerializerMethodField('get_links')
 36     user_address = UserAddressSerializer(read_only=False)
 37     
 38     class Meta:
 39         model = User
 40         fields = ('id', User.USERNAME_FIELD,'password','first_name','last_name','user_address','is_active','links')
 41 
 42 
 43     def get_links(self, obj):
 44         request = self.context['request']
 45         username = obj.get_username()
 46 
 47         return {
 48                 #'self': reverse('user-detail',
 49                 #kwargs = {User.USERNAME_FIELD: username}, request=request),
 50                 }

run the command below to install the drf_writable tool

pip install drf_writable_nested

All we are doing in the above code is creating serializer and deserializer classes for our three models: User model, UserProfile model, UserAddress model.
We are using the tool "drf_writable_nested" to serialize our nested classes, because of the one_to_one and one_to_many relationship between our model classes.

Next on the table is to create viewsets. Now, there are many ways to go about this but to make the process simple , i will be using viewsets.ModelViewSet, The ModelViewSet class inherits from GenericAPIView and includes implementations for various actions, by mixing in the behavior of the various mixin classes.
The actions provided by the ModelViewSet class are .list(), .retrieve(), .create(), .update(), .partial_update(), and .destroy().

Also note that although this class provides the complete set of create/list/retrieve/update/destroy actions by default, you can restrict the available operations by using the standard permission classes.

open the file "views.py" in the folder "graphRestAPI/CoreAPI/coreGraphRestAPI/Users",

views.py

 1 from django.contrib.auth import get_user_model                                                                                                                      
  2 from rest_framework import authentication, permissions, viewsets, filters
  3 from .serializer import UserSerializer
  4 from django_filters.rest_framework import DjangoFilterBackend
  5 
  6 
  7 
  8 from oauth2_provider.contrib.rest_framework import TokenHasReadWriteScope, TokenHasScope
  9 
 10 
 11 User = get_user_model()
 12 
 13 class DefaultsMixin(object):
 14     """Default settings for view authentication, permissions, filtering and pagination."""
 15     authentication_classes = (
 16         authentication.BasicAuthentication,
 17         authentication.TokenAuthentication,
 18             )
 19     permission_classes = (
 20         permissions.IsAuthenticated,
 21         #TokenHasReadWriteScope,
 22         )
 23     paginate_by = 25
 24     paginate_by_param = 'page_size'
 25     max_paginate_by = 100
 26     filter_backends = (
 27         DjangoFilterBackend,
 28         filters.SearchFilter,
 29         filters.OrderingFilter
 30             )    
 31     
 32 class UserViewSet(DefaultsMixin, viewsets.ModelViewSet):
 33     """API endpoint for listing users."""
 34     
 35     lookup_field = User.USERNAME_FIELD
 36     lookup_url_kwarg = User.USERNAME_FIELD
 37     queryset = User.objects.order_by(User.USERNAME_FIELD) 
 38     serializer_class = UserSerializer

run the following in your terminal

pip install django-filter==21.1 django-oauth-toolkit==2.0.0

Notice Our DefaultsMixin Class above.A mixin is a class that provides method implementations for reuse by multiple related child classes. However, the inheritance is not implying an is-a relationship. A mixin doesn't define a new type.

Connecting to the Router
At this point the Users app has the basic data model and view logic in place, but it has not been connected to the URL routing system. Django-rest-framework has its own URL routing extension for handling ViewSets, where each ViewSet is registered with the router for a given URL prefix. This will be added to a new file in graphRestAPI/CoreAPI/coreGraphRestAPI/urls.py.

So create a urls.py file in the folder "graphRestAPI/CoreAPI/coreGraphRestAPI" and add the following code:

  1 from rest_framework.routers import DefaultRouter                                                                                                                    
  2 from .Users.views import UserViewSet
  3 
  4 
  5 router = DefaultRouter()
  6 
  7 router.register(r'users', UserViewSet)
  8

Finally, this router needs to be included in the root URL configuration in /urls.py.

  1 from django.contrib import admin                                                                                                                                    
  2 from django.urls import path
  3 from django.conf.urls import include, url
  4 from rest_framework.authtoken.views import obtain_auth_token
  5 from coreGraphRestAPI.urls import router
  6 
  7 
  8 
  9 urlpatterns = [
 10     url(r'^admin/', admin.site.urls),
 11     url(r'^api/token/', obtain_auth_token, name='api-token'),
 12     url(r'^api/', include(router.urls)),
 13     
 14 ]

Create Product application

Next we want to create the second application called Products, the first application was Users application.
We follow the same flow as we just did to create the user app:

Go to the folder "graphRestAPI/CoreAPI", then run the command below to create the app:

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ cd coreGraphRestAPI/ && python ../manage.py startapp Products

then to create Businesses app:

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ cd coreGraphRestAPI/ && python ../manage.py startapp Businesses

Then to create the Cart app

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ cd coreGraphRestAPI/ && python ../manage.py startapp Cart

Then to create the Wish app

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI$ cd coreGraphRestAPI/ && python ../manage.py startapp Wish

now if you cd into the folder "graphRestAPI/CoreAPI/coreGraphRestAPI", it looks like below:

(venv) mykmyk@skynet:~/graphRestAPI/CoreAPI/coreGraphRestAPI$ ll
total 40
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun 20 12:03 ./
drwxrwxr-x 6 mykmyk mykmyk 4096 Jun 20 13:52 ../
-rw-rw-r-- 1 mykmyk mykmyk  164 Jun 19 09:55 apps.py
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 20 13:47 Businesses/
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 20 14:17 Cart/
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun 19 09:55 __init__.py
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 20 12:39 Products/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 20 12:07 __pycache__/
-rw-rw-r-- 1 mykmyk mykmyk  486 Jun 20 12:03 urls.py
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 20 13:52 Users/
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 20 14:17 Wish/

proceed into the folder "graphRestAPI/CoreAPI/coreGraphRestAPI/Products", and the open the "apps.py" just as we did in the Users app above, then add the following code:

apps.py

  1 from django.apps import AppConfig
  2 
  3 
  4 class ProductsConfig(AppConfig):
  5     default_auto_field = 'django.db.models.BigAutoField'
  6     name  = 'coreGraphRestAPI.Products'                                                                                                                             
  7     label = 'coreGraphRestAPI_Products'

Next, open the file "init.py" in the same folder and add the following code:

init.py

  1 default_app_config = 'coreGraphRestAPI.Products.apps.ProductsConfig'

Repeat the two processes above for the three new apps that we have created i.e Cart , Wish, Businesses

then we head to our settings.py file in the folder : graphRestAPI/CoreAPI/CoreAPI

settings.py

  33 INSTALLED_APPS = [
 34     'django.contrib.admin',
         ....
 40     'rest_framework',
 41     'django_filters',
 42     'coreGraphRestAPI',
 43     'coreGraphRestAPI.Users',
 44     'coreGraphRestAPI.Products',
 45     'coreGraphRestAPI.Businesses',
 46     'coreGraphRestAPI.Cart',                                                                                                                                        
 47     'coreGraphRestAPI.Wish',
 48 ]

Notice above we have added the second application "coreGraphRestAPI.Products", the third Businesses , fourth Cart and finally Wish.

The next task is defining the models for this "Products" application, to do this we head to the folder: graphRestAPI/CoreAPI/coreGraphRestAPI/Products

open the file "models.py" and add the following code,

models.py

  1 from django.db import models                                                                                                                                        
  2 from coreGraphRestAPI.Users.models import ImageUpload
  3 from coreGraphRestAPI.Businesses.models import Business
  4 from django.contrib.auth import get_user_model
  5 
  6 User = get_user_model()
  7 
  8 class Category(models.Model):
  9     name = models.CharField(max_length=100, unique=True)
 10     created_at = models.DateTimeField(auto_now_add=True)
 11 
 12     def __str__(self):
 13         return self.name
 14 
 15 class ProductImage(models.Model):
 16     image = models.ForeignKey(ImageUpload, related_name="product_image", on_delete=models.CASCADE)
 17     is_cover = models.BooleanField(default=False)
 18     created_at = models.DateTimeField(auto_now_add=True)
 19     update_at = models.DateTimeField(auto_now=True)
 20 
 21     def __str__(self):
 22         return f"{self.product.business.name} - {self.product.name} - {self.image}"
 23 
 24 class ProductComment(models.Model):
 25     user = models.ForeignKey(User, related_name="user_comments_on_product", on_delete=models.CASCADE)
 26     comment = models.TextField()
 27     rate = models.IntegerField(default=3)
 28     created_at = models.DateTimeField(auto_now_add=True)
 29 
 30 class Product(models.Model):
 31     category = models.ForeignKey(Category, related_name="product_categories", on_delete=models.CASCADE)
 32     business = models.ForeignKey(Business, related_name="belongs_to_business", on_delete=models.CASCADE)
 33     name = models.CharField(max_length=100)
 34     price = models.FloatField()
 35     total_available = models.PositiveIntegerField()
 36     total_count = models.PositiveIntegerField()
 37     description = models.TextField()
 38     product_images = models.ForeignKey(ProductImage, related_name="product_images", on_delete=models.CASCADE)
 39     product_comments = models.ForeignKey(ProductComment, related_name="product_comments", on_delete=models.CASCADE)
 40     created_at = models.DateTimeField(auto_now_add=True)
 41     update_at = models.DateTimeField(auto_now=True)
 42 
 43     def __str__(self):
 44         return f"{self.business.name} - {self.name}"
 45 
 46     class Meta:
 47         ordering = ("-created_at",)

After defining the model Product the next natural step is to define the serializer. Create a file "serializer.py" open it and add the following code:

serializer.py

  1 from django.contrib.auth import get_user_model                                                                                                                      
  2 from rest_framework import serializers
  3 from rest_framework.reverse import reverse
  4 from .models import Product, ProductImage, ProductComment, Category, ImageUpload
  5 from drf_writable_nested import WritableNestedModelSerializer
  6 
  7 User = get_user_model()
  8 
  9 class CategorySerializer(serializers.ModelSerializer):
 10     class Meta:
 11         model = Category
 12         fields = "__all__"
 13 
 14 class ImageUploadSerializer(serializers.ModelSerializer):
 15     class Meta:
 16         model = ImageUpload
 17         fields = "__all__"
 18 
 19 class ProductImageSerializer(WritableNestedModelSerializer, serializers.ModelSerializer):
 20     image = ImageUploadSerializer(read_only = False)
 21     class Meta:
 22         model = ProductImage
 23         fields = "__all__"
 24 
 25 class ProductCommentSerializer(serializers.ModelSerializer):
 26     class Meta:
 27         model = ProductComment
 28         fields = "__all__"
 29 
 30 class ProductSerializer(WritableNestedModelSerializer, serializers.ModelSerializer):
 31     links = serializers.SerializerMethodField('get_links')
 32     category = CategorySerializer(read_only = False)
 33     product_images = ProductImageSerializer(read_only = False)
 34     product_comments = ProductCommentSerializer(read_only = False)
 35     class Meta:
 36         model = Product
 37         fields = ['id','category','business','name','price','total_available','total_count','description','product_images','product_comments','links']
 38 
 39     def get_links(self, obj):
 40         request = self.context['request']
 41         links = {
 42                 'self': reverse('product-detail',
 43                 kwargs = {'pk': obj.pk}, request=request),
 44                 'business': None,
 45                 }
 46 
 47         if obj.business:
 48             links['business'] = reverse('business-detail',
 49                 kwargs = {'pk': obj.business}, request=request)        
 50         return links

After the serializing our data the next logical step is to piece together our Filters, create a file called "forms.py"

Open it and add the following code:

  1 from django_filters.rest_framework import BooleanFilter, FilterSet, DateFilter                                                                                      
  2 from django.contrib.auth import get_user_model
  3 from .models import Product
  4 
  5 
  6 class NullFilter(BooleanFilter):
  7     """ Filter on a field set as null or not."""
  8     def filter(self, qs, value):
  9         if value is not None:
 10             return qs.filter(**{'%s__isnull' % self.name: value})
 11         return qs
 12 
 13 
 14 class ProductFilter(FilterSet):
 15     class Meta:
 16         model = Product
 17         fields = ('business','total_count','price','category')

The above file is uses to define our filters that will be used on our API data.

Next logical step is to create our viewsets, similar to the ones we defined when creating our Users.

open the "views.py" file and add the following code :

views.py

  1 from django.contrib.auth import get_user_model
  2 from rest_framework import authentication, permissions, viewsets, filters                                                                                           
  3 from .serializer import ProductSerializer
  4 from .models import Product
  5 from .forms import ProductFilter
  6 from django_filters.rest_framework import DjangoFilterBackend
  7 from oauth2_provider.contrib.rest_framework import TokenHasReadWriteScope, TokenHasScope
  8 
  9 
 10 class DefaultsMixin(object):
 11     """Default settings for view authentication, permissions, filtering and pagination."""
 12     authentication_classes = (
 13         authentication.BasicAuthentication,
 14         authentication.TokenAuthentication,
 15             )
 16     permission_classes = (
 17         permissions.IsAuthenticated,
 18         #TokenHasReadWriteScope,
 19         )
 20     paginate_by = 25
 21     paginate_by_param = 'page_size'
 22     max_paginate_by = 100
 23     filter_backends = (
 24         DjangoFilterBackend,
 25         filters.SearchFilter,
 26         filters.OrderingFilter
 27             )    
 28 
 29 
 30 class ProductViewSet(DefaultsMixin, viewsets.ModelViewSet):
 31     """API endpoint for listing Products."""
 32     
 33     queryset = Product.objects.order_by('name',) 
 34     serializer_class = ProductSerializer   
 35     filter_class = ProductFilter
 36     search_fields = ('name','price','business')                                                                                                                     
 37     ordering_fields = ('price','name','total_available')

To finally add this view to our urls.py file in our root application "coreGraphRestAPI" folder , it will then look like this together with the other new applications:

CoreAPI/coreGraphRestAPI/urls.py

  1 from rest_framework.routers import DefaultRouter                                                                                                                    
  2 from .Users.views import UserViewSet
  3 from .Products.views import ProductViewSet
  4 from .Businesses.views import BusinessViewSet
  5 from .Cart.views import  CartViewSet
  6 from .Wish.views import  WishViewSet
  7 
  8 router = DefaultRouter()
  9 
 10 router.register(r'users', UserViewSet)
 11 router.register(r'products', ProductViewSet)
 12 router.register(r'businesses',BusinessViewSet)
 13 router.register(r'carts', CartViewSet)
 14 router.register(r'wishes', WishViewSet)
~

Now to avoid repeating myself, the code for the models, serializers, views and Filters for the Businesses, Cart, Wishes applications can be found at the following repository:
e-commerce repo

In this part we will be using dbsqlite, but we will migrate to postgresql in the next chapter, so dont worry about that.

After you finish creating the above applications you can run the following command to migrate your data

python manage.py makemigrations
python manege.py  migrate

Then create a superuser to log into your browsable API

python manage.py createsuperuser

Then launch the application

python manage.py runserver

Next, before we proceed to the next part of this series there are things we have to get out of the way

We need to move our data to postgresql or mysql (help me choose which one)
We need to wire up token authentication for our API
We need to write tests for our API application
Containerze our app using docker Till then, Keep Safe and stay curious.

Maps : In Go-lang and Python.

myk_okoth_ogodo — Fri, 03 Jun 2022 22:07:53 +0000

This will be my second article in my "data structures in Python and Go-lang series". In my first article i talked about the word slice as applied to python and Go-lang sequences. Just to recap that topic.Slice is an in-built python function that returns a slice object that we can initialize with the arguments "start", "end" and "step" as indices, we can then use this slice object to "slice/extract" a contiguous number of elements from our sequences. In Go-lang slice is a mutable data structure that can be used to hold and manage collections of elements of a similar type. Today we talk about Maps in Go-lang an Python.

The Map data structure

Known in some quarters as associative arrays or dictionaries. Its one of the most important data structure in many programming languages, it involves the "mapping" or "associating" unique keys to their respective values.
Take for example the mapping below, made using keys as country names and their associated values as the unit of currency used in that country:

Kenya => Shilling
Austria => Euro
Belgium => Euro
Chile => Peso
China => Yuan
Denmark => Krone
America => Dollar

As illustrated above maps' keys should be unique while their values can be shared for example above both "Austria" and "Belgium" keys have a similar value of "Euro".
Unlike a standard array,our maps indices do not have to be in numeric format or consecutive. For example a normal array should look like :
countries =["Kenya","Austria","Belgium","Chile","China","Denmark","America"]

in the above case the indices to the individual elements of the array are numeric and consecutive. Such that to access the individual elements we use these numeric indices like below:

countries[0] = "Kenya"
countries[1] = "Austria"
countries[2] = "Belgium"
countries[3] = "Chile"
.....
countries[6] = "America"

PS: remember arrays in almost all the languages are zero-indexed.

However to access values in our map structure .....

Maps use an array-like syntax for indexing, unlike a standard array, indices for a map(i am using a python dictionary for illustration) need not be consecutive or numeric.

currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"]

//to access the elements;  
currencies["Kenya"]   =  "Shilling"
currencies["Austria"] =  "Euro"
currencies["Belgium"] =  "Euro"
.....
currencies["China"]   =  "Yuan"

Creation of maps in Python and Go-lang

in python the map is implemented as an abstraction called dictionary, in which unique keys are mapped to associated values:

syntax is : identifier = {"key1":"value1","key2":"value2","key3":"value3"}

Example:

currencies = {
    "Kenya"  : "Shilling",
    "Austria": "Euro",
    "Belgium": "Euro",
    "Chile"  : "Peso",
    "China"  : "Yuan",
    "Denmark": "Krone",
    "America": "Dollar",
}

in Go-lang maps are an unordered collection of key/value pairs, the ability to retrieve data quickly based on use keys as indices is an attractive character of maps as data stores:
Their creation involves the syntax below:

var identifier = map[key-type]value-type{"key1":"value1","key2":"value2","key3":"value"}

Notice above due to the strict typing nature of Golang we are explicitly stating the type of our keys and value during map instantiation.
Example:

package main
import "fmt"

var currencies = map[string]string{"Kenya":"Shilling","Austria":"Euro", "Belgium":"Euro","Chile":"Peso","China":"Yuan","America":"Dollar"}
func main(){
    fmt.Println(currencies) 
}

var identifier = make(map[key-type]value-type)

Notice in the syntax above, we use the Go-lang built-in make function to create a map object.

Example:

package main
import "fmt"

func main(){
    var currencies = make(map[string]string)
    currencies["Kenya"]   = "Shilling"
    currencies["Austria"] = "Euro"
    currencies["Belgium"] = "Euro"
    currencies["Chile"]   = "Peso"
    currencies["China"]   = "Yuan"
    currencies["America"] = "Dollar"

    fmt.Println(currencies)    //will print the whole map
}

Accessing and Assigning values in Go-lang and Python maps ....

In python to access and assign the values of our map, we use the keys as indices into our map,the syntax is; identifier[key] = value

For example using our currency map created above

print(currencies["Kenya"])    // will print "shilling" value
print(currencies["Austria"])  // will print "Euro" value
print(currencies["Belgium"])  // will print "Euro" value
print(currencies["Chile"])    // will print "Peso" value
print(currencies["Chine"])    // Will print "Yuan" value
print(currencies["America"])  // will print "Dollar" value

In Go-lang, we use a similar indexing structure to access individual elements in our map, the syntax is: identifier[key] = value

Example:

package main
import "fmt"

func main(){
    var currencies = make(map[string]string)
    currencies["Kenya"]   = "Shilling"
    currencies["Austria"] = "Euro"
    currencies["Belgium"] = "Euro"
    currencies["Chile"]   = "Peso"

    fmt.Println(currencies["Kenya"])
    fmt.Println(currencies["Austria"])
    fmt.Println(currencies["Belgium"])
    fmt.Println(currencies["Chile"])
}

Determining Map length in Go-lang and Python

In python the map class implements a len() function, that counts the number of key-value pairs

the syntax is , len(identifier)

Example

currencies = {
    "Kenya"  : "Shilling",
    "Austria": "Euro",
    "Belgium": "Euro",
    "Chile"  : "Peso",
    "China"  : "Yuan",
    "Denmark": "Krone",
    "America": "Dollar",
}

print len(currencies)   // will print out the value 7(number of currency key-value pairs)

In Go-lang we stick to an identical syntax to python that looks like len(identifier)

Example

package main
import "fmt"

func main(){
    var currencies = make(map[string]string)
    currencies["Kenya"]   = "Shilling"
    currencies["Austria"] = "Euro"
    currencies["Belgium"] = "Euro"
    currencies["Chile"]   = "Peso"
    currencies["China"]   = "Yuan"
    currencies["America"] = "Dollar"

    fmt.Println(len(currencies))    //will print length of the whole map
}

Adding items to maps in Go-lang and Python ....

In python to add a key-value pair to an existing map object we use the following syntax: identifier["new-key"] = "new-value"

Example


currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}

//To add Germany Franc key value pair to the above map
currencies["Germany"] = "Franc"

print(currencies)

The print statement will output:

{"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan", "Germany":"Franc"}

In Go-lang to add an item to an existing map, we introduce a new index key and assigning a value to it, syntax is: indentifier["new-key"] = "new-value"


Example:
package main
import "fmt"

func main(){
    var currencies = map[string]string{"Kenya":"Shilling","Austria":"Euro","Belgium":"Euro"}
    fmt.Println(currencies)  //initial Map
    currencies["Chile"] = "Peso"  // add chile-peso key-value pair
    currencies["China"] = "Yuan"
    currencies["China"] = "Yuan"
}

Updating a value in existing maps in Python and Go-lang ....

In Python you can update the value of a specific item by referring to its key name, the syntax is] identifier["old-key"] = "new-value"

Example:

currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}

// To update the Belgium item value to "dollar"
print(currencies)   // Initial map
currencies["Belgium"] = "Dollar"
print(currencies)   // Updated map

In Golang we follow a similar path to Python above

Example

package main
import "fmt"

func main(){
    var currencies = map[string]string{"Kenya":"Shilling","Austria:"Euro","Belgium":"Euro","Chile":"Peso","China":"Yuan"}
    fmt.Println(currencies)  // Initial map
    //updating the value of the Belgium key
    currencies["Belgium"] = "Dollar"
    fmt.Println(currencies)
}

Deleting items in Go-lang and Python Maps ....

To delete an item in Python we use the built in del function, syntax will look like : del(identifies["key"])

Example:


currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}
print(currencies) // initial map
del(currencies["Belgium"])
print(currencies)  // Map with Belgium deleted

-Still in python Map we can use the built in pop function to delete an item

syntax ; identifier.pop("Key")

Example:

currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}
print(currencies) // initial Map
// to delete the Belgium item
currencies.pop("Belgium")
print(currencies)  // Map with the Belgium item deleted

In Go-lang the built-in delete function deletes an item from a given map associated with the provided key

Example

package main
import "fmt"

func main() {
    var currencies = make(map[string]string)
    currencies["Kenya"] = "Shilling"
    currencies["Austria"] = "Euro"
    currencies["Belgium"] = "Euro"
    currencies["Chile"]   = "Peso"
    currencies["China"]   = "Yuan"

     fmt.Println(currencies)
     delete(currencies, "Belgium")
     fmt.Println(currencies)
}

Iterating over a Map in Go-lang and Python

-In Python we can use the for loop to iterate over a map using the syntax:
for key,value in range(identifier.items()) , to iterate over key,value pairs
for key in range(identifier.keys()), to iterate over keys in a map
for value in range(identifier.value()), to iterate over the values of a map

Example:

currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}

for key,value in range(currencies.items()):
    print(key, ":" ,value)

//to print only the keys
for key in range(currencies.keys()):
    print(key)

//to print only the values
for value in range(currencies.values()):
    print(value)

In Go-lang the for..range loop statement can be used to fetch the index and element of a map: syntax for key, value := range identifier{ //do something }

Example:

package main
import "fmt"

func main() {
    var currencies = map[string]string{"Kenya":"Shilling","Austria":"Euro","Belgium":"Euro","Chile":"Peso","China":"yuan","Amerika":"Dollars"}

for key,value := range currencies {
    fmt.Println("Key: ", key, "=>", "Value: ", value)
}
}

Sorting of Map keys and Values in Golang and Python ....

In Go-lang we can sort kays and values of maps as follows; A keys slice is created to store keys value of map and then sort the slice. The sorted slice is used to print values of map in key order.

Example

Package main
import (
    "fmt"
    "sort"
)

func main() {
    unSortedCurrencies := map[string]string{"Kenya":"Shilling", "Austria":"Euro","Belgium":"Euro","Chile":"Peso","China":"yuan","America":"Dollar"}

keys := make([]string, 0, len(unSortedCurrencies))
for k := range unSortedCurrenciesMap{
    keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
    fmt.Println(k, unSortedCurrencies[k])
}

}

To sort Map Values


Package main
import (
    "fmt"
    "sort"
)

func main() {
    unSortedCurrencies := map[string]string{"Kenya":"Shilling", "Austria":"Euro","Belgium":"Euro","Chile":"Peso","China":"yuan","America":"Dollar"}

values := make([]string, 0, len(unSortedCurrencies))
for _,v := range unSortedCurrenciesMap{
    values = append(values, v)
}
//sort slice values
sort.String(values)

//Print values of sorted Slice
for _, v := range values {
    fmt.Println(v)
}
}

In Python , we can sort our maps as follows

To sort the keys we use the in-built sorted function,
syntax : sorted(identifier)

Example:

currencies = {"Kenya":"Shilling", "Austria":"Euro", "Belgium":"Euro",Chile:"Peso","China":"Yuan"}

sortedCurrencyKeys = sorted(currencies)
print(sortedCurrencyKeys)

To sort the values of a Map, we can use the following structure

currencies = {"Kenya":100, "Austria":200, "Belgium":110,Chile:150,"China":98}

sortedCurrencies = {k :v for k, v in sorted(currencies.items(), key=lambda item: item[1])}

Print(sortedCurrencies)

Merging two or more maps ....

In Go-lang we can merge two maps as shown below Example

package main
import "fmt"

func main() {
    firstCurrencies = map[string]int{"Kenya":1,"Belgium":2,"China":3}
secondCurrencies = map[string]int{"Austria":2,"Belgium":4,"Chile":6, "America":7}

for key, value := range secondCurrencies {
    firstCurrencies[key] = value
}

fmt.Println(firstCurrencies)
}

-In python we can merge dictionaries using the method update() and also using (**)operator.

Example

firstCurrencies = {"Kenya":"Shilling","Austria":"Euro","Belgium":"Euro"}
secondCurrencies = {"Austria":"Dollar","Chile":"Peso","China":"Yuan"}

secondCurrencies.update(firstCurrencies)
print(secondCurrencies)

using ** is a trick where a single expression is used to merge two dictionaries and stored in a third dictionary

firstCurrencies = {"Kenya":"Shilling","Austria":"Euro","Belgium":"Euro"}
secondCurrencies = {"Austria":"Dollar","Chile":"Peso","China":"Yuan"}

combinedCurrencies = {**firstCurrencies, **secondCurrencies}
print(combinedCurrencies)

Conclusion

We have managed to look at Maps in Go-lang and Python, interrogating things like , accessing and addition of items, iterating through the items , deleting the items, merging of maps. I will continuously update this article with any information that i have overlooked into the future. Thank you. See you soon.

Using graphQL+gRPC+Golang to Create a Bike Rental Microservices, with persistence on ArangoDB.

myk_okoth_ogodo — Thu, 02 Jun 2022 15:12:10 +0000

ArangoDB,Golang,Grpc(The holy Trinity of Microservices)

In this two part series i want us to build a simple bike rental micro-services application to demonstrate the integration of gRPC with graphql-client using go and persistence of data on an ArangoDb database. It will be divided into the following parts:

In part one of this series we will develop two APIs based on the gRPC framework, a "Users API", that will represent people who will be renting our bikes. The second API will be a "Bikes API", we will then have these two services maintain independent database instances on an ArangoDB database.
In our second part we will link the above two services to a graphql-client.

Find the code to this project "here".

The Technologies, architectures and FrameWorks will be implementing and using are:

Golang:

Officially known as the Go programming language. Go is a language that has been built ground-up for concurrency. It was conceptualized and built by Google engineers.Its a statically typed,natively compiled, garbage-collected ,concurrent, post-OOP(object oriented language).
In current landscape of distributed cloud computing, a little primer: A distributed system is a collection of independent computers that appears to its users as a single coherent system.Now, Golang particularly shines in this sphere using Concurrency. Concurrency in Go is the ability for functions to run independent of each other. Its concurrency mechanisms make it easy to write programs that get the most out of multi core and networked machines(distributed systems). To accomplish this it uses Goroutines, which are independent units of work that get scheduled and then executed on available logical processor. Check out their ofial [sficial site.

Microservices.

Micro-services architecture allow developers to decompose our apps into relatively small app(services) that are loosely coupled to each other. These small apps can then be built to address particular unique business concerns independently using different software and hardware stacks, the deployment and scaling once online can also be done independently too.

This design structure provides a lot of advantages like flexibility in development,robustness in deployment and operation and modularity in scaling . Dive deeper here.

gRPC(google Remote Procedure Call)

This is an RPC(Remote Procedure Call) framework developed by engineers at google, it allows us to innplement RPC based Application Programming Interfaces(API's) on HTTP/2 in contrast to REST based APIs.

Due to the use of HTTP/2 gRPC is more efficient, HTTP/2 can make multiple requests in parallel on a long-lived connection in a thread safe way. The payloads in gRPC are binary-based making them smaller than JSON based messages.Futhermore,HTTP/2 has built-in header compression.
Check out their site

GrapQL

Graphql is an API querying language that was developed to cure the shortcomings of REST such as the more prevalent over-fetching and under-fetching. It was specifically designed for flexibility and efficiency. It also allows for rapid Product iterations on our front-end. Dive deeper into this subject here.

Protocol buffer

Protobuf are googles platform agnostic, language neutral data serialization method, that allows us to initially describe our data in the form of messages. It then allows us to define a set of operations on the "messages" we just defined in the request/response format. Dive deeper into this subject here.

Code-Generators

We will be using code generators in our construction process to generate some of the boiler plate code of our application. This feature allows us to spend our time sewing together the services' business logic as opposed to low-level API implementation logic that should be automatically taken care off. We will be using Prototool in this first part and gqlgen in our second part.

ArangoDB

This a NOSQL database built for high availability and high scalability, a perfect fit for implementing persistence in microservices. ArangoDB is an open source native multi-model database that supports graph, document and key-value data models allowing users to freely combine all data models in a single query. Dive deeper into this database and its features here.

Lets start ....

First things first, lets make sure all our code generators are in place and functional,

prototool instalation:

Prototool primarily provides a centralized location of mantaining consistency across an applications Protobuf files especially in large applications.
For linux users,

curl -sSL \
https://github.com/uber/prototool/releases/download/v1.8.0/prototool-$(uname -s)-$(uname -m) \
  -o /usr/local/bin/prototool && \
  chmod +x /usr/local/bin/prototool

or simply download prototool-Linux-x86_64.tar.gz and then extract the binary file placing it in the folder /usr/local/bin to make it executable

Check the version to confirm success of installation:

mykmyk@skynet:~$ prototool version
Version:                 1.9.0
Default protoc version:  3.8.0
Go version:              go1.12.4
OS/Arch:                 linux/amd64

Dive deeper into the prototool subject here.

Now lets install protoc...

The full name is protobuf compiler, its used to compile .proto file. Dive deeper here.

For linux users:

$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28


$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2

update the PATH environment var to aid the compiler locate the above plugin "protoc-gen-go-grpc"

$ export PATH="$PATH:$(go env GOPATH)/bin"

GOPATH must be a root folder for the binary files located in $GOPATH/bin and must be the root folder for projects.
To confirm the success of the above installation, use the following command:

mykmyk@skynet:~$ protoc --version
libprotoc 3.15.8

Project Architecture

The finished project will be made up of three services : two microservices that we will construct in this first part and a graphql gateway service that we will work on in the second part of this series.
The gateway is intended to be the ingress of all the client requests associated with Bikes and their Rentees, and the two services will store their own respective data on ArangoDB. When a Rentee rents a Bike, the Rentee and the Bike are associated with the Bike's ID. Based on this link we can retrieve the Rentees' ID by using the Bike's ID.

Project File Structure

create a root folder "bikerenting" with the subfolders,

gen to hold our generated Go code,
Proto to hold our proto files ,
bikes to hold our implementation of bikes API,
rentees to hold our implementation of the rentees API and - graph_api to house our graphl-gateway implementaion
dockerfiles for containerization purposes.


mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ ll
total 32
drwxrwxr-x  8 mykmyk mykmyk 4096 Jun  1 23:18 ./
drwxrwxr-x 24 mykmyk mykmyk 4096 Jun  1 23:16 ../
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 bikes/
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 db/
-rw-rw-r--  1 mykmyk mykmyk    0 Jun  1 23:18 docker-compose.yml
-rw-rw-r--  1 mykmyk mykmyk    0 Jun  1 23:18 Dockerfile.dev
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 gen/
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 graph_api/
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 proto/
drwxrwxr-x  2 mykmyk mykmyk 4096 Jun  1 23:17 rentees/

We next want to generate a prototool.yaml config file using the prototool code generation tool that we installed at the inception of this project. To create a prototool.yaml file in the projects root folder lets use the following command:

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ prototool config init

you will notice that a "prototool.yaml" file will be generated,

  1 protoc:                                                                     
  2   version: 3.8.0
  3 lint:
  4   group: uber2

now add the following information;

1 protoc:
  2   version: 3.8.0
  3 lint:
  4   group: uber2
  5 
  6 generate:                                                                       
  7   go_options:                                                                   
  8     import_path: bikerenting
  9   plugins:
 10     - name: go
 11       type: go
 12       flags: plugins=grpc
 13       output: gen/go

We are simply giving paths to locate/store our generated Go code in the above code.

Create Our Bikes and Rentees API

We are going to generate boiler plate code, i.e client and server stubs for both APIs. What we need to do first is define our data structures and operations on these data structures on our proto files for both APIs.

Lets "cd" into the root proto folder that we had created above and create two new folders to hold our respective proto files, bikes folder and rentees folder, it will look as below.

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto$ ll
total 16
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun  1 23:44 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun  1 23:36 ../
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  1 23:44 bikes/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  1 23:44 rentees/

now move into the "bikes" folder and create the following files;

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto/bikes$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  1 23:46 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun  1 23:44 ../
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  1 23:46 bikes_messages.proto
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  1 23:46 bikes.proto

open up the "bikes_messages.proto" and add the following code:

  1 syntax = "proto3";
  2 
  3 package bikerenting.grpc.bikes.v1;
  4 option go_package = "bikes";
  5 
  6 // Bike definition                                                          
  7 
  8 message Bike {
  9   string id     = 1;
 10   string owner_name = 2;
 11   string type   = 3;
 12   string make   = 4;
 13   string serial = 5;
 14  }

Then open up the file "bikes.proto" and add the code below:

syntax = "proto3";
package bikerenting.grpc.bikes.v1;
option go_package = "bikes";
import "proto/bikes/bikes_messages.proto";


//API for managing bikes
service BikesAPI {
    //Get all bikes
    rpc ListBikes(ListBikesRequest) returns (ListBikesResponse);
    //Get bike by id
    rpc GetBike(GetBikeRequest)     returns (GetBikeResponse);
    //Get bikes by ids
    rpc GetBikes(GetBikesRequest)   returns (GetBikesResponse);
    // Get bikes by type
    rpc GetBikesByTYPE(GetBikesByTYPERequest) returns (GetBikesByTYPEResponse);
    // Get bikes by make
    rpc GetBikesByMAKE(GetBikesByMAKERequest) returns (GetBikesByMAKEResponse);
    // Get bikes by owner_name
    rpc GetBikesByOWNER(GetBikesByOWNERRequest) returns (GetBikesByOWNERResponse);
    // Add new bike
    rpc AddBike(AddBikeRequest)     returns  (AddBikeResponse);
    // Delete bike
    rpc DeleteBike(DeleteBikeRequest) returns (DeleteBikeResponse);
  }

  message ListBikesRequest {
  }
  message ListBikesResponse {
     repeated Bike bikes = 1;
  }

  message GetBikeRequest {
    string id = 1;
  }
  message GetBikeResponse {
    Bike bike = 1;
  }

  message GetBikesRequest {
    repeated string ids = 1;
  }
  message GetBikesResponse {
    repeated Bike bikes = 1;
  }

  message GetBikesByTYPERequest {
    string type = 1;
  }
  message GetBikeByTYPEResponse {
    repeated Bike bikes = 1;
  }

  message GetBikesByMAKERequest {
    string make = 1;
  }
  message GetBikesByMAKEResponse {
    repeated Bike bikes = 1;
  }

  message GetBikesByOWNERRequest {
    string owner_name = 1;
  }
  message GetBikesByOWNERResponse {
    repeated Bike bikes = 1;
  }

  message AddBikeRequest {
    Bike bike = 1;
  }
  message AddBikeResponse {
    Bike bike = 1;
  }

  message DeleteBikeRequest {
    string id = 1;
  }

  message DeleteBikeResponse { 
  }

After defining the two file, one to hold our message structure and another to define operations on our structure.
Next run the commands "go mod init" and "go mod tidy" on your terminal, while still in the bikes folder.

In our code above

;
Proto3; (stands for protocol buffers version 3)specifies what protocol we are using for our API definition, the code generator will then use this version to interpret our defined messages and operations.

option go_package = "bikes"; - In this case we are locally specifying the Go import path in this .proto file.

package package bikerenting.grpc.bikes.v1; - this enable the generator to place generated bikes server-client stub code in this location.

For our rentees, the code will take a similar structure, open the proto/rentees folder and create the following files.

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto/rentees$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 00:45 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun  1 23:44 ../
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 00:45 rentees_messages.proto
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 00:45 rentees.proto

The file "rentees_messages.proto" will look like:

  1 syntax = "proto3";                                                          
  2 package bikerenting.grpc.rentees.v1;
  3 option go_package ="rentees";
  4 
  5 // Customer definition
  6 message Rentee {
  7   string id         = 1;
  8   string first_name = 2;
  9   string last_name  = 3;
 10   string National_Id_Number = 4;
 11   string phone      = 5;
 12   string email      = 6;
 13   repeated string held_bikes = 7;
 14 }

The "rentee.proto" file will look as below:

syntax = "proto3";
package tutorial.grpc.rentees.v1;
option go_package = "rentees";
import "proto/rentees/rentees_messages.proto";

//API for managing rentees
service RenteesAPI {
    //Get all rentees
    rpc ListRentees(ListRenteesRequest) returns (ListRenteesResponse);
    // Get rentee by bike id
    rpc GetRenteeByBikeId(GetRenteeByBikeIdRequest) returns (GetRenteeByBikeIdResponse);
    // Get rentee by bike type
    rpc GetRenteesByBikeTYPE(GetRenteesByBikeTYPERequest) returns (GetRenteesByBikeTYPEResponse);
    // Get rentee by bike make
    rpc GetRenteesByBikeMAKE(GetRenteesByBikeMAKERequest) returns (GetRenteesByBikeMAKEResponse);
    // Get rentee by bike owner
    rpc GetRenteesByBikeOWNER(GetRenteesByBikeOWNERRequest) returns (GetRenteesByBikeOWNERResponse);   
    // Get rentee by id
    rpc GetRentee(GetRenteeRequest) returns (GetRenteeResponse);
    // Add new rentee
    rpc AddRentee(AddRenteeRequest) returns (AddRenteeResponse);
    // Update rentee
    rpc UpdateRentee(UpdateRenteeRequest) returns (UpdateRenteeResponse);
}

message ListRenteesRequest {
}

message ListRenteesResponse {
  repeated Rentee rentees = 1;
}

message GetRenteeByBikeIdRequest{
  string id = 1;
}
message GetRenteeByBikeIdResponse {
  Rentee rentee = 1;
}

message GetRenteesByBikeTYPERequest{
  string type = 1;
}
message GetRenteeByBikeTYPEResponse {
  repeated Rentee rentees = 1;
}

message GetRenteeByBikeMAKERequest{
  string make = 1;
}
message GetRenteeByBikeMAKEResponse {
  repeated Rentee rentees = 1;
}

message GetRenteeByBikeOWNERRequest{
  string owner_name = 1;
}
message GetRenteeByBikeOWNERResponse {
  repeated Rentee rentees = 1;
}

message GetRenteeRequest {
  string id = 1 ;
}
message GetRenteeResponse {
  Rentee rentee = 1;
}

message AddRenteeRequest {
  Rentee rentee = 1;
}
message AddRenteeResponse {
  Rentee rentee = 1;
}

message UpdateRenteeRequest {
  Rentee rentee = 1;
}
message UpdateRenteeResponse {
  Rentee rentee = 1;
}

After defining our rentees and bikes APIs our our .proto files,Run the commands "go mod init" and then run "go mod tidy". we will now genate code for the respective client-server stubs using our prototool:
At our root folder i.e "bikerenting" where our prototool.yaml file is located,lets run the following command:

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ prototool generate

Prototool will find all our protofiles and use the prototool.yaml configuration to stub our client-server service interfaces in the path that we had directed those files to be deposited as per our configurations in the prototool.yaml file. Now if you navigate to the folder "gen/go/proto". We find autogenerated support modules , the shell for the API servers and client is ready and thus we can the concentrate of sewing together our business logic or controller (or view is Django) for our two APIs,
The folder "gen/go/proto" looks like :

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto$ ll
total 16
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun  2 01:28 ./
drwxr--r-- 3 mykmyk mykmyk 4096 Jun  2 01:28 ../
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 01:28 bikes/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 01:28 rentees/

Now if we "cd" into bikes folder we find the structure below:

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto/bikes$ ll
total 40
drwxrwxr-x 2 mykmyk mykmyk  4096 Jun  2 01:28 ./
drwxrwxr-x 4 mykmyk mykmyk  4096 Jun  2 01:28 ../
-rw-rw-r-- 1 mykmyk mykmyk  2728 Jun  2 01:28 bikes_messages.pb.go
-rw-rw-r-- 1 mykmyk mykmyk 25373 Jun  2 01:28 bikes.pb.go

if we "cd" into the rentees folder, we find the following structure:

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto/rentees$ ll
total 40
drwxrwxr-x 2 mykmyk mykmyk  4096 Jun  2 01:28 ./
drwxrwxr-x 4 mykmyk mykmyk  4096 Jun  2 01:28 ../
-rw-rw-r-- 1 mykmyk mykmyk  3637 Jun  2 01:28 rentees_messages.pb.go
-rw-rw-r-- 1 mykmyk mykmyk 27424 Jun  2 01:28 rentees.pb.go

You can look inside the file to check the serialization that has been done by grpc to our messages:
Make sure you run the commands , "go mod init" and "go mod tidy", in both the rentees folder and the bikes folder.

Creating database connection.....

Download and install the Arangodb database , the official page here has an easy to follow instruction set on that whole process(Download the community edition).

PS: if you run into the error "

Setting up arangodb3 (3.2.10) ... FATAL ERROR: EXIT_FAILED - "exit with error" dpkg: error processing package arangodb3 (--configure): subprocess installed post-installation script returned error exit status 1 Errors were encountered while processing: arangodb3 E: Sub-process /usr/bin/dpkg returned an error code (1)

check out my answer on this error on stack-overflow here

To confirm if the database is correctly installed , use the command "arangosh" as below:

mykmyk@skynet:~$ arangosh
Please specify a password: 

                                       _     
  __ _ _ __ __ _ _ __   __ _  ___  ___| |__  
 / _` | '__/ _` | '_ \ / _` |/ _ \/ __| '_ \ 
| (_| | | | (_| | | | | (_| | (_) \__ \ | | |
 \__,_|_|  \__,_|_| |_|\__, |\___/|___/_| |_|
                       |___/                 

arangosh (ArangoDB 3.3.0 [linux] 64bit, using jemalloc, VPack 0.1.30, RocksDB 5.6.0, ICU 58.1, V8 5.7.492.77, OpenSSL 1.1.0f  25 May 2017)
Copyright (c) ArangoDB GmbH

Pretty printing values.
Connected to ArangoDB 'http+tcp://127.0.0.1:8529' version: 3.3.0 [server], database: '_system', username: 'root'

Please note that a new minor version '3.7.11' is available
Type 'tutorial' for a tutorial or 'help' to see common examples
127.0.0.1:8529@_system>

create a new database and user on the arangosh shell as follows:

127.0.0.1:8529@_system> db._createDatabase("bikesrentees_db");
true

127.0.0.1:8529@_system> var users = require("@arangodb/users");

127.0.0.1:8529@_system> users.save("root@bikesrentees","rootpassword");
{ 
  "user" : "root@bikesrentees", 
  "active" : true, 
  "extra" : { 
  }, 
  "code" : 201 
}

127.0.0.1:8529@_system> users.grantDatabase("root@bikesrentees", "bikesrentees_db");

127.0.0.1:8529@_system>

After creating database and user, keep the credentials and details and from our root folder navigate to the "db" folder, in the db folder that stands for database , create two files , connect.go and query.go,
That folder will now look like :

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/db$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 02:06 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun  1 23:36 ../
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 02:06 connect.go
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 02:06 query.go

The connect.go file will contain code that we will invoke when we want to establish a connection to our arangodb database that we just created;

The code will look like below, notice we plug in databasename, databaseuser etc:

package db

import (
    "context"
    "fmt"
    "log"
    "os"
    "github.com/joho/godotenv"
    "github.com/arangodb/go-driver"
    "github.com/arangodb/go-driver/http"
)

func parseEnvVars(key string) string {
    //load .env file
    err := godotenv.Load(".env")

    if err != nil {
       log.Fatalf("Error loading .env file")
    }

    return os.Getenv(key)
}


const (
    DbHost     = "http://127.0.0.1"
    DbPort     = "8529"
    DbUserName = "root@bikesrentees_db"
    DbPassword = "rootpassword"
)


type DatabaseConfig struct {
    Host     string
    Port     string
    Username string
    Password string
    DatabaseName string
}


func Connect(ctx context.Context, config DatabaseConfig)(db driver.Database, err error){
    conn, err := http.NewConnection(http.ConnectionConfig{
        Endpoints: []string{fmt.Sprintf("%s:%s", config.Host, config.Port)},
    })
    if err != nil {
        return nil, err
    }
    cl, err := driver.NewClient(driver.ClientConfig{
        Connection: conn,
        Authentication: driver.BasicAuthentication(config.Username, config.Password),
    })

    if err != nil {
        return nil, err
    }

    db, err = cl.Database(ctx, config.DatabaseName)
    if driver.IsNotFound(err) {
        db, err = cl.CreateDatabase(ctx, config.DatabaseName, nil)
    }
    return db, err
}


func AttachCollection(ctx context.Context, db driver.Database, colName string)(driver.Collection, error){
    col, err := db.Collection(ctx, colName)
    if err != nil {
        if driver.IsNotFound(err){
            col, err = db.CreateCollection(ctx, colName, nil)
        }
    }
    return col, err
}

func GetDbConfig() DatabaseConfig{
    dbName := parseEnvVars("ARANGODB_DB")
    if dbName == "" {
        log.Fatalf("Failed to load environment variable '%s'", "ARANGODB_DB")
    }
    return DatabaseConfig {
        Host:     DbHost,
        Port:     DbPort,
        Username: DbUserName,
        Password: DbPassword,
        DatabaseName: dbName,
    }
}

Next open up our query.go file and add the following code

package db

import "fmt"

func ListRecords(collectionName string) string {
    const listRecordsQuery =   `
    FOR record IN %s
        RETURN record`
    return fmt.Sprintf(listRecordsQuery, collectionName)
}

Remember to run the now monotonous command pair or "go mod init" and "go mod tidy " in the db folder.
Now that we have our database connection and query logic, lets finally create our rentees and bikes servers.
From the root of the project folder head to the bikes folder, create a folder called server and a file main.go, the folder will look like below:

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/bikes$ ll
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun  2 02:19 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun  1 23:36 ../
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 02:19 main.go
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 02:19 server/

"cd" into the just created server folder and create a file called "server.go"

Open the server.go using your favorite text editor and add the following code:

package server


import (
    "context"
    "fmt"
    "log"
    "net"
    "os"
    "github.com/myk4040okothogodo/bikerenting/db"
    bikesv1 "github.com/myk4040okothogodo/bikerenting/gen/go/proto/bikes"
    "github.com/arangodb/go-driver"
    "google.golang.org/grpc"
    "google.golang.org/grpc/reflection"
)

const (
    bikesCollectionName = "Bikes"
    defaultPort         = "60001"
)

type Server struct {
    database             driver.Database
    bikesCollection      driver.Collection
}

func NewServer(ctx context.Context, database driver.Database)(*Server, error) {
    collection, err := db.AttachCollection(ctx, database, bikesCollectionName)
    if err != nil {
        return nil, err
    }

    _, _, err = collection.EnsurePersistentIndex(ctx, []string{"serial"}, &driver.EnsurePersistentIndexOptions{Unique: true})
    if err != nil {
        return nil, err
    }

    return &Server{
        database:        database,
        bikesCollection: collection,
    }, nil
}


func (s *Server) Run() {
    port := os.Getenv("APP_PORT")
    if port == "" {
        port = defaultPort
    }

    listener,err := net.Listen("tcp", fmt.Sprintf("0.0.0.0:%s", port))
    if err != nil {
        log.Fatal("net.Listen failed")
        return
    }

    grpcServer := grpc.NewServer()

    bikesv1.RegisterBikesAPIServer(grpcServer, s)
    reflection.Register(grpcServer)

    log.Printf("Starting Rental Bikes server on port %s", port)

    go func() {
        grpcServer.Serve(listener)
    }()
}


func (s *Server) ListBikes(ctx context.Context, in *bikesv1.ListBikesRequest)(*bikesv1.ListBikesResponse, error){
    if in == nil {
        return nil,  fmt.Errorf("Request is empty")
    }

    cursor, err := s.database.Query(ctx, db.ListRecords(s.bikesCollection.Name()), nil)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over documents: %s", err)
    }
    defer cursor.Close()

    allBikes := []*bikesv1.Bike{}
    for {
        bike := new(bikesv1.Bike)
        var meta driver.DocumentMeta
        meta, err := cursor.ReadDocument(ctx, bike)
        if driver.IsNoMoreDocuments(err){
            break
        } else if err != nil {
            return nil, fmt.Errorf("Failed to read bike document: %s", err)
        }
        bike.Id = meta.Key
        allBikes = append(allBikes, bike)
    }
    return &bikesv1.ListBikesResponse{Bikes: allBikes}, nil
}


func (s *Server) GetBike(ctx context.Context, in *bikesv1.GetBikeRequest)(*bikesv1.GetBikeResponse, error){
    if in == nil || in.Id == "" {
        return nil, fmt.Errorf("Bike id is not provided")
    }

    bike := new(bikesv1.Bike)
    meta, err := s.bikesCollection.ReadDocument(ctx, in.Id, bike)
    if err != nil {
        if driver.IsNotFound(err){
            err = fmt.Errorf("Bike with id '%s' not found", in.Id)

        } else {
            err = fmt.Errorf("Failed to get bike with id '%s'", in.Id, err)
        }
        return nil, err
    }
    bike.Id = meta.Key
    return &bikesv1.GetBikeResponse{Bike: bike}, nil
}


func (s *Server) GetBikes(ctx context.Context, in *bikesv1.GetBikesRequest) (*bikesv1.GetBikesResponse, error){
    if in == nil || len(in.Ids) == 0 {
        return nil, fmt.Errorf("Bikes ids are not provided")
    }

    const queryBikesByIds = `
    FOR bike IN %s
        FILTER bike._key in @ids
            RETURN bike`

    query := fmt.Sprintf(queryBikesByIds, bikesCollectionName)
    bindVars := map[string]interface{}{"ids":in.Ids}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over bike docments with query '%s': %s", queryBikesByIds, err)
    }

    defer cursor.Close()

    bikes := []*bikesv1.Bike{}

    for {
        bike := new(bikesv1.Bike)
        meta, err := cursor.ReadDocument(ctx, bike)
        if driver.IsNoMoreDocuments(err) {
            break
        } else if err != nil {
            log.Print(err)
            return nil, fmt.Errorf("Failed to read bike document: %s", err)
        }

        bike.Id = meta.Key
        bikes = append(bikes, bike)
    }
    return &bikesv1.GetBikesResponse{Bikes: bikes}, nil
}

func (s *Server) GetBikesByTYPE(ctx context.Context, in *bikesv1.GetBikesByTYPERequest)(*bikesv1.GetBikesByTYPEResponse, error){
    if in == nil || in.Type == "" {
        return nil, fmt.Errorf("Bike type is not provided")
    }

    const queryBikeByTYPE = `
    FOR bike IN %s
        FILTER bike.type == @type
            RETURN bike`
    query := fmt.Sprintf(queryBikeByTYPE, bikesCollectionName)
    bindVars := map[string]interface{}{"type": in.Type}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByTYPE, err)
    }
    defer cursor.Close()

   bikes := []*bikesv1.Bike{}
   for {
     bike := new(bikesv1.Bike)
     meta, err := cursor.ReadDocument(ctx, bike)
     if driver.IsNoMoreDocuments(err){
       break
     } else if err != nil {
         log.Print(err)
         return nil, fmt.Errorf("failed to read rentees document: %s", err)
     }
     bike.Id = meta.Key
     bikes = append(bikes, bike)
   }
   return &bikesv1.GetBikesByTYPEResponse{Bikes: bikes}, nil
}


func (s *Server) GetBikesByOWNER(ctx context.Context, in *bikesv1.GetBikesByOWNERRequest)(*bikesv1.GetBikesByOWNERResponse, error){
    if in == nil || in.OwnerName == "" {
        return nil, fmt.Errorf("Bike owner is not provided")
    }

    const queryBikeByOWNER = `
    FOR bike IN %s
        FILTER bike.owner_name == @ownername
            RETURN bike`
    query := fmt.Sprintf(queryBikeByOWNER, bikesCollectionName)
    bindVars := map[string]interface{}{"owner_name": in.OwnerName}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByOWNER, err)
    }
    defer cursor.Close()

     bikes := []*bikesv1.Bike{}
   for {
     bike := new(bikesv1.Bike)
     meta, err := cursor.ReadDocument(ctx, bike)
     if driver.IsNoMoreDocuments(err){
       break
     } else if err != nil {
         log.Print(err)
         return nil, fmt.Errorf("failed to read rentees document: %s", err)
     }
     bike.Id = meta.Key
     bikes = append(bikes, bike)
   }
   return &bikesv1.GetBikesByOWNERResponse{Bikes: bikes}, nil
}


func (s *Server) GetBikesByMAKE(ctx context.Context, in *bikesv1.GetBikesByMAKERequest)(*bikesv1.GetBikesByMAKEResponse, error){
    if in == nil || in.Make == "" {
        return nil, fmt.Errorf("Bike make is not provided")
    }

    const queryBikeByMAKE = `
    FOR bike IN %s
        FILTER bike.make == @make
            RETURN bike`
    query := fmt.Sprintf(queryBikeByMAKE, bikesCollectionName)
    bindVars := map[string]interface{}{"make": in.Make}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByMAKE, err)
    }
    defer cursor.Close()

     bikes := []*bikesv1.Bike{}
     for {
     bike := new(bikesv1.Bike)
     meta, err := cursor.ReadDocument(ctx, bike)
     if driver.IsNoMoreDocuments(err){
       break
     } else if err != nil {
         log.Print(err)
         return nil, fmt.Errorf("failed to read rentees document: %s", err)
     }
     bike.Id = meta.Key
     bikes = append(bikes, bike)
   }
   return &bikesv1.GetBikesByMAKEResponse{Bikes: bikes}, nil
}

//func (s *Server) GetBikeBySERIAL(ctx context.Context, in *bikesv1.GetBikesSERIALRequest)(*bikesv1.GetBikesBySERIALResponse, error){
//    if in == nil || in.Serial == "" {
//        return nil, fmt.Errorf("Bike serial is not provided")
//    }

//    const queryBikeBySERIAL = `
//    FOR bike IN %s
//        FILTER bike.serial == @serial
//            RETURN bike`
//    query := fmt.Sprintf(queryBikeBySERIAL, bikesCollectionName)
//    bindVars := map[string]interface{}{"serial": in.Serial}

//    cursor, err := s.database.Query(ctx, query, bindVars)
//    if err != nil {
//        return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeBySERIAL, err)
//    }
//    defer cursor.Close()

//    b := new(bikesv1.Bike)
//    meta, err := cursor.ReadDocument(ctx, b)
//    if driver.IsNoMoreDocuments(err){
//        return nil, fmt.Errorf("Bike with SERIAL '%s' not found: %s", in.Serial, err)
//    } else if err != nil {
//        return nil, fmt.Errorf("Failed to read bike document: %s", err)
//    }
//    b.Id = meta.Key

//    return &bikesv1.GetBikesBySERIALResponse{Bike: b}, nil
//}



func (s *Server) AddBike(ctx context.Context, in *bikesv1.AddBikeRequest)(*bikesv1.AddBikeResponse, error){
    if in == nil || in.Bike == nil {
        return nil, fmt.Errorf("Bike is not provided")
    }

    meta, err := s.bikesCollection.CreateDocument(ctx, in.Bike)
    if err != nil {
        return nil, fmt.Errorf("failed to create bike: %s", err)
    }

    in.Bike.Id = meta.Key
    return &bikesv1.AddBikeResponse{Bike: in.Bike}, nil
}

func (s *Server) DeleteBike(ctx context.Context, in *bikesv1.DeleteBikeRequest)(*bikesv1.DeleteBikeResponse, error) {
    if in == nil || in.Id == "" {
        return nil, fmt.Errorf("Bike id is not provided")
    }

    _, err := s.bikesCollection.RemoveDocument(ctx, in.Id)
    if err != nil {
        return nil, fmt.Errorf("Failed to remove existing bike: %s", err)
    }

    return &bikesv1.DeleteBikeResponse{}, nil
}
}

Run the commands "go mod init" and "go mod tidy".
All we are doing in the above code is implementing the logic for listing bikes using their ids, makes, types and serial numbers, we also implementing the logic for adding a new back to the system and removing a bike from the system, Now to invoke this server by creating a new server, go back one level "cd .." and open the "main.go" file and add the following code.

1 package main
  2 
  3 import (
  4     "context"
  5     "log"
  6     "os"
  7     "os/signal"
  8     "syscall"
  9     "time"
 10     "github.com/myk4040okothogodo/bikerenting/bikes/server"
 11     "github.com/myk4040okothogodo/bikerenting/db"
 12 )
 13 
 14                                                                                                                                                                     
 15 func main(){
 16   ctx, cancelFn := context.WithTimeout(context.Background(), time.Second*5)
 17   defer cancelFn()
 18 
 19   database, err := db.Connect(ctx, db.GetDbConfig())
 20   if err != nil {
 21       log.Fatalf("d.OpenDatabase failed with error: %s", err)
 22   }
 23 
 24   srv, err := server.NewServer(ctx, database)
 25   if err != nil {
 26       log.Fatalf("NewServer failed with error: %s", err)
 27   }
 28 
 29   srv.Run()
 30 
 31   sigChan := make(chan os.Signal, 1)
 32   signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
 33   signal := <-sigChan
 34   log.Printf("shutting down bikes server with signal: %s", signal)
 35 
 36 }

in the code above we are connecting with our database as per the configurations we supplied in the file "db/connect.go" then creating a New server as per our definition in our "server.go" and running this server, remember this server will be a Go-routine and so we are using a channel to listen for any server killing signals i.e "SIGTERM", to kill our server

We will replicate the above code to create our rentees API, Now head over to the rentees folder from the projects root folder and add the following code , The file structure will be identical to the one we just created above and so to avoid repetition i am not going to be repeating explanations.

mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/rentees$ ll
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun  2 10:49 ./
drwxrwxr-x 9 mykmyk mykmyk 4096 Jun  2 02:44 ../
-rw-rw-r-- 1 mykmyk mykmyk    0 Jun  2 10:48 main.go
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun  2 10:49 server/

for our "bikerenting/rentees/server/server.go"

package server

import  (
    "context"
    "fmt"
    "log"
    "net"
    "os"
    "github.com/myk4040okothogodo/bikerenting/db"
    renteesv1 "github.com/myk4040okothogodo/bikerenting/gen/go/proto/rentees"
    "github.com/arangodb/go-driver"
    "google.golang.org/grpc"
    "google.golang.org/grpc/reflection"

)


const (
    renteesCollectionName = "rentees"
    defaultPort           = "60002"
)


type Server struct {
    database           driver.Database
    renteesCollection  driver.Collection
}

func NewServer(ctx context.Context, database driver.Database)(*Server, error){
    collection, err := db.AttachCollection(ctx, database, renteesCollectionName)
    if err != nil {
        return nil, err
    }

    return &Server {
        database: database,
        renteesCollection: collection,
    }, nil
}

func (s *Server) Run() {
    port := os.Getenv("APP_PORT")
    if port == "" {
        port = defaultPort
    }
    listener, err := net.Listen("tcp", fmt.Sprintf("0.0.0.0: %s", port))
    if err != nil {
        log.Print("net.Listen failed")
        return
    }
    grpcServer := grpc.NewServer()
    renteesv1. RegisterRenteesAPIServer(grpcServer, s) // use autogenerated code to register the server
    reflection.Register(grpcServer)

    log.Printf("Starting Rentees server on port %s", port)
    go func() {
        grpcServer.Serve(listener)
    }()
}

func (s *Server) ListRentees(ctx context.Context, in *renteesv1.ListRenteesRequest)(*renteesv1.ListRenteesResponse, error){
    if in == nil {
        return nil, fmt.Errorf("Request is empty")
    }

    cursor, err := s.database.Query(ctx, db.ListRecords(s.renteesCollection.Name()), nil)
    if err != nil {
        return nil, fmt.Errorf("failed to iterate over documents: %s", err)
    }

    defer cursor.Close()
    allRentees := []*renteesv1.Rentee{}
    for {
        rentee := new(renteesv1.Rentee)
        var meta driver.DocumentMeta
        meta, err := cursor.ReadDocument(ctx, rentee)
        if driver.IsNoMoreDocuments(err) {
            break
        } else if err != nil {
            return nil, fmt.Errorf("Failed to read rentee document: %s", err)
        }
        rentee.Id = meta.Key
        allRentees = append(allRentees, rentee)
    }
    return &renteesv1.ListRenteesResponse{Rentees: allRentees}, nil
}


func (s *Server) GetRentee(ctx context.Context, in *renteesv1.GetRenteeRequest)(*renteesv1.GetRenteeResponse, error) {
    if in == nil || in.Id == "" {
        return nil, fmt.Errorf("Rentee id is not provided")
    }

    rentee := new(renteesv1.Rentee)
    meta, err := s.renteesCollection.ReadDocument(ctx, in.Id, rentee)
    if err != nil {
        if driver.IsNotFound(err) {
             err = fmt.Errorf("Rentee with id '%s' not found", in.Id)
        } else {
             err = fmt.Errorf("Failed to get rentee with id '%s':'%s'", in.Id, err)
        }
        return nil, err
    }
    rentee.Id = meta.Key

    return &renteesv1.GetRenteeResponse{Rentee: rentee}, nil
}


func (s *Server) GetRenteeByBikeId(ctx context.Context, in *renteesv1.GetRenteeByBikeIdRequest)(*renteesv1.GetRenteeByBikeIdResponse, error){
    if in == nil || in.Id == "" {
        return nil, fmt.Errorf("Bike id is not provided")
    }

    const queryRenteeByBikeId = `
    FOR rentee IN %s
        FOR bikeId IN rentee.held_bikes
           FILTER bikeId == @bikeId
               RETURN rentee`
    query := fmt.Sprintf(queryRenteeByBikeId, renteesCollectionName)
    bindVars := map[string]interface{}{"bikeId": in.Id}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeId, err)
    }
    defer cursor.Close()

    r := new(renteesv1.Rentee)
    meta, err := cursor.ReadDocument(ctx, r)
    if driver.IsNoMoreDocuments(err){
        return nil, fmt.Errorf("Rentee that held bike with id %s not found: %s", in.Id, err)
    } else if err != nil {
        return nil, fmt.Errorf("Failed to read rentee document: %s", err)
    }
    r.Id = meta.Key
    return &renteesv1.GetRenteeByBikeIdResponse{Rentee: r}, nil
}


func (s *Server) GetRenteesByBikeTYPE(ctx context.Context, in *renteesv1.GetRenteesByBikeTYPERequest)(*renteesv1.GetRenteesByBikeTYPEResponse, error){
    if in == nil || in.Type == " " {
        return nil, fmt.Errorf("Request is empty")
    }

    const queryRenteeByBikeTYPE = `
    FOR rentee IN %s
        FOR bikeType IN rentee.held_bikes
           FILTER bikeType == @type
               RETURN rentee`
    query := fmt.Sprintf(queryRenteeByBikeTYPE, renteesCollectionName)
    bindVars := map[string]interface{}{"bikeType": in.Type}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeTYPE, err)
    }
    defer cursor.Close()

    rentees := []*renteesv1.Rentee{}

    for {
        rentee := new(renteesv1.Rentee)
        meta, err := cursor.ReadDocument(ctx, rentee)
        if driver.IsNoMoreDocuments(err){
            break
        } else if err != nil {
            log.Print(err)
            return nil, fmt.Errorf("failed to read rentees document: %s", err)
        }
        rentee.Id = meta.Key
        rentees = append(rentees, rentee)
    }
    return &renteesv1.GetRenteesByBikeTYPEResponse{Rentees: rentees}, nil
  }

func (s *Server) GetRenteesByBikeMAKE(ctx context.Context, in *renteesv1.GetRenteesByBikeMAKERequest)(*renteesv1.GetRenteesByBikeMAKEResponse, error){
    if in == nil || in.Make == " " {
        return nil, fmt.Errorf("Request is empty")
    }

    const queryRenteeByBikeMAKE = `
    FOR rentee IN %s
        FOR bikeMake IN rentee.held_bikes
           FILTER bikeMake == @make
               RETURN rentee`
    query := fmt.Sprintf(queryRenteeByBikeMAKE, renteesCollectionName)
    bindVars := map[string]interface{}{"bikeMake": in.Make}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeMAKE, err)
    }
    defer cursor.Close()

    rentees := []*renteesv1.Rentee{}

    for {
        rentee := new(renteesv1.Rentee)
        meta, err := cursor.ReadDocument(ctx, rentee)
        if driver.IsNoMoreDocuments(err){
            break
        } else if err != nil {
            log.Print(err)
            return nil, fmt.Errorf("failed to read rentees document: %s", err)
        }
        rentee.Id = meta.Key
        rentees = append(rentees, rentee)
    }
    return &renteesv1.GetRenteesByBikeMAKEResponse{Rentees: rentees}, nil
  }

func (s *Server) GetRenteesByBikeOWNER(ctx context.Context, in *renteesv1.GetRenteesByBikeOWNERRequest)(*renteesv1.GetRenteesByBikeOWNERResponse, error){
    if in == nil || in.OwnerName == " " {
        return nil, fmt.Errorf("Request is empty")
    }

    const queryRenteeByBikeOWNER = `
    FOR rentee IN %s
        FOR bikeOwner IN rentee.held_bikes
           FILTER bikeOwner == @owner
               RETURN rentee`
    query := fmt.Sprintf(queryRenteeByBikeOWNER, renteesCollectionName)
    bindVars := map[string]interface{}{"bikeOwner": in.OwnerName}

    cursor, err := s.database.Query(ctx, query, bindVars)
    if err != nil {
        return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeOWNER, err)
    }
    defer cursor.Close()

    rentees := []*renteesv1.Rentee{}

    for {
        rentee := new(renteesv1.Rentee)
        meta, err := cursor.ReadDocument(ctx, rentee)
        if driver.IsNoMoreDocuments(err){
            break
        } else if err != nil {
            log.Print(err)
            return nil, fmt.Errorf("failed to read rentees document: %s", err)
        }
        rentee.Id = meta.Key
        rentees = append(rentees, rentee)
    }
    return &renteesv1.GetRenteesByBikeOWNERResponse{Rentees: rentees}, nil
  }

func (s *Server) AddRentee(ctx  context.Context, in *renteesv1.AddRenteeRequest) (*renteesv1.AddRenteeResponse, error) {
    if in == nil || in.Rentee == nil {
        return nil, fmt.Errorf("Rentee is not provided")
    }

    meta, err := s.renteesCollection.CreateDocument(ctx, in.Rentee)
    if err != nil {
        return nil, fmt.Errorf("Failed to create rentee: %s", err)
    }

    in.Rentee.Id = meta.Key
    return &renteesv1.AddRenteeResponse{Rentee: in.Rentee}, nil
}


func (s *Server) UpdateRentee(ctx context.Context, in *renteesv1.UpdateRenteeRequest)(*renteesv1.UpdateRenteeResponse, error){
    if in == nil || in.Rentee == nil || in.Rentee.Id == "" {
          return nil, fmt.Errorf("Existing rentee is provided")
    }

    _, err := s.renteesCollection.ReplaceDocument(ctx, in.Rentee.Id, in.Rentee)
    if err != nil {
        return nil, fmt.Errorf("Failed to update with id %s", in.Rentee.Id, err)
    }
    return &renteesv1.UpdateRenteeResponse{Rentee: in.Rentee}, nil
}

For our "bikerenting/rentees/main.go" we will add the following code to that file.

  1 package main                                                                                                                                                        
  2 
  3 import (
  4     "context"
  5     "log"
  6     "os"
  7     "os/signal"
  8     "syscall"
  9     "time"
 10     "github.com/myk4040okothogodo/bikerenting/db"
 11     "github.com/myk4040okothogodo/bikerenting/rentees/server"
 12 )
 13 
 14 
 15 func main() {
 16     ctx, cancelFn := context.WithTimeout(context.Background(), time.Second*5)
 17     defer cancelFn()
 18 
 19     database, err := db.Connect(ctx, db.GetDbConfig())
 20     if err != nil {
 21         log.Fatalf("db.OpenDatabase failed with error: %s", err)
 22     }
 23 
 24     srv, err := server.NewServer(ctx, database)
 25     if err != nil {
 26         log.Fatalf("NewServer failed with error: %s", err)
 27     }
 28 
 29     srv.Run()
 30 
 31     sigChan := make(chan os.Signal, 1)
 32     signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
 33     signal := <-sigChan
 34     log.Printf("shutting down Rentees servers with signal: %s", signal)
 35 }

Run the commands "go mod init" and "go mod tidy".

The article has grown too big i will have to create an "interlude" article before the second part, this article will majorly dwell on Writing tests for our API, and also using the grpcurl to test our endpoints. See you then.

Comparing "SLICE" in Python and Go-lang.

myk_okoth_ogodo — Sat, 21 May 2022 17:24:44 +0000

In this new series i will be talking about data structures in two of the biggest back-end languages Go-lang and Python. I intend to look at a data structure that are common to both languages, comparing their implementation, speed, methods, commonalities and differences in the two languages.
Python and Go-lang both find a heavy use cases in today's server-side application development, with python particularly shinning through its most popular frameworks, Django, Flask, Tornado . Python as a language also finds solid uses in the following spheres;

Data science, using tools like NumPY, Pandas, Matplotlib, Keras.
AI applications, using tools like TensorFlow, PyTorch, Keras.
Game development, using tools like Pygame, PyKyra, Pyglet, Kivy, Panda3D.

Go-lang on the other hand is a language built for concurrency, it is designed to take full advantage of distributed cloud architectures that are what we mostly use for our current application development and management. Some of its web development frameworks are Gin, Gorilla and Martini. I should probably write a dedicated article comparing the two languages,watch out for this in my next article .
In this first article of the series i am probing what "slice" means in python and go-lang , as we will see in greater detail below, Slice is an in-built python function that returns a slice object that we can initialize with start, end and step indices which can then be used in "slicing" or extracting contiguous elements of our sequences. In Go-lang slice is a mutable data structure that can be used to hold and manage collections of elements of a similar type. Go-lang slices are index-able and can be initialized with capacity and length properties .

Python "SLICE"

In python we have an inbuilt slice() function, that returns a slice object. The slice object in python can only be applied on python sequences.
Python sequences refer to an ordered set that restricts our access of its constituent items to the order in which we input the items. There are six primary types of sequences in python, these are;

Strings : an array of bytes representing Unicode characters
Lists : a mutable(changeable,malleable) data structure in python made up of an ordered sequence of elements.
Tuples : a non-mutable(unchangeable, un-malleable) built-in data structure made up of an ordered sequences of elements.
Byte sequences : a mutable(changeable) data structure made up of a fixed-length sequence bytes.
Byte arrays : created using bytearray(), is mutable array(sequence) of given bytes in the range 0 and 256.
Range objects : created using the range() function, its an immutable sequence of numbers starting from the given start integer to the stop integer. i.e range(start integer, stop integer, step).

The slice() function, takes three arguments,

start : The first argument is the index at which to begin the slice operation.
stop : The second argument is the index up to(but not including) which to perform the slicing,
step : The third argument is the an integer to specify the step of the slicing syntax; slice(start, end, step) if we supply three integer arguments, it defaults to slice(start, stop) when we supply two integer arguments and slice(stop) when we supply one integer argument. Example: Lets assume we have a list sequence of characters below; seq = ["a" , "b" , "c" , "d" , "e" , "f" , "g" , "h" , "i" , "j" , "k" , "l" , "m" , "n" , "o" , "p" ] x = slice(2) #slice(stop), slice from the first index up to but not including the second index, remember sequences in python are zero-indexed print(seq[x]) results in the following output ["a" , "b"] x = slice(2, 6) #slice(start, stop), slice starts from the second index up to but not including the element at index six print(seq[x]) results in the output ["c" , "d" , "e" , "f"] x = slice(2, 8, 2)#slice(start,stop,step), slice starts from the second index up to but not including the element in the eighth index, but skip every second element with each step print(seq[x]) results in the output ["c" , "e" , "g"] colon operator(:) To accomplish the above we can also use the slice operator ,a colon(:), in this case we specify our starting and ending index on either side of the colon i.e sequence[start:end], if we do not provide an end index and only provide the start index i.e sequence[start: ], the slice goes up to last index of our sequence, similarly when we do not provide the start index and only provide the end index i.e sequence[:end], the slicing starts at the first index of our index, to accomplish the above using the colon operator seq = ["a" , "b" , "c" , "d" , "e" , "f" , "g" , "h" , "i" , "j" , "k" , "l" , "m" , "n" , "o" , "p" ] print(seq[0:2]) results in the output ["a" , "b"] print(seq[:2]) results in the output ["a", '"b"] print(seq[2:]) results in the output ["c" , "d" , "e" , "f" , "g" , "h" , "i" , "j" , "k" , "l" , "m" , "n" , "o" , "p"]

print(seq[2:6])
results in the output ["c" , "d" , "e" , "f"]
print(seq[:]) reproduces a copy of the list ["a" , "b" , "c" , "d" , "e" , "f" , "g" , "h" , "i" , "j" , "k" , "l" , "m" , "n" , "o" , "p"]
note: a neat trick to reverse a python sequence using the colon operator is as below
print(seq[::-1])results in the output ["p", "o" ,"n","m","l","k","j","i","h","g","f","e","d","c","b","a"]

Go-lang "SLICE"

In Go-lang a slice refers to an array that can be dynamically sized. Slices are made to hold multiple elements and those elements should be of the same type. A slice data structure can shrink and grow as necessary or as required.
Slices have two properties:
capacity , this is the maximum holding capacity of the slice
length , this is the number of elements currently held by the slice.

syntax
var identifier []type ; where the empty square brackets is our empty slice and the "type" identifies the type of element that the slice will hold:
var myIntegerSlice []int ; creates a slice object to hold elements of type integer
var myStringSlice []string ; creates a slice object to hold elements of type string
Creating a slice using "make":
We can also use the built-in "make" function to create a go-lang slice, "make", allows us to specify the length and capacity of the slice as below;
var myIntegerSlice = make([]int, 20)
The above line of code creates a slice whose length and capacity is equal to 20 integer elements
var myStringSlice = make([]string, 20, 30)
The above line of code creates a lice whose length is 20 and capacity is 30 string type elements.
Creating a slice with initial values:
to create a slice object and initialize it with values at creation time we do the following
var myIntegerSlice = []int{100, 200, 300, 400, 500}
The above code creates a slice called "myIntegerSlice" that is supposed to hold integer values and gives it initial values of 100,200,300,400,500.
var myStringSlice = []string{"this","that","those","them"}
the above code creates a slice called "myStringSlice" that is supposed to hold string values and gives it initial values of , "this","that","those","them"
Appending to a slice:
We can add an item to end of a go-lang slice using the append keyword just like in python where we can also use the same append keyword;
seq := make([]int , 5, 10) // a slice called seq to hold int elements ,with length 5 and capacity 10.
to insert values ;

seq[0] = 10
seq[1] = 20
seq[2] = 30
seq[3] = 40
seq[4] = 50

To append values :
seq = append(seq, 60,70,80,90,100)
Accessing elements of a slice:
We can access the elements of a slice by using indexes of the elements, for example in our sequence "seq" above:
fmt.Println(seq[0]) results in the output 10, fmt.Println(seq[1]) is 20, fmt.Println(seq[2]) is 30 and so on.
Changing the elements in a slice:
In this case we simply use the index to access the element position and change the value held in that position, i.e in our slice seq above;
seq[1] = 200
the seq is now [10, 200, 30, 40,50, 60,70, 80, 90, 100]
Removing an element from a slice:
Using our sequence "seq" above we can remove an element from an index in the sequence as below:
we create our own "removeElemfunction";

func removeElem(seq []int, index int) []int {
return append(seq[:index], seq[index+1:]…]
}

then in our func main;

func main() {
var myIntegerSlice = []int{10,20,30,40,50}
fmt.Println(myIntegerSlice) // prints out [10,20,30,40,50]
myIntegerSlice = removeElem( myIntegerSlice, 3)
fmt.Println(myIntegerSlice) // prints out [10,20, 30,50]
}

"Slicing" a Slice in Go-lang.
Similar to python's slicing of sequences , we can perform a slicing of go-lang slice data type. we use the colon operator in a much similar way to the python slicing discussed above.
For example, lets create a full example that can run, use the go-lang playground to test it.

package main
import "fmt"
func main() {
var bodyParts = []string{"eyes","ears","skin","mouth","nose","hair"}
fmt.Printf("body parts: %v\n", bodyParts)
fmt.Printf(":3 %v\n", bodyParts[:3])
fmt.Printf("2: %v\n", bodyParts[2:])
fmt.Printf("1:3 %v\n", bodyParts[1:3])
fmt.Printf("last body part: %v\n", bodyParts[5])
fmt.Printf("last body part: %v\n", bodyParts[len(bodyParts)-1])
fmt.Printf("copy of the bodyParts sequence : %v\n", bodyParts[:])
}

The results in order of printing are;
body parts : [eyes, ears,skin,mouth,nose,hair]
:3 [eyes,ears,skin]
1:3 [ears,skin]
last body part: hair
copy of the bodyParts sequence: [eyes, ears,skin,mouth,nose,hair]
Similarities between go-lang slice and python list;
1.len

We can get the len of the sequnce using the "len" keyword,
i.e
k = [10,20,30,40,50] //python list var v = []int{10,20,30,40,50} //golang slice
to get the length of both sequences we ; len(k) in python and len(v)
2. makecopy
Still using the sequences created above in python and go-lang, to make copies using the colon operator;
Kcopy = k[:] // in python
var Vcopy = v[:] // in golang
3. append
Still on the sequences created we can add an element at the end of the sequence i.e;
k.append(60) // in python k is now [10,20,30,40,50,60] v = append(v , 60) // in golang

Conclusion

As i have demonstrated above python slice is a built in function that returns a slice object that we can then used to "slice" our sequences i.e lists, strings and tuples, while a slice in go-lang is a data type that is extensible used to implement and manage collections of data its similar to a list data type in python.
Next i will be talking about maps in go-lang and python. Goodbye, see you then.