Forem: rohit20001221

Building a Full-Stack Todo App with Esbuild, React & Golang

rohit20001221 — Sat, 15 Feb 2025 19:13:47 +0000

In this post, we explore how to build a full-stack to-do application using Go, React, Esbuild, and import maps—eliminating the need for traditional Node modules. Our setup enables an efficient workflow with server-side rendering, client-side routing, and seamless state management.

Backend: Go HTTP Server and Page Rendering

Our backend is a simple HTTP server built with Go. It serves static files, handles API requests, and dynamically renders pages using Esbuild.

Initializing the Server

The main.go file defines our HTTP server. It sets up a SQLite database for managing to-do items and builds necessary frontend files:

func init() {
    stdLibFiles, _ := filepath.Glob("ui/lib/*.jsx")
    api.Build(api.BuildOptions{
        EntryPoints:      stdLibFiles,
        Bundle:           true,
        Write:            true,
        Outdir:           ".web/lib",
        JSX:              api.JSXAutomatic,
        External:         []string{"react", "react-dom", "@mui/material", "@emotion/react", "@emotion/styled"},
        Format:           api.FormatESModule,
        MinifyWhitespace: true,
    })

    controller = *controllers.New(db.OpenConnection())

    if _, err := controller.DB.Exec(`
        CREATE TABLE IF NOT EXISTS todos (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            title TEXT,
            description TEXT,
            is_completed BOOLEAN
        )
    `); err != nil {
        panic(err)
    }
}

Serving Pages

We define routes for the home and about pages, leveraging the RenderPage function to dynamically inject data:

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    defer controllers.HandleError(w)

    todos := controller.GetTodos()
    isCode := r.URL.Query().Get("isCode") == "true"
    content := controller.RenderPage("index", map[string]any{"todos": todos}, isCode)

    w.WriteHeader(http.StatusOK)
    w.Write(content)
})

The RenderPage function compiles JSX files using Esbuild and injects context data:

func (c *Controller) RenderPage(page string, ctx map[string]any, isCode bool) []byte {
    tmpl, err := template.New("__WEB_ROOT__").Funcs(template.FuncMap{
        "marshal": marshal,
    }).Parse(HTML_PAGE)

    if err != nil {
        panic(err)
    }

    result := api.Build(api.BuildOptions{
        EntryPoints: []string{fmt.Sprintf("ui/pages/%s.jsx", page)},
        Bundle:      true,
        Write:       false,
        JSX:         api.JSXAutomatic,
        External:    []string{"react", "react-dom", "@lib", "@mui/material", "@emotion/react", "@emotion/styled"},
        Format:      api.FormatESModule,
    })

    codeBytes := result.OutputFiles[0].Contents
    code := b64.StdEncoding.EncodeToString(codeBytes)
    code = fmt.Sprintf("data:text/javascript;base64,%s", code)

    if isCode {
        return []byte(code)
    }

    buf := new(bytes.Buffer)
    args := map[string]any{
        "Page":    page,
        "Context": ctx,
        "Code":    code,
    }

    if err := tmpl.Execute(buf, args); err != nil {
        panic(err)
    }

    return buf.Bytes()
}

var HTML_PAGE = `<html>
    <head>
        <link rel="preconnect" href="https://fonts.googleapis.com" />
        <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
        <link
            rel="stylesheet"
            href="https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;500;700&display=swap"
        />
    </head>
    <body>
        <div id="root"></div>

        <script id="__WEB_ROOT_DATA__" type="json">
            {{ if .Context }}
              {{ .Context | marshal }}
            {{ else }}
                {}
            {{ end }}
        </script>

        <script type="importmap">
            {
                "imports": {
                    "react": "https://esm.sh/react@18.2.0",
                    "react/": "https://esm.sh/react@18.2.0/",
                    "react-dom": "https://esm.sh/react-dom@18.2.0",
                    "react-dom/": "https://esm.sh/react-dom@18.2.0/",
                    "@lib/": "/_lib/",
                    "@mui/material": "https://esm.sh/@mui/material?external=react,react-dom,@emotion/react,@emotion/styles&exports=Box,Typography,TextField,Button,List,ListItem,ListItemText",
                    "@emotion/react": "https://esm.sh/@emotion/react?external=react,react-dom",
                    "@emotion/styled": "https://esm.sh/@emotion/styled?external=react,react-dom"
                }
            }
        </script>

        <script type="module">
            import ui from "{{ .Code }}";
            import { createElement } from "react";
            import { createRoot } from "react-dom/client";

            globalThis.____WEB_ROOT = createRoot(
                document.getElementById("root"),
            );

            globalThis.____WEB_ROOT.render(createElement(ui));

            window.addEventListener("popstate", async (e) => {
                const code = await (
                    await fetch(`${e.target.location.pathname}?isCode=true`)
                ).text();

                const m = await import(code);
                globalThis.____WEB_ROOT.render(createElement(ui));
            });
        </script>
    </body>
</html>
`

Frontend: Import Maps, Client-Side Routing, and Context Management

Our frontend architecture follows a modular approach, using import maps to manage dependencies instead of Node modules.

Client-Side Navigation (`link.js`)

The Link component provides seamless navigation without full-page reloads:

import { createElement } from "react";

export const Link = ({ to, children }) => {
  return (
    <div
      onClick={async () => {
        const path = `${to}?isCode=true`;
        const code = await fetch(path);
        const m = await import(await code.text());

        history.pushState({}, "", to);
        const root = globalThis.____WEB_ROOT;
        root.render(createElement(m.default));
      }}
    >
      {children}
    </div>
  );
};

Managing Global State (`context.js`)

The context.js file extracts and provides server-injected data:

export const usePageContext = () => {
  return JSON.parse(document.getElementById("__WEB_ROOT_DATA__").text);
};

Home Page (`index.jsx`)

The home page displays the list of to-do items retrieved from the server:

import { useState } from "react";
import { Link } from "@lib/link.js";
import { usePageContext } from "@lib/context.js";

import {
  Button,
  Typography,
  Box,
  TextField,
  List,
  ListItem,
  ListItemText,
} from "@mui/material";

const HomePage = () => {
  const [count, setCount] = useState(0);

  const data = usePageContext();

  return (
    <Box>
      <Box
        borderBottom={2}
        borderColor={(theme) => theme.palette.divider}
        p={1}
        display={"flex"}
      >
        <Box flex={1}>
          <Typography>Dashboard</Typography>
        </Box>
        <Link to="/about">
          <Button>About Page</Button>
        </Link>
      </Box>
      <Box p={4}>
        <Typography>Todo List</Typography>

        <form method="POST" action={"/createTodo"}>
          <Box display="flex" flexDirection="column" gap={2} p={2}>
            <TextField
              type="text"
              required={true}
              name="title"
              placeholder="Title"
              variant="outlined"
            />
            <TextField
              type="text"
              required={true}
              name="description"
              placeholder="Description"
              variant="outlined"
            />
            <Box>
              <Button type="submit" variant="contained">
                Create
              </Button>
            </Box>
          </Box>
        </form>
        <Box mt={2}>
          <List>
            {data.todos.map((todo) => (
              <ListItem key={todo.id}>
                <ListItemText
                  primary={todo.title}
                  secondary={todo.description}
                />
              </ListItem>
            ))}
          </List>
        </Box>
      </Box>
    </Box>
  );
};

export default HomePage;

About Page (`about.jsx`)

A simple about page:

import { Typography, Box, Button } from "@mui/material";
import { Link } from "@lib/link.js";

const AboutPage = () => {
  return (
    <Box>
      <Typography>About Page</Typography>
      <Link to="/">
        <Button>Home</Button>
      </Link>
    </Box>
  );
};

export default AboutPage;

Conclusion

This article demonstrates how we can leverage import maps and esbuild to create a React-based full-stack application without relying on Node modules. By dynamically injecting server-side data into the frontend, we achieve a lightweight and efficient architecture. The approach outlined ensures seamless bundling, efficient routing, and a structured way to manage UI components while keeping dependencies minimal.

For the full code example, visit:
https://github.com/rohit20001221/go-esbuild-react-todo.git

Stop Using Go Templates for SQL (Until You Read This)

rohit20001221 — Wed, 05 Feb 2025 19:26:52 +0000

I initially considered using Go's text/template package to improve productivity when developing backend APIs by dynamically generating SQL queries. However, I discovered that this approach introduces a significant security risk—SQL injection—if not handled correctly.

Example of a Vulnerable SQL Query Using text/template

package main

import (
    "os"
    "text/template"
)

const queryTemplate = `SELECT * FROM users WHERE username = '{{.Username}}';`

func main() {
    tmpl, err := template.New("sql").Parse(queryTemplate)
    if err != nil {
        panic(err)
    }

    data := map[string]string{
        "Username": "admin' OR '1'='1", // Malicious input
    }

    tmpl.Execute(os.Stdout, data)
}

Output:

SELECT * FROM users WHERE username = 'admin' OR '1'='1';

Why is this Vulnerable?

Since the query is built using string interpolation, an attacker can manipulate the input ("admin' OR '1'='1") to bypass authentication or retrieve unauthorized data. If executed, this query would return all users instead of just the intended one.

A Secure Approach

To address this issue, I have developed a utility package that enables secure and dynamic SQL query generation using Go templates while ensuring protection against SQL injection.

Installation
You can install the package using:

go get github.com/rabbit-backend/template

Using the Execute Method for Safe Query Generation

The Execute method transforms SQL queries into parameterized queries, making them safe from SQL injection.

package main

import (
    "fmt"

    engine "github.com/rabbit-backend/template"
)

func main() {
    query, args := engine.Execute(
        "test/app.sql",
        map[string]map[string]string{
            "args": {"user": "admin' OR '1'='1"},
        },
    )

    fmt.Println(query, args)
}

SQL Template (test/app.sql)

SELECT * FROM users WHERE username = {{.args.user | __sql_arg__}};

output:

SELECT * FROM users WHERE username = $1  ["admin' OR '1'='1'"]

How This Works

Unlike raw string substitution, the Execute method sanitizes the SQL by converting it into a parameterized query. The query and arguments can then be passed to sql/db for safe execution against your database, preventing SQL injection attacks.

you can find the source code of the package at
https://github.com/rabbit-backend/template

Deploying Flask app to raspberry kubernetes cluster Part - 2

rohit20001221 — Sat, 19 Aug 2023 08:48:35 +0000

before proceeding i would suggest you to go through the setup of k3s in raspberry pi

https://dev.to/rohit20001221/deploying-flask-app-to-raspberry-kubernetes-cluster-part-1-environment-setup-3nc5

now we will create a simple flask app and create a docker image out of it

note:- (all the below code is written in your local system and deployed to raspberrypi)

first we will create a project folder

mkdir flask-project
cd flask-project

now inside the flask project lets create app.py and paste the following code

app.py

# app.py
from flask import Flask, render_template
import os

app = Flask(__name__)

@app.route('/')
def index():
    return render_template("index.html"), 200

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=os.environ.get("PORT", 8000))

templates/index.html

# templates/index.html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Hello World</title>
  </head>
  <body>
    <h1>This is a sample flask project.</h1>
  </body>
</html>

requirements.txt

flask

now lets create a Docker file for our project

FROM python:3-alpine

WORKDIR /app

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV PORT 80

CMD ["python", "app.py"]

this is a simple docker file which runs our flask application

now we will create a simple shell script to build and push our dockerfile to dockerhub

build.sh

docker build . -t flask-starter
docker tag flask-starter:latest <username>/flask-starter
docker push <username>/flask-starter

you can replace username with the dockerhub username

once you create the shell script run it by the following command it will push the docker image to the docker hub

sh ./build.sh

# [out]

in my case you can find the docker image at rohit20001221/flask-starter

pi@raspberrypi:~/starter-app-kubernetes-tutorial $ sudo sh build-docker.sh 
Sending build context to Docker daemon  72.19kB
Step 1/7 : FROM python:3-alpine
 ...
 ...
Successfully tagged flask-starter:latest
Using default tag: latest
The push refers to repository [docker.io/rohit20001221/flask-starter]

now lets start creating out config files for our kubernetes
create a folder called k8s in your project directory

inside the k8s folder create flask-starter.yaml

flask-starter.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: flask-starter
  labels:
    app: flask
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flask
  template:
    metadata:
      labels:
        app: flask
    spec:
      containers:
        - name: flask-starter
          image: rohit20001221/flask-starter:latest
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: flask-starter
spec:
  selector:
    app: flask
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: flask-starter-ingress
spec:
  rules:
    - host: "raspberrypi.local"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: flask-starter
                port:
                  number: 8080

the yaml file contains 3 parts

1. Deployment
the main purpose of deployments is to create pods and manage them
it has the following functionalities
1. creating replica set -> we can run multiple instances of our app
2. auto healing -> if any our our pod's go down the deployment will spin up new instances of our pod

as you can see in the above deployment we created 3 replica sets and assigned each replica set a label and also we have mentioned the docker hub image name to tell the deployment from where it can find our application code

2. Service

In simple words we can think of Service as a discovery server with the help of which we can access our application pod's

as you can see above we need to specify the app label which we created in the deployment in the service and also we have to specify the port number at which our application pods run

3. Ingress controller

Ingress controller exposes our application to the external world through a domain since the raspberry pi can be accessed through raspberrypi.local we will assign it as the host name

and we will define the service which we want to expose along with the port number of that service

by path: / we are telling the Ingress controller that what ever requests come to the following url http://raspberrypi.local/ we must send that request to our service flask-starter

now its time to deploy the application cd into the k8s folder
cd ./k8s and run the following command

kubectl apply -f flask-starter.yaml

now our flask app is deployed to the raspberry pi you can verify by SSH'ing into the pi and running the following command

# inside raspberry pi
pi@raspberrypi:~ $ sudo kubectl get pods
NAME                             READY   STATUS    RESTARTS   AGE
flask-starter-5f5bf855dc-ljctk   1/1     Running   0          6s
flask-starter-5f5bf855dc-tnzbt   1/1     Running   0          6s
flask-starter-5f5bf855dc-5p86s   1/1     Running   0          6s

now open your browser and type http://raspberry.local and you will see that our flask app has opened

Deploying Flask app to raspberry kubernetes cluster Part - 1 (Environment Setup)

rohit20001221 — Sat, 19 Aug 2023 06:17:23 +0000

recently i have bought two raspberry pi's and taught of converting it into a server to host backend code for free 😉
yaa you heard it right you can host you website or the backend api for free on raspberry pi

i will walk you through the steps required in this process

first you need a raspberry pi i have two but you can even use one for this guide and it is completely fine, and you need to install the raspberry pi debian OS i would suggest you to install the raspberry pi light 64bit os we dont need desktop verison it could create unessary load

you can find this video helpful to install raspberrypi os

now lets get into the main part !!

1. Setting up K3s in RaspberryPi

k3s is a lightweight kubernetes which can run on low end devices
with two raspberry pi's we can configure one raspberry pi as master node and other as worker node

Configuring the Master Node

SSH into raspberry pi and edit the /boot/cmdline.txt and add the following line at the end of the file

# /boot/cmdline.txt
# (append this to the end of file don't replace)
cgroup_memory=1 cgroup_enable=memory

Reboot your raspberry pi using

sudo reboot

once the raspberry pi is rebooted SSH into it again now its time to install k3s

start installing k3s using the following commands we will be using docker backend for k3s

Install docker

curl https://releases.rancher.com/install-docker/20.10.sh | sh

Install k3s with --docker option

curl -sfL https://get.k3s.io | sh -s - --docker

you can verify the k3s installation using sudo docker ps
you will get the following output

CONTAINER ID        IMAGE                     COMMAND                  CREATED              STATUS              PORTS               NAMES
3e4d34729602        897ce3c5fc8f              "entry"                  About a minute ago   Up About a minute                       k8s_lb-port-443_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
bffdc9d7a65f        rancher/klipper-lb        "entry"                  About a minute ago   Up About a minute                       k8s_lb-port-80_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
436b85c5e38d        rancher/library-traefik   "/traefik --configfi…"   About a minute ago   Up About a minute                       k8s_traefik_traefik-758cd5fc85-2wz97_kube-system_07abe831-ffd6-4206-bfa1-7c9ca4fb39e7_0
de8fded06188        rancher/pause:3.1         "/pause"                 About a minute ago   Up About a minute                       k8s_POD_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
7c6a30aeeb2f        rancher/pause:3.1         "/pause"                 About a minute ago   Up About a minute                       k8s_POD_traefik-758cd5fc85-2wz97_kube-system_07abe831-ffd6-4206-bfa1-7c9ca4fb39e7_0
ae6c58cab4a7        9d12f9848b99              "local-path-provisio…"   About a minute ago   Up About a minute                       k8s_local-path-provisioner_local-path-provisioner-6d59f47c7-lncxn_kube-system_2dbd22bf-6ad9-4bea-a73d-620c90a6c1c1_0
be1450e1a11e        9dd718864ce6              "/metrics-server"        About a minute ago   Up About a minute                       k8s_metrics-server_metrics-server-7566d596c8-9tnck_kube-system_031e74b5-e9ef-47ef-a88d-fbf3f726cbc6_0
4454d14e4d3f        c4d3d16fe508              "/coredns -conf /etc…"   About a minute ago   Up About a minute                       k8s_coredns_coredns-8655855d6-rtbnb_kube-system_d05725df-4fb1-410a-8e82-2b1c8278a6a1_0
c3675b87f96c        rancher/pause:3.1         "/pause"                 About a minute ago   Up About a minute                       k8s_POD_coredns-8655855d6-rtbnb_kube-system_d05725df-4fb1-410a-8e82-2b1c8278a6a1_0
4b1fddbe6ca6        rancher/pause:3.1         "/pause"                 About a minute ago   Up About a minute                       k8s_POD_local-path-provisioner-6d59f47c7-lncxn_kube-system_2dbd22bf-6ad9-4bea-a73d-620c90a6c1c1_0
64d3517d4a95        rancher/pause:3.1         "/pause"

now type the following command to get the master node's token we will need this to configure the other raspberry pi as worker node

sudo cat /var/lib/rancher/k3s/server/node-token

# [out]: K106e6f015b9399c74bab39bf6be3b53bf9661fbc64d6da8ab713544843399418cb::server:04cf766c678cf022a6f80d296b4125c8

now lets SSH into our other raspberry pi and configure it as the worker node

you need to follow the above step of editing the /boot/cmdline.txt even for this node and reboot the Pi

once you have done editing the /boot/cmdline.txt just install the docker with the command

curl https://releases.rancher.com/install-docker/20.10.sh | sh

2. Worker node setup

after installing the docker we will configure the worker node

curl -sfL https://get.k3s.io | K3S_URL=http://<IP_OF_MASTER_NODE>:6443 sh -s - agent --token <masternodes token> --docker

you are done with the setup process of the master and worker nodes

you can verify the setup using sudo kubectl get nodes inside the master node

now we will copy the maters node kubectl config file so that we can access it from our local machine

type the following command

sudo cat /etc/rancher/k3s/k3s.yaml

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ...
    server: https://127.0.0.1:6443  # <-- replace this with local ip
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
  user:
    client-certificate-data: ...
    client-key-data: ...

replace 127.0.0.1 with the local ip of the pi in my case it is 192.168.1.10 you can get it by typing following command ip a

now copy the config into ~/.kube/config in your local system

note: you need to install kubectl in you local system

you can follow this guide to install kubectl
https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

we are done with the setup now in part 2 we will deploy a flask application to kubernetes

part 2
https://dev.to/rohit20001221/deploying-flask-app-to-raspberry-kubernetes-cluster-part-2-3bld

Building Simple Recommendation Engine With KNN Algorithm using Redis

rohit20001221 — Sat, 11 Feb 2023 13:00:36 +0000

Hi guys today i tought of building a recommendation engine which recommends us music on giving a audio file as input

let us see how we can build this engine using redis with the help of inbuilt vector similarity search

start the redis stack server

docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

preparing the dataset

for the dataset i had downloaded the music from this channel https://www.youtube.com/@TheFatRat i had downloaded the music in .wav format

and i had create a data.json

[
    {
        "file": "1.wav",
        "link": "https://www.youtube.com/watch?v=2QdPxdcMhFQ"
    },
    {
        "file": "2.wav",
        "link": "https://www.youtube.com/watch?v=2Ax_EIb1zks"
    },
    {
        "file": "3.wav",
        "link": "https://www.youtube.com/watch?v=wgip631nFdY"
    },
    {
        "file": "4.wav",
        "link": "https://www.youtube.com/watch?v=bgyO9bNbfb8"
    },
    {
        "file": "5.wav",
        "link": "https://www.youtube.com/watch?v=gHgv19ip-0c"
    },
    {
        "file": "6.wav",
        "link": "https://www.youtube.com/watch?v=3VTkBuxU4yk"
    },
    {
        "file": "7.wav",
        "link": "https://www.youtube.com/watch?v=cJglBxApcDM"
    },
    {
        "file": "8.wav",
        "link": "https://www.youtube.com/watch?v=j-2DGYNXRx0"
    },
    {
        "file": "9.wav",
        "link": "https://www.youtube.com/watch?v=M-P4QBt-FWw"
    },
    {
        "file": "10.wav",
        "link": "https://www.youtube.com/watch?v=cMg8KaMdDYo"
    }
]

now to insert the data into redis we can run the below script

import librosa
import threading
import numpy as np
import json
import os
import uuid

dataset = json.load(open("./dataset/dataset.json", "r"))

music_data = []

def load_music_file(data):
    vector, sr = librosa.load("./dataset/" + data['file'])
    music_data.append([vector, sr, data['link']])

threads = []

for data in dataset:
    t = threading.Thread(target=load_music_file, args=[data,])
    t.start()
    threads.append(t)

for t in threads:
    t.join()

features = []

for y, sr, url in music_data:
    feature = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=200)
    features.append([np.mean(feature.T, axis=0), url])

import redis

redis_client = redis.Redis()

pipeline = redis_client.pipeline()

for feature, url in features:
    pipeline.hmset(
        "music:" + str(uuid.uuid4()),
        {"url": url, "vec": feature.astype(np.float32).tobytes()}
    )

pipeline.execute()

we are using librosa python package to preprocess the audio files and to convert the music file into a vector representation we are using mfcc https://medium.com/prathena/the-dummys-guide-to-mfcc-aceab2450fd

now if you navigate to http://localhost:8001 you can find that the music files have be added into our redis database

using redis knn command to get the recommendations

first step you need to do inorder to use knn feature of redis you must create a vector index which can be done via following command

FT.CREATE "idx:music"
    ON HASH
        PREFIX 1 "music:"
    SCHEMA
        "url" TEXT
        "vec" VECTOR HNSW
            6
            "TYPE" "FLOAT32"
            "DIM" 128    // <-- 128 because the mfcc vector has a dimension of 1 x 128
            "DISTANCE_METRIC" "COSINE"

now if you want to query the items using redis command you can use the following command

FT.SEARCH idx:music 
    "*=>[KNN $K @vec $query_vector as vector_score]"  
    "PARAMS" "4" 
        K 2                 
        "query_vector"      
            <binary_form_of_vector>
    RETURN 2 vector_score url 
    SORTBY vector_score
    DIALECT 2

but i recommend you to use python instead of the above query because it is easy you can query recommendations using the below code

y, sr = librosa.load("./example.wav")
feature = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=200)
vector = np.mean(feature.T, axis=0)
vector = vector.astype(np.float32).tobytes()

q = Query(
    f"(*)=>[KNN 3 @vec $vec_param as vector_score]"
).sort_by("vector_score").return_fields("url","vector_score").dialect(2)

params = {
    "vec_param": vector
}

results = []
query_results = redis_client.ft("idx:music").search(query=q, query_params=params)

for result in query_results.docs:
    results.append(result.url)

print(results)

i have created one flask api so that we can run this model in the browser

# app.py
import tempfile

import librosa
import numpy as np
import redis
from flask import Flask, jsonify, render_template, request
from redis.commands.search.query import Query

redis_client = redis.Redis()
app = Flask(__name__)

@app.route('/')
def index():
    return render_template("index.html")

@app.route("/get_recommendations", methods=['POST'])
def get_recommendations():
    tmp = tempfile.NamedTemporaryFile(delete=True)
    request.files.get("music_file").save(tmp)

    y, sr = librosa.load(tmp.name)
    feature = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=200)
    vector = np.mean(feature.T, axis=0)
    vector = vector.astype(np.float32).tobytes()

    q = Query(
        f"(*)=>[KNN 3 @vec $vec_param as vector_score]"
    ).sort_by("vector_score").return_fields("url", "vector_score").dialect(2)

    params = {
        "vec_param": vector
    }

    results = []
    query_results = redis_client.ft("idx:music").search(query=q, query_params=params)

    for result in query_results.docs:
        results.append(result.url)

    tmp.close()
    return jsonify({
        "results": results
    })

if __name__ == '__main__':
    app.run(debug=True)

<!--- templates/index.html --->
<script>
    async function submitForm(event) {
        event.preventDefault()

        const fd = new FormData(event.currentTarget)
        const data = await (await fetch("{{url_for('get_recommendations')}}", {
            method: 'POST',
            body: fd,
        })).json()

        document.getElementById("results").innerHTML = ""
        data["results"].forEach((url) => {
            document.getElementById("results").innerHTML += `<a href="${url}">${url}</a><br/>`
        })

        return false
    }
</script>

<form action="#" onsubmit="submitForm(event)" enctype="multipart/form-data">
    <input type="file" name="music_file" id="">
    <button type="submit">submit</button>
</form>

<div id="results"></div>

you can find the complete code in the github
https://github.com/rohit20001221/music_recommendation_engine

Crypto Bot

rohit20001221 — Sun, 28 Aug 2022 19:07:17 +0000

Overview of My Submission

Crypto Bot is a collection of micro services which uses algorithmic trading to generate the BUY / SELL Signals based on the data received from the crypto market (Binance API)

the bot uses the pubsub to publish the live crypto prices to the strategy micro service and redis json to store the orders data and the micro services communicate via gRPC with each other

custom strategies can be writen under the strategy-service/strategies

an example ema.py (Exponential Moving Average) strategy is given in the repo

Submission Category: Microservice

Language Used: Python, Protobuf

Link to Code

rohit20001221 / trading-bot-binance

Crypto Bot

crypto bot uses algorithmic trading strategies to generate buy and sell signals for trading the crypto currency

How it works

How the data is stored:

    //historical data

    {
        "open": float,
        "high": float,
        "low": float,
        "close": float,
        "volume": float
    }

    // live data

    {
        'open': float,
        'high': float,
        'low': float,
        'close': float,
        'volume': float,
        'is_interval': boolean,
        "symbol": string, // btcusdt
        "interval": string // "5m", "1m", "1h", "1d"
    }

How the data is accessed:

# subscribing to the live data service
REDIS> SUBSCRIBE <channel-name>

# reading historical data
REDIS> JSON.GET historical-<tradingsymbol> '$'

How to run it locally?

Prerequisites

python
gRPC
Redis Stack Server
Docker
Binance API Key and Secret
Pushbullet Access Token

Local installation

to start the…

View on GitHub

Additional Resources / Info

Collaborators

@rohit20001221

Check out Redis OM, client libraries for working with Redis as a multi-model database.
Use RedisInsight to visualize your data in Redis.
Sign up for a free Redis database.

Django Signals

rohit20001221 — Mon, 11 Apr 2022 09:53:59 +0000

Hi guys here i am with an other post about django signals
first of all why do we need django signals. django signals can become useful when we try to develop some event based functonality in our backend

in django we have two entities one is sender and other is receiver the sender will send a signal on any particular event and all the receivers who are listening for that event are notified with the signal and perform certain kind of actions

Signals based on Djano Models

lets see few examples by using django's inbuild signals

let's define few models in django's models.py

# models.py

from django.db import models
from django.contrib.auth.models import User
import datetime

# Create your models here.
class Profile(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE)
    profile_url = models.URLField()
    mobile = models.CharField(max_length=20, default='999-999-9999')
    funds = models.FloatField(default=2000)

    def __str__(self):
        return "[PROFILE]:" + self.user.username


class Log(models.Model):
    timestamp = models.DateTimeField(default=datetime.datetime.now)
    message = models.TextField()

    def __str__(self):
        return self.message

here we have defined two models Profile which extends the inbuilt User model by models.OneToOneField and another is Log model which is simply used to store the Log messages in database

now lets define a reveiver which uses post_save signal which is triggered after a particular model is saved into database

# models.py

# .... models defined above

from django.db.models.signals import post_save
from django.dispatch import receiver


# triggred when User object is created
@receiver(post_save, sender=User)
def create_profile(sender, instance, created, **kwargs):
    if created:
        Profile.objects.create(
            user=instance
        )

# triggred when User object is saved
@receiver(post_save, sender=User)
def log_user_saved(sender, instance, **kwargs):
    Log.objects.create(
        message=f"user {instance.username} is saved"
    )

# triggred when Profile object is saved
@receiver(post_save, sender=Profile)
def log_profile_saved(sender, instance, **kwargs):
    Log.objects.create(
        message=f"profile {instance} is saved"
    )

as you can see above we have created 3 receiver functions lets see each one of them

create_profile this receiver is triggred when the post_save signal on User model is generated i.e after the user model has been saved to the database the post_save signal is generated by User model and the create_profile receiver is triggred

in create_profile we are creating Profile for the User we can verifiy this by going to django admin and creating a user

Creating User

User Saved

Profile Created

as we can see in the above image the profile is created automatically for the user with the help of post_save signal

similarly if we go to the Log model we can see the logs created for each signal

in the above image we can see that Log objects are created when a signal on User and Profile are triggered

similar to post_save we have signals like pre_save, pre_delete, post_delete, pre_init,
post_init, m2m_changed

The receiver functions for various signals

pre_init

receiver_function(sender, *args, **kwargs)

post_init

receiver_function(sender, instance)

pre_save

receiver_function(sender, instance, raw, using, update_fields)

post_save

receiver_function(sender, instance, created, raw, using, update_fields)

pre_delete

receiver_function(sender, instance, using)

post_delete

receiver_function(sender, instance, using)

m2m_changed

receiver_function(sender, instance, action, reverse, model, pk_set, using)

Destructuring and Spread Syntax in javascript

rohit20001221 — Wed, 06 Apr 2022 08:27:19 +0000

Hi Guys today i am going to share about what i have learned about destructure and spread operator in javascript

Destructuring Syntax

first lets see about destructure operator in javascript, this operator is used to extract variables out of an array or and object

lets see the below examples to understand more clearly


// let us declare an array here
let arr = [2, 1, 3, 5];

/*
 suppose we want to store the individual numbers into
 variables we can use destructure operator 
*/

const [a] = arr;
console.log(a); // [output]: 2 

const [l, m, n] = arr;
console.log(`${l} ${m} ${n}`); // [output]: 2 1 3

const [p,q,r,s] = arr;
console.log(`${p} ${q} ${r} ${s}`); // [output]: 2 1 3 5

const [z,x,c,v,b] = arr;
console.log(b); // [output]: undefined // as there are only 5 elements in array

as you can see we can destructure any number of elements from an array if the number of variable while destructuring are more than number of elements in array then we get undefined in those extra variables


const student = {name: 'Krishna', age:21, rollNo: '21JK15'};

const {name} = student;
console.log(name); // [output]: Krishna

const {age, rollNo} = student;
console.log(`${age} ${rollNo}`); // [output]: 21 21JK15

const {marks} = student;
console.log(marks); // [output]: undefined

as you can see above the destructuring is used to even extract the properties of an object and extracting the properties not present in object gives us undefined

Spread Syntax

what i have found about this syntax is that, we can use it for passing arguments to a function or we can use it to extend an array or an object lets see few examples here

in javascript we use ... 3 dots as the spread operator

function addNums(a, b) {
   return a+b;
}

const nums = [1,2];

console.log(addNums(...nums)); // [output]: 3

in above example we can see that spread operator is used to pass arguments to a function

const arr1 = [1,2];
const arr2 = [3,4];


const arr3 = [...arr1, ...arr2]; // concat two arrays
const arr4 = [...arr1, 7, 8]; // append new elements to array at last
const arr5 = [7, 8, ...arr1]; // append to the front of array

in the above example we can see that spread operator is used to extend an array

let student = {name:'Krishna', age:21};

student = {...student, marks: 85};
console.log(student); // [output]: {name:'Krishna', age:21, marks:85}

in above example we use the spread operator to add new properties to an object

let student = {name:'Krishna', age:21};

student = {...student, age:18}; // wii modify the age to 18
console.log(student); // [output]: {name:'Krishna', age:18}

student = {name:'Krishna', age:21};

student = {age:18, ...student}; // wrong way to modify object
console.log(student); // [output]: {name:'Krishna', age: 21}

as seen above we can even modify the properties of an object using spread operator note that second method of modifing object is wrong we must first spread the object and then modify if not the other way this is the common mistake that i did many times so i am sharing with you now so that you would be careful next time

Google Authentication in Django The Simplest way

rohit20001221 — Mon, 04 Apr 2022 11:31:39 +0000

Hi guys this time lets see how we can add google authentication to our django webapp

1. Setting up the django project



django-admin startproject GoogleAuthentication

here we are creating a django project called GoogleAuthentication the name of project can be any thing literally.

2. Installing the django-allauth package

now let's install the required packages needed for adding google authentication to our web application



pip install django-allauth

3. Adding Authentication Backends in settings.py

you need to specify AUTHENTICATION_BACKENDS propery in your settings.py



# settings.py

# all the below property in your settings.py file for django allauth to work
AUTHENTICATION_BACKENDS = [
    # django's inbuild authentication backend
    'django.contrib.auth.backends.ModelBackend',

    # django's allauth authentication backend
    'allauth.account.auth_backends.AuthenticationBackend',
]

4. Adding the required Apps in settings.py

you need to add the following apps to your settings.py note that you need to add django.contrib.sites to INSTALLED_APPS if it is not present



# settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',

    'django.contrib.sites', # <-- add the sites app here

    # apps needed for django all auth
    'allauth',
    'allauth.account',
    'allauth.socialaccount',

    # add the below app for google authentication to work
    'allauth.socialaccount.providers.google',



    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

5. Adding allauth urls to the main urls.py

include the django's allauth urls into our project using the code snippet below



# main urls.py file

from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('accounts/', include('allauth.urls')), #  <- include the allauth urls here
]

6. Run the migrations

to run the migrations use the below two commands



python manage.py makemigrations



python manage.py migrate

7. Create a superuser

to create a super user use the following command



python manage.py createsuperuser

output



Username (leave blank to use 'rohit'): admin
Email address: admin@admin.com
Password: 
Password (again): 
Superuser created successfully.

8. Create Google client-id and secret

now visit google developer dashboard by visiting the following link : google api console

now in the search bar search for Gmail and under Market Place select the Gmail API

now enable the Gmail API on clicking the Enable button

now visit the credentials page and click on " + CREATE CREDENTIALS " button and click on "OAuth Client ID"

now select the application type as Web as shown below

now name the application what ever you want i am naming it as Django Google Auth but the main thing we need to keep in mind is the redirect url and javascript origin



# add the following to redirect urls

http://127.0.0.1:8000/accounts/google/login/callback/
http://localhost:8000/accounts/google/login/callback/



# add the following to javascript origins

http://127.0.0.1:8000
http://localhost:8000

after adding the above urls click on create button

copy the client id and secret which appear on the screen

9. Configure the client id and secret

in the settings.py add the following line of code



# settings.py

SITE_ID = 1

now login to the django admin page and edit the site example.com to localhost

now go to Social Applications and add the google client id and secret there

you can see above how we have added the cliend id and secret and also dont forget to add localhost to chossen sites

on visiting : http://127.0.0.1:8000/accounts/login/

you can see a link for login with google now you can login using google clicking on the link

LOGIN PAGE

GOOGLE AUTH

you can create custom login page using the below code snippet



<html>
<head>
  <!--Dont forget to replace with client id below-->
  <meta name="google-signin-client_id" content="YOUR_CLIENT_ID.apps.googleusercontent.com">
</head>
<body>
  <div id="my-signin2"></div>
  <script>
    function onSuccess(googleUser) {
      console.log('Logged in as: ' + googleUser.getBasicProfile().getName());
    }
    function onFailure(error) {
      console.log(error);
    }
    function renderButton() {
      gapi.signin2.render('my-signin2', {
        'scope': 'profile email',
        'width': 240,
        'height': 50,
        'longtitle': true,
        'theme': 'dark',
        'onsuccess': onSuccess,
        'onfailure': onFailure
      });
    }
  </script>

  <script src="https://apis.google.com/js/platform.js?onload=renderButton" async defer></script>
</body>
</html>

Simple Pub-Sub System using Redis and Python

rohit20001221 — Sun, 27 Mar 2022 11:23:06 +0000

Hi guys i am here with an another blog post now let us build a simple publish subscribe system using python and redis

Running the redis server

i am using docker to start a redis instance in my computer. it's very easy to run a redis instance using docker and also it's very easy to remove redis instance, and the fun part is redis running through docker would not affect our system 😄

running redis instance



docker run --name demo-redis -p 6379:6379 -d redis:alpine

if you want to stop or remove redis instance running through above command you can do the following



docker container stop demo-redis # <-- stop the redis instance

docker container rm demo-redis # <-- remove the redis container

Creating the publisher



import redis

# initializing the redis instance
r = redis.Redis(
    host='127.0.0.1',
    port=6379,
    decode_responses=True # <-- this will ensure that binary data is decoded
)

while True:
    message = input("Enter the message you want to send to soilders: ")

    r.publish("army-camp-1", message)

let's think publisher as a commander in army and he wants to send message to the soilders so here the the topic that he wants to publish message is army-camp-1. this analogy is only for fun 😂

Creating the subscriber



import redis

r = redis.Redis(

    host='127.0.0.1',

    port=6379,

    decode_responses=True

)

# pubsub() method creates the pubsub object

  
  
  but why i named it mobile 🧐


  
  
  just kidding 😂 think of it as the waki taki that listens for incomming messages


mobile = r.pubsub()

# use .subscribe() method to subscribe to topic on which you want to listen for messages

mobile.subscribe('army-camp-1')

# .listen() returns a generator over which you can iterate and listen for messages from publisher



for message in mobile.listen():

    print(message) # <-- you can literally do any thing with this message i am just printing it

Here is a demo video of running the above code

Creating Forms in Django

rohit20001221 — Sat, 26 Mar 2022 20:19:58 +0000

Hi guy's today i am going to show you how can we create forms in django

Step 1 : Setting up the django Project

create a django project using the following command
django-admin startproject DjangoForms

Step 2 : Creating a app in our project

fist cd into your project directory and type the following command

python manage.py startapp demo

Step 3 : Creating Forms 😄

creating a normal form

first let me show how to create a normal form (i.e this form is not connected to any of the models)

create a file called forms.py in your app folder

we will be importing the froms from django using following code
from django import forms
every form in django will inherit the following class forms.Form
for example we will create a form called ContactForm as follows
class ContactForm(forms.Form)
now lets add some fileds to our form

from django import forms
from django.core.validators import MaxLengthValidator

class ContactForm(forms.Form):
    name = forms.CharField()
    email = forms.EmailField()
    mobile = forms.CharField(
        validators=[MaxLengthValidator(10)]
    )

we can use form.TypeField for the fields Type can be anything like Char, Email, Integer,Float etc ...

and also note that we are adding MaxLengthValidator to the validators of mobile field this ensures that when the form is submitted the length of value of mobile field is less than 10

now lets create a view in views.py which will render the form

# views.py
from .forms import ContactForm

def contact_form(request):
    form = ContactForm() # <-- Initializing the form

    # sending the form to our template through context
    return render(request, "form.html", {"contactform": form})

mapping the view to the urls.py

# urls.py of the project
from django.contrib import admin
from django.urls import path
from demo.views import contact_form

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', contact_form),
]

rendering the form in form.html

there are two ways to render the form in our template

rendering using form.as_p

   <form method="POST" action="">
       {{form.as_p}}
       <button>submit</button>
   </form>

run the development server using python manage.py runserver

if you navigate to http://localhost:8000 the form is displayed as shown below

handling the data submitted by the form

from django.shortcuts import render

# Create your views here.
from .forms import ContactForm

def contact_form(request):
    form = ContactForm() # <-- Initializing the form

    if request.method == 'POST':
        form = ContactForm(request.POST) # <-- populating the form with POST DATA

        if form.is_valid():  # <-- if the form is valid
            print("[**] data received [**]")  # <-- we handle the data 
            print(form.cleaned_data['name'])
            print(form.cleaned_data['email'])
            print(form.cleaned_data['mobile'])


    # sending the form to our template through context
    return render(request, "form.html", {"contactform": form})

When the form is submitted successfully we see the data logged to terminal

# output in the terminal
March 26, 2022 - 20:06:02
Django version 3.0.7, using settings 'DjangoForms.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[**] data received [**]
blog
rohit20001221@gmail.com
9999999999

but when the data entered in the forms is incorrect the error message is displayed

we can see the mobile number is more than 10 characters long so the error message is displayed in the form

Alternate way to render a form

render each field manually

<form method="POST" action="">{% csrf_token %} 
    <div>
        {{ form.name }}

        <!---Rendering the error message--->
        {% for error in form.name.error %}
            {{error}}
        {% endfor %}
    </div>
    <div>
        {{ form.email }}

         <!---Rendering the error message--->
        {% for error in form.email.error %}
            {{error}}
        {% endfor %}
    </div>
    <div>
        {{ form.phone }}

         <!---Rendering the error message--->
        {% for error in form.phone.error %}
            {{error}}
        {% endfor %}
    </div>
    <button>submit</button>
</form>

Forem: rohit20001221

Building a Full-Stack Todo App with Esbuild, React & Golang

Backend: Go HTTP Server and Page Rendering

Initializing the Server

Serving Pages

Frontend: Import Maps, Client-Side Routing, and Context Management

Client-Side Navigation (link.js)

Managing Global State (context.js)

Home Page (index.jsx)

About Page (about.jsx)

Conclusion

Stop Using Go Templates for SQL (Until You Read This)

Deploying Flask app to raspberry kubernetes cluster Part - 2

Deploying Flask app to raspberry kubernetes cluster Part - 1 (Environment Setup)

1. Setting up K3s in RaspberryPi

Building Simple Recommendation Engine With KNN Algorithm using Redis

Crypto Bot

Overview of My Submission

Submission Category: Microservice

Language Used: Python, Protobuf

Link to Code

rohit20001221 / trading-bot-binance

Crypto Bot

How it works

How the data is stored:

How the data is accessed:

How to run it locally?

Prerequisites

Local installation

Additional Resources / Info

Collaborators

Django Signals

Signals based on Djano Models

The receiver functions for various signals

Destructuring and Spread Syntax in javascript

Destructuring Syntax

Spread Syntax

Google Authentication in Django The Simplest way

1. Setting up the django project

2. Installing the django-allauth package

3. Adding Authentication Backends in settings.py

4. Adding the required Apps in settings.py

5. Adding allauth urls to the main urls.py

6. Run the migrations

7. Create a superuser

8. Create Google client-id and secret

9. Configure the client id and secret

Simple Pub-Sub System using Redis and Python

Running the redis server

Creating the publisher

Creating the subscriber

but why i named it mobile 🧐

just kidding 😂 think of it as the waki taki that listens for incomming messages

Here is a demo video of running the above code

Creating Forms in Django

Step 1 : Setting up the django Project

Step 2 : Creating a app in our project

Step 3 : Creating Forms 😄

creating a normal form

Alternate way to render a form

Client-Side Navigation (`link.js`)

Managing Global State (`context.js`)

Home Page (`index.jsx`)

About Page (`about.jsx`)