Forem: krishna

Grafana 414 - Network error request URI too large

krishna — Thu, 24 Sep 2020 16:38:44 +0000

When your filters and eventually your GET URL is too long Grafana panel says 414-Network error request URI too large

But therez a simple fix to it.
Change your HTTP method to POST, default is GET, in your Datasource settings

Quick and Clean Nginx dashboard

krishna — Sat, 15 Aug 2020 05:43:26 +0000

Goaccess helps you to quickly put together a dashboard based on Nginx (and many other web servers) access logs.

This solution is very neat as there is no need to run any frameworks for UI, no charting library plugin etc.

Goaccess can parse the access logs, calculate the HTTP stats based on every log entry and generate a self contained .html file. As a bonus it als allows csv, json exports

Docker command to generate a on-demand one time access log report

cat access.log | docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED - > report.html

Docker command for real-time report

cat access.log | docker run -p 7890:7890 --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED --real-time-html - > report.html

There is much more to Goaccess, In this story I just wanted to share my enthusiasm in using this cool piece of OSS.

Read blocked medium stories without subscription

krishna — Mon, 29 Jun 2020 17:13:05 +0000

tl;dr — In medium.com, if a premium story is blocked like below, open the article link in browser’s private tab without logging into medium account. The article will get opened. Been using this hack for a while now. Make hay while the sun shines!

Blocked content

Same content unblocked in browser's private window

I started to not like medium after they switched from free to premium service. Users can read only 3 premium stories for free/month, common guys that’s too less, also we, the users read a title and open it if we find it interesting. We do not bother whether it is a premium story or not and before we realize the count is 3 and we cannot read more premium content. IMHO this is not fair, users contribute amazing content to the site and the business is expecting the same users to subscribe makes me feel that the business is too money-minded and I don like it.

These days I follow and write in https://dev.to more than medium.com. dev.to feels at home, with fellow developers contributing amazing content.

API versioning in golang ECHO

krishna — Mon, 29 Jun 2020 15:34:07 +0000

package main

import (
    "net/http"

    "github.com/labstack/echo"
)

func main() {
    e := echo.New() 

    v1 := e.Group("/v1")
    groupV1Routes(v1)

    v2 := e.Group("/v2")
    groupv2Routes(v2)

    e.Logger.Fatal(e.Start(":8090"))
}

func groupV1Routes(e *echo.Group) {
    grA := e.Group("/groupa")
    grA.GET("/event", handlerGet1)

    grB := e.Group("/groupb")
    grB.PATCH("/:id", handlerPost)
}

func groupv2Routes(e *echo.Group) {
    grA := e.Group("/groupa")   
    grA.GET("/event", handlerGet2)

}

func handlerGet1(c echo.Context) error {
    return c.String(http.StatusOK, "THIS IS VERSION 1!")
}

func handlerGet2(c echo.Context) error {
    return c.String(http.StatusOK, "THIS IS VERSION TWO")
}

func handlerPost(c echo.Context) error {
    return c.String(http.StatusOK, c.Param("id"))
}

Creating block diagrams from docker-compose files

krishna — Sun, 22 Mar 2020 08:56:38 +0000

A docker-compose file contains the definition of all the services that needs to run in harmony to successfully deploy an app stack.

When services and their interactions are defined clearly in a compose file manually charting a block diagram to list those services and their interaction is redundant, isn't it ? that was the thought I had when I needed to chart a block diagram for documenting my architecture.

A quick search using "tool to convert a docker compose file to a block diagram" yielded this awesome repo https://github.com/pmsipilot/docker-compose-viz

The code in this repo will help us to chart a block diagram from a docker compose file.

Lets create one!

Sample docker-compose of a simple SaaS stack

Command to execute

docker run 
--rm -it \
--name dcv \ 
-v /home/user/blog:/input pmsipilot/docker-compose-viz \
render -m image \
--force docker-compose.yml \
--output-file=topology.png \
--no-volumes \
--no-ports \
--no-networks

Ouptput

Command - With port numbers

docker run 
--rm -it \
--name dcv \ 
-v /home/user/blog:/input pmsipilot/docker-compose-viz \
render -m image \
--force docker-compose.yml \
--output-file=topology.png \
--no-volumes \
--no-networks

Ouptput - with ports
Command - With volumes

docker run 
--rm -it \
--name dcv \ 
-v /home/user/blog:/input pmsipilot/docker-compose-viz \
render -m image \
--force docker-compose.yml \
--output-file=topology.png \
--no-ports \
--no-networks

Ouptput - with volumes

Designs I like — Swiggy’s popup driver map

krishna — Tue, 17 Mar 2020 04:40:58 +0000

Who doesn’t order food online these-days, swiggy is a popular food delivery app in India. After ordering the food, I keep looking at my mobile freq, but why? becoz I am hungry mate… I need my food right away. I need to know where my delivery guy is, i would like to call him if he takes a wrong turn, tell him some shortcuts, know how long it would take for him to handover so that I can eat something else in the meanwhile ;)

I need to do all the above and then more, all the while using other apps. That’s where I find this pop-up map design very much helpful.
This is an awesome design and flexibility from android for letting app developers do this intuitive and helpful design.

Swiggy’s user empathy, in this regard, is better than Uber’s I would say.
Though this is a common design in 2019, Uber for some reason doesn’t have this. One reason would be Uber’s driver arrival is supposed to be quicker and the passenger is expected to be ready while booking the cab. Is it?

In real world, many a times Uber arrivals are easily > 15m and also there are scheduled rides. Consumers play 30s games these days to kill boredom so 15m is way too much for a ask, uber cannot argue about being quicker than swiggy. What do you think can be uber’s reason for not having this feature?

Be a developer before being a devops

krishna — Sun, 08 Mar 2020 07:37:50 +0000

In my opinion a devops person with software developer background can do better in devops than direct devops dive-in.

Writing substantial amount of code, trying to scale it, refactor, maintain, observe, debugging, unit testing, picking a framework for a project are all a must-need experience for a devops person to function effectively in their devops role. In short, there cannot be a junior devops engineer.

Without coding experience it will be difficult to understand concepts like state management, code versioning, code performance, library dependencies, packaging, need for containers, distributed tracing, why centralized logging and the list will go on. And these concepts are very much part of devops.

A devops role demands someone to learn a suite of tools to implement end-end automation. Anyone can learn these suite of tools without prior software development but that doesn’t make them devops ready. Being a software developer and working for few years as a developer is the very first step in becoming devops ready.

Software development is a very wide spectrum, every software dev solves different kinds of problems with different software development tool sets. Having hands-on in all these tools is close to impossible. With plethora of tools in devops space, sometimes even experienced developers find it little intriguing to understand what a particular tool is for and what kind of problem it solves. Learning to use a tool is easy but the ability to decide right tool for the right job and the optimal usage of the tool will come with years of experience on software development.

Python programming and scripting

krishna — Sat, 07 Mar 2020 13:38:34 +0000

I think python scripting and python programming are two different things. Python scripts can be used to automate a specific task or glue scripted tasks. Whereas programming is an organised structure, modular, unit-testable, easily maintained compared to scripts.

Python is a versatile dynamic language and can be leveraged both as a scripting and programming language.

If you are very clear that your task is going to accomplish just one specific objective and does not scale up in future write a python script to execute just this one piece or glue other scripts written already. But if you foresee that your task can scale up tomorrow and can grow like a mindmap please do not write ten scripts and then write a script gluing these scripts, rather organize a structure using classes or try to leverage existing frameworks to accomplish your objective.

Check this awesome python repo https://github.com/vinta/awesome-python

Build a wingman UI for your product

krishna — Thu, 05 Mar 2020 12:15:41 +0000

Imagine you are a startup trying to pitch your idea with the prototype to potential customers, then there will be several instances where you need to generate data, content, user login sessions etc. With all this simulated data properly sitting in the UI, you can plan and pitch your demo better. In order to do this consistently and repeatedly, a separate monolith UI+scripts totally other than your product interface just to simulate actual use cases would help you to deliver your pitch better and might even save time.

I have been working as full-stack developer with infrastructure based startups for the past several years. The product that gets built in these startups involves state, data, sessions, health checks, multiple user logins from several actual services, infrastructure and cloud resources. In all these products, we built and separate monolith demo UI to bring up and tear-down resources and services on-demand. These services would then push the actual data into the portal’s database and eventually into product portal’s UI. This method has helped us better rather than triggering the simulation using scripts.

For example, lets say you are building a highly scalable and performant video conference app and you would like to demonstrate the scalability of the product by really starting, not just emulating, 100’s of conferences with 10’s of users in each conference. This can be built using a script. Instead built a Demo UI and trigger this script from a UI button. This way you can clearly showcase what you are trying simulate.

Advantages of building a demo portal

Repeatability

Flexibility to pass dynamic inputs
Better clarity on what the simulation script does
Your CEO doesn’t have to SSH into a machine and trigger the scripts
Extend and scale the portal to show more info like dev metrics, logs, etc
Can be easily containerized and used by many

Demo portal can be really lean

Just one user login with a strong password
UI need not be aesthetically pleasing a decent one would do
Monolith app. API, microservices based development is not mandatory
If code is containerized, hosting in public cloud’s managed container service would help
Must have features in Demo UI

In all the demo UI built for infrastructure products there are these common functionalities

Tear-down all the resources
Reset Configuration
As the demo clicks are launching resources , its very important to tear them down and save cloud cost. As you are playing around changing configurations of the services and resources, we will screw up several things in the process and these configurations needs to be reset to some defaults.

Technologies I prefer building this portal - nodejs, docker, bootstrap, jquery, keeping it simple.