Forem: Soumyajit Deb

Lessons I learned from my first fullstack project

Soumyajit Deb — Sun, 03 May 2020 05:01:49 +0000

I recently completed my first fullstack project. Well, I still think it is far from completion as I keep getting ideas about new features I can add to my project but now for the least I can be proud of it.
The project was a real estate dealing site. The purpose of the site was to help people look for houses they want to buy and also help real estate agents to sell any houses they have by listing them on the site. Once a person likes a house and want to buy/rent it, the person can contact the real estate agent and talk about it and make a deal.

While the idea seemed pretty simple to me at first but as I started coding, then I realized that it is not the case as I kept stumbling upon design problems, database deadlock situations, concurrency issues and many more.
I will write about the project in greater details in future post, for now I want to share the lessons I learned while doing the project and which can be prevented with some precaution.

Well, the technology stack I used in my project are:

For frontend:

HTML
CSS
Bootstrap framework
Javascript

FOr backend:

Node
Express.js framework

For database:

MySQL

Before starting the project, I had the knowledge and experience of working with HTML,CSS,Bootstrap and MySQL extensively while Javascript, Node and Express.js were completely new to me and I had no previous knowledge of them.

By doing the project, I learned a lot of new things and also I made a lot of mistakes which I only came to realize at the end of the project.

1. Plan everything before you start programming

One of the grave mistakes I did was I didn't plan what I needed for the project, what are the features that needed to be present and I didn't put too much thought into the database schema design. All of this led to a lot of trouble in the later part of the project. As I didn't plan the features to be added to the project in the beginning, I kept adding new features on the go as I kept getting the idea about them. But this created a lot of hassle for me!
As the kept adding new features, I had to change my database design a lot. I kept on adding new tables, sometimes deleting current ones and many times changing the schema of the current relations. All of this led to a whole lot of frustration and confusion which clearly disturbed the smooth flow of the project.

2. Design your database schema before you start programming.

Although, this point is part of the first point but I want to put more emphasis on it. One of the big mistakes I did in the beginning of my project is I didn't put much thought or work in the schema design of my database. This led to a huge pile of problems in later part of the project. One of the main problems I faced was while creating triggers in the MySQL database. Due to no previous planning of my database schema, I stumbled across database deadlock situation while creating one of the triggers. Since, I was already near the end of my project, changing my database schema at this point felt a lot tough with a huge data population. Thus, I was not able to resolve the deadlock situation and had to drop the idea of adding a feature which was dependent on the trigger. Also, at the end of the project, I realized how no planning of the database schema led to a poorly constructed and highly inefficient database. The database was not at all scalable and if it was not a personal project but a product to be used by people, it would have caused a lot of trouble as number of users increased. So, it is better to design the database properly at the beginning of the project than to go through all the trouble later on.

3. Know the framework/language in depth before you start doing some project.

I coded the backend bit of my project in node. Node was completely new to me before starting the project. Before starting the project, I spent a lot less time than it was needed to learn node and it surely created a lot of problem while doing the project. While doing the project I didn't know that Nodejs is asynchronous in nature. While I primarily have experience with C,C++ which are synchronous in nature, the asynchronous nature of node came as a big shock to me. I learned about the asynchronous nature when I was sending query to my database from the server using node. The flow of the code was not the one I was expecting. So after sending a query to the database, node didn't wait for the result and simply started executing the next line of code. It created a lot of errors and unexpected results. I struggled to understand the problem when it occurred and it took up a lot of my time to figure out the reason behind the unexpected behavior of the code. How I solved this problem has also taught me a lesson.

4. While taking each decision, always consider the long term solution.

In the above point, I mentioned the problem I faced due to the asynchronous nature of node. While I recognized the problem after some time, I realized I lacked the knowledge of promises and async/await which are able to solve the problem more efficiently. Soon I started to read about them and I realized if I implement them, I have to modify a huge chunk of my code. So, instead of doing that I took the short term solution which is executing each query in nested form. Although this solution worked perfectly, it created a problem of it's own. This problem is something known as the callback hell. It basically means is that I kept nesting my function calls and the nesting got so deep that my code became somewhat cumbersome and unreadable.
Even for me, the code became somewhat cumbersome and I can imagine how difficult it would be for someone other to read the code and understand it. As I kept nesting, the lines of code kept shifting rightwards and soon it became difficult to code in the editor itself. So, it is wise to plan everything always consider the long term solution.

5. Choose the appropriate framework before starting the project.

In my first attempt of building the site, I chose to do the server side code using pure node. Soon I realized it was a wrong decision and had to restart the whole project. It was because as humongous the project was, coding it with pure node led to lot of programming from scratch and created a lot of frustration. I basically had to code a lot of the components from scratch as pure node don't have them in the native library. It took a lot of time and if you are under time constraint, it would be better to use a framework such as express.

6. Always make your code modular

I always had a bad habit of coding all of my methods, structures and everything in a single file.Although this worked for small projects but while working on somewhat large projects, it created a lot of trouble. If the code is not modular, first it creates a lot of confusion during debugging and secondly it makes the code unreadable. To be able to write a clean code, it in necessary to make the code modular. It not only helps to deliver your intent, idea to the other person reading your code clearly and effectively but it also helps you to have a smooth flow during the coding process and be less frustrated while debugging.

Unlocking some features of bash terminal

Soumyajit Deb — Tue, 10 Dec 2019 05:16:55 +0000

Well, most of us are familiar with bash terminal one way or another.Terminal is the most commonly used tool by developers and in general, for every person who is trying to execute a code on a computer.
Recently, while working on a project, I have learned a lot about some features of the bash terminal which are usually hidden and can be unlocked by doing some low level programming. Some of this features are pretty cool and can be helpful in many situations.These features of bash terminal are by default switched off.

Now without further delay, let's unlock those features!

1. ECHO

The Echo feature causes each key you type to be printed on the terminal, so you can see what you are typing. This can be turned off by changing some flags which control various attributes of the terminal. This can be done by doing some low level programming in C.

#include <termios.h>
#include <unistd.h>

void enableRawMode() {
  struct termios raw;
  tcgetattr(STDIN_FILENO, &raw);
  raw.c_lflag &= ~(ECHO);
  tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw);
}

int main() {
  enableRawMode();
  char c;
  while (read(STDIN_FILENO, &c, 1) == 1 && c != 'q');
  return 0;
}

struct termios, tcgetattr(), tcsetattr(), ECHO, and TCSAFLUSH all come from termios.h.
After you execute this program, anything you type will not be printed on the terminal. You may be familiar with such feature of the terminal if you ever had to type a password at the terminal.
In the above program, we have used the termios header file, which contains functions which can change the attributes of the terminal.

struct termios is pre-defined structure which contain the terminal flags as the fields.

tcgetattr() helps to read the current attributes of the terminal and write them to the struct termios.

raw.c_lflag is the flag, which contains the bit, which controls the echo feature of the terminal.
ECHO is a bitflag, defined as 00000000000000000000000000001000 in binary. We use the bitwise-NOT operator (~) on this value to get 11111111111111111111111111110111. We then bitwise-AND this value with the flags field, which forces the fourth bit in the flags field to become 0, and causes every other bit to retain its current value. Flipping bits like this is common in C.

Terminal attributes can be read into a termios struct by tcgetattr(). After modifying them, you can then apply them to the terminal using tcsetattr(). The TCSAFLUSH argument specifies when to apply the change: in this case, it waits for all pending output to be written to the terminal, and also discards any input that hasn’t been read.

After the program quits, depending on your shell, you may find your terminal is still not echoing what you type. Don’t worry, it will still listen to what you type. Just press Ctrl-C to start a fresh line of input to your shell, and type in reset and press Enter . This resets your terminal back to normal in most cases.

The above program will quit as soon as you type q .

Bash terminal has primarily two modes, one is canonical mode and other one is raw mode. Now, bash terminal is by default in canonical mode. In canonical mode, the input in the terminal is taken line by line rather than byte by byte. Although line by line mode of input is helpful as in this case, we can backspace if some typo has occurred but sometimes this feature is not needed and byte by byte mode of input is of much help.Byte by byte mode of input is provided in raw mode if the terminal.

Similarly, we can change the mode of our terminal from canonical to raw mode by doing some low level C programming and changing some flags.

Canonical to Raw mode

There is an ICANON flag that allows us to turn off canonical mode. This means, now the terminal will be reading input byte by byte rather than line by line.

#include <stdlib.h>
#include <termios.h>
#include <unistd.h>
struct termios orig_termios;

void enableRawMode() {
  tcgetattr(STDIN_FILENO, &orig_termios);
  struct termios raw = orig_termios;
  raw.c_lflag &= ~(ECHO | ICANON);
  tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw);
}

 int main() {
  enableRawMode();
  char c;
  while (read(STDIN_FILENO, &c, 1) == 1 && c != 'q')
{
   if (iscntrl(c)) {
       printf("%d\n", c);
     } else {
      printf("%d ("%c")\n",c,c);
      }
}

  return 0;
}

You just have to BITWISE-OR(|) ICANON with ECHO in the above program and it will turn of the canonical mode and the terminal will be in raw mode.

The program will quit as soon as you press q .

To get a better idea of the raw mode, the program prints the key presses as soon as they are typed unlike canonical mode, where they were printed after a line is given as input and enter is pressed.