Forem: shyynux

snow : saas starter kit

shyynux — Sun, 01 Sep 2024 22:24:09 +0000

This is a submission for the Neon Open Source Starter Kit Challenge : Ultimate Starter Kit

About Snow

Snow is a comprehensive developer kit designed to supercharge your SaaS development process. Built with simplicity and efficiency in mind, Snow provides everything you need to build, deploy, and scale your SaaS application quickly and effortlessly.

Snow is designed for developers who want to accelerate their SaaS development without sacrificing quality. With built-in integrations and a streamlined setup, Snow eliminates the hassle of configuring essential tools, allowing you to focus on what truly matters—building great software.

Link to Kit

This is the link to the kit : link.

Why I built Snow?

A lot of times when I get ideas, I quickly want to build, but the setup and everything takes so much time. It kind of sucks out the fun of building.
I know most of us can relate to this feeling.
With Snow, You do not need to worry about all this, since the hero section, the landing page, the ORM (drizzle) and the database postgres(neon) is already set up for you. I also plan to setup auth and payments integration. The verification takes some time for Stripe so I will be including it a few days after the launch, but if your country supports quick, I request you to do for the meantime.

Comparable to products which cost around $150+, snow promises to provide the same services with 1/3rd of the cost. It has a free tier as well.

I am excited to share snow with everyone.

Also, this product is specifically for builders, so let me know what is that one thing that annoys you or you constantly need, I will add it in the starter kit. Open to feedback and suggestions.

Have a nice week :)

What is System design? & terms you should know before diving deep into system design.

shyynux — Thu, 14 Mar 2024 19:31:39 +0000

Hi everyone, recently I got hooked into system design and I wanted to share this knowledge. A lot of things started making sense once I started deep diving into system design. I am very excited to share this with the world. Before we go deeper, let us understand what System design actually means.

System design aims to design scalable, reliable and maintainable systems. Every system is unique and the design should be catered to the specific situation. (I talk about the terms like scalable and reliable below, don’t worry).

System design is basically designing scalable, reliable and highly efficient systems that are fault tolerant and meets all the business requirements as well.

It simplifies and explains the components, APIs, data models and the relationship and dependencies among them.

Obviously no design is perfect and there is always a scope of improvement, but we always try to pick the most optimized and correct solution according to our needs. We also consider our limitations like cost etc.

For example - If we have a lot of unstructured data, we would prefer NoSQL or maybe a combination of the different types of databases. Or if we are expecting a lot of data in blob format (images/videos), we will use something like S3 buckets with a CDN solution. If these terms scare you, don’t worry, coz I used to feel the same whenever I used to see the word CDN. (it gets better)

🤔 But why system design, why can’t we code directly?

You can, it will take more time, more redos and it will be far less efficient and way CHAOTIC.
Why would you hammer a nail with your hand, when you can actually use a hammer. I know lame example, but I hope you got the point. Doing a huge task without planning and with planning, the difference is insane. A well designed system is scalable, performs better and makes your life easier as a dev.

Let us understand the building blocks or the basic stuff you need to know before attempting to design systems:

Scalable: Scalability means the system should function properly when the system has grown from x to 100x (or 200x) users, or when the system is processing larger volumes of data than before. In other words, the system should be ‘scalable’ as in able to handle scale.
Reliable: The system should perform as expected by the user in all conditions, it should not break in case the user makes mistakes, and should be able to handle the load. Basically it should be reliable in all scenarios and work as expected.
Maintainable: Once the system is up and running, over the years, the main task is to maintain it and keep it running. Being maintainable means the system should function properly over time and adapting new functionalities should be smooth.
Load balancer: Responsible for dividing the load among servers, so that the service functions correctly in case the load is increased. It distributes the load coming from users to all the different servers so there is not load only on one server. Helps in achieving scalability.
Caching: By caching we mean that storing frequently needed data or computational results, by doing this we do not need to query the database or compute those results for each request.. So you can think of a cache like a smaller and faster storage. Common examples: redis, memcached. It reduces latency and improves user experience.
CDN: CDN or content delivery network, caching and distributing data to hundreds or billions of users. Distributing our data to a number of users.
API: APIs or application programming interface is an effective way to communicate between users and backend servers or from one server to another. Commonly we use REST APIs in modern system designs.
DNS: DNS or domain name server,translates domain names to IP addresses so browsers can load Internet resources. In simple words, the public address of the server, in human readable form.
Databases: Storage for data so to access it later, when you are designing your system, you would not want to worry about how to store the data, where to store the data so we depend on databases or databases management systems. We have a wide range of databases, and we choose them according to the type and amount of data and our budget. A lot of times we use a combination of different database options to fulfill our needs. Examples include PostgreSQL, MySQL, MongoDB, AWS DynamoDB, and Cassandra. (etc.)
Database Sharding: Dividing your database into shards, or partitions, makes it faster as queries now has to deal with lesser data. Ex- 4 million data is divided into 4 shards, so now every query will read just 1 million data, so faster. We need to consider query routing mechanisms.
Messaging queues: Helpful in asynchronous flow and makes the system fault tolerant. Examples: Rabbit MQ, Amazon SQS (Read bout it)
Rate limiter: No matter how scalable your system is, there is always a limit, suppose there is a malicious user who is sending too many requests, there will come a point when your service will start overloading or malfunctioning, we do not want this scenario. We want our users to be able to rely on our system, Rate limiting means limiting the number of requests per second or per minute according to our need. There are multiple ways, for ex: we can limit API calls made to our backend per second from an IP address.
Micro-services and monolithic architecture: Well in simple words, micro services based architecture has different independent services for different things or functions, code is loosely coupled, easier to maintain and update, flexible. Monoliths are the opposite, one whole service has all the parts. It is faster and simpler to develop, easier to debug, you might have seen this a lot in open source or personal projects.
Logging: A mechanism to keep logs of what happened in our system, so that we can debug later in case our system runs into an issue. For ex: whenever a metric spiked we can see on AWS Cloudwatch to see what went wrong.
Elastic Search: A lot of systems need search functionalities these days, where the user can enter a text or query and get results. We can query in a traditional database but it has a lot of intricacies, here comes elastic search. It is a text search engine built on Apache Lucene, (it has Fuzzy search, because we do not want the system to break when the user types bracekets instead of bracelets).

A lot of the topics need detailed discussion which I will talk about in later blogs. This is a system design blog series. We will learn more as we go down this journey. Feel free to google the topics and learn more about them.

I hope you enjoy designing systems. :)

Introduction to typescript

shyynux — Fri, 20 Oct 2023 14:39:04 +0000

Hi friends, so I am from the C++ and the Java world, but I have always been interested in web development so I finally took the plunge and went into the JS world.

JS is wild. I was amazed, it’s like you have no rules. You don’t even need to write ‘int x’ if you want to use x as an integer, just do ‘let x’ and x is yours. I was enjoying building my project using MERN but as an open-source lovr, I wanted to contribute to projects. While diving into those projects, I realised that big projects that interest me use Typescript and something called monorepo (nothing but one single repo for all the things your project needs).

Now TypeScript, sweet TypeScript, is actually a superset of JS, whatever you write in JS you can write in TS(but don’t). Coming from a C++ background I felt validated, because in TS you have to define the types, for example you can do this in JS,

var x = 42;
x = “noob”;

and no one will say a word.

But in TS, if you do this, it will start crying because you defined x as a number.

var x: number = 42
x = “noob” /* invalid */

which is not a bad thing tbh.

🚀 How does the ts code is compiled?

The typescript compiler or the tsc does not compiles the code, it does two things:

type checks the code
converts ts code to js

And then the js code is compiled.
Pretty cool, right? 🚀 You can actually see that js file being created beside your ts file.

Anyways getting to the official definition,

“TypeScript is a strongly typed programming language that builds on JavaScript, giving you better tooling at any scale.”

Also fun fact, it was developed by Anders Hejlsberg, from Microsoft.

I will share more technical details later in this series where I deep dive and code with TypeScript.

What is tree data structure? 🌳

shyynux — Tue, 17 Oct 2023 19:39:21 +0000

Today we are going to discuss:

What is a tree?
Basic tree terminologies
Types of trees
Operations on trees
Some practice questions

🌳 What is a tree?

Trees are hierarchical data structures used to organize and represent data in a manner that resembles a tree with a root at the top and branches connecting nodes. Each node can have child nodes, creating a parent-child relationship.

Tree data structures have various real-world applications:

File Explorer: Trees organize files and folders hierarchically, making it easy to navigate and locate data on your device.
Parsers (XML parser): Trees structure and interpret data from files like XML, allowing software to understand and process information effectively.
Database for Indexing: Trees enable efficient data storage and retrieval in databases, speeding up searches.
DNS (Domain Name System): Trees convert domain names into IP addresses, facilitating internet communication.
Trie (Autocomplete): Trees power autocomplete suggestions in search engines by quickly searching for and suggesting relevant terms or phrases as you type.

🌳 Basic tree terminologies

Parent: A node in a tree data structure that has one or more child nodes directly connected to it.
Child: A node in a tree data structure that is connected to a parent node.
Root: The topmost node in a tree, serving as the starting point for traversing the structure. It has no parent.
Leaf: A node in a tree that has no children, meaning it is a terminal node.
Internal Node: A node in a tree that is not a leaf, which means it has one or more children.
Level: The position of a node within a tree, with the root node at level 0 and each subsequent level increasing by 1.
Sibling: Nodes that share the same parent in a tree.
Degree of a Node: The number of subtrees (children) connected to a node. Leaf nodes have a degree of 0.
Height of a Tree: The length of the longest path from the root to a leaf node in the tree.
Height of a Node: The length of the longest path from the node to a leaf node in the tree.
Depth of a Node: The length of the path from the root to that node, measured in terms of the number of edges in the path.

🌳 Types of trees

Now that we know what a tree is, let use look into a few different types of trees. You must have came across a binary tree or a binary search tree.

General Tree: A tree structure where nodes can have any number of children, providing flexibility in organising hierarchical data.
Binary Tree: A tree in which each node can have at most two children, commonly used for efficient data organisation and traversal.
Binary Search Tree (BST): A binary tree where values are organised so that left children are smaller, and right children are equal to or greater than the parent, enabling efficient searching and sorting.
AVL Trees: A self-balancing binary search tree that ensures the height difference between left and right subtrees is kept to a minimum, guaranteeing efficient operations.
Red-Black Trees: Another self-balancing binary search tree that maintains balance by using colored nodes and performing rotations, suitable for a variety of applications.
N-ary Trees: Trees where nodes can have more than two child nodes, allowing for versatile data representation.
B-Trees: Self-balancing trees used for managing large datasets, frequently employed in databases and file systems.
B+ Trees: A variant of B-trees with enhanced properties, particularly useful for indexing large amounts of data in databases.
Trie (Prefix Tree): A tree structure optimized for storing and searching strings, commonly employed in applications such as autocomplete and spell-checking.
Ternary Tree: A tree with nodes that have exactly three children, often used in specialised scenarios requiring a three-way decision structure.
Segment Tree: A binary tree structure designed for efficient range queries, especially useful for finding intervals or segments that contain a given query point, offering fast retrieval in O(log n + k) time, where k represents the number of retrieved intervals or segments.

🌳 Operations on trees

We can perform a variety of operations on trees. Let us have a look on some of the common ones.

Insertion: Add a new node with a given value to the tree.
Deletion: Remove a specific node (and its subtree) from the tree.
Search: Find a specific node with a given value in the tree.
Traversal:
- Inorder: Visit nodes in sorted order (for binary search trees).
- Preorder: Visit the current node before its children.
- Postorder: Visit the current node after its children.
- Level-order: Visit nodes level by level, starting from the root.
Find Minimum/Maximum: Find the node with the smallest or largest value in the tree.
Balancing: Rebalance the tree to ensure that it maintains a specific structure, such as in AVL trees or Red-Black trees.
Height Calculation: Calculate and return the height (or depth) of the tree.
Count Nodes: Count and return the number of nodes in the tree.
Diameter Calculation: Find the length of the longest path between any two nodes in the tree.
Lowest Common Ancestor (LCA): Find the lowest common ancestor of two nodes in the tree.
Checking for Balance: Verify whether the tree is balanced, ensuring the heights of subtrees do not differ by more than a certain threshold.
Serialization and Deserialization: Convert the tree into a serial format for storage or transmission, and then reconstruct the tree from the serialized data.
Prefix Search (in Tries): Search for words or strings based on a common prefix in a Trie data structure.
Traversal for Specific Orderings: Perform various tree traversals (e.g., in-order, preorder, postorder) with specific requirements or rules for processing nodes.
Pruning: Remove specific nodes or subtrees from the tree based on certain conditions or criteria.

🌳 Some practice questions

To understand trees better and develop an intuition for solving tree problems, I recommend the following problems.

Before anything else, Implement a simple BST with the functions to create node, insert node, delete node and tree-traversal, after that do the following questions:

Links -

More questions relevant to BST :
Run your c++ code online : https://ideone.com/
Structures in C++

What are messaging queues?

shyynux — Thu, 12 Oct 2023 14:03:55 +0000

One day a team-mate told me during a call to poll some messages from SQS, I was like ‘What’s SQS or polling’, they told me it is an asynchronous queue. I was like whaaat (in my mind), I was a new joinee(shy), I felt so dumb. But i hope you don’t (and you shouldn’t), so today we are going to understand -

What is a messaging queue?
An exploration of some popular messaging queues.
Common terms related to messaging queues (SQS), yes polling as well.

💡 What is a messaging queue?

A messaging queue is a way for services to communicate with each other without having to wait for a response(asynchronously). Messages are stored in a queue until they are processed, and each message is only processed once.
Message queues can be used to break down complex tasks into smaller, more manageable ones, and to make systems more scalable and reliable. They are used in serverless and micro-services architectures.

A component called a producer adds a message to the queue. The message is stored on the queue until another component called a consumer retrieves the message and processes it.

A simple example:

Imagine a website that sells products. When a customer places an order, the website creates a message and puts it in a queue. A separate process then picks up the message from the queue and processes the order.
This allows the website to continue processing other orders while the first order is being processed. It also makes the website more reliable, because if one process fails, another process can pick up the messages from the queue and continue processing them.

💡 Some popular messaging queues:

Amazon SQS: Amazon Simple Queue Service (Amazon SQS) offers a secure, durable, and available hosted queue that lets you integrate and decouple distributed software systems and components.
Rabbit MQ: An open-source message broker that implements the Advanced Message Queuing Protocol (AMQP).
Azure Service Bus: A cloud-based message broker that provides reliable and scalable messaging services for applications.
Apache Kafka: A distributed streaming platform that allows applications to process streams of data in real time.
Apache ActiveMQ: ActiveMQ is a popular messaging service that facilitates disparate data at scale in enterprise systems.

What and when to use should be decided according to the scale, service, tech stack choice etc. factors.

💡 Useful terms associated with SQS

Amazon SQS(Simple queue Service):
As mentioned above, it is a messaging queue service. Our system has producers, consumers using the queue i.e., SQS. The queue stores each message on multiple Amazon SQS servers to ensure that messages are not lost if a server fails.
Producer: The service or application that sends messages to a queue.
Consumer: The service that retrieves and processes messages from a queue, which were placed there by producers.
The Queue: The specific messaging queue being used (e.g., SQS has both simple and FIFO options).
Polling: The process of checking a queue for new messages.
By default, short polling is used, where your application sends a ReceiveMessage request to AWS SQS. SQS checks a subset of servers for messages and sends a response immediately, even if no messages are found. On the other hand, long polling waits for new messages, checking all servers for a specific duration, and returns a response if messages are found, otherwise, it sends an empty response when the polling wait time expires.
Dead-letter Queue (DLQ): A queue that stores messages that cannot be processed successfully by other source queues. This can happen for various reasons, such as if the message is invalid, too large, or if the consumer crashes. However, they are not created automatically; we need to create them first.
Queue Attributes: These include maximum message size, message retention period, redrive policy, and visibility timeout, which all affect how the messaging queue functions.

I really hope this information will help in understanding MQs better. If you want to read more on this topic I will recommend a few resources -

Let me know what messaging queue you used for your service and why.
Have a nice day. 🤍