Forem: om pharate

Twitter System Design

om pharate — Wed, 19 Mar 2025 13:29:24 +0000

Let’s look at the requirements to start with

⚙️📌 Functional Requirements

Tweet - allow users to post text, images, video, links, etc
Re-Tweet - allow users to share someone's tweets
Follow/unfollow
Search

⚙️📌 Non Functional Requirements

Read Heavy
Fast Rendering
Fast Tweet
The system should be scalable and efficient.

👥🧑‍🤝‍🧑 Types Of Users

Famous Users - celebrities, sportspeople, politicians, or business leaders
Active Users
Live Users
Passive Users
Inactive Users

🔍👀 Capacity Estimation for Twitter System Design

Let us assume we have 1 billion total users with 200 million daily active users(DAU). and on average each user tweets 5 times a day

200M * 5 tweets = 1B/day

Tweets can also contain media such as images, or videos. we can assume that 10 percent of tweets are media files shared by users.

10% * 1B = 100M/day

1 billion Requests per day translate into 12k requests per second.

1B / (24 hrs * 3600 seconds) = 12k requests/seconds

Let's assume each message on average is 100 bytes, we will require about 100 GB of database storage every day.

1 billion * 100 bytes = 100 GB/day

🗄️💾 Data Storage

User accounts - User data like username, email, password, profile
picture, bio, etc. in a relational database like (PostgreSQL)
Tweets - Store tweets in a separate table within the same database,
including tweet content, author
ID, timestamp, hashtags, mentions, retweets, replies, etc
Follow relationships: Use a separate table to map followers and
followers, allowing efficient retrieval of user feeds.
Additional data: Store media assets like images or videos in a
dedicated storage system like S3 and reference them in the tweet table.

📝 Note: Remember how Twitter is a very read-heavy system? Well, while designing a read-heavy system, we need to make sure that we are precomputing and caching as much as we can to keep the latency as low as possible.

🔥📈 High Level Design

View High-Quality Image - Download

🔹 Register & Login Flow 🔑👤
When a user registers or logs in, the request goes through a Load Balancer (LB) to the User Service. The service interacts with the User DB MySQL Cluster, which stores authentication details. To speed up login verification, Redis is used for caching user session tokens. After authentication, a session or JWT is issued to the user for subsequent requests.

🔹 User Follow/Unfollow Flow 🔄👥
When a user follows or unfollows someone, the request is sent via the LB to the Graph Service, which manages user relationships. This service updates the User Graph DB (MySQL Cluster) and caches frequently accessed follow data in Redis to reduce DB queries. Updates to the user’s feed can be sent to Kafka for processing, ensuring scalability.

🔹 Tweet Flow 📝📢
When a user posts a new tweet, the request is handled by the Tweet Service, which stores the tweet in the Cassandra Cluster (User Tweets DB) for scalability. If the tweet contains media, it is stored in Blob Storage and served via the CDN. The tweet event is also pushed to Kafka, where different consumers (such as Tweet Preprocessing Service and Analytics Service) process it for engagement tracking and recommendations.

🔹 Search & Analytics Flow 🔍📊
The Search Service queries Elasticsearch, which is updated via the Search Consumer that listens to Kafka events. To improve performance, Redis caches frequently searched terms and trending topics. The Analytics Service tracks user interactions and forwards insights to Apache Spark, which processes large-scale data and stores it in a Hadoop Cluster. Processed insights are then used to enhance recommendations and notifications.

💯 Conclusion

This system is well-architected with horizontal scalability, event-driven processing, and efficient caching strategies! 🚀

Chat Application Architecture | AWS | API Gateway | WebSockets | Microservices | System Design

om pharate — Mon, 17 Mar 2025 16:35:23 +0000

In this blog, we’ll dive into one of the ways to build a scalable chat application using AWS Cloud. No fluff—let’s get straight to it!

Functional Requirement

User should be able to chat using we or mobile app.

⚡🖥️Tech Stack

🌐 Front End
Website: 📄 HTML5 / 🟨 JavaScript (SPA) / ✨ Nextjs
Mobile App: 📱 Appropriate Tools

🤝 Interactions
All Client to Server interactions over 🔌 WebSockets
Request/response payload: 📦 JSON

⚙️ Backend
Backend will be 🚫 Serverless as far as possible
Deployed on ☁️ Cloud
Scale on demand: 📈

Logical Architecture

AWS Architecture

🧩 Components and Their Roles

Clients (Client 1, Client 2, ... n Clients)🖥️
- These are WebSocket clients that connect to the API Gateway WebSocket API.
- They send and receive messages through WebSockets.
API Gateway (WebSocket API)🚪
- API Gateway manages WebSocket connections.
- It provides endpoints for WebSocket connection, disconnection, and message handling.
- It routes events to the appropriate AWS Lambda functions.
Lambda Functions🟧
- Connect Lambda Function
  - Executes when a client establishes a WebSocket connection.
  - Stores the client’s connection ID in DynamoDB for message routing.
- Disconnect Lambda Function
  - Triggers when a client disconnects.
  - Removes the connection ID from DynamoDB to prevent unnecessary message delivery.
- onMessage Lambda Function
  - Handles incoming messages from WebSocket clients.
  - Retrieves connection IDs from DynamoDB to forward messages to the correct recipient.
DynamoDB🔷
- Stores active WebSocket connections (connection IDs).
- Used for tracking which clients are online and enables message delivery to the correct recipients.

How This 🏗️ Architecture Works

Client 1 and Client 2 connect to the WebSocket API via API Gateway.
API Gateway triggers the Connect Lambda function, which stores the connection ID in DynamoDB.
When a client sends a message, API Gateway invokes the onMessage Lambda function.
The onMessage Lambda function retrieves connection IDs from DynamoDB and forwards messages to the appropriate recipients using API Gateway.
When a client disconnects, API Gateway calls the Disconnect Lambda function, which removes the connection ID from DynamoDB.

How Far Can It Scale 📈?

API Gateway supports up to millions of concurrent WebSocket connections.
Lambda can handle thousands of requests per second (subject to AWS account limits).
DynamoDB scales on-demand and can handle millions of transactions per second with proper indexing and partitioning.

🔚 Conclusion
This architecture is fully serverless, scalable, and cost-efficient for handling real-time WebSocket connections. It works well for medium to large-scale applications and can handle millions of users with minimal infrastructure management. 🚀

Create Todo in React JS with typescript?

om pharate — Thu, 18 Jul 2024 12:45:07 +0000

Creating a to-do application in ReactJS is a great first step towards upskilling. However, incorporating TypeScript into your project will also be beneficial for improving your TypeScript skills.

This blog post will teach you how to create a basic to-do app using TypeScript. The app will include essential features such as:

⚒️ Creating new tasks
❌ Deleting tasks
↕️ Updating tasks
✅ Marking tasks as complete

Getting Started With Vite.

npm create vite@latest

npm install

npm run dev

Now the basic folder structure will look like this.

Do not worry in case you have any additional files.
I have set up the tailwind CSS if you don't know how to set up tailwind CSS in React JS click here.

Here are the components/modules we are going to develop:

Navbar
Form
Todo List

Create components/Navbar.tsx

create a basic navbar with a title and about.

Attach the navbar in app.tsx

Create components/Form.tsx

Create this basic form having an input field and a button "Add".
and a use state for getting value from the input field.

However there is a problem in this code.

the problem with the typescript.
because we are using typescript we are supposed to tell the datatype or what type of value It will hold in future to state(task).
and our state is empty there it will be considered undefined only.
and below in the input field, we are assigning a string to the state in onChange.
but the task(state) value was supposed to be undefined.

fix:

the value it will hold must be a string.
which will come from the user input.

Add onClick functionality

Import Form in App.jsx

Create a global state of tasks array

create an array of tasks.
A task interface consists of a single task.
pass the setTasks to the Form component.
- accept task the setTask in Form component.
- the type of prop would be: React.Dispatch<React.SetStateAction<task[]>>
- in the typescript which type of props you are going to receive is required to mention.
- in this case, setTask is a State action.

and on the onSubmit function we have pushed the task value (local state) to tasks (global state array).

Forem: om pharate

Twitter System Design

⚙️📌 Functional Requirements

⚙️📌 Non Functional Requirements

👥🧑‍🤝‍🧑 Types Of Users

🔍👀 Capacity Estimation for Twitter System Design

🔥📈 High Level Design

💯 Conclusion

Chat Application Architecture | AWS | API Gateway | WebSockets | Microservices | System Design

Functional Requirement

Logical Architecture

AWS Architecture

How This 🏗️ Architecture Works

How Far Can It Scale 📈?

Create Todo in React JS with typescript?

Getting Started With Vite.

Now the basic folder structure will look like this.

Create components/Navbar.tsx

Attach the navbar in app.tsx

Create components/Form.tsx

However there is a problem in this code.

Add onClick functionality

Import Form in App.jsx

Create a global state of tasks array

Create components/Todos.tsx

Final app.jsx

Output

Forem: om pharate

Twitter System Design

⚙️📌 Functional Requirements

⚙️📌 Non Functional Requirements

👥🧑‍🤝‍🧑 Types Of Users

🔍👀 Capacity Estimation for Twitter System Design

🔥📈 High Level Design

💯 Conclusion

Chat Application Architecture | AWS | API Gateway | WebSockets | Microservices | System Design

Functional Requirement

Logical Architecture

AWS Architecture

How This 🏗️ Architecture Works

How Far Can It Scale 📈?

Create Todo in React JS with typescript?

Getting Started With Vite.

Now the basic folder structure will look like this.

Create components/Navbar.tsx

Attach the navbar in app.tsx

Create components/Form.tsx

** However there is a problem in this code. **

Add onClick functionality

Import Form in App.jsx

Create a global state of tasks array

Create components/Todos.tsx

Final app.jsx

Output

However there is a problem in this code.