Forem: Rashid

jokes

Rashid — Tue, 19 Nov 2024 20:47:26 +0000

From Technical Debt to Massive Layoff

Rashid — Tue, 17 Sep 2024 15:39:59 +0000

In this post, I will share my experience witnessing the failure of a large company in the EU, employing over 500 people. This company faced massive layoffs due to technical debt and poor decision-making.

"Hey everyone. Today, some of you will lose their jobs." - CEO

We'll dive into what happened, analyze the causes, and understand the lessons learned from this unfortunate event.

Feature-Focused Development

The Project Manager (PM) consistently prioritized new features over addressing technical debt, believing that time spent on code clean-up and optimisation was a waste. Convincing the PM to allocate even one sprint to deal with technical debts was an uphill battle. From his perspective, time spent on cleaning up the codebase and eliminating technical debt was a strategy developers used to waste time.

This approach led to a vicious cycle where the development team was always under pressure to deliver new features quickly, often at the cost of quality. It's challenging for non-technical stakeholders to grasp the intricacies of software development. When developers release features under pressure, they often skip essential testing, optimization, and sometimes even resort to temporary solutions. This approach inevitably leads to new bugs and system inconsistencies.

Moreover, the lack of focus on technical debt meant that the codebase became increasingly difficult to manage and maintain. Over time, this resulted in slower development cycles, higher defect rates, and a significant drop in overall productivity. The PM's inability to understand the long-term benefits of addressing technical debt was a critical factor in the company's downfall.

Temporary Fixes

The situation worsened whenever something broke down. We were instructed to implement quick fixes due to time constraints. As a result, the codebase became riddled with temporary fixes, leading to a cascade of bugs that grew larger and more frequent over time. These band-aid solutions, while addressing immediate issues, created a fragile and unstable codebase that was prone to frequent failures.

Each temporary fix introduced new inconsistencies and potential points of failure, making the system increasingly difficult to manage. This approach also demoralized the development team, as they were constantly firefighting instead of building robust, scalable solutions. The culture of quick fixes and shortcuts undermined the quality of the product and eroded trust in the development team's capabilities.

The accumulation of technical debt through temporary fixes reached a tipping point where the system became unsustainable. The constant need for quick fixes not only slowed down new feature development but also damaged current system with critical failures.

Tight Deadlines

Unrealistic deadlines were another significant contributor to the company's technical debt. At times, the deadlines set were beyond achievable, making it impossible to deliver a quality product. When we raised concerns about this issue, the response from the staff engineer was like:

"Just release something working, then we can do something about it."

This mindset led to a culture where the primary goal was to push something into production, regardless of its quality. As a result, crap products were shipped, compounding the existing technical debt and creating a backlog of issues that needed to be addressed.

The pressure to meet tight deadlines also meant that corners were cut, best practices were ignored, and thorough testing was often skipped. This approach not only increased the likelihood of defects but also made it difficult to identify and fix root causes of problems. The relentless pace of development left little room for reflection or improvement, trapping the team in a cycle of reactive problem-solving.

Messed Roadmap

Despite having set OKRs (Objectives and Key Results), we frequently received unexpected projects in the middle of the quarter. This lack of a proper roadmap led to chaos and further contributed to technical debt. The constant influx of new priorities disrupted the development process and made it difficult to maintain any semblance of strategic direction.

"Hey, we just need it now because there is a huge potential in that area"

The absence of a clear, coherent roadmap meant that the team was always in a state of flux, constantly shifting focus from one project to another. This instability not only slowed down progress but also increased the likelihood of mistakes and oversights. The lack of a well-defined roadmap also made it challenging to plan for long-term improvements, further exacerbating the issue of technical debt.

The frequent changes in priorities undermined the team's morale and productivity, as they were never given enough time to complete tasks properly. This chaotic environment made it difficult to establish a sense of accomplishment or progress, leading to frustration and burnout among the team members.

Skill Issues

Compounding the problem was the lack of technical expertise among the leadership. Our tech lead, who should have bridged the gap between business requirements and the development team, lacked the necessary technical skills. Essentially, the tech lead functioned as a second PM with limited technical knowledge.

A tech lead's responsibilities include managing business expectations based on capacity and technical limitations, and assisting the development team with delivery. Unfortunately, our tech lead lacked these crucial skills and had no authority to influence decisions. This deficiency can be traced back to HR, which failed to identify the missing competencies during the hiring process.

The tech lead's inability to provide technical guidance and support left the development team without a clear direction. This lack of leadership further compounded the issues of technical debt and poor decision-making, as there was no one to advocate for best practices or push for necessary improvements.

The HR department's failure to identify and hire qualified technical leaders was a significant oversight. The lack of skilled leadership created a vacuum that contributed to the company's decline, as there was no one to steer the team towards sustainable development practices.

Micromanagement

This topic deserves a post of its own, but I'll provide a quick overview here since it had a huge impact on the situation. The lack of a flat organizational structure, combined with a culture where everyone constantly watched each other, exacerbated the problems. Higher-level management made all the decisions, and no one could question them. Despite being told to "feel free to speak up," it was clear that doing so wouldn't change anything.

Bureaucracy and groups of people with very poor decision-making abilities created an environment where micromanagement thrived. This stifled innovation, hindered productivity, and contributed significantly to the company's downfall.

Conclusion

This experience taught me invaluable lessons about the importance of balancing feature development with technical debt management. It's crucial for non-technical stakeholders to understand the long-term impact of technical debt and to allocate sufficient time and resources to address it. Additionally, having skilled technical leads and realistic deadlines is essential for delivering high-quality products and maintaining a healthy codebase.

By sharing this story, I hope to shed light on the critical factors that led to the downfall of this company, providing insights for others to avoid similar pitfalls.

Key Takeaways:

Prioritize addressing technical debt alongside new feature development to maintain a healthy codebase.
Ensure that deadlines are realistic and allow for thorough testing and optimization.
Develop a clear and coherent roadmap to provide direction and stability for the development team.
Hire skilled technical leaders who can bridge the gap between business requirements and development capabilities.
Foster a culture of quality and continuous improvement to avoid the pitfalls of quick fixes and shortcuts.
By learning from these lessons, other companies can avoid the devastating consequences of technical debt and poor decision-making, ensuring long-term success and stability.

Also, I would highly recommend to use tools like TechDebtMonster, which is a tool I built to help teams manage and eliminate technical debt. This platform turns the daunting task of dealing with technical debt into a gamified experience, making it engaging and less intimidating for developers.

With seamless integration into Slack and JIRA, it allows teams to set schedules, monitor progress, and tackle tech debt monsters at various stages. This tool is a must-have for any team looking to manage technical debts.

Build MSG File Analyzer using NextJS

Rashid — Mon, 17 Jun 2024 05:47:53 +0000

When applying for a tech job it is important to have a strong portfolio to impress the company. Especially for beginner developers the first impression of resume and projects are playing crucial role. The most common project types are To-do app, Weather app, Blog and etc. which is boring.

This post cross-published from Notion with OnePublish.

In this new series we will build unique and creative stuff and ship it to production. Check out previous publication from this series below:

https://www.developertimeline.io/build-a-phishing-site-detector-with-nextjs-and-virustotal/

So, in this episode we will build Email Detail Viewer that will take the MSG file as input, read its content and display it to the user.

We’re going to use a NextJS to build our app fast and ship it to production easily using Vercel. Each time we make a new commit Vercel will update our app so we don’t need to worry about deployment but only focus on building.

Why this project is great?

Because it involves many skills those required to have as a developer:

TypeScript.
Third-party integration.
Use of NextJS itself.
UI/UX skills.
Tailwind CSS (eh)
Deployment

What we are building?

This project is going to be a single page app that will let user to upload MSG File or in other terms Microsoft Outlook Item. After user uploads file, it should extract the email headers, content, attachments and other details of email then display the result to the user with nice UI/UX.

Getting Started

Let’s start by creating a new NextJS project. I will be using latest version of NextJS which is version 14 at time of writing this post.

npx create-next-app@latest

It will prompt few questions to configure the NextJS project. I will select to use TypeScript, initialise the Tailwind CSS and use App router.

✔ What is your project named? … <your_app_name_here>
✔ Would you like to use TypeScript? … No / **Yes**
✔ Would you like to use ESLint? … No / **Yes**
✔ Would you like to use Tailwind CSS? … No / **Yes**
✔ Would you like to use `src/` directory? … **No** / Yes
✔ Would you like to use App Router? (recommended) … No / **Yes**
✔ Would you like to customize the default import alias (@/*)? … No / **Yes**

Now you should have initialised NextJS project ready to use.

Building Backend

First, we need to find a library that can read the content of msg file. There is a npm package named @kenjiuno/msgreader which can read and extract the details from the .msg files.

Let's start by creating and endpoint called /api/reader that will handle POST request from frontend and pass file data to the msg reader.

/api/reader/route.ts

import { reader } from "@/app/utils/reader";

export async function POST(request: Request) {
  const formData = await request.formData();

  const file: any = formData.get("file");
  const buffer = Buffer.from(await file.arrayBuffer());

  const readerRes = await reader(buffer);
  return Response.json({ reader: readerRes });
}

Simply, getting and reading the file from request body and storing the buffer inside buffer variable.

There is no need to save uploaded file locally since reader accepts Buffer as well. That saves us from a lot of file IO operations.

Now, let's add a reader util function:

/utils/reader.ts

import MsgReader from "@kenjiuno/msgreader";

export async function reader(buffer: Buffer) {
  const testMsg = new MsgReader(buffer);
  const testMsgInfo = testMsg.getFileData();
  return testMsgInfo;
}

We're using @kenjiuno/msgreader library to extract the details from file buffer.

That's pretty all from backend side!

Building Frontend

On the frontend side, we're going to use DaisyUI and some Flowbite components to quickly build UI. Both built on top of Tailwind CSS.

Here's the index page that holds all components for this app:

app/page.tsx

"use client";

import { useState } from "react";
import FileInput from "./components/FileInput";
import EmailHeaders from "./components/Headers";
import AttachmentsHeaders from "./components/Attachments";
import { FieldsData } from "@kenjiuno/msgreader";

const Home: React.FC = () => {
  const [details, setDetails] = useState<FieldsData | null>(null);

  return (
    <>
      <FileInput onDataExtracted={(details: FieldsData) => setDetails(details)} />
      {/* Results */}
      <EmailHeaders headerDetails={details?.headers} />
      <AttachmentsHeaders attachmentsDetails={details?.attachments} />
    </>
  );
};

export default Home;

The first component will be FileInput that will allow user to upload their MSG file to the drag & drop input area.

/components/FileInput.tsx

"use client";

import { FieldsData } from "@kenjiuno/msgreader";
import axios from "axios";
import { ChangeEvent, useState } from "react";

interface FileInputProps {
  onDataExtracted: (details: FieldsData) => void
}

const FileInput: React.FC<FileInputProps> = ({ onDataExtracted }) => {
  const [alertMessage, setAlertMessage] = useState<string>("");

  const submitMsgFile = async (event: ChangeEvent<HTMLInputElement>) => {
    const file = event.target.files?.[0];
    if (file) {
      try {
        const formData = new FormData();
        formData.append("file", file);
        const res = await axios.post("/api/reader", formData);
        onDataExtracted(res?.data?.reader);
        console.log(res.data.reader);
        setAlertMessage("Success! Check results below.");
      } catch (error) {
        console.log(error);
      }
    } else {
    }
  };
  return (
    <div className="hero min-h-screen bg-base-200">
      <div className="hero-content text-center">
        <div className="max-w-lg">
          <h1 className="text-5xl font-bold mb-5">MSG Analyzer</h1>
          <p className="mb-5">Drop your .MSG file below to get results</p>

          <div className="flex items-center justify-center w-full">
            <label className="flex flex-col items-center justify-center w-full h-64 border-2 border-gray-300 border-dashed rounded-lg cursor-pointer bg-gray-50 dark:hover:bg-bray-800 dark:bg-gray-700 hover:bg-gray-100 dark:border-gray-600 dark:hover:border-gray-500 dark:hover:bg-gray-600">
              <div className="flex flex-col items-center justify-center pt-5 pb-6">
                <svg
                  className="w-8 h-8 mb-4 text-gray-500 dark:text-gray-400"
                  aria-hidden="true"
                  xmlns="http://www.w3.org/2000/svg"
                  fill="none"
                  viewBox="0 0 20 16"
                >
                  <path
                    stroke="currentColor"
                    strokeLinecap="round"
                    strokeLinejoin="round"
                    strokeWidth="2"
                    d="M13 13h3a3 3 0 0 0 0-6h-.025A5.56 5.56 0 0 0 16 6.5 5.5 5.5 0 0 0 5.207 5.021C5.137 5.017 5.071 5 5 5a4 4 0 0 0 0 8h2.167M10 15V6m0 0L8 8m2-2 2 2"
                  />
                </svg>
                <p className="mb-2 text-sm text-gray-500 dark:text-gray-400">
                  <span className="font-semibold">Click to upload</span> or drag
                  and drop
                </p>
                <p className="text-xs text-gray-500 dark:text-gray-400">
                  SVG, PNG, JPG or GIF (MAX. 800x400px)
                </p>
              </div>
              <input
                id="dropzone-file"
                type="file"
                className="hidden"
                onChange={submitMsgFile}
              />
            </label>
          </div>

          {alertMessage && (
            <div role="alert" className="alert">
              <svg
                xmlns="http://www.w3.org/2000/svg"
                className="stroke-current shrink-0 h-6 w-6"
                fill="none"
                viewBox="0 0 24 24"
              >
                <path
                  strokeLinecap="round"
                  strokeLinejoin="round"
                  strokeWidth="2"
                  d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z"
                />
              </svg>
              <span>{alertMessage}</span>
              <div>
                <a href="#results" className="btn btn-sm btn-primary">
                  Get Results
                </a>
              </div>
            </div>
          )}
        </div>
      </div>
    </div>
  );
};

export default FileInput;

The submitMsgFile handler function listens onChange event from input field. That mean's once user uploads file it wil trigger the handler function.

Next, passing the file to the FormData and sending to the endpoint.

As you noticed this component has callback prop which sends result of reader to the parent component. In page level, we will pass the result to other components to display the result.

Let's now move to the Headers component that displays result of email headers:

components/Headers.tsx

"use client";

interface EmailHeadersProps {
  headerDetails: string | undefined;
}

const EmailHeaders: React.FC<EmailHeadersProps> = ({ headerDetails }) => {
  return (
    <div className="px-16 my-8" id="results">
      {headerDetails && (
        <>
          <h3 className="text-2xl font-bold mb-2">Email Headers</h3>
          <div className="mockup-code">
            <pre data-prefix="~" className="text-xs text-sky-500">
              <code
                style={{
                  display: "block",
                  marginLeft: "50px",
                  whiteSpace: "pre-wrap",
                  wordWrap: "break-word",
                  overflowWrap: "break-word",
                  wordBreak: "break-all",
                }}
              >
                {headerDetails}
              </code>
            </pre>
          </div>
        </>
      )}
    </div>
  );
};

export default EmailHeaders;

and finally Attachments to view attached file names.

/components/Attachments.tsx

"use client";

interface AttachmentsInterface {
  attachmentsDetails: Array<any> | undefined;
}

const AttachmentsHeaders: React.FC<AttachmentsInterface> = ({
  attachmentsDetails,
}) => {
  return (
    <div className="px-16 my-8">
      {attachmentsDetails && attachmentsDetails.length > 0 &&(
        <>
          <h3 className="text-2xl font-bold">Attachments</h3>
          <dl className="max-w-md text-gray-900 divide-y divide-gray-200 dark:text-white dark:divide-gray-700">
            {attachmentsDetails.map((item) => (
              <>
                <div className="flex flex-col pb-3">
                  <dt className="mb-1 text-gray-500 md:text-lg dark:text-gray-400">
                    {item.attachMimeTag}
                  </dt>
                  <dd className="text-lg font-semibold">{item.fileName}</dd>
                </div>
              </>
            ))}
          </dl>
        </>
      )}
    </div>
  );
};

export default AttachmentsHeaders;

Results

At the end the result will look like below:

What’s next?

One more project added yo your portfolio! Feel free to display more details or change the design as you want.

You need to deploy your NextJS to Vercel to make it publicly accessible for others. Also, remember to explain your solution well in ReadMe file since it shows that you’re documenting your work.

PylotStuff / msg-analyzer

Extract msg file details

MSG READER

Key Features:

MSG File Upload: Users can upload MSG files through an intuitive interface with drag-and-drop support.
Data Extraction from MSG Files: The application uses the @kenjiuno/msgreader library to extract headers and other data from MSG files.
Header Display: Extracted data is displayed in an easy-to-read format.

Usage Example:

Navigate to the main page of the application.
Upload an MSG file using the upload area.
View the headers and content of the file displayed on the screen.

View on GitHub

Let me introduce you OnePublish | Notion-based cross-publishing tool

Rashid — Sun, 16 Jun 2024 09:17:22 +0000

Hi everyone, let me introduce you OnePublish which is a cross-publishing tool that allows you to manage all your content in Notion and cross-publish it with few clicks.

Inspiration and Idea 💫

I have been posting tech articles since 2018, but I have always been too lazy to publish them on all of my blogging platforms.

Because, each platform comes with its own unique markdown requirements, image uploading processes, tag settings, and more. Also, simple copy-pasting doesn't work effectively due to the varying ways editors render content.

Yeah.. you guessed it right, I am too lazy for 15 minutes of dedicated work so I spent 5-6 month to build this chrome extension to make publishing easier.

All your content in Notion 🗒️

Have you ever lost your articles? I have been through this, and it was hard for me to rewrite everything. I felt like there was no single place where I could manage all my content.

Then I realized Notion is a great place to manage content with all its tools and easy-to-use markdown editor. So, why not manage content in Notion and make a tool to cross-publish it to other blogging platforms.

No Complex Dashboards. Simple Extension as a Gateway to Publishing. 🎯

I was thinking of making something simple and easy-to-use where there is no need to register on some website and then get lost inside the dashboard.

That idea lead me to make a chrome extension where user don't need to navigate to separate website and manage all the content there.

Minimizing the onboarding process plays crucial role in not getting bored when starting to use the product. I often find myself closing the tab in the middle of registration because of the long setup process.

In the current state of OnePublish, users can connect to Notion with an OAuth flow (few clicks) and then it's ready to use.

I am using my own product 😎

I am cross-publishing to DEV, Hashnode, Medium, and Ghost CMS (soon LinkedIn). And all my content is in managed in Notion.

Looking forward to hear your feedbacks!

// Detect dark theme var iframe = document.getElementById('tweet-1799712258299486630-448'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1799712258299486630&theme=dark" }

Boost MongoDB Performance: Motor Client vs PyMongo - Which is Faster?

Rashid — Tue, 04 Jun 2024 06:55:55 +0000

Once I was tasked with improving the database and general app operations of backend services which are using MongoDB as their main database.

These services were part of huge infrastructure where millions of messages coming through the queues and needed to be processed based on the message actions. That means tons of DB ops each second and other additional checks while processing.

This post cross-published with OnePublish

Real Case Scenario

The processing layer of service was using pymongo to interact with MongoDB and service itself was running in synchronous environment. Even the database operations was handled in bulk still performance was not capable of handling incoming data.

Synchronous code was making things even worse. The code execution waits result from current operation to move forward. That's a serious bottleneck in scalable systems.

This was causing queue overflows and potential data loss every time.

Asynchronous Environment

The solution I implemented was combination of:

Motor
Asyncio
Uvloop

Let's quickly go through the definitions of these items.

PyMongo is the official MongoDB driver for Python, providing a simple and intuitive way to interact with MongoDB databases. It's synchronous, meaning each database operation blocks the execution of your program until it completes, which can be a bottleneck in I/O-bound tasks.

Motor is the asynchronous driver for MongoDB, built on top of PyMongo and designed to take advantage of Python's asyncio library. Motor allows you to perform non-blocking database operations, making it suitable for high-performance applications that require concurrency.

To illustrate the performance differences, I prepared a stress test using two scripts: one using Motor (asynchronous) and the other using PyMongo (synchronous). Both scripts performed the same task of reading and writing documents to MongoDB in batches.

Both scripts reading 300k documents from source collection and migrating them to new target collection.

Asynchronous Script (Motor)

import logging
import asyncio
import time
from bson import ObjectId
from motor.motor_asyncio import AsyncIOMotorClient

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# MongoDB setup
MONGO_URI = 'mongodb://root:root@localhost:27019'
DB_NAME = 'products'
COLLECTION_NAME = 'gmc_products'

client = AsyncIOMotorClient(MONGO_URI)
db = client[DB_NAME]
collection = db[COLLECTION_NAME]
target_collection = db["new_collection"]

async def fetch_products(batch_size, last_id=None):
    query = {'_id': {'$gt': last_id}} if last_id else {}
    cursor = collection.find(query).sort('_id').limit(batch_size)
    products = await cursor.to_list(length=batch_size)
    return products

async def bulk_write_to_mongo(products):
    for product in products:
        product['_id'] = ObjectId()  # Generate a new ObjectId for each product

    try:
        result = await target_collection.insert_many(products, ordered=False)
        logger.info(f'Inserted {len(result.inserted_ids)} products into MongoDB.')
    except Exception as e:
        logger.error(f'Error inserting products into MongoDB: {e}')

async def process_batches(batch_size, concurrency_limit):
    tasks = []
    last_id = None
    while True:
        products = await fetch_products(batch_size, last_id)
        if not products:
            break
        last_id = products[-1]['_id']
        tasks.append(bulk_write_to_mongo(products))
        if len(tasks) >= concurrency_limit:
            await asyncio.gather(*tasks)
            tasks = []
    # Process remaining tasks if any
    if tasks:
        await asyncio.gather(*tasks)

async def main():
    batch_size = 1000
    concurrency_limit = 10

    start_time = time.time()
    await process_batches(batch_size, concurrency_limit)
    end_time = time.time()
    logger.info(f'Total time: {end_time - start_time:.2f} seconds.')

if __name__ == '__main__':
    asyncio.run(main())

Synchronous Script (PyMongo)

import logging
import time
from bson import ObjectId
from pymongo import MongoClient

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# MongoDB setup
MONGO_URI = 'mongodb://root:root@localhost:27019'
DB_NAME = 'products'
COLLECTION_NAME = 'gmc_products'
TARGET_COLLECTION_NAME = 'new_collection'

client = MongoClient(MONGO_URI)
db = client[DB_NAME]
collection = db[COLLECTION_NAME]
target_collection = db[TARGET_COLLECTION_NAME]

def fetch_products(batch_size, last_id=None):
    query = {'_id': {'$gt': last_id}} if last_id else {}
    cursor = collection.find(query).sort('_id').limit(batch_size)
    products = list(cursor)
    return products

def bulk_write_to_mongo(products):
    for product in products:
        product['_id'] = ObjectId()  # Generate a new ObjectId for each product

    try:
        result = target_collection.insert_many(products, ordered=False)
        logger.info(f'Inserted {len(result.inserted_ids)} products into MongoDB.')
    except Exception as e:
        logger.error(f'Error inserting products into MongoDB: {e}')

def process_batches(batch_size):
    last_id = None
    while True:
        products = fetch_products(batch_size, last_id)
        if not products:
            break
        last_id = products[-1]['_id']
        bulk_write_to_mongo(products)

def main():
    batch_size = 1000

    start_time = time.time()
    process_batches(batch_size)
    end_time = time.time()
    logger.info(f'Total time: {end_time - start_time:.2f} seconds.')

if __name__ == '__main__':
    main()

Results and Analysis

Execution Time of Migrating 300k documents:

Asynchronous script: 17.15 seconds
Synchronous script: 23.26 seconds

The asynchronous script completed the task 6.11 seconds faster than the synchronous script. While this might not seem like a significant difference for a single run, it becomes more pronounced in high-load scenarios or when processing large datasets continuously.

Benefits of Using Motor and Asynchronous Environment

Improved Throughput: Asynchronous operations can handle more tasks concurrently, increasing overall throughput. This is especially beneficial in applications with high I/O operations, such as web servers handling multiple database queries simultaneously.
Non-Blocking I/O: Asynchronous operations do not block the main thread, allowing other tasks to continue running. This results in better CPU utilization and smoother application performance, particularly under load.
Scalability: Asynchronous code scales better with the number of concurrent operations. For example, a web application using Motor can handle more simultaneous requests compared to one using PyMongo.
Resource Efficiency: Asynchronous operations can lead to more efficient use of system resources. For instance, the event loop in asyncio allows the application to switch between tasks, reducing idle times and improving overall efficiency.

Source Code and Video Explanation

You can find the source code on Github repository below:

PylotStuff / motor-asyncio-performance

Boost DB ops performane with AsycioMotorClient & Asyncio & Uvloop

motor-asyncio-performance

Boost DB ops performane with AsycioMotorClient & Asyncio & Uvloop

YouTube Video

Getting Started

Run Mongodb in Docker

docker-compose up -d

Set up Virtual Environment & Install Requirements

python3 -m venv venv
source venv/bin/activate
pip3 install -r requirements.txt

Populate product data as a source collection

python3 populate.py

Now, you should have 1.8M document in the source collection. What's next?

Benchmark Write Operations

python3 db-migrate-slow.py

python3 db-migrate-fast.py

Benchmark Read Operations (Existence Cheking)

With field existence vs assing default value

python3 db-bad-document-structure.py

python3 db-good-document-structure.py

View on GitHub

Youtube - Boost MongoDB Performance: Motor Client vs PyMongo - Which is Faster?

Conclusion

The choice between Motor and PyMongo depends on the specific needs of your application. For applications that require high concurrency and efficient I/O handling, Motor and the asynchronous approach offer significant advantages. However, for simpler applications or scripts where ease of implementation is a priority, PyMongo's synchronous approach might be sufficient.

By leveraging asynchronous operations with Motor, you can build more scalable and performant applications, making it a worthwhile consideration for modern web development.

Strategies I Used To Land My First Job as a Software Engineer

Rashid — Tue, 28 May 2024 18:22:23 +0000

I discovered two main methods that worked for me to land a remote work as software engineer. The Machine Gun Strategy and the Sniper Approach. By understanding and applying these techniques, hopefully you'll be well on your way to securing your first software engineering position.

This post cross-published with OnePublish

The Machine Gun Strategy

The "Machine Gun Strategy" involves sending out as many resumes as possible, targeting companies within a specific industry that aligns with your interests and projects. This approach might seem overwhelming, but it's all about maximizing your exposure.

Why It Works

Volume and Visibility: By applying to a large number of positions, you increase the likelihood that your resume will be seen by multiple hiring managers.
Industry Alignment: Targeting a specific industry helps you tailor your resume and cover letter to showcase relevant skills and experiences, making you a more attractive candidate.
Persistent Presence: Continuously applying ensures that your name stays fresh in the minds of recruiters, increasing the chance of a response.

How to Implement the Machine Gun Strategy

Identify Your Target Industry: Start by pinpointing the industry that aligns with your skills and interests. Whether it's fintech, health tech, or gaming, having a clear focus helps streamline your search.
Tailor Your Resume and Cover Letter: Customize your application materials for each job. Highlight relevant projects, experiences, and skills that match the job description and realted industry.
Leverage Job Boards and Networks: Utilize platforms like LinkedIn, Indeed, Glassdoor, and industry-specific job boards. Also, tap into your professional network for leads.
Track Your Applications: Use a spreadsheet or a job application tracker tool to keep track of where you've applied, the status of each application, and follow-up dates.
Follow Up: After applying, don't hesitate to send a follow-up email to express continued interest in the position. It shows initiative and keeps your application top-of-mind.

The Sniper Approach: Precision Targeting with Cold Emails

While the Machine Gun Strategy focuses on volume, the Sniper Approach is all about precision. This method involves reaching out directly to companies or industry professionals with cold emails, explaining your interest, and expressing your willingness to learn, even if it means starting as an intern.

Why It Works

Direct Connection: Cold emails allow you to bypass traditional application processes and connect directly with decision-makers.
Personal Touch: A well-crafted email can convey your enthusiasm, personality, and fit for the company more effectively than a standard application.
Demonstrates Initiative: Taking the initiative to reach out shows that you're proactive and genuinely interested in the company.

How to Implement the Sniper Approach

Research Target Companies: Identify companies that not only interest you but also align with your career goals and values. Look for companies that are known for hiring entry-level engineers or offering internships.
Find the Right Contacts: Use LinkedIn, company websites, and professional networks to find the contact information of hiring managers, team leads, or department heads.
Craft a Compelling Email: Your email should be concise, personalized, and professional. Here’s a basic structure to follow:
- Subject Line: Keep it clear and relevant.
- Introduction: Briefly introduce yourself and mention how you found their contact information.
- Body: Explain why you're interested in the company and how your skills and experiences align with their needs. Mention any relevant projects or achievements.
- Call to Action: Politely ask for an opportunity to discuss potential openings or internships. Provide your contact information and availability.
Follow Up: If you don't hear back within a week or two, send a polite follow-up email to reiterate your interest and availability.

Combining Both Strategies for Maximum Impact

While both strategies have their strengths, combining them can amplify your job search efforts. Here’s how to effectively merge the Machine Gun Strategy and the Sniper Approach:

Initial Wide Reach: Start with the Machine Gun Strategy to get your resume in front of as many eyes as possible. This can help you get initial responses and possibly some interviews.
Targeted Follow-Up: For companies you’re particularly interested in or haven’t heard back from, use the Sniper Approach to send a personalized cold email. This dual effort ensures you're not only casting a wide net but also making precise, targeted connections.
Leverage Networking Opportunities: Attend industry events, webinars, and meetups to meet professionals and learn about job opportunities. Networking can provide leads that complement both strategies.
Continuous Improvement: Regularly update and refine your resume, cover letters, and email templates based on feedback and results. This iterative process will help you improve your chances over time.

Conclusion

Landing your first job as a software engineer might seem challenging, but with the right strategies, you can increase your chances of success. The Machine Gun Strategy helps you maximize your reach, while the Sniper Approach allows you to make direct, meaningful connections. By combining both methods and staying persistent, you'll be well on your way to starting your career in software engineering.

I made a browser extension that cross-publish post from Notion to DEV, Hashnode, Medium and more

Rashid — Mon, 22 Apr 2024 18:40:18 +0000

Hi everyone, let me introduce you OnePublish which is a cross-publishing tool that allows you to manage all your content in Notion and cross-publish it with few clicks.

Inspiration and Idea 💫

I have been posting tech articles since 2018, but I have always been too lazy to publish them on all of my blogging platforms.

Yeah.. you guessed it right, I am too lazy for 15 minutes of dedicated work so I spent 5-6 month to build this chrome extension to make publishing easier.

All your content in Notion 🗒️

Have you ever lost your articles? I have been through this, and it was hard for me to rewrite everything. I felt like there was no single place where I could manage all my content.

No Complex Dashboards. Simple Extension as a Gateway to Publishing. 🎯

I was thinking of making something simple and easy-to-use where there is no need to register on some website and then get lost inside the dashboard.

That idea lead me to make a chrome extension where user don't need to navigate to separate website and manage all the content there.

In the current state of OnePublish, users can connect to Notion with an OAuth flow (few clicks) and then it's ready to use.

I am using my own product 😎

Of course I am using it. I already boosted my SEO with canonical links and domain rating is increasing because of backlinks. I am cross-publishing to here DEV, Hashnode, Medium, and Ghost CMS. And all my content is in managed in Notion.

See it in action

Here's how I am cross-publishing with OnePublish:

Waiting for your feedbacks :)

What I learned from managing hundred millions of data in MongoDB

Rashid — Sat, 20 Apr 2024 18:16:28 +0000

In this post, I will share real experience that I gained while working with hundred's of millions of pieces of data in MongoDB.

This post cross-published with OnePublish

Don't store all data in a single mongo collection

This was the worst mistake that we made, which became the root cause of all issues. We used to store 500 million complex structured data in single collection, that caused following side effects:

Indexes creation takes very long time.
If collection dropped accidentally, all data is gone.
Querying data takes longer than usual, even though with indexes.
Counting with filters is timing out because of large document scanning.

Probably there will be even more issues the data increases on a daily basis.

Bad document structure

Invest some time to define document structure precisely. This is a critical part that never gets attention when starting the project. Our mistake was excluding the field entirely from some of the documents instead of assigning a default value for the field.

That ended up with slow query scanning because of existence checking. In MongoDB, it's possible to give partial indexes but it's really not working that well where field existence is checked.

The example below shows null value assigned for phone instead of excluding it from the document completely.

{
  "_id": "507f191e810c19729de860ea",
  "name": "John Doe",
  "age": 30,
  "email": "john.doe@example.com",
  "phone":null, //assign default value instead of removing
  "isActive":true,
  "tags": ["user", "active", "new"]
  "registeredOn": "2024-04-19T15:00:00Z"
}

{$exists: true} filter is slow so consider adding default value and index the fields for fast query results.

Use Bulk Operations

Bulk operations are used to execute multiple write operations (inserts, updates, deletes) efficiently in a single command. This method can significantly reduce the number of round trips between your application and the MongoDB server, leading to performance improvements, especially when dealing with large volumes of data.

Since we were to processing millions of data that from RabbitMQ queues it requires to handle messages in batches and make bulk DB operations for each batch.

Here's the documentation for pymongo how to use bulk operations for better write performance.

Use Indexes

Indexing is playing crucial part in DB performance. Create required indexes in collection based on what filters you are using in the codebase. That also refers the part about creating valid document structure. Instead of removing the field completely assign default value to benefit from indexing power.

Instead of count use aggregation pipeline

If you need get count of data based on specific filter I wouldn't recommend using count() , instead use combination of $match and $count aggregation for fast performance.

$match filters the documents first, which means only the relevant documents that meet the specified conditions are passed down to the $count stage. This is more efficient than counting all documents and then filtering them, as count() might initially do without any conditions.

Wrap up

That's all what I have been experienced so far with MongoDB. I will update the article once I find more hints and best practices to help optimise the performance.

Debugging Setup Guide for Python Projects

Rashid — Tue, 19 Mar 2024 20:41:18 +0000

In this post, I will show you proper debugging of a real-world project. Debugging is not adding print or log statements and rely on output of those methods. There could be underlying cause of a bug that is not visible without debugging the project.

We will set up debugger for the project by testdriven.io that processing asynchronous tasks with FastAPI and Celery. You can clone the repository from below:

testdrivenio / fastapi-celery

Example of how to handle background processes with FastAPI, Celery, and Docker

Asynchronous Tasks with FastAPI and Celery

Example of how to handle background processes with FastAPI, Celery, and Docker.

Want to learn how to build this?

Check out the post.

Want to use this project?

Spin up the containers:

$ docker-compose up -d --build

Open your browser to http://localhost:8004 to view the app or to http://localhost:5556 to view the Flower dashboard.

Trigger a new task:

$ curl http://localhost:8004/tasks -H "Content-Type: application/json" --data '{"type": 0}'

Check the status:

$ curl http://localhost:8004/tasks/<TASK_ID>

View on GitHub

Identifying Dependencies

Each project has dependencies such as databases, queues, message brokers or any other external service layer that needs to be considered when setting of debugging. Because the core of our application needs to interact with these services to have entire logic working.

The project we have right now requires to interact with Celery and Redis. So, we can use Docker containers to run and up required services in seconds and make it ready to use.

That’s the beauty of Docker allows you to run multiple services isolated from your environment and works perfectly with local development environment setups.

There are already docker-compose.yml file inside the project which manages the containers and makes it easy to setup local environment. The following services are listed in compose file:

web - FastAPI container runs the core logic of application
worker - Celery workers
redis - Redis
dashboard - Flower dashboard to monitor and manage Celery jobs and workers.

Preparing Dependencies

First, when we open project we can see a folder named project that hold FastAPI application or core logic under it and docker-compose.yml to run containers.

Now, we want to debug the internal logic of our core FastAPI application, we will exclude the web and instead run it through debugger. The FastAPI will be launched from VSCode debugger with all external running dependencies inside Docker and we will map ports of dependency services to our local machine in order to able to connect them outside of Docker network.

Also, we want to debug celery worker which means worker container also has to be launched by debugger.

Let’s create a separate docker compose file for debugging purposes.

docker-compose-dev.yml

version: '3.8'

services:

  redis:
    image: redis:7
    ports:
      - 6379:6379

  dashboard:
    build: ./project
    command: celery --broker=redis://redis:6379/0 flower --port=5555
    ports:
      - 5556:5555
    environment:
      - CELERY_BROKER_URL=redis://redis:6379/0
      - CELERY_RESULT_BACKEND=redis://redis:6379/0
    depends_on:
      - redis

In this compose file we only have redis with mapped ports to our local machine and Flower dashboard for task monitoring.

Now, you can run the required services by following command:

docker-compose -f docker-compose-dev.yml up -d

Setting up VSCode Debugger

Now we have containers running, it’s time set up debugger for the project.

Navigate debugging screen of VSCode (locates on left sidebar) you will see create a launch.json file option where it automatically creates a starter configuration based on project.

Once you have launch.json created replace the entire configuration with one below:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python Debugger: FastAPI",
            "type": "debugpy",
            "request": "launch",
            "module": "uvicorn",
            "args": [
                "main:app",
                "--reload"
            ],
            "env": {
                "PYTHONPATH": "${workspaceRoot}/project",
                "CELERY_BROKER_URL": "redis://localhost:6379/0",
                "CELERY_RESULT_BACKEND": "redis://localhost:6379/0"
            },
            "jinja": true,
            "justMyCode": false
        },
        {
            "name": "Celery Worker",
            "type": "python",
            "request": "launch",
            "module": "celery",
            "console": "integratedTerminal",
            "justMyCode": true,
            "env": {
                "PYTHONPATH": "${workspaceRoot}/project",
                "CELERY_BROKER_URL": "redis://localhost:6379/0",
                "CELERY_RESULT_BACKEND": "redis://localhost:6379/0"
            },
            "args": [
                "-A",
                "worker.celery",
                "worker",
                "--loglevel=info",
                "--logfile=project/logs/celery.log"
            ],
        }
    ]
}

There are key points need to consider when creating debug configurations:

Define PYTHONPATH environment variable to point the location of Python libraries. The folder named project is the parent module so we set it as python path.
The application internally using environment variables to connect dependencies. That means we need to set all required environment variables properly to make app work. We mapped redis to port 6379 on our local machine that’s why we’re not using container name directly because debugger launching from our local machine and has no idea of internal docker network.
The args is holding arguments that defined in docker-compose file that passed main command to run specific service. Both FastAPI and Celery Worker using exact same arguments defined in docker compose file but in a way that debugger configuration supports.

Starting Debugger and Refactoring Codebase

The last steps are to create virtual environment with all installed packages from requirements.txt and make a slight tweak inside the app to make it run properly.

Now, if you start the debugger for FastAPI it will throw an error:

RuntimeError: Directory 'static' does not exist

That means, python can’t find the static directory even we defined PYTHONPATH in debugger configuration. So, let’s take a look the code:

project/main.py

app = FastAPI()
app.mount("/static", StaticFiles(directory="static"), name="static")
templates = Jinja2Templates(directory="templates")

Seems StaticFiles class can not initialise properly. However, it works fine when running the app from Docker (web service) because of volume mapping (take a look to docker-compose file). It means that the directory ./project on the host machine will be mounted into the directory /usr/src/app within the container. So, just passing a directory name is enough to find it in container.

For debugging purposes, we will slightly refactor the code and provide correct path to find static directory within our local machine:

app = FastAPI()
app.mount("/static", StaticFiles(directory=f"{os.environ.get('PYTHONPATH')}/static"), name="static")
templates = Jinja2Templates(directory="templates")

Concatenation of PYTHONPATH and /static will do the trick and create and absolute path to the required folder. Try to start the FastAPI debugger again and it will work smoothly. Then we can also run Celery Worker debugger as well to debug celery worker.

That’s it and our application is ready to debug by setting breakpoints to required places.

Wrap up

We created debugging environment for a random project on internet. It’s similar in real-world applications where you need to identify the dependencies, setting up debugger configuration with proper environment variables, launch types, commands, arguments and make slight refactoring in the core logic if necessary.

The main goal is to make the sure the part or service you are debugging is communicating properly with it’s dependencies.

This post cross-published with OnePublish

Build a Phishing Site Detector with NextJS and VirusTotal

Rashid — Fri, 01 Mar 2024 17:37:32 +0000

In this new series we will build unique and creative stuff and ship it to production. So, in this episode we’re going to build a simple app that will check phishing risk against provided URL.

Why this project is great?

Because it involves many skills those required to have as a developer:

TypeScript.
Third-party integration.
Use of NextJS itself.
UI/UX skills.
Tailwind CSS (eh)
Deployment

What we are building?

This project is going to be a single page app that will let user to enter a URL. After user enters the URL, it should extract the domain and send it to third-party service which will check the phishing risk of that domain. Then, on the backend we need to evaluate the response from that service and display the result to the user with nice UI/UX interface.

Getting Started

Let’s start by creating a new NextJS project. I will be using latest version of NextJS which is version 14 at time of writing this post.

npx create-next-app@latest

It will prompt few questions to configure the NextJS project. I will select to use TypeScript, initialise the Tailwind CSS and use App router.

✔ What is your project named? … <your_app_name_here>
✔ Would you like to use TypeScript? … No / <u>**Yes**</u>
✔ Would you like to use ESLint? … No / <u>**Yes**</u>
✔ Would you like to use Tailwind CSS? … No / <u>**Yes**</u>
✔ Would you like to use `src/` directory? … <u>**No**</u> / Yes
✔ Would you like to use App Router? (recommended) … No / <u>**Yes**</u>
✔ Would you like to customize the default import alias (@/*)? … No / <u>**Yes**</u>

Now you should have initialised NextJS project ready to use.

Integration with VirusTotal

Virus Total is an online service that analyses suspicious files and URLs to detect types of malware and malicious content using antivirus engines and website scanners.

Create an account on VirusTotal to get an API key for interacting with their service programmatically.

Since we want to check domain safety we had to use Domain Report endpoint to gather deep analyse of the submitted domain.

We can test the endpoint with Postman to see which part of the response we need to calculate the risk of phishing. Here’s the results:

There are lots of useful information on domain analysis based on various needs. But for now, we will continue with selected part which will help us to calculate the risk.

Building Backend

Since we have all information about what we’re going to build, it’s time to put these knowledge into practice. First, create a new API route that will send request to VirusTotal:

app/api/virustotal/route.ts

import { calcRisk } from '@/utils/calcRiskcomponents'
import axios from 'axios'
import { type NextRequest } from 'next/server'
import { getDomain } from 'tldts'

export async function GET(request: NextRequest) {
    const urlToAnalyze = request.nextUrl.searchParams.get('urlToAnalyze')
    if (!urlToAnalyze) {
        return new Response("URL must not be empty.", {status: 400})
    }
    try {
        const domain = getDomain(urlToAnalyze)
        const virustotalRes = await axios.get(`https://www.virustotal.com/api/v3/domains/${domain}`, {
            headers: {
                "x-apikey": process.env.VIRUSTOTAL_API_KEY
            }
        })
        const stats = calcRisk(virustotalRes?.data?.data?.attributes?.last_analysis_stats)
        return Response.json({ stats: stats })
    } catch (err) {
        return new Response("Internal Server Error", {status: 500})
    }

}

This route will handle the GET request that will be sent from frontend part of our app. Let’s break down the code:

Validating if required search param named urlToAnalyze is present. This param should be URL that provided by user.
If urlToAnalyze is present then we will extract the domain from that URL using npm module named tldts.
Then, based on VirusTotal docs we’re sending the domain to gather analysis report. Also we’re including the API Key in the headers.
We have a helper util function calcRisk which will evaluate the result of last_analysis_stats part from the response.

So, the calcRisk is a simple helper function that will determine if URL is risky or not. You can change the logic as you want such as assigning weights to the states of analysis. I would mark the domain “High Risk” even if there are only single report from antivirus engines or scanners.

So here’s the code for better understanding:

utils/calcRisk.ts

interface VirustotalStatsForDomain {
    harmless: number,
    malicious: number,
    suspicious: number,
    undetected: number,
    timeout: number
}

export function calcRisk(virsusTotalStatsForDomain: VirustotalStatsForDomain) {
    const { malicious, suspicious } = virsusTotalStatsForDomain;

    if(malicious > 0) {
        return "High Risk"
    }

    if (suspicious > 0) {
        return "Moderate Risk"
    }

    return "Low Risk"
}

Cool, that’s all for backend. Now, let’s move on to frontend part.

Building Frontend

We’re using app router in our NextJS project, so we should create the pages as <folder-name>/page.tsx which in this case I will create it as check-site/page.tsx. That will be routable on browser as /check-site.

If I would be brutally honest, Tailwind CSS is something that I don’t prefer but it’s useful for small projects like this.

Well, here’s the code for frontend:

/app/check-site/page.tsx

"use client"

import Hero from "@/components/Herocomponents"
import axios from "axios"
import { useState } from "react"

export default function Page() {
    const [url, setUrl] = useState<string>("");
    const [analyzeResult, setAnalyzeResult] = useState<string | null>(null)
    const handleUrlAnalyze = async () => {
      const res = await axios.get(`/api/virustotal?urlToAnalyze=${url}`)
      setAnalyzeResult(res?.data?.stat)
    }

    const icon: any = {
      "Low Risk": <svg xmlns="http://www.w3.org/2000/svg" width="150" height="150" viewBox="0 0 24 24" fill="none" stroke="green" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" className="lucide lucide-shield-check"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10"/><path d="m9 12 2 2 4-4"/></svg>,
      "Moderate Risk": <svg xmlns="http://www.w3.org/2000/svg" width="150" height="150" viewBox="0 0 24 24" fill="none" stroke="yellow" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" className="lucide lucide-shield-alert"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10"/><path d="M12 8v4"/><path d="M12 16h.01"/></svg>,
      "High Risk": <svg xmlns="http://www.w3.org/2000/svg" width="150" height="150" viewBox="0 0 24 24" fill="none" stroke="red" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" className="lucide lucide-shield-x"><path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10"/><path d="m14.5 9-5 5"/><path d="m9.5 9 5 5"/></svg>,
    }

    return (
      <section className="flex flex-col justify-between bg-white dark:bg-gray-900 min-h-screen">
        <div>
          <Hero />
          <div className="px-4 mx-auto max-w-2xl flex flex-col items-center justify-center">
            <div className="w-full flex flex-col items-center">
              <div className="sm:col-span-2 w-full">
                <label className="block mb-2 text-md font-medium text-gray-900 text-center dark:text-white">Domain or URL to Analyze</label>
                <input type="text" name="name" id="name" className="bg-gray-50 border border-gray-300 text-gray-900 text-sm rounded-lg focus:ring-primary-600 focus:border-primary-600 block w-full p-2.5 dark:bg-gray-700 dark:border-gray-600 dark:placeholder-gray-400 dark:text-white dark:focus:ring-primary-500 dark:focus:border-primary-500" placeholder="Type domain to check"
                onChange={(e) => setUrl(e.target.value)}
                />
              </div>
            </div>
            <button type="button" className="inline-flex items-center px-5 py-2.5 mt-4 sm:mt-6 text-sm font-medium text-center text-white bg-blue-900 rounded-lg hover:bg-primary-800"
            onClick={handleUrlAnalyze}
            >
              Check URL
            </button>
          </div>
          <div className="flex flex-col justify-center items-center w-full py-5">
            {analyzeResult && icon[analyzeResult]}

            <h3 className="text-md">{analyzeResult}</h3>
          </div>
        </div>

        {/* Image container positioned at the bottom */}
        <div className="flex justify-center items-center w-full py-5">
          Made with
          <img className="ml-1" src="https://upload.wikimedia.org/wikipedia/commons/thumb/b/b7/VirusTotal_logo.svg/2560px-VirusTotal_logo.svg.png" width={100} alt="VirusTotal logo" />
        </div>
      </section>

    )
  }

The only hidden part in the code is <Hero /> component which is below:

components/Hero.tsx

const Hero = () => {
    return (
        <section className="bg-white dark:bg-gray-900">
            <div className="py-8 px-4 mx-auto max-w-screen-xl text-center lg:py-16 lg:px-12">
                <h1 className="mb-4 text-4xl font-extrabold tracking-tight leading-none text-gray-900 md:text-5xl lg:text-6xl dark:text-white">Secure Your Surfing</h1>
                <p className="mb-8 text-lg font-normal text-gray-500 lg:text-xl sm:px-16 xl:px-48 dark:text-gray-400">Navigate the web with confidence using instant site safety checker.</p>
                <div className="flex justify-center">
                    <img className="h-auto max-w-full" src="logo.png" width={200} alt="image description" />
                </div>
            </div>
        </section>
    )
}

export default Hero;

And yes, I generated the tailwind code with ChatGPT.

Result

Now, we have full working, single feature app that will analyse the phishing risk of submitted URL. Here’s how it looks when you run the app:

What’s next?

You need to deploy your NextJS to Vercel to make it publicly accessible for others. Also, remember to explain your solution well in ReadMe file since it shows that you’re documenting your work.

PylotStuff / phishalayzer

Single feature app to check phishing risk of URLs

This is a Next.js project bootstrapped with create-next-app.

Getting Started

First, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.

This project uses next/font to automatically optimize and load Inter, a custom Google Font.

Learn More

To learn more about Next.js, take a look at the following resources:

Next.js Documentation - learn about Next.js features and API.
Learn Next.js - an interactive Next.js tutorial.

You can check out the Next.js GitHub repository - your feedback and contributions are welcome!

Deploy on Vercel

The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.

Check out our Next.js deployment documentation for more details.

View on GitHub

Cross-Published with OnePublish.

Best Practices for Pull Request Management and Code Review

Rashid — Tue, 30 Jan 2024 09:46:00 +0000

Let's go through the best practices I noted during my experience for both authors and reviewers to create an efficient code review workflow.

Don't rely on the reviewer to find bugs

The reviewer should not be considered as Quality Assurance (QA) to find bugs and errors in the pull request. The author needs to carefully review the code on their own before requesting a review. This doesn't mean the PR should be perfect and error-free.

We're all human, and it's possible for things to slip past us. That's where the reviewer comes in as a second pair of eyes to check for any missing parts. However, solely relying on the reviewer and not checking anything in the code before creating a PR will waste time for both sides. It will result in multiple back-and-forth changes and discussions, ultimately delaying the delivery.

Make sure there are no blockers for a reviewer

If your project has CI/CD workflows or pipelines, ensure that all checks pass before requesting a review. Sometimes authors are overly confident that checks will pass and request a review immediately, without waiting for the status. However, when reviewers open the pull request, they discover failed checks and need to communicate with the author to fix them.

This not only applies to CI/CD, but also to situations where there are merge conflicts or outdated commits in the pull request, making it difficult to review and considered as uncompleted work.

Provide a clear and detailed description

The reviewer needs to understand the goal of the pull request. Poor descriptions make it difficult to understand the business logic behind the code, and this negatively impacts the review process.

The best way to provide a detailed description is to use templates and helper indicators to provide more context for the reviewer. For instance, defining the goal of the PR, specifying its desired outcome, including technical notes and labels to indicate whether the PR is ready for review or still in progress, and so on.

Check out BattlePlan ticket templates for examples.

Be considerate when providing feedback in review comments

When reviewing code, it is important to consider the author's feelings and avoid offending them with your comments. If the reviewer has suggestions for improvement, it is best to phrase them in a non-confrontational manner. Instead of directly instructing the author to make changes, it is recommended to start sentences with phrases like "I would suggest to...", or "It could be more efficient to..." and continue explaining the reason behind the suggested changes.

No one is perfect, and demonstrating superiority on a pull request is the worst toxic behavior a reviewer can exhibit. If you find yourself in such situations, it is best to directly inform the reviewer that their communication style is unprofessional.

Sanity checks might not be enough

If it is a large pull request that potentially introduces breaking changes, relying solely on sanity checks might not be sufficient. This is because the actual logic needs to be tested locally to ensure that it aligns with the provided requirements.

It seems that you understand all the steps and the code appears to be fine. However, there may be underlying issues that are not visible until the entire application is run locally and thoroughly debugged.

Dealing with a procrastinating reviewer

If you have created a pull request and your reviewer is procrastinating on it, here is the best way to handle the situation.

This is a joke. Do not try at work.

Conclusion

By following these practices, authors and reviewers can work together effectively and deliver high-quality code. Remember, the goal of code review is to improve the code and collaborate as a team not compete over who writes better code or flex on each other.

I built a chrome extension that can cross-publish Notion page to other blogging platforms

Rashid — Mon, 29 Jan 2024 21:39:51 +0000

I love writing tech articles but I am too lazy to publish it all of my blogging platforms.

Yeah.. you guessed it right👉, I am too lazy for 15 minutes of dedicated work so I spent 5-6 month to build this chrome extension🫠

Anyway.. I always use Notion to manage my content and other personal goals. Then I thought… why not organising all my content on Notion and publish the page directly from there?

After hitting the wall too many times, I came up with concluded solution. Well, of course it’s not purely dependent on me because each platform has its own API and some of them are still need improvements. But I tired my best to use all of features that API’s provide.

Is that all? No. I am planning to add more platforms there based on feedback (adding reddit might be good🤔). So, I would love to hear your feedback on extension and how it performs and what other platforms I can integrate.

Because it’s newly published, anyone from my connections can reach me out from DM and ask for few extra credits for free. I always appreciate supportive community and fellow devs out there.🫡

OnePubli.sh

// Detect dark theme var iframe = document.getElementById('tweet-1752049887238103296-633'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1752049887238103296&theme=dark" }