Forem: Akash Rajpurohit

Getting paid in foreign currency as a software developer in India

Akash Rajpurohit — Fri, 21 Feb 2025 11:24:01 +0000

Introduction

As the world becomes increasingly remote-first, more Indian software developers are finding opportunities to work with clients and companies based outside India. While this opens up exciting possibilities for career growth and better compensation, it also comes with its own set of challenges - particularly around receiving payments and staying compliant with Indian regulations.

After spending considerable time researching this topic, going through countless blogs, Reddit threads, and discussions with fellow developers and professionals, I realized there wasn't a comprehensive guide that addressed all aspects of receiving foreign income as an Indian developer. This blog post aims to fill that gap.

This article is based on my personal research and experience. While I strive to provide accurate information, I am not a Chartered Accountant (CA) or tax expert.

Tax laws and regulations can change, and individual situations may vary. Please consult with a qualified CA or tax professional for advice specific to your situation.

Receiving Payments from Foreign Clients

One of the first challenges you'll face is deciding how to receive payments from foreign clients. The reason why this is a challenge and tricky thing to do is because you are dealing with foreign currency and at the end of the day, you need to convert that foreign currency to Indian Rupees.

If you are doing this less frequently then maybe you don't need to worry much about this, but if you are going to be receiving payments from foreign clients on a regular basis then you need to find a way to optimize this process as much as possible.

The reason is simple, a small difference in currency exchange rates, transaction fees, bank charges and processing fees and time can add up to a significant amount of money over time (as much as 4-5% of the total amount which can be a lot depending on your volume of transactions).

With that being said, let's look at the options available to you.

1. Direct Wire Transfer (with SWIFT code)

The traditional method of receiving foreign payments directly to your bank account. How this process works is that you will open a current account in your name which has global trade account setup.

Once that is done, you will be provided with the account number and the SWIFT code of the bank. You can then use this information to receive payments from your foreign clients.

Pros:

No intermediary fees
Direct deposit to your bank
Transparent process

Cons:

High transfer fees (both sender and receiver)
Poor exchange rates
Longer processing time
Complex documentation requirements

To find what exchange rate you will be getting from different banks, search for "TT buying rate bank-name" on Google. Each bank releases their daily exchange rates on their website. Check the rates for your foreign currency and see how much less than the market rate you will be getting.

Now, coming back to the reality, what you can do instead is you can go to the bank, talk to the manager (or any representative who handles the foreign income transactions) and ask them for a better rate. Based on the volume of the transaction you plan to do, they might give you a better rate.

So if you are planning to do it via this way, then I would highly recommend that you talk and get the best rate possible. You can even get 15-20 paise less than the current market rate if you are doing a large volume of transactions. To compare this with standard market rate, you might be getting 1-1.5 rupees less than the market rate which is awfully low by default.

2. Skydo

A relatively new player in the market, Skydo is designed for Indian freelancers and agencies receiving foreign payments.

Few things that I found quite interesting about them is firstly they give you the live FX rates (which means that what you see is what you get) and their pricing model is very simple with a flat fee based on the amount of money you are receiving which makes it very transparent and easy to understand.

Pros:

Competitive exchange rates
Lower fees compared to traditional banks
Designed specifically for Indian businesses
Quick setup and easy documentation
Direct integration with accounting software
Excellent customer support (I can vouch for this personally)

Cons:

Limited to specific countries
Relatively new service

The flow will look like this:

You will be provided with a virtual bank account number depending on the currency and country you are receiving the payment from.

For example, if you are receiving the payment from the USA, you will be provided with a USA bank account number, ACH / Fedwire details etc.

Transfer the money to your Skydo account using those details.
Link your Indian bank account to your Skydo account by providing the account number and IFSC code.
Once you receive the payment, you will be able to see the amount in your Skydo dashboard which will get converted and transferred to your Indian bank account.

3. Wise (Formerly TransferWise)

A popular modern solution for international transfers. This was probably the first one that comes into mind when you think about receiving payments from foreign clients.

Pros:

Better exchange rates than banks
Lower fees
Transparent pricing
Multiple currency support

Cons:

May require additional documentation
Some banks may have issues with Wise transfers
Not all countries supported
Recently their sign up process has become quite of a hassle for indian users and hence I have not been able to test it out myself.

4. PayPal

While popular globally, PayPal might not be the best option for Indian recipients for larger amounts but it is still a good option to consider for smaller and infrequent transactions.

Pros:

Widely accepted
Easy to set up
Good for small amounts

Cons:

High fees (from my personal experience its the worst, even compared to the direct wire transfer)
Poor exchange rates
Funds can be held without notice
Withdrawal to Indian banks can be slow

Understanding Tax Implications

Section 44ADA: A Blessing for Professionals

If your gross receipts are under ₹75 lakhs per year, Section 44ADA of the Income Tax Act can be particularly beneficial.

Key points about 44ADA:

Presumptive Taxation:

50% of your gross receipts are considered as profit
Remaining 50% is assumed as business expenses
No need to maintain detailed books of accounts

Eligibility:

Applicable to professionals (including software developers)
Gross receipts should not exceed ₹75 lakhs in a financial year (provided 95% of the income is via online mode and not cash)
Only available to individuals and partnership firms (not companies)

Benefits:

Simplified tax filing
No need for audit
Can still claim actual expenses if they're lower than 50%

Beyond ₹75 Lakhs: Regular Taxation

If your income exceeds ₹75 lakhs, you'll need to:

Maintain Books of Accounts:

Regular books of accounts required
Need professional help (CA) for maintenance
Annual audit mandatory

Actual Expense Deduction:

Can claim actual business expenses
Need to maintain proper documentation
Common deductible expenses include:
- Home office expenses
- Internet and phone bills
- Computer equipment and software
- Professional development courses
- Travel expenses (if applicable)

One important thing to note here is that you need to make sure whatever platform you use, it is compliant with the Indian regulations and laws and also that you are receiving FIRA/FIRC document from the bank for each of your transactions.

This document is required by the Indian government to prove that the money you are receiving is from a foreign source and not from an Indian source and will be very much needed when you file your taxes and want to claim the foreign income under 44ADA.

PS: With Skydo, you don't need to worry about this as they will provide you with the FIRA document for each of your transactions.

GST Registration and Compliance

When to Register for GST

GST registration is mandatory if you're providing services to foreign clients (considered as export of services) and you are earning more than 20 lakhs in a financial year.

While you will be registering for GST, you need not worry about charging your foreign clients for the GST as it will be zero rated for you. To enable this you will be applying for the LUT (Letter of Undertaking) from the GST portal for every financial year.

Key points about GST:

Registration Requirements:

Mandatory regardless of turnover for export services
Need to register in the state where you're based

Documentation Needed:

PAN card
Aadhaar card
Bank account details
Address proof
Passport size photograph

GST Compliance Requirements

Once registered, you need to:

File Regular Returns:

GSTR-1 (monthly/quarterly)
GSTR-3B (monthly/quarterly)
Annual returns

Maintain Records:

Invoices
Payment receipts
Bank statements
Other relevant documents

Zero-rated Supply:

Export of services is considered zero-rated
No GST charged to foreign clients
Can claim input tax credit

Consider using accounting software that supports GST compliance and can generate GST-compliant invoices. This will save you significant time and reduce errors in compliance.

Best Practices and Tips

Separate Bank Account:

Maintain a separate bank account for business transactions
Makes accounting and tax filing easier
Better tracking of income and expenses

Documentation:

Keep all client agreements in writing
Maintain proper invoices
Document all business expenses
Keep foreign inward remittance certificates (FIRC)

Professional Help:

Engage a CA for tax planning and compliance
Consider using professional accounting software
Stay updated with regulatory changes

Currency Management:

Consider maintaining a multi-currency account
Plan for currency fluctuations
Use forward contracts if dealing with large amounts

Regular Compliance:

File returns on time
Pay advance tax if applicable
Keep track of important dates and deadlines

Tools and Resources

Here are some tools and resources that can help you manage your finances better:

Payment Platforms:

Tax Resources:

Conclusion

Working with foreign clients can be rewarding both professionally and financially. While the initial setup and compliance requirements might seem overwhelming, with proper planning and professional help, you can create a smooth system for managing your international income.

Remember to:

Choose the right payment platform
Understand your tax obligations
Stay compliant with GST requirements
Maintain proper documentation
Seek professional help when needed

Have you been working with foreign clients? What challenges have you faced? Share your experiences in the comments below, or reach out to me on Twitter / Reddit.

Happy freelancing! 🚀

Increase Swap Memory on Raspberry Pi

Akash Rajpurohit — Sat, 24 Feb 2024 07:01:30 +0000

Introduction

A Swap file is a file on the disk that can be used to extend the RAM on your system. It is used when the RAM is full and your system needs more memory resources. It is also used to store the data that is not accessed frequently.

This is especially useful on Raspberry Pi with limited amount of RAM. For example, I have a Raspberry Pi Zero 2W which has 512MB of RAM. When I started adding more services on it, I could see with htop that the RAM is becoming almost full. This is where the swap file comes in handy.

The default swap file size on Raspberry Pi is 100MB. This is might be enough for some use cases, but if you need more, you can increase it.

Let's see how to do it.

Increase Swap Memory

We will use the dphys-swapfile package to increase the swap memory. This package is installed by default on Raspberry Pi OS. If you don't have it installed, you can install it with the following command:

sudo apt install dphys-swapfile

First thing to do it to turn off the swap file to avoid any issues:

sudo dphys-swapfile swapoff

Next, we will update the CONF_SWAPSIZE variable in the /etc/dphys-swapfile file. This variable is used to set the size of the swap file. The default value is 100MB. We will set it to 500MB

I am using vi to edit the file, but you can use any editor you like:

sudo vi /etc/dphys-swapfile

Search for the CONF_SWAPSIZE variable and update it to 500:

CONF_SWAPSIZE=500

Save the file and exit the editor.

Now we will reinitialize the swap file setup using the following command:

sudo dphys-swapfile setup

Finally, we will turn on the swap file:

sudo dphys-swapfile swapon

Once this is done, we can reboot the system to make sure everything is working fine:

sudo reboot

After the reboot, you can check the swap file size using the htop command and check for the Swp value, it should be 500M.

One thing to note is if you want to increase the swap file size to more than 2GB then you would need to uncomment the CONF_MAXSWAP variable in the /etc/dphys-swapfile file and set it to the desired value.

I have personally never needed it, but if you do, then you can do it (not verified by me).

Conclusion

In this post, we saw how to increase the swap memory on Raspberry Pi. This can be useful if you are running out of RAM on your Raspberry Pi.

I hope you liked this post. If you have any questions or feedback, reach out to me on X / Twitter or follow my blog at akashrajpurohit.com

See you in the next post. Until then, keep HomeLabbing! 👋

Fix Sudo not asking Password on Raspberry Pi

Akash Rajpurohit — Sat, 24 Feb 2024 06:54:04 +0000

Introduction

When you install a fresh copy of Debian or Raspbian on your Raspberry Pi you might notice that sudo does not ask for a password. This is a security risk and should be fixed. Let's see how to fix it.

Fix Sudo not asking Password on Raspberry Pi

This happens due to the NOPASSWD option for your user. We need to make sure that this option is not set.

We will edit the file named 010_pi-nopasswd. To find its location we will use the find command.

sudo find / -name 010_pi-nopasswd

This will return the location of the file. In my case it is /etc/sudoers.d/010_pi-nopasswd. Now we will edit this file using visudo.

sudo visudo /etc/sudoers.d/010_pi-nopasswd

The current content on the file will look like this.

<username> ALL=(ALL) NOPASSWD: ALL

Where <username> is your username. We need to remove the NOPASSWD option. So the file will look like this.

<username> ALL=(ALL) ALL

Save the file and exit. Now when you run sudo it will ask for a password.

Conclusion

We looked into how to fix sudo not asking password on Raspberry Pi. This is a security risk and should be fixed.

Follow my blog at akashrajpurohit.com

How to use ES6 import syntax in Node.js

Akash Rajpurohit — Fri, 09 Jul 2021 13:52:39 +0000

Introduction

A module is a JavaScript files that can export one or more values. These values can be variables, objects, or even functions.

JavaScript does not have built-in support for modules, but the community has created impressive work-arounds. The two most important standards are:

CommonJs Modules - What we know as the require() method for importing modules and module.exports for exporting. This is designed for synchronous loading and mainly used for server.
Asynchronous Module Definition (AMD) - The most popular implementation of this standard is RequireJS. This is designed for asynchronous loading and mainly used for browsers.

ES6 Import

The goal for ECMAScript 6 modules was to create a format that both users of CommonJS and of AMD are happy with. The syntax for importing modules looks like this..

import Express from 'express'; // for default imports

import { foo } from 'bar'; // for named imports

and exporting with named and default exports like this..

export const MAX_USERS = 20; // Named exports

function sayHi(name) {
  console.log(`Hi ${name}!`);
}

export default sayHi; // Default export

How to use it

If you try to run this code directly then you would get an error something like

SyntaxError: Cannot use import statement outside a module

Now, if you are using Node.js version 14.x.x or above then the simplest fix is to add "type": "module" in your package.json file.

{
  "type": "module"
}

However if you are on any version below 14, you would have to use Babel to compile your code before running it.

Note: Babel is not framework or platform opinionated. This means that it can be used in any project that is written in JavaScript and thus, in a Node.js project as well.

Let's start by installing some dev dependencies,

yarn add -D @babel/core @babel/preset-env @babel/node

# OR

npm install -D @babel/core @babel/preset-env @babel/node

Next is to create a babel.config.json file and add the following config,

{
  "presets": ["@babel/preset-env"]
}

Now you just have to use babel-node instead of node, so your start/dev script may look like this now

{
  "scripts": {
    "dev": "nodemon --exec babel-node server.js"
  }
}

Hope this was helpful. See you in another one 👋

Increase Root Partition Size On Fedora

Akash Rajpurohit — Mon, 01 Mar 2021 14:38:05 +0000

Introduction

I recently was in the need of extending the root partition size on my system and spent quite some time figuring out the approach to follow in doing so without losing data.

My scenario was I wanted to shrink the /home partition and use that space to increase root partition size. In this blog, I will be sharing how I was able to do the same quickly as possible.

Backup, Backup, and Backup

Always take a backup before shrinking/expanding or doing any configurations on your partitions. If you already have backups in your place then you are good to go, but If you don't then you can follow along from this tutorial to create a backup using tar command.

Now if you already have enough space, you can skip the next step of shrinking any partition to get the space available. Although in my case, I did not have extra space, I had to shrink it from the home directory.

Shrink Home

There are some prerequisites before we start the shrinking process.

To follow along, the first thing that you want to do is to switch yourself to the root user. And by switching means logging out from your non-root user and login in with username as root and your root password.

Note: You have to go into root user because your non-root (the current user) would be mounting the home directory, so for shrinking, you would have to unmount the home partition but since some processes might be using this partition it might throw you the error like

Do you want to unmount "/home" ? [Y|n] y
umount: /home: target is busy.
fsadm: Cannot proceed with mounted filesystem "/home".
  /usr/sbin/fsadm failed: 1
  Filesystem resize failed.

You can check if the target is busy or not using the fuser -mv /dev/mapper/fedora_localhost--live-home.

Remember you can check where your fedora root and home partition is mounted using fdisk -l command. In my case it is fedora_localhost--live-root and fedora_localhost--live-home)

Now to shrink this partition we will use lvresize tool.

lvresize -L -20G --resizefs /dev/mapper/fedora_localhost--live-home

This will shrink 20Gb from home.

Extend Root Partition

Now you can see available space using the vgs command.

vgs

The output will be like this

  VG                    #PV #LV #SN Attr   VSize     VFree
  fedora_localhost-live   1   3   0 wz--n- 475.35g    20g

Now we can add this extra space into the root using the same lvresize tool. But now instead of -20G we will add +20G to root.

lvresize -L +20G --resizefs /dev/mapper/fedora_localhost--live-root

And that's it, reboot your system and login with your normal non-root user and you would have your root size increased.

Hope this helped you. See you in the next one.

Backup Users Home Directory In Linux Using Tar Command

Akash Rajpurohit — Mon, 01 Mar 2021 12:33:59 +0000

Introduction

We often need to do things that may put us at risk of losing data in Linux, nevertheless, it's always better to take systematic backups from time to time.

Today we will be learning about how to use the powerful tool called tar in Linux to help in the backup process.

Tar Command

Let's take a quick look at the tar command and the flags that we will be using with it.

tar -zcvpf /[Backup_Location]/[Backup_Filename] /[User_Home_Directory_Location]

Now let's understand these flags.

z : Compress the backup file with ‘gzip’ to make it a small size.
c : Create a new backup archive.
v : verbosely list files that are processed.
p : Preserves the permissions of the files put in the archive for later restoration.
f : use archive file or device ARCHIVE.

Usage

Let us now backup the home directory, in my case, the user name is akash.

tar -zcvpf /backup/akash-backup-$(date +%d-%m-%Y).tar.gz /home/akash

This will produce the output (for the /backup directory) as

ls -lh /backup

total 15G
-rw-r--r--. 1 root root 15G Mar  1 12:09 akash-backup-01-03-2021.tar.gz

If you wish to exclude some directory to be archived, you can use --exclude flag, something like this

tar --exclude='/home/akash/Documents/test-folder' -zcvpf /backup/akash-backup-$(date +%d-%m-%Y).tar.gz /home/akash

This will archive everything excluding the test-folder inside Documents.

And that's it, this will help you create a backup and store it with the date inside /backup directory.

Bonus

Doing this once is fine, but what if there was a way to regularly backup your content and automatically wipe out the old backups, wouldn't that be great. So let's just do that.

We will use a shell script and cron job to automate this task for us.

Backup Script

First, let's move into a better directory where we will store our script.

nano /opt/scripts/home-dir-backup.sh

Now copy the mentioned script, it is pretty straightforward.

#!/bin/bash
DATE=$(date +%d-%m-%Y)
BACKUP_DIR="/backup"

# To backup akash's home directory
tar -zcvpf $BACKUP_DIR/akash-$DATE.tar.gz /home/akash

# To delete files older than 15 days
find $BACKUP_DIR/* -mtime +15 -exec rm {} \;

Set executable permissions for this file.

chmod +x home-dir-backup.sh

Now your script is ready to be used.

Cron Job

The final step is to set up the cron job to run this script automatically. We will enter into crontab using

crontab -e

Inside it paste the following lines at the bottom.

0 12 * * 5 /opt/scripts/home-dir-backup.sh

This will run this script every Friday at 12:00. To learn more about how to configure these values, I would highly recommend using crontab.guru.

Hope you found this helpful, see you in the next one.

Top Free Services To Deploy Full-Stack Applications (Back-end, Front-end, Databases)

Akash Rajpurohit — Sun, 14 Feb 2021 05:03:41 +0000

Introduction

So you have developed a full-stack web application and now you are wondering how to take it online?

Is it just a side project that you want to showcase or a small scale project to test it out with real users and get some feedback?

One question you must be having is how can I get it done in the least cost possible right? Today we will see a list of amazing top services, using which you can easily go live with your project for FREE!

We will see services for Back-end, Front-end, and even Databases 🤯. In the end, I'll drop in my personal choice and views regarding the same.

Note - I am not being paid or sponsored by any of these services. This is just advice based on my experience as a freelancer, and on how I host my web applications.

Back-End

1. Heroku

Heroku is a PaaS i.e Platform as a Service, you can deploy your backend applications very quickly and easily on Heroku. They have a free tier where you can run up to 100 backend apps at the same time for free. One catch is that you collectively have only 1000 free dyno/app-hours per month but its more than enough for a hobby or side project.

Also, Heroku has a wide variety of add-ons which makes this entire ecosystem very easy to use and setup.

2. Vercel

Vercel or Formerly known as Zeit is another great option to deploy both Backend and Frontend applications, It uses serverless architecture to set up their infrastructure for deployment. Most people use vercel to deploy frontend applications but you can also deploy backend apps with vercel. This guide shows you how to deploy Node.js API with MongoDB on vercel.

Vercel has a very generous free tier with these specs

Feature	Free	Pro
Serverless Functions Created per Deployment	12	24
Serverless Functions Created per Month	160	640
Serverless Function Duration (Seconds)	10	60

3. Firebase

Firebase is not just a hosting service but a google backed product that provides lots of features along with hosting. Firebase gives you functionality like analytics, databases, messaging and crash reporting, etc.

Along with hosting, many of the other services are also available in free tier and as you scale up, you can go ahead with their Pay-as-you-Go model.

You can host serverless functions called Cloud Functions, however recently google has announced that to create functions that use Node.js version 10 and above, you have to enable billing for the account.

4. Fly.io

Fly.io is a relatively new platform (founded in 2017) which has a free tier to test out the platform or host small side projects. They give each user \$10/mo of service credit that automatically applies to any paid service.

Type	RAM	Monthly (approx)
micro-1x	128MB	$2.67
micro-2x	512MB	$8

You can easily follow along with their quickstart guide to set up the deployment workflow using their CLI tool.

5. Freemium IaaS

I have clubbed together AWS, GCP, and Azure into this category. All three of these provide a free tier with a basic machine to set up for a year.

Read more about their services in the free tier at

These all are indeed good but sometimes can become too overwhelming for beginners by looking at their vast number of services and interconnection between them.

Front-End

1. Netlify

Netlify is an amazing service for hosting you static websites, you can easily host unlimited projects, invoke serverless functions, add a custom domain with HTTPS, Store large files, Preview Deploys and many more for free.

Recently Netlify also released Plugins Ecosystem which allows you to customize your build process.

2. Vercel

Vercel as mentioned above, not only can be used to deploy backend APIs but in fact has a better ecosystem built to deploy static frontend applications. They have solutions for Next.js, Gatsby, React, Vue, Angular, Nuxt out of the box with Zero Configuration.

Vercel also provides Edge Network or Content Delivery Network and has multiple layers of caches to ensure that your Vercel deployments are fast around the world. You can read more about it on their docs section which in my opinion is nicely structured for better understanding.

3. Github Pages

If you are a heavy user of GitHub and just want to quickly have a deployed version of your project then Github Pages is the best choice for you. Github Pages host your code directly from your GitHub repository, so any changes/edits you make and push it to the repository are automatically pushed to your live website.

Follow this amazing guide by github to set up your next project using Github Pages.

4. Surge.sh

Surge.sh is one of the simplest static site publishing providers. You can use their CLI tool to publish any project directly from the terminal or you could use its integration with various build tools like Grunt, Gulp, Travis CI and many more.

Try out surge free tier which is more than enough for any small to medium size front end application.

5. Render

Render is another great service to host your both front end and back end applications and database, Although they don't have any free tier for back end and databases (as of now) but they have a really good free tier for static web hosting.

The services are more or less similar to the other services mentioned above, so in the end, it all depends on your experience as a developer using these services.

Database

1. MongoDB Atlas

The company MongoDB itself created this service called MongoDB Atlas and provides a really good free tier with 512 MB of Storage, Shared RAM and Highly available replica sets, end-to-end encryption, automated patches, REST API.

Previously, I was using mLab as Database-as-a-Service for most of my projects, but a few months back mLab was acquired by MongoDB and for the greater good, as MongoDB provides additional services and better security rules for your shared MongoDB Databases. There is no doubt that Atlas is the best Cloud Database you can get for rapid prototyping and small to medium-sized projects.

2. Elephant SQL

When I am not using a NoSQL database like MongoDB and need an RDBMS database for any project, I mostly use PostgreSQL. Elephant SQL is a great PostgreSQL as a Service for the cloud. Although they have a variety of plans for their service but their free tier is great for getting started with the PostgreSQL database in the cloud.

3. Heroku Postgres

As I have mentioned above in the Heroku section about the additional add-ons that Heroku provides, one of them is Heroku Postgres. This is what I was using previously before I discovered about elephantSQL and I am pretty much satisfied with the services that Heroku Postgres provides.

Moreover, it's not just great but also completely free, you don't even need to add credit card details to verify your account to use this (as for some other service like Heroku Redis).

4. Redis Lab

Most of my side/client projects involve tasks where I require a fast, in-memory datastore either for caching, rate limiting or session management, for that, I mostly go with Redis. For the cloud Redis database, I use Redis Lab. You can signup for an account and get 1 database with 30 MB storage for free. This is more than enough for me for prototyping and testing things out.

Conclusion

So these were most of the services that I have personally used and would recommend to others as well. My final thoughts are

I mostly use Heroku and Vercel for deploying most of my backend applications
For frontend I mostly choose between Netlify and Vercel from the mentioned options, but all of these options provide very similar kind of services so at the end it boils down to our own experience while using them, my suggestion would be to try each one out and choose for yourself which is your preference
For Databases, I have laid down all the services I use for different types of databases, there may be other service providers but these are the ones I tend to choose.

Let me know if something is missing in the list which you would've liked to see. 😊

API Rate Limit for Small Size to Large Production Ready Applications in Node.js

Akash Rajpurohit — Fri, 12 Feb 2021 07:33:08 +0000

Introduction

Rate limiting is one of the most important security feature that you must add for securing backend APIs from malicious attacks like brute-forcing /login or /admin and for handling unwanted flood of requests from users. In simple terms, rate-limiting allows us as a developer to control the rate at which user requests are processed by our server.

In this guide, we will learn how to add rate limiting for a small-sized project as quickly as possible and how to scale such a method for a production-ready application.

Why do you want to add rate-limiting?

Simply said, you want to reduce the risk of DOS attacks and make sure that your server is never overloaded.

For example, you have to build a Public API and you want to allow unsubscribed users to make only 100 requests per hour. Once the user has exceeded that limit, you just want to simply ignore the request and send them an error indicating that they have exhausted their free limit and consider to subscribe for your API or anything or that sort.

Kindly bear in mind that for implementing a rate-limiting technique, there must be a clearly defined constraint, which could be based on any of the following:

IP address: The constraint is on the IP address of the device from where the request was initiated.
Location: The constraint here is based on the geographical region which is implemented based on the location from where the request was made.
Users: The constraint is made on the particular user and is implemented by using a unique identifier for the particular user like userId, API KEY, etc.

Several algorithms can be used to implemented rate limiting, you can read more about them here.

Given that, let us start with the practical implementation of API rate limiting.

For Small & Medium-Sized Applications

We will use a third-party npm package called express-rate-limit for this purpose. Surely we can build a custom middleware ourselves but there is no need to reinvent the wheel.

Step 1: Setup Basic Project

I'm assuming that you already have a project set up with express. If not then quickly setup an express project using boilerplate-gen package.

npx boilerplate-gen

and choose express for the project template.

Step 2: Install the third party package

Install the package.

yarn add express-rate-limit

Step 3: Create the rate limit middleware

const rateLimit = require('express-rate-limit');

// Rate limit middleware
const rateLimitMiddleware = rateLimit({
  windowMs: 60 * 60 * 1000,
  max: 100,
  message: 'You have exceeded your 100 requests per hour limit.',
  headers: true,
});

// Export it
module.exports = rateLimitMiddleware;

Let's now quickly try to understand what we are doing here.

We are exporting a function called rateLimitMiddleware which invokes the rateLimit function which we installed from the package. This middleware enforces rate limiting based on the options we have passed in, which are -

windowMs - The window size in milliseconds, which in our case is 1 hour.
max - Maximum number of requests which can be allowed in the given window size.
message - The error message that the user would get when they exceed their limit.
headers - This option automatically adds the appropriate headers showing that this API resource is rate limited (X-RateLimit-Limit), current usage (X-RateLimit-Remaining) and time to wait before retrying (Retry-After).

Now that we have created our middleware, we have simply configure our application to use this middleware when handling the requests.

Step 4: Use the middleware

const express = require('express');
const rateLimitMiddleware = require('./middlewares/ratelimit');

const app = express();
// Use Ratelimit Middleware
app.use(rateLimitMiddleware);

Voilà! We are done here. Now all the requests will be rate-limited according to your configurations. Also you can add multiple middlewares with different sets of configurations for certain routes.

For eg, normal routes can be rate-limited to 100 requests per hour and /login or /admin can be rate-limited to 20 requests per hour to avoid brute force password attacks

Note: This package identifies users by their IP addresses using req.ip by default

Great! So now you have added a rate limiter for your API with just 4 simple steps.

Now let's move to the other half section of this blog, which is...

For Large Applications

The above-mentioned implementation is really good if you are building a small to medium size application. However, this approach will not scale for large applications.

Why is that?? You must be asking right.

Firstly, if your application size is large, most likely you won't be having a single node process on a single server. Rather you will have multiple node processes running on a distributed system and the above third party package does not share state with other processes/servers by default.

So using the same configurations you won't be able to scale.

So what's the solution there? How can I share the state between multiple server instances?

The answer is quite simple

You use an External Data Store to store all the information.

express-rate-limit package uses Memory Store by default which stores hits in-memory in the Node.js process, hence cannot share the state between processes.

So we can use an external data store to store this information and that way we can have multiple processes/servers using that external store and that way we can scale our application.

Now the question is what should we use as our Data Store. Well, there are many choices like -

Redis Store
Memcached Store
Mongo Store
PostgreSQL
MySQL etc.

I would prefer to choose Redis because it is fast and flexible with support for various types of data structures.

Also we will use another third-party package called rate-limiter-flexible as it works with Redis, process Memory, Cluster or PM2, Memcached, MongoDB, MySQL, PostgreSQL and allows to control requests rate in a single process or distributed environment.

Let's now start with the implementation part. Assuming you have an existing project, or use the same method from above to set up a new one quickly.

Step 1: Install the package

yarn add rate-limiter-flexible redis

Step 2: Setup the middleware

We will use Redis for the project, if you don't have Redis installed then first download and install it from here

const redis = require('redis');
const { RateLimiterRedis } = require('rate-limiter-flexible');

// Create redis client
const redisClient = redis.createClient({
  host: 'redis',
  port: 6379,
});

// Setup Rate Limiter
const rateLimiter = new RateLimiterRedis({
  redis: redisClient, // redis client instance
  keyPrefix: 'appname:rl', // prefix your keys with some name
  points: 10, // 10 requests
  duration: 1, // per 1 second by IP
});

// Setup the middleware using the rate limiter config
const rateLimiterMiddleware = (req, res, next) => {
  // On the basis of ip address, but can be modified according to your needs
  rateLimiter
    .consume(req.ip)
    .then(() => {
      next();
    })
    .catch(() => {
      res.status(429).send('Too Many Requests');
    });
};

module.exports = rateLimiterMiddleware;

Let's break it down by each section.

We import the packages, both Redis and rate-limiter-flexible, and use the RateLimiterRedis since we are implementing with Redis.
We create Redis client connecting to the local machine on Redis default port 6379. You can use a remote hosted machine with Redis as well here (which you might do for large systems).
We create the rateLimiter instance with some configuration options

redis - The redisClient instance we created.
keyPrefix - Adding a prefix to all the keys generated from it, we can use it like appname:rl, rl is for ratelimit. Feel free to choose any other keyPrefix.
points - Maximum number of points can be consumed over the duration.
duration - Number of seconds before consumed points are reset. If it is set to 0 then it never resets.

You can view more options here

Finally, we set up our middleware which uses the rateLimiter instance we created above and we consume based on the req.ip i.e IP address of the user.

Finally, we will use this middleware in our app.js

Step 3: Use the middleware

const express = require('express');
const rateLimiterRedisMiddleware = require('./middleware/rateLimiterRedis');

const app = express();
app.use(rateLimiterRedisMiddleware);

And..that's it. Now you can scale your application with rate limiter. I would highly suggest you to check the docs of the package for more details and configuration options.

TL;DR

We learn how to set up rate limiter for nodejs and expressjs for small and large size applications with implementation with code examples.

Disable Source Maps in GatsbyJs V2 for Production

Akash Rajpurohit — Fri, 12 Feb 2021 07:19:10 +0000

Introduction

GatsbyJS in version 2 automatically includes source maps when you build your site for production using the yarn build command. This is a really handy feature that enables you to debug and view your code in production.

If your website or app is open source then you would not mind people being able to view your code directly when visiting your site, but if your code is a private source or some client projects for which you don't want to explicitly allow others to view your code, you would have to disable the generation of source maps when gatsby builds your site.

In this blog, we will see how to configure source maps settings for your gatsbyjs v2 websites i.e have source maps in development so that your life is a bit easy as a developer but disable them on production builds.

Customize Webpack

GatsbyJs uses Webpack for bundling the files, don't worry if you don't know how to configure webpack because for this task we won't be doing any high-level configurations.

What we have to do is disable the devtool flag when we are in the production environment.

So, since now we know what we have to do, let's take a look at how we are going to do it.

We can modify the configuration options for webpack in gatsby by exporting a function called onCreateWebpackConfig in the gatsby-node.js file. If you don't have a gatsby-node.js file then create a new one (most likely you would already be having that at the root of your folder).

Copy the snippet below in the gatsby-node.js file



exports.onCreateWebpackConfig = ({ actions, getConfig }) => {
  const config = getConfig();
  if (config.mode === 'production') {
    // Turn off source maps in production
    actions.setWebpackConfig({
      devtool: false,
    });
  }
};

What's Happening

So first we are destructing actions and getConfig from this function from the list of available configurations.

We will first get the current config by calling the getConfig function, it has a lot of properties but the ones in which we are interested are devtool and mode.



{
    ...
    devtool: 'source-map',
    mode: 'production',
    ...
}

when NODE_ENV is set to production, we would get the mode value set to production in the config as well, so we would first check if this mode is production or not.

If it is production then we would call the method setWebpackConfig on the actions object and pass the devtool option to false (which is initially set to 'source-map').

And that's it, we have successfully disabled source maps for our website in production.

Note: GatsbyJS creates the public directory and it is usually not cleared automatically when you are running gatsby build, so to check the effects you have to clear the public and .cache directory (use gatsby clean command). You can add these scripts to your package.json file.



{

    "scripts": {

        ...

        "build": "gatsby build --log-pages",

        "clean": "gatsby clean",

        ...

    }

}

Pro Tip for Netlify

If you are hosting your site on Netlify then to see the changes, you have to clear the cache and deploy the site again. You can find this option on your dashboard at https://app.netlify.com/sites/SITE_NAME/deploys

TL;DR

We learn how to disable source maps in production for the GatsbyJS version 2 website.

Forem: Akash Rajpurohit

Getting paid in foreign currency as a software developer in India

Introduction

Receiving Payments from Foreign Clients

1. Direct Wire Transfer (with SWIFT code)

2. Skydo

3. Wise (Formerly TransferWise)

4. PayPal

Understanding Tax Implications

Section 44ADA: A Blessing for Professionals

Beyond ₹75 Lakhs: Regular Taxation

GST Registration and Compliance

When to Register for GST

GST Compliance Requirements

Best Practices and Tips

Tools and Resources

Conclusion

Increase Swap Memory on Raspberry Pi

Introduction

Increase Swap Memory

Conclusion

Fix Sudo not asking Password on Raspberry Pi

Introduction

Fix Sudo not asking Password on Raspberry Pi

Conclusion

How to use ES6 import syntax in Node.js

Introduction

ES6 Import

How to use it

Increase Root Partition Size On Fedora

Introduction

Backup, Backup, and Backup

Shrink Home

Extend Root Partition

Backup Users Home Directory In Linux Using Tar Command

Introduction

Tar Command

Usage

Bonus

Backup Script

Cron Job

Top Free Services To Deploy Full-Stack Applications (Back-end, Front-end, Databases)

Introduction

Back-End

1. Heroku

2. Vercel

3. Firebase

4. Fly.io

5. Freemium IaaS

Front-End

1. Netlify

2. Vercel

3. Github Pages

4. Surge.sh

5. Render

Database

1. MongoDB Atlas

2. Elephant SQL

3. Heroku Postgres

4. Redis Lab

Conclusion

API Rate Limit for Small Size to Large Production Ready Applications in Node.js

Introduction

Why do you want to add rate-limiting?

For Small & Medium-Sized Applications

Step 1: Setup Basic Project

Step 2: Install the third party package

Step 3: Create the rate limit middleware

Step 4: Use the middleware

For Large Applications

Step 1: Install the package

Step 2: Setup the middleware

Step 3: Use the middleware

TL;DR

Links

Disable Source Maps in GatsbyJs V2 for Production

Introduction

Customize Webpack

What's Happening

Pro Tip for Netlify