Hey there, devs! 👋 If you've ever struggled with paginating large datasets efficiently, you're in the right place. Today, we'll implement cursor-based pagination in a NestJS API using TypeORM. This approach is far superior to offset-based pagination when dealing with large databases. Let's dive in! 🏊♂️
What We'll Cover 🔥
- Using a
createdAtcursor to fetch records efficiently. - Implementing a paginated endpoint in NestJS.
- Returning data with a cursor for the next page.
1️⃣ Creating a DTO for Pagination Parameters
First, let's define a DTO to handle pagination parameters:
import { IsOptional, IsString, IsNumber } from 'class-validator';
import { Transform } from 'class-transformer';
export class CursorPaginationDto {
@IsOptional()
@IsString()
cursor?: string; // Receives the `createdAt` of the last item on the previous page
@IsOptional()
@Transform(({ value }) => parseInt(value, 10))
@IsNumber()
limit?: number = 10; // Number of items per page (default: 10)
}
2️⃣ Implementing the Query in the Service
Now, let's create the logic in our service:
import { Injectable } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { User } from './user.entity';
import { CursorPaginationDto } from './dto/cursor-pagination.dto';
@Injectable()
export class UserService {
constructor(
@InjectRepository(User)
private readonly userRepository: Repository<User>,
) {}
async getUsers(cursorPaginationDto: CursorPaginationDto) {
const { cursor, limit } = cursorPaginationDto;
const queryBuilder = this.userRepository
.createQueryBuilder('user')
.orderBy('user.createdAt', 'DESC')
.limit(limit + 1); // Fetching one extra record to check if there's a next page
if (cursor) {
queryBuilder.where('user.createdAt < :cursor', { cursor });
}
const users = await queryBuilder.getMany();
const hasNextPage = users.length > limit;
if (hasNextPage) {
users.pop(); // Remove the extra item
}
const nextCursor = hasNextPage ? users[users.length - 1].createdAt : null;
return {
data: users,
nextCursor,
};
}
}
3️⃣ Creating the Controller
Finally, let's expose our paginated endpoint:
import { Controller, Get, Query } from '@nestjs/common';
import { UserService } from './user.service';
import { CursorPaginationDto } from './dto/cursor-pagination.dto';
@Controller('users')
export class UserController {
constructor(private readonly userService: UserService) {}
@Get()
async getUsers(@Query() cursorPaginationDto: CursorPaginationDto) {
return this.userService.getUsers(cursorPaginationDto);
}
}
4️⃣ Defining the Database Model
Here's our User entity:
import { Entity, Column, PrimaryGeneratedColumn, CreateDateColumn } from 'typeorm';
@Entity()
export class User {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column()
name: string;
@CreateDateColumn()
createdAt: Date;
}
How Cursor-Based Pagination Works ⚡
1️⃣ The first request to GET /users does not include a cursor. It fetches the first limit records.
2️⃣ The backend returns a nextCursor, which is the createdAt timestamp of the last user in the response.
3️⃣ To fetch the next page, the frontend makes a request to GET /users?cursor=2024-03-09T12:34:56.000Z, and the backend will return users created before that timestamp.
4️⃣ This process continues until nextCursor is null, meaning there are no more records left.
Example JSON Response 📝
{
"data": [
{ "id": "1", "name": "John", "createdAt": "2024-03-09T12:00:00.000Z" },
{ "id": "2", "name": "Anna", "createdAt": "2024-03-09T11:45:00.000Z" }
],
"nextCursor": "2024-03-09T11:45:00.000Z"
}
Why Use Cursor-Based Pagination? 🤔
✅ Better Performance: Avoids OFFSET, which slows down large datasets.
✅ Scalability: Works seamlessly with millions of records.
✅ Optimized Queries: Using indexed fields like createdAt makes queries lightning-fast. ⚡
Conclusion 🎯
Cursor-based pagination is a game-changer for handling large datasets in APIs. 🚀 It's faster, more efficient, and ensures a smoother experience for your users. Now you’re ready to implement it in your own NestJS project! 💪
Got questions or improvements? Drop them in the comments! 💬 Happy coding! 😃
Top comments (3)
Hi @victorsfranco Great question!
Yes, to go back to the previous page when using cursor-based pagination, you need to implement a bit of reverse logic.
Here's how it usually works:
Store the cursors on the client side: For each page you load, save both the nextCursor and prevCursor (if provided). Think of it like a breadcrumb trail.
Use a flag (e.g., isPrevious: true): When requesting the previous page, you send the prevCursor along with a flag that tells the server you want to go backward.
Reverse the query on the backend: The backend should apply the cursor with a reversed comparison (e.g., createdAt < cursor instead of createdAt > cursor), and also reverse the sort order temporarily to fetch the correct set of records.
Reverse the result before sending: Since you fetched in reverse order, reverse the result before returning it so it appears in the correct chronological order.
This way, you can paginate both forward and backward using cursors without relying on offset-based logic, which can become unreliable with dynamic data.
I just implemented it on my API and its working fine. Thank you!
I got the point, but I have one question: In this example, I could understand how I can go to next page, but how can I came back to previus page? Should I additionaly apply something as a previus cursor, writing the reverse logic?
Thank you for this valuable and useful content!