Forem: Tom Bloom

Infinite Pagination with Ruby on Rails API and TanStack Query

Tom Bloom — Wed, 02 Jul 2025 16:16:38 +0000

And with filtering and sorting too...

Building efficient pagination can be tricky, but combining Ruby on Rails with TanStack Query (formerly React Query) makes it surprisingly elegant. In this article, I'll show you how to implement infinite scroll pagination that's both performant and developer-friendly.

The Rails API Setup

Let's start with a Ruby on Rails API that has these characteristics:

JWT authentication/authorization in v1/application_controller.rb
pagy gem for pagination
blueprint gem for JSON serialization
has_scope gem for filtering

Here's our v1/groups_controller.rb index endpoint:

class V1::GroupsController < V1::ApplicationController
  before_action :set_group, only: [:show, :update, :destroy]

  has_scope :search

  def index
    sort_by = params[:sort_by] || 'created_at'
    direction = params[:sort_direction] || 'desc'

    @groups = apply_scopes(current_account.groups).order(sort_by => direction)
    @pagy, @groups = pagy(@groups, limit: params[:per_page])

    render json: {
      data: V1::GroupBlueprint.render_as_hash(@groups),
      meta: pagy_metadata(@pagy)
    }
  end

  # rest of the controller...
end

The beauty of this setup is its simplicity. The has_scope gem automatically handles our search filtering, while pagy takes care of pagination metadata.

The TanStack Infinite Query Hook

Now for the frontend magic. Here's our useInfiniteGroups hook:

export const useInfiniteGroups = (params?: {
  per_page?: number;
  search?: string;
  sort_by?: string;
  sort_direction?: string;
  [key: string]: any; // Allow any additional parameters
}) => {
  const api = useApi();

  return useInfiniteQuery({
    queryKey: ['groups'],
    queryFn: async ({ pageParam }) => 
      await api.get<GroupsResponse>('/v1/groups', { 
        ...params, 
        page: pageParam, 
        per_page: params?.per_page || 20 
      }),
    getNextPageParam: (lastPage) => lastPage.meta.next,
    initialPageParam: 1,
  });
};

The hook is beautifully simple:

queryKey identifies our query for caching
queryFn fetches the data with pagination parameters
getNextPageParam tells TanStack Query when there are more pages
All filtering parameters are passed through seamlessly

The React Component with Infinite Scroll

Here's where it all comes together in our GroupsList component:

import { Button } from '@/components/ui/button';
import { useInfiniteGroups } from '@/lib/hooks/useGroups';
import { queryClient } from '@/main';
import { Link } from '@tanstack/react-router';
import { PlusIcon } from 'lucide-react';
import { useCallback, useEffect, useRef, useState } from 'react';

export default function GroupsList() {
  const [search, setSearch] = useState('');
  const [sortBy, setSortBy] = useState('created_at');
  const [sortDirection, setSortDirection] = useState('desc');

  const {
    data,
    fetchNextPage,
    hasNextPage,
    isFetchingNextPage,
    isLoading
  } = useInfiniteGroups({
    search,
    sort_by: sortBy,
    sort_direction: sortDirection
  });

  // Reset infinite query when parameters change
  useEffect(() => {
    queryClient.resetQueries({ queryKey: ['groups'] });
  }, [search, sortBy, sortDirection]);

  const observer = useRef<IntersectionObserver | null>(null);

  const sentinelRef = useCallback((node: HTMLDivElement) => {
    if (isFetchingNextPage) return;
    if (observer.current) observer.current.disconnect();

    observer.current = new IntersectionObserver(entries => {
      if (entries[0].isIntersecting && hasNextPage) {
        fetchNextPage();
      }
    });

    if (node) observer.current.observe(node);
  }, [isFetchingNextPage, hasNextPage, fetchNextPage]);

  if (isLoading) return <div>Loading...</div>;

  // Flatten all pages into a single array
  const allGroups = data?.pages.flatMap(page => page.data) ?? [];

  return (
    <div className="space-y-4">
      <div className="flex justify-between items-center">
        <h1 className="text-2xl font-bold">Groups</h1>
        <Link to='/admin/groups/new'>
          <Button>
            <PlusIcon className='w-4 h-4 mr-2' />
            Add Group
          </Button>
        </Link>
      </div>

      {/* Search and sorting controls */}
      <div className="flex gap-4 p-4 bg-gray-50 rounded-lg">
        <input
          type="text"
          placeholder="Search groups..."
          value={search}
          onChange={(e) => setSearch(e.target.value)}
          className="flex-1 px-3 py-2 border border-gray-300 rounded-md"
        />

        <select
          value={sortBy}
          onChange={(e) => setSortBy(e.target.value)}
          className="px-3 py-2 border border-gray-300 rounded-md"
        >
          <option value="created_at">Creation Date</option>
          <option value="name">Name</option>
          <option value="updated_at">Last Modified</option>
        </select>

        <select
          value={sortDirection}
          onChange={(e) => setSortDirection(e.target.value)}
          className="px-3 py-2 border border-gray-300 rounded-md"
        >
          <option value="desc">Descending</option>
          <option value="asc">Ascending</option>
        </select>
      </div>

      {/* Groups list */}
      <div className="space-y-4">
        {allGroups.map(group => (
          <div key={group.id} className="p-4 border border-gray-200 rounded-lg">
            <h3 className="text-lg font-semibold">{group.name}</h3>
            <p className="text-gray-600">{group.description}</p>
          </div>
        ))}
      </div>

      {/* Invisible element that triggers loading */}
      <div ref={sentinelRef} style={{ height: '1px' }} />

      {isFetchingNextPage && (
        <div className="text-center py-4">Loading more groups...</div>
      )}

      {hasNextPage && (
        <div className="text-center py-4">
          <Button
            onClick={() => fetchNextPage()}
            disabled={isFetchingNextPage}
            variant="outline"
          >
            {isFetchingNextPage ? 'Loading...' : 'Load More'}
          </Button>
        </div>
      )}
    </div>
  );
}

The Magic: Automatic Cache Invalidation

The most elegant part of this solution is how filtering works. Notice this simple useEffect:

useEffect(() => {
  queryClient.resetQueries({ queryKey: ['groups'] });
}, [search, sortBy, sortDirection]);

When any filter parameter changes, we invalidate all cached queries that start with the ['groups'] key. This forces a fresh fetch with the new parameters, and TanStack Query handles all the complexity behind the scenes.

Why This Approach Works So Well

As a Rails lover, I appreciate the simplicity of this solution:

Server-side: Rails with pagy provides robust, efficient pagination out of the box
Client-side: TanStack Query handles caching, background updates, and loading states automatically
Developer Experience: The API is intuitive and the React component is clean and maintainable
Performance: Only fetch what you need, when you need it
Flexibility: Easy to add new filters or sorting options

What's Next?

This example covers the core infinite pagination pattern, but there's much more you can do:

Loading states and error handling
Optimistic updates for CRUD operations
Real-time updates with WebSockets
Advanced filtering with multiple parameters
Virtualization for extremely large lists

Would you like to see a full CRUD example with authentication, API request generators, TypeScript types, and all the CRUD queries, mutations, and components? Let me know in the comments!

Have you tried this pattern in your Rails + React apps? What challenges have you faced with infinite pagination? Share your experience in the comments below!

Learn React: the same TODO list app, upgraded step by step

Tom Bloom — Thu, 12 Sep 2024 09:20:23 +0000

Recently, I wanted to understand deeply how to create React apps today, which technologies to chose between Redux, RTK Query, React Query, Zustand, NextJS.

I also wanted to understand perfectly the old and new Hooks available in React. And how to connect a store-based app to an external API.

So I decided to create this project of a TODO list application, first static, then connected to an API, then connected to an API with authentication. Feel free to create other versions of the app, it will be very useful.

The rest of this article is simply extracted from the github project README.

Todo List experiments

This is a set of different versions of the very same todo list application in React to experiment with different React architectures and technologies.

Why

There are many ways to create React applications nowadays, and it is useful to have a look at different versions of the same app using different hooks, frameworks, libs, etc.

These version are either :

static (client only)
API (connected to an API)
API + auth (connected to an API and with authentication)

(API) versions are using the API located in task_apifolter.
(API + auth) verssions are using the API located in task_api_with_authfolter.

Versions

done :

with useState
with useReducer
with useReducer and Context
with old school redux (connect state to props and dispatch to props)
with redux hooks (useDispatch and useSelector)
with redux and more hooks (useMemo, useEffect, useCallback)
with modern redux (slices)
with zustand
(API) with modern redux and async thunks
(API) with rtk-query
(API) with react-query
(API + auth) with react-query and zustand

doing:

(API + auth) with nextjs (and server actions)

todo:

with tanstack router
with old school redux + thunk (manual thunks)
with Remix

About the API

task_apiand task_api_with_auth are developped in Ruby on Rails in 5 minutes for the first one and add 10 minutes for the second one.

They provide a simple CRUD interface for tasks :

GET /tasks
POST /tasks
PATCH /tasks/:id
DELETE /tasks/:id

The version with autentication provide a kind of oauth authentication with an access token and a refresh token. The routes are :

POST /auth_tokens           # login with email and password
POST /auth_tokens/refresh   # refresh the access token
DELETE /auth_tokens         # logout

The strategy for the client app is to have an apiClient with 2 interceptors to manage the access token and refresh token.

adds a Bearer token to the Authorization header
if the request is rejected and we have a refresh token, we try to refresh the access token and retry the request

A normalization function for json-api format

Tom Bloom — Fri, 02 Feb 2024 20:37:12 +0000

Recently I had to build a mobile application on top of an API respecting the json-api format.

Json-api is a great format for REST apis because it is normalized (follows a normalization pattern) and you can include all the data you want in the json response.

However, it is a bit messy on the client-side when you want to access the data. It is a bit too verbose and too deep, with "data", "type", "attributes", "relationships" and relationships have a similar structure recursively. So I wanted to re-normalize the data into a more javascript-friendly format.

In React it is better (and easier) to use the normalization pattern recommanded by the React or the Redux communities. react-query also talks about that.

So, the normalization function I came up with looks like this. First, let's define our entities in Typescript :

// File: entities.ts

export interface ApiResponse {
  data: Array<DataEntity>;
  included?: Array<DataEntity>;
}

export interface DataEntity {
  id: string;
  type: string;
  attributes: any;
  relationships?: { [key: string]: RelationshipData };
}

export interface NormalizedData {
  [entityType: string]: {
    [entityId: string]: DataEntity & RelationshipObject;
  };
}

export type RelationshipObject = {
  [key: string]: string | string[];
}

export interface RelationshipData {
  data: RelationshipItem | RelationshipItem[] | null;
}

export interface RelationshipItem {
  id: string;
  type: string;
}

Then, let's write our normalization function. In Json-API, you always have the main data (in "data" key) and the included data (in the "included" data).

// File: normalization.ts

import { ApiResponse, NormalizedData, RelationshipData, DataEntity, RelationshipObject } from './entities';

const singularize = (word: string): string => {
  if (word.endsWith('ies')) {
      // e.g., "categories" to "category"
      return word.slice(0, -3) + 'y';
  } else if (word.endsWith('s')) {
      // e.g., "projects" to "project"
      return word.slice(0, -1);
  } else {
      // no change for words that do not end in 'ies' or 's'
      return word;
  }
}

const processRelationships = (relationships: { [key: string]: RelationshipData }) => {
    if (!relationships) {
        return {};
    }
    const result: RelationshipObject = {};
    for (const [key, relationship] of Object.entries(relationships)) {
        const relationshipData = relationship.data;
        if (Array.isArray(relationshipData)) {
            // Plural relationships
            const singularKey = singularize(key); // Convert to singular form
            result[`${singularKey}Ids`] = relationshipData.map(item => item.id);
        } else if (relationshipData) {
            // Singular relationships
            result[`${key}Id`] = relationshipData.id;
        }
    }
    return result;
};

export const normalizeData = (responseData: ApiResponse): NormalizedData => {
  const normalized: NormalizedData = {};

  const normalize = (item: DataEntity) => {
    const entityType = item.type + 's';
    normalized[entityType] = normalized[entityType] || {};
    normalized[entityType][item.id] = {
      ...item.attributes,
      id: item.id,
      ...processRelationships(item.relationships),
    };
  };

  responseData.data.forEach(normalize);
  responseData.included.forEach(normalize);

  return normalized;
};

We use a singularize function to map the OneToMany relationships. E.g. if a Project has projectActivities, you want to transform the key to projectActivityIds and the value is an array of ids. If it is a ManyToOne relationship, you just have a single string for the Id.

Like this :

      "groupId": "178",
      "userId": "345",
      "projectActivityIds": [],
      "invoiceIds": [
        "24639",
        "24640",
        "25070",
        "25071",
        "84898"
      ]

I had success with this format in my application so I recommand it. I added a test with jest to be sure that the json-api input gives exactly the right output.

But I actually should put it in an external package.

With react-query, I use it like this. Actually, it has nothing to have with react-query. I just use it in the basic fetch function that calls get() on the api object, a simple wrapper of the axios lib.

// File : queries.ts
import { useQuery } from '@tanstack/react-query';
import { api } from "./api";
import { normalizeData } from './normalization'

const fetchPhases = async (params) => {
  const { fromDate, toDate, userId } = params;
  const response = await api.get('/v5/plannings/phases', {
    params: {
      from: fromDate,
      to: toDate,
      users_id_in: [userId],
      include_invoices: true
    },
  });
  //console.log('response data', JSON.stringify(response.data, null, 2));
  return normalizeData(response.data);
}

export const usePhases = (params) => {
  return useQuery({
    queryKey: ['phases', params],
    queryFn: () => fetchPhases(params),
  });
};

Deploy a React Monorepo to Firebase with Multiple Environments

Tom Bloom — Tue, 13 Dec 2022 09:37:31 +0000

Deploy a React Monorepo to Firebase with Multiple Environments

Managing multiple React applications in a monorepo while maintaining separate staging and production environments can be challenging. In this guide, I'll show you exactly how to deploy a JavaScript monorepo with multiple React apps to Firebase, complete with proper environment separation and CI/CD integration.

The Setup: What We're Building

Let's say you have a monorepo containing multiple React applications managed by Lerna. In our example, we have:

📱 Frontend app - Main user-facing application
⚙️ Admin app - Internal administration panel
🎨 Design system app - Component library documentation

We need at least two environments:

Staging - For testing and QA
Production - For live users

We'll use separate Firebase projects for each environment, and within each project, we'll create different Firebase targets (sites) for each app in our monorepo.

Step-by-Step Firebase Configuration

1. Create Firebase Projects

For each environment, you'll need to:

Create a Firebase project in the Firebase Console
Initialize Firebase in your local repo using:

   firebase init

2. Deploy Initial Content

This step is crucial and often overlooked:

firebase deploy

Deploy an empty index.html file first. This step is essential because it activates the hosting service and allows you to create additional targets (sites) in your Firebase project.

3. Create Firebase Sites for Each App

For each app in your monorepo, create a separate site:

Go to the Hosting section in your Firebase Console
Click "Add another site"
Create sites like:
- myproject-app1-staging
- myproject-app2-staging
- myproject-app3-staging

Repeat this process for your production project.

💡 Pro tip: While you can use the Firebase CLI to create sites, I recommend using the console to avoid accidentally overwriting your .firebaserc file.

Configuration Files

firebase.json - Define Your Targets

This file specifies how each target maps to your build folders:

{
  "hosting": [
    {
      "target": "app1",
      "public": "packages/app1/build",
      "ignore": [
        "firebase.json",
        "**/.*",
        "**/node_modules/**"
      ],
      "rewrites": [
        {
          "source": "**",
          "destination": "/index.html"
        }
      ]
    },
    {
      "target": "app2", 
      "public": "packages/app2/build",
      "ignore": [
        "firebase.json",
        "**/.*",
        "**/node_modules/**"
      ],
      "rewrites": [
        {
          "source": "**",
          "destination": "/index.html"
        }
      ]
    },
    {
      "target": "app3",
      "public": "packages/app3/build", 
      "ignore": [
        "firebase.json",
        "**/.*",
        "**/node_modules/**"
      ],
      "rewrites": [
        {
          "source": "**",
          "destination": "/index.html"
        }
      ]
    }
  ]
}

I've added rewrites rules to ensure your React Router works properly in production.

.firebaserc - Environment and Target Mapping

This file maps your environments to Firebase projects and targets to specific sites:

{
  "projects": {
    "default": "myproject-staging",
    "staging": "myproject-staging", 
    "prod": "myproject-prod"
  },
  "targets": {
    "myproject-staging": {
      "hosting": {
        "app1": ["myproject-app1-staging"],
        "app2": ["myproject-app2-staging"],
        "app3": ["myproject-app3-staging"]
      }
    },
    "myproject-prod": {
      "hosting": {
        "app1": ["myproject-app1"],
        "app2": ["myproject-app2"], 
        "app3": ["myproject-app3"]
      }
    }
  }
}

Deployment Commands

Once configured, deploying is straightforward:

# Deploy to staging
firebase deploy -P staging

# Deploy to production  
firebase deploy -P prod

# Deploy specific targets only
firebase deploy -P staging --only hosting:app1

CI/CD Integration with GitLab

Here's a complete GitLab CI configuration that builds and deploys your monorepo:

image: node:18.12.0

stages:
  - build
  - deploy

variables:
  # Disable source maps in CI for faster builds
  GENERATE_SOURCEMAP: false

# Build stage - shared by both environments
.build_template: &build_template
  stage: build
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
      - packages/*/node_modules/
  script:
    - yarn install --frozen-lockfile
    - NODE_OPTIONS=--openssl-legacy-provider yarn build
  artifacts:
    paths:
      - packages/*/build/
    expire_in: 1 hour

# Staging deployment
deploy-staging:
  <<: *build_template
  stage: deploy
  only:
    - develop
  environment: 
    name: staging
    url: https://myproject-app1-staging.web.app
  variables:
    REACT_APP_API_URL: "https://api-staging.myproject.com"
    REACT_APP_SENTRY_ENVIRONMENT: "staging"
  after_script:
    - npm install -g firebase-tools
    - firebase deploy -P staging --token $FIREBASE_TOKEN

# Production deployment  
deploy-production:
  <<: *build_template
  stage: deploy
  only:
    - main
  environment:
    name: production
    url: https://myproject-app1.web.app
  variables:
    REACT_APP_API_URL: "https://api.myproject.com"
    REACT_APP_SENTRY_ENVIRONMENT: "production"
  after_script:
    - npm install -g firebase-tools
    - firebase deploy -P prod --token $FIREBASE_TOKEN
  when: manual # Require manual approval for production

Setting Up Firebase Authentication

To get your FIREBASE_TOKEN for CI/CD:

# Generate token (legacy method, but still works)
firebase login:ci

# Modern approach - use service account
firebase projects:list

⚠️ Security note: Store your FIREBASE_TOKEN as a protected variable in GitLab CI/CD settings, not in your code.

Common Gotchas and Solutions

1. "Project not initialized" Error

If you get this error when trying to deploy to staging, it means you skipped step 2. You need to:

Delete your firebase.json
Run firebase init
Deploy an empty index.html
Create your sites in the console
Restore your multi-target firebase.json

2. Build Optimization

For faster CI builds, consider:

Using --frozen-lockfile with yarn
Caching node_modules appropriately
Disabling source maps in CI (GENERATE_SOURCEMAP=false)
Using the legacy OpenSSL provider if needed

3. Environment Variables

Make sure your environment variables are set correctly for each environment in your CI/CD pipeline. React apps need variables prefixed with REACT_APP_.

Benefits of This Approach

✅ Clean separation between environments

✅ Independent scaling for each app

✅ Atomic deployments - deploy apps independently

✅ Easy rollbacks - each app can be rolled back separately

✅ Cost-effective - Firebase hosting is generous with free tier

What's Next?

This setup gives you a solid foundation, but you can enhance it further:

Add preview deployments for pull requests
Implement blue-green deployments
Add automated testing before deployment
Set up monitoring and alerting
Configure custom domains for each app

Have you set up a similar monorepo deployment strategy? What challenges did you face? Share your experience in the comments below!

If you found this helpful, follow me for more DevOps and React content! 🚀