How I Optimized API Calls in Flutter to Handle 10,000+ Concurrent Users

How I Optimized API Calls in Flutter to Handle 10,000+ Concurrent Users

Optimizing API calls is crucial for ensuring the scalability of mobile applications. In this post, I'll share how I successfully optimized API interactions in my Flutter app, enabling it to handle over 10,000 concurrent users efficiently.

The Challenge

Initially, our Upalerts Flutter application experienced significant slowdowns and occasional crashes under heavy load, particularly when reaching around 2,000 concurrent users. Users reported long loading times, failed requests, and poor overall user experience. This issue was critical as it directly impacted user retention and satisfaction.

Original Approach

We initially used standard HTTP requests for every API interaction without any optimization:

Future<Response> fetchData() async {
  final response = await http.get(Uri.parse('https://api.example.com/data'));
  return response;
}

Each user action triggered a separate API request without checking for cached responses or managing concurrent connections.

Issues Encountered:

• High latency under heavy load: Response times spiked dramatically as the user count increased, frequently exceeding several seconds.
• Frequent timeouts: Users frequently encountered request timeouts, leading to incomplete data or failed requests.
• Increased server load: The lack of caching or optimization meant repetitive requests were flooding the server, quickly overwhelming its resources.
• Poor resource utilization: Connections were not efficiently reused or managed, leading to unnecessary overhead and connection establishment latency.

Optimized Solution

To address these issues, I implemented three key improvements:

1. Connection Pooling: Reusing established connections rather than creating new ones for each request.
2. Caching responses: Implemented a caching layer to store frequently accessed data, significantly reducing redundant API calls.
3. Concurrency control: Limiting the number of simultaneous requests to maintain optimal performance and server health.

Here's the updated implementation using Dio for better control over HTTP calls:

final dio = Dio(BaseOptions(
  baseUrl: 'https://api.example.com/',
  connectTimeout: 5000,
  receiveTimeout: 3000,
));

final cacheManager = DefaultCacheManager();

Future<Response> fetchOptimizedData() async {
  final cacheKey = 'dataEndpoint';

  // Check cache first
  final cachedData = await cacheManager.getFileFromCache(cacheKey);
  if (cachedData != null) {
    return Response(
      requestOptions: RequestOptions(path: 'data'),
      data: cachedData.file.readAsStringSync(),
    );
  }

  // API request with concurrency control
  final response = await dio.get('data');

  // Cache successful responses
  if (response.statusCode == 200) {
    cacheManager.putFile(
      cacheKey,
      Uint8List.fromList(response.data.toString().codeUnits),
      maxAge: Duration(minutes: 10),
    );
  }

  return response;
}

Before vs. After Performance Metrics

Conclusion

By implementing connection pooling, caching, and concurrency management, we significantly improved the performance and scalability of our Flutter app. These changes enabled us to reliably support over 10,000 concurrent users, drastically reducing latency and virtually eliminating request failures.