Forem: Kashyap Bhanu Das

Engineering Remote Logging in Mobile Apps: Zero-Loss, Offline-First & Scalable

Kashyap Bhanu Das — Wed, 20 Aug 2025 15:29:27 +0000

Monitoring isn't just about logging; it's about building observability that scales to handle (200KB to 3MB)/minute per user across multiple remote services, and provides the debugging capabilities needed for a financial trading platform where every millisecond matters.

We are not going into backend design here. Let me know if you would be interested in that as well.

Why This Matters

In a trading application, monitoring serves multiple critical purposes:

Debugging: Real-time issue identification during development and production
Audit Trail: Regulatory compliance for financial transactions
Performance Monitoring: Latency tracking for trading operations
Error Tracking: Proactive issue detection before users report problems
Analytics: User behavior and system performance insights

The monitoring system must handle:

Volume: (200KB to 3MB)/minute per user
Reliability: Zero message loss, even during network failures
Multi-Service: Simultaneous logging to Service-1 and Service-2
Offline Support: Seamless operation without internet connectivity
Caching: Shouldn’t burden app operations by continuously pushing logs
Scale: 10,000+ concurrent users

Architecture Overview


┌─────────────────────────────────────────────────────────────┐
│                    Application Layer                        │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐        │
│  │   Local     │   │   Remote    │   │     Logs    │        │
│  │  Logging    │   │  Logging    │   │   UI Screen │        │
│  └─────────────┘   └─────────────┘   └─────────────┘        │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│                   Facade Layer                              │
│  ┌─────────────┐            ┌─────────────┐                 │
│  │ Local       │            │ Remote      │                 │
│  │ Logger      │            │ Logger      │                 │
│  │             │            │ Facade      │                 │
│  └─────────────┘            └─────────────┘                 │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│                Coordination Layer (Ports)                   │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │              Remote Log Coordinator                     │ │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐        │ │
│  │  │ Timer   │ │ Cache + │ │ Sender  │ │ Config  │        │ │
│  │  │         │ │ Batching│ │         │ │         │        │ │
│  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘        │ │
│  └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│                Implementation Layer (Adapters)              │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐            │
│  │ Native  │ │ Hive    │ │ HTTP    │ │ Timer   │            │
│  │ Logger  │ │ Cache   │ │ Client  │ │ Service │            │
│  └─────────┘ └─────────┘ └─────────┘ └─────────┘            │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│                External Services                            │
│  ┌─────────────┐                    ┌─────────────┐         │
│  │  Service 1  │                    │  Service 2  │         │
│  │   (HTTP)    │                    │   (HTTP)    │ etc     │
│  └─────────────┘                    └─────────────┘         │
└─────────────────────────────────────────────────────────────┘

`

Local Logging Architecture

Why Talker?

When evaluating logging solutions for the new app, we needed more than just console prints. We needed a production-ready, testable, and maintainable logging layer that fits clean architecture principles.

1. Production-Ready Debug UI

Talker gives us a built-in, zero-effort debug screen:

// Built-in debug screen with real-time filtering
Widget debugScreen() {
  if (kReleaseMode) return const SizedBox.shrink();
  return TalkerScreen(talker: _instance._talker);
}

Out of the box, we get:

Real-time log streaming
Log-level filtering (Debug, Info, Warning, Error)
Search and export
Performance metrics and insights

2. Zero Configuration

LoggerImpl._()
  : _talker = TalkerFlutter.init(
      settings: TalkerSettings(enabled: !kReleaseMode),
      logger: TalkerLogger(
        settings: TalkerLoggerSettings(enable: !kReleaseMode),
      ),
    );

Talker handles release detection, formatting, performance optimization, and memory management internally.

3. Clean Architecture Integration

We abstracted Talker behind our own Logger interface:

abstract class Logger {
  void init({required bool enabled});
  void debug(String message);
  void info(String message);
  void warn(String message);
  void error(String message, [Object? err, StackTrace? st]);
  Widget debugScreen();
}

Benefits:

Mock and unit-test logging in isolation
Inject loggers seamlessly across layers
Swap to another logger without touching business logic

Bottom line: Talker hit the sweet spot—powerful tooling with a minimal footprint, perfectly aligned with architectural principles.

Remote Logging Architecture

As part of a Flutter app, we designed a remote logging system that goes far beyond simple print() statements. It’s engineered to handle scale, offline scenarios, and multiple logging backends without sacrificing maintainability.

Design Principles

Zero Message Loss — every log is guaranteed to be delivered
Offline-First — logs queue and persist seamlessly without connectivity
Multi-Service Support — simultaneous dispatch to multiple providers
Scalable Batching — network-efficient, configurable batch sizes
Clean Architecture — loosely coupled, testable, and maintainable components

Components

Remote Log Coordinator

The brain of the system:

Routes logs (immediate vs cached)
Manages connectivity state
Coordinates periodic flushes
Handles retries, errors, and recovery
Owns cache lifecycle

class RemoteLogCoordinator {
  final LogCache _cache;
  final LogSender _sender;
  final ConnectivityChecker _connectivity;
  final PeriodicTimer _timer;
  final ConfigManager _configManager;
  final MessageBatcher _batcher;
}

Hive-Based Cache

Mutex-protected for thread safety
Survives restarts (zero data loss)
getAllAndClear() ensures atomic operations
Lazy loading keeps memory footprint low

Multi-Service Sender

Parallel HTTP requests with timeouts
Lazy service initialization
Batch-level success/failure metrics

Message Batching

class MessageBatcherImpl implements MessageBatcher {
  @override
  int get maxBatchSizeBytes => 600 * 1024; // 600 KB
}

600 KB max per batch
Oversized single messages isolated automatically
Optimal grouping for throughput and latency

Remote Logging Flow

Scalability Considerations

1. Performance at Scale

Throughput: ~3.3 MB/s sustained
Batch Size: 600 KB (max)
Send Interval: 3 minutes

2. Memory Management

Lazy loading with Hive
Chunked batch processing
Automatic cleanup
Configurable cache size

3. Network Efficiency

Fewer HTTP calls → less overhead
Larger payloads → better compression
Connection reuse with keep-alive
Per-batch timeouts to isolate failures

Error Handling & Resilience

Graceful Degradation — fallback to cache if direct send fails
Service Failure Isolation — one service failing doesn’t block others
Cache Recovery — failed sends restored to cache for retry

Testing Strategy

Unit Tests — mocked interfaces for coordination logic
Integration Tests — real service contract validation
Performance Tests — load simulation under production-like conditions

Security Considerations

Authentication: HMAC-SHA256 signatures per request
Data Privacy: no PII, enforced HTTPS
Access Control: service-specific tokens
Rate Limiting: batch caps and configurable intervals

Conclusion

This architecture ensures logs are:

Reliable — Zero message loss with robust offline support
Flexible — Multi-service logging via swappable interfaces
Maintainable — Layered clean architecture with strong test coverage
Observable — Real-time debug screens and deep system insights

We've struck the right balance between simplicity and power—an architecture that scales with app growth while keeping us confident in production diagnostics and rapid iteration.

Provider vs Bloc vs Riverpod vs GetX: Deep Dive for Flutter Engineers

Kashyap Bhanu Das — Sun, 27 Jul 2025 17:21:13 +0000

Whenever we revisit architecture decisions—especially in large-scale apps—foundational choices like state management deserve a fresh evaluation. This isn't about what's popular or convenient, but about what enables a scalable, testable, and maintainable architecture over time.

Why This Comparison Matters

A codebase is expected to evolve, scale, and often outlive its original authors. This post is aimed at engineering teams building large-scale, multi-module Flutter apps, where velocity must be balanced with architecture, traceability, and long-term maintainability.

A good state management solution must:

✅ Support modular architecture
✅ Enable testability and mocking
✅ Handle async state, errors, and lifecycles cleanly
✅ Work seamlessly with dependency injection
✅ Align well with clean separation of business logic

This article breaks down four major approaches to state management: Provider, Bloc, Riverpod, and GetX, through the lens of production readiness, clean architecture compatibility, and long-term team impact.

How Each State Management Tool Holds Up in Production

Provider — Simple but Limited

Great for quick state access and learning Flutter. But for serious products, it breaks down:

Logic leaks into UI
Tightly bound to the widget tree context
Difficult to scale or test in modular codebases

Why not go with it: Lacks structure, context-bound, DI control, and long-term testability.

Bloc — Structured, Explicit, Traceable

Bloc introduces structure through events and states. It works well when:

Every state transition matters (e.g., onboarding, order flow, KYC)
Teams need traceability and a strict unidirectional flow
Business logic must be fully decoupled from UI

Why it's a strong candidate: Predictable, testable, and great for compliance-heavy flows.

Riverpod — Modular, Clean, Testable

Riverpod fixes Provider's flaws:

No context needed
Async handling and DI are built-in
Modular, test-first friendly
Works well with freezed and build_runner

Especially with @riverpod codegen and AsyncNotifier, Riverpod becomes low-boilerplate while still enforcing structure.

Why it's leading our evaluation: Best balance of control, scalability, and team velocity.

GetX — Fast, but Needs Guardrails

GetX excels at speed—minimal setup, built-in routing, reactive patterns. But:

Uses global state extensively (Get.put, Rx)
Business logic often tied to UI controllers
Poor separation and testability
Easy to leak memory or lose control at scale

Why I'm cautious: Fast to start, but risky for shared modules unless sandboxed and structured with discipline.

Testing and Clean Architecture

Tool	Testability	DI Override	Business Logic Separation
Bloc	✅ Strict state flow	✅ Yes	✅ Fully decoupled
Riverpod	✅ Context-free testing	✅ Yes	✅ Modular & clean
Provider	❌ Context-bound	❌ Difficult	❌ UI-bound logic
GetX	⚠️ Requires decoupling	⚠️ Manual mocks	⚠️ Global controller scope

For apps with complex business logic, testability and mockable DI are non-negotiable.

Summary: Pick Based on Use Case

Provider: Easy to start, but brittle at scale.
Bloc: Heavy at first, but long-term clarity and testability pay off.
Riverpod: Cleanest path to scalable, async-safe architecture.
GetX: Only for tightly scoped modules with enforced cleanup.

Verdict

Why Bloc?

Bloc remains my first choice for features that demand strict control, testability, and explicit state modeling—like onboarding or compliance-heavy flows.

For an app as large and feature-rich as ours, boilerplate is a valid concern. But when Bloc is paired with:

New lint rules
Dependency Injection
Cubit
Code generation tools like freezed
on<Event> API
Feature-specific Bloc separation
hydrated_bloc for persistence

…it enables a scalable, disciplined architecture that keeps complexity in check.

The Bloc infra stack (Cubit, on, hydrated_bloc, freezed) has reduced onboarding time for new engineers by ~30%, thanks to familiar patterns and predictable flow structure.

Why Not Riverpod, Yet?

I'm a big fan of Riverpod—especially:

@riverpod codegen
AsyncNotifier patterns
Context-free, DI-friendly design

It aligns exceptionally well with clean architecture principles and enables modular codebases with minimal boilerplate.

That said, in large teams, its flexibility can become a double-edged sword. Without strong conventions and enforcement, it's easy for patterns to diverge and create inconsistent architecture across teams.

How are you approaching state management in large-scale Flutter apps? Would love to learn from other teams—drop a reply!

📚 Want More?

To follow other decisions we're making across our stack, check out the rest of the series:

🌐 Dio vs Retrofit vs Chopper: Choosing the Best Networking Tool for the New Flutter Application

Choosing the Best Networking Tool for a Scalable Flutter App: Dio vs Retrofit vs Chopper

Kashyap Bhanu Das — Fri, 25 Jul 2025 05:23:56 +0000

When designing the architecture of a large-scale Flutter app, the networking layer is one of the most consequential decisions you'll make. It's not just about making HTTP requests — it directly impacts:

How scalable and maintainable your codebase is
How easily teams can onboard and iterate
How robust your error handling, testing, and debugging flows are

We evaluated Dio, Retrofit, and Chopper — not based on surface-level ergonomics, but on:

How they scale across hundreds of endpoints
How they handle evolving backend contracts
How testable and mockable they are
How safely teams can build in parallel without regressions

🚦 Why This Comparison?

Our requirements for the networking layer were clear. It had to:

Fit clean architecture principles (separation between domain, data, and presentation layers)
Be mockable and testable (abstract service interfaces, interceptors)
Handle dynamic headers, tokens, and retries with clean layering
Scale to hundreds of APIs without growing unmaintainable
Integrate well with CI/CD and analytics tools

🔧 Option 1: Dio — The Manual but Powerful Approach

Dio is a low-level but full-featured HTTP client. It offers:

✅ Fine-grained interceptors, cancellation, and retry support
✅ Great for custom logic, debugging hooks, and analytics
✅ Actively maintained and widely adopted

But it comes with tradeoffs:

⚠️ No compile-time safety — typos or structure mismatches show up at runtime
🧱 High boilerplate for large codebases
🔍 Harder to mock and test — requires custom wrappers or adapters

Example:

Future<LoginResponse> login(LoginRequest request) async {
  final response = await _dio.post('/auth/login', data: request.toJson());
  if (response.statusCode == 200) {
    return LoginResponse.fromJson(response.data);
  } else {
    throw Exception('Failed');
  }
}

Repeat that for 150+ endpoints — and it quickly becomes unscalable.

📦 Option 2: Retrofit — The Declarative, Type-Safe Layer on Dio

Retrofit builds on Dio and uses annotations + code generation to define your APIs.

✅ Minimal boilerplate with clean interfaces
✅ Compile-time safety for endpoints and payloads
✅ Abstract classes make mocking and testing easy
✅ Codegen integrates directly with Dio interceptors

Example:

@RestApi(baseUrl: 'https://api.example.com')
abstract class AuthApi {
  factory AuthApi(Dio dio, {String baseUrl}) = _AuthApi;

  @POST('/auth/login')
  Future<LoginResponse> login(@Body() LoginRequest request);
}

This simplifies both implementation and testing. Your repositories call api.login(), and the response is already deserialized.

Tradeoffs:

⚠️ Less flexibility for highly dynamic APIs
⚠️ Regeneration required on API contract changes
⚠️ No per-method converters or custom response envelopes out-of-the-box

🧩 Option 3: Chopper — Extensible and Service-Oriented

Chopper offers Retrofit-like structure but is built on top of http.Client, not Dio.

✅ Supports service-oriented patterns
✅ Response wrappers include headers and status
✅ Custom converters and plugin-style extensibility

Example:

@ChopperApi(baseUrl: "/v1")
abstract class ApiService extends ChopperService {
  static ApiService create() => _$ApiService();

  @Get(path: "/user/json")
  Future<Response<User>> getJsonUser();
}

Tradeoffs:

📉 Smaller ecosystem than Dio or Retrofit
⚠️ No Dio support (interceptors, retries, etc.)
🛠️ Slightly more boilerplate than Retrofit
🔌 Fewer plugins for advanced use cases (e.g. file upload, custom auth flows)

🔄 Real-World Refactor Example

Before (Dio):

Future<OtpResponse> requestOtp(OtpRequest request) async {
  final response = await _dio.post('/auth/request-otp', data: request.toJson());
  if (response.statusCode == 200) {
    return OtpResponse.fromJson(response.data);
  } else {
    throw Exception('Failed');
  }
}

After (Retrofit):

@POST('/auth/request-otp')
Future<OtpResponse> requestOtp(@Body() OtpRequest request);

Now it's just:

final result = await api.requestOtp(request);

✅ Deserialized
✅ Error-wrapped
✅ Testable and clean

🧠 Decision: Retrofit + Dio

For a Flutter app built at scale, stacking Retrofit on top of Dio gave us the best of both worlds:

Structure and codegen from Retrofit
Power and configurability from Dio
Clear architectural boundaries for domain/data separation
Easy testing and mocking via abstract interfaces

🚫 Why Not Chopper?

Chopper is conceptually solid and extensible, but the lack of Dio support, smaller ecosystem, and limited plugin integrations made it harder to justify. If it supported Dio natively, we might have made a different call.

🧱 Dio Remains the Foundation

Even with Retrofit, Dio stays under the hood — handling interceptors, base config, cancellation tokens, and retries. Retrofit simply adds a declarative, testable layer on top.

🔍 Impact

✅ When having 150+ endpoints, we estimate 5,000–8,000 lines of code will be either eliminated or simplified by switching to Retrofit.
✅ Onboarding new devs is faster — clear Retrofit interfaces, minimal custom parsing
✅ ~4x faster test setup, fewer test-specific implementations per feature

🧩 Bonus: What Else

As part of architecture evolution, we’re also reviewing:

State management (Bloc vs Riverpod vs Provider vs GetX)
Flutter CI/CD pipelines (Azure DevOps, Firebase, fastlane)
Error and analytics instrumentation

💬 Your Turn

How is your team structuring the networking layer in large Flutter apps?
Are you using raw clients, Retrofit, or something else?

Let’s exchange notes — drop your thoughts below 👇

🔗 Further Reading

📖 Full article on Medium