Forem: Artem Goncharov

MR/AR devices as the next generation of exocortexes

Artem Goncharov — Tue, 29 Nov 2022 07:55:43 +0000

TL;DR

The next gen exocortex would proactively help us to do whatever we do by offloading some work to itself without us telling it to do so. The future App Store would be the Skill Store — we will be able to download skills right into our MR/AR devices, which are the best-equipped computing devices to be the next gen exocortex.

What is an exocortex?

An exocortex is an external information processing system that augments the brain’s biological high-level cognitive processes. [1]

In Wikipedia, there is no such article and I’m being redirected to the page about BCI (brain-computer interface) [2] –computer_interface which is a bit different from the notion of exocortex and describes the interface between two computing devices — the brain and the exocortex itself.
Strictly speaking, we can say that exocortex is any external system that enhances our learning or thinking process. A notepad is one of the examples: when we write, think, read what we write, then write again — we use the simplest exocortex ever.

Currently, any personal electronic device is an exocortex, especially a mobile phone. We offload a lot of storage tasks to our phones, and we use it as a notepad, as a scheduling tool, it helps with learning and solving simple tasks.

What an exocortex consists of?

I think the modern exocortex is a computing device with an interface to our internal computing device (brain). So, very high level there are three parts it contains:

The computing device is just a CPU/GPU + memory. Nothing interesting here.
Interfaces to other computing devices, including the human brain are very diverse — starting from Wi-Fi, mouse, keyboard, and monitor for PC, and touchscreen for mobile phones and ending with BCI ( that interacts with brain neurons directly.
Model — it’s a tricky thing. The models inside the exocortex are being computed with the computing device and should imitate models in our brain on multiple layers. One of the layers is a communication layer and here an interaction design can be used [3]. The core part of the model in the device is the set of very high-level abstractions that have counterparts in our brain — we name them applications. For instance, let’s take the application “Notes” — it models the notes catalog, it means in the brain we have a model/idea of the catalogue/list of notes, where each note is a piece of content about something. So the exocortex supports our brain by helping to offload computations of some models to itself and provides the result of computations using a protocol (interaction design) over the interface from (2).

How exocortex works?

The main process looks like this: our brain gets some input from the external world or from the internals of the brain itself, then it decides to activate the exocortex, sends some info to the exocortex using available interfaces, the exocortex makes some models calculations and then returns the result of the calculations, which our brain uses for its own calculations.

There are a lot of moving parts. The brain needs to switch between different activities here — thinking about a problem, deciding to use the exocortex, deciding what interface to use, encoding the data to be processed, passing the encoded data to the exocortex, then periodically checking if the results are already available, decoding the results. Of course, our brain is super adaptable and we usually don’t notice all those activities — they just occur. However, there is a cognitive power we spend on these activities anyway, any switch means losing focus and increased chances to be distracted by something else (new message from a friend while translating the word in the translate app).

v How to make a better exocortex?

I believe the fewer moving parts we have the more efficiently we use our exocortex and the more enjoyable is the process. It’s one of the reasons we start using phones as an exocortex — switch time is lesser than when we use a PC, the same is for the voice assistants which make switch for some tasks super quick.
For a PC or a notebook one needs to turn it on, find the app, run it, use mouse and keyboard as a rather slow input. For mobile phones usually, there is no hassle of finding the device and turning it on, the input is quicker in some tasks and slower in others. For voice assistant there is almost no time for a switch — just ask the question (yes, we still need encoding, communicating, and decoding but for most of people speech is faster than typing + no need to run the specific app).

What would be the next steps to make the exocortexes even more efficient? I see two ways of doing it:

We can continue optimizing interfaces to be less disruptive, probably the final target here is neural BCI which connects the brain and exocortex directly. However, we would still need to have an interactions layer and a high level protocol that connects models in a brain and in a computing device. So it could turn out that we will not save a lot of time because of this direct interface.
We can make exocortex to be proactive, driven by its own AI. In this case, the exocortex should have the same inputs as our brain and then it would predict the tasks our brain is trying to solve and proactively pre-calculate some models and provide the results to the brain. Of course, the AI should learn to provide what is really needed for this particular brain at this particular moment. This way can considerably decrease the switch time or even get rid of it completely!

The next gen exocortex

I think the second way is very promising. Just imagine, you are somewhere in France in a bakery shop and want to buy a loaf of artisan bread, but unfortunately, you can’t speak French, however, your AI driven exocortex can easily understand what task you are working on (buying bread) and proactively provides you with some phrases that can be of use now. You choose and say one of them — AI understands what phrase you use and learn from this fact, making the following advice even more accurate. The way it works looks super useful to me.

The only problem with the second way is that the exocortex device should have the same input your brain has + the output/results of models calculations should be available to our brain constantly. Obviously, the future MR/AR devices [4] are the best equipped for this task as they see and hear the same as you do + the results of models calculations are available to you any time as an overlay above the real world.
Interestingly, in this case, the models (or apps) would be very different from the current apps that we see every day in our phones or PCs. The future models would more resemble the real models in our brain that are being activated by specific contexts automatically — so it could be that multiple models would be activated by the same context and will fight for the chance to provide help. It resembles one of the models of consciousness (Global Workplace Therory) where different object on the scene of our view fight for the attention of different parts of our brain areas and one with the largest number of activated areas wins the attention [5]. For instance, when you cook, two models, for cooking and for healthy habits could fight each other because the context activates both of them. In this case, you would get help from both or from one that wins.

The result would be that the App Store for MR/AR devices as an exocortex will look like the Skill Store: users would install models that support the required activity to their exocortex (MR/AR device) — one for translating, one for calculating, one for drawing, one for woodworking, one for management and so on.

For example, I’m going to learn to play the guitar, I open the Skill Store and choose the skill “guitar playing “ — it’s downloaded to my brand new MR device and once I take a guitar and look at the strings the new skill is being activated. It starts showing me accords right on the guitar, drums the beat and attracts my attention to mistakes, and provides some help to fix them.

Another example: when I install the skill “bouldering”, go to an indoor bouldering center and start climbing, my AR device starts helping me by counting the climbs and their complexity, highlighting the holds in the route, and letting me know about breaks when my heart rate is spiking.

A new way of developing models for the next gen exocortex

The way the models/apps/skills are being developed would change as well. Skills are not the mobile apps, they are much more complex and we may need to change our thinking process in order to learn how to develop skills for the next gen exocortex.

For instance, we may apply systems thinking and notice that if we use the usual exocortex, like a mobile phone or assistant, we use it as a technology in one of our practices. However, if we use an exocortex that is equipped with AI, we can consider the exocortex as the active team member, which at each moment of time knows what practice we are doing, what the lifecycle of the outcome of the practice is, and proactively plays some sub-roles of our role, does its practices and provides outcomes of those practices, so once we need them — they are already ready.

Systems thinking can help us to understand what practices the exocortex would help us with, what would be the lifecycle of the outcome for each practice and how we may split the models in the most efficient way.

References

[1] Exocortex | Transhumanism Wiki | Fandom
[2] Brain — Wikipedia
[3] Interaction design — Wikipedia
[4] Virtual Reality vs. Augmented Reality vs. Mixed Reality — Intel
[5] Frontiers | Global Workspace Theory (GWT) and Prefrontal Cortex: Recent Developments

Some resources about systems thinking: isss.org, eem.institute.

Disclaimer: Opinions expressed in this article are solely my own and do not express the views or opinions of any company including my previous or current employers.

Future of E2E testing

Artem Goncharov — Sun, 07 Aug 2022 14:26:56 +0000

What do we want from E2E testing?
E2E testing outcome
Current situation with E2E testing
Problems we have now with E2E testing
One ideal solution
The ideal solution outcome
Virtual users solution
Would the outcome be the same as in the ideal solution?
Some words about the design and implementation of the virtual users
Summary

In this article I’m trying to anticipate the evolution of End-to-End (E2E) testing in the near future.

There is a lot of info about E2E testing, for example, this one.

What do we want from E2E testing?

By using E2E testing we want to make sure that from the real user perspective all flows in our application work fine and that the user can successfully reach their goal using the application.

E2E testing outcome

As a result of testing we would have the following artefacts:

All the testing scenarios with the % of testers/runs who successfully finish them.
Bug reports with enough info to reproduce the bugs.
Using the above outcome we can decide if the new version of our app is ready to be released in production and give us enough info to understand in which system we have bugs and how to repro them.

Current situation with E2E testing

Currently, in order to implement E2E testing, we need to prepare/update the testing scenarios, prepare test data (test accounts, test cards, etc), ask manual testers to test the application using the testing scenarios, or automate running those scenarios using different tools and methods.

Problems we have now with E2E testing

There are quite a few problems with E2E testing. We can enlist some:

Considerable efforts to keep testing scenarios up to date every time a new version is being released or some code changes are being done.
In the case of manual testing, we have to sacrifice some combination of user parameters like country, language, etc., in order to reduce the number of testing hours that can make the results of testing not so reliable.
In the case of automation, there are a lot of technical challenges on each platform, especially on mobile, mostly related to the way of searching the proper UI elements and reliability of interaction with them and with device/browser.

One ideal solution

Then the question is how the ideal solution could look like. How do we see the perfect E2E testing?

What if we can have all our real users use our application, do whatever they need, and report to us if they find a bug or inconvenience. They should not stop using the app or leave a bad rating in stores or call our support, instead, they should just send us well-prepared reports with enough info to reproduce and fix bugs.

This approach definitely satisfies the definition of the E2E testing and it doesn’t have those problems from the previous chapter and actually, it sounds like perfect E2E testing, at least from my point of view.

The ideal solution outcome

In the case of the ideal solution we would have the same artefacts as from the usual E2E testing plus some more:

By using the business analytics and metrics that we anyway should have in our application we would know how many users participated, how many of them successfully do what they want, how many errors were caught, and a lot of other product specific metrics that we would not have in case of usual E2E testing.
Bug reports with enough info to reproduce the bugs.
Reports about services load from our backend.
Reports about any errors in our systems, especially on the boundaries of the systems.
We even can make some A/B testing before officially releasing the app :)

Sounds rather cool. Of course, there are its own difficulties around usage of special environments and organisation of the process but it seems possible to solve them as we’ve already done that for the usual E2E testing.

The next question is how we can make that dream come true without bothering our real users.

Virtual users

What if we can invent a virtual user (AI driven) who would install our application or open our website and use it as the real user with a purpose in their mind (or maybe more correct — in our Neural Network). Then we summon as many such users as we need and they will be doing exactly what we described in the ideal solution.

However, there are two requirements that make the task rather hard:

Those users should behave differently in order to imitate the diversity in the real users. And they shouldn’t behave chaotically, they should be trying to reach their goal (buy a chair, search for a movie, or anything else).
In order to reduce the load to our environment and to the machinery that emulate those virtual users we’d like to make their number as small as possible but enough to cover all the functionality.

One way of solving the both above requirements is to set up different types of users that will match the different groups of the real users. For example, our app is a mega online store, so we can, probably, define the following groups of users using their intentions:

Search for one specific thing a lot, add many options in the basket, then choose one and buy it (intent to choose one thing from one category and buy it ASAP)
Search for a group of similar things, rarely add something to the basket, almost never buy (intent to explore a broad category of goods)
Come by a direct link from outside and immediately buy smth (intent to buy the specific thing)
…

I’m not an expert in user behaviour, it’s just what comes to my mind first.

The main idea is that there are some common behavior patterns and they can be reproduced by our virtual users. However, the above patternd are not the testing scenarios. The testing scenarios are hardcoded and should give the tester or testing pipeline clear instructions: tap this, input that, check this, and so on. On the contrast the behaviors or patterns are very general, they are on a higher level of abstraction: “search for black office chair under 100 pounds with an overall rating bigger than 4.5 and check all reviews so there are no any injures mentioned, then buy it using {card} and {address}”.

A virtual user will then imitate the real users — trying to understand UI, find the needed elements on the screen, and interact with them in the same way the real users do.

Using this approach we can summon few virtual users per group and “ask” them to use our app in the same way the real users do it.

Would the outcome from the virtual users solution be the same as in the ideal solution?

The outcome would be pretty much the same, so we will have the same stats about users who participated and successfully reached their goals, the same reports about any bugs that prevent our virtual users to reach their goals, reports about backend errors that didn’t impact user experience, and so on.

What will be missing:

Load testing. It depends on the capacity of our environment and testing pipeline actually, so theoretically we can summon thousands or millions of virtual users but the costs of such testing would be super high.
Proper business metrics about the real users’ behavior. There are a lot of factors that impact the real users’ behavior — from the political situation in the world to the weather and modern trends. But nothing impacts our virtual users — they will behave almost the same way every time we summon them. Sure thing we can re-train them if something really important changes in the world. Thus the data about virtual users behavior can’t help product owners to evolve the product but it can help them to measure how easy using our application is by measuring the time the virtual users spend to reach their goals. It’s, actually, a really interesting idea to use the virtual users in order to do UX research of our apps. If an image on a button is not easily understandable for the virtual user it will be the same for the real one.

Some words about the design and implementation

How would we implement the virtual users?

Let’s see what would we need to make the virtual users do what we want.

There could be the following layers:

[UI] UI that we should already have in the app
[UI Parser] A system that understands the UI. So it can extract all the UI elements, be able to interact with them, and understand their purpose using text or images on elements. Ideally it shouldn’t rely on any knowledge about the internals of the UI — it should analyse a plain picture of the interface in the same way a real user does it.
[Solver] A system that can expand the goal into a series of steps to reach it. If you want to buy a black chair you need to search for one first, you want to use filters and keywords, then you want to check it out and finally pay for it.
[Goal setter] The very high-level system that translates the pattern or the needed behavior in a text form into the virtual user’s goal.

We can think of layer 3 as a core of the system, layer 4 as an input, layer 2 as an adapter from core to any UI (web, mobile, PC apps, or even something special like medical equipment with touch screens or industrial monitoring systems with hardware buttons).

How could the above example "Buying a black office chair" work here?

Layer 4

Layer 4 gets the text input, analyzes and extracts all the important features from it. It could look like this:

Main goal: buy
Subject: chair
Subject properties: black, office
Limitations: price (100 pounds), rating (better than 4.5), reviews (not contain injures).
Account: …
Card: …

I suppose the best way to do that is to use machine learning and train it to extract the meaning from the text, probably, there are already some on the market.

Layer 3

This one is a bit tricky. It should look like a problem solver and for each task provides the algorithm or the sequence of steps to solve it. In our example, the steps could look like this:

Open the search page
Search for a subject “chair” with keywords: black and office.
Check the list of options, if there is none -> return error
Filter out all the items with a price bigger than 100 and rating less than 4.5
Iterate the list and choose the first one where reviews don’t contain keyword injures
Proceed to checkout of the chosen item
If it’s available buy it using credentials {card}

I can see two ways of implementing this:

Use the real users’ analytics data, label a lot of different funnels and user’s journeys with goals and properties, and then use this data to train NN that can do the opposite — using a goal and properties suggests the sequence of events that lead to the goal.
Once layers 1 and 2 are ready we can use reinforcement learning to train NN to reach different goals in the real application.

I appreciate that I don't have much knowledge in machine learning and maybe there are many other ways to implement this layer but I’m sure that the task is very interesting, challenging, and, what is more important, promising as the results can be reused in many other fields like robotics, game AI and many more.

Layer 2

This layer is very important as it gives us the real interface to any application. What I mean here is that UI is rather different in web, mobile apps, and PC apps and the way we can interact with the UI is different too. Layer 2 abstracts it and provides us with something more high level.

In our example, Layer 2 will be constantly analyzing the page it sees (using screenshots, some special tools like ‘adb’ for android or something else), extract all UI elements, and “understand” them, so when it receives command “Open search page” from Layer 3 it looks up something that has the meaning of search — button with the text “Search”, magnifying glass icon, menu with the text “Search”, etc, and then activates it by tapping on it (using special tools or just tapping on screen using elements coordinates).

This layer alone can be rather useful for UI testing as it gives the opportunity to interact with any UI universally and does not depend on special ‘ids’ of elements or special tooling for interacting with internal UI representation.

We have already made a PoC for this layer and it looks very promising. Hopefully, in the next articles, we can reveal more about it.

Layer 1

This is just the UI of our app and the implemetation of a bidirectional way of interacting with it. We already mentioned some options in the previous section — like taking screenshots and tapping on screen using coordinates of the UI elements.

Summary

In this article, we discussed one possible way of E2E testing evolution and suggested some ways of going there.

We think that the future of E2E testing in the hands of virtual users (AI) who can test our apps in the same way our real users use them every day.

It’s definitely the challenge to make that AI work but each step to the end state can bring its own value and can reveal new unknown ways of testing, using the apps, and even thinking about the apps.

First of all, I’m super happy that somebody reaches this far and I’ll be even happier to discuss this topic in comments to the article or in any other appropriate place.

Feel free to share your ideas, critics, and comments.

Three things you didn’t know about exception handling in Dart and Flutter

Artem Goncharov — Sat, 06 Aug 2022 18:23:04 +0000

What is an exception?
Exceptions handling in different languages
Exception handling in Dart and Flutter
* Synchronous code
* Asynchronous code
* Zones and runZonedGuarded
* Error handling in streams
* Unhandled exceptions
Wrap up

Sorry for the clickbait title of the article, I’m experimenting with different styles… But I’d like to not disappoint you and will provide all those 3 things :)

What is an exception?

There are two types of exception definition:

An exception is an inability of an operation to fulfill its contract
An exception is an unexpected behavior

It seems that the first definition can include the second one as the unexpected behavior can be one of the reasons why the operation couldn’t fulfill its contract. The other reasons could include more or less expected problems — like connection loss, corruption of the file, and so on.

As an example let’s consider this function:

Future<Money> getRemainingAmount({Account account}) async {
  final response = await backend.getAcccountInfo(account);
  return response.remainingAmount;
}

This function returns the remaining money in our account. In order to do this, the function should hit our backend and request the account balance and this operation can fail because of various reasons — the network is down, the server is down, the contract between frontend and backend was changed, etc… All those reasons lead to an inability of our operation to fulfill its contract. However, all of those problems can be foreseen and we can add them to the contract. The only thing we can’t expect is unexpected behavior :). For instance, we anticipated all the problems except one: when backend in some cases returns text “negative” instead of the balance of our account — so this becomes unexpected for us (it’s a completely artificial example but just to give you an idea).

The more we know about our system the less chance some unexpected behavior occurs.

The next question is what we are going to do in all those exceptional cases. Do we want to react differently for each case or maybe we don’t care much if the server is down or returns some unexpected data? It’s the question of our system design but anyway we need a way to catch those exceptions separately or as a general exceptional situation so that we can respond accordingly.

Exceptions handling in different languages

The above considerations were taken into account differently in different programming languages. Let’s consider three approaches that Go, Rust and Dart use to handle exceptions (I'm not an expert in Go and Rust so feel free to comment if you found a problem with the code).

Dart:

try {
  final remaingAmount = await getRemainingAmount(account: account);
  displayAccountAmount(remaingAmount);
} on SocketException catch (_) {
  displayNoConnectionError();
} catch (e, s) {
  logException(exception: e, stackTrace: s);
  rethrow;
}

Go:

func Sqrt(f float64) (float64, error) {
    if f < 0 {
        return 0, errors.New("math: square root of negative number")
    }
    // implementation
}

f, err := Sqrt(-1)
if err != nil {
    fmt.Println(err)
}

Rust:

use std::fs::File;
fn main() {
   let f = File::open("main.jpg");   // main.jpg doesn't exist
   match f {
      Ok(f)=> {
         println!("file found {:?}",f);
      },
      Err(e)=> {
         println!("file not found \n{:?}",e);   //handled error
      }
   }
   println!("end of main");
}

The most common approach which was used in C++ and is being used in Dart is to pay attention mainly to the happy path. So the code is rather easy to read and it’s not contaminated with many error branches. On the other side, it’s not that clear which block of code can cause an error and what kind of error it could be. For instance, in the above example, if you look inside the function getRemainingAmount you, most likely, will not be able to say if it would throw an exception or not.
In Go it’s all the way around, the main focus is on error handling. So a function, besides a value, should return an error that has to be handled right after a function call. If the function can’t produce an error it should return a result only, and, it will clearly indicate that function is error-free. There is no way to silently ignore an error in a hope that upstream code will handle it — sure thing you can ignore an error but that error will not be automatically propagated to the upstream, you have to do it explicitly.
In Rust, there is the third way of dealing with exceptions/errors. It looks similar to Go: function can return Either type with value OR error and then you can use pattern matching to understand if it’s value or error and what kind of error was returned. It’s not mandatory but there is no other way to throw an exception (except panic but it’s another story) so that you probably want to return Either type from all the functions that can cause errors. Code looks really readable and you can’t just pass exception/error upstream without handling it (only if the exception type is exactly the same as in an upstream function).

Any of those three approaches can be used in your code in Dart, you can return an object with value AND error, or result with Either value OR error, or use traditional try-catch blocks and other options Dart provides you out of the box.

Let’s explore what exceptions handling options Dart provides us with.

Exception handling in Dart and Flutter

What are the situations where we need to use exception handling?

Synchronous code
Asynchronous code
Timers
Streams
Flutter app as a whole

Synchronous code

The main tool for handing the exceptions in Dart is try-catch-finally construction — the same as in many other languages.

In order to catch all the exceptions in a block of code you wrap the code in try block and use multiple on-catch instructions to handle some specific exceptions, then use catch to handle all other unexpected exceptions, and finally use finally to run the code that should be invoked after the code in try block and in any of the catch blocks. Code in finally will be called even if you return from try or catch blocks or re-throw the exception in catch block.

Instruction rethrow helps you to throw the same exception after you handle it so the upstream code will take care of it.

try {
    final _ = 100 ~/ 0;
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } finally {
    print('Clean-up done 1');
  }

  print('\n\n');

  try {
    throw FormatException('format is wrong');
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } on FormatException catch (e) {
    print('Format exceptions $e');
  } finally {
    print('Clean-up done 2');
  }

  print('\n\n');

  try {
    print('Allocate something 3');
    throw 'Other error occurred';
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } on FormatException catch (e) {
    print('Format exceptions: $e');
  } catch (e, s) {
    print('Unknown exceptions: $e, with stack: \n$s');
    rethrow;
  } finally {
    print('Clean-up something 3');
  }

Output:

Cant divide to zero
Clean-up done 1
Format exceptions FormatException: format is wrong
Clean-up done 2
Allocate something 3
Unknown exceptions: Other error occurred, with stack: 
#0      main (file:///Users/user/Downloads/dart_sample/bin/sync_exceptions.dart:26:5)
#1      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#2      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
Clean-up something 3
Unhandled exception:
Other error occurred
#0      main (file:///Users/user/Downloads/dart_sample/bin/sync_exceptions.dart:26:5)
#1      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:299:32)
#2      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

We can see that all three try-catch blocks catch the exceptions and in the third block, the exception was re-thrown after clean-up was done.

This is pretty much it for the handling exceptions in synchronous blocks of code. You can check for more details here — https://dart.dev/guides/language/language-tour#catch

Asynchronous code

Let’s consider this example:

try {
    Future.delayed(Duration(seconds: 1), () {
      final _ = 100 ~/ 0;
    });
    print('Everything is fine!');
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } finally {
    print('Clean-up done');
  }

Output:

Everything is fine!
Clean-up done
Unhandled exception:
IntegerDivisionByZeroException
#0      int.~/ (dart:core-patch/integers.dart:22:7)
#1      main.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/async_exceptions.dart:6:21)
#2      new Future.delayed.<anonymous closure> (dart:async/future.dart:325:39)
#3      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#4      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#5      _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#6      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

We’d like to handle the exception in the Future delayed for one second. It’s natural that this code fails to catch the exception as the code in the try-catch block just schedules the _computation_ (Futures callback/closure) to be invoked one second after try-catch finishes its work.

It would look something like this on a timeline:

How to fix it? There are many ways of fixing it, for instance, by adding await instruction before the Future. It forces computation to be invoked inside the try-catch and thus the try block will be able to catch and handle the exception successfully.

try {
    await Future.delayed(Duration(seconds: 1), () {
      final _ = 100 ~/ 0;
    });
    print('Everything is fine!');
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } finally {
    print('Clean-up done');
  }

Output:

Cant divide to zero
Clean-up done

Now it works fine, we managed to catch the exception and make a clean-up after it.

We can represent the new code like this:

What if we have a chain of futures? No problem at all, try-catch will catch exceptions in all of them as long as we insert await before the first of them.

try {
    await Future.delayed(Duration(seconds: 1), () {
      print('Its ok so far');
    }).then((value) {
      final _ = 100 ~/ 0;
    });
    print('Everything is fine!');
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } finally {
    print('Clean-up done');
  }

Output:

Its ok so far
Cant divide to zero
Clean-up done

If you don’t like the above way of catching the exceptions you can use a dot syntax and catchError function.

await Future.delayed(Duration(seconds: 1), () {
    print('Its ok so far');
  }).then((value) {
    final _ = 100 ~/ 0;
  }).catchError((error) {
    print('Cant divide to zero');
  }).whenComplete(() {
    print('Clean-up done');
  });

Output:

Its ok so far
Cant divide to zero
Clean-up done

In the above code, catchError is an analog of catch block and whenComplete is an analog of finally block. Function catchError catches exceptions in all the code that located above it, so if you suspect that whenComplete can throw an exception you can add one more catchError after it.

await Future.delayed(Duration(seconds: 1), () {
    print('Its ok so far');
  }).then((value) {
    final _ = 100 ~/ 0;
  }).catchError((error) {
    print('Cant divide to zero');
  }).whenComplete(() {
    print('Clean-up done');
    throw 'ss';
  }).catchError((error) {
    print('Oh, error in the finally block');
  });

Output:

Its ok so far
Cant divide to zero
Clean-up done
Oh, error in the finally block

You can use exceptions matching using test parameter of a catchError function.

await Future.delayed(Duration(seconds: 1), () {
    print('Its fie so far');
  }).then((value) {
    print('Then started');
    throw 'random exception';
  }).catchError((error) {
    print('Cant divide to zero');
  }, test: (e) => e is IntegerDivisionByZeroException).catchError((error) {
    print('All other exceptions: $error');
  }, test: (e) => true).whenComplete(() {
    print('Clean-up done');
  });

Output:

Its fie so far
Then started
All other exceptions: random exception
Clean-up done

Looks not super readable, right? That’s why a Dart code style recommends using await (code style guide)and thus try-catch in such cases. It could look like this:

try {
    await Future.delayed(Duration(seconds: 1), () {
      print('Its fie so far');
    });
    await Future(() {
      print('Then started');
      throw 'random exception';
    });
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } catch (e, s) {
    print('All other exceptions: $e');
  } finally {
    print('Clean-up done');
  }

Output:

Its fie so far
Then started
All other exceptions: random exception
Clean-up done

From my personal point of view, it’s much clearer and more readable.

Additionally, It has one more advantage — it can handle both synchronous and asynchronous code, so if you need to make some calculations between two asynchronous calls you don’t have to create additional synchronous Future (as you would have done in the case of dot syntax), just add this code into try block.

Ooook, this was easy, right? What about this piece of code?

try {
    Timer.run(() {
      print('Timer runs.');
      throw Exception('[Timer] Bad thing happens!');
    });
    print('Everything is fine!');
  } on IntegerDivisionByZeroException {
    print('Cant divide to zero');
  } finally {
    print('Clean-up done');
  }

Everything is fine!
Clean-up done
Timer runs.
Unhandled exception:
Exception: [Timer] Bad thing happens!
#0      main.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/async_exceptions.dart:7:7)
#1      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#2      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#3      _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

:( (smile with crying). Yeah, bad things happen in timers too. And the above code is not good to deal with those bad things as again timer’s callback will be invoked not in the try-catch block and we can’t fix it with await instruction as we can’t wait on Timer.

An obvious solution is to add try-catch inside the timer callback but if we don’t want to contaminate the callbacks with exceptions handling or there is a much complicated asynchronous logic inside the timer’s callback and we want to handle exceptions from that logic in the place where we run timer?

Zones and runZonedGuarded

runZonedGuarded can help you with this and with more complicated cases too!

Zones (dart help) is a concept in Dart that helps to divide your code into error free zones where any exceptions in any pieces of asynchronous code will be handled in one place: zone’s uncaught error handler.

How does it work? Each callback or computation in timers, microtasks, and Futures is being wrapped into a special handler in the Zone. The handler wraps the callback in try-catch and if catch is being triggered Zone invokes an uncaught error handler where you can handle those errors.

Each piece of code is being run in some Zone, even you don’t create your custom Zone it would be invoked in the Root Zone which means we are safe anyway!

Let’s see how we can fix the problem with the timer:

runZonedGuarded((){
    Timer.run(() {
      print('Timer runs.');
      throw Exception('[Timer] Bad thing happens!');
    });
    print('Everything is fine!');
  }, (e,s) {
    print('Exception handled $e, \n$s');
  });

Output:

Everything is fine!
Timer runs.
Exception handled Exception: [Timer] Bad thing happens!, 
#0      main.<anonymous closure>.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/async_exceptions.dart:7:7)
#1      _rootRun (dart:async/zone.dart:1182:47)
#2      _CustomZone.run (dart:async/zone.dart:1093:19)
#3      _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#4      _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#5      _rootRun (dart:async/zone.dart:1190:13)
#6      _CustomZone.run (dart:async/zone.dart:1093:19)
#7      _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:1021:23)
#8      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#9      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#10     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

Easy! Just wrap your code into runZonedGuarded and add an error handler. No exception can squeeze through this construct… until we will not throw the exception in the error handler itself as the error handler is being invoked in the parent zone, so, be careful.

You can notice that the stack trace becomes longer as now async callbacks are wrapped into our _CustomZone handlers.

Zones is a very powerful tool that can help you to isolate some parts of your complex asynchronous logic from the other parts of the app. As we will see later it’s also rather useful when you use streams and for handling unhandled exceptions in Flutter apps.

If we would try to represent handling of exceptions is zones it could look like this:

In the about image we can see that all the asynchronous callbacks are covered by the zone’s exception handling if we schedule those callbacks inside the zone.

Additionally, Zones helps you to intercept the creation of timers and print operators, and, keep some parameters in the zone’s context (it’s a dictionary that you can pass to the zone while you creating it). All those advanced things are being used in tests mainly, as for my knowledge.

This is an example of how we can use nested zones and zone’s context parameters — I added the parameter ZoneName where I write Zone name and display it in prints.

 print('Current zone start in: ${Zone.current.toString()}');
  runZonedGuarded(() {
    print(
        'Current zone inside runZoned: ${Zone.current.toString()} with name ${Zone.current['ZoneName']}');
    Timer.run(() {
      print('Timer runs.');
      throw Exception('[Timer] Bad thing happens!');
    });
    runZonedGuarded(
          () {
        print(
            'Current zone (1) inside runZoned: ${Zone.current.toString()} with name ${Zone.current['ZoneName']}');
        Timer.run(() {
          print('Timer 1 runs. ');
          throw Exception('[Timer 1] Bad thing happens!');
        });
      },
          (e, s) {
        print(
            'Current zone (1) inside catch: ${Zone.current.toString()} with name ${Zone.current['ZoneName']}');
        print('Exception handled (1) $e, \n$s');
      },
      zoneValues: {'ZoneName': 'Second zone'},
    );
    print('Everything is fine!');
  }, (e, s) {
    print(
        'Current zone inside catch: ${Zone.current.toString()} with name ${Zone.current['ZoneName']}');
    print('Exception handled $e, \n$s');
  }, zoneValues: {'ZoneName': 'First zone'});

Output:

Current zone start in: Instance of '_RootZone'
Current zone inside runZoned: Instance of '_CustomZone' with name First zone
Current zone (1) inside runZoned: Instance of '_CustomZone' with name Second zone
Everything is fine!
Timer runs.
Current zone inside catch: Instance of '_RootZone' with name null
Exception handled Exception: [Timer] Bad thing happens!, 
#0      main.<anonymous closure>.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/async_exceptions.dart:10:7)
#1      _rootRun (dart:async/zone.dart:1182:47)
#2      _CustomZone.run (dart:async/zone.dart:1093:19)
#3      _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#4      _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#5      _rootRun (dart:async/zone.dart:1190:13)
#6      _CustomZone.run (dart:async/zone.dart:1093:19)
#7      _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:1021:23)
#8      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#9      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#10     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
Timer 1 runs. 
Current zone (1) inside catch: Instance of '_CustomZone' with name First zone
Exception handled (1) Exception: [Timer 1] Bad thing happens!, 
#0      main.<anonymous closure>.<anonymous closure>.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/async_exceptions.dart:18:11)
#1      _rootRun (dart:async/zone.dart:1182:47)
#2      _CustomZone.run (dart:async/zone.dart:1093:19)
#3      _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#4      _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#5      _rootRun (dart:async/zone.dart:1190:13)
#6      _CustomZone.run (dart:async/zone.dart:1093:19)
#7      _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:1021:23)
#8      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#9      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#10     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

From the above example, you can see that it’s rather convenient to use nested zones to divide your code into error free zones and, probably, use zone’s context to keep so useful info for logging purposes.

Error handling in streams

I promised to reveal how zones can help with handling exceptions when we use streams.

Let’s consider the simple code where one piece of code emits some events to the stream and another one consumes them.

import 'dart:async';

void main(List<String> arguments) async {
    startListener(getStream());
}

Stream<int> getStream() {
  final intsController = StreamController<int>.broadcast();
  int i = 42;
  Timer.periodic(Duration(seconds: 1), (timer) {
    i += 1;
    if (i == 46) throw '[Stream] Bad thing happens!';
    intsController.add(i);
  });

  return intsController.stream;
}

void startListener(Stream<int> ints) {
  ints.listen((event) {
    print(event);
    if (event == 44) {
      throw '[Listener] Bad thing happens!';
    }
  });
}

Output:

43
Unhandled exception:
[Listener] Bad thing happens!
#0      startListener.<anonymous closure> (file:///Users/artemgoncharov/Downloads/dart_sample/bin/streams_exceptions.dart:23:7)
#1      _RootZone.runUnaryGuarded (dart:async/zone.dart:1384:10)
#2      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:357:11)
#3      _DelayedData.perform (dart:async/stream_impl.dart:611:14)
#4      _StreamImplEvents.handleNext (dart:async/stream_impl.dart:730:11)
#5      _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:687:7)
#6      _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#7      _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#8      _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#9      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:404:11)
#10     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#11     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
44

I know this code is not perfect at all :) but it demonstrate the problem — we would have an unhandled exception in the listener. An obvious solution is to add try-catch inside the listener. And again, it could be hard because there could be multiple listeners and some asynchronous logic in place and we may want to handle all the exceptions from the listeners and the business logic in one place.

We wrap listeners creation into one zone and producers creation in another one, and, now we can handle all the exceptions from both sides and log them and/or recover in one place that is very convenient:

import 'dart:async';

void main(List<String> arguments) async {
  Stream<int> stream;

  runZonedGuarded(() {
    stream = getStream();
  }, (e, s) {
    print('PRODUCER ZONE Exception handled: $e, $s');
  });

  runZonedGuarded(() {
    startListener(stream);
  }, (e, s) {
    print('CONSUMER ZONE Exception handled: $e, $s');
  });
}

Stream<int> getStream() {
  final intsController = StreamController<int>.broadcast();
  int i = 42;
  Timer.periodic(Duration(seconds: 1), (timer) {
    i += 1;
    if (i == 46) throw '[Stream] Bad thing happens!';
    intsController.add(i);
  });

  return intsController.stream;
}

void startListener(Stream<int> ints) {
  ints.listen((event) {
    print(event);
    if (event == 44) {
      throw '[Listener] Bad thing happens!';
    }
  });

Output:

43
44
CONSUMER ZONE Exception handled: [Listener] Bad thing happens!, #0      startListener.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/streams_exceptions.dart:35:7)
#1      _rootRunUnary (dart:async/zone.dart:1206:13)
#2      _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#3      _CustomZone.runUnaryGuarded (dart:async/zone.dart:1005:7)
#4      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:357:11)
#5      _DelayedData.perform (dart:async/stream_impl.dart:611:14)
#6      _StreamImplEvents.handleNext (dart:async/stream_impl.dart:730:11)
#7      _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:687:7)
#8      _rootRun (dart:async/zone.dart:1182:47)
#9      _CustomZone.run (dart:async/zone.dart:1093:19)
#10     _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#11     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#12     _rootRun (dart:async/zone.dart:1190:13)
#13     _CustomZone.run (dart:async/zone.dart:1093:19)
#14     _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#15     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#16     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#17     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#18     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#19     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:404:11)
#20     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#21     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)
45
PRODUCER ZONE Exception handled: [Stream] Bad thing happens!, #0      getStream.<anonymous closure> (file:///Users/user/Downloads/dart_sample/bin/streams_exceptions.dart:24:18)
#1      _rootRunUnary (dart:async/zone.dart:1198:47)
#2      _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#3      _CustomZone.runUnaryGuarded (dart:async/zone.dart:1005:7)
#4      _CustomZone.bindUnaryCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1042:26)
#5      _rootRunUnary (dart:async/zone.dart:1206:13)
#6      _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#7      _CustomZone.bindUnaryCallback.<anonymous closure> (dart:async/zone.dart:1026:26)
#8      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:397:19)
#9      _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:428:5)
#10     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)

Warning! I’m not saying we have to give up handling errors as close to the place where they can arise, I’m just saying that sometimes it’s very useful to wrap the complex features to zones in order to catch and properly log the errors that sometimes squeeze through all our downstream bastions.

On this optimistic note, I’m going to get into the last topic: Unhandled exceptions.

Unhandled exceptions

As we saw before all the unhandled exceptions are intercepted by the zone the code is being invoked in. So that if you’d like to log the unhandled exceptions in you flutter app you need just wrap your runApp call into the runZoneGuarded and add logging in the error handler. If you do something else in main function — create providers or singletons — you may want to wrap them to the runZonedGuarded as well.

This helps you to be aware of any unhandled exceptions in your app.

You may argue that you can see all of them in the terminal when you debug your app. This is super convenient for debugging but it has two nuances:

If you debug your app directly on a device (not on Simulator) you will not be seeing the exceptions in the asynchronous code (unless you use runZonedGuarded)
In release builds, I’m sure, you want to be aware of all the unhandled exceptions even if it seems that app is working fine. So that you need to add runZonedGuarded and log all the exceptions to your backend or Firebase/Sentry, or whatever system you use for remote logging

However, runZoneGuarded can’t help you with the exceptions which were already handled by the Flutter framework.
All those exceptions can be logged using FlutterError.onError callback. If you add logging to this callback then anytime Flutter catches an error with rendering or anything in widgets it will reach that callback and will be logged by you. Don’t forget to WidgetsFlutterBinding.ensureInitialized() before adding the onError callback.

void main() {
  WidgetsFlutterBinding.ensureInitialized();

  FlutterError.onError = (FlutterErrorDetails errorDetails) {
    print("onError Exception: $errorDetails was caught by Flutter framework - redirect to Sentry or Firebase.");
  };

  runZonedGuarded(() {
    createProviders();
    runApp(MyApp());
  }, (e, s) {
    print(
        "Synchronous or Asynchronous Exception: $e (stack $s) was caught in our custom zone - redirect to Sentry or Firebase.");
  });
}

Wrap up

It seems that brevity is not one my features but I hope the article was still useful and can be of help for you.

Additionally, if you still haven’t forgotten about the 3 things from the title, here they are:

There are many ways of dealing with exceptions in different languages, try-catch is not the only one, and, it depends on your design what exception handling method you want to use — in Dart you can implement many of them
Use runZoneGuarded to catch exceptions in the complex asynchronous code, streams, and in a whole Flutter app too
Unhandled exceptions in asynchronous code can be silently swallowed (not printed in the terminal) if you debug your app on the real device

This article was written after my presentation (YouTube link) in Singapore Flutter Meetup which you can find here: https://www.meetup.com/Singapore-Flutter-Meetup/events/276139032/

List of sources
https://api.flutter.dev
https://dart.dev/get-dart
https://github.com/flutter/flutter

Yet another mobile architectures comparison

Artem Goncharov — Sat, 06 Aug 2022 17:02:08 +0000

There are a lot of articles about different architecture patterns for mobile applications which try to convince you that this or that architecture pattern is the best one. Sure thing, we all know that there is no a silver bullet… unfortunately. However, it’s a very good exercise to compare the most popular mobile architecture patterns (MVC, MVP, MVVM, VIPER, Riblets and Redux) and find out:

what we can learn from them,
how they can help us to design our app architecture.

Multitier architecture

In order to compare the above mentioned architecture patterns we need a baseline. I chose the multitier (actually 5 tier) architecture as it’s very popular, universal and has tiers/layers that are in fact very high level abstractions so we can use them to understand the abstraction level of the layers in mobile architecture patters.

There are a lot of information about the multitier architecture [1], depending on the type of system some authors find 3 layer (pure backend — Data, Business Logic, Presentation), some 4 or 5.

Each layer is a high level abstraction and the architecture has only one restriction: components in the layer can interact only to the components from the neighbour layers. It allows to decrease the coupling between parts of the application and keep a dependency graph in a manageable state.

So let’s map each mobile architecture pattern to our baseline and see what we can find out.

MVC

Let’s start from MVC: View can be easily mapped to the View layer, Controller we can map to Presentation layer, and Model then should span over the rest of layers and it depends on your product whether you need them or not (in most cases you do). So MVC adds nothing to the multitier architecture and actually just simplifies it a bit.

MVP

Next is MVP. We can map it in the same layers as MVC — just change Controller to Presenter. Additionally, MVP restricts interactions between components so that Model can interact only with Presenter and View can interact only with Presenter as well. In the multitier architecture we have the same restrictions, so MVP again adds nothing.

MVVM

MVVM has a pretty similar mapping. View, as in the previous examples, can be mapped to View layer, View Model in this pattern usually maps to Presentation layer and Model is the rest. The only difference is in a more advanced decoupling between View and ViewModel — here ViewModel knows nothing about View and all interactions from ViewModel to View happen through the binding of UI elements in View layer components to exposed properties in ViewModel components. So MVVM adds conception of binding between View and Presentation layers to the multitier architecture.

VIPER

What about VIPER? It looks like a different beast. View as always maps to View layer, Presenter and Router go to the Presentation layer, Interactor maps to Business logic, and, Entity spans over Service and Data layers. Thus VIPER adds Router layer that helps to extract routing business logic from Presentation layer. But, what is more important, VIPER introduces conception of VIPER modules and describes a universal structure for such a module in an app and by this it starts the confusion between layers and classes names (Interactor usually is a class name that doesn’t make sense as it this name can’t help us to understand the business logic inside the Interactor). Splitting the app to modules is a very useful concept but there is no clear understanding how those modules can interact with each other and moreover share some state.

Redux

Redux seems to be the only pattern that doesn’t cover View at all and concentrates mostly on Business logic as Reducers and State, and, Data + Service Layers as Middleware. Redux is a very good attempt to define the conception of building the “backend” part of our layered model(data, service and business+state layers). By introduction the concepts of state, subscriptions, reducers, actions and middleware Redux helps to fully decouple Business layer from Service and Data layers, and additionally, View + Presentation layers from all the other layers. So Redux doesn’t claim to be a universal architecture pattern, it just provides a way to organise interactions between the layers and keep them independent from each other.

Riblets

Lastly there is Riblets (or RIBs). This pattern can actually be 1–1 mapped to the layered model, Services maps to Data + Service layer, Interactor to Business logic layer, Model Stream looks like State inside Business layer, Presenter and Router maps to Presentation layer and View is actually View layer. Riblets looks very similar to the VIPER as it helps to split your application not only to the vertical layers but horizontally to modules as well. However Riblets fixes some VIPER problems and modules can be united to the hierarchical structure using Routers and communicate and share some state using Interactors.

Wrap up

How this analysis can help us to choose or understand existed mobile architecture patterns? Good question.
Firstly, we can see that not all the design patterns are on the same level of abstraction and moreover they have different intention.

MVC and MVP are very high level design patterns, they are on the same level of abstraction as the multilayered architecture and add nothing to it. So it worths to be aware that Model, View, Controller and Presenter are not the classes but rather abstract layers and you need to come up with the components, classes and interactions between them in each layer of the multitier architecture. They are just hints how you can split functionality of your application.

MVVM is more about interaction between ViewModel(Presenter) and View. If you want to isolate UI from components in Presenter layer — you can try the MVVM way of binding.

VIPER might help you to divide your app into modules with the strictly defined structure and interactions between components in the module. So it’s a lower level of abstraction and it has less freedom and flexibility if you follow VIPER ideology but it’s good as a starting point to learn about modularisation.

Redux is the only design pattern that takes the state and the business logic very seriously. Again Redux is located on the even lower level of abstraction and will considerably restrict you in a way of layers/components interaction if you chose it as it is. But Redux gives us an idea about the state and the business rules management and additionally it can be easily coupled with any of above patterns in this role (for instance, you may try to use Redux as a Model layer in MVP pattern).

Riblets seems to be on the lowest level of abstraction and it dictates you how to make hierarchical modules in your app, how to build them, interact between components in a module and even tries to take care of the app state. So if you think of you application as a hierarchy of modules and you have no idea where to start — it’s time to learn Riblets and see how it’s done there.

References:

[1] https://en.wikipedia.org/wiki/Multitier_architecture, https://www.researchgate.net/figure/The-5-tier-Architecture-model_fig1_280235378, https://www.oreilly.com/library/view/core-j2eetm-patterns/0130648841/0130648841_ch06lev1sec1.html

Grokking the Mobile System Design interview

Artem Goncharov — Sat, 06 Aug 2022 14:21:59 +0000

Purpose
How mobile system design interview usually looks like
Common recommendations
Proposed interview answer plan
- Business idea
- Requirements clarification
- Mathematical model design (if needed)
- High level system design
- API design
- High level client side design
- Detailed design of some modules
- One complex case and detailed discussion
Summary
References

Purpose

Being a mobile developer you may know that it’s a problem to find any preparation materials for mobile system design interview. There are plenty of videos about backend system design interviews and just few of them about mobile one. The same situation is with the articles, one can find a lot of articles and even courses that would help to be prepared to the backend system design interview and there is no articles at all about mobile system design interviews. That’s why I thought it would be great to share my experience of the preparation and passing the mobile system design interviews.

Sure thing this article is just one way of grokking the interview, and the outcome will depend not only on information you’ve got from this article but on your knowledge, skills and on interviews skills and knowledge as well, so don’t consider this info as a silver bullet.

How mobile system design interview usually looks like

First let’s explore how mobile system design interview looks like, what are the input, the process and the expectations.

Process

Usually the interview takes about 45 minutes:

5 minutes - acquaintance
5–10 minutes - defining the task
25–30 minutes - solution
5 minutes - your questions to an interviewer.

Commonly your task is to design the mobile application or a part of the application and sometimes to design some part of backend that is relevant to the mobile application as well. An interviewer lets you know all the needed information during the second 5–10 minutes, the input can vary from company to company.

Input

Let’s see what can be provided to you:

The idea of the new feature or of the whole application. Usually it sounds like — “Let’s design the Instagram like application” or “Given you work in Twitter let’s add the the feature of combining some of your posts to stories”. Nothing more you commonly hear from the interviewer, all additionally info you need to get from them using properly formulated questions — the technique will be described later in the article.
The sketches of the screens needed to implement the idea from the first item. It’s a very important piece of the information and you have to be very attentive in order to absorb all the nuances.
Some requirements and limitations, like in what extent you need to design the system — mobile only, or mobile and API only, or mobile and backend.

That’s it, no more input you most likely have from the interviewer.

Expectations

High-level expectations:

Ability to understand and solve complex problems
Communicate clearly and effectively
Having relevant experience and knowledge The way of going deeper from this abstract definitions mainly depends on the interviewer, theirs skillset, experience and position in the company. They can define complex problems very differently, but we will try to formulate the most common concrete signals you should demonstrate.

For the first item, in order to show your ability to understand and solve complex problems, you should demonstrate the following:

Ability to think about the business part of the task as this is the first step in understanding the problem and answering the question “Why?”. You would show the attitude to understand the business purpose of the feature or the whole application and apply this knowledge in the process of clarifying or generating the technical requirements.
Working well with the vague requirements. The interviewer usually gives you very less information and this info is definitely not enough to even start designing, thus you should be able to ask proper questions in order to clarify the requirements. The requirements clarification process should be driven by definite and clear aims, questions shouldn’t be random, but should rather follow some system.
Ability to synthesise the solution for a given problem. In the first two items you demonstrated ability to understand the problem, next step is to apply all the available knowledge and design a reasonable solution. You would synthesise the design using the “top to bottom” approach in order to provide the context and the main conceptions first, and then, using divide and conquer technics, gradually split the system to smaller and smaller pieces.

The second item — effective and clear communication — could include:

Voice communication. You should speak for about 30 minutes without any big pauses explaining all your solutions and ideas with enough confidence, knowledge and ability to defend your ideas
Whiteboard communication. A whiteboard is super helpful to document all your thoughts so you can return back to any step and see what you decided earlier and whether all of your decisions are consistent.

With regards to the relevant experience and knowledge:

Have experience in the development of the huge mobile applications or better the whole client-server systems. You are expected to show the knowledge of the application to server communications design (one-way, two-ways, API design), pagination, cache and state management, modularisation techniques, different mobile application design patterns and their pros and cons. In addition here you can demonstrate knowledge of the most popular problems in mobile apps and methods of avoiding them (like state mutation, storing of the big images in the app state and overuse of singletons and god objects).
Ability to foresee the problems or difficulties in the solution. Some solutions can be very hard to implement and you are expected to clearly communicate about it, explain where the possible problem or the trade-off is.
Have experience working with the different quality assurance and risk mitigation techniques. Here you are expected to mention how you make your design good for testing, blue-green deployments, fail fast, etc.
Ability to design the modules on the class level, awareness of the main principles and design patterns. Very unlikely you will have time to do this but still there is a chance the interviewer will ask you questions like “How would you implement updating of the unread messages count label?” — you will need to decide will it be observer pattern or delegate or polling or whatever else it could be and draw the simple diagram.

Common recommendations

Summarising the most popular recommendations companies provide before the interview:

You need to gather as many requirements as possible from your interviewer before starting designing. Don’t hesitate to ask as many questions as needed.
You should show your knowledge so it’s possible to start from the part of design you know the best.
You shouldn’t be silent, speak all the time, communicate clearly, don’t wait for hints from the interviewer.
Always provide information about the alternatives and defend your choice.

Proposed interview answer plan

I believe that if in your company you have a good tradition to do a design review before implementation of any big (or cross team) feature you can notice that actually a system design interview looks like a complex design review along with the preliminary work when you gather the requirements, think about the design and draw diagrams.

So you can conduct design review for 3–5 times and it should be enough to understand the idea. But if you don’t have such an option let’s explore the good plan of your system design interview.

This could be the high level plan:

Business idea (what we are selling to user?, usually it’s the services and the data) : 3–4 minutes
Requirements clarification (5 dimensions to help with limitations) : 1–2 minutes
Designing of a mathematical model (if needed) : 0–5 minutes
Define how to split that state and functionality between the server and the client sides : 3–5 minutes
API design : 7–10 minutes
High level client side design (layered structure) : 4–5 minutes
Detailed design of the one chosen module : 4–8 minutes
One complex/tricky case and detailed discussion : 5–10 minutes

Let’s explore each item from the below plan in more detail.

Business idea

This is mainly to understand the context, to formulate what exactly we are going to sell users (what kind of services and data) and to start thinking about the task from the very top.

The outcome of this step is as the following:

The list of services we sell to our users
The data entities access we sell to our users
Understanding the main idea of the product or the feature (if you don’t understand something, don’t hesitate to ask all the needed questions. The more you know about the product the easier will be making decisions during the the rest of the process)

For instance, if you were asked to design the simplified WhatsApp-like application without registering/login, groups, broadcast lists, E2E encryption and reading the status then the outcome of this step would be something like this:
We sell the following services:

- Safely storing and providing read/write access to your address book
- Providing the ability to make/receive Audio and Video calls to contacts from your address book
- Providing the ability to send/receive text messages and images to contacts from your address book and safely store the history of all the chats

Services should be as independent from each other as possible so it will help you to design API faster. In our example all services depend on address book service but they are independent from each other.

We sell the access to the following data:

Address book item:
- First name
- Family name
- Phone number
Chat history message:
- User opponent
- Inbound or outbound
- Date and time of the message
- Message text or Image

So the above services list and the data entities (+ UI sketches commonly provided by the interviewer) form the basis for all your future work. The data entities are not the final ones and are not prepared to be stored or transferred by network, it’s just a high level description of the data mostly from the end-user perspective.

You can start clarifing requirements right here, for instance you can ask whether we need to add user avatar image but in this case you should be ready to answer this question by your self since the interviewer can ask you back the same question and you should reason your choice.

Requirements clarification

It’s very important to gather all the possible requirements beforehand. We started doing that with the business requirements on the previous step, now we need to clarify the technical requirements, the limitations and the context. Why do we need it? Because your design should depend on the context, development processes and actually the ask from the customer/PO/interviewer.

So the first question should be about the context, whether we have any in our task, whether we have the host application we’re adding new functionality to, whether we have backend already done for the web version, how big our team is and what their expertise is. In 99% of the cases I believe the interviewer will answer that there is no context at all and you should start from the scratch but these questions will show that you do care about the context and take it into account during the solution design process.

Concerning the limitations I prefer to use a checklist for evaluation of the design solutions. The following list contains the checklist items and questions that can be asked regarding each of the item related to the mobile system design. You can come up with different questions and use another list, this one just for reminding you of the questions during the interview.

Maintainability — are we designing MVP or PoC or full scale system? How big is the team which will implement my design?
Testability — we can leave this dimension, we definitely need to think about some type of testing. No limitations here, just not to forget to mention about testing on the latest steps of you design.

Scalability/Performance — here we can treat scalability as the scalability of our team — ability to reuse their code in different platforms and though we can ask about cross platform option since it can impact our design considerably. Also you can ask questions about some parameters that could impact the performance of the applications where performance is crucial and where it’s better to define limitations beforehand. For instance, if you are designing an application with recognition of many objects in the live video using ML you could mention that we limit our selves to the latest models of Android or iOS devices only. Otherwise we need to use third party libraries and use main processor for making all the calculation that can affect the battery life and the quality of recognition. This will not impact design so seriously but it’s worth mentioning since it shows your experience and an ability to foresee the problems.

Security — it’s very important to discuss the limitations caused by security reasons. Especially if you’re asked to store some information on the device, or if you should work with the Personally Identifiable Information (PII) or sensitive PII. So it’s better to clarify with the interviewer that persisting and transferring requested information should be safe and if you think it’s not safe you should suggest what have to be done to protect it.

Availability — here we can ask about the ability to work offline, probably limitations of OS versions, phone/tablet choice, using limited set of languages and screen sizes. Of course all those questions should be asked only if it’s relevant to your task.

There is no need to enlist all the dimensions on the whiteboard as they work only as the keys to recall what should you think and ask about. Later on you can use the same dimensions if you will be asked to compare two solutions.

It’s better to write all the answers to the above questions on the whiteboard so you can reference them later.

Mathematical model design

This step is relevant only to a limited subset of the tasks. Those tasks would sound like following:

You are a Facebook developer and you have to add new feature to show the two most popular posts at the top of the feed. In this case you need to clarify/come up with what it means — “most popular” and then describe how you can calculate it: should you use views or comments or likes in order to figure it out, should it be instant popularity or moving average and so on.
You are a developer in a rapidly growing super hotel aggregator and your task is to develop a bot detection feature. Here you should define the input data you need to detect bots and come up with the algorithm and mathematical model of how to use this data in order to distinguish bots from the real users. For instance, you can calculate the requests rate using moving window and compare it to a given limit.

Outcome of this step would be a simple schema of the module with inputs, formula(s) inside and outputs.

High level system design

Next step is to decide what part of the services and the data should be implemented/stored on the server and what part should be implemented/stored on the mobile device. Here you can use the output of the first step — list of services and data entities.

Let’s consider previous example — WhatsApp-like app.

Can we implement address book storage on device only? No, because we need to have one shared store so all the clients can use it in order to be able to contact to the needed correspondent (We don’t consider distributed storage schemes but you can think about it and compare with the above schema using dimensions from the second step).

Can we implement Audio calls using client side only? Yes, but you need to let the interviewer know about all the tradeoffs in that case in comparison to the server side service (like E2E encryption, better scalability, worse availability because of NAT traversal and so on).

For instance, you decide to put all the services to the server side, and so all the data will reside on the server as well.

Then since all the data and services reside on the backend we need to choose the client-server communication type for each of them one by one.

For WhatsApp-like application example, address book service can be easily implemented as a simple REST HTTP calls with JSON payload (given that each user can use only one device and so we don’t need server to client notifications about changes in the address book contacts), Audio/Video calls service would use its own communication interfaces — bidirectional TCP-based connection for a signal protocol (WebSocket probably) and UDP-based for the streaming of the data (RTP probably). On this step it should be enough to just mention type of communication (bidirectional or unidirectional, streaming or request-response-based, then more details — HTTP REST (simple, long-polling), web-socket, raw-socket, etc…)

For the text/images chats service it is a bit harder to decide what communication channel to use. It could be REST HTTP + push notification or bidirectional web-sockets or custom TCP based channel or something else. All the options have their own pros and cons so you can choose any option, just let the interviewer know about the other alternatives and be ready to defend the chosen one.

The outcome of this step should be something like on the following scheme where you define the place where the services and the data will reside + communication channels between the server and clients.

This is the first very high level structure of our system and most likely it will be the only scheme where you draw any backend details.

API design

This topic is rather controversial, there are a lot of ways to design the API, so you can use any of them, the only thing is to define all the exact data entities and API calls as an outcome of this step. Side effect of API design is the clarification of some server-side solutions.

From my experience it’s good to start from the service and the data entity that other services/data entities depend on and expand it to one or more API calls. For instance, our first service is address book, so there would be 4 API calls:

GET /address-book?page,limit -> [AddressBookItem]. Read the address book data with paging (it’s very important not to forget about paging — better if you consider paging by default to any call and then remove it if it’s not needed than to forget about it at all). I intentionally don’t mention any user identifier as we need to mention in the interview that all the requests should contain header with authorisation token which was received earlier during login procedure. If the interviewer wants to know more about login you can let him know one or more ways of the register/login implementation.
POST /address-book?user_id,first_name,family_name,phone -> AddressBookItem. Add new record to the address book.
PUT /address-book?usert_id,first_name,family_name,phone -> AddressBookItem. Update existed record in the address book.
DELETE /address-book?user_id. Delete existed record from the address book.

Where AddressBookItem looks like:

AddressBookItem:
    user_id: int64
    first_name: string
    family_name: string
    phone: string

Read chat message API call would look like:
GET /chat-history?user_id,page,limit -> [ChatMessage, ChatImageMessage]

Where ChatImageMessage looks like:

ChatImageMessage:
    user_id: int64
    msg_id: int64
    dt: timestamp
    is_inbound: bool
    img_url_thumb: string
    img_url_full_size: string

I wrote API calls in a custom form that is easy to show in one line, so question mark doesn’t mean we will use request path parameters, it’s just delimiter between the call name and parameters list. Moreover word “GET” doesn’t imply we will use HTTP requests, it could be WebSocket according to our high-level scheme and this API call can be encoded with Google Protobuf, for instance, but idea will remain the same — it should be an idempotent call with three input parameters that returns array of 2 types of chat messages (simple text message and image message).

You can come up with your own format of writing API calls if you think it would better represent the call name, input and output data.

At this step we define data with more details taking into account server side implementation and trying to foresee future problems. For instance, we added unique identifiers to users and messages and represent image as two direct links — one for a thumbnail and one for a full size image.

Most likely in your task you will have to work with images so it’s better to think about the possible problems beforehand, like saving a lot of traffic for users by downloading only thumbnails and not the full size images. It sounds obvious but those small optimisations make your solution more interesting to design and discuss.

Designing an API is an iterative process, you can change API later when you will be designing client-side and it’s OK. This step is rather important and hard since you need to think simultaneously about server-side design and client-side design trying to foresee future problems with the connection speed, throughput and usage convenience. The previous steps are intended to prepare you by providing with the services list, data entities and the type of communication channel between the client and the server.

After finishing this step you can spend some time analysing the result — probably there are some obvious optimisations you can apply to the API, for example, you may want to merge some APIs to one, or split some of them, or even decide to change communication channel because, for instance, it turns out that bidirectional communication better suits for one of the cases.

High level client side design

Finally, you can start drawing the client modules structure scheme. Layered design, commonly, the easiest way to organise modules in some meaningful groups in almost all Frontend applications.

It’s better to start drawing this scheme from the central and most important part of it — the state and services that doesn’t related to sate. These are things you provide to your users, so let’s start designing from them. You can name this layer business entities or Business Layer and then start going downwards in order to meet the server-side.

The next layer could be Service Layer, here I use name “service” in a different meaning than before— interval Services Layer helps to isolate concrete implementations of the low level communication and persistent modules (Data Layer) from the Business Layer. So Business Layer shouldn’t bother about the place where we get the data from — persistent storage or communication channels, and what kind of SDK we use to implement audio/video calls. Another purpose of the Service Layer is to convert data from multiple formats to the one Business Layer is using. At this step we don’t make any assumptions about implementation of the services — will it be the set of middleware in Redux or services distributed among VIPER modules or something else, we can just mention that our purpose is to make services stateless and to try to keep explicit state in the Business Layer only.

Data Layer contains SDKs, frameworks, libraries that provide your application with such features like network communication, local DB access, logging, or, complete fully functional features like Audio/Video calls. Of course you can split Data Layer to multiple internal layers if it makes sense, for instance if you develop some low level codecs you would have two layers in the Data Layer — one for high level orchestration and one for low level access to device hardware, but it should be hidden in the Data Layer and exposed only in Dependency Injection phase where we can define what kind of modules and codec should be used in Data Layer.

Going upwards from business layer to meet user we should draw the Presentation Layer. The main purpose of this layer is to prepare the data from Business Layer to be displayed to user in UI Layer. Again it worth to mention that we want to keep the Presentation Layer stateless.

The last layer to draw on the high level design structure scheme is UI Layer. It’s a very platform specific layer where the business data, converted by the Presentation Layer, is being displayed on the device screen using OS SDKs. The second purpose of this layer is to interact with the user and pass all those interactions throughout the Presentation Layer to the Business Layer and further if needed.

This layered design can be mapped to any concrete architecture patterns like MVC, MVI, Redux, VIPER, RIBs, etc, but it’s very important to remain on the higher level of abstraction at this step and not to go deeper to those concrete patterns. You can mention such thing like “module” here but postpone the concrete definition of the “module” to the next step as module’s meaning depends on the architecture pattern you will choose later, for instance, module in VIPER lays horizontally and crosses all the layers except the Data Layer, so each module contain small piece if each layer. In opposite, Redux, usually, doesn’t have modules but more like features instead.

Detailed design of some modules

Here we should finally choose one feature/module and define the architecture pattern that suits it best in your opinion. Let’s consider the chat feature in a WhatsApp-like application. As the chat feature crosses all the layers, the chosen pattern will influence the whole application, so we need to think not only about chosen feature but rather about the app in a whole.

Actually, our WhatsApp-like application works with rather limited types of data entities that are related to each other and all changes in the data should be immediately reflected in different part of the application as messages in the chat list and the chat screen, as a notification when user browses the other parts of the app and as a counter of the unread messages. It means we can have the one periodically updated state with the ability to subscribe to those state’s changes. Looks like Redux is a good option for the Business Layer + middleware as the Service Layer + MVP or MVVM or MVI patterns for the Presentation and UI layers with the Redux store playing the Model role. VIPER seems a bit excessive for just a few screens with minimum number of routes between them and RIBs is better suits very complex screens with many teams working on the same screen. Of course, you can choose any pattern or mention none of them, but then you would have to to explain where store/state should be placed, how it should work, how we would implement subscription for state changes, how services should be used and how they will update the state.

I intentionally didn’t mention any concrete platform as it actually doesn’t matter if it’s Android or iOS as there are multiple implementations of Redux store and reactive frameworks for implementation of the MVVM/MVI for both platforms whereas MVP doesn’t need any.

Let’s draw a possible structure scheme for the chat feature:

Here it’s important to describe how all the cycle of the data update would work, demonstrate the advantages of the pattern you have chosen and mention some tradeoffs and hard problems you would encounter during the implementation of this solution.

For instance, we can enlist the following advantages of the chat feature’s Redux design using the previous dimensions:

Maintainability: We fully decouple data receiving part (Data and Service Layer) and data consumer part (Presentation and UI Layer) using store with updates subscriptions, so it doesn’t matter from which source the data comes — from a persistent store or a network communication channel. It helps to change some parts of the app without impacting the others + reuse as much code as possible.
Scalability: The store could have multiple subscribers out of the box, so Presenter of our chat feature will be triggered and will pass the latest changes to View in order to display them + Presenter of the top panel, for instance, will be triggered as well and will update the counter of the unread messages. Adding one more subscriber is easy.
Testability: State is immutable and located in one place (not distributed across the app) which help to avoid many mistakes related to simultaneous changes of the state. Also it helps to test the app as you are sure that state can be changed only in reducers.

Disadvantages:

Deployability (and partly Maintainability): complex project structure and the tricky modularisation concept. In order to change something in the chat feature developers will have to make changes in many places — state, reducers, actions, middleware, Presenter, View. Moreover, all those places are spreader among the project (we can come up with a project structure that minimise this effect by uniting the most of the parts in one place but it could lead with the repeating code and problems with changing the other teams code) which makes modularisation a bit tricky. So every team will be most likely touching the store/state and it means rebuilding of all the modules as all the modules depend on the state.
Scalability: the bigger become the application the bigger become the state. Usually we split it into sub-states so each feature has it’s own sub-state but anyway huge state could lead to performance and multithreaded access issues.

It’s worth mentioning that your pattern/approach/design is good for testing, for instance, in our example we can mention that we should test middleware, services, presenters and reducers and it’s very easy since they are stateless.

Additionally, here you can demonstrate the skill of class diagrams designing. Specific implementation will depend on the platform (because, for instance, table rendering concepts are different in Android and iOS).

One complex case and detailed discussion

At this step you can show the knowledge of the platform and problem solving skill. It’s better to choose one interesting case that you met before and successfully resolved and softly lead the conversation to this case.
For instance, you can say that it’s interesting problem of caching the images in the chats. Each image has direct link to it’s thumbnail and full size image and once image comes to the visible area of the screen it will be requested from, let’s say, an image download service which downloads image using those direct links. Weak point of such a solution is that if user has slow network connection images will appear too late, especially if user scrolls chat rather fast. Obviously we need a cache. The easiest solution is to add to the image download service a memory cache which represents the dictionary with keys as direct links and values as images but what will happen when we restart the app? All the text messages will be restored from the local DB (assume it) but should we save thumbnails and full size images in DB as well? Or should we save only thumbnails in DB and full size images in the file system? If we will save images in the filesystem how can we find them later on? How can we housekeep folder with images if for instance device, in case of iOS, asks to free some space up? There are a lot of questions and answering them could demonstrate your experience, knowledge of the platform and problem solving skills.

So the trick here is to choose one common hard enough problem beforehand and lead the conversation to this problem. The interviewer can reject that and choose the random one, but still you will show that you aware of some problems and you would have a chance to talk a lot about the problem you know the best.

Summary

Summarizing the above information I can say that it’s always better to elaborate a plan of the common system design interview beforehand, should it be like one I suggested in this article or another one. It doesn’t matter which plan you will chose but it should clearly demonstrate your ability to tackle a very big ambiguous problem — split it to some meaningful pieces, absorb them one by one, process that information and produce another type of relevant information as an outcome.

Good luck at your interviews and don’t hesitate to write comments to this article or share your experience.

References

Layered architecture pattern: https://towardsdatascience.com/software-architecture-patterns-98043af8028
Architecture/systems quality attributes/dimensions: https://sites.google.com/site/misresearch000/home/software-architecture-quality-attributes
There are a lot of books about API Design, it does worth to read at least one
Mobile Architecture patterns: MVC, MVP, MVVM, MVI, VIPER, REDUX, etc. It would be good to know about them and about the conceptions behind each of them even if you don’t want to mention any of them on the interview. It’s easy to find a lot of info about each of them.