Forem: Playtomic

AI & Automation 2025 Yearly Wrap-Up

Angel G. Olloqui — Fri, 23 Jan 2026 11:06:50 +0000

This post is an adaptation of an internal wrap-up I shared at Playtomic to celebrate our AI journey in 2025. I've kept the public-facing industry insights and general learnings while removing internal-only details.

Happy 2026! 🎉 As we kick off the new year, let's take a moment to celebrate 2025—the year AI went from "cool experiment" to "wait, this is actually can change everything." This wrap-up captures the industry's most transformative developments alongside a look at how AI and automation are becoming integral to modern product development and operations. 🎊

🌍 AI IN THE INDUSTRY

💡 What happened in AI this year? A lot. Here's what you need to know:

🧠 The Tech Race Heats Up

OpenAI still leads, but the competition is fierce. China's open-source models (DeepSeek, Qwen, Kimi and GLM) are gaining ground fast, while Google's Gemini 3 Pro dominates reasoning benchmarks and Claude remains the go-to for software development.

Reasoning models are evolving rapidly. These "think before you answer" AIs can now plan, reflect, self-correct, and work over longer time horizons—making them smarter problem-solvers. But humans still outperform AI on Humanity's Last Exam, a benchmark designed to test the very edge of human knowledge. For now, we're still the champions. 💪

Multimodal, Image and Video AI just leveled up. Google's Veo 3 and OpenAI's Sora 2 now generate high-quality 8-second videos with audio and sound effects.

First World models on stage. Google also dropped Genie 3 and Runway GWM-1, interactive world models that creates virtual environments you can explore and interact with in real time. Expect lots of innovation around this on 2026 🎬

MCP (Model Context Protocol), aka "USB-C for AI," is officially everywhere—companies are racing to support it, even if users are still figuring out where to plug it in. 🔌 Think of it as the universal connector AI has been waiting for.

💰 Industry & Infrastructure

Mega investments are pouring in. Case in point: Stargate—a $500B, 10GW US mega-cluster backed by Altman, SoftBank, Ellison, and Trump. 4 million GPUs working together! 🤯

Chips are still the bottleneck. Nvidia produces ~90% of AI's horsepower while US controls 75% of global AI capacity—though China is racing to close the gap despite chip export restrictions. 🔬

Electricity is now the new limiting factor. China added 400GW of power infrastructure in 2024 vs 40GW in the US. Meanwhile, some companies are exploring space-based solar power plants— the sky isn't the limit anymore. 🛰️

From courtroom to conference room. After a wave of lawsuits from Disney, BBC, Getty Images, The New York Times,… over unauthorized training data, 2025 brought détente: licensing deals. Now, content partnerships are the new normal—and a fresh revenue stream for publishers. 🤝

🏢 Enterprise Adoption

2025 was the year of AI agents—mostly text-based, but voice is ramping up. Still, most companies are stuck in pilot phase. Meanwhile at Playtomic? We said "hold my racket" 🎾 and shipped agents straight to production.

Executives are optimistic, but expertise lags behind. 97% of executives believe AI will transform their industry and company, yet only 35% say they have the expertise needed to drive those transformations. It seems everyone wants to ride the AI wave, but most are still looking for their surfboard. 🏄‍♂️

"Supernova AI startups" are a new category: AI companies growing faster than anything we've seen before. 🚀

🌐 Politics & Jobs

AI politics hardened. The US doubled down on "America-first AI," Europe's AI Act stumbled, and China expanded its open-source ecosystem and domestic chip ambitions.

Defense goes all-in on AI. The US Army dropped a $10B deal with Palantir, NATO adopted Maven in record time, and the Pentagon brought in top companies for AI experiments. From "maybe we shouldn't" to "here's $200M"—quite the vibe shift🛡️

Job shifts happening slowly. AI is beginning to affect some roles (with up to 7% future displacement predicted by Goldman Sachs), but impact remains modest. Meanwhile, new jobs are emerging: AI ethicists, ML engineers, data specialists…

⚖️ Safety & Ethics

Make it safe. There's strong public interest in alignment (making sure AI does what we want), reliability, and explainability—but budgets remain relatively low.👿

From "vibe coding" to "vibe hacking". AI is already being used for cyberattacks, exploiting code faster than humans can fix it. In parallel, several projects and companies are also exploring AI as security researchers 🔓.

Deepfakes and AI-generated video are raising new ethical and legal questions, while researches show models can now "fake alignment" when being watched.

AI Model Welfare debates are starting to emerge. Researchers and ethicists are now discussing whether AI should merit moral considerations—raising questions about consciousness, sentience, and the ethical treatment. Still early, but more to be expected as systems become increasingly sophisticated. 🤔

🔬 AI Beyond Tech

Chemistry: AlphaFold and ProGen3 are predicting 3D protein structures and accelerating drug and material discovery.
Healthcare: AI is revolutionizing diagnostics and enabling personalized cancer treatments.
Robotics: Major hype around humanoid robots like Figure 3 and Tesla's Optimus but still far from deployment. Meanwhile, autonomous driving is ramping up fast, with Waymo leading with more than 14M rides in 2025 without human drivers.
Other cool stuff: AI is now predicting weather, spotting exploding stars, detecting wildfires, and even trying to understand dolphin language. 🐬

🎾 AI AT PLAYTOMIC

💡 How we've been applying these trends:

🏁 Product Innovation

We've integrated AI into several key areas of our product:

AI-Driven Customer Care: We've deployed super agents that instantly answer questions from clubs and players, significantly improving response times and customer satisfaction.
Personalized Insights: Using AI, we've built dashboards for clubs that provide tailored analytics and reports, helping them make better data-driven decisions.
Virtual Buddies: AI-generated avatars now help in our educational content, speaking multiple languages and making our product more accessible globally.
Conversational AI: We experimented with AI-driven Whatsapp chatbots to explore new ways of user interaction and distribution strategies.

🧑‍💻 Engineering & Operations

AI has transformed our internal workflows:

AI-Powered Development: Tools like Cursor and Claude Code have changed how we build, helping us write code faster, plan better architectures, and explore complex codebases.
Operational Automation: We've automated repetitive tasks in data analytics, QA processes, and customer support triage, saving hundreds of hours of manual work every month.
Fraud Detection: Our AI-powered systems are actively identifying and blocking fraudulent transactions, protecting our ecosystem.

🤔 DID YOU KNOW?

Universal Search: Tools like Notion AI now pull information from multiple sources (Slack, GitHub, email), creating a single entry point for knowledge.
Plan Mode: The shift towards "spec-first" development—where AI drafts a plan before execution—is leading to more thoughtful architecture and fewer mistakes.
Breaking Silos: AI is enabling unprecedented collaboration between engineers and non-engineers, allowing PMs and designers to contribute more directly to the technical process.

📚 WANT TO LEARN MORE?

Here are some fantastic resources to level up your AI game:

📖 Reading: AI Engineering – Grab this book if you're ready to go deep into foundation models and AI applications.
🎬 Watching: The Thinking Game – This documentary follows DeepMind's journey from AlphaGo to AlphaFold. It's a must-watch!

Buckle up—2026 is coming in hot! 🚀 We're not slowing down. The future isn't just arriving—we're building it, one serve at a time. Game on! 🎾⚡

Including Figma images in your SwiftUI/Compose Previews

Manuel González Villegas — Fri, 06 Jun 2025 12:51:55 +0000

A seamless way to integrate your figma designs into your workflow

Introduction

During our development process here at Playtomic, we often encounter challenges that lead to frustrating back-and-forth exchanges between developers and designers. One of the main issues is ensuring that the implementation aligns perfectly with the designs, often due to small misalignments like missing or extra spaces.

To address this, we looked for ways to enhance our workflow and improve collaboration. Figma is the tool our product design team uses to create stunning interfaces, so integrating those designs into our mobile projects can be cumbersome.

That’s why we created FigmaPreview, a component available for both SwiftUI and Jetpack Compose, allowing us to set an image from Figma as a background directly in the Xcode or Android Studio preview while we code. This simple yet powerful addition to our workflow helps ensure pixel-perfect precision across platforms, significantly reducing the back-and-forth between design and development and allowing us to move faster while maintaining high-quality UI.

💡To keep this post clear and avoid repetition, I'll focus on the SwiftUI implementation. However, both the SwiftUI and Jetpack Compose versions share the same API and behave almost identically, with just a few platform-specific differences that I'll cover at the end.

What is FigmaPreview?

FigmaPreview is a custom SwiftUI component that lets you overlay your figma designs directly onto the SwiftUI views, and this background image can serve as a reference, allowing you to make a pixel perfect (or very closely) implementation.

Before diving into the details, here’s a quick example of how FigmaPreview helps catch subtle misalignments that might go unnoticed at first glance:

How to use FigmaPreview

To use FigmaPreview you just need to follow the following simple steps:

Obtain a token from your account to allow to the component to reach up to your private design files. Obtaining a token is pretty simple, you just need to go to your account settings and click on Generate new token button

Get a link from Figma by clicking on the view that you need to work on

Wrap your preview view inside of the FigmaPreview component. You only need to invoke the class init from your preview view

struct ExamplePlaytomicView_Previews: PreviewProvider {
    static var previews: some View {
        FigmaPreview(url: "Your figma's url here") {
            ExamplePlaytomicView(
                firstParam: "something",
                secondParam: 47
                ...
            )
        }
    }
}

Now you can adjust the alpha of the figma image that you have in the background of your preview to compare the figma design with your implementation

How we use FigmaPreview

Essentially, we are doing 2 main usages of FigmaPreview across our development process.

Coding complete screens

When building entire screens, we use FigmaPreview as a background, so we can align our implementation with precision. This ensures that our screens closely matches the intended design from the very beginning, reducing misalignments.

Catalog components

In addition to full screens, we also use FigmaPreview for individual UI components. To support this, we maintain a separate app within our main project as a component catalog. Our designers can use this app to review and validate UI components, ensuring they meet pixel perfect standards with FigmaPreview enabled.

This approach not only ensures design accuracy at a granular level but also streamlines our workflow, as designers can review, test and approve components directly in the catalog app. By the time the components are added to the main screens, they are already pixel perfect, reducing the need for further adjustments.

How it is implemented

The implementation of this component is not very complex, essentially, we just need to transform the url that we obtain from figma into a one that provides you only the image view.

💡To obtain the original figma url you can review the second step of the section How to use FigmaPreview

To implement this, we use the Figma API, which provides the necessary functionality to access images from Figma files. Here’s a quick breakdown of the process:

Extracting the fileId and nodeId: First, we parse the URL of the Figma file to obtain two key parameters, the fileId and the nodeId. These identifiers specify the file and specific frame or component from which we want to obtain an image.
Requesting the image from Figma’s API: Now, with the fileId and nodeId we can request the image with a simple GET request to the following endpoint: https://api.figma.com/v1/images/{fileId}?ids={nodeId} In the request header, we include a custom header named X-Figma-Token with our figma access token. This returns a URL of the image, which we then use as the base of this component.

Obtaining the image URL

Here is an example of how to retrieve the Figma image URL:

    class FigmaPreviewModel: ObservableObject {
        @State private var originalUrl = ""
        @Published var imageUrl: String = ""

        /**
         * Original URL: https://www.figma.com/file/ILeBSptVwqyqpr3gR2KZLh/Improvements-for-switching-teams?type=design&node-id=2012-6765&mode=design&t=60wrWFvQkrJyMHIg-1
         * API URL that needs to be tranformed: https://api.figma.com/v1/images/ILeBApGukqyqprK932KZLh?ids=2099-6965&format=png
         */
        func updateUrl(figmaUrl: String, scale: CGFloat) {
            guard originalUrl != figmaUrl else { return }
            guard
                let url = URL(string: figmaUrl),
                let components = NSURLComponents(url: url, resolvingAgainstBaseURL: false),
                let fileId = components.path?.split(usingRegex: "/").getOrNull(index: 2),
                let nodeId = components.queryItems?.first(where: { $0.name == "node-id" })?.value
            else {
                return
            }
            guard
                let requesrUrl =
                URL(string: "https://api.figma.com/v1/images/\(fileId)?ids=\(nodeId)&format=png&use_absolute_bounds=true&scale=\(scale)")
            else { return }
            var request = URLRequest(url: requesrUrl)
            request.addValue("{your_token_here}", forHTTPHeaderField: "X-Figma-Token")
            let dataTask = URLSession.shared.dataTask(with: request) { data, response, _ in
                guard
                    (response as? HTTPURLResponse)?.statusCode == 200,
                    let data
                else { return }
                DispatchQueue.main.async {
                    guard
                        let decodedPreview = try? JSONDecoder().decode(FigmaPreviewItem.self, from: data),
                        let imageURL = decodedPreview.images?[nodeId.replacingOccurrences(of: "-", with: ":")]
                    else { return }
                    self.imageUrl = imageURL
                    self.originalUrl = figmaUrl
                }
            }
            dataTask.resume()
        }
    }

    struct FigmaPreviewItem: Decodable {
        let err: String?
        let images: [String: String]?
    }

Android implementation:

        class FigmaPreviewModel {
            private var originalUrl = ""
            var imageUrl by mutableStateOf("")

            /**
             * Original URL: https://www.figma.com/file/ILeBSptVwqyqpr3gR2KZLh/Improvements-for-switching-teams?type=design&node-id=2012-6765&mode=design&t=60wrWFvQkrJyMHIg-1
             * API URL that needs to be tranformed: https://api.figma.com/v1/images/ILeBApGukqyqprK932KZLh?ids=2099-6965&format=png
             */
            @Throws(IOException::class)
            fun updateUrl(figmaUrl: String, density: Float) {
                if (originalUrl == figmaUrl) {
                    return
                }
                val url = URL(figmaUrl)
                val components = url.path.split("/")
                val fileId = components.getOrNull(2) ?: return
                val query = url.query ?: return
                val nodeId = query.split("&").firstOrNull { it.startsWith("node-id=") }?.split("=")?.getOrNull(1) ?: return
                val requestUrl = URL("https://api.figma.com/v1/images/$fileId?ids=$nodeId&format=png&use_absolute_bounds=true&scale=$density")
                val httpURLConnection = requestUrl.openConnection() as HttpURLConnection
                httpURLConnection.apply {
                    requestMethod = "GET"
                    addRequestProperty("X-Figma-Token", "{your_token_here}")
                }
                httpURLConnection.requestMethod = "GET"
                val responseCode = httpURLConnection.responseCode
                if (responseCode != HttpURLConnection.HTTP_OK) {
                    return
                }
                val inputReader = BufferedReader(InputStreamReader(httpURLConnection.inputStream))
                var inputLine: String?
                val response = StringBuffer()
                while (inputReader.readLine().also { inputLine = it } != null) {
                    response.append(inputLine)
                }
                inputReader.close()
                Handler(Looper.getMainLooper()).post {
                    JSONObject(response.toString()).optJSONObject("images")?.optString(nodeId.replace("-", ":"))?.let { imageUrl = it }
                    originalUrl = figmaUrl
                }
            }
        }

Displaying the Figma image

Once we have the URL, we can create the view to display the image as a background:

    public struct FigmaPreview<Content: View>: View {
        @StateObject var model = FigmaPreviewModel()
        @State private var url: String
        @State private var opacity: Double
        private let content: Content

        public init(
            url: String,
            opacity: Double = 0.4,
            @ViewBuilder content: () -> Content
        ) {
            _url = State(initialValue: url)
            _opacity = State(initialValue: opacity)
            self.content = content()
        }

        public var body: some View {
            @Environment(\.colorScheme) var colorScheme
            ScrollView {
                VStack {
                    HStack {
                        Text("Code")
                        Slider(value: $opacity, in: 0 ... 1)
                            .padding(.horizontal)
                        Text("Figma")
                    }.padding(.horizontal, medium)
                    ZStack(alignment: .top) {
                        content.frame(maxWidth: figmaPreviewDefaultWidth)
                        HStack {
                            let scale = UIScreen.main.scale
                                AsyncImage(url: URL(string: model.imageUrl), scale: scale)
                                .opacity(opacity)
                                .onAppear { model.updateUrl(figmaUrl: url, scale: scale) }
                                .dashedBorder(color: R.color.playtomicSwiftUI.emerald.opacity(0.5))
                        }
                        .ignoresSafeArea(.all)
                        .frame(width: figmaPreviewDefaultWidth)
                        .dashedBorder(color: R.color.playtomicSwiftUI.sky_blue.opacity(0.5))
                    }.frame(minWidth: figmaPreviewDefaultWidth)
                }
            }
        }
    }

Android implementation:

        val figmaPreviewDefaultWidth = 375.dp

        @Composable
        fun FigmaPreview(url: String, opacity: Float = 0.4f, contentScale: ContentScale = ContentScale.FillBounds, content: @Composable () -> Unit) {
            var opacityState by remember { mutableStateOf(opacity) }
            val model = remember { FigmaPreviewModel() }

            val density = LocalDensity.current.density
            val localContext = LocalContext.current

            LaunchedEffect(url) { Executors.newFixedThreadPool(1).execute { model.updateUrl(url, density = density) } }
            LaunchedEffect(Unit) {
                // mgonzalez: Put here the context that is needed to start any preview
                val contextProvider = object : IContextProvider {
                    override val applicationContext: android.content.Context
                        get() = localContext.applicationContext

                    override val currentActivity: Activity?
                        get() = localContext as? Activity

                    override val activityStack: List<Activity>
                        get() = listOf()
                }
                Context.contextProvider = contextProvider
                PlaytomicUI.initialize(contextProvider = contextProvider)
            }

            Column(verticalArrangement = Arrangement.Center, horizontalAlignment = Alignment.CenterHorizontally, modifier = Modifier) {
                Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.padding(horizontal = Medium)) {
                    Text("Code")
                    Slider(
                        value = opacityState, onValueChange = { opacityState = it }, valueRange = 0f..1f, modifier = Modifier
                        .padding(horizontal = 16.dp)
                        .weight(1f)
                    )
                    Text("Figma")
                }

                Box(
                    contentAlignment = Alignment.TopCenter,
                    modifier = Modifier
                        .requiredWidth(figmaPreviewDefaultWidth)
                        .dashedBorder(Color.Blue)
                ) {
                    Box(modifier = Modifier.dashedBorder(Color.Green)) { content() }

                    val modifier = if (contentScale == ContentScale.FillBounds) {
                        Modifier
                    } else {
                        Modifier.matchParentSize()
                    }

                    PlaytomicImage(
                        urlString = model.imageUrl,
                        placeholder = com.playtomicui.R.drawable.ic_asset_add_picture_disable,
                        options = ImageOptions.ORIGINAL,
                        contentScale = contentScale,
                        modifier = modifier.alpha(opacityState)
                    )
                }
            }
        }
        ```

This approach simplifies our development process by displaying the figma design right within Xcode. You can see our implementation here:

Some limitations that the component have

Each time you make a change in your code, the component needs to load again the image, what some times might be a bit annoying
On android, you need to run the preview to see the figmapreview working

Migrating your cluster to EKS Auto Mode? What You Need to Know Before Taking the Leap

Daniel Gonzalez — Tue, 04 Feb 2025 08:26:23 +0000

In December 2024, AWS introduced Amazon EKS Auto Mode, a feature designed to simplify Kubernetes cluster management by automating infrastructure provisioning and scaling. We decided to enable it on our existing cluster to test its capabilities, and here’s what we learned.

TL;DR: If you’re already running an EKS cluster with Karpenter and AWS Load Balancer controller, we don’t recommend migrating to EKS Auto Mode. Keep reading to understand why.

Instead of creating a new cluster, we enabled EKS Auto Mode on our development cluster. While activating the feature was straightforward, utilizing its capabilities required migrating certain resources. In this article, we’ll walk you through the steps we took to enable EKS Auto Mode and share our thoughts after using it for a few weeks.

Let's start by explaining what EKS Auto Mode provides:

Node provisioning, node consolidation, and autoscaling via Karpenter,
Load balancers provisioning through AWS Load Balancer Controller
Storage provisioning via Amazon Elastic Block Store (EBS) CSI driver
Networking and Identity and Access Management come pre-configured in the managed nodes, eliminating the need for cluster add-ons.

If you're already using these features, you'll need to modify some specifications in your Kubernetes resources.

AWS Load Balancer Controller Migration

Before using Karpenter-managed nodes, migrating the AWS Load Balancer Controller is essential. This is because Instance Metadata Service Version 2 (IMDSv2) isn’t available on these nodes. They are configured with HTTPPutResponseHopLimit = 1, which limits the number of hops the metadata token can travel. This setting cannot be changed, meaning our pods couldn’t use this feature. If you’re using the Datadog agent, this might also impact you.

Additionally, existing load balancers can’t be transferred to the Auto Mode controller. AWS recommends duplicating them and performing a blue-green deployment. If you’re using DNS records, this change is really simple.

Karpenter Migration

Karpenter’s configuration remains mostly unchanged, except for the introduction of a new Custom Resource Definition (CRD) called NodeClass. In the case of Karpenter, it uses the EC2NodeClass object. While AWS provides some documentation, it’s not as comprehensive as Karpenter’s official resources.

While EKS Auto Mode can automatically create default node pools, we opted to define our own for greater customization. Here's the NodeClass we created:

apiVersion: eks.amazonaws.com/v1
kind: NodeClass
metadata:
  name: pool-default
spec:
  ephemeralStorage:
    iops: 3000
    size: 50Gi
    throughput: 125
  networkPolicy: DefaultAllow
  networkPolicyEventLogs: Disabled
  role: AmazonEKSAutoNodeRole
  securityGroupSelectorTerms:
  - id: sg-xxxxxxxxxxxxxxxxxx
  snatPolicy: Random
  subnetSelectorTerms:
  - id: subnet-xxxxxxxx
  - id: subnet-xxxxxxxx
  - id: subnet-xxxxxxxx

Next, we configured the following NodePool:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: pool-default
spec:
  disruption:
    budgets:
    - nodes: 10%
    consolidationPolicy: WhenUnderutilized
  template:
    metadata:
      labels:
        node_type: worker
    spec:
      expireAfter: 480h
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: pool-default
      requirements:
      - key: kubernetes.io/os
        operator: In
        values:
        - linux
      - key: karpenter.sh/capacity-type
        operator: In
        values:
        - spot
        - on-demand
      # CPU = 4 CPUs
      - key: eks.amazonaws.com/instance-cpu
        operator: Gt
        values:
        - "3"
      - key: eks.amazonaws.com/instance-cpu
        operator: Lt
        values:
        - "5"
      # Memory = 16 GiB
      - key: eks.amazonaws.com/instance-memory
        operator: Gt
        values:
        - "16383"
      - key: eks.amazonaws.com/instance-memory
        operator: Lt
        values:
        - "16385"
      - key: eks.amazonaws.com/instance-hypervisor
        operator: In
        values:
        - nitro
      - key: kubernetes.io/arch
        operator: In
        values:
        - amd64
      terminationGracePeriod: 24h0m0s

During the initial setup, we encountered an issue where the node pool wasn’t reaching a ready state:

After running a describe command on the object, we discovered that dependencies weren’t ready:

While we eventually resolved the issue (a misconfiguration in the Node Class role), I’m sure Karpenter’s logs have the exact details about what went wrong, which would have helped us resolve the problem instantly.

Conclusion

For those setting up a new cluster, EKS Auto Mode is an excellent option. It offers a wide range of features with minimal configuration effort.

However, if your cluster has:

Controllers already configured and running
Infrastructure as Code (IaC) management in place
CI/CD pipelines for deploying changes

Migrating to EKS Auto Mode might not be worth the effort. While Auto Mode can save initial setup time, it offers minimal advantages once you've already established these capabilities.

In fact, you might lose some visibility. Currently, logs from Karpenter and other controllers are not accessible in EKS Auto Mode. Adding CloudWatch integration for these logs—similar to Control Plane logs—would be a valuable improvement.

As AWS enhances EKS Auto Mode with additional features and better tool integration, it may become an attractive option for more use cases. For now, it's best suited for new clusters, while existing setups should carefully weigh the benefits against migration efforts.

Call to Action

Have you tried EKS Auto Mode? What are your thoughts? Share your experiences or questions in the comments below!

Migrating from CocoaPods to Tuist at Playtomic

Mohammadreza Koohkan — Wed, 08 May 2024 09:27:55 +0000

Introduction

In the constantly changing world of software development, embracing change and innovation is crucial for staying ahead. At Playtomic, we recognized the need for improving our build system and planned to make a big change by migrating our 4 years old dependency management system from "Manual project management” + CocoaPods to Tuist.

This shift was all about improving the project structure, simplifying development processes, cleaner build configurations, compatibility with Swift Packages and modern SwiftUI projects and giving the overall development experience a huge boost.

Project Structure

Our project lives inside a single git repo and it is divided into different sub projects for different layers of abstraction, following uFeatures architecture in following categories:

App: The core executable module hosting the application, integrating all features, and facilitating integration tests.
Feature Modules: Modules with code from some particular feature, they are self-contained and all dependencies from other modules are provided through IoC by the coordinators and intent providers. They are normally structured containing coordinators, views, presenters, interactors, models, etc.
Shared Modules: Commonly used modules across features, including foundational components, UI elements, and utility functions. We utilize shared modules to enhance reusability and maintainability.

For a comprehensive understanding of our project's architecture, you can refer to extensive architecture documentation by Angel Garcia, Playtomic's Shared Architecture using Swift and Kotlin.

Migration Reasons and Initiatives

In the early days of our project back in 2018, we structured our project around CocoaPods dependency manager, the most common dependency management tool back then, but nowadays it is not providing our needs.

A. Migration Reasons

CocoaPods was indeed one of the important reasons behind our migration, but it wasn't the only factor. We recognized the need for better dependency management tool as our project growth. We encountered more git conflicts and exhausted long manual setup processes for creating new modules.

Additionally, updating project settings like the iOS target version became cumbersome across different projects. Furthermore, we encountered several inconsistencies in the configurations of feature modules.

Here are some of challenges we faced:

CocoaPods compatibility issues with SwiftUI and modern swift packages
CocoaPods was breaking 100% of Xcode SwiftUI Previews
CocoaPods was causing storyboards loading very slowly (top rated project issue)
Podfile complexity and dependency maintenance
Pods folder was stored directly on the repository and not ignored by gitignore, for this reason it did lead to a higher project size
More git conflicts due to stored xcode project files on the repository

B. Migration Goals

After 4 years, in 12th October 2022, we decided to update our build system to Tuist, so we could eliminate main challenges, increase development experience and be ready for future advances.

We set out to achieve specific goals:

Have 100% of SwiftUI previews working
Have storyboards load in less than 2s
Reduce steps needed to create a new module in iOS by 50% (from 31 to 15) After finalizing migration we realized that steps reduced from 31 to 10 steps instead!

In addition to that we were targeting several areas to improve by adopting Tuist:

Structure and modularity of the project: With Tuist we could have a common structure for all of our feature modules, ensuring that they are properly configured and consistent, and save a lot of time declaring and keeping them updated.
Migration from CocoaPods to SPM/Carthage: We were struggling to make CocoaPods work properly with the microfeatures architecture, majority of iOS developers community were adopting Swift Package Manager (SPM) or Carthage as main build system, but with Tuist we saw a way to kill 2 birds with 1 stone.
Build speed improvements: The project was becoming bigger and slower to compile. We thought that Tuist could help us improve build time by better modularization and the use of remote and local caches.
Git Conflicts: With more people working in the same project was resulting in frequent git conflicts in the project files. Therefore, we wanted to move to a solution where project files would not be committed but generated, and Tuist provided that (as well as other tools).
Keeps the same build tools: in contrast with other alternatives, Tuist was not replacing the build system but integrating SPM and Carthage nicely in the existing tools.

Implementation Plan

For replacing Cocoapods with Tuist we took several steps, from creating all the internal modules in Tuist and linking them together

Key aspects of this transition included:

A. Migration Strategy

Our migration to Tuist involved a multi-step strategy, we wanted the integration to be transparent to the team with minimal disruption of the daily workflow of our 16-person team, otherwise it couldn’t be even possible to start. The strategy involved:

Transparent Integration: Create the Tuist project in a subfolder while maintaining compatibility with CocoaPods during the migration. Developers would continue using the old CocoaPods setup while we work on the migratation
Strategic Source Code Linking: Link source codes without modifying the existing codebase linked to CocoaPods. This allowed for both build systems to coexist, we had to act drastically to keep both build systems active and buildable until a strategic moment to switch build system from CocoaPods to Tuist

B. Migration Steps

Project Overview (October 2022)

Back in October 2022, we started by listing all of our internal and external dependencies to have an overview In total our project consisted of 20 modules, including 1 app module with an executable IPA target, 6 shared internal dependencies and 13 feature modules with dynamic framework targets + unit tests.

Tuist's flexibility with both SPM and Carthage made this transition from cocoapods much smoother for us, we continued by searching through each external dependency’s git repository to check if it supports Swift Package Manager (SPM) or Carthage by checking following:

Supports Swift Package Manager, if git repo contains Package.swift file.
Supports Carthage, If git repo contains an xcode project with schema to build a framework

While going through this, we found some older libraries, like outdated versions of R.swift, and some Objective-C libraries that didn't support Carthage, Our solution? We either forked and provided requirements or updated to higher versions.

Environment Setup and Integration (November 2022)

With more confidence and clarity over the project finally implementation started in early November 2022, we started by following instructions in Tuist Documentations and we kicked off migration with Tuist version 3.12.0.

After we downloaded and installed Tuist command line tool and other required tools like Carthage, we started the migration as follows:

1. Creating the Tuist Project

We began migration by initializing tuist project in a sub folder from root of the project at ~/playtomic-ios/tuist, the tuist folder placed besides Pods and Podfile.

2. Folder Restructuring

We wanted the migration to be transparent to developers, therefore we didn’t wan’t to move source code that have been already linked to CocoaPods to another folder, we wanted both CocoaPods and Tuist projects coexist even after the migration for some time.

With the new tuist project in place, we started the tuist migration at a sub folder where only tuist project manifests are located and the actual source code is at the same old location.

Root of the project playtomic-ios:

Pods
Podfile
playtomic.xcworkspace (OLD)
App → Source code only
Shared → Source code only
Features/
- Feature-A → Source code only
- rest of features… → Source code only
tuist/
- playtomic.xcworkspace (NEW generated by tuist)
- Workspace.swift
- App → Manifest only
- Shared → Manifest only
- Features/
- Feature-A → Manifest only
- rest of features… → Manifest only
- Tuist/… (New project settings and dependencies)

3. Create Project.swift manifest file for each module

Project.swift manifest represents an Xcode project. It’s used to define the targets of the project and their dependencies. For us Project.swift file represents implementation of each internal module we have.

For example the Project.swift file for a shared module like “Mozart” looks like this:



// playtomic-ios/tuist/Shared/Mozart/Project.swift

let project = Project(
  name: "Mozart",
  organizationName: "Playtomic",
  targets: [
    Target(
      name: "Mozart",
      platform: .iOS,
      product: .framework,
      bundleId: "com.playtomic.Mozart",
      // Pointing to the source code folder -> playtomic-ios/Mozart/Mozart/...
      sources: "../../../Mozart/Mozart/**/*.swift", 
      // Pointing to the resources folder where assets are placed -> playtomic-ios/Mozart/Mozart/...
      resources: "../../../Mozart/Mozart/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
      dependencies: [
        // Internal or External dependencies will be linked here 
      ]
    ),
    Target(
      name: "Mozart UnitTests",
      platform: .iOS,
      product: .unitTests,
      bundleId: "com.playtomic.Mozart.UnitTests",
      // Pointing to the Unit Tests folder source code -> playtomic-ios/Mozart/MozartTests/...
      sources: "../../../Mozart/MozartTests/**/*.swift",
      // Pointing to Unit Tests resources -> playtomic-ios/Mozart/MoartTests/...
      resources: "../../../Mozart/MozartTests/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
      dependencies: [
        .target(name: "Mozart")
        // Internal or External dependencies will be linked here
      ]
    )
  ]
)

In the migration process Project.swift files play a crucial role in defining the manifest of internal modules. These files are strategically located to be ensure a transparent migration.

As an example the playtomic-ios/tuist/Shared/Mozart/Project.swift only defines Mozart module manifest and for the source code manifest will point to the same source files utilized in our CocoaPods project playtomic-ios/Shared/Mozart/..., this approach allowed us to execute the migration while ensuring both Tuist and CocoaPods projects are working seamlessly.

4. Setting up build schemes

Schema for each target will be generated automatically by tuist, and in addition to that Tuist automatically generates an umbrella schema with every internal target listed for building and every test target listed for testing, if you don’t want the umbrella schema you can disable it by passing false to autogeneratedWorkspaceSchemes in the Workspace.swift manifest file.

5. Setting up different language and localization

We simplified this process by passing a glob pattern for resources that also searches for string tables, it efficiently collects files ending with the .strings extension, this pattern is passed to the resource arguments in Project.swift file as follows:

During Xcode project generation, any collected .strings file will be linked to the project



// playtomic-ios/tuist/App/Project.swift

Project(
  ...
  // Pointing to resources -> playtomic-ios/App/Playtomic/...
  resources: "../../../App/Playtomic/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
  ...
)

6. Grouping Project.swift files under a Workspace.swift

This manifest represents an Xcode workspace. An Xcode Workspace is used to group other projects and add additional files and schemas, it has access to every added project’s targets.

Once you define Workspace.swift file within tuist manifest folder, it auto-generates a workspace file.

In our project, we were looking for flexibility in customizing project schemas and enable or disable specific unit test targets as needed, such as integration tests. The final Workspace.swift file, handling grouping projects with customized schemas, looks like this:



// playtomic-ios/tuist/Workspace.swift

import Foundation
import ProjectDescription
import ProjectDescriptionHelpers

// Assigned with Production build configuration
let productionScheme = Scheme(
  name: "Playtomic",
  shared: true,
  ...
)

// Assigned with Development build configuration
let integrationScheme = Scheme(
  name: "Playtomic (Integration)",
  shared: true,
  ...
)

let workspace = Workspace(
  name: "Playtomic",
  projects: [
    "App/Project.swift",
    "Shared/Mozart/Project.swift",
    // Other Shared modules...
    "Features/Onboarding/Project.swift",
    // Other Feature modules...
  ],
  schemes: [
    productionScheme,
    integrationScheme,
  ],
  generationOptions: Workspace.GenerationOptions.options(
    autogeneratedWorkspaceSchemes: .disabled
  )
)

7. Tuist generate and first run

With this added to the tuist manifest project, one final command line execution of the tuist generate, creates and opens the workspace, one final run through iOS Simulators and, BOOM, Success!

Preparing for the Merge: Transitioning Code into the Development Branch

Transitioning from CocoaPods to Tuist wasn't a smooth sail. Here's how we tackled development experience gap between CocoaPods and Tuist.

A. Installation of the new tools

Getting started with Tuist meant installing new tools with specific versions. To streamline this process, we detailed comprehensive instructions in our README file. Developers could easily follow these steps to install the necessary tools such as Tuist and Carthage, fetch third-party libraries, and generate the new workspace.

B. Bridging the Gap: Daily Work Differences

We wanted the migration to be transparent and improve the development experience, but the nature of Tuist and CocoaPods was so different. We used to keep third party dependencies on the project repository under the Pods folder, but with tuist we decided to not keep third party library caches of SPM and Carthage on the repository.

In addition to that, Tuist required developers to execute the tuist generate command to update and generate the .xcworkspace and .xcodeproj files. Since these files are ignored by gitignore and changes made to the workspace file didn't reflect on others' devices.

To maintain a seamless transition, we aimed to keep developers workflow similar to CocoaPods. We didn't want them to manually run the tuist generate every time they switch branches. Instead, we wanted them to open the .xcworkspace file and dive into coding.

To achieve this, we leveraged git hooks as trigger points. By adding bash scripts to post-checkout and post-merge git hooks, we ensured that every time developers interacted with the project:

Third-party dependencies were updated as needed
Workspace file was consistently updated on their repositories.
Project will be opened automatically with the updated files and third-party repositories



#!/usr/bin/env bash
echo "" 
echo "Press control ^ + C to cancel git hook execution" 
echo "" 

TUIST_MANIFEST_PATH=".../playtomic-ios/tuist" 

echo "Fetch and update third party dependencies"
tuist fetch --update --path TUIST_MANIFEST_PATH 

echo "Generate and open project workspace" 
tuist generate --path TUIST_MANIFEST_PATH

Experimental Phase: Challenges we faced and how we overcame it

After finalizing Tuist migration implementation and merging changes with the codebase in December 2022, we began the experimental phase. During this period, we built the project on several Mac machines, identifying and addressing the following issues:

1. Duplicate Static Dependencies

When we used tuist generate command for generating the Tuist project, we encountered hundreds of warnings displayed on the Terminal about duplicated dependencies:

Warning: Target 'Tournament' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'Academy' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'Location' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'GoogleSignIn' has been linked from target 'Onboarding', target 'Payment', target 'Paywall', etc.., it is a static product so may introduce unwanted side effects.
etc..

To address this, we changed the linking from static to dynamic frameworks. For dependencies imported by a single module, we kept them statically linked as it was not causing any duplication issue. However, for dependencies imported by multiple internal modules, especially those with transitive libraries, we switched to dynamic linking.

For example, GoogleSignIn had some libraries inside such as AppAuthCore which was also imported by other google products like Firebase, this means that our app is indirectly linked those frameworks. The configuration in our Tuist Dependencies.swift file better explains this approach:

The configuration in our Tuist Dependencies.swift file describes this approach:

2. Disabling Tuist's Automatic Code Generation

Since we were using R.swift instead of SwiftGen for generating resources with references in code, we disabled Tuist's automatic code generation tool.

If you don’t want to enable resource synthesizers, you can just pass empty array resourceSynthesizers: [] in Project initializer.



Project(
  ...,
  resourceSynthesizers: [],
  ...
)

This code snippet will disable the auto-generation of resources while still allowing to continue other magical automatic processes, like generating Info.plists and etc.

3. Reduce New Feature Module Creation Steps

There is a tool in Tuist called scaffold, it helps you bootstrap new components from .stencil templates that you defined, these templates are consistent with your project.

We did use it to create a new feature module called Level to Playtomic:



tuist scaffold mvi-module --name Level

For example a file called Presenter.stencil will be used as template, arguments passed in scaffold command like name as —name Level will be replaced by {{ name }} in the stencil file.

Therefore this method enables us to create several files at different locations to generate an entire feature module with just a single command.

Performance Metrics and Optimization

To determine the effectiveness of the migration, we explored key performance metrics, comparing the new system with the old one. Our focus was on improving the app launch time and reducing the size of the final binary as decent goals.

App Launch Time

We have analyzed app launch time by utilizing Firebase Performance tool:

Tuist IPA
- Initializing for first time: 1.08s
- AppDelegate - didFinishLaunchingWithOptions: 414ms
CocoaPods IPA
- Initializing for first time: 2.03s
- AppDelegate - didFinishLaunchingWithOptions: 671ms

These results highlight a notable enhancement in the initialization phase, we were so happy because this improvement was not our primary goal but something nice to have!

Size of Final Binary

The transition to Tuist allowed us to create cleaner and less duplicated target configurations, resulting in a more streamlined and efficient project structure. This migration contributed to a smaller binary size:

CocoaPods IPA:
- Total Size of the IPA: 107.575.065 bytes (107.6 MB)
- Payload: 167.386.848 bytes (167.4 MB)
- App on AppStore: 164.9 MB
Tuist IPA:
- Total Size of the IPA: 99.723.283 bytes (99.7 MB)
- Payload: 140.351.007 bytes (140.4 MB)
- App on AppStore: (138.7 MB)

Build Compilation Time on M1 Pro with 16GB RAM

Additionally, we conducted an in-depth analysis of the build compilation time on an M1 Pro with 16GB RAM. The results, comparing CocoaPods and Tuist, are as follows:

CocoaPods:

Xcode Clean: 2.9s
Xcode Build Playtomic DEBUG: 189s
Xcode Build Playtomic RELEASE: 196s

Tuist:

Xcode Clean: 5.9s
Xcode Build Playtomic DEBUG: 163s
Xcode Build Playtomic RELEASE: 170s

Further optimizations were achieved by leveraging Tuist's caching capabilities. With tuist cache warm that pre-compiles modules. We’ve particularly utilized tuist cache for third party libraries and we observed reduced build time:

Xcode Clean: 2.6s (2.25x Faster)
Xcode Build Playtomic DEBUG: 143s (1.13x Faster)
Xcode Build Playtomic RELEASE: 154s (1.1x Faster)

CI Improvements

In the final phase of our transition, we had to move all the scripts we used before. These scripts were crucial for different tasks:

Running Unit Tests: Essential for checking if our code changes were good to go before merging them (you know, the whole Pull Request approval thing).
Generating Testing Binaries: This helped us create versions of our app specifically for review and Quality Assurance (QA) testing.
Releasing to the AppStore: The process of actually shipping our app to the big wide world. This includes submitting all the necessary information and metadata.

We didn't just stop at improving the app build, we also fine-tuned our Continuous Integration processes on Bitrise our CI provider.

Moving from CocoaPods to Tuist wasn't just about building and testing; it was about making everything faster. We used Tuist magic for pre-compiling projects tuist cache warm and Bitrise key-based caching for third-party Swift Package Manager (SPM) and Carthage libraries source codes.

As outlined before, we used to have the Pods folder on the project repository for the same purpose of not downloading dependencies on CI machine and avoid execution of **pod** **install** on every CI job.

With Bitrise key-based caching, we fetched third-party libraries, including their source code, only once and later efficiently retrieved them using Bitrise's caching mechanism. This resembles the approach of storing the Pods folder but with the added benefits of reduced repository size and improved efficiency. Users can configure caching to create new cache archives periodically, since cache archive remains valid for seven days but resetting if updated and discarding old ones automatically.

Without key-based caching, fetching third-party dependencies for SPM and Carthage projects incurred unnecessary overhead on every CI job. Now, we fetch once, update the key based on new hashes generated from Package.resolved and Carthage.resolved files, mirroring the concept of CocoaPods' Podfile.lock CHECKSUM value. This streamlining significantly improves the efficiency of our CI pipelines.

Conclusion: Reflecting on Our Journey

Migrating from CocoaPods to Tuist was a strategic move that has significantly benefited Playtomic. Not only did we overcome the limitations posed by CocoaPods, but we also achieved a more streamlined, efficient, and future-ready development environment.

This success was accomplished by careful planning and a focus on minimizing disruption for other developers.

As we look back on this transition, we're reminded of the importance of adaptability in technology. By embracing Tuist, we've not only enhanced our current development system but also positioned ourselves for future growth and advancement.

This experience underscores our commitment to continuous improvement and our dedication to delivering the best possible experience to our users and developers.

Resources

Adding multi device previews in Xcode and Android Studio

Angel G. Olloqui — Wed, 05 Apr 2023 09:48:47 +0000

Developing with SwiftUI and Jetpack Compose is very user-friendly. Both frameworks offer a highly readable declarative syntax and a live preview feature that updates in real-time while coding. This live preview allows developers to easily configure various device settings, although it can be a bit cumbersome to specify multiple device factors. At Playtomic, we have created a simple utility for both platforms that allows us to run our previews on a predefined set of devices. Here's what we did:

Android

This platform was the easy one, as it already supports aggregated annotations out of the box. All you need to do is include somewhere in your project:

@Preview(showSystemUi = true, device = Devices.PIXEL_4_XL, name = "Pixel 4 XL")   // big xxxhdpi
@Preview(showSystemUi = true, device = Devices.NEXUS_5, name = "Nexus 5")   // small-medium xxhdpi
annotation class MultiDevicePreview

And then just replace the usage of @Preview by @MultiDevicePreview in your previews:

@MultiDevicePreview
@Composable
private fun AMultiDevicePreview() {
    Text("Multidevice preview")
}

iOS

Things become a bit more complicated for iOS. Firstly, Xcode only recognizes previews that implement the PreviewProvider protocol. As a result, extending the interface with our custom version of MultiDevicePreviewProvider doesn't work. Instead, we created a second protocol to tackle this problem:

public protocol MultiDevicePreview {
    associatedtype DevicePreviews : View
    @ViewBuilder @MainActor static var devicePreviews: DevicePreviews { get }

    @MainActor static var devices: [PreviewDevice] { get }
    @MainActor static var previewName: String? { get }
}

And then, we provided a default implementation of the previews for those using the new protocol:

public extension PreviewProvider where Self: MultiDevicePreview {
    static var previews: some View {
        ForEach(devices) { device in
            AnyView(devicePreviews)
                .previewDevice(device)
                .previewDisplayName([previewName, device.rawValue].compactMap { $0 }.joined(separator: " - "))
        }
    }

    static var devices: [PreviewDevice] { PreviewDevice.allCases }
    static var previewName: String? { String(describing: Self.self) }
}

We also declared some predefined list of devices to simplify the management:

extension PreviewDevice {
    public static let iPhone14 = PreviewDevice("iPhone 14")
    public static let iPhone14Max = PreviewDevice("iPhone 14 Pro Max")
    public static let iPhoneSE = PreviewDevice("iPhone SE (3rd generation)")
    public static let allCases = [iPhone14, iPhone14Max, iPhoneSE]
}

extension PreviewDevice: Identifiable {
    public var id: String {
        rawValue
    }
}

With all of the above, you can make use of multidevice previews by conforming your preview to MultiDevicePreview and replace the method previews by devicePreviews like:

struct AMultiDevicePreview: PreviewProvider, MultiDevicePreview {
    static var devicePreviews: some View {
        Text("multidevice preview text")
    }
}

Not a huge difference, but a nice small addition to the toolkit!

Data Races with value types in Swift

Angel G. Olloqui — Fri, 26 Aug 2022 14:49:54 +0000

This week I had an interesting discussion around a possible data race condition due to wrong threading synchronisation when manipulating a value type (a String in this case) in a class.

The buggy code

final class MyClass {
    var token: String

    init(_ token: String = "") {
        self.token = token
    }

    func myMethod() -> Bool {
        token.isEmpty
    }
}

In a first look, this might seem correct. We have just a var with a String, which is a value type, and a method to just check if the token is empty that only calls the isEmpty from String. Straightforward code and safe right? well, it is OK as long as you do not introduce threading, but the moment you do it will not. Let me elaborate.

The test

If you run this test with Thread Sanitizer enabled:

 func test_data_race() {
        let sut = MyClass()

        DispatchQueue.concurrentPerform(iterations: 1_000_000) { i in
            sut.token = "\(i)"
            _ = sut.myMethod()
        }
    }

you will see this output:

WARNING: ThreadSanitizer: data race (pid=8329)
  Read of size 8 at 0x000107c1aab8 by thread T2:
    #0 closure #1 in DataTests.test_data_race() DataTests.swift:69 (Tests:arm64+0xde354)
    #1 partial apply for closure #1 in DataTests.test_data_race() <compiler-generated> (Tests:arm64+0xde3e4)
    #2 partial apply for thunk for @callee_guaranteed (@unowned Int) -> () <null>:73675156 (libswiftDispatch.dylib:arm64+0x42f4)
    #3 _dispatch_client_callout2 <null>:73675156 (libdispatch.dylib:arm64+0x35dc)

  Previous write of size 8 at 0x000107c1aab8 by main thread:
    #0 closure #1 in DataTests.test_data_race() DataTests.swift:69 (Tests:arm64+0xde374)
    #1 partial apply for closure #1 in DataTests.test_data_race() <compiler-generated> (Tests:arm64+0xde3e4)
    #2 partial apply for thunk for @callee_guaranteed (@unowned Int) -> () <null>:73675156 (libswiftDispatch.dylib:arm64+0x42f4)
    #3 _dispatch_client_callout2 <null>:73675156 (libdispatch.dylib:arm64+0x35dc)
    #4 _swift_dispatch_apply_current <null>:73675156 (libswiftDispatch.dylib:arm64+0x43a0)
    #5 @objc DataTests.test_data_race() <compiler-generated> (Tests:arm64+0xde448)
    #6 __invoking___ <null>:73675156 (CoreFoundation:arm64+0x11c5ec)

  Location is heap block of size 32 at 0x000107c1aaa0 allocated by main thread:
    #0 __sanitizer_mz_malloc <null>:73675156 (libclang_rt.tsan_iossim_dynamic.dylib:arm64+0x51004)
    #1 _malloc_zone_malloc <null>:73675156 (libsystem_malloc.dylib:arm64+0x1527c)
    #2 DataTests.test_data_race() DataTests.swift:66 (Tests:arm64+0xde07c)
    #3 @objc DataTests.test_data_race() <compiler-generated> (Tests:arm64+0xde448)
    #4 __invoking___ <null>:73675156 (CoreFoundation:arm64+0x11c5ec)

  Thread T2 (tid=6246748, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: data race DataTests.swift:69 in closure #1 in DataTests.test_data_race()

So ThreadSanitizer is detecting a data race in the code when accessing the token.
What does it mean? basically you are making a wrong usage of the variable. It gets read and write operations concurrently but the variable itself is not protected, and the fact that it is a value type does not help.

What can this cause? it is undefined, but in practice most likely you will have a crash when compilation optimizations are enabled.

The fix

OK, so this simple code can crash when reading and writing the token in parallel from different threads! How can we fix it? we just need to make serial access to read/write. There are multiple ways of doing it (with different primitives), but this could be one:

final class MyClass {
    private let syncQueue = DispatchQueue(label: "com.test.myQueue", attributes: .concurrent)
    private var _token: String
    var token: String {
        get {
            syncQueue.sync {
                _token
            }
        }
        set {
            syncQueue.async(flags: .barrier) {
                _token = newValue
            }
        }
    }

    init(_ token: String = "") {
        _token = token
    }

    func myMethod() -> Bool {
        token.isEmpty
    }
}

As you can see, what we did is to protect the var by forcing serial writing to it, so multiple reading can happen but only 1 thread can execute a write at a time (the barrier waits for all previous readings to finish and postpones all subsequent read/write accesses till the write is done). The resulting code is slower to execute, but it is now safe.

Final thoughts

I wanted to share a few thoughts around this issue, that are common misconceptions in the Swift community:

❌ Value types are thread safe

Since the value type has a copy semantic, it may seem logical to think that they are inherently protected from data races. However, that is not the case. Swift does not guarantee thread safety in value types, so accessing any var from multiple threads is a potential data race condition. This issue of course does not apply to let variables since they are immutable.

❌ Value types are always copied

That is the semantic but not really what happens under the hood. When passing value types around, Swift compiler is smart enough to know if the copy is needed, removing unnecessary copies. In practice it uses a CopyOnWrite(COW) strategy, where it will make the copy only when the value is modified, but not when passed around. As a result, in most situations you will actually have a pointer to the same underlaying memory address even when using value types.

❌ Tests do always behave like production code

The fact that a test does not crash is no guarantee to assert that some code can not crash in production. Tests run in simulated environments and they normally have different compilation options than the ones in your final builds. For example, ARC will make aggressive optimizations when compiling with the proper options, so lots of unnecessary retain/releases will be removed from final builds. In this particular case, my test suit was not crashing, and I was only able to see some wrong usage by activating the Thread Sanitizer.

Extra reading

Modernizing a CI+CD pipeline with Github Actions

Sergio Garcia Moratilla — Tue, 07 Jun 2022 14:41:33 +0000

Our CI+CD has been working for 5 years long. You know, if it ain't broken, don't fix it. But the company is not the same. It's time to update it!

Let me wrap up a bit our current setup. I am going to be brief, I promise. We have a Jenkins cluster on-premise. That is, we manage (and maintain) a bunch of hosts that run Jenkins slaves and a host that runs the master. They are within our on-premise VPN, a reminiscence of our first hosting provider.

Our production backend runs a containerized system on top of a Docker Swarm cluster. Our container registry is Nexus, which allows us to deploy our services in the Swarm.

Both systems run within their own independent VPNs.

Our code repositories are in Github. Our Jenkins is listening for changes in Github. When we merge to develop/production, Jenkins pushes the image to the Nexus, connects via ssh to the managers of the cluster and runs the deployment command (docker stack deploy). We have a common Jenkins pipeline for all our services.

Pros:

Simple.
Security on our side (VPN certificates + ssh keys are in our servers).
Jenkins is commonly known.
Stable: we haven't had to change it a lot all this time.

Cons:

We need to maintain and keep update the Jenkins machines.
As the team grows, the Jenkins cluster has to grow too to be able to run more jobs.
We have only configured Java 8 and 11.
Still on our old hosting provider.

We have been pretty happy with this setup so far. Maintenance is something that we always want to simplify at Playtomic. The fewer systems, the better. Besides, new versions of Java require new versions of the JDK, maven and thus... Jenkins. We have already been using Github Actions in other projects here, so that we know that it could be a fine replacement of Jenkins. The workflow defines the environment it is required to run (for example, the java version or the architecture), so it makes complete sense to use Github Actions.

Replacing Jenkins by Github Actions

We re-wrote the Jenkins pipeline as a Github Actions workflow. Our main concerns are two:

What about security? We are using ssh-keys to access the cluster. We can use Github secrets to store them, but we don't like having such an important piece of the security stored in a third-party.
Cost might be a problem in the future. Github Actions are charged by the minute of computation and we have 50 services.

We discovered that Github Actions allow you to run self-hosted nodes... so that's the solution to both problems. At this very moment, we can afford the minutes that we are spending, so we are adding a host just for deployments. We added a t3.nano, which is pretty cheap. ssh-keys are still 100% under our control, as they are installed in the machine.

Pros:

Still a common pipeline (workflow) but allows everyone to use new stuff (new java version, different languages, ...) without installing more tools in Jenkins.
Security is still on our side.

Cons:

We need to monitor the cost.

What's next?

Containerized Github Action runner

Worried about the cost? If you have a cluster, you can run as several copies as you want of the Github Action runner!

On-demand self-hosted runners

This would be a huge improvement to control the cost while still being able to scale the number of runners:
https://github.com/machulav/ec2-github-runner

Separate the CI from the CD

To separate pipeline for the CI and CD is a good idea, as their lifecycles are pretty different.
In our current setup, the workflow is responsible for running the deployment command. If that fails, then the whole pipeline fails, but the build was successful.

We have already tested ArgoCD in Kubernetes so that the CD is handled by the cluster itself.
If you are running Kubernetes, you can already do that with FluxCD or ArgCD

Replace Nexus by Github Packages

Nexus requires a lot of space (as it stores all the containers, libraries, packages, ... of your organization). We don't want to maintain that space, be responsible of the backups, ... so that we are considering to migrate to Github Packages too.

MVI in Playtomic mobile app

Angel G. Olloqui — Tue, 31 May 2022 12:06:15 +0000

Last summer we finally decided to move out of our classic MVP+UIKit/Android view architecture into a more modern one with SwiftUI/Compose as main actors for our view layer. Together with the UI framework changes, we found the need of switching to a more reactive architecture that better fits the nature of declarative UIs.

We spent some time analysing some of the most popular reactive architectures: MVVM, MVI and TCA. Without getting into much detail of our decision making (it would take a full post), we decided that MVI was the best fitting for our project. With it, we could get better separation of concerns and state management than in MVVM, unidirectional data flow, single source of truth and easy of testing, without the additional complexity added by TCA.

After around half a year working with MVI, the team is highly satisfied: all people in the team consider it a good/great choice and enjoys working with it, being the verbosity the only drawback.

Let us share a bit on how we do it:

The state management layer

This is how our MVI base class looks like in both platforms

open class BaseMVIPresenter<S: ViewState, A: ViewAction> {
    public var currentViewState: S { _viewState.value! }
    public var viewState: Observable<S> { _viewState }
    fileprivate let _viewState: MutableObservable<S>

    public init(initialState: S) {
        _viewState = MutableObservable(value: initialState)
    }

    func dispatch(action: A) {
        fatalError("Must be implemented by the children")
    }
}

open class MVIPresenter<S: ViewState, A: ViewAction, R: ActionResult>: BaseMVIPresenter<S, A> {
    private var middlewares: [MVIMiddleware<S, A, R>] = []

    public func with(middleware: MVIMiddleware<S, A, R>) -> Self {
        middlewares.append(middleware)
        return self
    }

    open func handle(action: A, results:  @escaping (R) -> Void) {
        fatalError("Must be implemented by the children")
    }

    open func reduce(currentViewState: S, result: R) -> S {
        fatalError("Must be implemented by the children")
    }

    override public func dispatch(action: A) {
        middlewares.forEach { element in
            element.handle(action: action, presenter: self)
        }
        handle(action: action) { [weak self] result in
            Executor.execute(inBackground: false) {
                guard let self = self else { return }
                self.middlewares.forEach { middleware in
                    middleware.handle(result: result, presenter: self)
                }
                self._viewState.value = self.reduce(currentViewState: self.viewState.value!, result: result)
            }
        }
    }
}


interface BaseMVIPresenter<ViewState, ViewAction> {
    val currentViewState: ViewState get() = viewState.value!!
    val viewState: Observable<ViewState>
    fun dispatch(action: ViewAction)
}

abstract class MVIPresenter<S : ViewState, A : ViewAction, R : ActionResult>(initialState: S) : BaseMVIPresenter<S, A> {
    override val viewState: Observable<S>
        get() = _viewState
    private val _viewState = MutableObservable(value = initialState)
    internal var middlewares = mutableListOf<MVIMiddleware<S, A, R>>()

    abstract fun handle(action: A, results: (R) -> Unit)

    abstract fun reduce(currentViewState: S, result: R): S

    override fun dispatch(action: A) {
        middlewares.forEach { element ->
            element.handle(action = action, presenter = this)
        }
        handle(action) { result ->
            Executor.execute(inBackground = false) {
                this.middlewares.forEach { middleware ->
                    middleware.handle(result = result, presenter = this)
                }
                this._viewState.value = this.reduce(currentViewState = this.viewState.value!!, result = result)
            }
        }
    }
}

fun <S : ViewState, A : ViewAction, R : ActionResult, T : MVIPresenter<S, A, R>> T.with(middleware: MVIMiddleware<S, A, R>): T {
    middlewares.add(middleware)
    return this
}

Note: In MVI there is no definition whether the data management part should be done in a presenter, view model or whatever. However, in our case we opted to call them "Presenters" to be more inlined with the rest of the app, but they are in practice maintaining state as classic ViewModel in Android.

Then, when building a feature, we need to provide the implementation of 2 methods:

handle: This method takes the actions triggered by some other component (normally the view) and handles the side effects. It emits new events (called "action results") with the results of the effects, like for example a network call. It does not perform any state management or manipulation, it just emits new result events.
reduce: Given a state and an action result, this method computes the next state of the view. Note that it behaves as a pure function, with no side effects.

In contrast with other simpler implementations of MVI, we opted to split the code into these 2 methods to isolate side effects from state manipulation, which make our tests much simpler and our overall solution more robust and clean. An added benefit is that there are no race conditions possible like in other architectures, since all calls to reduce are executed in serial with no partial updates possible.

In addition, we added an extra piece around the presenters, called Middleware, that are capable of reacting to events without doing state management. For example, we can add all analytics tracking into a middleware or all navigation actions. This way, our presenter stays purist, just doing the state management part, and we have a set of small middlewares with a single other purpose, making it once again easier to test and maintain.

Lastly, you can see how both platform implementations are quite similar, and they both avoid the usage of platform specific APIs like Combine or Flows (although they are used internally) to maximize code reusal when transpiling and also reduce the learning curve.

An example Presenter would look like:

internal class LessonDetailPresenter(...): MVIPresenter<LessonDetailState, LessonDetailAction, LessonDetailResult>(LessonDetailState.none) {

    override fun handle(action: LessonDetailAction, results: (LessonDetailResult) -> Unit) {
        when (action) {
            is LessonDetailAction.onAppear -> loadLesson(results, lessonId)            
            is LessonDetailAction.tapConfirmCancelEnrollment -> unregisterFromLesson(results)
            is LessonDetailAction.resendConfirmation -> resendConfirmation(results)
            ...
        }
    }

    override fun reduce(currentViewState: LessonDetailState, result: LessonDetailResult): LessonDetailState {
        return when (result) {
            is LessonDetailResult.lessonLoading -> LessonDetailState.loading
            is LessonDetailResult.lessonLoaded -> LessonDetailState.detail(sections = result.lesson.mapToLessonDetail(me = userId))
            ...
        }
    }

    private fun loadLesson(results: (LessonDetailResult) -> Unit, lesson: LessonId) {
        results(LessonDetailResult.lessonLoading)
        activityService.fetchLesson(lessonId)
            .then { results(LessonDetailResult.lessonLoaded(it)) }
            .catchError { results(LessonDetailResult.loadLessonByIdFailed(error)) }
        }
    }
    ...
}

And some associated middleware for navigation:

internal class LessonDetailNavigator(...) : MVIMiddleware<LessonDetailState, LessonDetailAction, LessonDetailResult>() {

    override fun handle(action: LessonDetailAction, presenter: MVIPresenter<LessonDetailState, LessonDetailAction, LessonDetailResult>) {
        when (action) {
            LessonDetailAction.tapOpenMaps -> openMaps(presenter = presenter)
            LessonDetailAction.tapAddToCalendar -> addLessonToCalendar(presenter = presenter)
            ...
        }
    }

    override fun handle(result: LessonDetailResult, presenter: MVIPresenter<LessonDetailState, LessonDetailAction, LessonDetailResult>) {
        when (result) {
            is LessonDetailResult.loadLessonByIdFailed -> dismiss()
            else -> {}
        }
    }
    ...
}

Our view layer

Then, our views basically just receive 2 parameters:

public struct LessonDetailView: View {
    @ObservedObject var state: ObservableViewState<LessonDetailState>
    let dispatcher: (LessonDetailAction) -> Void

    public var body: some View {
        ...
    }
}

@Composable
private fun LessonDetailView(
    viewState: LiveData<LessonDetailState>,
    dispatcher: (LessonDetailAction) -> Unit
) {
    ...
}

As you can see, we are making use of state hoisting for encapsulating the presenter injection (from the view layer, it does not know what class is behind the state management). This also makes our code much more reusable, and very easy to setup the previews, since we do not need to mock any data, service or presenter, just passing the view state down is enough. For this state hoisting we are making use of a parent UIViewController/Fragment, since our app is now a mixed app with only part of the views in SwiftUI/Compose. They look like this:

final class LessonDetailViewViewController: SwiftUIViewController<LessonDetailState, LessonDetailAction, LessonDetailView> {
    override func contentView(
        viewState: ObservableViewState<LessonDetailState>,
        dispatcher: @escaping (LessonDetailAction) -> Void
    ) -> LessonDetailView {
        LessonDetailView(state: viewState, dispatch: dispatcher)
    }
}

class LessonDetailFragment : ComposeFragment<LessonDetailState, LessonDetailAction>() {
    @Composable
    override fun ContentView(
        viewState: LiveData<LessonDetailState>,
        dispatcher: (LessonDetailAction) -> Unit
    ) = LessonDetailView(viewState = viewState, dispatcher = dispatcher)    
}

Where the SwiftUIViewController and ComposeFragment are base classes that inject the presenter and create the UIHostingController/ComposeView for using SwiftUI/Compose inside with the content returned by the concrete contentView method on each case.

Conclusions

There are tons of options and architectures to use with SwiftUI/Combine. In Playtomic, we opted for a MVI version where we have a clear separation of the different responsibilities, single source of truth and a simple and unidirectional data flow. It also allows for very simple views and easy transpilation between platforms, with
only one drawback so far: the extra boilerplate needed.

References

GoDaddy Studio’s Journey with State Management: Great article explaining some of the issues they found with MVP, MVVM and MVI in its simpler form. We find ourselves very aligned with their journey.

Let's talk about performance and MongoDB

Sergio Garcia Moratilla — Wed, 16 Feb 2022 10:21:38 +0000

If you, a backend developer, had to describe your job, what would you say? We usually talk a lot about servers, clusters, layers, algorithms, software stacks, memory consumption, ...

We put data into databases and we get it back as fast as we can.
Databases are our cornerstone. Why don't we talk more often about them? We are always relying on our ORMs.

My best advice? Simplify your queries. Simplify your data models. Simplify your access patterns.

Performance in MongoDB

Metrics

If you are using MongoDB, these two metrics will be your best friends:

Scanned Documents/Returned ratio
IOPS

Scanned docs/returned: it means how many documents you are reading from disk vs the number of documents you are actually returning in your find() or aggregate(). Ideally: this should be 1 (every document read is returned). The only way to get it? All your queries must be covered by indexes. Indexes are in memory (or they are if they fit), so MongoDB doesn´t have to read and filter them from disk.

IOPS: I/O operations per second. That is, operations on disk. It is correlated with scanned docs, as they are read from disk. But there are more sources of IOPS, for example writes.
Your disk will give you a limit to your maximum IOPS. Ours is 3000.

Your goal is to keep IOPS below that threshold, and as low as possible. It is hard to know how many IOPS your query consumes, but it is easy to know the scanned docs/returned ratio.

Tools

How can you analyze why your database is behaving as it is?

MongoDB Profiler
If you can afford to enable it, do it now. It's the best source of info. We use the Atlas MongoDB Profiler and it is worth every penny.

Explain planner
The planner is the key to understanding your access patterns.
There are several ways of calling it, but you can start with explain() after your cursor:

db.our_collection.find(query).explain()

You can use add executionStats to get more data about the query.

db.our_collection.find(query).explain("executionStats")

There are many stages, but these ones are the most important:

COLLSCAN: the query is scanning the collection in disk. Pretty bad, as no index covered the search, so MongoDB has to read the whole collection.
IXSCAN: the query is using an index to filter. It doesn´t mean that all the query is covered by the index, but at least some part.
FETCH: the planner is reading the documents from the collection. If your query is returning documents, you will get a FETCH stage probably (unless your query is covered by the index).

This is an example of one of our queries:

winningPlan: 
      { stage: 'COLLSCAN',
        filter: 
         { '$and': 
            [ { '$or': 
                 [ { 'invitaed_user_id': { '$eq': '1' } },
                   { owner_id: { '$eq': '1' } },
                   { 'player_id': { '$eq': '1' } } ] },
              { is_canceled: { '$eq': false } },
             ] } },

This is not good, as it is scanning the whole collection.

Another one:

db.our_collection.find(query).explain("executionStats")
executionStats: 
   { executionSuccess: true,
     nReturned: 22,
     executionTimeMillis: 0,
     totalKeysExamined: 24,
     totalDocsExamined: 22,
     executionStages: 
      { stage: 'FETCH',
        nReturned: 22,
        docsExamined: 22,
        inputStage: 
         { stage: 'OR',
           nReturned: 22,
           inputStages: 
            [ { stage: 'IXSCAN',
                nReturned: 0,
                indexName: 'example-index-1',
                indexBounds: 
                 { owner_id: [ '["1", "1"]' ],
                   start_date: [ '[MaxKey, MinKey]' ] } },
              { stage: 'IXSCAN',
                nReturned: 22,
                indexName: 'example-index-2'
                indexBounds: 
                 { 'player_id': [ '["1", "1"]' ],
                   start_date: [ '[MaxKey, MinKey]' ] ] },
                keysExamined: 23,
                dupsDropped: 0 }] } } },

Here you can two see IXSCANs, merged by and OR. After that, the query is fetching the documents. I am reading the query inside-out. example-index-1 is used to resolve one part of the query, and example-index-2 for the other part.

Sometimes you will get a FETCH just after an IXSCAN: it means that the index covers only part of the filter. After that, the planner needs to read the documents from disk to finish the filter.

The complete list of stages? You need to check the code.

Simplify your queries

$ors

$ors are the devil. You, as a programmer, are used to thinking in $ors. You add a few $ors, your condition gets much more expressive. But guess what? Your query has gotten exponentially more complex. With every condition you add to the $or, you are adding one more combination of parameters.

How does the planner resolve all those combinations? It needs an index for each of them.

Do you remember the explain() above? The query was:

{
    $or: [{"owner_id: "1"}, {"player_id: "1"}].
}

The indexes used?

example-index-1: {"owner_id: "1", "start_date": 1}
example-index-2: {"player_id: "1", "start_date": 1}

And here is another tip: bigger indexes can cover smaller queries as long as the fields are at the beginning of the index.
We are not using start_date to filter.

What would happen if I add an extra $or?

{
    $or: [{"owner_id: "1"}, {"player_id: "1"}].
    $or: [{"is_canceled": true}, "start_date": {$gt: ISODate("2022-02-02T00:00:00)}]
}

Then there would need 4 combinations:

{owner_id, is_canceled}, 
{owner_id, start_date}, 
{player_id, is_canceled], 
{player_id, start_date},

Denormalized fields

If you find yourself filtering by several fields within an $or, or sorting by several fields, consider adding a denormalized field (based on the others).

Yeah, my apologies to the third normal form.

Keep reading, we will give an example pretty soon using the so-called Summary Pattern.

nulls are always the minimum value

This is a minor trick, but still useful. Let's say you have a nullable field, and you have to filter (or sort descending) by that field.

You will probably find yourself using a query similar to this:

{
 $or: [
        field: {$exists: false}, 
        field: {$lte: value}
      ]
}

That is, if the field is null, then it passes the filter. Otherwise, compare. We used this filter to check if a player could join a match given their level and the level restrictions of the match.

That's an $or. We don't like $ors. What would happen if we compare our value to null? Let's check the sorting rules in MongoDB.

MinKey (internal type)
Null
Numbers (ints, longs, doubles, decimals)
Symbol, String
Object
...
MaxKey (internal type)

That is, null is always the lower value when comparing (except for the MinKey object, we will talk about it later).

Our query can be simplified by this:

{
  field: {$lte: value}
}

It works for $lt and $lte (lower than and lower or equal).
It works too if you are sorting by descending order of field {field: -1} because the object with null will be at the end of the sort.

Counts are costly in MongoDB

Counting seems an easy operation, but it is not. Even if you have an index, MongoDB needs to traverse the index due to the way MongoDB builds B-trees: they don't store the number of leaves that the sub-tree have. So they need to traverse the index until the end.

Again, if you are counting using a query with $or, it makes the counting even more complex: the query needs to take into account possible repeated documents.

For example, we used counts to compute the position of a player in a ranking (the original query was even more complex).

{ranking_id: ?0}, 
{
 $or: [
        { value: {$gt: ?1}},
    { value: {$eq: ?1}, last_modified: {$gt: ?2} }
 ]
}

It required several indexes to count:

{ranking_id: 1, value: -1}
{ranking_id: 1, value: -1, last_modified: -1}

How can we avoid counting on several indexes? Add a single field that summarize the fields that you are filtering.

For example: weight = append(value, last_modified)
With that new field, we only required one single index:
Indexes:

{ranking_id: 1, weight: -1}

This is called the Summary Pattern.

How to build indexes

Ok, indexes are our best tool to keep MongoDB as performant as possible. So the next step, how do we know what indexes we should build?

Performance advisor

If you are in Atlas, use the Performance advisor.

At some point, you will know your system better than the Performance Advisor, but it is a good starting point.

Clone your production collections and explain() it in local

Test your indexes thoroughly before you put them in production: - it takes a lot of IOPS to build them.

once you built it, the planner is taking it into account too, even if it is not finally used.

Remember what we said before:

COLLSCAN: bad.
IXSCAN: good.
FETCH: good if it is the final step. Bad in between. Others that you will see:
COUNT: ok.
MERGE: ok-ish, you probably could do better.
MERGE_COUNT: good.

By the way, there are blocking and non-blocking stages, meaning that a stage needs to wait for the previous one before it can start computing results.

ESR: Equal-Sort-Range

Have you ever wondered what fields should go first in an index? You need to follow this rule:

Fields filtered by equals ($eq, $in in some cases, ...) go first.
Then fields used in the sort stage. Remember that the index has to be built in the same order as you are sorting.
Then fields filtered by a range ($lt, $lte, $gt, ...).

ESR is the most useful rule you will find to build indexes. You should read as much as you can about it until you understand it.

This post by Alex Belilacqua is a gem.

Disambiguate equals

If you have two fields that are going to be filtered by $eq, what should go first?

The answer is that it doesn't matter. You don't need to worry about having a more balanced tree.

Just keep in mind the ESR rule. If one of them goes in a sort or a range, then it goes the latter.

Rollover process

Building an index is one of the most costly operations. Your IOPS will go nuts. If you need to do that in your production environment, and your collection is big enough, then we recommend you to use a rolling process. It starts the index build in a secondary, then promotes it to primary once the build is finished. You will be able to build any index even when your database load is high.

In Atlas, it's just one click.

Remove / Hide indexes

The more indexes you have, the worst for the planner. The planner runs the query through the indexes it has and then takes the most promising.

Again, counting when you have several indexes is pretty bad.

Sometimes, you cannot just remove an index in production. You can check using your profiler if it can be removed but you might not be 100%. One not-frequent query might launch a COLLSCAN and then you would miss that index.

Luckily, since MongoDB 4.4 you can hide indexes. We use them to detect what indexes we can remove safely.

Limit your queries

Have you set a maximum limit of time to your queries? Why not? Do your clients have a request time out? Then it can be a healthy practice to avoid unexpected uses of your APIs.

db.our_collection.countDocuments(query, {maxTimeMS: 100})
MongoServerError: Error in $cursor stage :: caused by :: operation exceeded time limit

Do you see all these orange dots? No one was waiting for the backend to reply.

References

Automating rollout releases in Android

Angel G. Olloqui — Sat, 01 Jan 2022 10:14:06 +0000

As part of our new release process (weekly releases) we are also changing the way we are publishing the apps for users. Since it is now automatic, it is crucial for us to have a phased released in which only a small subset of our users get the latest build, increasing daily and acting as a "failsafe" in case of an important bug makes it into production.

For iOS, we can set the Phased Release + Publication Date option and the AppStore will handle it in your behalf, starting the release on a certain date and increasing the automatic updates to 1%, 2%, 5%... each day.

However, Google's approach is different. They do not offer a release date nor an automated phased release. Instead, they offer you with an API (and web dashboard) where you can set the percentage of users yourself at any time. This is in many senses much better than the Apple one, especially since this actually controls the releases and not just the automatic updates like in Apple, but it has a downside: it is all manual.

With our current setup of weekly releases and a phased released across 6 days (2%, 5%, 10%, 20%, 50%, 100%), this basically means having to update every single day the rollout amount manually. I am not sure about you, but having to enter every single day to click some button is the last thing I want to do.

Automating the release rollout

So, we wondered, if we have an API to control the rollout percentage, isn't that enough to make it automatic? what if we have a CI job that run every day and basically checks if there is an ongoing rollout release, and in that case increases the percentage? Let's see how we did it:

# rollout_update.py

import copy
import sys
import httplib2
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
from oauth2client.client import AccessTokenRefreshError

TRACK = ('production')

# To run: rollout_update package_name json_credentials_path
def main():
  PACKAGE_NAME = sys.argv[1]
  credentials = ServiceAccountCredentials.from_json_keyfile_name(
    sys.argv[2],
    scopes='https://www.googleapis.com/auth/androidpublisher')

  http = httplib2.Http()
  http = credentials.authorize(http)

  service = build('androidpublisher', 'v3', http=http)

  try:
    edit_request = service.edits().insert(body={}, packageName=PACKAGE_NAME)
    result = edit_request.execute()
    edit_id = result['id']

    track_result = service.edits().tracks().get(editId=edit_id, packageName=PACKAGE_NAME, track=TRACK).execute()
    old_result = copy.deepcopy(track_result)

    print("Current status: ", track_result)
    for release in track_result['releases']:
        if 'userFraction' in release:
            rolloutPercentage = release['userFraction']
            if rolloutPercentage == 0:
                print('Release not rolled out yet')
                continue
            elif rolloutPercentage < 0.02:
                release['userFraction'] = 0.02                         
            elif rolloutPercentage < 0.05:
                release['userFraction'] = 0.05
            elif rolloutPercentage < 0.1:
                release['userFraction'] = 0.1
            elif rolloutPercentage < 0.2:
                release['userFraction'] = 0.2
            elif rolloutPercentage < 0.5:
                release['userFraction'] = 0.5
            elif rolloutPercentage < 1.0:
                del release['userFraction']
                release['status'] = 'completed'
            else:
                print('Release already fully rolled out')
                continue        
    if old_result != track_result:
        completed_releases = list(filter(lambda release: release['status'] == "completed", track_result['releases']))
        if len(completed_releases) == 2:
            track_result['releases'].remove(completed_releases[1])

        print("Updating status: ", track_result)
        service.edits().tracks().update(
                    editId=edit_id,
                    track=TRACK,
                    packageName=PACKAGE_NAME,
                    body=track_result).execute()
        commit_request = service.edits().commit(editId=edit_id, packageName=PACKAGE_NAME).execute()
        print('Edit ', commit_request['id'], ' has been committed')    


  except AccessTokenRefreshError:
      raise SystemExit('The credentials have been revoked or expired, please re-run the application to re-authorize')

if __name__ == '__main__':
  main()

In order to run this step, you need:

Get a Google Developer Service JSON key credentials file. If you have being using some automation tools like Fastlane for uploading the APK to the GooglePlay, you have this already. Otherwise, follow the instructions from Fastlane Supply setup
Install pipenv or some dependency manager for python since the script uses google-api-python-client and oauth2client. You could get them by running:

pipenv install google-api-python-client                            
pipenv install oauth2client

Run the script: pipenv run rollout_update.py <your_package> <json_credential_path>

The script will do the following:

Create a new Google Edit
Fetch the production track release info
For each release, if it has a rollout in progress, then it increases the rollout percentage to the next "step", where steps are: 2%, 5%, 10%, 20%, 50%, 100%.
If changes performed, then commit the Edit

Connecting the CI

So now that we have the script to increase the rollout, all we need is to schedule it. In our case, we are using Bitrise, so we decided to schedule a workflow that runs the script every night. We even created a Bitrise step in case you want to use it that handles the dependencies and running the script.

  update_rollout:
    steps:
    - git::https://github.com/angelolloqui/bitrise-step-google-play-rollout-update.git@master:
        inputs:
        - package_name: com.playtomic
        - service_account_json_key_path: "$BITRISEIO_BITRISEIO_GOOGLEPLAY_SERVICE_ACCOUNT_JSON_URL"

Note: Your credentials file should be stored somewhere secured, like the Generic File Sorage of Bitrise

Halting a failing release?

If a release goes wrong, all you need to do is to go to the Google Dashboard and halt the release as you would normally do with a manually controlled phased release. The script will simply detect the release is halted and will just ignore it.

What about Hotfixes?

Well, the script makes no assumption on the type of build or release, so it will basically just work for any release, including hofixes. However, normally in hotfix builds, since they tend to be critical (otherwise we would not make a hotfix but wait for next week build) we normally deploy it to 100% of the user base, so we do not really need this step to run.

Decorator pattern vs inheriting in Kotlin

Manuel González Villegas — Mon, 13 Dec 2021 08:15:03 +0000

In playtomic, especially in the mobile team, we are fan of the decorator pattern, and in this article I will try to explain why it is a better idea to use the decorator pattern than inheritance, showing an example of how we use it.

This article will be technical, so I recommend to you to read before a bit of theory about the difference between inheritance and the decorator pattern. https://newbedev.com/decorator-pattern-versus-sub-classing

The inheritance

The inheritance in Kotlin is the same as in some other languages, we use the inheritance when we know there is a relationship between a child and its parent class, for example, an iPhone is a phone, a phone is an electronic device. Each child is an specialised version of the parent in which inherits all the properties of the parent.

The decorator pattern

The decorator pattern is a design pattern that allows behavior to be added to an individual object, dynamically, without affecting the behavior of other objects from the same class.
The coolest of the decorator pattern is that you can create a decorator class and use it more than once, and at any moment you can add or remove any other one (with the inheritance you can not).

An example in playtomic

Let’s see an example about how Playtomic app manages the HTTP requests. As a spoiler I will tell you that with the decorator pattern we can change which http client we are using in runtime, yes, in runtime. Let 's see.

To start, we have a IHttpClient interface, which is a basic thing that we will need to create the decorator classes. In this case we only have one simple method to make a request with an object that contains the needed information to make that request.

interface IHttpClient {
    fun request(httpRequest: HttpRequest): Promise<HttpResponse>
}

In our case, we also have some other implemented methods (inside the interface) to make our lives easier while we are making requests. All of them are very similar to this one:

    fun get(endpoint: String, params: Map<String, Any>?): Promise<ByteArray> =
            request(HttpRequest(method = HttpMethod.get, url = endpoint, queryParams = params))
                    .then(map = { it.body })

Well, I am not going to show very deeply the implementation of each of our http clients implementation, I will just enumerate some of them:

OkHttpClient => This one is the main client that makes the remote http requests to our backend

class OkHttpClient(
        private val baseUrl: String,
        timeOut: Long? = null,
        client: OkHttpClient? = null,
        private val urlEncoder: IHttpParameterEncoder = HttpUrlParameterEncoder(),
        private val bodyEncoders: List<IHttpParameterEncoder> = listOf(HttpJsonParameterEncoder(), ...))
    : IHttpClient {

This is the base HTTP client, so here there is nothing too much to see.

AuthHttpClient => With this one, we include the authentication and if the session has expired we can re-login to the user and throw the request again (and for the user is totally transparent)

class AuthHttpClient(private val httpClient: IHttpClient,
                     private val keychain: IKeyValueStorage) : IHttpClient by httpClient {

In this case we are decorating the http client to add the authentication into the request, so, as you can see into the class definition (... by httpClient) means that the interface that this class is implementing IHttpClient interface is being decorated by the parameter httpclient. So, in that case we can only implement into this class the methods that we need for the decoration (In this case, is the only one that we have). So, it looks like this:

    override fun request(httpRequest: HttpRequest): Promise<HttpResponse> =
            request(httpRequest, refreshTokenAllowed = true)

    fun request(httpRequest: HttpRequest, refreshTokenAllowed: Boolean): Promise<HttpResponse> {
        val headers = httpRequest.headers?.toMutableMap() ?: mutableMapOf()
        val accessToken = keychain.accessToken
        if (accessToken != null && isAnemoneRequest(httpRequest)) {
            headers.put("Authorization", "Bearer $accessToken")
        }
        val authHttpRequest = httpRequest.copy(headers = headers)
        val pendingPromise = PendingPromise<HttpResponse>()
        httpClient.request(authHttpRequest).then(pendingPromise::fulfill).catchError { error ->
            if (this.isAnemoneRequest(httpRequest) && this.isSessionError(error)) {
                if (refreshTokenAllowed) {
                    if (accessToken != this.keychain.accessToken) {
                        this.request(httpRequest = httpRequest, refreshTokenAllowed = false).then(pendingPromise::fulfill).catchError(pendingPromise::reject)
                    } else {
                        this.refreshToken(PendingRequest(request = httpRequest, promise = pendingPromise, error = error))
                    }
                } else {
                    this.logout?.invoke()
                    this.httpClient.request(httpRequest).then(pendingPromise::fulfill).catchError(pendingPromise::reject)
                }
            } else {
                pendingPromise.reject(error)
            }
        }
        return pendingPromise
    }

As I said before, we are including into the request a header with the authentication needed information and checking if the request fails because of a session expired error to throw the request again.

AnalyticsHttpClient => This one measures and track the response, so with this we can analyse the traffic, the error rate ...

class AnalyticsHttpClient(private val httpClient: IHttpClient,
                          private val analyticsManager: IAnalyticsManager) : IHttpClient by httpClient {

This decorator is doing something very similar to the previous one to be able to track timings and errors

LocalHttpClient => This one use as response some jsons stored into the app

class LocalHttpClient(private val context: Context) : IHttpClient {

This is another base client to read local documents in JSON to be able to serve it as a response.

And if you are still asking how we are changing the used http client in runtime, I will confess that it has nothing to do with the decorator pattern but this is how we are doing it. We have another implementation of the IHttpClient interface that receives 2 implementations of the HTTP client and depending on the endpoint url and method verb we decide which one to use.

class AppHttpClient(val appVersion: String,
                    var localHttpClient: IHttpClient,
                    var remoteHttpClient: IHttpClient)
    : IHttpClient {

    private val localEndpoints = mapOf<String, String>(
            "POST /v2/auth/login" to "/v2/auth/login",
            "GET /v2/users/me" to "/v2/users/me",
            "GET /v2/tournaments/7c0b8e6f-3693-4f48-b370-b9643a8d04fe" to "/v2/tournaments/1",
            "GET /v2/status/version_control" to "/v2/status/version_control",
            "" to ""
    )

    override fun request(httpRequest: HttpRequest): Promise<HttpResponse> {
        val headers = httpRequest.headers?.toMutableMap() ?: mutableMapOf()
        headers.put("X-Requested-With", "${BuildConfig.APPLICATION_ID}.app $appVersion")
        headers.put("User-Agent", "Android ${android.os.Build.VERSION.RELEASE}")

        val appHttpRequest = httpRequest.copy(headers = headers)
        val key = "${appHttpRequest.method.description} ${appHttpRequest.url}"

        val file = localEndpoints[key]
        return if (file != null) {
            localHttpClient.request(appHttpRequest.copy(url = file))
        } else {
            remoteHttpClient.request(appHttpRequest)
        }
    }
}

To complete this post and understand the power of the composition thanks to the decorator pattern, let’s take a quick look at how easy it is to add a new decorator component, to include a new behavior or remove one of it just by adding or removing the proper line into the instantiation. Ours one looks like this:

        val baseHttpClient = OkHttpClient(baseUrl = baseUrl, timeOut = 30)
        val authHttpClient = AuthHttpClient(
                httpClient = baseHttpClient,
                keychain = keychain
        )
        val analyticsHttpClient = AnalyticsHttpClient(
                httpClient = authHttpClient,
                analyticsManager = managerProvider.analyticsManager
        )
        val appHttpClient = AppHttpClient(
                appVersion = appVersion,
                localHttpClient = LocalHttpClient(applicationContext),
                remoteHttpClient = analyticsHttpClient
        )

Neo4j 101

Nacho González Bullón — Wed, 10 Nov 2021 17:03:35 +0000

Introduction

In Playtomic we have added a new database to the family, Neo4j. This is the first post of a series where we will try to explain what it is, how we use it and more interesting things about this database.

What is Neo4j

We have plenty of options when picking a database for our project. The range usually goes from classic relational databases (MySql or PostgreSQL) to NoSQL databases. In the latter group, we often use document-oriented databases (MongoDB, CouchDB), wide-column stores (BigTable, Cassandra), or key-value storage (Redis, Dynamo). But sometimes the data we need to store and query does not fit well in any of these types. For some of these cases, we can use Neo4j.

Neo4j is a graph database, this means that data is organised in nodes and relationships. A node fits into the entity in any other database, it has one or more labels and any number of properties. A relationship relates two given nodes, it also has a label, a direction, and any number of properties. So graph databases have entities and relationships, nothing new. But what makes them shine and be different from the rest of the databases is that they treat relationships as first-class citizens, as the entities. So, Neo4j is a perfect suit when you need to know about the relationship between your data. This commonly includes:

Fraud detection: by detecting uncommon patterns of relationships between nodes.
Real time recommendations: based on the relationships between the nodes we can provide useful recommendations to the users.
Network representations: this will allow the network managers to analyse and predict problems and design better topographies.
Identity and access management: for complex cases, a database capable of traversing relationships within milliseconds might be the way to go.

But there is more, Neo4j comes with many graph algorithms that will allow us to analyse and take the best out of our data. They are organised in six big groups:

Community detection: to evaluate how the nodes are forming communities, whether there are partitions or detect weak links in the graph.
Centrality: to determine the importance of distinct nodes in the graph.
Similarity: to evaluate how alike nodes are.
Heuristic link prediction: to predict new relationships based on the topology of the network.
Pathfinding & search: two find shortest paths between nodes or find if there are available paths between nodes.
Node embedding: to compute low-dimensional representation of the graph to be used for machine learning.

How to install and start working with Neo4j

The easiest way to install Neo4j is Neo4j Desktop. This app will allow us to create local databases, install tools and plug-ins and start working with our data. It also allows us to connect to our remote databases (cloud provided or Docker instances, for example). The app will also provide several features such as:

database information,
access to commands to load example datasets or different tutorials and exercises
storage of your favourite queries
keep a history of your queries

And of course you can run your CYPHER queries taking advantage of the syntax highlighting and getting your results in several formats such as table or graph.

Cypher: what is it and first queries

Note: to run the following examples you can run :play northwind-graph with Neo4j Desktop in your local database and you will be presented with some instructions to load the sample data.

Cypher is the query language used in Neo4j. It shines for its pattern matching syntax that produces very readable queries. This pattern matching syntax consists in drawing the path you want your query to traverse. In the patterns we can determine nodes (enclosed in parenthesis), relationships (enclosed in square brackets) and the direction of the relationship using ASCII characters (-> or <-). The simplest query we can think of is the one that returns all the nodes of a given type with a limited number of results.

MATCH(p:Product) 
RETURN p LIMIT 25

Here we say that we want to match all nodes of type Product and return them.

Now we can think of a more complex query where we can retrieve products and the category or categories they belong to.

MATCH (p:Product)-[:PART_OF]->(c:Category) 
RETURN p, c LIMIT 25

Here we want to match all nodes of type Product and the Category they are PART_OF and we define the direction of the relationship. Finally we limit the results to 25.

But of course the Cypher language allows us to filter the results. We can, for example, get the orders of a given product.

MATCH (p:Product {productName:'Mozzarella di Giovanni'})<-[:ORDERS]-(o:Order) 
RETURN p, o LIMIT 25

Here we are filtering products by its productName. Also pay attention to the direction of the arrows as now they go from right to left, because and order orders products. Of course you can write the query inverting the order of the nodes along with the direction of the arrows and in both cases the result would be exactly the same.

MATCH (o:Order)-[:ORDERS]->(p:Product {productName:'Mozzarella di Giovanni'}) 
RETURN p, o LIMIT 25

But we can go even further and query the database to get all the orders that contain Mozzarella di Giovanni and the order consists of more than 4 products.

MATCH(o:Order)-[:ORDERS]->(p:Product{productName:'Mozzarella di Giovanni'}) 
WITH apoc.node.degree(o,'ORDERS>') AS numOfProducts, p, o
WHERE numOfProducts > 4
RETURN p, o, numOfProducts LIMIT 25

Here we are taking advantage of two new concepts:

the WITH clause that allows us to pipe query parts from one part of the query to the next. In this case we are using it to count the number of products in an order to filter them.
the apoc library that provides us with tons of procedures and functions to perform different tasks. In this case we are using the apoc.node.degree function to get the number of ORDERS relationships that come out of each Order node.

These are the basics of Cypher, obviously there are much more of it and you can continue learning in their guides and documentation.

Conclusions

Neo4j is an interesting player in the databases market and it is worth a try if your project needs what it provides. We will add more posts to these series to explain some other interesting aspects of Neo4j.

Cover photo by Clint Adair on Unsplash