Migrating from CocoaPods to Tuist at Playtomic

Mohammadreza Koohkan — Wed, 08 May 2024 09:27:55 +0000

Introduction

In the constantly changing world of software development, embracing change and innovation is crucial for staying ahead. At Playtomic, we recognized the need for improving our build system and planned to make a big change by migrating our 4 years old dependency management system from "Manual project management” + CocoaPods to Tuist.

This shift was all about improving the project structure, simplifying development processes, cleaner build configurations, compatibility with Swift Packages and modern SwiftUI projects and giving the overall development experience a huge boost.

Project Structure

Our project lives inside a single git repo and it is divided into different sub projects for different layers of abstraction, following uFeatures architecture in following categories:

App: The core executable module hosting the application, integrating all features, and facilitating integration tests.
Feature Modules: Modules with code from some particular feature, they are self-contained and all dependencies from other modules are provided through IoC by the coordinators and intent providers. They are normally structured containing coordinators, views, presenters, interactors, models, etc.
Shared Modules: Commonly used modules across features, including foundational components, UI elements, and utility functions. We utilize shared modules to enhance reusability and maintainability.

For a comprehensive understanding of our project's architecture, you can refer to extensive architecture documentation by Angel Garcia, Playtomic's Shared Architecture using Swift and Kotlin.

Migration Reasons and Initiatives

In the early days of our project back in 2018, we structured our project around CocoaPods dependency manager, the most common dependency management tool back then, but nowadays it is not providing our needs.

A. Migration Reasons

CocoaPods was indeed one of the important reasons behind our migration, but it wasn't the only factor. We recognized the need for better dependency management tool as our project growth. We encountered more git conflicts and exhausted long manual setup processes for creating new modules.

Additionally, updating project settings like the iOS target version became cumbersome across different projects. Furthermore, we encountered several inconsistencies in the configurations of feature modules.

Here are some of challenges we faced:

CocoaPods compatibility issues with SwiftUI and modern swift packages
CocoaPods was breaking 100% of Xcode SwiftUI Previews
CocoaPods was causing storyboards loading very slowly (top rated project issue)
Podfile complexity and dependency maintenance
Pods folder was stored directly on the repository and not ignored by gitignore, for this reason it did lead to a higher project size
More git conflicts due to stored xcode project files on the repository

B. Migration Goals

After 4 years, in 12th October 2022, we decided to update our build system to Tuist, so we could eliminate main challenges, increase development experience and be ready for future advances.

We set out to achieve specific goals:

Have 100% of SwiftUI previews working
Have storyboards load in less than 2s
Reduce steps needed to create a new module in iOS by 50% (from 31 to 15) After finalizing migration we realized that steps reduced from 31 to 10 steps instead!

In addition to that we were targeting several areas to improve by adopting Tuist:

Structure and modularity of the project: With Tuist we could have a common structure for all of our feature modules, ensuring that they are properly configured and consistent, and save a lot of time declaring and keeping them updated.
Migration from CocoaPods to SPM/Carthage: We were struggling to make CocoaPods work properly with the microfeatures architecture, majority of iOS developers community were adopting Swift Package Manager (SPM) or Carthage as main build system, but with Tuist we saw a way to kill 2 birds with 1 stone.
Build speed improvements: The project was becoming bigger and slower to compile. We thought that Tuist could help us improve build time by better modularization and the use of remote and local caches.
Git Conflicts: With more people working in the same project was resulting in frequent git conflicts in the project files. Therefore, we wanted to move to a solution where project files would not be committed but generated, and Tuist provided that (as well as other tools).
Keeps the same build tools: in contrast with other alternatives, Tuist was not replacing the build system but integrating SPM and Carthage nicely in the existing tools.

Implementation Plan

For replacing Cocoapods with Tuist we took several steps, from creating all the internal modules in Tuist and linking them together

Key aspects of this transition included:

A. Migration Strategy

Our migration to Tuist involved a multi-step strategy, we wanted the integration to be transparent to the team with minimal disruption of the daily workflow of our 16-person team, otherwise it couldn’t be even possible to start. The strategy involved:

Transparent Integration: Create the Tuist project in a subfolder while maintaining compatibility with CocoaPods during the migration. Developers would continue using the old CocoaPods setup while we work on the migratation
Strategic Source Code Linking: Link source codes without modifying the existing codebase linked to CocoaPods. This allowed for both build systems to coexist, we had to act drastically to keep both build systems active and buildable until a strategic moment to switch build system from CocoaPods to Tuist

B. Migration Steps

Project Overview (October 2022)

Back in October 2022, we started by listing all of our internal and external dependencies to have an overview In total our project consisted of 20 modules, including 1 app module with an executable IPA target, 6 shared internal dependencies and 13 feature modules with dynamic framework targets + unit tests.

Tuist's flexibility with both SPM and Carthage made this transition from cocoapods much smoother for us, we continued by searching through each external dependency’s git repository to check if it supports Swift Package Manager (SPM) or Carthage by checking following:

Supports Swift Package Manager, if git repo contains Package.swift file.
Supports Carthage, If git repo contains an xcode project with schema to build a framework

While going through this, we found some older libraries, like outdated versions of R.swift, and some Objective-C libraries that didn't support Carthage, Our solution? We either forked and provided requirements or updated to higher versions.

Environment Setup and Integration (November 2022)

With more confidence and clarity over the project finally implementation started in early November 2022, we started by following instructions in Tuist Documentations and we kicked off migration with Tuist version 3.12.0.

After we downloaded and installed Tuist command line tool and other required tools like Carthage, we started the migration as follows:

1. Creating the Tuist Project

We began migration by initializing tuist project in a sub folder from root of the project at ~/playtomic-ios/tuist, the tuist folder placed besides Pods and Podfile.

2. Folder Restructuring

We wanted the migration to be transparent to developers, therefore we didn’t wan’t to move source code that have been already linked to CocoaPods to another folder, we wanted both CocoaPods and Tuist projects coexist even after the migration for some time.

With the new tuist project in place, we started the tuist migration at a sub folder where only tuist project manifests are located and the actual source code is at the same old location.

Root of the project playtomic-ios:

Pods
Podfile
playtomic.xcworkspace (OLD)
App → Source code only
Shared → Source code only
Features/
- Feature-A → Source code only
- rest of features… → Source code only
tuist/
- playtomic.xcworkspace (NEW generated by tuist)
- Workspace.swift
- App → Manifest only
- Shared → Manifest only
- Features/
- Feature-A → Manifest only
- rest of features… → Manifest only
- Tuist/… (New project settings and dependencies)

3. Create Project.swift manifest file for each module

Project.swift manifest represents an Xcode project. It’s used to define the targets of the project and their dependencies. For us Project.swift file represents implementation of each internal module we have.

For example the Project.swift file for a shared module like “Mozart” looks like this:



// playtomic-ios/tuist/Shared/Mozart/Project.swift

let project = Project(
  name: "Mozart",
  organizationName: "Playtomic",
  targets: [
    Target(
      name: "Mozart",
      platform: .iOS,
      product: .framework,
      bundleId: "com.playtomic.Mozart",
      // Pointing to the source code folder -> playtomic-ios/Mozart/Mozart/...
      sources: "../../../Mozart/Mozart/**/*.swift", 
      // Pointing to the resources folder where assets are placed -> playtomic-ios/Mozart/Mozart/...
      resources: "../../../Mozart/Mozart/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
      dependencies: [
        // Internal or External dependencies will be linked here 
      ]
    ),
    Target(
      name: "Mozart UnitTests",
      platform: .iOS,
      product: .unitTests,
      bundleId: "com.playtomic.Mozart.UnitTests",
      // Pointing to the Unit Tests folder source code -> playtomic-ios/Mozart/MozartTests/...
      sources: "../../../Mozart/MozartTests/**/*.swift",
      // Pointing to Unit Tests resources -> playtomic-ios/Mozart/MoartTests/...
      resources: "../../../Mozart/MozartTests/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
      dependencies: [
        .target(name: "Mozart")
        // Internal or External dependencies will be linked here
      ]
    )
  ]
)

In the migration process Project.swift files play a crucial role in defining the manifest of internal modules. These files are strategically located to be ensure a transparent migration.

As an example the playtomic-ios/tuist/Shared/Mozart/Project.swift only defines Mozart module manifest and for the source code manifest will point to the same source files utilized in our CocoaPods project playtomic-ios/Shared/Mozart/..., this approach allowed us to execute the migration while ensuring both Tuist and CocoaPods projects are working seamlessly.

4. Setting up build schemes

Schema for each target will be generated automatically by tuist, and in addition to that Tuist automatically generates an umbrella schema with every internal target listed for building and every test target listed for testing, if you don’t want the umbrella schema you can disable it by passing false to autogeneratedWorkspaceSchemes in the Workspace.swift manifest file.

5. Setting up different language and localization

We simplified this process by passing a glob pattern for resources that also searches for string tables, it efficiently collects files ending with the .strings extension, this pattern is passed to the resource arguments in Project.swift file as follows:

During Xcode project generation, any collected .strings file will be linked to the project



// playtomic-ios/tuist/App/Project.swift

Project(
  ...
  // Pointing to resources -> playtomic-ios/App/Playtomic/...
  resources: "../../../App/Playtomic/**/{*.strings,*.xcassets,*.storyboard,*.xib}",
  ...
)

6. Grouping Project.swift files under a Workspace.swift

This manifest represents an Xcode workspace. An Xcode Workspace is used to group other projects and add additional files and schemas, it has access to every added project’s targets.

Once you define Workspace.swift file within tuist manifest folder, it auto-generates a workspace file.

In our project, we were looking for flexibility in customizing project schemas and enable or disable specific unit test targets as needed, such as integration tests. The final Workspace.swift file, handling grouping projects with customized schemas, looks like this:



// playtomic-ios/tuist/Workspace.swift

import Foundation
import ProjectDescription
import ProjectDescriptionHelpers

// Assigned with Production build configuration
let productionScheme = Scheme(
  name: "Playtomic",
  shared: true,
  ...
)

// Assigned with Development build configuration
let integrationScheme = Scheme(
  name: "Playtomic (Integration)",
  shared: true,
  ...
)

let workspace = Workspace(
  name: "Playtomic",
  projects: [
    "App/Project.swift",
    "Shared/Mozart/Project.swift",
    // Other Shared modules...
    "Features/Onboarding/Project.swift",
    // Other Feature modules...
  ],
  schemes: [
    productionScheme,
    integrationScheme,
  ],
  generationOptions: Workspace.GenerationOptions.options(
    autogeneratedWorkspaceSchemes: .disabled
  )
)

7. Tuist generate and first run

With this added to the tuist manifest project, one final command line execution of the tuist generate, creates and opens the workspace, one final run through iOS Simulators and, BOOM, Success!

Preparing for the Merge: Transitioning Code into the Development Branch

Transitioning from CocoaPods to Tuist wasn't a smooth sail. Here's how we tackled development experience gap between CocoaPods and Tuist.

A. Installation of the new tools

Getting started with Tuist meant installing new tools with specific versions. To streamline this process, we detailed comprehensive instructions in our README file. Developers could easily follow these steps to install the necessary tools such as Tuist and Carthage, fetch third-party libraries, and generate the new workspace.

B. Bridging the Gap: Daily Work Differences

We wanted the migration to be transparent and improve the development experience, but the nature of Tuist and CocoaPods was so different. We used to keep third party dependencies on the project repository under the Pods folder, but with tuist we decided to not keep third party library caches of SPM and Carthage on the repository.

In addition to that, Tuist required developers to execute the tuist generate command to update and generate the .xcworkspace and .xcodeproj files. Since these files are ignored by gitignore and changes made to the workspace file didn't reflect on others' devices.

To maintain a seamless transition, we aimed to keep developers workflow similar to CocoaPods. We didn't want them to manually run the tuist generate every time they switch branches. Instead, we wanted them to open the .xcworkspace file and dive into coding.

To achieve this, we leveraged git hooks as trigger points. By adding bash scripts to post-checkout and post-merge git hooks, we ensured that every time developers interacted with the project:

Third-party dependencies were updated as needed
Workspace file was consistently updated on their repositories.
Project will be opened automatically with the updated files and third-party repositories



#!/usr/bin/env bash
echo "" 
echo "Press control ^ + C to cancel git hook execution" 
echo "" 

TUIST_MANIFEST_PATH=".../playtomic-ios/tuist" 

echo "Fetch and update third party dependencies"
tuist fetch --update --path TUIST_MANIFEST_PATH 

echo "Generate and open project workspace" 
tuist generate --path TUIST_MANIFEST_PATH

Experimental Phase: Challenges we faced and how we overcame it

After finalizing Tuist migration implementation and merging changes with the codebase in December 2022, we began the experimental phase. During this period, we built the project on several Mac machines, identifying and addressing the following issues:

1. Duplicate Static Dependencies

When we used tuist generate command for generating the Tuist project, we encountered hundreds of warnings displayed on the Terminal about duplicated dependencies:

Warning: Target 'Tournament' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'Academy' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'Location' has duplicate project dependency specified: 'GoogleSignIn'
Warning: Target 'GoogleSignIn' has been linked from target 'Onboarding', target 'Payment', target 'Paywall', etc.., it is a static product so may introduce unwanted side effects.
etc..

To address this, we changed the linking from static to dynamic frameworks. For dependencies imported by a single module, we kept them statically linked as it was not causing any duplication issue. However, for dependencies imported by multiple internal modules, especially those with transitive libraries, we switched to dynamic linking.

For example, GoogleSignIn had some libraries inside such as AppAuthCore which was also imported by other google products like Firebase, this means that our app is indirectly linked those frameworks. The configuration in our Tuist Dependencies.swift file better explains this approach:

The configuration in our Tuist Dependencies.swift file describes this approach:

2. Disabling Tuist's Automatic Code Generation

Since we were using R.swift instead of SwiftGen for generating resources with references in code, we disabled Tuist's automatic code generation tool.

If you don’t want to enable resource synthesizers, you can just pass empty array resourceSynthesizers: [] in Project initializer.



Project(
  ...,
  resourceSynthesizers: [],
  ...
)

This code snippet will disable the auto-generation of resources while still allowing to continue other magical automatic processes, like generating Info.plists and etc.

3. Reduce New Feature Module Creation Steps

There is a tool in Tuist called scaffold, it helps you bootstrap new components from .stencil templates that you defined, these templates are consistent with your project.

We did use it to create a new feature module called Level to Playtomic:



tuist scaffold mvi-module --name Level

For example a file called Presenter.stencil will be used as template, arguments passed in scaffold command like name as —name Level will be replaced by {{ name }} in the stencil file.

Therefore this method enables us to create several files at different locations to generate an entire feature module with just a single command.

Performance Metrics and Optimization

To determine the effectiveness of the migration, we explored key performance metrics, comparing the new system with the old one. Our focus was on improving the app launch time and reducing the size of the final binary as decent goals.

App Launch Time

We have analyzed app launch time by utilizing Firebase Performance tool:

Tuist IPA
- Initializing for first time: 1.08s
- AppDelegate - didFinishLaunchingWithOptions: 414ms
CocoaPods IPA
- Initializing for first time: 2.03s
- AppDelegate - didFinishLaunchingWithOptions: 671ms

These results highlight a notable enhancement in the initialization phase, we were so happy because this improvement was not our primary goal but something nice to have!

Size of Final Binary

The transition to Tuist allowed us to create cleaner and less duplicated target configurations, resulting in a more streamlined and efficient project structure. This migration contributed to a smaller binary size:

CocoaPods IPA:
- Total Size of the IPA: 107.575.065 bytes (107.6 MB)
- Payload: 167.386.848 bytes (167.4 MB)
- App on AppStore: 164.9 MB
Tuist IPA:
- Total Size of the IPA: 99.723.283 bytes (99.7 MB)
- Payload: 140.351.007 bytes (140.4 MB)
- App on AppStore: (138.7 MB)

Build Compilation Time on M1 Pro with 16GB RAM

Additionally, we conducted an in-depth analysis of the build compilation time on an M1 Pro with 16GB RAM. The results, comparing CocoaPods and Tuist, are as follows:

CocoaPods:

Xcode Clean: 2.9s
Xcode Build Playtomic DEBUG: 189s
Xcode Build Playtomic RELEASE: 196s

Tuist:

Xcode Clean: 5.9s
Xcode Build Playtomic DEBUG: 163s
Xcode Build Playtomic RELEASE: 170s

Further optimizations were achieved by leveraging Tuist's caching capabilities. With tuist cache warm that pre-compiles modules. We’ve particularly utilized tuist cache for third party libraries and we observed reduced build time:

Xcode Clean: 2.6s (2.25x Faster)
Xcode Build Playtomic DEBUG: 143s (1.13x Faster)
Xcode Build Playtomic RELEASE: 154s (1.1x Faster)

CI Improvements

In the final phase of our transition, we had to move all the scripts we used before. These scripts were crucial for different tasks:

Running Unit Tests: Essential for checking if our code changes were good to go before merging them (you know, the whole Pull Request approval thing).
Generating Testing Binaries: This helped us create versions of our app specifically for review and Quality Assurance (QA) testing.
Releasing to the AppStore: The process of actually shipping our app to the big wide world. This includes submitting all the necessary information and metadata.

We didn't just stop at improving the app build, we also fine-tuned our Continuous Integration processes on Bitrise our CI provider.

Moving from CocoaPods to Tuist wasn't just about building and testing; it was about making everything faster. We used Tuist magic for pre-compiling projects tuist cache warm and Bitrise key-based caching for third-party Swift Package Manager (SPM) and Carthage libraries source codes.

As outlined before, we used to have the Pods folder on the project repository for the same purpose of not downloading dependencies on CI machine and avoid execution of **pod** **install** on every CI job.

With Bitrise key-based caching, we fetched third-party libraries, including their source code, only once and later efficiently retrieved them using Bitrise's caching mechanism. This resembles the approach of storing the Pods folder but with the added benefits of reduced repository size and improved efficiency. Users can configure caching to create new cache archives periodically, since cache archive remains valid for seven days but resetting if updated and discarding old ones automatically.

Without key-based caching, fetching third-party dependencies for SPM and Carthage projects incurred unnecessary overhead on every CI job. Now, we fetch once, update the key based on new hashes generated from Package.resolved and Carthage.resolved files, mirroring the concept of CocoaPods' Podfile.lock CHECKSUM value. This streamlining significantly improves the efficiency of our CI pipelines.

Conclusion: Reflecting on Our Journey

Migrating from CocoaPods to Tuist was a strategic move that has significantly benefited Playtomic. Not only did we overcome the limitations posed by CocoaPods, but we also achieved a more streamlined, efficient, and future-ready development environment.

This success was accomplished by careful planning and a focus on minimizing disruption for other developers.

As we look back on this transition, we're reminded of the importance of adaptability in technology. By embracing Tuist, we've not only enhanced our current development system but also positioned ourselves for future growth and advancement.

This experience underscores our commitment to continuous improvement and our dedication to delivering the best possible experience to our users and developers.

Forem: Mohammadreza Koohkan