Forem: Anton Logvinenko

Modernizing Legacy Applications in PHP: Challenges and Approaches

Anton Logvinenko — Wed, 05 Apr 2023 17:37:24 +0000

The fact that PHP projects can run for many years is one of the most awesome strengths of this technology. But at the same time, outdated versions of PHP with unsupported frameworks and libraries cause a lot of problems for product owners. Your system may still work but have some issues preventing it from adding new features and scaling. Still, the main challenge of modernization is to update without harming the current product, which may lead to losing customers and money.

If you are among those who experience difficulties in finding PHP developers able to scale hours immediately to modernize your app, or your previous developers have not lived up to their commitments and left you with nothing right before the deadline, this article is for you. As a PHP/DevOps Group Leader at MobiDev, I have worked on dozens of PHP projects and would like to share our approach to modernizing legacy applications to help you tackle this task in the most efficient way.

6 Signs Your PHP Application Needs Modernization

First of all, let’s start with a quick diagnostic process to find out if your PHP app really needs to be modernized. There are certain signs which can tell you that something is wrong.

FROM A BUSINESS POINT OF VIEW

Unpredictable behavior of the application causes poor UX
The price of infrastructure support is too high
Security breaches cause a bad reputation
You lose your clients because your application is too slow

FROM A TECH POINT OF VIEW

Unstable work of the application: errors, outages, long downtime
Security issues caused by an outdated tech stack and/or vulnerable code
The current tech stack no longer meets your business needs. There are better technical alternatives.
Hard to add new features because of messy source code and complex business logic
There are unsupported or unmaintained libraries/technologies

If you come across the above-mentioned issues, your app really needs to be updated to stay competitive in the market. The more points that are true for you, the sooner you should start reviving your PHP project.

Why You Should Use Supported PHP Versions

At the time of writing this article, the officially supported versions of PHP are 8.0, 8.1, and 8.2. But according to statistics, about 80% of PHP users are still on PHP 7.4 and lower. PHP 8 has been out since November 2020, but only 7.9% of websites have made the switch to it:

Source

There are many reasons to stay on the supported version of PHP and keep your product updated. The most preferable option is to stay on the latest stable version since it provides you with better UX and improved application performance. Among other reasons, you should consider the following:

Security. Most of the security issues in PHP apps are related to outdated versions of PHP. Security vulnerabilities can cause data loss, data leaks, outages, etc.
Support of modern libraries. Outdated versions of PHP may cause issues with updates to newer versions of libraries/frameworks. Libraries and frameworks can also be vulnerable and sometimes you may be unable to update separate libraries without PHP update.
Integrations. To speed up development, you often need to use third-party services that can provide libraries, clients, and SDKs integrations. You may not be able to use such services until you update to newer versions of PHP. Same story with PHP frameworks. There is a higher chance to find integrations for modern frameworks than for outdated ones.
Adding new features. Newer versions of PHP provide a variety of capabilities to speed up the development of new features and improve the readability of the code. That means you can easily expand your team with new developers. Also, many engineers prefer to work with modern tech stack rather than with legacy applications. This is also a reason to think about regular updates and stay on a stable modern version.
Performance. Core PHP developers are constantly working on improving PHP performance. For example, JIT-compilation has been added since PHP 8, which opened the door for enhanced data processing.

Challenges of Migrating Legacy PHP Applications (To Migrate or Not to Migrate?)

Modernization of outdated code is crucial, but it comes with some risks. Migrating a legacy PHP application presents challenges for both a development team and the product owner. Let’s imagine how developers and product owners see the same situation:

And they are both right. They have to find some kind of compromise so that the project effort is justified. In most cases, it is possible to support current applications and do updates step by step while also adding new features. The development speed would not be so great in this case though.

The main point is that developers need to do extra steps before starting the migration process or adding new features such as:

understand legacy logic laid by previous developers
do refactoring in case of dirty code
prepare code for adding new features

Only after this work is done will PHP developers be able to add new functionality. Product owners will benefit from such work in the long run. After some time, the speed of the app will increase, and the stability and security will also grow. This is a kind of investment in the future of your application.

Two Approaches to PHP App Modernization

When it comes to PHP app modernization there is one difficult question: should you continue using the existing application after updates or should you rewrite everything from scratch on a modern tech stack? Answering this question gives us two main approaches to PHP app updates.

APPROACH 1. REWRITE EVERYTHING FROM SCRATCH

Rewriting your app from scratch is the most guaranteed way to achieve quality and stability of the application. But a lot depends on the state of the current software product. Sometimes it is really better to rewrite everything from scratch because it will take less time than fixing and updating legacy code.

In this case, a development team can start working on a new application with modern technologies using the old app as a reference to business logic. The newest versions of PHP or some alternatives like Node.js or Ruby can be used here. This will provide a better understanding of the business needs, taking into account previous user experience, parts of the application that work well for users and parts that can be improved. As a result of this work, you will get a beautiful modern application based on your legacy one.

Let me answer the questions that you may have at this stage:

Is it an expensive approach? Yes. Such work will consume a lot of time and money.
Should I do a feature freeze? Probably, yes. You would pay twice if you create new features on a legacy application and after that you need to create the same features on a modern application again.

APPROACH 2. STAY WITH A CURRENT APPLICATION

The previous approach will not work for most businesses. In most cases, a business needs new features to live. Using the current application is possible in cases when the approximate time for an upgrade will take less time than rewriting everything from scratch. A lot depends on the current technology stack and the technical state of the system.

What parts are most critical to update? Are there any security vulnerabilities? Are there any libraries that are not supported anymore?

Only after analysis, can technical experts suggest an upgrade plan. The most valuable activities will be done first. During the upgrade process, PHP developers can add new features to the current application as well.

What issues can bring such an approach?

Potential non-obvious bugs. Fixing one bug may bring on another bug because of the level of complexity and quality of the legacy code.
Difficulties with compatibility. You can update the version of PHP and some parts will work well but some parts will not work because of breaking changes.
Complex logic. If you have a complex application with a huge code base, then upgrades should be done step by step. That means there would be part of logic with updates and part of logic without updates. These may bring some temporary dirty hacks, extra levels of complexity, and cause some difficulties in the onboarding of new developers who are not familiar with the project.

Workflow for PHP App Modernization

Regardless of which of the above-mentioned approaches you choose, the process of PHP app modernization will look almost the same. Based on our experience with many legacy applications, I can say that this workflow is the most efficient for both the product owner and the development team. It consists of 10 steps:

Step 1. The client sends a request for a project review to a technical expert.

Step 2. The technical expert analyzes the project and discusses with the client the possible ways of modernization and estimated timelines.

Step 3. The client requests a development team that will start working on a selected approach.

Step 4. Containerize the application. Docker will help with this. This allows for adding an extra isolation layer for the parts of the app. With Docker, we can easily manipulate any version of the service that we need for application functionality.

Step 5. Deploy everything in a sandbox environment for testing and development.

Step 6. Involve the QA engineer. Start testing the application to find the most critical issues.

Step 7. Prioritize the list of issues and start working on them.

Step 8. Work on updates and fixes iteration by iteration.

Step 9. (optional). After the stabilization of the application, new features can be added as a priority to updates.

Step 10. Your software is upgraded and ready to go.

Our Case Studies for PHP Legacy Software Modernization

Let’s go through some cases from our experience that will illustrate the solution of business challenges with the modernization of legacy PHP applications. These cases include comments from project developers that will help you better understand why a specific solution was chosen.

CASE STUDY #1. UPDATING LEGACY LARAVEL-BASED SUBSCRIPTION PLATFORM

The client came to us with a request for the modernization of the PHP platform in a critically short time frame. Since the previous developers of the project failed and didn’t do what they promised, the client needed a reliable team that would take up the challenge and meet the deadlines.

Main challenges:

Build the back end for the existing front end
Find and fix issues preventing the system from working correctly
Prepare the product for adding new features

Solution

The initial request was to rebuild the platform, but after code assessment and analysis of the client’s business needs, our PHP development team suggested improving the existing system instead of rewriting the product from scratch. Such a decision was influenced by the following factors:

The platform was already in production and had an active user base. It was crucial for the business to keep the product running.
Our client had a limited budget and timelines to make the system stable and prepare it for adding new features.

1.Build the back end for the existing front end

When we delved into the project, we found out that its deployment, including the assembly of the front end, took place directly on the server, which required routine manual work, and significantly increased the time and cost required for debugging.
We’ve integrated Ansible, a software provisioning and configuration management system, and added rules to automatically build, deploy and update the software. This helped the client to save on servers since the assembly of the front end on the server required quite large capacities, which was unreasonable for everyday tasks.

2.Find and fix issues preventing the system from working correctly

The fact that the platform was written on an outdated version of the Lavarel framework led to unstable operation of the entire system (unpredictable errors, inability to use stable modern libraries, slow response from the server, etc.). Moreover, the PHP version also needed an update. Bugs in the framework plus PHP bugs were superimposed on project code bugs that created a danger to the security of customer data. So our PHP team decided to update Laravel to 5.5 LTS and PHP version to 7.0 on all servers as the latest versions at that time. This made it possible to eliminate system vulnerabilities and fix critical bugs.

3.Prepare the product for adding new features

The poor quality of the code was a real challenge for the team, as a large part of the code needed refactoring. Plus, the project had no documentation, which complicated the process. Also, in order to use a third-party payment system, it was necessary to update the SDK since its version conflicted with the modules used by the project.

So code refactoring and regression work was done. After that, we were able to continue adding new features with modern technologies and an up-to-date version of our built-in and external modules. We created tech documentation that will allow the client to significantly save on onboarding new developers and adding new functionality in the future.

"It was really challenging to update the framework version by version: 5.2 -> 5.3 -> 5.4 -> 5.5. because all of them were not backward compatible and each subsequent version radically changed either the architecture or the logic or even the names of the framework functions.
It was also important for the client to implement full GDPR compliance, at that time there was little information about it and it required additional research from the team. Overcoming these challenges assured the client that we have great expertise in PHP app development and they still trust us to introduce new features into the platform."
Oleh Sichevskyi
PHP Team Leader

CASE STUDY #2. MODERNIZING A LEGACY APP ON CODEIGNITER FRAMEWORK

Our communication with the customer started with the request to solve problems with the current API for CRM and mobile app with legacy code (Codeigniter 3, PHP 7.3). The client had tight deadlines: the software had to be updated within a month.

It is difficult to complete refactoring of the entire system in such a short period of time, so we decided to approach the improvement of the code very selectively and gradually. We drew up a plan with the most critical issues and started working on it.

Main challenges:

Update the highly outdated version of the framework to improve system performance
Improve code quality under tight deadlines
Configuring the mechanism of code delivery to the server
Build new scaling infrastructure

Solution

1.Update the highly outdated version of the framework

As the Codeigniter version of the framework was no longer supported, our PHP developers suggested gradually migrating the entire application to a newer version. The team created a plan with prioritization and division of all functionality into modules. Each module is a fully isolated functionality in a new project with up-to-date versions of dependencies and environments.

2.Improve code quality under tight deadlines

The poor quality of the code caused additional problems. We had to spend more time studying the logic of each method, even for the smallest changes. At the same time, there was no way to fix everything at once due to the large size of the project and the large amount of functionality. Each change required a full understanding of the requirements for a specific feature, which could take a very long time and thus delay the release date.The solution was to refactor only those places whose functionality we were currently working on and proceed to other parts gradually. It made it possible to meet the deadline and save the client’s money. Within six months, about 80 percent of the code was already covered.

3.Configuring the mechanism of code delivery to the server

At the start of the project, we had only one production environment available. Making edits to the code was very time-consuming and involved the risk of human error, which put the seamless operation of the application under attack.To mitigate these risks we needed to prepare a separate environment for testing and development. This was done with an automated code delivery process with Docker Stack + Ansible + Jenkins CI/CD, first for the test environment and then for the production environment.

4.Building new scalable infrastructure

The old system was a monolithic server with a database and a folder to store all uploaded files. This architecture makes server load scaling very difficult and is obsolete in 2023, mostly suitable only for CMS out-of-the-box solutions but not for frameworks. Deploying new code to the server was only possible via FTP and had to be done manually.

To resolve these issues, first of all we moved the database and file storage from the server to remote services (AWS RDS, AWS S3). Third-party services are more stable and already have a backup mechanism that is more secure and helps to prevent data leakage.

The next step was to wrap architecture with Docker and isolate each service (NGINX, PHP and Socket) into containers. This made it possible to configure a stable automatic code delivery to the server (CI/CD) and also added opportunities for the optimization of the server resources.

Finally, we gradually moved the code to a newer framework (Codeigniter 4). For this purpose, a separate Docker container with a newer version of PHP was prepared.

Our team also created API Documentation (Postman Collection) with sample requests/responses from the server to help front-end and back-end developers easily communicate about the nuances of the API.

"Initially, the project only had a month’s work to do. It was difficult to fit all the necessary improvements into such a limited period of time, and it was a really interesting challenge for us to choose what was the most critical.
It’s important to find ways to communicate the need for some code changes to the customer, even if the end user won’t see a clear benefit from it, but it’s crucial for functionality in the future. You need to have the trust of the customer and provide transparency in your decisions.
As a result of a month’s work, the client decided to continue cooperation with our team, and it made it possible to improve the code according to the plan prepared earlier without stopping the delivery of new functionality."
Georgii Mirgorod
PHP Team Leader

CASE STUDY #3. REWRITING A LEGACY PHP COMMUNICATION APPLICATION FROM SCRATCH

The client’s request was to add an iOS app to the existing website, as well as improve the overall stability and maintainability of the system. In the process of the initial investigation, our team discovered that it was quite difficult to do this with the current architecture, which presented us with the following challenges:

Bad app performance because of poor source code organization and quality
Data split across several data storages made the support of the system complicated and time-consuming
Multiple bugs and errors prevented the system from adding new features

Solution

Our PHP experts suggested rewriting the app from scratch since:

Working with the existing code base and infrastructure would take more time than rewriting the application
The client was not limited in time and was focused on maximum improvement
The product was not used actively in the production environment

1.Fixing bad app performance because of poor source code organization

During the analysis stage, our team discovered that the current application architecture was quite difficult to maintain and it did not allow for the easy addition of an iOS API. The project included pieces of code written in different languages and pieces of data stored on different platforms, such as MySQL, Neo4j and Google Cloud storage & Firebase. The logic was mostly client-side, with the notifications system being the only server-side asset.

So we set about creating a new version from scratch, moving the logic to the server side and consolidating the existing data into a single database. The existing legacy project helped us define the ultimate vision for the new app. We chose a new technology stack with a modern PHP framework and created an effective and scalable architecture.

The iOS app and its API were the main development focus and in the later stages, we recreated the existing website as a SPA using the same API. This approach was intended to provide easier iOS app integration, improve stability and performance, and provide opportunities for potential future improvements.

2.Consolidating data into a single database

The existing MySQL database was taken as the basis of development, as it contained the most data; however, we made improvements to its structure and indexes to provide better performance and data consistency. We also added a migration tool. Since then, the old version of the project has not been used for reference; instead, the functionality was rebuilt based on the customers’ current vision and ideas.

The development and testing servers were created on Amazon Cloud. We also added an admin panel to simplify user data management and an automatic documentation page for the API that updates in real-time and provides the ability to test API functionality for client-side developers.

3.Adding new features to the system

After the successful launch, we continued to improve the project. A cache was added and the database was moved to RDS to improve performance. Websocket notification was implemented to provide real-time chat. We’ve also updated the notification system to be easily configurable and include email and SMS notifications. The admin panel receives constant quality-of-life updates, including the ability to communicate directly with users through the associated account.

"After about a year of cooperation, the new project branched off from the existing one, taking its data as a basis. This new project was later successfully migrated to another cloud service due to a number of legal and geographic reasons, and now it’s in constant development alongside the original one.
Currently, both projects are still actively used and allow users around the world to connect and share their experiences."
Dmytro Lisovin
PHP Developer

Why PHP is Still Good in 2023

If you have concerns about whether you should stay with PHP and opt for modernization or better migrate to another tech stack, let me dispel your doubts and tell you why PHP is still in demand in 2023.

Being an open-source PHP has a great development community that constantly improves this programming language. New frameworks and tools open the door for new opportunities and help PHP stay competitive in the market. Low cost for development and maintenance with very high performance and reliability made it the main choice for 77.5% of all the websites, according to W3Tech.

PHP is used by both enterprises and startups. It is a great option for long-term projects that want to grow since PHP is scalable by design. Also, it has a good loading speed, even on slow Internet connections. Since PHP 5.6, its performance has increased almost two times over.

In our experience, PHP is a great tool for the following types of projects:

Server-side rendering (SSR) websites
RESTful web services (API)
Social applications that require scalable architecture and specific features like likes, dislikes, messaging, comments
Server-less applications
Complex systems with great functionality like ERP and CRM platforms
E-commerce, marketplace, booking platforms
Mobile Device Management (MDM) systems
Healthcare projects that focus on security

Of course, PHP is not a silver bullet and it has certain drawbacks. For example, it’s not the best choice for CPU-consuming tasks. If you need to run heavy tasks with the best possible performance then it’s better to use other compiled languages like Golang, C++, or C.

How to Scale Software Product Development at Fast Growing Tech Startups

Anton Logvinenko — Sun, 17 Jul 2022 19:59:57 +0000

The correct choice of infrastructure for a tech-based startup may help to cope with unpredictable user base growth, volatile demand, and server load. In other words, it’s crucial for scaling. And you should take a hard look at such infrastructure options as Firebase, Lambda on PHP, Kubernetes, and Docker Swarm to understand how to scale a software product correctly and timely, in alignment with business flow. Therefore, our journey starts from dwelling into types of startup businesses.

C2C startups – BlaBlaCar, eBay, Amazon, and others – are characterized by linear load change, but there are certain nuances. The number of users, who interact while trading or agreeing on the trips, is changing not smoothly according to the geographical area, day, and time. We can observe peak loads for C2C startups in the evening and during the weekend. And this is the main point: in such times, C2C services should be scaling dynamically, with high speed that corresponds to the unforeseeable workload. AWS Lambda and Kubernetes are suitable for such C2C products.

B2C businesses work in a wide range of areas, from consumer products to beauty products, where workload is heavily influenced by marketing activity. And that’s why B2C companies can plan this activity and load. The daytime use of most B2C services isn’t as intense, except for food delivery. In general, we can see enhanced predictability compared to C2C startups. However, in terms of predictability, the most winning are B2B products, which we’ll discover in a moment.

B2B startups involve the projected growth and reduction of the load. It’s the simplest option for optimizing scaling costs. Here you have a clear vision of what and how the business will sell, so it’s possible to opt for Docker Swarm, as an example.

Foreseeing the Scaling Before MVP Development

Regardless of the type of business that is mentioned above, tech-based startups are premised on the idea of scaling realized as soon as a launched product reaches the necessary capacity. Scaling foresees process automation without adding extra costs to the business, and it may be tough. That’s why startup founders should take into account important factors even before embarking on the MVP development.

If you neglect these factors while choosing technologies for startups and working on the interface, design, and architecture, then scaling of the MVP will be highly implausible. Hence, it seems reasonable to consider important details for creating a product that will take its place in the market under the sun.

The first nuance is premature scaling. RewardMe can serve as an example here. It was an intelligent CRM solution for local commerce that provided merchants with customer insights. RewardMe raised $1.1M in the first round but couldn’t provide customers with features that were available in competitors’ products. Founders decided to scale, though their product wasn’t ready for it and the product failed.

The second nuance that leads to failure is misreading market demand. According to CBInsights, 35% of startups face a situation where there is no need for their product.

Are premature scaling and misreading market demand the only reasons for a startup’s failure? Obviously no, and this is the third nuance. A lot of startups have no clear vision of how to earn money and foresee revenue. They may use a dynamic scaling that implies high costs.

And as an example, we can consider a C2C startup that revolves around the idea of building a social network and earning money from paid features. A few users are interested in this product because they have Instagram and Facebook for free and don’t want to pay for such platforms. The intention to scale and combine this C2C product or a similar startup with income generation becomes fatal.

In a business model, it is always important to consider that free functionality is also scalable, as well as the cost of it, so it is important to forecast revenue and expense during the growth of the load. Startups with properly structured business processes can scale and thrive, overcoming tech limitations and challenges we are going to touch further.

When considering optimal conditions for startup scaling, you should take into account the potential load level. Since additional pressure may be an obstacle for onboarding millions of users, opt for a scalable interface and infrastructure.

Building a Scalable Interface

The UI/UX design phase is essential for a startup. To be scalable, the core of your product can be developed through the design thinking approach. It consists of five stages during implementation during which the team emphasizes and defines users’ needs, brainstorms and validates ideas, creates design solutions, and tests them.

When going through these five stages, experienced designers should foresee possible future changes and apply different approaches to tackle the product’s growth. For instance, a scalable interface can be based on reusable components. If such components weren’t foreseen, it would be difficult to maintain the speed of development and create a consistent and intuitive UI. Just imagine an ordinary situation. A startup requires adding a few menu items. Reusable components are of value in this case, yet without a scalable infrastructure their significance for scaling is doubtful.

Building a Scalable Infrastructure

The wrong infrastructure accelerates the downward trajectory of a startup. To create a scalable solution, address all infrastructure options.

1. DOCKER SWARM

One of the most easy-to-use container orchestration tools. It is open-source and supported by American technology company Docker, Inc. As a mode, it’s suitable for the deployment of applications to production and may serve as an alternative to Kubernetes and Mesos. Docker Swarm is an optimal solution for a beta version of a startup with a limited number of users. However, unlike Kubernetes, it cannot scale automatically according to the load. So, Docker Swarm is usually used only for the project kick-off or an MVP, where servers are added manually. In case of unpredictable growth, you can switch from Docker Swarm to Kubernetes, thus avoiding failure risks related to the manual scaling and a great rise in load.

2. KUBERNETES

A system that may be suitable for startups that need to deploy, manage, and scale a containerized app. With Kubernetes, containers are grouped into services. This approach simplifies discovery and management. It may be perceived as a convenient tool for scaling, which doesn’t require a massive expansion of the ops team. Kubernetes is worth considering if the product has an understandable or predictable growth and workload from users and can require autoscaling and a competent use of billing in the future. It’s easy to add or remove servers, while auto-scaling allows for changing the quantity of running containers.

3. AWS LAMBDA

A serverless computing service that enables running code for virtually any app or backend service without the need to bother with servers. You may trigger Lambda from AWS services and SaaS apps, paying only for what is used. As was previously mentioned, AWS Lambda is okay in the beginning, when the number of users varies dramatically, from zero to any required value. Although, switching from AWS Lambda to another solution can be difficult and depends on the programming language. For instance, it would be easy for Serverless PHP, where PHP frameworks like Laravel or Symfony can be used.

Please take a look at Lambda Symphony and Lambda Laravel REST API demos made by George Mirgorod, PhP Team Leader.

Here you can get acquainted with serverless PHP applications on AWS Lambda and paths to be taken to prepare serverless.yml, deploy, run migrations and fixtures, and test REST API.

4. FIREBASE

A platform or a serverless solution for creating mobile and web apps. Should you want to test an idea and reduce time to market, begin with Firebase. It’s a suitable platform with lots of extensions, integrations, and ready-to-use functionality.

Should you want to test an idea and reduce time to market, begin with Firebase. Firebase allows you to quickly investigate the market and get funding, but it has limitations in terms of functionality. Moreover, switching from Firebase to something else can be problematic, and sometimes you may need to code from scratch, but it really depends on the business goals.

Our team has successfully used Firebase to develop a platform for the mutual support that people give to each other https://dopomoga.co.ua.

And since Firebase is a Google product, I can mention that it has a lot of advantages, namely proficient analytics dashboards, free hosting on Google cloud, multiple authentication methods, etc. That’s why Firebase is listed among the top software development trends for startups.

CI & CD to Speed Up Delivery Process

Before we even think about scaling, we should introduce process automation, simplifying repetitive procedures. Among such procedures are code deployment, testing, and release automation.

The main idea is to set up a system in a way that enables you to modify existing features or launch a new version of the product by speeding up the release cycle. Ideally, it should be possible to introduce changes in hours and enable an engineering team to spend less time on the preparation, release, manual testing, code deployment, and so on. Through this approach, large projects can save significant time and costs. And since we have already begun to speak about cost efficiencies, it may be instructive to examine the case study.

Firebase is easily combined with Flutter, serving the well-balanced tech stack of MVP Lite. This Agile-like approach enables the implementation of the needed features in the short term since there is no need for the final design and full-fledged testing. MVP Lite foresees the development of 80% of the key functionality, which reduces the overall development costs (savings are estimated at 30–80%).

Note: you can build an MVP or POC for a startup using no-code platforms. For example, Webflow, Bubble, Adalo, or Zapier. It’s faster and cheaper, but scaling of such startups is too complicated and unreasonably expensive.

Scaling Startups Down

Should you ever get to visit the heart of San Francisco, it’s a good chance to paint the city red. And when you investigate bars downtown, you’d definitely find yourself in a crowded one, somewhere at Polk st. Every second bar there runs the venue management & POS system developed by MobiDev.

Before 2020, operating of the product was based on the scheme where each bar or nightclub had a dedicated server and paid its subscription. The architecture design made it possible to quickly deploy the system at venues. It was even able to automatically deliver unique features to certain locations. But when the COVID-19 pandemic began, downscaling was required. To avoid overpayment and reduce costs on infrastructure while scaling down, we went ahead and considered adopting Kubernetes or a similar solution.

From this case, we can see that final business requirements for startups must be foreseen in terms of different operating scenarios. This complete transparency is crucial and simplifies scaling up and down without excessive costs.

Image credit

Mobile Device Management Guide 2022: AOSP for custom MDM

Anton Logvinenko — Mon, 04 Apr 2022 14:25:20 +0000

Mobile Device Management or MDM is an administrative area that deals with security and monitoring of corporate mobile devices. MDM software suggests methods to quickly deploy, integrate, and monitor a network of certain smartphones or tablets.

One of the primary concerns of MDM is security. Since corporate mobile devices can access critical business data, they can threaten enterprise databases. Administrating devices through MDM provides the means to distribute software packages, set permissions, and optimize device functionality. But, ideally, MDM has to provide a way to oversee mobile devices as easily as desktop computers.

There are MDM systems for all mobile operating systems, including cross-platform ones. But today we’ll focus on Android as the only operating system that offers fully-fledged customization in terms of MDM. With the biggest market share and open-source nature, let’s look at how we can approach building mobile device management for Android devices.

Mobile device management features and capabilities

Before analyzing the actual Android solutions, we need to clearly understand what MDM is capable of. Mobile device management systems provide quite similar capabilities across the market. Some of the most common functions are:

Enrollment — a procedure that entails installing MDM on a mobile device. The majority of MDM software supports bulk enrollment to install software packages on multiple devices at once. An Over-the-air (OTA) enrollment suggests distribution and installation of MDM through a dedicated web page or app.
Profile management — once we install the MDM package, we can now assign working profiles for the device users.
Policy management — a policy is a bunch of rules or permissions set for a given device. By providing policies, we may lock or unlock some of the software or hardware functions, denote rules for accessing corporate data, etc.
Device administration and troubleshooting — further, all the policies can be updated remotely. This part of MDM functionality implements monitoring of the device and troubleshooting.
Device location tracking – detecting device location via GPS.
Remote wiping — once we enroll MDM on the device, all the corporate data accessed through it will be stored on a protected profile. This data and the MDM itself can be wiped to factory settings remotely to protect any data leakage from the stolen, lost, or compromised device.

MDM is a complex solution. However, it consists of modular small parts that after closer inspection are actually not that hard to implement as long as a skillful team is involved. Generally, these parts can be divided into an admin panel which includes a device policy controller (DPC), and middleware to orchestrate enrolled devices. By middleware we mean an API interface that performs all the policy updates, and transfers data between the MDM server and smartphones.

So, MDM solutions basically provide management of such features as WiFi, Bluetooth, NFC, USB file transfer, location tracking, phone, SMS, or just simply allowing users to access some settings on devices that otherwise wouldn’t be available. If any additional functionality is required, the best thing to do is to extend the existing API by adding methods, managers and services. It is done with certain policies on the backend, which would be later pushed to devices. Then, a system service is created as a simple way of sharing these settings across devices.

Based on our experience, such systems might scale to thousands of devices and require numerous custom policies. The majority of the security requirements are covered by the available MDM solutions on the market. However, some systems may require enhanced security and independence of operating system providers. This will impose a challenge to build a custom Android MDM where any policy can be deployed on the device.

So first, let’s look at the standard solutions for Android MDM, and then talk about the customizable options.

Android Management API: out-of-the-box MDM

Android Management API is a managed solution for building mobile device management systems for Android. Google ships the whole package of MDM software, including backend, based on their Cloud Platform, a device policy controller, and user interface to administrate corporate devices out of the box. All of these components become available after a few steps of registration we’ll describe later.

The Android API itself is required to build your own MDM solution and create custom policies. Currently, the registration for new solutions is open, except for creating a custom DPC. Which means, it is possible to develop an MDM platform based on the Android management API, but you won’t be able to apply custom policies.

To use a managed MDM, you’ll have to create a dedicated Google account to log into Google Workspaces. The account cannot be associated with the existing enterprise account. Google provides device policies for over 80+ types of Android smartphones, except Samsung KNOX. There are three ways we can implement MDM with Android management API:

Work profile. A dedicated account on a device that stores and transfers corporate data without affecting personal data. MDM policies are also applied to the profile data only.
Managed device, a smartphone or tablet enrolled with MDM.
Dedicated device, a separate device used with restricted functionality. For example, this can be a tablet used as a bulletin board.

The system allows enrolling target devices over-the-air, which means no cables are required. You can use a wireless connection to enroll and manage mobile devices on Android.

The list of available policies includes nearly all of the native functionality for Android devices. While the existing policies are not customizable, the ease of deployment and no need for Android development outweigh this flaw. So now let’s look at how to approach Google MDM.

HOW TO SET UP ANDROID MANAGEMENT API MDM

The registration procedure to Google’s MDM platform imposes several requirements:

Android device version 6.0 or higher
Access to Google Cloud Platform
Enabled Android management API in Google Cloud Platform

After all the requirements are completed, there are two ways to enroll your devices. The easiest one is a quickstart procedure suggested by Google. This entails a few steps of creating a Google Cloud project, generating a QR code and enrolling the device with a default policy. Note that a device can only have a single policy at a time.

The more advanced way to complete setup is meant for creating your own MDM solution based on Google’s API. For a full setup procedure please check the article PDF, where all the corresponding steps are described.

Now, let’s move on and sum up the strengths and weaknesses of Google’s MDM.

BENEFITS AND LIMITATIONS OF ANDROID MANAGEMENT API

In terms of a mobile device management platform, Google suggests a solid base either for deploying a ready-made ecosystem, or using its API to build your own one. So here we’ll highlight some significant factors in choosing the solution type:

Out-of-the-box solution. Google’s MDM requires just a few steps to enroll your first devices. After that, the full range of capabilities for device management becomes available for the users. It doesn’t require any additional Android development, so we can say it’s a low-code solution. Since minimal manipulations on the server side are required.

Large list of supported devices. Usually, different types of Android devices can be used in the organization. Integrating different devices and writing custom enrollment methods might be a serious pain, unless your company uses a single type of Android device. Android management API solves this problem by providing support for over 80 Android devices by default.

DPC is updated with Android. A device policy controller is a module that sits on your target device and implements policies sent from a server. This bit of software is operating system dependent, meaning the updates in the OS will require DPC to be rewritten.

Default DPC from Google is updated with the Android version, since it’s a managed solution. So the only possible flaw here is that Google will abandon some of the older Android versions in the future.

This becomes beneficial if you lack flexibility within Google’s solution or want another set of policies. Or, you can use Android management API as a groundwork for your own solution. All of the developed solutions using Google’s DPC have to be registered and certified by Google. So here, you’ll need an experienced engineering team that has expertise in mobile application development and mobile device management systems specifically.

Besides these advantages, there are also two important factors you want to keep in mind.

No customization options. In terms of Google’s managed platform, there is no way you can customize the existing software. Building your own solution based on Android management API allows you to customize the backend part, but the set of policies remains unchanged.

Vendor lock-in. Another problem is security. Since the backend part of a system runs on Google Cloud servers, all of your corporate data will pass through it. This doesn’t necessarily mean your sensitive data is compromised. However, some organizations want to protect their information and keep it in house.

To overcome these limitations, another option for Android mobile device management can be approached.

Android Open Source Project (AOSP): custom mobile device management

Android Open Source Project or AOSP is an open source software stack for a wide range of mobile devices and a corresponding open source project led by Google. It can be used to create your own custom variants of the Android OC and mobile applications, and connect it with your custom back-end device management platform. AOSP provides a number of benefits for custom Android development:

Active open-source solution. Some features that might be necessary are either already implemented, some might be available in the nearest future. As soon as a security patch is committed to AOSP, it can be applied and pushed to users.

Full control over product life cycle. The product owner decides when to deliver a new feature or security update. Product needs can be prioritized, instead of waiting for something that might never be rolled out by vendors, who naturally prioritize their own needs.

Customization at any level. Linux kernel contains all the essential hardware drivers like camera, keypad, display, and others. Above it, there is a set of libraries, including an open-source Web browser engine WebKit, libc library, SQLite database, which is a useful repository for storage and sharing of application data, libraries to play and record audio and video, SSL libraries responsible for Internet security, and others.

Then, the Android Framework layer provides many higher-level services to applications in the form of Java classes.

The application level is where Android developers usually work. Android applications extend the core Android operating system. There are two primary sources for applications:

Pre-Installed Applications: Android has a set of pre-installed applications, including phone, email, calendar, web browser, and contacts. These function as user applications as well as providers of key device capabilities that can be accessed by other applications. Pre-installed applications may be a part of the open source Android platform, or they may be developed by an OEM for a specific device.
User-Installed Applications: Android provides an open development environment supporting any third-party applications. Our device administration app would fall into this category, and our main work will be done primarily at the framework and application levels. Applications at these layers are written with Java, so a regular Android team will feel comfortable working with them.

As a solution for businesses that use mobile device management, AOSP helps to resolve concerns for security of functionality limitations and ownership over the product. As it’s possible to create an Android stack that will provide any features and means of device management that they might require. So let’s quickly enlist the pros and cons of this approach.

BENEFITS AND LIMITATIONS OF CUSTOM MDM

Flexibility. As we mentioned, custom development allows you to choose any architecture and solution type you want for mobile device management. This means you can deploy a custom backend with policies of your choice. You can provide any enrollment methods, security management, and reporting.

Independence of data collection. While AOSP is an open-source project led by Google, the solution based on it doesn’t depend on their infrastructure. This means, your mobile device management system will be secured from data collection by third-party organizations.

Operating system customization. Additionally, the operating system itself can be customized to bring extra functionality and enhance security of the system. The only limitation here is hardware capabilities of the target device.

Complexity. The approach of customizing the OS, creating your own applications and back-end management platform is a complex and time-consuming project. Moreover, customizing Android OC core requires not only an experienced and skilled development team, but extensive quality assurance as well.

How to choose the best solution?

Given the described options, how do you choose? Here is a quick run through the key points:

Google mobile device management platform is capable of closing the majority of needs for mobile device management systems. The only tangible limitation here is the closed registration for developing custom device policy controllers, so no custom policies can be implemented. Another solution is using Google Cloud platform as your backend, which can be a security, or architectural concern for some organizations.

Using Android management API as a basis for your system allows some customization, but still relies on Google’s infrastructure. So approaching AOSP-based MDM might be the best choice if you require a high degree of customization. Based on our experience, working with AOSP might be a time-consuming task. But at the end of the day, it fulfills project objectives at the same level as native Android MDM solutions.

Mobile Device Management solutions for iOS

It’s a rare case for the enterprise to use devices only on Android. So the first thing that pops up is the question, what about iOS? Apple devices have a built-in framework for enrolling and managing iOS smartphones and tablets. The capabilities are similar to Android management API, including enrollment options, and management flexibility. Because it’s a huge topic, we’re going to describe MDM for iOS in a dedicated article. Please stay tuned to learn about the implementation of the iOS framework in future material or contact us if you’ve got interested in Android MDM solution.