Forem: Emre MERT

Migrating from Monolith to Microservices for Faster Feature Implementation

Emre MERT — Thu, 28 Dec 2023 10:11:09 +0000

Migrating for scalability? No, this time it’s for faster feature implementation. Does it work? No pain, no gain, right? But what if there’s much pain and so little gain? Is this paying off technical debt, an investment for the future, or just hopping on the hype train?

TL;DR: What feature? You knew it!

We have a project that integrates with other systems. These integrations involve numerous business rules that change constantly. No one remembers these business rules. A developer needs to read the documentation, coordinate meetings, implement a new feature, and fix new bugs. It’s a trial-and-error process every time. Let’s create a microservice and assign responsibility to a new team. A microservice for a team — what a great idea! Let’s implement the microservices architecture first, and we’ll build teams later. We must sacrifice the current team for the greater good of the new teams!

Let the meetings begin! Everyone has ideas about the design of the new microservices architecture. Everyone is excited and wants to talk. Developers who have worked with microservices architecture before are our knights — they are on the front lines, fighting the monster! Monster? Microservices are a monster? Come on, that can’t be true!

We needed some mediator microservices, workers, queue infrastructure, and some state machines. We converted some libraries to NuGet packages. So much extra work for migration. But what about features?

“Network. What? I’m a software developer,” said nobody. “Network? Not a problem, bro, bring it on.” And the monster is hunting us. Connectivity problems are the new bugs. Increased latency is another story.

Log aggregation. Oh, correlation, my new friend. There’s a new bug. Where? It’s somewhere in the logs, if it’s logged. And if it’s logged correctly. And with enough data. It should be there. Oh my God, please help me! Oh, I found it. No, it’s another thing. Is this really it? This one. Hmm. Let’s see where it starts, goes, and stops. You found it. Good. Is it different from your assumption? Good. You need to suffer more. Breakpoint. You can’t hit the breakpoint.

You fixed it. Built it. Deployed it. Deployed the new version? Oh, let’s check it. We have a cool CI/CD. Don’t worry. What? A deployment error?

Deployment to production, adding a new microservice, staging a new environment, and simulating production locally are all fun. Just saying. Deploying to different cloud services is even more fun.

Test it? You can’t give your local machine to testers. They can sense if you’re weak. Stay strong! Let’s build a container and deploy it to the test stage. The test stage is running, right? If not, testers will bury you alive.

Have you ever heard of different databases? Yes, bro, it sounds cool. Okay. What about different data sources? Do you mean different databases? No. Think about configuration file and other folder paths. If you survive that, think about different event sources. What triggers what? What if a microservice retries something again and again? What if a microservice is in panic mode and another microservice tries to kill others? Who will survive? I wish for data or states. The flow of code is complicated. No matter how clean your code is.

Technology stack diversity. It’s a good thing. Right? I’m sure you hired all team members for the unknown new stacks. Learning a new programming language is fun, but in production, it’s hell.

Security vulnerabilities. New attack vectors or something like that. I’m not a security expert. Our security team opened many tickets for new vulnerabilities. I don’t understand most of them. Kidding! All I think about is features. What about features?

Cultural shift? We don’t have time for it. There is a bug in production! Not kidding!

You get the point. It is hard to deal with it. We have learned a lot, and we are still learning. All you need is to be prepared for what is coming for you. Features? They are accustomed to waiting.

Code is cheap, show me the product!

Emre MERT — Tue, 03 Jan 2023 14:29:10 +0000

Coding is more complex than ever. Creating a new product is much more complex.

A software developer must have deep knowledge of a subject and knowledge of many other technologies. This t-shaped skill helps developers to build software from end to end. If you are a backend developer you need to understand how frontend works and vice versa. And add some Docker, cloud systems (AWS, Azure …), CI/CD knowledge. Don’t forget to add database knowledge.

Now you are a complete development team, right? No. This is how things work nowadays. Literally. If your code is not working, you need to know the problem’s source. Is it your code, infrastructure, or pipeline? Is data orphaned? Accessing the logs and reading them is also getting complex. You need to use a few tools to just read logs. This is getting crazy.

Despite all this, coding is not so important. Yep! You heard it right.

People care about products, not codes.

Who is using your product?

Who is paying for that?

What difference does your product make?

What is the added value?

How do you differentiate from your opponents?

and many other questions that disturb you about your product

Getting customers is hard. Keeping them is too hard. Word-of-mouth marketing and becoming a love brand is every product owner's dream. But we live in the real world. Who cares about your product? You answer "why", and I answer you "who". Think about why and add some difference. Then sell it. People will love it if you focus on why. People will talk about it if they love it.

Developers? Don't listen to them too much. Give developers freedom, and power but always listen to your clients more. Some solutions don't require coding, some solutions only need integration which can be solved with no-code integration tools. Focus on finding a solution to problems, not on coding.

Implementing a new feature is hard. Implementing the right feature is too hard. Users always request new features. They want a magic button that solves everything. Measure it! Use analytic tools, A/B testing, and ask your clients. But measure it, twice.

Fixing the bugs is hard. Prioritizing bugs is too hard. You always have bugs in your code. You always complaining customers. It is the nature of software. You have limited time and limited resources. You need to prioritize.

If it is not a bug, it is a feature.

Product is the new king! Developers can code any product (just ignore the pain). Even ChatGPT can write code. But what is the right product?

Source: https://emremert.dev/code-is-cheap-show-me-the-product

Lessons I Learned from Developing an IoT Project from Scratch

Emre MERT — Tue, 09 Jun 2020 13:13:11 +0000

I am working on an Internet Of Things (IoT) project. I work with medical IoT devices in my project.

These are the lessons I learned from this project:

Networking: You must know how the network works. You must have a good knowledge of network protocols such as TCP and UDP. If you will work with IoT devices that have multiple network interfaces, you must know about CIDR. Some IoT devices use different technologies such as Bluetooth and Serial port (COM) for data transmitting. Also, there are IoT specific network technologies such as LPWAN.

Data Structures: IoT devices use different data structures. They tend to compress data and send/receive small data because of limited resources. Some old devices which are converted to IoT use weird ways to process data. So you must be aware of endianness, signed and unsigned numbers and encodings. IoT devices can produce large data. Design your system to save and process big data.

Parallel Processing: Asynchronous programming has advantages and disadvantages. Every situation is different and there is no real grab-bag solution. It is not a silver bullet. You must know the differences between process and thread, debugging and tracking asynchronous functions.

Building Embedded Softwares: It is now easier than before. You can easily build a specific solution with Arduino, Raspberry Pi and many other platforms. I couldn't find an IoT device that fits my need. So I decided to develop one. There is huge information about Arduino. I implemented an IoT solution with Arduino in 3 days.

Encrption: It is not only needed for confidentiality. It is also used for user authentication and checking the integrity of data messages. A basic understanding of encryption is enough. There are too many tools and libraries for it.

Building a Mock Device: I was working on a medical IoT device (defibrillator). Preparing this device and configuring the setup takes one hour at some times. So I built a mock device which is completely a software that simulates this device. I used this mock device for too many reasons: traffic monitoring, security and integrity check but mostly for demo purposes in meetings. We had only one device but more than one developer could be able to work on the mock device simultaneously.

Use the same device as much as possible: Edit a configuration and boom! Nothing works! Documents about IoT devices are not up-to-date. Don't make assumptions when changing a configuration or when an IoT device will work as expected. Test it!

You will deal with a lot of cables, triple plugs, batteries, screwdrivers, wi-fi devices, SD cards, USB flash drives.

Working with IoT devices is fun when everything works as expected :)

Original Post: https://www.saascommando.com/2020/06/lessons-i-learned-from-developing-iot.html