Forem: Laurentiu Raducu

Microservices are bad for your mental health

Laurentiu Raducu — Wed, 14 Aug 2024 12:08:46 +0000

Short, tldr version here: https://www.youtube.com/watch?v=wtXrOXYO0LA
Long version below 👇

Last month I wrote a hit piece on software in general, venting my decade-long frustrations as a user of commercial software. While I briefly touched on today’s topic, I didn’t delve into it deeply. So, I plan to share my experiences with microservices and explain why this trend is causing burnout and immense frustration among developers.

There are many myths surrounding microservices. Some people advocating for them say that scaling-related problems are solved by following this architecture. Some say that it enables high development speed, while others exaggerate how great it is in comparison to monoliths. These myths were propagated by tech gurus and influencers, people who don’t usually write code to create successful products, but rather philosophize about different tech topics. They don’t have the time to develop products, as their role as professional conference speakers occupies most of their time.

Engineers mistakenly look for answers formulated by these philosophers (or thought leaders). Instead of thinking critically about problems and trying to simulate what the outcome will be, they will just go to a conference, listen to these professional conference speakers, and start idolizing the nice stories they’ve heard.

As a consultant working mostly with large enterprises, I am amazed to see how huge budgets are being allocated to moving away from monolithic to microservice architectures. The motivation seems always to be related to scaling, but the only thing that scales in the end is their cloud costs. I tried to track down some of these initiatives, and most of them come from non-technical CTOs trying to impress the board members by shoving the latest buzzwords in their faces. A limited subset of these initiatives are influenced by visits from tech gurus, and only in one case was this architecture style a promising change and came organically from the engineering teams.

Why Microservices Suck?

In my latest blog post I got some really nasty answers, as well as some death threats on my YouTube channel, therefore I have to give you the following disclaimer. In this post, I am going to give my personal opinion based on anecdotes from my experience as a consultant. Although this might seem controversial, my aim is to raise awareness around the pitfalls of this architectural choice. My aim is not to insult or challenge anyone personally, so don’t take this post too seriously. If you don’t agree and you cannot supply constructive criticism, just stop reading, it’s that simple.

Microservices do not scale better than monoliths

Established tech voices always tell you that rather than scaling vertically, by adding hardware resources to a server, like it’s being done in the case of monoliths, it’s better to do it horizontally, by adding another server. This is often false. (or “This is not always true.”)

Contrary to what you’ve heard so far from the biggest voices in tech, there is no superiority when it comes to scaling with neither of these architectures. Let’s break this down with a clear example:

Imagine you have a monolith service for an e-commerce app, which is containerized and deployed on two machines, with a load balancer balancing the requests between the two. You are convinced by a tech guru that in order to scale, you must convert everything to microservices. You decide to split your monolith into multiple services:

Authentication
Products
Orders
Payments
Stock

Of course, you won’t go with these simple names, so you get your team to brainstorm some cool names for your services, inspired by either comic books, mythology, or astronomy. Since you are also agile, you need to add story points to this effort, so you will end up losing some time in endless debates about what the difference between 2, 3, and 5 story points is, and you finish by adopting the t-shirt size estimation method.

After all this effort, you end up with two machines for each service and a load balancer for each, which means a total of 10 machines (without the gateway service), a huge amount of bloatware given by the many HTTP requests going between services, and a new living hell for maintaining the contracts between these new services. How is this scalable if, for each new service, you need to provision a new machine (or two), find the best strategy for balancing the load, and make sure the contracts won’t break when developing new features?

Microservices are not testable

As humans, we are not able to predict what a change is going to cause in a complex system. Without proper simulations or testing against a system that replicates the real one, you cannot predict what will happen. In software, testing is absolutely critical for ensuring software quality, we all know that. However, in the case of microservices, testability is often lower than in the case of monoliths.

On the one hand, you have the tests developed for the service itself. Let’s stick with the same e-commerce app, where we want to define tests for our payments service. Devs will write unit and system tests, which will assess how individual components behave when fed certain inputs. They need to make a change to adapt the quantity field, which is the field that is supposed to hold the information on the quantity of the product that the customer needs to pay for, from integer to double. After deploying the change, production gets broken, and the company loses thousands of dollars for not allowing customers to buy anything for a few minutes.

Even though the tests were green, the integration between the service supplying the quantity and the payment service wasn’t tested properly. The quantity was still sent as an integer instead of being adapted to double, to match the expectation of the consumer (payment service).

There are ways to address this issue of testing the integration of the services, but none of these ways are easy enough, and require constant maintenance. Contract testing is one of the options. In contract testing, you define the consumer’s HTTP request and the provider’s expected response. You send the request and the expectation to a third-party service (in the case of Consumer-driven Contract Testing from Pact, the broker), and this service will execute the request against the provider, and check the response against the expectation. This sounds great in theory, but:

Both the consumer and provider need to make assumptions about the behavior of specific fields in the contract that are not explicitly enforced. For example, a consumer might expect a “quantity” field to be always positive, while the provider might allow zero to indicate “out of stock”. The consumer assumes it’s always positive because that fits its usage pattern. Meanwhile, the provider assumes it needs the flexibility of zero for inventory management. In the context of verifying the schema, this kind of assumptions would never be checked.
The effort for maintaining these tests is huge, on one hand, in the case of teams that are in early-stage development. It doesn’t provide flexibility in dealing with the rapidly evolving interfaces. On the other hand, legacy systems might need to invest in performing big amounts of refactoring in order to enable this type of testing, which is impractical for most of the organizations.
The learning curve is steep – devs will have to fully understand the concept, spend time developing proof of concepts, test their assumptions, which drains a lot of time and energy from focusing on critical business requirements.
You can always default to end-to-end tests, which will give you the confidence and fidelity that the changes won’t break your system. However, these tests are slow, and in the context of microservices, in a really entangled architecture, devs will spend a lot of time defining, maintaining, debugging and even running those tests. It’s just not worth it to wait hours and hours for the tests to end in order to deploy a change.

Development is very slow for microservices

You’ve heard that right! It takes a lot of time for a developer to push a change in this architecture, which leads to sluggish development cycles. A primary reason for this is the increased cognitive load and context switching that developers face. With numerous independent services, each with its own codebase, dependencies, and deployment processes, developers must constantly switch between different contexts to understand and modify the system. This continuous mental juggling can significantly impede development speed, especially for complex systems with a multitude of microservices.

Furthermore, debugging and troubleshooting become more cumbersome in a microservice environment. When an issue arises, pinpointing the root cause can be time-consuming as developers need to trace the interaction path through multiple services. This process involves analyzing logs, tracking requests, and potentially reproducing the issue in different environments, further adding to the development time. The distributed nature of microservices can also make it challenging to maintain a holistic view of the system, leading to inefficiencies and delays in resolving problems.

All in all, I think microservices are bad for your mental health as an engineer. What appears to be a huge jump in quality of life, leading to fast development, better flexibility and improved scalability, turns out to be a huge pain. Constant context switching, increased stress from observing and understanding all the interactions, and the hazardous nature of adding changes to your services will easily make you burn out and lose motivation for your work.

Simplify Your Taxes with AI: How to Create a Tax Consultant Using OpenAI API and Python

Laurentiu Raducu — Tue, 18 Apr 2023 07:29:24 +0000

Tax consulting can be a complex and time-consuming process for individuals and businesses alike. With changing tax laws and regulations, it’s crucial to stay up-to-date and accurate in tax planning and compliance. Fortunately, artificial intelligence (AI) has the potential to streamline the tax consulting process, and one powerful tool in this realm is the OpenAI API.

OpenAI is a leading AI research organization that offers a powerful language generation API, which can be leveraged to create a tax consultant using Python. By utilizing the OpenAI API, you can automate the generation of tax-related information, forms, deductions, and credits, making the tax consulting process more efficient and accurate.

In this blog post, we will explore how to create a tax consultant using the OpenAI API and Python. We will delve into the capabilities of the OpenAI API, provide step-by-step instructions on setting up your tax consultant project, discuss the development process, and explore how machine learning can enhance the tax consultant’s capabilities. So, let’s dive into the world of AI-powered tax consulting and see how it can simplify your taxes and save you time and effort.

Setting Up Your Tax Consultant Project

Setting up your tax consultant project with Python and the OpenAI API is a straightforward process. Here’s an outline of the steps you’ll need to follow:

Step 1: Prerequisites

Before getting started, ensure that you have the following prerequisites in place:

Python: Make sure you have Python installed on your machine. You can download the latest version of Python from the official Python website (https://www.python.org/).
OpenAI API Credentials: You’ll need to obtain API credentials from OpenAI to access their API. You can sign up for an account and obtain the necessary credentials from the OpenAI website (https://www.openai.com/).
Python Libraries: Install the necessary Python libraries for interfacing with the OpenAI API. The main library you’ll need is the openai library, which provides Python bindings for the OpenAI API. You can install it using pip, the Python package manager, with the following command: pip install openai

Step 2: Setting Up the Project Environment

Once you have the prerequisites in place, you can set up your Python project environment for creating the tax consultant. Here are some instructions and code snippets to guide you:

Create a New Python Project: Start by creating a new directory for your tax consultant project. You can use your favorite Python IDE or text editor to create a new Python file (.py) within this directory.
Import the Required Libraries: In your Python file, import the openai library and any other libraries you may need for your project. Here's an example:
import openai
Authenticate with OpenAI API: Use your OpenAI API credentials to authenticate with the API. You can do this by setting the OPENAI_API_KEY environment variable in your Python script, or by passing your API key as a parameter when initializing the openai library. Here's an example:

import openai

openai.api_key = "YOUR_API_KEY"

Define Functions and Variables: Define any functions or variables that you’ll need for your tax consultant project. For example, you might define a function to generate tax-related information, or variables to store input data.
Test the API Connection: Finally, test your API connection by making a simple API call. For example, you can use the openai.Completion.create() method to generate a basic text completion. Here's an example:

import openai

openai.api_key = "YOUR_API_KEY"

response = openai.Completion.create(
  prompt="Hello, I'm testing the OpenAI API!",
  max_tokens=50
)
print(response.choices[0].text)

If you receive a response from the API, it means your project environment is set up correctly, and you’re ready to start building your tax consultant using the OpenAI API and Python!

With the project environment set up, you can now move on to the development of your tax consultant, which involves leveraging the capabilities of the OpenAI API to generate tax-related information and provide accurate responses. In the next section, we’ll explore the development process in more detail, so you can start creating your own AI-powered tax consultant.

Section 3: Developing the Tax Consultant

Once you have set up your project environment, you can start developing your tax consultant using the OpenAI API and Python. In this section, we will discuss the approach for developing the tax consultant and explore the different components involved in the process.

Data Input: The first step in developing the tax consultant is to define the input data that will be provided to the API. This can include information such as the user’s income, deductions, credits, and other relevant tax details. You can collect this information from the user through a user interface or by programmatically retrieving data from external sources.
Processing Data: Once you have collected the input data, you can process it to generate meaningful queries or prompts to send to the OpenAI API. This can involve formatting the data in a way that is compatible with the API’s input requirements, such as creating a prompt that asks a specific question or provides a context for generating tax-related information.
Generating Responses: The core functionality of the tax consultant is to generate accurate and relevant tax-related information using the OpenAI API. You can utilize the OpenAI API’s language generation capabilities to generate responses that include tax forms, deductions, credits, and other tax-related information. You can experiment with different prompt engineering techniques, such as tweaking the prompt, adjusting the temperature and max tokens parameters, to fine-tune the responses to meet your desired level of accuracy and comprehensiveness. Here’s an example of how you can use the OpenAI API to generate a tax-related information response:

import openai

openai.api_key = "YOUR_API_KEY"

income = 50000
deductions = 10000
prompt = f"Generate a tax form for a person with an income of ${income} and deductions of ${deductions}."

response = openai.Completion.create(
  prompt=prompt,
  max_tokens=200
)
tax_info = response.choices[0].text

processed_tax_info = process_tax_info(tax_info)

print(processed_tax_info)

Customizing the Tax Consultant: You can also customize the behavior of your tax consultant by training it with your own data. The OpenAI API allows for fine-tuning using custom datasets, which can enable your tax consultant to provide more accurate and tailored responses based on your specific use case or domain expertise. Fine-tuning involves training the model with additional data, such as tax laws, regulations, and specific tax scenarios, to make it more proficient in generating tax-related information.

The tax consultant developed using the OpenAI API and Python can provide automated and accurate tax-related information, including forms, deductions, and credits.

Creating a tax consultant with the OpenAI API and Python can be a game-changer in the world of taxes! Gone are the days of sifting through stacks of forms and scratching your head over complicated deductions.

It’s difficult to say at this point how hard will this industry be disrupted by AI. According to this paper published by OpenAI, tax consultants are 100% exposed to the capabilities of AI. However, without any legislation in place, it is difficult to say if AI will replace anyone. In the meantime, we can enjoy a productivity boost given by these tools. Happy coding!

Optional Chaining in JavaScript: Simplify your Code and Avoid Unwanted Errors

Laurentiu Raducu — Mon, 20 Feb 2023 19:01:19 +0000

If you've been writing JavaScript code for a while, you've probably encountered situations where you need to access properties or methods of an object, but you're not sure if the object is defined or has the desired property or method. In the past, you may have written verbose code with many conditional statements to avoid errors caused by undefined objects. However, with the introduction of Optional Chaining in JavaScript, you can simplify your code and avoid these unwanted errors.

What is Optional Chaining?

Optional Chaining is a feature that was introduced in ECMAScript 2020 (ES2020) and is now widely supported by modern browsers. It allows you to access properties and methods of an object without the need for null checks or conditional statements. Instead, you can use the ?. operator to access the property or method, and if the object is null or undefined, the expression will evaluate to undefined without throwing an error.

Syntax of Optional Chaining

The syntax for Optional Chaining is straightforward. You simply add a ?. operator between the object and the property or method you want to access. Here's an example:

const user = {
  name: 'John',
  age: 30,
  address: {
    street: '123 Main St',
    city: 'Anytown',
    state: 'CA'
  }
};

const state = user.address?.state;

In this example, we use Optional Chaining to access the state property of the address object, even if address is undefined. If address is undefined, the state variable will be set to undefined as well, without throwing an error.

Use Cases for Optional Chaining

Optional Chaining can be used in a variety of situations where you need to access properties or methods of an object, but you're not sure if the object is defined. Some common use cases include:

Accessing properties of nested objects:

const user = {
  name: 'John',
  address: {
    street: '123 Main St',
    city: 'Anytown',
    state: 'CA'
  }
};

const state = user?.address?.state;

In this example, we use Optional Chaining to access the state property of the address object, even if address is undefined.

Accessing methods of an object:

const user = {
  name: 'John',
  getAddress: function() {
    return {
      street: '123 Main St',
      city: 'Anytown',
      state: 'CA'
    }
  }
};

const state = user?.getAddress()?.state;

In this example, we use Optional Chaining to access the state property of the object returned by the getAddress method, even if getAddress is undefined.

Checking for the existence of a property or method:

const user = {
  name: 'John',
  age: 30
};

const hasAddress = user?.address !== undefined;

In this example, we use Optional Chaining to check if the address property exists on the user object, without throwing an error if user is undefined.

Optional Chaining is a powerful feature that simplifies your code and helps you avoid unwanted errors caused by undefined objects. It can be used in a variety of situations where you need to access properties or methods of an object, and it is now widely supported by modern browsers. If you're not already using Optional Chaining in your JavaScript code, it's definitely worth considering as a way to improve your code quality and reduce the risk of errors

My Journey Creating an API to Check Terms and Conditions Using ChatGPT

Laurentiu Raducu — Fri, 17 Feb 2023 15:01:53 +0000

Hey there, fellow developers! Today, I want to talk about my latest project called terms-and-conditions-verifier. The aim of this API is to check any terms and conditions sent by a client app for any irregularities or fishy stuff. To achieve this, I have integrated ChatGPT, which is an advanced language model that can understand natural language, into the API.

To give you a brief idea of how the code works, let's take a look at the following file:

import dotenv from "dotenv-safe";
dotenv.config();
import express from "express";
import bodyParser from "body-parser";
import cors from "cors";
import { ChatGPTAPIBrowser } from "chatgpt";
import { oraPromise } from "ora";
import config from "./config.js";

const app = express().use(cors()).use(bodyParser.json());

const gptApi = new ChatGPTAPIBrowser({
  email: process.env.OPENAI_EMAIL,
  password: process.env.OPENAI_PASSWORD,
});

await gptApi.initSession();

const Config = configure(config);

class Conversation {
  //...
}

const conversation = new Conversation();

app.post("/eula-summary", async (req, res) => {
  //...
});

const EnsureAuth = new Promise((resolve, reject) => {
  //...
});

async function start() {
  //...
}

function configure({ plugins, ...opts }) {
  //...
}

start();

The first few lines of code import necessary modules such as dotenv, express, body-parser, cors, ChatGPTAPIBrowser and oraPromise. dotenv is used to read environment variables from a .env file, express is used to create an instance of the express application, body-parser is used to parse incoming request bodies, cors is used to enable Cross-Origin Resource Sharing (CORS), ChatGPTAPIBrowser is used to interact with the ChatGPT API via the browser, and oraPromise is used to create loading spinners for various functions.

After importing the modules, the code initializes the ChatGPT API by passing the credentials to the ChatGPTAPIBrowser constructor. It then creates a new Conversation instance and configures the API by calling the configure function.

The configure function collects rules and parsers from all plugins and trains the ChatGPT model with all the rules. It then runs the ChatGPT response through all plugin parsers.

The start function connects to ChatGPT and trains the model with the rules collected from all plugins. It then starts the express application, and the app.post method listens for incoming requests. Once a request is received, the API sends the terms and conditions to the Conversation instance for processing.

This project can prove to be a powerful tool that can help you detect any irregularities or fishy stuff in terms and conditions. With the help of ChatGPT, you can easily process natural language and provide quick responses to your clients. I hope this post has given you a better understanding of the code behind the terms-and-conditions-verifier API. Happy coding!