Forem: alony

Monorepo Backend Application with Bundled Packages

alony — Fri, 10 May 2024 10:54:46 +0000

I spent hours searching and struggling to set up a server within a monorepo alongside private packages, trying various scripts and methods. Little did I know, the solution was right in front of me the whole time, simple and straightforward to implement.

The frustrating part is, when you're working on a Next.js project within a monorepo, adding your module to the transpilePackages entry in the configuration is all it takes. However, for a backend applications with a custom build step, it's not as straightforward.

I began by creating a monorepo using turbo with applications and packages. It took me 5 minutes with everything installed and working like a charm! Then I was setting up the backend application, utilizing swc to transpile TypeScript into CommonJS format.

My goal was to facilitate type sharing between my web application and backend, utilizing a shared library named "common". Implementing this in the web application, built with Next.js, was straightforward. However, I encountered challenges with the backend. While the transpilation process worked seamlessly on project files, the bundled version required a TypeScript file from the common package, causing issues.

Determined to find a solution, I rolled up my sleeves and delved into options for transpiling my common module. Despite exhaustive searching, swc didn't offer the necessary functionality, and I wasn't keen on resorting to Babel. Then, I stumbled upon esbuild, which promised to transpile all my code into a single bundle. It was a game-changer!

The Solution

To build the project I had to create a build.mjs file.
To transpile and bundle the project, simple run:

> node build.mjs

This is the content of the file:


import * as esbuild from "esbuild";
import packageJson from "./package.json" assert { type: "json" };

// Bundle internal dependencies
const external = Object.keys(packageJson.dependencies).filter(
  (dep) => !dep.startsWith("@repo")
);

await esbuild.build({
  entryPoints: ["./src/index.js"],
  sourcemap: true,
  bundle: true,
  tsconfig: "tsconfig.json",
  platform: "node",
  outfile: "dist/index.js",
  external,
});

Why mjs ?

The build process is waiting for the result, and I'm utilizing import statements. While I could include "type": "module" in my package.json file, doing so risks the bundle targeting esm, which would stress all the dependencies to use esm. However, it's worth noting that some libraries still don't support esm, which can be frustrating, but it's the reality we face.

External Dependencies

Initially, I attempted to run the build process without including the external entry. This led to lots of errors as it attempted to bundle all external dependencies into my bundle. I had hoped for a more robust solution. Consequently, I sought a method to bundle only my private packages, leaving the external ones outside.

Iterating through the dependencies and excluding my private ones, identifiable by their known names, assures me that they will be included in the bundle. For instance, @repo/common will be bundled, while @fastify/cors will not.

The Result

A bundle with my private packages inside!
There is no need to a build step in the common package, only on the top level. The checks I do have in the packages is lint, type checking and tests.

I hope that by sharing this article, I've saved someone valuable time in their search. This approach offers a practical method for managing a backend application within a monorepo while leveraging private packages.

To Review or Not to Review: The Debate on Mandatory Code Reviews

alony — Wed, 24 Apr 2024 10:17:46 +0000

Code reviews play a pivotal role in today's software development lifecycle, serving as a quality assurance mechanism and code-sharing platform. As software projects grow in complexity, ensuring the reliability, maintainability, and security of the codebase becomes increasingly challenging. Code reviews offer a structured process for developers to collaboratively assess code changes, identify potential issues, and provide constructive feedback when merging new code.

Code reviews potential downside is the time and resources required to conduct thorough reviews, which can introduce delays in the development process and impact project timelines. Additionally, code reviews may sometimes result in interpersonal conflicts or hurt feelings, particularly if feedback is perceived as overly critical or if there are disagreements about coding style or approach. Moreover, enforcing mandatory code reviews can create bottlenecks in the workflow if team members have conflicting priorities.

Another point I want to emphasize is that mandatory code reviews can create the impression that they should prevent production errors and bugs in the workflow. However, this is often not the case. In reality, it's impossible to catch every aspect of a change without a full QA cycle. In some cases, the pressure of the review process can lead the individual being reviewed to make unnecessary or inconsistent changes simply to appease the reviewer and gain approval for their request.

Code reviews often come too late in the process. It's a case of "too little, too late." In the following text, I want to address some of the reasons teams are enforcing code reviews as a mandatory part of the workflow.

Code Review For Style Guide

We don't have to block the workflow and wait for a code review to validate the style guide. Instead, we can utilize automated tools and adopt a Shift-Left approach, allowing us to receive continuous feedback throughout the development process.

Automating code checks with static code analysis allows us to enforce code styling effectively. By integrating tools into our workflow, we can identify errors at an early stage, while coding instead of blocking us at the end. For instance, flake8 checks Python code for style and errors, eslint performs similar checks for JavaScript, and prettier automatically formats code to maintain consistency.

If you're actively testing your codebase, which I hope you are, consider integrating a code coverage automatic checker such as codecov. This tool can alert if the coverage drops below a threshold. While I've had positive experiences with such tools, it's worth mentioning that the adoption process may pose some challenges.

It's worth considering investing time and budget into automatic tools for checks done during the review process by us, particularly the more tedious ones. Consider investing in a code analysis service like SonarCloud, a personal favorite of mine. This approach allows us to maintain focus on achieving our objectives without getting bogged down in repetitive tasks.

Code Review For System Design

Code review validates the system design and ensures proper planning of architecture. reusing existing capabilities, avoiding redundant code flows, preventing duplication, and enhancing efficiency. I argue that evaluating the system design at the end of the process is misguided; investing more time in upfront planning ensures a cycle concludes with a sound design. What are the consequences of changing the design late in the process, and what are the outcomes of such adjustments? A drawback of this is the urge to implement and not invest enough time for planning.

I once had a debate with a colleague about time estimations in a TDD environment. We discussed the complexity of estimating tasks, recognizing that they include more than just writing tests. Tasks involve various elements such as planning, translations, data analytics, rollout plans, and testing is just one component. Evaluating tasks requires a holistic approach, considering all aspects including coding time, testing, and QA. It's an art mastered by a few.

We need to have enough time for planning changes properly, allowing for discussions and consensus on architectural adjustments or understanding the necessary system modules for a feature. Neglecting proper planning can lead to unforeseen challenges later, resulting in higher costs in the long run.

"R&D stands for Research and Development. Sometimes, we forget the 'R'." - unknown

Code Review for Checkup Points

Code reviews frequently serve as checkpoints to ensure the integrity and consistency of feature design and system architecture across the codebase. The question arises is how do we validate changes? My stance is that this is a result of providing context and empowering developers to self-assess, seeking validation from peers when necessary.

We need to build an embracing ecosystem that fosters a sense of responsibility among individuals and empowers team members with a strong sense of accountability. This is a bug challenge that deserves its own post. It involves trusting and empowering to succeed, fostering growth with honesty and candor.

Creating this kind of ecosystem, combined with the context of change, works wonders. It enables team members to consult and offer input on changes beforehand, preventing roadblocks and fostering developer growth. I would argue that, even in the absence of mandatory code reviews, team members will seek reviews when necessary to ensure changes are implemented optimally. Only that way, the process of code review is positive by nature.

Code Review to Prevent Errors and Bugs

When reviewing code, we often hunt for nonsensical segments and offer feedback to clear potential bugs. However, I argue that this approach creates a false sense of confidence as one can not find all the bugs as we are exposed to human error and mental overload.

Code review is not a monitoring tool, but a code quality tool. Yes, it helps prevent errors and bugs. However, doing reviews is done at the end of each change and is inconsistent and costly. Instead, establish rollout plans with feature toggles and KPIs and integrate QA processes pre-production. Optimize the workflow for top developers and work your way so everyone aligns, rather than compromise to the lowest common denominator.

Conclusion

In my experience, I've worked in environments both with and without mandatory code review steps, serving in roles as both a developer and a leader. Interestingly, I found that when code reviews were mandatory, the process often felt burdensome and lacked enthusiasm from team members. On the other hand, in environments where code reviews were not mandatory, code reviews still happened often
and was highly valued, yielding significant benefits for the team.

The debate on whether to mandate code reviews continues, but perhaps it's time to drop the restrictions and embrace a more agile approach. By leveraging automated tools, fostering a culture of accountability, and emphasizing upfront planning, we can simplify the development process without sacrificing quality. So, let's bid farewell to mandatory code reviews and start on a journey toward more efficient and enjoyable coding experiences!

Harmony in Complexity: The Fusion of Simplicity and Clean Code

alony — Fri, 19 Apr 2024 18:20:12 +0000

As a software engineer, my passion lies in crafting not just functional but elegantly simple code. Amidst the chaos of complex systems, lines, and algorithms, simplicity is like a breath of fresh air. It allows me to cut through the noise, to see the patterns and structures more clearly. To me, simplicity isn't just a preference; it's a necessity. It makes my code more maintainable, understandable, and reliable.

"Simplicity is the ultimate sophistication." - Leonardo da Vinci

The first time I came across Clean Code, I was captivated. This aligns perfectly with my preference for organized environments where I can think clearly. I always strive to tidy up my workspace before starting any task, as it helps me gain clarity. Even in my children's education, I'm drawn to the Montessori method for its elegant environment. I've observed how children thrive in organized spaces, fostering a sense of calmness and boosting self-esteem.

Clean Code represents a set of principles and practices that prioritize simplicity, clarity, and maintainability in software development. It emphasizes writing code that is easy to understand not only by the author but also by fellow developers who may encounter it in the future. We follow a set of rules and principles designed to promote simplicity, clarity, and maintainability. For instance, meaningful naming, minimizing duplications, IOC (Inversion of Control), SOLID principles, and KISS (Keep It Simple, Stupid).

In this post, I want to share some useful tips for crafting code out of my experience. I’ll try to offer my way of thought for your delightful. We'll look at examples from the world of online shopping, like products, orders, coupons, and more.

Function Structure

"Thinking is the hardest work there is, which is probably the reason why so few engage in it." - Henry Ford

As simple as it looks, composing functions can be a turning point for creating neat and beautiful functions. When writing new functions with clear input and output, I often begin by typing out hypothetical function calls that eventually lead to the desired outcome.

For this example, we will implement applyCoupons function. Each coupon comes with its own set of rules (such as minimum purchase thresholds or specific product requirements). The task at hand is to determine the best coupon to apply to a given shopping cart.

Here's a basic initial implementation:

export function applyCoupons(cart, coupons) {
 let resultCart = cart;
 let resultCoupon = null;


 for (const coupon of coupons) {
   const cartAfterDiscount = applyDiscount(cart, coupon);
   if (cartAfterDiscount.price < resultCart.price) {
     resultCart = cartAfterDiscount;
     resultCoupon = coupon;
   }
 }
 return [resultCart, resultCoupon];
}

Let's examine how the function might appear by typing out hypothetical function calls:

export function applyCoupons(cart, coupons) {
 const carts = coupons.map((coupon) => applyDiscount(cart, coupon));
 return chooseBestCart(zip(carts, coupons));
}

Creating hypothetical function calls:

Helps break down the problem into smaller, simpler tasks - it's easier to solve small problems. If the function is well-defined, readers don't have to check the function itself, reducing mental strain.
These smaller tasks can be combined into other functions, as long as they're general enough. I'd say it's not a worry for now, only for later uses.
It prompts you to think of a clear sequence that moves forward only - stopping spaghetti code that declares and updates variables all over the place.

The biggest challenge that might come up is the temptation to dive straight into coding to avoid mental strain. Let me tackle that in the next tip.

The Importance of Refactor

"Change is the only constant in life." - Heraclitus

I first came across the term refactor while practicing Test-Driven Development (TDD). In TDD, the process involves writing a failing test, then writing the code to pass the test, and finally refactoring the code. Over time and through experience, I realized that the refactoring step is, actually, the most important part of this process. The cycle of failing tests transitioning to passing tests allows us to refactor without the fear of breaking, a power that is nothing short of remarkable.

When composing a new function or changing an existing one, I always begin and end the process with refactoring. Why? Because I aim to start with a clean slate and ensure the space remains clean afterward.

To illustrate the concept, let's consider the initial version of applyCoupons and introduce some complexity. For instance, suppose we aim to apply all coupons, starting with the one with the most discount. Our approach would be first to get the best coupon to apply, then update the cart and filter the coupon out of the coupon list.

import { range } from 'lodash';

export function applyCoupons(cart, coupons) {
 const [cartWithDiscount] = range(coupons.length).reduce(
   ([processedCart, couponsToApply]) => {
     let bestCoupon;
     let cartWithDiscount = processedCart;
     for (const coupon of couponsToApply) {
       const cartAfterDiscount = applyDiscount(processedCart, coupon);
       if (cartAfterDiscount.total < cartWithDiscount.total) {
         bestCoupon = coupon;
         cartWithDiscount = cartAfterDiscount;
       }
     }
     return [
       applyDiscount(processedCart, bestCoupon),
       couponsToApply.filter((c) => c !== bestCoupon),
     ];
   },
   [cart, coupons]
 );
 return cartWithDiscount;
}

You may notice that I'm using the reduce function, even though it may seem more complex than just a for loop. However, using reduce allows us to work with the same variables without needing to update anything outside of the current scope. Now, we can proceed to refactor further by extracting a getBestCoupon function and trim it down:

export function applyCoupons(cart, coupons) {
 const [cartWithDiscount] = range(coupons.length).reduce(
   ([processedCart, couponsToApply]) => {
     const bestCoupon = getBestCoupon(processedCart, couponsToApply);
     return [
       applyDiscount(processedCart, bestCoupon),
       remove(couponsToApply, bestCoupon),
     ];
   },
   [cart, coupons]
 );
 return cartWithDiscount;
}

I believe the result could have further refactoring, possibly using a more functional approach.

export function applyCoupons(cart, coupons) {
 const [cartWithDiscount] = range(coupons.length).reduce(
   chain(applyBestCoupon, filterCoupon),
   [cart, coupons]
 );
 return cartWithDiscount;
}

After refactoring, we can see that the code becomes more readable and adaptable to future modifications. I would argue in favor of refactoring, even in the absence of automatic test coverage, as the code becomes easier to comprehend and potential bugs can be identified during code review, just do it wisely and add some manual tests if needed, but it is worth the effort.

Cognitive Complexity

"Simplicity is the key to brilliance." - Bruce Lee

Cognitive complexity is the level of mental effort required to understand a piece of code. It focuses on how challenging it is for a human programmer to comprehend and reason about the code. We take various factors into account:

Nesting depth: The level of nesting within control structures such as loops and conditionals.
Cyclomatic complexity: The number of independent paths through a piece of code.
Blocks length: The number of lines or statements. Code duplication.
Variable naming and documentation: The clarity and descriptive nature of variable names.

To simplify code and lower cognitive complexity, we often refactor by extracting blocks of code into separate functions, reducing the block length. Use early return statements when the exit terms are clear to reduce cyclomatic complexity, reusing existing code and name variables and functions in a meaningful name.

One point I want to pick your brain with is avoiding the use of control structures, such as if and for blocks, to simplify code. These structures can lead to nested blocks that run based on certain conditions, which can complicate code readability and maintenance. For instance, consider filtering products by name; let's explore an example.
Here is the basic, initial implementation of products filter:

function filterProducts(products, name) {
 const result = [];
 for (const product of products) {
   if (contains(product.name, name)) {
     result.push(product);
   }
 }
 return result;
}

Consider the following code instead:

function filterProducts(products, name) {
 return products.filter(productContains({ name }));
}

The second version is shorter and easier to understand. One point I want to highlight here is that we don't always have to perform all tasks within the same loop. As we've learned, O(n) is the same as O(n) + O(n) theoretically. I often prefer to execute certain operations in separate loops rather than overcomplicating a single loop for the sake of performance. However, it's essential to note that this approach isn't always applicable, and there may be cases where performing tasks in the same loop is necessary.

Another approach to minimize the use of control structures is by storing the logic in a configuration object and accessing it from there. For instance, let's consider the scenario of ordering products from various providers:

async function placeOrder(order) {
 let result;
 if (order.provider === 'provider1') {
   result = await orderFromProvider1(order);
 } else if (order.provider === 'provider2') {
   result = await orderFromProvider2(order);
 } else if (order.provider === 'provider3') {
   result = await orderFromProvider3(order);
 } else {
   throw new Error('Unknown provider');
 }
 return result
}

And we can do it with holding the login in a configuration object:

const providers = {
 provider1: orderFromProvider1,
 provider2: orderFromProvider2,
 provider3: orderFromProvider3
};


function placeOrder(order) {
 const orderFromProvider = providers[order.provider];
 if (!provider) {
   throw new Error('Invalid provider');
 }
 return orderFromProvider(order)
}

Clearly, this is a simplified example, but I argue that it's more maintainable because we can easily maintain the configuration object and make changes. Implementing a typing system to specify the input and output of all functions can be incredibly advantageous. Furthermore, if the creation criteria become more complex, creating a factory pattern can create the correct function or the appropriate class instance.

Conclusion

By prioritizing simplicity, we can enhance the maintainability, understandability, and reliability of our codebase. From function composition to minimizing control structures, I provided practical insights and tips for crafting clean and elegant code. Moreover, it underscores the significance of adhering to clean code principles, such as meaningful naming and minimizing complexity, to build robust and scalable software solutions.

As a passionate advocate for simplicity, I could talk all day about it. I have plenty more topics in mind, such as YAGNI (You Ain't Gonna Need It), SOLID principles, system design, and dependency management. However, let's keep it simple as suggested in the title.