Forem: Chakrit Likitkhajorn

Things Java (and C#) developer should unlearn when start learning new languages (Part 1)

Chakrit Likitkhajorn — Sat, 11 Mar 2023 15:34:24 +0000

Java and C# are very famous for being good programming languages for enterprise work. It's pretty stable. Patterns and practices are known and well-established. They are good boring technology that works.

However, since the community and developer of these languages usually work in enterprise development. Enterprise love having a standard. Enterprise love having design patterns. Enterprise love having best practices. In Java and C# world, there are established patterns and best practices for almost everything. Many developers are rigidly thought that there is only one single best way to achieve something, and every alternative is undesirable. This belief is embedded in the culture of the community.

For the sake of future reference, I will call these developer JC developer.

That culture well in the enterprise environment. It's not good to have every single developer invent their own pattern and make every part of a big codebase become weirdly unique.

However, I found many JC developers struggle to learn a new programming language.

Other programming languages such as Go, Elixir, Python, Javascript, TypeScript, Clojure, Ruby, Scala or Rust have a different style, semantic and idiomatic which looks and feels different from JC. I've seen many
JC developers try to force JC-ish code into other programming languages and it usually does not end well.

There are many patterns, best practices and beliefs that JC developer should let go of while learning other programming language. I'm here to talk about it.

Dependency Injection (or IoC container) isn't always necessary to make code testable

This is the most common one. JC developer usually asked where is dependency injection and where is IoC container that I can config dependencies.

In many programming language, you don't need IoC container in order to switch between test double and actual code.

Assuming you have to call PaymentGateway to pay for things. If the result is success, you need to save some data in response into transaction database.

In Ruby, you can just switch test and code using Mocha. The language allow us to do a simple monkey patching. Here are some examples.

Ruby

class Pay
  def pay(payment)
    response = PaymentService.new.pay(payment)
    transaction = Transaction.new
    transaction.amount = response[:amount]
    transaction.ref = response[:ref]
    raise 'Cannot save' unless transaction.save

    transaction
  end
end

class PayTest < ActiveSupport::TestCase
  test 'pay when success' do
    payment = { amount: 500 }
    Transaction.any_instance.stubs(:save).returns(true)
    PaymentService.any_instance.expects(:pay).with(payment).returns({ amount: 500, ref: 'super' })
    t = Pay.new.pay(payment)
    assert_equal 500, t.amount
    assert_equal 'super', t.ref
  end

  test 'raise when pay failed' do
    Transaction.any_instance.stubs(:save).returns(false)
    PaymentService.any_instance.stubs(:pay).returns({ amount: 500, ref: 'super' })
    exception = assert_raise(StandardError) do
      Pay.new.pay({ amount: 500 })
    end
    assert_equal 'Cannot save', exception.message
  end
end

In Ruby, even though class Pay require PaymentService object to connect to payment gateway and Transaction object to connect to database. We don't really need dependency injection or IoC container in order to make this code testable. Ruby allows us to simply say: For every transaction instance, save will return true and PaymentService will return our particular result without connecting to an actual third-party.

Transaction.any_instance.stubs(:save).returns(true)
PaymentService.any_instance.expects(:pay).with(payment).returns({ amount: 500, ref: 'super' })

Javascript/TypeScript also have same monkey-patch approach. But since class isn't even necessary, we can even reduce PaymentService and Transaction to be mere functions.

// Implementation
const transaction = require("./transaction");
const paymentService = require("./paymentService");

async function pay(payment) {
  const response = await paymentService.pay(payment);

  const newTransaction = await transaction.save({
    amount: response.amount,
    ref: response.ref,
  });

  if (!newTransaction) {
    throw new Error("Cannot save");
  }
  return newTransaction;
}

// ----------------- Test file -----------------
jest.mock("./transaction");
jest.mock("./paymentService");

const transaction = require("./transaction");
const paymentService = require("./paymentService");
const pay = require("./pay");

describe("pay", () => {
  it("should be able to pay with correct amount", async () => {
    const payment = { amount: 300 };
    paymentService.pay.mockResolvedValue({ amount: 300, ref: "myref" });
    transaction.save.mockResolvedValue({ amount: 300, ref: "myref" });

    const result = await pay.pay(payment);
    expect(transaction.save).toHaveBeenCalledWith({
      amount: 300,
      ref: "myref",
    });
    expect(result.amount).toEqual(300);
    expect(result.ref).toEqual("myref");
  });

  it("should be throw when cannot saved", async () => {
    const payment = { amount: 300 };
    paymentService.pay.mockResolvedValue({ amount: 300, ref: "myref" });
    transaction.save.mockResolvedValue(null);

    await expect(() => pay.pay(payment)).rejects.toThrowError("Cannot save");
  });
});

First of all, in Javascript, you don't always need an object or class to do a simple thing (I will emphasize this later). Second, you can use jest.mock to force return value and even implementation of a class or function. This means you don't need a dependency injection or IoC container to create a testable code.

While there might be some merit to using IoC and dependency injection anyway, I have seen many JC developers who insist that you can't write testable code without dependency injection and IoC container. That is not true for many other languages.

If all you need is testable code, mocking and monkey-patching (like jest.mock and stub) can be simpler and more direct to achieve the effect. You don't need to bring the whole IoC container library into the equation.

In Go, dependency injection in general is still a common way to write a test. However, since Go does not confine you to a class system, dependency injection can be done at a function level. Here is an example from gitbook Learn Go with test.

package main

import (
    "fmt"
    "io"
)

type Sleeper interface {
    Sleep()
}

const finalWord = "Go!"
const countdownStart = 3

func Countdown(out io.Writer, sleeper Sleeper) {
    for i := countdownStart; i > 0; i-- {
        fmt.Fprintln(out, i)
        sleeper.Sleep()
    }

    fmt.Fprint(out, finalWord)
}

// ----------------- Test -------------------
type SpySleeper struct {
    Calls int
}

func (s *SpySleeper) Sleep() {
    s.Calls++
}

func TestCountdown(t *testing.T) {
    buffer := &bytes.Buffer{}
    spySleeper := &SpySleeper{}

    Countdown(buffer, spySleeper)

    got := buffer.String()
    want := `3
2
1
Go!`

    if got != want {
        t.Errorf("got %q want %q", got, want)
    }

    if spySleeper.Calls != 3 {
        t.Errorf("not enough calls to sleeper, want 3 got %d", spySleeper.Calls)
    }
}

Notice that the SpySleeper is injected directly into the function. You can see that in other languages it isn't necessary to inject dependency via an object constructor.

I have demonstrated how you can write unit testable code via different method aside from what you usually do in JC world, a constructor-based dependency injection with IoC container in framework.

There are many ways to make code testable in other languages. Dependency injection isn't the only option and even with Go, dependency injection isn't necessarily about the constructor.

So the first thing I suggest JC developer unlearn is the fixation toward dependency injection and IoC container. A good code should be testable and it's true that in JC environment, you should use always use dependency injection to write good code. That's the only way in JC.

However, when you start learning other languages, you can write testable code without them.

When JC developer learn new language, I suggest to stay open-minded about testing approach. I know many were thought that "you must use DI and never instantiate anything directly because you need to be able to test your code". Again, it is 100% correct for JC but not necessary true in other languages.

I'm not saying that you should never use IoC container in another programming language. I'm saying that IoC container and constructor-based dependency injection is not the only one and true way to achieve testable code. JC developer would become much better programmer if they are open minded about this.

JC developer perspective can be greatly broadened if they let their enterprise one-and-true-best-practice go while learning a new programming language paradigm.

In the end, if they find that the IoC container is the best solution because of any other factor outside of testing, then please use it. Go ahead.

What I've seen is many JC developers throw all other approaches out of the window because it doesn't conform to what they learned as best practices. As a result, their perspective become narrow and they did not learn anything from a new programming language. Such a pity, I would say.

I have a mixed feeling in how many TypeScript framework adopt IoC container approach. I understand it helps JC developer to be at ease with TypeScript, but at the same time I always think it is unnecessary if all you want to do is just write a testable code.

Stay tuned for part 2: Everything don't need to be an object.

Simplicity fallacy of "Just use X (for everything)"

Chakrit Likitkhajorn — Wed, 14 Dec 2022 02:21:24 +0000

For the past few years I found one commmon fallacy trope.

"Stop introduce more technology. Just use X for everything. It will be much simpler."

This trope can range from Postgres, Make, HTML-CSS-JS and so-on.

While I think there is a lot of merit in limiting amount of technology stacks in organization and I've seen many unjustifiably complicated systems consist of 10 frameworks and 20+ tech stack behind the scene just because devs want to play with new shiny toy. I also think the position of just use "X" is pretty bad in its own way.

Because, simply put, why "X" when we can also use "Y" for everything as well.

Take Postgres for example: technically speaking we can use Postgres to build queue, to do a full-text search and possibly Redis replacement according to this article. But one thing that I might argue: Why Postgres over, let say, SQL Server or MySQL or just plain old boring file?

There might be some technical answer to this question. But I guarantee most of the times it's because the speaker is proficient in Postgres.

Implementing queue in Postgres is not that easy and require a lot of experience to nail it right. I would bet using Postgres as Redis replacement would have even more edge cases and concern.

In the eye of Postgres expert, it is simple. In the eye of everyone else, it's super complicated.

And this is where the fallacy lie in. Someone who familiar with X will say that using X for everything is simple because you just need to learn X. And they might forget how it took them years of experience until one start to find X simple.

C++ can also be very simple for someone btw.

I know someone who advocate for using Makefile for everything infra relate because he is so proficient at it. He can't see why we need to embed our script into package.json or using few lines of Python/Go for cli task. Then I know someone who advocate for using Python for every cli task because "It's simpler. Everyone know about Python right?".

And I can imagine how they will fight and pointing finger to each other for "introducing needless complexity". Why don't you embrace simplicity and just use X for everything?

I agree with general concept of limiting technology stack. Sadly most of the times it isn't really about general concept of simplicity, it's about using technology that the advocater familiar with.

Here is a litmus test. If you advocate for using X for everything because simplicity then consider this. When your team said that Y can achieve the same and everyone else except you is more productive with Y, will you change your mind and advocate for using Y for everything instead? Will you drop your expertise on X and advocate for Y for sake of simplicity? (Assuming Y can achieve this technically speaking)

In general, is it really about priniple of simplicity by using one technology or is it about using what you already familiar with?

I usually agree with principle of simplicity but I find it is very rare to see true believer. It's usually more about just get back and use what I learned for the past 10 years rather than simplifying technology stacks.

Also, we can technically use C or Assembly for everything. Radical simplicity has its own limit. What grind my gear is how people usually draw the line. It's almost always like what I am good inherently simple and what I don't know is needless complexity since I can use what I know to achieve the same.

Modified from Douglas Adam quotes, it's almost like this:

Anything that is in the world when you’re start your tech career is normal and ordinary and is just simple technologies we all should rely on.
Anything that’s invented between when you’re first to ten years of tech career is new and exciting and revolutionary and have so much potential.
Anything invented after that is a blatantly complex technology created by clueless kids who just reinvent the wheel.

Moral of story: If you believe in simplicity of using fewer framework and tech, don't stick with just technologies that you know. Otherwise, you aren't into principle that much to begin with.

TDD Misconception: Yes, you can write code before test in TDD

Chakrit Likitkhajorn — Sun, 24 Apr 2022 10:29:04 +0000

I start a title with a controversial statement. Yes, you can write code before a test in TDD.

After I pair with many TDD practitioner, especially the beginner one. I found that many of them usually discourage me to write any code before test. I could not touch a codebase, keyboard or play around with anything before I write a test case.

So here I am, making a statement as a TDD practitioner too. You can write a code before test in TDD.

It is clear that any TDD tutorial and content says that you need to write tests before code. Why do I claim this?

Let me explain.

The step zero of TDD: design

When we normally talk about TDD we usually just talk about the test-code-refactor cycle. We often forget or take a pre-requisite step for a grant.

We need to design the code we want to write first

We cannot to TDD without designing an interface, class or function. We need to figure out where will we put a new code first. Is it in an existing method? Is it in a new class? Is it in a new method?

Without this step, we cannot even write a test. At least we need to have method(s), procedure(s) or function(s) in our mind to write a test.

One thing why people advocate for TDD is because TDD encourages good design. You need to put some thought into where would you put the implementation, what should be an interface and all the design stuff. You need to make a decision on the interface in order to write a test before a code. And that's why generally speaking, TDD encourages good design. (This topic is still in a heated debate. Some claimed that TDD hurt the design.)

The full TDD cycle is not just test, code and refactor. It's actually a design, test, code and then refactor. There is a zero-phase involved, design how and where your implementation should be.

Coding can be part of designing

There are many ways to come up with an interface design. You can think in your head. You can use a notebook. You can use a whiteboard. You can use a virtual whiteboard. There is no limitation on how would you design your interfaces.

And one approach to the design is to write some POC (aka. throwaway) code.

When I am not confident about having all the details I need to design an appropriate interface for a business requirement, writing a code (maybe working one or not working one) can help a lot in surfacing missing details.

The act of coding can help in figuring out the design. And no rule in software engineering states that the design must not involve any act of coding. And if someone tries to declare and enforce this rule, it will be an obviously stupid rule.

The point here is that in TDD, you can code as much you want if it helps you discover an appropriate design. TDD only says that any production code must have tests first. Not any type of code.

So the misconception here is that you cannot touch your keyboard and type any code before writing a test. You can, as long as it is not a production code.

The key here is that the code generated in this design phase is to discover an appropriate design. It is a throwaway code.

You should not expect to use code generated in this phase as a production code. Well, you can copy some of it into production code but you need to treat it as some kind of code in the documentation, Google or Stackoverflow. You generated a test and a blank method first, you wrote some code that satisfied the test. Some of the code might be copied from some kind of documentation with few modifications.

You can treat throwaway code this way instead of trying to use the throwaway code as much as possible.

Don't fall into the sunk-cost fallacy here. After the design, the test-code-refactor cycle still remains.

Dogma about not writing any code

Looking back, somehow many TDD practitioners have an idea that we need to constrain ourselves to design an interface within only our mind and somehow in the end this will result in better architecture.

Duh, right?

I think sometimes we take TDD practice too literally and we lost sight of how this concept actually works. The underlying idea of TDD is not about writing a test first. The underlying idea of TDD is that if you can't think of a way to test just your new implementation, you might design it wrong.

That is why we are forcing ourselves to write a test first. We want to force ourselves to think about how to test as opposed to "it works now. the test can be an afterthought".

Test-driven development offers more than just simple validation of correctness, but can also drive the design of a program. By focusing on the test cases first, one must imagine how the functionality is used by clients (in the first case, the test cases). So, the programmer is concerned with the interface before the implementation. This benefit is complementary to design by contract as it approaches code through test cases rather than through mathematical assertions or preconceptions. Source

And that is what actually makes TDD generally lead to a better code architecture. You can argue about how TDD can lead to a bad design all you want but one thing you need to admit is that TDD generates testable code with a lot of testable units. Always.

You can write some code before tests in TDD, but only as a design tool and without any expectation that the code written in this phase should be used in production. You don't need to unnecessary constraint yourselves to put down the keyboard and forbid any code writing activity during the design phase just because someone says in TDD you need to write tests before code.

Let us try not to take software development practices too literally and dogmatic (and disguise this short-sightedness as "disciplined"). Let us try to be more understanding of the underlying concept of software development practices.

CRUD is simple until it's not

Chakrit Likitkhajorn — Sun, 16 Jan 2022 09:17:58 +0000

Every app is just a glorified CRUD app with extra steps.

There are a lot of debates happening in the dev community between using complex architecture (such as CQRS) and viewing everything as a CRUD.

I want to clarify when CRUD is simple enough for the job, and when it is not.

Let say you want to create an accounting system. You have an invoice. Invoice can be created, read, updated and deleted.

Sounds like a CRUD app.

So you start with a simple Rails-style framework with REST-style API surface, where everything is just a resource that can be CRUDed.

The code can look like this (simplified version)

def update
  invoice = Invoice.get(request.body[:id])
  invoice.update_by(request.body)
  invoice.save()
end

Fit everything to CRUD

Let say that your invoice can be approved. You added status field to the model. The status can be Draft, Confirmed, Approved, Commented and Rejected.

And then you some business requirement

When an invoice is approved, send a notification to someone so they can proceed to pay.
When an invoice is commented or rejected, send a notification to the creator so they can be revised.
When a confirmed invoice has a total of more than 100,000 USD, send a notification to the director level.

If we still model by CRUD, we need a code that looks like this:

def update
  invoice = Invoice.get(request.body[:id])
  old_status = invoice.status
  invoice.update_by(request.body)
  invoice.save()
  new_status = invoice.status

  # Send notification to payment department if status changed to approved
  if old_status != :Approved and new_status == :Approved
    Notification.send_to(:payment_department)
  end

  # Send notification to invoice owner if invoice was commented or rejected
  if old_status == :Confirmed and [:Commented, :Rejected].included?(new_status)
    Notification.send_to(invoice.owner)
  end

  # Send notification to management board if invoice exceed particular amount
  if old_status == :Draft and new_status == :Confirmed and invoice.total > 100000
    Notification.send_to(:C_level)
  end
end

So now we can generalize everything as CRUD.

Six months later, we want to add functionality to the invoice approval step.

We want to ensure that any invoice exceeding 100,000 USD must be approved by someone with a proper level of authorization
We want to lock any invoice that has been rejected more than 3 times.

If we wrote code as CRUD, to implement these requirements you must:

Carefully read through update hoops and see which line is related to which.
If 2 programmers work in parallel, they might need to resolve to merge conflict.

Use business intent

Another type of implementation is to put a business intent into our model.

Instead of update, we have these:

def approve
  invoice = Invoice.get(request.body[:id])
  invoice.status = :Approved
  invoice.save
  Notification.send_to(:payment_department)
end

def reject
  invoice = Invoice.get(request.body[:id])
  invoice.status = :Reject
  invoice.save
  Notification.send_to(:invoice.owner)
end

# And so-on

Instead of translating business intention such as approve to be like: approval is just an update with extra steps. We instead embedded business intention directly into our method.

When the business what to modify the approval process, we can search for approve method.
When the business what to modify the confirmation process, we can search for confirm method.

Even if this contains more code, it's become easier to modify and reason about.

This is the power of moving away from CRUD based modelling.

It's not black and white

CRUD is a simple model. There are only four concepts and it can handle almost every requirement imaginable.

Everything can be viewed as "CRUD with extra steps". Even network communication can be viewed as "CRUD of network packet with extra steps".

It is a simple and powerful model. But it has its own limitation.

CRUD based method works well until collaboration break.

You can fit nearly every app into the CRUD model, but at what cost?

The question is here is not can you, it's should you.

If your software grows based on some particular business process, such as if your accounting module is famous for a world-class approval system, it's worth you putting a highlight to those business intent rather than generalize it to "just CRUD with extra steps".

I don't have a clear line between when to model the app as CRUD and when to do the domain event-based app.

But I know that if communication starts to break down. If a requirement needs 2 days of dev assessment just to translate the requirement into codebase modelling, check where to change, what is the impact.

Your model might not be sufficient.

And that's when you might need to move on from the simple CRUD model.

Common misconception of dependency inversion

Chakrit Likitkhajorn — Sun, 26 Dec 2021 08:37:22 +0000

Dependency inversion is widely misunderstood. I will explain why.

According to the definition: the principle itself consists of two rules

High-level modules should not depend on low-level modules. Both should depend on abstractions.
Abstractions should not depend on details. Details should depend on abstractions.

In this context, high-level modules mean domain logic and low-level modules mean technology stack. For example: If you are working on an accounting system. All tax formulas are high-level modules, and the database which you fetch invoices and receipts to calculate tax refunds is a low-level module

You can imply that the gran idea is to decouple business or domain logic from the technology used. If you look back into Robert Martin (the originator of the principle) works on software architecture, you will find that the idea of decoupling business logic out of the technology stack is always his theme. If you want to have a quick look: here is his idea of Clean Architecture.

Common understanding

Back from the idea world to the actual world. Technically speaking, every system heavily depends on a low-level technology stack to perform.

For example: If a user want see an amount of taxes for invoice number INV001, here is what you normally need to do

Fetch invoice INV001 from the database
Apply tax calculation
Send the calculation result to the client, probably via HTTPS protocol and internet

You can see that we depend on database implementation, HTTPS and the whole internet to work exactly and consistently as we expected to implement this feature.

Let say you have to implement a method CalculateTax(string invoiceNumber), you need to at least be able to fetch invoice data from the database. How can we make this "de-coupled" from the database implementation? At a first glance, this seems to be impossible.

The common understanding is that you can achieve this by using a technique called dependency injection. Instead of depending on concrete implementation

There are many frameworks out there such as Spring or C# MVC which implement something called dependency injection framework and IoC container. So we can globally register services like this:

using ConfigSample.Options;
using Microsoft.Extensions.DependencyInjection.ConfigSample.Options;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddScoped<IInvoiceRepository, InvoiceRepository>();
builder.Services.AddScoped<IMyDependency2, MyDependency2>();

var app = builder.Build();

and this is where I think the idea become misunderstood.

Let's get back to our tax calculation on invoice example, let say you have

public class TaxCalculator {
  private IInvoiceRepository invoiceRepository;

  public TaxCalculator(IInvoiceRepository invoiceRepository) {
    this.invoiceRepository = invoiceRepository;
  }

  public double CalculateTax(String invoiceNumber) {
    Invoice thisInvoice = this.invoiceRepository.getById(invoiceNumber);
    // Calculate tax
  }
}

At a first glance, this seems to be a normal dependency inversion principle implemented using dependency injection and IoC container. We seem to be able to decouple a concrete database implementation out of tax calculation by making TaxCalculator depending on just an interface.

I want to step back from the code and get back to the main idea. All the main idea of this dependency injection and framework stuff is to make high-level modules independent of low-level modules right.

My question is: What does it mean to be independent?

What does it mean to be independent?

Well, it depends.

To answer this question in a precise manner, I need to bring an extreme situation that will make the dependency graph clear: High bureaucracy, low-trusted environment.

Let say there are two vendors binding by just contract and neither of them is willing to expose their code. One is working on a class TaxCalculator class, which is a high-level module. Another one is working on InvoiceRespository class, which is low-level module.

Let's call the first team as Tax team and the second team as Repo team.

In the spirit of dependency inversion principle, the Tax team should be independent of the Repo team while the Repo team can still depend on the Tax team, right? That's mean any changes from the Repo team should not affect the Tax team, but some changes from the Tax team might affect the Repo team.

That's one way to make the dependency graph clear.

In a real Java or C# codebase, we usually see the repository interface that looks like this

interface IInvoiceRepository {
  Invoice getById(String invoiceId);
  List<Invoice> findAllByUserId(String userId);
  List<Invoice> findAllByEmail(String email);
  void AddInvoiceItem(InvoiceItem item);
  // And more methods
}

This is a clear signal that the interface is not owned by the Tax team. First of all, the interface has many methods unrelated to what Tax team need. Second of all, according to my experience, the interface changes is likely to be dictated by either database implementation or global system requirement.

Does the Tax team is independent and decoupled from database implementation, in this case, the Repo team?

I highly doubt that.

Assuming that dependency inversion is about making high-level modules independent of low-level modules, it safe to say that just using dependency injection and IoC container that the most famous framework provided does not automatically make the codebase achieve dependency inversion principle yet.

This is where I believe most of the misconception happens. Many people see that dependency inversion must be done using dependency injection (which make sense). The common implementation of dependency injection is using a global IoC container. Hence, most people believe that by just using Spring or C# MVC IoC container, they achieve dependency inversion.

Understandable, but not true.

Real world is messy, but we can make it better

At this point, you might argue that it is impossible to make the Tax team truly independent of the technical implementation of the database. And I agree.

Let's say Tax team want to have a GUID-based invoice id but the database only supports an integer-based id, what realistically can we do.

Many hard technical limitations make it impractical to be idealistic and dogmatic about the dependency inversion principle. Some tax calculations might be impractical to do without the help of database-level aggregation.

However, we can still be better by simply asking ourselves

How about we let Tax team decide what interface they want?

namespace TaxModule {
  interface IGetInvoiceById {
    Invoice getById(String invoiceId);
  }
  public class TaxCalculator {
    private IGetInvoiceById getInvoiceById;

    public TaxCalculator(IGetInvoiceById getInvoiceById) {
      this.getInvoiceById = invoiceRepository;
    }

    public double CalculateTax(String invoiceNumber) {
      Invoice thisInvoice = this.IGetInvoiceById.get(invoiceNumber);
      // Calculate tax
    }
  }
}

Hence, the Tax team and TaxModule defined what interface they want instead of relying on either.

It's a matter of who owns the interface.

If high-level modules own all the interfaces. This means high-level modules dictate changes, and low-level modules need to adhere to what high-level modules define.

This makes high-level modules truly independent of changes made by low-level modules or global requirements.

This is better aligned with a vision of dependency inversion.

And in case the Repo team need to change the interface because of technical limitation, they need to inform the Tax team and ask for them to neither change their interface nor implement some kind of anti-corruption layer. Collaboration still needs to happen. But at least, all Tax team objects depends on the code in TaxModule, including all required interfaces. The team become more independent.

Simply put:

I suggest that to truly invert dependency, we should let high-level modules define required interfaces instead of global interface or worse, interface dictate by low-level code.

Language limitation

The problem with this approach is that, sadly, the global interface approach is endorsed by IoC container design of the most common framework in practice.

If you are using Java, Kotlin or C# we don't have type-inference.

interface IGetInvoiceById
{
    Invoice getById(String invoiceId);
}

interface IInvoiceRepository
{
    Invoice getById(String invoiceId);
}

In this case, the interface IInvoiceRepository cannot be used as IGetInvoiceById. What we can only do is to have

interface IInvoiceRepository: IGetInvoiceById
{
}

And in a large system, you can imagine an explosion of small interfaces

interface IInvoiceRepository: IGetInvoiceById, ISetInvoiceId, IAddInvoiceItem, IGetInvoiceByUsername // ... and 8 more
{
}

This is where I think GoLang did a good job on type-inference and defining Go interface philosophy.

Go interfaces generally belong in the package that uses values of the interface type, not the package that implements those values. The implementing package should return concrete (usually pointer or struct) types: that way, new methods can be added to implementations without requiring extensive refactoring.

Anyway, we need to be aware of a language limitation and why are we not doing this yet.

As a polyglot developer, I believe we need to be aware of both sides of the coin

What the ideal principle looks like, and how is it helpful?
What is the language and framework limitation. Is it practical to adhere with the principle to the letter based on the limitation we have?

There are two extremes stances. One is to dismiss the purity just because of current limitations ie. "Stupid. We don't do that here.". Another one is to be a purist and make a codebase become an unmaintainable mess.

Both stances are unproductive.

In Golang, I would obviously adhere to the principle of letting high-level modules define required interfaces instead of global interfaces.

In C# or Java in a current version, I might not. In the future, who knows?

Last note

I think the wide misconception occurred because the famous concrete implementation of DI is to use IoC container.

But that's just a form, born out of many limitations and legacies.

The principle itself should be language independent.

The implementation still depends on the language and framework you use.

You cannot understand principle from the eyes of just "C# Developer", "Java developer" or "React Developer". You need to understand what's out there in the programming world.

And you might disagree with the principle. But at least, to be a better programmer, you need to understand

Ideal situation of programming. Such as principles,
Current state of programming. Such as technical limitations.

Adhere to the ideal, you become a purist. Adhere to the current state, you become stagnant.

What I try to say in this article are

Current implementation of Dependency Inversion principle on some famous frameworks is not really inverting dependency.
Ideal situation is to make high-level module code does not affect by low-level module change at all.
We might get closer to the ideal once we have a better language or framework. For example, type-inference is one killer feature that I hope to have in enterprise-focus programming languages.

That's one way to consolidate between the ideal and the practical side of programming.

That's all for today. Thanks for reading this far!

Start with Microservices. Don't start with Microservices

Chakrit Likitkhajorn — Sat, 23 Oct 2021 04:37:56 +0000

This article has an oxymoron title.

I want to tell two stories. One which makes a case for don't start with Microservice, one for start with Microservice. And I believe by looking from both side of thing, we will understand more about what is the actual benefit of Microservice.

So, let's get start.

Don't start with Microservice

Imagine you are working for a big e-commerce platform. And like all e-commerce platform, there is a product list page which show all the product. There is also a checkout page which handling a shopping cart and a checkout process.

A long time ago, John, CTO of this company read something about Microservices. John thought that this is a great idea. So John decided to have a Microservice which responsible for the Product list domain, and Shopping Cart domain.

The microservices architecture allows John to have a product list team and a checkout team. Both of them have their own business objectives, metrics, KPI, SLA and so on. Both teams are cross-functional teams responsible for the product list page and checkout process end-to-end.

Each of them can go wild with what they want to do. For example, the checkout team can implement their own AB Testing with multiple localization and a nice holiday theme on top of it. The product team can focus on optimizing the product list by suggesting relevant items. Both of them can focus on their area without being afraid of making breaking changes to other areas, as long as the backward compatibility of API communication between domains is maintained.

Furthermore, we can partially scale the system. Let say we have a big Friday promotion, the customer will browser products and promotions a lot more. We might need a much bigger Product list backend while a little bit bigger checkout page.

Perfect sounds good. Life is nice.

Next year, Jane the CEO want to have a promotion sales. Jane also figured out from some research that if show relevant products on the landing page and during the checkout process can significantly increase sales.

There are some specific pricing scheme, relevancy (a customer who buy a lot of IT-related items get more discount on IT items), and customer privileged classes. These should be managed by the product list team since they have done much research on how to display relevant attractive products. Also, the product list team know exactly how to show the product in an attractive manner. They know exactly what color, font, image size and margin yield the best conversion rate.

However, this change going to be tough.

Since both the backend and the frontend for the Checkout process and product page belong to different microservice, so we need to share code, design, logic, etc. between both teams. We can do a simple copy-paste, we can create a shared library, we can expose some kind of API.

While those are all technically possible, implementing this type of change in this microservices architecture required significantly more effort compared to a monolith architecture.

What if we need to iterate the design, data, and everything in promotion? Every change in each iteration must be reflected on both the checkout page and product list page. If we don't do it right, we will need to double all the work or keep redesigning API between these two teams again and again.

Once we use microservice architecture, we essentially make any change within service become easier, while a change across services become harder.

Since we as a human are bad at predicting future, we should never start with microservices. We don't know what business going to expect from us 6-12 months down the line. However we split microservice, it is likely to be wrong.

Let's never start with Microservice.

Start with Microservice

Let's rewind back and see how things can go differently.

John, CTO of this company read something about Microservices. John thought that this is a disastrous idea. Everything Microservices technically solve can also be done with Monolith, so why add distributed part into the equation? Most of our programmers will suffer from the fallacies of distributed computing.

John insisted on using Monolith architecture.

John has a team of around 30 developers. At this scale, it is not possible for John to take reviewed every single change to his big monolith. So he checked the code once in a while.

The company took the opposite direction: They want to implement a single Buy now!! button, which essentially is a single-step checkout.

Since this is obviously on the Product List page, so stakeholders ask the Product List team to develop it. One of the developers looked at the requirement. They thought that since this page is all about the product, we should implement it in a product class like this.

class Product {
    public void CheckoutWithOneClick() {
      this.PaymentService.Pay(this.Price);
    }
}

From a seasoned developer perspective, this is a very very strong code smell. Somehow, this passed through code review, got check-ins, and become the main part of the monolith.

Six months later, everything relate to checkout become really hard to implement and buggy. If we want to be able to apply promo code, we need to remember that there are one-click checkout implemented in a product class. If we want to provide a discount for a special class of customers, we also need to remember this as well. There were many instances where the checkout team was asked to implement some sophisticated checkout scheme, and it does not work well with this one-click button.

Feature development, bug fixes and everything related to the checkout process was implemented separately. And since Product Team own the Product class and Checkout team own the CheckoutService, each team interpreted requirement a little bit differently and implement business logic a different way. One might check for promo code first and the other might check for customer privileged first and so on.

John saw this once it is too late. There are many code duplications and separate logic branches between CheckoutWithOneClick and the normal checkout process. It would take quite a significant investment to refactor this piece of code.

At some point, John was fired and Jane hired a new CTO.

The first question Jane asked was: Why is it so hard for us to do anything in the checkout process?

New CTO: Well, there is a different logic implemented in one-click checkout and normal checkout. This makes it hard to do anything in the checkout process.

Jane: How did this happen?

Random Engineer in the team: Everyone know about this. It's started by one of the developers decide to have separate logic for one-button check-in. That's bad, but I don't know how John could do it better. He could not possibly check every piece of code.

New CTO: Well, John should have started with Microservice. That way, the product list team would never be able to come up with this design. They would be blocked by Checkout team service ownership.

The value of Microservice

Well, John was in a weird situation. Doom if he used Microservice, also doom if he didn't.

Because essentially, Microservice is a management tool.

Microservice prevent some type of changes and enable some type of changes to be easier.

In the first example, the Microservice architecture prevents desirable changes. In the second example, the lack of Microservices architecture enables an undesirable set of changes.

That is why John is doomed either way. The problem is not about Monolith vs. Microservice. The problem is about architecture choice misalignment with business growth.

Well, you can argue that in the second story John can work harder and make sure all the code have design review and throughout audit and etc. But that’s the point. That kind of management solution can be costly and add a lot of overhead into the development process.

Microservice can reduce that cost by making it inherently hard to mess the design up so we don’t need all those heavy processes. But when we actually want to mess the design up for a good reason, well… you got the scenario in the first story.

In the end, the architecture must align with business growth. That is all to it.

When you use microservice, you enable some set of growth to be easier and simultaneously disable some type of growth to be harder.

If you have a checkout microservice, the checkout team would be able to churn out new features much faster. They can release on their own. They can use a programming language they prefer. They don't depend on the other team. They can manage the scalability of their service independently.

That is, what I believe, the core benefit of microservice architecture.

Every technical aspect of Microservice, such as independently scale some part of the system or using multi-language implementation, can be solved using Monolith. You can use a build and configure it to open a specific endpoint for some containers and there you go independently scaling just a part of the system. You can use cross-process communication at the OS level and manage binary distribution and there you go, multi-language implementation. The latency might be even better than the network.

What Monolith cannot solve is when you want to have a certain type of management structure and incentivize growth and creativity in a certain area by giving autonomy and a frequent independent release cycle. That is what really really hard to do in Monolith architecture.

So I would say the deciding factor for Microservice implementation: Is it aligned with how your organization want to grow?

Because again, Microservice essentially is a technical solution to the management problem. And it is a real problem that requires a good solution.

If you think it is a stupid management problem, well I invite you to try solve that. If you can solve just a simple question of "how to make programmers collaborate effectively?" in a scalable and repeatable way, I believe you have the potential to be richer than Jeff Bezos.

Amazon has been working on this. Google has been working on this. Facebook has been working on this. Every behemoth tech company in existence have been working on this for decades and they came up with Microservice stuff. If you have a better answer, then go ahead, let me and all those guys know. They are ready to invest.

Is this too complex?

Chakrit Likitkhajorn — Sun, 25 Jul 2021 12:46:57 +0000

I made a comment in Reddit and I want to blog it here as well. Because my sentiment toward someone claim that we have unnecessary complexity usually goes like this.

Disclaimer: All I want to say here is more often than not, people deemed system to be "too complex" based on their familiarity and skillset. And I think that is very unfair.

Re: Back to the '70s with Serverless

Original reply

I don't really understand how managing VPS can be simpler than Serverless. What is the definition of complex?

I can deploy my apps into Serverless within few minutes. And with VPS, I need to ensure I have the right version of Linux, dependencies installed with correct versions and proper resources (CPU, Ram, Disk) + uptime monitoring.

I can manage VPS dependencies and stuff with some tools like Ansible, but how it that provide a better feedback loop? It's so slow compared to Serverless.

If you are already familiar with all Linux commands, concepts, and toolsets, everything I mentioned will be easy and "feel simple" to you. But that is easy, not simple. And it applied only to you.

Even if we don't go with Rich Hickey's definition of simple, you still need to take a layman's perspective if you want to claim a system is easy. I would like to see how fast junior developers can get VPS up and running with the same quality as Serverless.

Sometimes I feel like an old programmer complaint goes like this:

Senior: Back in the day, everything was simpler with stack X
Junior: That is interesting. Let me learn X.
Senior: You need to read this big book, and it takes a year of experience
Junior: Wait, but how can X be simpler than new shiny toy A that I can learn and get it run within 10 minutes?
Senior: You lack fundamental young Padawan. Once you dig into X long enough, you will understand how good X is.
Junior: Wait, are we still talk about how X is simpler than A?
Senior: Kid these days......... so entitled and soft.

Maybe this is a classic case for the (curse of knowledge)[https://en.wikipedia.org/wiki/Curse_of_knowledge].

By the way, I am also a fan of Erlang/Elixir ecosystem, but I strongly disagree with "running Erlang cluster is simpler than Kubernetes". For a start, there are very few resources online. I figured out how to do Elixir code hot-reload using edeliver and I cannot even find out how to do some simple stuff until I dig into some kind of hidden Webboard and mailing group.

Funny enough end, I found it easier for someone like me, without years of OTP experience, to run Elixir apps in Kubernetes cluster and achieve hot-reload from K8S stack. Of course, this is conceptually sucks since I am fully aware that OTP is much more potent and powerful than Kubernetes in many things. But in the end, I have some apps to deploy.

I want to point out that the resource I found when I googled "Hot Code Reload Deployment Elixir" ended up recommend me to not fully rely on it. So while I agree that Erlang and OTP is technically way more powerful and more amazing than new shiny toy such as Kubernetes, I strongly disagree with it being simpler.

Toward solving interruption in Programming

Chakrit Likitkhajorn — Sun, 11 Jul 2021 10:44:07 +0000

Recently there are developers and creative vocal about how interruption ruin their work.

There are many articles about flow state and funny cartoon on how interruption ruin serious thinking work. There is even a passive-aggressive piece talking about teaching non-geeks cost of interruption.

As a developer, while I agree that interruption can ruin work to some degree (which differ for each people), I don't think the passive-aggressive stance most programmer take is productive nor helpful.

First of all, according to my experience in many companies both as a full-time developer, management and consultant, I want to say this out loud:

No one really want to interrupt your work

There is no conspiracy where every non-geek role held a secret meeting to figure out how to make programmer life most miserable and decide that throwing non-sense interruptions every hour is the best course of action.

No, that is not the case.

Every interruption has its own reason. Your teammate has some problem they want to solve, and they think you can help.

That's why they come to you. So tell people to stop interrupting will not work, especially if the problem is a shared problem. For example, what happens if sales are about to commit to some scope but don't know the feasibility? Would you want sales to stop interrupting the programmer and figure this out on their own? That obviously will make the matter worse.

There are some necessary interruptions and some not. You cannot just tell people to leave you alone to solve the problem. After all, you don't get paid to solve your personal problem. It is always a shared problem. Everyone in the team is at stake. Cutting interruptions and work in a silo is not a solution.

I believe the solution is to let them know how to interact with you. You have to set your own boundary.

For example, you can say if anything urgent, please call me instead of a message. You can say that if you leave a message, I will read it within a day. If you ping me directly, I will respond every hour. You can also block your flow time on the calendar.

Those are all valid boundaries to set.

And at first, people might mistake the urgency of the situation. Some people might call you on the tiny issue they have. That is fine if it happened the first time. You take that opportunity and categorize this issue. You tell them that next time this type of issue is not as urgent as they might think. If it happens repeatedly, we have to talk with a team about urgency and tradeoffs since interruption has some cost.

In short, rather than complaining about how others are wrong, it's better to tell what's right. We should let others know how to interact with us productively.

And there might be some constraints that we don't know. For example, we might legally bind to report production issues within a specific amount of time. We can work together to solve that constraint.

But first, you need to believe that people are not out there to get you. As I said, there is no conspiracy of non-geeks. Everyone, geeks or non-geeks, just wants to get the best results. We might have a different opinion on how to get it. That's true. They have a reason, strong or weak, they have a reason.

And shutting others of is not going to solve anything. It is a bad stance to take on the interruption problem. It is much better to take a stance of

I understand why you need to interrupt me. However, this is not productive for all of us. Can we do it differently?

And that is my rant for today.

บ่นโค้ดใน Smart Contract ของ Pancake Bunny

Chakrit Likitkhajorn — Thu, 20 May 2021 05:59:39 +0000

อันนี้จะบ่นสั้นๆ คือวันนี้อยากรู้ว่า Pancake Bunny มันพลาดตรงไหน แล้วเห็นโค้ดชุดนึงที่น่าพูดถึง

คือออกตัวก่อน เวลาเขียน Smart contract มันจะมีเรื่องค่า Gas ซึ่งมันมีผลให้การเขียนโค้ดแบบปกติอาจจะสิ้นเปลือง จุดนี้ผมยังไม่เข้าใจมาก และที่เขาเขียนมันอาจจะดีที่สุดในโดเมน Smart Contract แล้วก็ได้นะครับ

แต่ถ้าเราเอามาเขียนในภาษาอื่นๆ ที่ไม่มีค่า Gas อย่าลอกเขามาทื่อๆ นะ

จากภาพนี้

(ต้นทาง https://peckshield.medium.com/pancakebunny-incident-root-cause-analysis-7099f413cc9b)

ขอลอกส่วนนี้มา

if (IPancakePair(asset).token0() == WBNB) {
  valueInBNB = amount.mul(reserve0).mul(2).div(IPancakePair(asset).totalSupply());
  valueInUSD = valueInBNB.mul(priceOfBNB()).div(1e18);
} else if (IPancakePair(asset).token1() == WBNB) {
  valueInBNB = amount.mul(reserve1).mul(2).div(IPancakePair(asset).totalSupply());
  valueInUSD = valueInBNB.mul(priceOfBNB()).div(1e18);
}
// ต่อ ไม่ได้เอามาด้วย

จาก

valueInBNB = amount.mul(reserve0).mul(2).div(IPancakePair(asset).totalSupply());

// และ 

`valueInBNB = amount.mul(reserve1).mul(2).div(IPancakePair(asset).totalSupply());

สองบรรทัดนี้เหมือนกันเด๊ะๆ เลยแค่ใช้ reserve คนละตัว แล้วพอรวมกับ if token1() == WBNB ทำให้รู้ว่าจุดประสงค์ของโค้ดนี้คือมันต้องการจะเอา Reserve ของ WBNB มาใช้ในสูตร amount * reserve * 2 / totalSupply() นั่นแหละครับ

ดังนั้นการเขียนให้เจตนาตรงนี้เคลียร์กว่า ทำได้แบบนี้

uint reserveBNB
if (IPancakePair(asset).token0() == WBNB) {
  reserveBNB = reserve0
} else if (IPancakePair(asset).token1() == WBNB) {
  reserveBNB = reserve1
}

valueInBNB = amount.mul(reserveBNB).mul(2).div(IPancakePair(asset).totalSupply());

โค้ดชุดหลังมันทำให้อ่านปราดเดียวรู้เลยว่า มันมีสูตรคำนวนแค่สูตรเดียวนะ และสูตรต้องการค่า BNB ที่อยู่ใน Pool มาใช้งานนะ ต่างกับชุดแรกที่ต้องเพ่งนานมากกว่าจะรู้ว่า "อ้อ สูตรเหมือนกันนั่นแหละไม่มีอะไรหรอก แค่ต้องเลือก Reserve มาให้ถูกที่" ผมเพ่งหลายนาทีกว่าจะถึงบางอ้อ

ผมเดาว่าที่เขาไม่เขียนแบบนี้เพราะจริงๆ มันมี Condition ที่ 3 แล้วใน Solidity การเช็ค if ทุกครั้งมันมีค่า Gas ซึ่งผมไม่เชี่ยวชาญพอที่จะบอกได้ว่าโค้ดแบบนี้ในบริบทของ Solidity ดีหรือไม่ดี อาจจะดีสุดแล้วก็ได้พอเอาเรื่องความประหยัดเข้ามาร่วม

แต่อยากจะบอกโปรแกรมเมอร์ที่เขียนโปรแกรมทั่วๆ ไปว่า โค้ดแบบนี้จริงๆ อย่าลอกไปใช้ในภาษาที่ไม่ติดเรื่องค่า Gas นะครับ มันอ่านยากครับ

Persona-based architecture

Chakrit Likitkhajorn — Tue, 27 Apr 2021 05:33:17 +0000

We developers always dream about a maintainable system. We longed for a simple system that is easy to work with. It should be easy to change, debug and observe.

We created a concept called clean code and clean architecture in order to pursue that dream.

What is clean architecture? There is a definition defined by Uncle Bob. It must be independent of a framework. It must be independent of UI, etc.

Everything seems good. Since we have a solid ground on what a clean code is, we should already achieve a maintainable codebase, right?

Surprisingly, whenever I had a conversation with developers who work with those clean architecture patterns. I asked them how it is in practice. Does the team become more effective? Do the team happy with the codebase?

The answer usually goes like this.

"Well, it is not really easy for a common developer. They need to learn multiple concepts and take a lot of time to become effective."

Wait. In a quest to chase a maintainable codebase, we accidentally created a codebase that is hard to maintain, did we?

Practical maintainability

Personally, I have no issue working with great architectural patterns such as Hexagonal Architecture, CQRS, Flux or Microservices. I learned about this pattern for quite a while.

But when I work in a team of 10 developers, does it matter if I am the only one out of nine who knows about it?

We know that a maintainable codebase must have dependency inversion, low coupling, high cohesion, follow SOLID principle, follow DRY principle, etc. Surprisingly, when we follow all the maintainable codebase rules and at the end, the team might find it hard to work with.

What happens? Are we chasing an illusion of a maintainable codebase? Are we a bunch of sheep that blindly follow the shepherd rules with false hope of promised maintainable land?

I think when we follow all those rules and theories, we achieve theoretical maintainability. Any codebase with all those characteristics should be maintainable, in theory.

In practice, I think maintainability should simply be defined like this.

A maintainable codebase is a codebase that a team can effectively work with.

That's it—such a simple definition.

All those rules about dependency inversion, coupling, cohesion, SOLID, DRY are just tools to achieve the dream. We should understand those concepts in order to create a maintainable codebase. But they should never be viewed as a goal in itself.

Is the CQRS codebase maintainable? If a team finds it easy to work with, then yes.

In the end, the maintainability of any codebase never depends on whether you do things in the right way or follow best practices. While it helps, it is not a deciding factor.

It all depends on humans. Yes, those fuzzy emotional unreliable humans.

Working with human

Once, a software architect said this to me: Software Architect is a very special role. We are responsible for the overall quality of the codebase. We know the quality of the codebase depends on developers. And yet, we don't have any power to hold those developers accountable. We can only design, convince, and maybe set some rules.

A path toward maintainable architecture is such a vague path. Since it all depends on humans, there is no correct answer. No matter what you do, you might be right, or you might be wrong.

Software engineers are familiar with working with binary. There will always be a right way. Code should either compiled or error. Code should either pass a test or fail a test. Everything should be consistent and reproducible.

On the other hand, humans are not consistent at all. Today a human can love one thing, and tomorrow they might hate it. They might say our codebase is the best codebase ever today, and in the next few months, they might want to rewrite everything from scratch.

How can we work with humans?

I would like to introduce you to two fields that are pretty much in the same spot.

The first field is the design field. Designers never have direct control over users, yet they need to design both UX and UI to guide users to do the right thing. Their career pretty much depends on the user doing the right thing while still happy with the overall experience.

The second field is the economic field. Economic planners never have direct control over the population behavior, yet they need to create a system that incentivizes people to a certain economic direction. Otherwise, the whole nation failed.

I am always fascinated by behavioral economics. How can you introduce a policy? How will people react to that? What will be the consequence?

There are many tools and ways of thinking from those fields that we can apply to software architecture. I can name many. Design thinking, User testing, Unintended consequence, Game Theory, etc. All those tools are applicable to software architecture.

And today, I will write about a tool that I steal from the design field: Persona

Persona

According to Wikipedia, here is a definition of persona

A persona (also user persona, customer persona, buyer persona) in user-centered design and marketing is a fictional character created to represent a user type that might use a site, brand, or product in a similar way

Basically, whenever designers design anything, they create a fictional character of their user. And we can adopt this practice of creating a fictional character of maintainers.

There are many ways to create a persona in product development. To get back to software architecture, I think persona of maintainers should consist of two parts:

Context. What is the experience level of our fellow developers? What is their role? What will they do on a day-to-day basis?
Expectation. Based on context and our design, what are our expectations?

The context helps you empathize with the capability, limitation, and habit of maintainers. That will be an architectural design constraint. On the other hand, expectation helps you understand what you expect from each.

For example, let say we want to adopt microservice architecture. Here is a persona example:

We might start with either the context side or the expectation side. We might start by asking ourselves: here what I want from our maintainers. What kind of people are they? Or you might start by: here are our maintainers. What can I expect? It works both ways.

Just by writing down persona, we make a connection between maintainer and expectation clearer and more realistic.

Persona testing: Context side

Once you have persona, you need to test your persona.

It is very simple. We just look at actual maintainers and see if they fit the description.

In this case, we identify the gap between the actual maintainers and our ideal maintainers. Mark doesn't have 6 years of experience, but he understands distributed systems.

Once all gaps are clearly identified, we have many choices to deal with them.

Aside from a clear choice of changing our architecture to fit the maintainers, we can also provide a training and capability improvement program. We can switch some team members around with other projects. We can hire someone from the outside that matches a requirement.

We can even set a standard and career development. We can bring your persona to all those junior and say: We want you to be this person within a year. Once you obtain these, we can talk about promotion and next step in your career.

All of the above are some examples of what we can do once gaps are clearly identified.

I hope you now see the value of writing down persona.

There is one thing I want you to be prepared. Seeing these gaps might lead to some hard conversations, and we might not feel comfortable about these.

All I can say is that this approach works much better than design for the best and pray for the rest approach. It works much better than blindly adopting microservice, and in the end, all maintainers just want to quit or kill the project.

I heard this kind of story so many times. Architecture implements fancy architecture and creates a big gap between what should happen and what actually happens.

Those gaps will not go away just because we don't think about it. So let us be mentally prepared and confront those gaps heads-on.

Persona testing: Expectation side

As we adjust our context, we still might be wrong about expectations.

For example, we might think that our maintainers can independently create new feature. We might think that a senior developer will be able to facilitate post-mortem.

In the end, we might be wrong.

When you have expectations and context written down, it opens up an opportunity to improve.

Maybe our maintainers need more training and knowledge than we initially thought. Perhaps the way we separate our domain does not enable our maintainer to contribute. Maybe our maintainers are stuck in some legacy process or legacy code that we don't know.

There are many possible causes and possible solutions. As in product development field said, we will never get our assumption right until we launch a product into the market.

And that is ok.

When we know exactly where we wrong, we can make it right.

It is better to say

I thought microservice was going to allow the team to be autonomous. It turns out that they still depend on many external services. We need to bring those services back together and rethink how we draw our domain boundaries.

than

We tried, and Microservices just does not fit us.

Also, it is better to say

I thought a programmer with 6 years of Java development experience which demonstrates capability in OOP should be able to handle this. That is where I am wrong. It seems that the Java experience is irrelevant here. We need someone who understands how the distributed system works to maintain this system.

than

We need better programmers to maintain this system. Please hire only top class programmers.

And personas allow you to say these types of sentences with confidence.

Whether the path forward is to change our architectural decision or improve maintainers' capability, it is much better to be explicit.

And it is ok if you don't optimize your architecture for maintainers, as long as you make it clear why and what do you expect instead.

With persona, you have a clear message to all stakeholders about what you expect. You can have a meaningful conversation on how to adjust according to hiring and capability limitation.

Side Benefits

Aside from a software architecture perspective, having a clear persona can be a tool to connect many aspects of software engineering management.

Hiring - What is a job description? Persona!
Career growth - How shall we set junior developer expectations? Persona!
Capability building - What kind of training should we provide this year? Persona!

I was in a position of tech lead who responsible for both management and architecture, and having a persona in mind helps a lot.

Endnote

Today, I introduce the concept of practical maintainability. And I think software architects should strive for it.

When it comes to practical maintainability, it's all about real humans rather than rules, processes, and practices.

In the software engineering field, we mainly focus on binary and concept. In some other fields, they have many tools to deal with human inconsistency. We can learn a lot from them.

To be successful in software architecture design, you rely on humans. Persona is one of many tools to help you layout dependency between your structure and human behavior. Persona testing helps you understand how your design actually works in practice.

Once you have everything visualize, you have the power to solve it.

I believe that just like every other type of architect, a software architect is supposed to solve a human problem. Architecture should serve human needs, not the other way around.

Thanks for reading!

Code quality แบบใช้ความคุ้นเคย

Chakrit Likitkhajorn — Sun, 07 Mar 2021 09:29:04 +0000

หลายๆ ครั้งผมพบว่าโปรแกรมเมอร์มักจะตัดสินคุณภาพของโค้ดจากปัจจัยเพียงแค่ว่า Pattern ที่เขาใช้ตรงกับที่เราใช้มั้ย

เช่น ถ้าสมมติเราทำโค้ด MVC อาจจะมีทีมนึงกำหนดกฎไว้แบบนี้

Controller ห้ามมี Logic
ใน View ให้เข้าถึง Model แล้วถ้าอยากแสดงอะไรพิเศษให้เพิ่มใน Model
Model เป็นที่รวม Logic ส่วนมาก รวมถึงการเข้าฐานข้อมูล
ให้ใช้ Dependency Injection ในการทำให้ Unit test ได้

ทีนี้เวลาเราไปเจอโค้ดของอีกทีมที่ใช้ Pattern อีกอย่างกับที่เราคุ้นชิน เช่น

หากต้องการแสดงอะไรพิเศษให้สร้าง Presenter หรือ ViewModel ขึ้นมา
Controller มี Logic ในการตรวจสอบ Request ได้
ให้ใช้ Stub ในการทำให้ Unit test ได้

เราก็อาจจะคิดว่าโค้ดนั้นห่วยเพราะมันไม่ถูกต้องตามที่เราเรียนมาหรือคุ้นชินมา

ผมมองว่าการใช้ความคุ้นชินแบบนี้เป็นตัวชี้วัดว่าโค้ดมีคุณภาพมั้ยที่ไม่ดีมากๆ โค้ดหลายตัวมีคุณภาพที่ดีโดยไม่จำเป็นต้องใช้ Design Pattern ที่เรารู้จัก ตัวเราเองก็ไม่สามารถไปอ่านหรือทำความเข้าใจทุกๆ Design Pattern บนโลกนี้ได้

ผมมีโอกาสอ่านโค้ดของ Candidate ทำงานกับ Legacy code มาเยอะ มีโอกาสเขียนโปรแกรมมาหลากหลายภาษา แต่ละภาษาแต่ละโค้ดเองก็มีทั้งสไตล์, Framework, Design decision, Best practices ที่แตกต่างกัน

ด้วยประสบการณ์แบบนี้ ผมไม่อยู่ในฐานะที่สามารถบอกได้ว่าโค้ดไหนแย่หรือดีจากเพียงแค่เพราะมันคุ้นตามั้ย แค่เปลี่ยนภาษาก็จบเห่แล้ว ทุกอย่างต้องลบทิ้งหมด อย่างเช่น

วันที่ผมเปลี่ยนจาก Angular มาเป็น React งี้ ผมไม่สามารถบอกได้ละว่าโค้ดที่ดีต้องมี Dependency injection ใน React มันไม่เหมาะเอามากๆ
วันที่ผมใช้เทคโนโลยี Web API กับ MVC เพื่อเขียน C# JSON API Application ก็มีความแตกต่างมากในแง่ของ Entity ที่ใช้ในระดับ API ผูกกับ Object ขนาดไหน อันนึงออกแบบบนฐานของ Coupling Entity, Response กับอีกอันคือ Decoupling เลย
ตอนผมทำ Rails, C# MVC, Phoenix ถึงแม้จะเป็น MVC เหมือนกันแต่สไตล์ก็แตกต่างกันมาก

ผมจึงอยากแชร์ว่าปกติแล้วผมดูอย่างไรว่าโค้ดไหนดีหรือโค้ดแบบไหนแย่ โดยไม่ขึ้นกับความคุ้นตาของตัวเอง

งั้นเรามาเริ่มกันเลยดีกว่า

Definition

กฎข้อแรกที่สำคัญที่สุดของโค้ดที่ดีสำหรับผมคือ

โค้ดที่ดีสำหรับผมคือมีนิยามของคำศัพท์ที่ชัดเจน และทำตามนิยามนั้น

ผมยกตัวอย่าง Rails บอกว่า Controller

Action Controller is the C in MVC. After the router has determined which controller to use for a request, the controller is responsible for making sense of the request, and producing the appropriate output. Luckily, Action Controller does most of the groundwork for you and uses smart conventions to make this as straightforward as possible.

แปลว่า Controller มีหน้าที่จัดการ Request และจัดการ Output ถ้าเรานิยามไว้แบบนี้ แปลว่าโค้ดที่เอา Controller ไปทำอย่างอื่นคือไม่ดีละ

แปลว่าผมอ่านโค้ด Rails ผมจะไม่อยากเห็น Controller ที่รู้เรื่องธุรกิจภายในมากกว่าการจัดการ Request/Response

class TransactionController
def create
  raise TooLowTransactionError if params[:amount] < 20 # บรรทัดนี้คือไม่ดี เพราะมันรู้ว่าในธุรกิจของเราห้ามสร้าง Transaction ต่ำกว่า 20 บาท
  # ....
end

ซึ่งแปลว่าถ้ามีใครก็ตามกำหนด Definition ของ Controller ต่างไปจากเรา เราอาจจะเอ๊ะในครั้งแรกว่าทำไมถึงกำหนดต่างกับมาตรฐานสากลล่ะ แต่ไม่ได้แปลว่าโค้ดไม่ดีนะครับ

สมมติใครซักคนกำหนดว่า Controller มีหน้าที่ประสานงานระหว่าง Model ในกรณีที่ API นั้นต้องใช้มากกว่า 1 Model แล้วเขาเขียนตามนั้น นี่ก็คือถือว่ายังเป็นโค้ดที่ดีในแง่ของ Honest to definition อยู่ครับ

ส่วนการกำหนด Definition แบบนี้มันเหมาะหรือไม่ จะทำให้ Controller บวมเกินเหตุมั้ย อันนั้นอีกเรื่องนึง ต้องไปดูตามบริบทของระบบที่พัฒนาอีกทีนึง

หรืออย่างเช่นใน MVC แบบ Rails ตัว Model จะเข้าถึงฐานข้อมูลได้ ก็จะเขียนบอกไว้ใน Definition ของ Model แต่ถ้าเราเริ่มทำ Hexagonal Architecture เราอาจจะมี Repository ที่เข้าถึงข้อมูลอีกที ไม่ให้ Model เขาถึงฐานข้อมูลโดยตรง

ทั้งสองอย่างผมมองว่าไม่ได้มีอันไหนเหนือกว่าอันไหน ขึ้นกับบริบทของระบบ

แต่สิ่งที่ดีเสมอโดยไม่ขึ้นกับบริบท คือคุณมีนิยามที่ชัดเจนว่าแต่ละคำศัพท์แปลว่าอะไร และคนร่วม Contribute เข้าใจตรงกันไม่สับสน สามารถทดสอบได้ง่ายๆ คือ สมมติพูดถึง Controller ถ้าผมเป็นสมาชิกใหม่เข้าไปถามว่า สิ่งนี้ควรอยู่ใน Controller มั้ย ทุกคนควรจะตอบได้ตรงกัน

สิ่งที่แย่โดยโดยไม่ขึ้นกับบริบท คือ นิยามที่ว่าแต่ละคำศัพท์คืออะไรมันกำกวม คนร่วมทีมอธิบายไม่ได้ว่า Model แปลว่าอะไร Controller แปลว่าอะไร ไปถามเจ้าตัวคนเขียนคนแรกก็ตอบไม่ได้ชัดเจนว่านิยามอะไร บอกได้แค่ประมาณนั้นอ่ะประมาณนี้อ่ะ แต่ละคนในทีมตอบไม่ตรงกันซักคน

อันนี้ผมมองว่าแย่เสมอในทุกบริบทของระบบ

Consistency

ต่อเนื่องจากนิยาม ความคงเส้นคงวาก็สำคัญ

ผมขอยกย่อหน้าเก่าขึ้นมา

หรืออย่างเช่นใน MVC แบบ Rails ตัว Model จะเข้าถึงฐานข้อมูลได้ ก็จะเขียนบอกไว้ใน Definition ของ Model แต่ถ้าเราเริ่มทำ Hexagonal Architecture เราอาจจะมี Repository ที่เข้าถึงข้อมูลอีกที ไม่ให้ Model เขาถึงฐานข้อมูลโดยตรง

ถ้าสมมติเราวางนิยามไว้แบบนี้ แต่บางทีเวลาเห็น Model บวมเราก็สร้าง Repository ขึ้นมา บางทีก็ใช้ บางทีก็ไม่ใช้ ไม่มีความคงเส้นคงวา อันนี้ก็ถือว่าไม่ดี

ไม่ว่าจะทำโค้ดในภาษาโบราณแค่ไหนไปจนถึงใหม่แค่ไหน ความคงเส้นคงวาของคำศัพท์และ Layering คือดีครับ และข้อยกเว้นยิ่งมีมากคือไม่ดี

หรือสมมติถ้าเราตกลงกันแล้วว่าเราจะใช้ map เสมอ เราก็จะใช้มันเสมอ ไม่ใช้ for loop และตรงข้าม ถ้าเราตกลงกันแล้วว่าจะใช้ for loop เสมอไม่ใช้ map ก็ควรจะคงเส้นคงวากับสิ่งนั้น

ถ้าจะใช้ผสมกันก็ต้องบอกได้ว่าตอนไหนใช้ map ตอนไหนใช้ loop ที่ชัดเจนเข้าใจตรงกัน

การมีข้อยกเว้นได้มั้ย ได้ ถ้า Justify the cost ได้ แต่ถ้ามีไปเฉยๆ โดยไม่มีต้นสายปลายเหตุอธิบายได้ดีว่าทำไมถึงเกิดข้อยกเว้น อันนี้ผมมองว่าแย่โดยไม่เกี่ยวกับว่าออกแบบยังไง

Appropriate coupling

โค้ดที่ดีอีกอย่างนึงคือ เราเข้าใจว่าโค้ดแต่ละบรรทัดแก้ไขแล้วมีสิทธิ์กระทบอะไรบ้าง

ผมยกตัวอย่างจากบทความที่ผมเขียนเองเรื่อง โค้ดเหมือนจะ Clean

class Invoice
  def pay_invoice
    make_sure_invoice_approved
    create_transaction_entry
    decrease_company_total_balance
    record_tax_deduction
  end
end

ถ้าเราเขียนแบบนี้แล้วเราไม่สามารถทำความเข้าใจได้ว่าการแก้ไข create_transaction_entry จะกระทบกับใบเสร็จส่วนไหนบ้าง จนกว่าจะขุดลึกๆ อันนี้คือแย่

ดังนั้นการออกแบบให้ Dependency ระหว่างโค้ดแต่ละส่วนเคลียร์ เข้าใจง่าย ไม่พันกันเกินไป จนสามารถพูดได้เต็มปากว่าถ้าผมแก้โค้ดบรรทัดนี้ ส่วนนี้อาจจะกระทบ และส่วนนั้นจะไม่กระทบแน่นอน

นั่นคือคุณลักษณะของโค้ดที่ดี โดยไม่เกี่ยวกับ Design pattern ที่ใช้

Local reasoning

คุณสมบัติสุดทายที่ผมจะพูดถึงวันนี้คือ Local reasoning หรือความสามารถในการทำความเข้าใจแยกส่วนได้

ถ้าคุณเห็นโค้ดแบบนี้

def run
  # ทำทุกอย่าง
end

โปรแกรมลักษณะนี้เราต้องเข้าใจทุกอย่างที่มีในระบบก่อนที่จะแก้ไขบรรทัดนึง เพราะแต่ละบรรทัดอาจจะขึ้นกับบรรทัดอื่นได้โดยที่เราไม่รู้ จึงทำให้การทำความเข้าใจแค่ส่วนที่เราสนใจทำไม่ได้

แต่ถ้าเป็นแบบนี้ล่ะ

def vat
  @amount * 0.07
end

เราเร่ิมเข้าใจได้ละว่าบรรทัดนี้ขึ้นกับจำนวน @amount ในคลาสของเราเท่านั้น โดยมั่นใจได้ 100% ว่าจะไม่มี Surprise ใดๆ ออกมาที่จะมากระทบกับสิ่งนี้แบบงงๆ ได้ นอกจาก Amount

การที่เราสามารถทำความเข้าใจโค้ดเป็นส่วนๆ ได้ก็สำคัญ และเป็นสาเหตุนึงที่โค้ดที่ดีมักจจะมีขนาดของ Function, Method ที่เล็ก (แต่ผมว่าเราเชียร์ว่าทำฟังก์ชั่นเล็กๆ มากไปหน่อย จนหลายๆ คนเชื่อว่าฟังก์ชั่นเล็กคือดี โดยไม่เข้าใจว่าดียังไง ผมตอบให้ว่าจริงๆ เราอยากได้ความสามารถในการทำ Local reasoning)

ส่งท้าย

จริงๆ แล้วมันมีปัจจัยอีกมากมายที่ผมดู แต่วันนี้ขอเสนอ 4 ข้อครับ

Domain alignment
Naming convention
etc.

แต่ 4 ข้อนี้เป็นข้อพื้นฐานที่ผมคิดว่าถ้าเริ่มมองโค้ดที่ไม่คุ้นชินจากมุมนี้ เราจะเข้าใจง่ายขึ้นว่าโค้ดที่ดีเป็นแบบไหน รวมไปถึงถ้าเราไปศึกษา Framework ใหม่ โดยเข้าใจว่า Framework ที่ดีจะมีคุณสมบัติพวกนี้ เราก็จะเข้าใจและคาดเดา (Deduce) Design บางอย่างได้อย่างรวดเร็วครับ อ่านแป๊ปเดียวก็อ๋อ เขาไว้ยังงี้นี่เอง ได้เร็วขึ้นครับ

Agile upfront design

Chakrit Likitkhajorn — Sat, 27 Feb 2021 08:21:57 +0000

One of the greatest questions in software design is how much should we put effort into upfront design?

Agile methodology has become a norm for software development, and many people argue for not design upfront. As a result, many of us claim that Agile prevents a good software design and aims to churn out endless features and technical debts.

While I don't believe that is true, I can see where this is coming from.

I want to share my approach toward architectural design.

(Disclaimer: I make a lot of bash in waterfall methodology in the article. When I bash it, I bash on waterfall done wrong. I am aware that effective waterfall will not end up this way.)

Two secrets of the agile design upfront

I think the first secret of upfront design can be simply put as:

Do the upfront design, not for the future, for a present moment

Agile never says that we should not do any upfront design. But contrary to the Waterfall approach, Agile is very against the design for anticipated requirements.

Most of the waterfall projects have a fixed budget and timeline. Most of the time, the IT team is separate from business. Sometimes it is an internal team. Sometimes it is a vendor.

Let say you work for HR, and you want software for your department. You need to state the requirement to the development team. They will tell you how long it would take.

After you get your software, the development team will move to the next project and become busy for years until you get the next development queue again.

What if you want a small workflow change after the project is delivered? You wait for a year since this year the development team will be focusing on manufacturing department projects.

In this case, it makes perfect sense that the HR team will make a lot of guesses on what they want in the future because changes after delivery are very slow.

Now you can see how Waterfall incentivizes guessing future requirements. Sometimes we even asked them to guess.

Will you need to do X in the future? Because we need to design for that upfront.

A team which familiar with waterfall methodology, with their best intention, will likely demand a customer to think of all possible requirements upfront. So they can design and lay the foundation to make it possible to scale for the future.

That is the waterfall upfront design.

I saw people blaming agile for bad software design and technical debt. I worked on a waterfall project, and I've seen worse. A strategy pattern for one strategy. An observer pattern for one observer. A factory of factory of adapter for only a single adapter implementation. I saw it all. Everything is abstract and extensible, without a clear need. Because of ....... future!!

Why this approach mostly fail? The harder we think about the future, the better software architecture should be, right?

Here comes the second secret:

No one can predict future requirements with certainty.

Yes. The problem lies in the fact that we basically squeeze out a future requirement from a customer in waterfall methodology.

Then we design upfront for that "future," and then blame customers when that future never comes, and we have to maintain a bloated software.

You know how we talk about how hard it is to estimate a timeline for feature development, right? We know that many factors come to play. There might be a bad tech debt in the current codebase. The specs might be unclear. The library might not work as well as we initially thought.

The requirement anticipation is also that hard. The business landscape might change. The person who wants the feature might get fired. The key employee might resign, and the whole organization structure might need to change.

I want us to be more empathetic toward business requirement gathering. It is very unfair for us to demand #NoEstimate because it is hard while also demanding to get every future requirement laid out upfront to design our program correctly.

Don't be that entitled.

In my experience, the waterfall project gets a lot of technical debt as well. There are much non-sense code or design pattern which, I believe, designed for those anticipated requirements that the team force out of business and never come true.

So two secrets come to play in the Agile way to design upfront:

Contrary to popular belief of design upfront is for the future: We focus on design for current requirements.
Future requirements are mostly bullshit. That is why we don't design for that.

Approach towards future

When I said stop design for future requirements, there is a big caveat.

There is no guarantee that any requirement will actually be something the user will want until we finished the software. So every requirement is essentially a future requirement.

In practice, I draw the line between present requirement and future requirement based on the confidence level.

For example, let say there are two requirement

The system needs to be able to handle 1,000 req/sec (99%)
The system needs to be able to handle 1,000,000 req/sec (30%)

Both of them are not required today since we don't have a single user yet. But since we are really confident that 1,000 req/sec is required, I will design for just that. I would not think about 1,000,000 req/sec.

Or this might hit closer to the home.

User needs to be able to login using Google Account (99% confidence. All of the target users are using Google)
User needs to be able to login using LDAP (30%, might happen when we know how to market to Enterprise)

Then I will design for only Google Account login. I might use a standard protocol implementation (OAuth), but in the end I would not worry about making sure the system can handle LDAP without refactoring.

You can imagine the difference. In the waterfall project, the customer might ask us to support LDAP even if it not need because the customer was afraid that they would need it in the future. We will be too busy to get back and help them. So we need to "design upfront" for that future requirement.

That leads to a terrible software design—a design for something that does not actually exist.

In Agile, I design for only current requirements. Since everything is essentially happening in the future, I define a requirement with a high confidence level as the current requirement.

For Google login, I can use OAuth. I can make an authentication services. I can put some thought on session timeout. I need to think about cookie or token. I need to think about interface of auth service and callback. That is the design phase. But I don't think about design and interface, network diagram and service layer that can handle LDAP at this point.

That is the Agile upfront design.

To wrap up: I believe that we should think about software design. We should not become a code-monkey and churning out feature regardless of tech debt.

We also need to accept that the future is unpredictable. Instead of upfronting design based on what might happen in the future, I believe we should do upfront design just for the current requirement (which can be defined by working closely with the customer as in Agile Manifesto).

That's all I have to say. Thanks for reading.