Forem: Horia Coman

Seven Anti-Patterns For Your Job Search

Horia Coman — Sun, 17 Oct 2021 07:00:00 +0000

Over the last year, I’ve reached out to almost 1000 engineers trying to get them to interview for Bolt. I’ve had some success and plenty of failures. But I have learned a lot about how people approach interviews and career development. And I’ve identified seven “anti-patterns” that I think are detrimental to a successful career.

In this article, I want to share these anti-patterns with you. I also want to share my approach to effective “job searches”. I hope these will prove useful to you.

I’m gonna work off some assumptions. You’re a software engineer with some years of experience. You want to get the most out of your career. You’re based somewhere in Europe, the US, LatAm, or so. Life is going OK for you and you want to optimize your long-term career. If this is not you, then you may or may not get something out of all this.

Now it’s no secret that we are living in a golden age for software engineers. Software is eating the world, and demand for devs is far outstripping supply. Developers have their pick of projects, high compensation, and cushy jobs. If you’re looking to have a great career it’s never been easier. But it still requires work, planning, keeping your ear to the ground, avoiding some common mistakes, and even taking a 40-year view of it.

And as some of us know, it wasn’t always like this. The .com crash was a situation opposite to what we have right now. And things have only really gotten good since the 2010s. A reversal to the mean is still within the realm of possibility. That’s why I think it’s important to make the most out of this situation and strike the iron while it’s hot. Avoiding the following anti-patterns is a good start.

My Background Doesn’t Match The Position

I was looking for backend engineers to work on our microservices-based systems. But I cast a wide net when reaching out. I wanted to speak with folks from embedded, security, low level, and kernels. And I was also keen on people working in compilers, networking, or automotive. But I got a lot of pushback and even questioning of my approach.

But “backend” development is different. It’s the place where “business logic” resides. Often, it is not so much about the technology as it is about describing the laws that govern the business. As such, any developer can be a backend dev! There are still notions of architecting and scaling a system that does require expertise. But that’s not a day-to-day thing. And in general, engineers help by solving problems. Not by knowing technologies or being experts in this or that tool. There’s great value in mastering tools, but only as a means to solving problems. Knowledge for its own sake is an anti-pattern too!

At Bolt, we have folks from a classical backend background, of course. But we also have former mobile devs, embedded ones, or ML people. They’re doing great and their previous experience comes in handy often. And they gravitate towards the projects that highlight their strengths.

I’m A Blub Developer And You’re Using Blab

Even when I reached out to backend developers I got way too many “I’m a Java/C#/C++ dev, so I’m not sure I’m what you’re looking for” answers.

The skillset you developed is being able to solve problems, having engineering maturity, and understanding what’s needed in large-scale software projects. That’s why we want to talk to you. That’s what companies value, in the end. Regardless of language or ecosystem, you’re going to need to solve problems. And you’re going to need to build reliable and scalable solutions. Or take part in code reviews, write tests, ensure the quality of your solution, etc.

Again, speaking from experience, very few of our devs have used Node+TypeScript before. Yet they still do their first release in their first or second week of working with us. And pick up the rest and become proficient in a matter of months.

I Have A Fancy Title And I Don’t Wanna Give That Up

This came up a bunch too. Folks had Architect, Staff, or Principal titles. But I was hiring “just” for a Senior Software Engineer. So I got a bunch of quick nopes. And a bunch of longer nopes from folks who wanted to understand a bit more but decided against a “downgrade”.

Titles do matter. But titles are very company-specific too. For example, product companies usually don’t have the Architect position. As a famous example, none of the FAANGs do. Likewise, startups usually don’t have more than “Engineer” or “Senior Engineer”. Even in established product companies, there’s a wide variety of post-senior role structures. There’s alsotitle inflation to consider. And different companies have different expectations. These things become more of a way to express something internal to the company and not some global truth. You’re limiting the kinds of cool companies you can work for by focusing too much on the title.

We’ve had many folks join as regular engineers who were Staff engineers, or Architects, or Tech Leads, or even managers. They’ve all thrived, managed to put their experience to good use, and then “got back” the title in our context too. And of course, even their compensation increased as a result of the move.

There Are Too Many Senior People

This one caught me off guard. Some folks worried that there would be too many other senior people in the engineering teams. And there would not be enough room to get to those limited leadership opportunities.

But this approach is counterproductive. You never want to be the best in the group. You rather want to surround yourself with people you have things to learn from. If it’s a good collective then it’s bound to also have success. And then at least you’ll catch the rising tide that lifts all boats. But most likely the organization will expand, and you’ll get that chance you were looking for, in the end. Being in a higher-performance team will reflect better on you at the end of the day.

At Bolt, there’s no limit to ownership or leadership. It’s more about the things you do, and not the position you are in. The same holds for managers and individual contributors. Case in point, a lot of the critical processes at Bolt are run by folks who took an interest in them. Our hiring pipelines, our coding interview process, etc. are good examples.

I’m Not Good For The Interviews Or For Bolt

Again, this one caught me off guard. Some folks found the position attractive. Yet they didn’t want to apply because they didn’t see themselves as ready or good enough.

This is an insidious anti-pattern. You might be judging yourself harshly, or us too well. You might be viewing the situation through the lens of impostor syndrome. Or of perfectionism. Even if the technical skills aren’t there yet, a company might still see enough potential to make an offer. Or the internal context is such that something can happen. It’s not all black and white, even at bigger companies.

We have folks from very diverse companies. Ex-FAANG engineers as well as 3-person shops. Product companies and outsourcing companies. Competitive programmers and self-taught devs. I’d dare say success is orthogonal to this sort of background.

I’m Looking For An Upward Move

A fair number of folks were looking for an upgrade to their current position. They were junior devs seeking a particular senior title. Or inexperienced managers seeking to transition to a full manager role. Or even folks without any management experience seeking to gain it via a move.

The truth is the scope of your work rarely increases when changing jobs. The salary - sure. Especially in the first part of your career. It’s even expected every job change has a comp increase. The “title” can change too. You can get an upgrade when moving to a startup or smaller company. Or a downgrade when going to a bigger company. But companies tend to be conservative on the experience side. They will look for “demonstrated experience”. And there’s nocompression algorithm for experience.

A common pattern at Bolt is for someone to join and take on more ownership and responsibility. They’re then swiftly promoted. We’ve even done promotions during the probation period for folks who were clearly next-level. It is not just the transition, but the growth that can happen after.

I’m Happy Where I Am / I’m Not Ready For A Change

This one is gonna ruffle some feathers. A fair number of folks didn’t want to talk because they were happy with their current gig. And not looking for a change.

Finding a place with great colleagues, challenging projects, and good compensation is hard. You don’t wanna lose that if you have it. But the number of these employers keeps increasing. Even in the outsourcing-dominated Eastern Europe where I’ve got the most context. And what was great a couple of years ago, might be OKish today.

In the end, leaving a company might not be the irreversible move it’s made out to be. I’ve seen enough folks jump from Company A to Company B and back to Company A. Sometimes in a month.

Conclusion & Next Steps

Thanks for coming to my TED talk. You now know some things to avoid. But the tone so far has been sorta’ negative and I don’t feel good ending it like this. So I want to give a small “job searching” framework you can use as a guide. It consists of three rules to replace the anti-patterns form above.

First, look at the team and project you’ll be working on as well as the wider product you’ll be embedded in. Think about the future growth of the company you’ll be helping out and how that can affect you. Tie all these to whatever career plans you have. Only then consider technology-specific reasons. Especially avoid going for one thing - playing with Scala, or using K8S.

Second, aim to find the highest performing collective that will have you. This is very important in your early career. The dividends it will pay off will more than offset “title deflation”. Once there, take on responsibility and ownership and try to grow as fast as the company at least.

Finally, treat interviews as a skill to practice and as a tool for exploring the “market”. Not as a chore to only do when you’re forced to switch jobs. Rather you should do them even when you’re not “actively” looking. You want to get good at them. And you want to use them to better understand the market.

Anyway, that’s enough from me. Thank you for reading this far! If I’ve managed to get you excited you should know thatBolt is hiring. If you’re an engineer who wants to avoid the career anti-patterns - or know one who does - then come our way at careers.bolt.eu. Or DM me on LinkedIn and I’ll sort things out.

Thanks to Mihnea DB, Mihai S, Radu V, Liviu C, Foti C, Radu S, Tomasz R, Sergey Z, Lauri L, Andreea C, and Nick G for tearing the first draft of this to shreds and forcing me to write a leaner and meaner text.

Jupiter Dev Log 5 - Architecture

Horia Coman — Tue, 08 Dec 2020 07:20:05 +0000

Photo credit James Barnett on Unsplash

In this latest post in the Jupiter dev log series I want to speak about the software architecture of Jupiter. It’s a topic I’ve wanted to write about for a while actually, but somehow never got the will to do it. With the limited time, I have for working on this, writing a new feature always trumped writing about writing a new feature. But I’ve finally decided to bite the bullet and commit to something because I recently finished reading and reviewing Clean Architecture. It has made quite an impression on me and made me think in a new light about some of the design decisions I made with Jupiter. So this post will be equal parts presentation and critique. The project has also reached a respectable 15 KLOC of code, so it has enough heft that the discussion isn’t strictly academic, and might prove useful to other folks too!

Architecture Cornerstones

I find the term “architecture” quite fluid. I’ve seen it mean everything from the choice of what patterns are used in the source code of a project, to building a system as a webapp, to using a microservices “architecture”. Clean Architecture focused a lot on “architecture in the small”: codebase level organization of classes within components and dependencies between components. But I liked the very general notion that “architecture is the important decisions” so I’ll cover the big decisions or cornerstones too.

Some of the choices I made will raise some eyebrows. They aren’t ones one would usually make for a product. They even seem both explicitly tailored at programmers and actively hostile to non-programmers. The underlying theme is that they made my life easier, and allowed Jupiter to quickly become usable and useful. Indeed, I started making use of it as my main tool as soon as a week after the git init was done. Had I went with a more traditional approach I would have statistically speaking never finished it and never used it. I’m happy with the outcome of these choices, but I realize they aren’t and cannot be final if I ever want user #2! Anyway, that’s about all for justification - take the choices as they are henceforth.

The first architecture cornerstone is that Jupiter is a self-hosted and locally operating CLI application. This means that a user (me) downloads a packaged and versioned release of the application and uses it locally to manage local data. I use Docker as packaging and releasing infrastructure, even if this sort of use-case isn’t its main one. Of course, Notion.so plays a big role as the nice user-interface. Jupiter interacts with Notion as a client which periodically checks to see if there’s something to be done. When I’ll add other integrations - with Google Calendar, Spotify, etc, the same interaction patterns will be followed. But for now, there isn’t an “always-on” component to Jupiter to act as a counterpart to these services. Rather a user’s install of Jupiter does something only as a result of the user asking it to!

The second architecture cornerstone is the use of text files as storage and reliance on git for persistence and cloud sync up. Storage itself - whether text files, or databases, or in the cloud, is an implementation detail in the end. Though it is a very visible one. But relying on mechanisms outside the control of the app for persistence and sync up is what essentially makes the app a “local” application rather than a “cloud” one.

The third architecture cornerstone is that the CLI application has a command-driven interface. It essentially presents a lot of commands to the user through which they do their work. Think git here. So there are commands like jupiter sync or jupiter smart-lists-create, etc. Jupiter is running only when such a command is invoked. If you run jupiter --help you’ll see a lot of commands. Suspiciously looking like a REST or RPC API even.

The fourth architecture cornerstone is that there are multiple ways of interacting with the system - the CLI and Notion. And they have separate storages, which need to be explicitly synchronized. Indeed, most of the interaction will occur through Notion. But any change made there won’t be seen by the local install. You need to run a jupiter sync command for that to happen. At first glance, this seems like simply a functional concern, but it does end up influencing several architectural aspects. From how code is organized to allow abstracting away the need for synchronization, to a focus on idempotence which permeates the codebase. I run the command about once a week, therefore the eventual consistency reconciliation window can get quite big!

The Code

This draws us down into the code for the first time. Commands imply a great deal of regularity in the way one interacts with the application - issue a command, wait for processing, observe results, repeat. And this indeed translates to the code. The app looks like any system for answering requests. Which in computer parlance we call a server! It’s not a web server like most things are these days. And it’s not always running like them. But it’s the same kind of thing in the end, and to a user it is transparent. As a historical aside - the first web applications were built with a server feature called CGI - which allowed it to invoke a binary on the same machine. Naturally, a new process was spawned for each request, much like we’re doing here

In the prehistory of the project, there was no code architecture to speak of. Or rather it was the venerable big ball of mud. The first-ever tagged commit v0.0.1 still had a lot of this. There were a bunch of files, all independent, all entry points, and with a very minimal structure around it.

But things have evolved quite a lot. And as of release v0.6.1 things look much different. For better or worse we have this component structure:

A listing of the source tree hints at this too, but it isn’t a 1:1 mapping with the above image.

$ tree -L 3 src
src
├── command
│ ├── __init__.py
│ ├── command.py
│ ├── garbage_collect.py
│ ├── generate_inbox_tasks.py
│ ├── inbox_tasks_archive.py
│ ├── ...
│ └── workspace_show.py
├── controllers
│ ├── big_plans.py
│ ├── common.py
│ ├── garbage_collect_notion.py
│ ├── ...
│ ├── vacations.py
│ └── workspaces.py
├── jupiter.py
├── models
│ ├── basic.py
│ └── schedules.py
├── remote
│ └── notion
│ ├── big_plans.py
│ ├── common.py
│ ├── inbox_tasks.py
│ ├── infra
│ ├── projects.py
│ ├── recurring_tasks.py
│ ├── smart_lists_manager.py
│ ├── vacations_manager.py
│ └── workspaces.py
├── repository
│ ├── big_plans.py
│ ├── common.py
│ ├── ...
│ └── workspace.py
├── service
│ ├── big_plans.py
│ ├── ...
│ └── workspaces.py
└── utils

I’ll go into a bit more details around what’s happening. There are three large components to the system, each split up into several sub-components.

CLI:
- This component handles the CLI functionality of the application. It concerns itself with arguments parsing , outputting things to the console, etc. It’s in fact the only component aware that this is a command-line app, as opposed to a web or desktop one.
- The Main subcomponent is the entry point to the application - the uncleanest of unclean components. It’s actually the src/jupiter.py file. This essentially handles wiring everything up together in the application - all repositories, commands, etc - and dispatching from the user’s intent to a command to execute.
- The Command subcomponent contains classes for handling each of the commands for the application. Here we’ll find a class like GarbageCollect which implements the jupiter gc command, and indeed, one for each command. The logic is very thin, however. A command simply parses arguments and invokes an appropriate domain-level controller with the required data.
- There’s even a small home-brewed framework of sorts since each command is so regular. Checkout command.py and jupiter.py for details.
- Overall I’m pretty happy with this part of the codebase. It’s self-contained, modular, and deals with exactly what it needs to.
Domain:
- This component handles the business logic of the application. It’s split across entities lines - projects, inbox tasks, big plans, smart lists, etc.
- The Controller subcomponent contains the so-called use-cases of the application. Here is where the magic happens many times - the business logic itself resides here. Here we’ll find a class like InboxTasksController which implements the CRUD operations on inbox tasks. There are also some specialized controllers like GarbageCollectNotionController or SyncLocalAndNotionController which frankly do the core and logic-heavy work. These are the reasons why the application was built in the first place - all the other controllers offer support.
- The Service subcomponent contains the lower-level entity-specific logic. There’s one service class - like InboxTasksService - for each entity. This is not exactly business-logic, nor is it storage or infrastructure code. What happens here usually is that mutations are applied to both the local storage and to the Notion storage. Similarly, the critical synchronization code between Notion and the local storage resides here. The need for such a layer - an architectural thing - is pushed by the functional choices. Though I admit the naming of service isn’t quite right.
- I’m not 100% happy with the Controller <> Service separation. In the future, I’ll probably merge the two, reify each use-case as a separate controller class, and figure out some better abstractions for Notion to local syncing.
Infra
- This component handles the interaction with the “outside” world. It’s split across entity lines - with projects, inbox tasks, etc. getting their own classes here, and along the local vs Notion divide.
- The Local subcomponent contains the classical repositories that code in the domain component uses to interact with persistent storage - for now text files on disk. It’s what you would expect these be. For example, there is an InboxTasksRepository class with methods such as create, load (by id), save, find_all, etc.
- The Remote subcomponent contains code for interacting with Notion in a principled way. There are a bunch of utility classes that abstract away the raw Notion client and provide some shielding for when the official SDK will need to replace the current one. But the main abstraction is the Notion “collection”. For example, there is an InboxTasksCollection class which deals with both setting up the Notion-side page structures and collections, but also mapping between Notion rows and in-memory data.
- Overall I’m decently happy with the architecture of this part of the code. But the actual code-quality could be improved. There’s some lower-level code here which could be made better, more generic, and more robust. But the biggest issue is probably that there’s a lot of business logic about Notion such as what pages there are, how collections are formed, and what views they have, which are coded at this lower level, rather than in the “Domain” component.

When you invoke a command such as jupiter inbox-task-set-name --id 10 --name “Foo”, the following happens:

The src/jupiter.py script is invoked and all the repositories, collections, services, controllers, and commands get initialized and wired together. The program decides which command to invoke based on the name of the command.
The InboxTasksSetName classes’ run command gets executed, which parses and validates command line arguments, and passes control over to the InboxTaskController.
The InboxTaskController’s set_inbox_task_name method gets invoked, which simply forwards the call to the InboxTaskService.
The InboxTasksService’s set_inbox_task_name method gets invoked. This loads the respective inbox tasks via the repository then applies the change to the entity, then saves the change locally. Finally, it applies the change remotely on Notion side.
The operation is not transactional between the local and Notion side. The latter change might fail, and a jupiter sync will be needed to reconcile the two.
This is an example of a simple CRUD operation. More complex sync, or gc commands have most of their logic in the controller, which orchestrates the changes across multiple entities.

Some Nice Things

To wrap the previous section off, there are some unambiguously good things about how the system as it is currently organized.

It is “layered”, and each layer deals with a specific thing. The directories structure better represents these layers, but each layer also is a specific component. There are no cycles between the layers.

It has a notion of controllers which capture the use-cases of the application. The logic is shared with services, and it’s quite clear what the application is about and can do.

It uses dependency injection quite heavily. Not IoC though, as the flow of control still follows the flow of dependencies. Regardless, the src/jupiter.py looks like a regular modern web application’s setup code.

It isn’t tied too hard to the CLI app paradigm and can relatively easily be extended to any “request-handling” interaction mode.

A Principled Critique

Let’s look now at the component-level principles presented in the book, and how the current system stacks against them.

The release/reuse equivalence principle is weakly followed. The application is monolithic, hence we can’t speak of reuse, and releases happen for the whole thing. In a future in which we have multiple notions of clients, the controllers, services, and other parts can be packaged independently, and various clients (or servers) could make use of them. It is versioned, however, and it’s using semantic versioning.

The common closure principle is quite well followed, by virtue of separating things into layers. Big changes have tended to affect things across layers, but once the thing becomes more mature I’ll expect the bigger chunk of work to be localized to a layer.

The common reuse principle is quite well followed, by virtue of separating things into layers. Again, it’s not a situation of reuse we’re looking at so far. However, because dependencies flow from more abstract to more concrete, rather than the other way around, reuse will be tricky for certain situations if future improvements aren’t done.

The acyclic dependencies principle is quite well followed. There are no loops in the dependencies.

The stable dependencies principle is weakly followed. Business logic classes routinely depend on infrastructure level concerns, which tend to change for various reasons (Notion API changes, updates in storage, etc.). Where it’s most hurting is the dependence on data-types exported by these packages, but functionality tends to be neatly abstracted.

The stable abstractions principle is weakly followed. Abstractions aren’t well-reified in the code - there’s a lot of dependencies expressed in terms of concrete classes. Since this is Python it still allows for decent substitution if we wanted, even though the type-checker would complain.

Future Improvements

In this section, let’s look at some architectural improvements which flow naturally from the previous discussion.

The biggest future improvements are around implementing more of the clean architecture as given in the book. This means at the minimum inverting the flow of dependencies. We can add a bunch of interfaces and types in the domain part, that the other more “concrete” components depend on and implement. src/jupiter.py can still wire them up well.

There are very few clear “Entities” encapsulating the clear and profound business logic. And that’s a shame because there are so many such concepts - from ids to schedules. I’d like to separate this sort of central logic and reify entities and what can be done with them better.

All of the major objects are somehow containers of objects and singletons. Even in the business logic layers, it’s all about “repositories” doing things rather than entities themselves doing the things. For example, a smart list item doesn’t get to add its own tag, rather the smart list service adds a tag to a smart list item entity. It’s all very “procedural”. I’d like to change this and move some of the logic to the entity classes themselves.

Controller/use-case logic is kind of thin, and I think some of it can be centralized there instead of “services”. Some better abstraction for syncing between Notion and local will be needed to avoid repetition and/or missing out on some cases.

Too much business-logic about Notion is embedded in the lower-level Infra/Remote subcomponent. It should offer a more flexible interface, which the Domain component uses to build the appropriate structures.

Changes tend to be quite vertical, from the smallest to the largest. I’d prefer it if only larger changes - those warranting a minor version bump - would be thus, and smaller changes would completely be isolated in a component.

Conclusion

This about wraps it. Hopefully, it’s been an interesting and useful read. I expect the doc to reflect the reality of the codebase for quite some time. But as with everything, the truthness of this document will slowly start to erode as I make improvements, extend functionality, and refactor the codebase. However, the major elements I expect to stay in place for a long-long time, and everything else to gradually move in the direction highlighted in the “to improve” sections of the doc.

Stay safe and see you next time!

Clean Architecture Review

Horia Coman — Tue, 01 Dec 2020 07:00:00 +0000

This is my review of Clean Architecture by Uncle Bob. The book is undoubtedly a classic and I feel bad for not having read it earlier. I think it might have helped with a project or three. Better late than sorry for sure, and unlike other books I read too late (SICP primarily) I did feel I got a lot out of this book, and I came out a better developer on the other side. It certainly gave me a bunch of ideas to try out in Jupiter.

I must admit I’ve been skeptical in the past of this whole clean code, clean architecture, design patterns, etc part of the software engineering toolset. Not the things in themselves. But what I perceived as the bias towards “the enterprise” on the one hand, and towards the world of software consultancies and “project”-work on the other hand. It didn’t help that I felt there was a lot of cargoculting in the mix too. As well as a feeling that this stuff is the easy part of software engineering. As opposed to algorithms, machine learning, functional programming, distributed systems, or other “hard” things. In any case, I got some wisdom in the end and decided to complete this part of my education before I slip too much into management. The fact that my side project Jupiter is hitting some architecture limitations is purely coincidental.

Anyway, back to regular programming. The topic of the book is software architecture. From the start, we’re faced with the problem of what exactly is software architecture. Uncle Bob settles on the very wide “every important decision about the technical aspects of the system being built”. Which has the nice property that it applies just as well to the 100 line utility script as it does to the Linux kernel, and just as well in the lowliest embedded system as in the loftiest distributed one. It’s not very concrete, unfortunately. However the book essentially then focuses on the “architecture in the small” (if I may) of components and component dependencies and their proper delineation and orientation.

The book has a lot of chapters. This is great for snacking and context switching and getting quick wins, but bad for chapter-by-chapter summaries as I’m used to doing. So instead I’ll offer a mish-mash of the actual lessons of the book. More work, but hopefully more useful too. I’ll even throw in a diagram.

The main problem is that of selecting a good architecture for the system that we are designing. By good here we mean that the system is easy to understand and extend. A good architecture allows for this growth and change to continue. It’s interesting to note that architecture in the sense discussed is not about the functionality of the system, though the performance, security, etc non-functionals do drive it.

The biggest take away is a set of design principles that should be followed when designing a system. There are two classes of them: the lower-level and more commonly known SOLID principles, and the higher-level component-level principles. By component here Uncle Bob means something like a jar, or dll. But in many ways, it can be extended to larger-grained things like processes, or services. I’ll summarise them here:

The release/reuse equivalence principle: the “unit” of release is the unit of “release”. That is, versioned libraries/components can be used and reused, not anything “smaller”.
The common closure principle: a component should contain classes that change for the same reason.
The common reuse principle; a component should contain classes that tend to be used and reused together.
The acyclic dependencies principle: the dependencies graph between components should not have cycles - it should be a DAG.
The stable dependencies principle: dependencies should flow in the direction of stability.
The stable abstractions principle: a component should be as stable as it is abstract. Abstract components (consisting of interfaces, ABCs, etc.) should be stable, and concrete ones less stable.

The “Clean Architecture” itself is quite simple and follows from the principles. The main idea is to split the code into two layers - the inner “domain” layer and the outer “infrastructure” layer. The inner layer is the pure business logic of the system, while the outer layer handles things like persistence, communication, serialization, etc. Crucially the dependencies are from the outer layer to the inner layer. This is against the flow of control (and thus doing “Inversion Of Control” proper), and also from more concrete to more abstract, and from more variable to more stable.

The domain layer can be further split into two sub-layers. The first or “core” layer deals with the very abstract and high-level procedures of a business. Things like computing interest rates, or scoring a loan application. These things tend to change very slowly and would exist regardless of the physical instantiation of a system. They would look the same regardless of whether the system is a mainframe program, web service, blockchain contract, , etc. The “application logic” layer deals with the specific use cases of the current system. Things like a clerk verifying the loan documents of a potential client, or a salesman marking an account as inactive. These use cases tend to change with some frequency, and they are bound to the physical instantiation of a system. They would look different depending on whether the system is a mainframe program, an IoT controller code, or low-code automation.

The infrastructure layer has some canonical splits between presentation and persistence, but in general, here you can put anything which deals with the “world outside” the domain layer. From the database and the realities of turning entities into SQL rows in various tables to capturing user input and transforming it into commands of the various use cases. This layer is concrete and tends to change quite a lot. For that reason, it is called “dirty”. And the dirtiest of components is the main one. This is the component where the system starts - so perhaps it’s a class with the main function or the initial script file - and where it configures all other classes, linking them together, wiring the abstract with the concrete.

This means that in the domain layer we should have only simple classes. There should be no ORM objects, no framework muddling, no concerns about serialization or input validation. Just simple classes doing simple things. Just like one would see in OOP tutorials as “simple” but “unrealistic” examples. I find this thing very attractive intellectually. First because it’s indeed “clean” of external libraries or frameworks - it’s just the application’s own code. But it also allows for the clear separation between the part that codes the business processes and the one which is just accidental complexity. Indeed, this part could just as well be plucked from its most likely webapp form, into a CLI application, or a testing suite.

As an implementation note (mostly gotten from other sources though), the way the dependencies are manifest is that the domain layer will define some sort of interface - say LoansRepository, which deals with the persistence of Loan entities. The infrastructure layer will implement the LoansRepository via a PostgresLoansRepository that will handle speaking with a properly configured Postgres instance. It could also define MySqlLoansRepository or FlatFileLoansRepository. The main component will then need to configure that the LoansRepository is resolved to the appropriate implementation for the run. This way, the domain layer doesn’t use anything from the infrastructure layer explicitly - it just says what it expects some other components to provide, and leaves it up to main and polymorphism to do the rest.

The last section of the book is the catalogue of details. Essentially all the pieces of technology that we’d commonly understand as central to a system: databases, the web, frameworks, etc are relegated to mere details. Something which we should tightly keep in the application layer, but abstracted away to the needs of the domain layer, rather than running the show. This thing was both the most counterintuitive in the whole book, but also one of the truer and influential of the bunch. Indeed, building something with the “clean” architecture even imposes the use of libraries more than frameworks. Which is a trend I’ve seen in the industry anyway of shunning big and all-encompasing frameworks like Rails or Spring and moving towards simpler frameworks for special use-cases (and perhaps there’s even some causality there to look at).

There’s a link here to hexagonal architecture, the onion architecture and even domain driven design (to an extent). Indeed I’ve come to understand these architectures are a sort of large scale software design pattern for structuring the codebase. Ways in which it is useful to organize the classes/modules of a system, which we’ve observed to work and allow for easy understanding, reasoning, and expansion. On the other hand they’re not set in stone. Like all patterns they are (or should be) guidelines - each system will be different and have goals and constraints such that the form of $named architecture it uses will look different from another’s or from the standard. That’s why I think the principles are the most important part in the end. You can get something radically different than the “Clean Architecture”, but if it follows the principles, it would still be a good thing.

Besides reading this book I also watched a bunch of talks, both from Uncle Bob and from others. I’m linking them here because they serve to clarify, contextualizie, and expand the understanding of clean architecture. They are:

I couldn’t end before mentioning the final chapter. It’s a rundown of Uncle Bob’s past projects. It was a fun read in itself but two things stood out. One was that Bob’s first gig was in 1970 programming a computer for some union’s accounting needs. This was a mere 25 years after the first computers ever were built to help with the atomic bombs. That’s a massive achievement to go from the smartest minds of a generation working on the cutting edge to a fresh highschool grad of 18 years working on an old (by then) machine in the mundane field of union accounting. The second was just how brutal the developer experience was back then. Punched cards, days long edit-compile-test cycles, assembler everywhere. I’m not 100% I would have found being a programmer so interesting during those times. I’m hooked on our modern tooling.

Anyway, that’s it from me. There’s a lot more to the book, but hopefully, this has piqued your interest. Stay safe ‘till the next one.

Friday Blast #95

Horia Coman — Fri, 06 Nov 2020 07:00:00 +0000

Productionizing Spark streaming applications (2019) - we had to use Spark streaming for a project at Bolt, and this guide proved mighty useful for getting us in the right headspace for tackling the problem.

Asynchronous computing at Facebook (2020) - an overview of Facebook’s complex system for handling “notifications” and “notification-like events”. A message queue in a way. As a side-note, I’ve noticed that systems that are essentially message queues (chat systems, notification systems, etc.) don’t ever seem to be implemented as an infrastructure-level message queue (Kafka, SQS, etc.), but rather in a traditional fashion (mixes of databases, queues, caches, servers, etc.).

Big data architecture patterns (2014) - a short overview of how a modern-ish data infrastructure layer looks like - data lakes, ML systems, etc. included.

Mutation testing - a tale of two suites (2020) - about how Esty used mutation testing to migrate from one test framework to another. It was the first time I heard of mutation testing (with this definition at least - but the name sounds familiar). It is a test for tests essentially. It applies a batter of mutations to the code base, and runs the test suite on it. Ideally the test suite should fail in the appropriate manner. Quite an interesting and powerful code quality tool - a step above just having tests, and in the realm of code coverage tooling, fuzzing, etc.

An introduction to CQRS and Event Sourcing patterns (2017) - again an intro talk on these two more advanced architecture patterns.

Friday Blast #94

Horia Coman — Fri, 30 Oct 2020 07:00:00 +0000

Why write ADRs (2020) - architecture decision records (ADRs) are a form of technical documentation. They cover the big technical decision made on a project, in a sort of syntehsized fashion. Much like a design doc, but without going into the nitty-gritty.

Type in the exact number of machines to proceed (2020) - a nifty double-check for potentially destructive commands of all kinds. Don’t just do it or require a simple Yes/No answer. Rather ask the person to work a bit for the answer and also tie it to the magnitude of the mutation - ie number of machines affected by the change.

Achieving maintainability with hexagonal architecture (2019) #video - another short intro to hexagonal architecture.

Homesteading the Twittersphere (2020) - an interesting take on why there are so many high-quality analyst-type blogs and newsletters lately (the author included). Humans value status quite a lot, but in an abundance of information scenario, the old patterns of status via wealth or via power don’t really help in establishing the proper hierarchy. But status via gift does. Much like with open-source software many years earlier. Put out some complex and useful software for free, benefit from the recognition, and get extra IRL rewards from this.

Friday Blast #93

Horia Coman — Fri, 23 Oct 2020 07:00:00 +0000

Taming floating point error (2020) - a brief overview of what error means in floating point and how it affects work. What I found most interesting was the discussion on error analysis and error propagation through a computation. This was one area I didn’t have a good mental model of how to approach from uni days, and it turns out that (at least conceptually) it is quite straightforward and elegant. Indeed, I have an even higher appreciaton for the design of the floating point standard now.

Bloom filters debunked (2020) - apparently there was an error in the analysis of Bloom filters false positive rates lurking for 30 years. Indeed the last step of the analysis seemed a bit hand-wavy the first time I encountered it. The authors used coq to provide a formal proof of the known correct form and increase our confidence in it.

On the uses of a life (2020) - the author oftarsnap (and inventor of key derivation methods in cryptography) got called out that essentially he wasted his life making backup software. This is his response.

Finding the right company to reach Staff engineer (2020) - another one from Will Larson. Staff roles are those very-senior engineering roles in a company, and the article is about alignment and expectation setting in order to reach that.

What engineering managers should do and why we don’t (2019) #video - a short introduction to management for engineers. In video form.

Radical Candor Review

Horia Coman — Thu, 22 Oct 2020 07:00:00 +0000

This is my review of Radical Candor by Kim Scott. This is a management book as you can surely surmise. The central idea is that one should practice radical candor in a work environment - a style of interacting with others where one both seeks and offers good and constructive and sometimes hard to hear feedback.

While I found the idea worthy, the book itself felt quite the slog. The message itself is OK, and there’s ample storytelling, justification, and instruction around it. But that can only fill so much book. Hence the bigger part of the book deals with general “modern” management approaches. The kind of stuff you can read about in The Manager’s Path or An Elegant Puzzle. It’s also stuff that I am more or less doing as a manager. Hence my earlier characterization as a slog.

The most worthwhile part was the first one consisting of four chapters. These only dealt with radical candor in itself and why one should care, with copious examples and an analysis of the various facets of it - from relationship building to delegation.

The second part focused on implementation. From this only the last chapter I found useful, but mostly because it had a bunch of easy to apply tips and tricks for time management, meeting structuring, etc. Stuff I could steal right away.

Even given all of the above I am not 100% would want to commit to such a thing, and follow the protocols. I do think a large part of this is achieved via simpler things like building a human relationship with your colleagues and reports, and having structured career conversations. The rest can surely help, but the delta might not be sufficient.

Anywho, at least the highlighted chapters are interesting and worth to dig deeper into. And if you’ve never read any other management book this one isn’t a bad start at all.

Announcing Jupiter Release 0.5.1

Horia Coman — Sun, 18 Oct 2020 00:00:00 +0000

I’ve prepared a minor release - 0.5.1 - which contains a lot of goodies for recurring tasks. This version introduces:

Recurring tasks can have active intervals. Only during this interval will inbox tasks be generated. This can be open ended of course.
Inbox tasks from recurring tasks now have the year included in the name.
Reporting on habits also shows a “completion ratio” broken down by sub-periods.
Inbox tasks now have an actionable date. They’ll be visible to work on one week before this. Recurring tasks have been updated to allow generation of such fields too.
Fix a timezone issue with recurring tasks generation.
Fix an issue with smart list syncing which caused every smart list to contain every smart list item.

As always checkout the installation instructions to find out how to get hold of the latest version.

Friday Blast #92

Horia Coman — Fri, 16 Oct 2020 07:00:00 +0000

Why do ARM chips have instructions with JavaScript in the name (2018) - remember kids, if your programming language becomes popular enough, its primitive operations will become CPU instructions. I wonder if there’s been any analysis done on this? Like look at where a sufficient sample of programs spend their time and make that into instructions. You’d need a size/complexity heuristic to not just leave it atmain but I’m sure companies such as Google or AWS which run big sever workloads would have the data.

Why isn’t functional programming the norm? (2019) #video - the short answer is “it’s complicated”. The longer answer is that FP languages lack either a “killer app” or some serious corporate backing. Add do that a bit of a definitional problem - like what is an FP language? Is Common Lips where you can have imperative constructs more FP than Rust where everything is immutable by default? OTOH many modern languages have functional programming capabilities, so in that sense FP is the present.

The observer effect with Daniel Ek (2020) - an interview with Spotify’s CEO. There’s a lot to take in, but perhaps for me the most interesting part was about the coaching he provides to his reports.

AirBnBaller (2020) - an analysis of why in the long run AirBnB might be the “breakout” startup of the 2010s, eclipsing the likes of Uber, or Palantir. Right now, things are dire with the global pandemic, but it won’t last forever. If anything, as a supply gateway AirBnB will have a very strong hand to play vs traditional hotel chains.

40ms of latency that just won’t go away (2020) - another case of being tripped up by Neagle’s algorithm for TCP - which batches short message bursts into bigger chunks to more efficiently utilize bandwidth. At the expense of latency. Disabling it is set via some flag on the socket. What I always find peculiar about these low level APIs is how much they had that you’re doing a “big thing(tm)” with these things.Sometimes it’s not even clear it works or is enough to work. I definitely like the higher level API where you’d instantiate a NoBufferingConnection or something of the sort and make it really clear and semantic what you’re trying to achive.

Friday Blast #91

Horia Coman — Fri, 09 Oct 2020 07:00:00 +0000

OAuth 2.0 and OpenId Connect in plain english (2018) #video - A gentle guide to these protocols for managing authorization / identity and its delegation. It’s one of those things every developer building web applications should know about, but there’s a dearth of good material out there.

Modern banking in 1500 microservices (2020) #video - An overview of Monzo’s architecture and a little bit of their tech stack.

Did you accidentally build a database? (2016) #video - one of the founders of CochroachDB talks about various large-scale systems and how they’re essentially poorly implemented databases.

Understanding statistical error (2020) - many models can be written as R(X) + \epsilon, where \epsilon is some noise factor with following some distribution and R(X) is some deterministic relation. This \epsilon is not an error of reality, but of our own modelling.

Industrial literacy (2020) - a bunch of things to understand about our modern industrial society, before attempting to change anything.

Friday Blast #90

Horia Coman — Fri, 02 Oct 2020 07:00:00 +0000

More testable code with the hexagonal architecture #video (2019) - the hexagonal architecture is the name given to an clean and interesting approach to building large “business” applications. There is essentially a core of logic, surrounded by “ports” for persistency, communication, etc. and “adapters” that allow the logic to make use of the ports.

Hexagonal architecture in DDD #video (2015) - another take on this, with some domain driven design bits thrown in. A lot of folks are doing both of these at the same time, and the match seems good.

How we migrated Dropbox from Nginx to Envoy (2020) - in three words - the hard way! This is a surprisingly detailed look at what I can imagine is the equivalent of switching the engine in flight on an airplane.

Brief history of scaling LinkedIn (2015) - The article was kind of light on the details, and is quite out of date. On the other hand, LinkedIn is an often overlooked tech powerhouse, with a lot of custom technology they had to build in order to handle their scale. So it’s well worth a read to see when and how these things got their start.

Work on what matters (2020) - can’t have a friday blast without a Will Larson post. The main idea is that as one advances through a career (not in time, but in impact and scope), one needs to deliver more in less time. Hence, you need to focus on what matters, and aggresively cut non essential work (drop it, or delegate it, basically).

Announcing Jupiter Release 0.5.0

Horia Coman — Tue, 22 Sep 2020 07:20:05 +0000

As promised, the 0.5.0 release comes much faster than the 0.4.2 one. It includes a single change, namely smart list. It’s a significant addition however, and the first foray outside task-planning for Jupiter.

Smart lists essentially are lists. You can use them to build lists of books to read, movies you’ve watched, etc. Long term, there’s gonna be some actual smartness added to them - syncing with YouTube, or Amazon, or Spotify, but right now they’re pretty basic.