How to solve 5 kinds of architectural tech debt

Phillip Voyle — Mon, 09 Feb 2026 18:56:25 +0000

Tech debt is a pretty course term
In general it’s technology choices or practices that may have initially allowed your application or architecture to move fast, but that now slow you down

Coming through university I knew about reuse and polymorphism and once I got out into the industry, this was about 2002 I learned about design patterns and started to think clearly about the interactions between those structures that allow reuse, but it wasn’t until a few years later that I started to notice problems around me where rot had started to happen. When I think back, the first commercial position I took they had a pretty clear architecture, exceptionally high staff retention and some really talented programmers and really I didn’t get to see or experience technical debt until I got into the game industry a few years later, where the rush mindset and staff churn play a larger part

There are different kinds of tech debt and they are all caused by different forces in your organisation. Unfortunately it’s not always obvious you’re creating tech debt until you see the friction that a particular solution causes, but 20 years of software architecture and design has taught me that there are some constants

Here are 5 kinds of tech debt and what to do about them

Depreciation

Software doesn’t rot so much as go out of fashion
If you don’t regularly and carefully review the packages, libraries, methodologies, frameworks, runtimes and practices that you depend on you’ll find that one or all of them will become dated. Deprecated packages tend to form islands of supportability and at least some of the time you’ll find that a whole chunk of your system will have to be upgraded to deal with a dependency or platform going out of support. Not dealing with this regularly and repeatedly will require you’ll have to stop work and do an emergency upgrade which could mean months of development work and testing to ensure you haven’t broken anything

To deal with depreciation you need to book regular dependency reviews. Quarterly, biannually, or yearly are pretty good choices and won’t overwhelm your schedule with upgrades

Entropy and Mediocrity

Mediocrity will always tug at an elegant solution and try to pull them back into the mediocre. This often happens when a single developer has to deal with a special case that a common piece of code doesn’t strictly accommodate. Often this looks like the special case being hard coded into that general purpose logic, making it more fragile and specific and reducing its ability to be reused. Reusable logic takes maintenance and boundaries that are regularly enforced

There’s only one way to deal with this, it requires that code is consistently to a high standard. If you allow your expectations and standards to drop you’ll let mediocrity into your code base and that will set a diminishing expectation that will cause your application to rot from the inside. If you’re in charge of code quality, you need to set the bar and hold it high

Dependencies

This is almost a special case of the previous kind of tech debt. Developers will need a part of a library or package and simply add a reference. This may unwittingly cause them to bring in other dependencies that aren’t relevant to that problem. Over time your system will become a web of interconnected and implicit references which will slow builds, and make choices about testing and deployment boundaries difficult or impossible to make. There’s only one way to deal with this and that’s carefully reviewing dependencies, splitting components and intentional re-engineering of code that has a murky reuse proposition.

I’d urge you to decide whether your component is shared or not, and if it is, extract it and make it general purpose

Overengineering

Overengineering is often the response to a difficult problem that is impossible to generalise. The developer attempts to make concessions to the problem space by accepting complexity, and special cases. I’ve been guilty of this several times, and I think the problem here can be summed up by my attraction to problem solving and complexity. Solving hard problems is fun and challenging, but beware. The more complex your solution has to be to solve your general problem, the more difficult it will be for other developers to pick it up. If other developers can’t pick up your work there will be friction that makes change hard, and it will get you stuck on a project that you’ll never be able to progress from. This general solution will also attract entropy and over time will collect nicks, cuts, hacks, and open gashes.

To avoid overengineering, shrink your problem space and allow yourself and your team to create a second solution for a special case

Manual Processes

Personally I think you have to face that not everything can be automated, and in this day and age that’s a good thing for us organic life forms. But there are a great many processes that if you spent a bit of time looking at how much you do it and how much variation there is in the process then you’ll find that it’s actually pretty samey and tedious. So maybe have a look at automating it. Automation does something else that’s pretty magical that I don’t think enough people talk about. Codifying a process also documents it in a formal way so that clicking the button isn’t the only thing another developer can do. Instead they can look at and understand the activities in the process and understand the mechanisms that produce the outcome

I will offer one caution about automation though. Ceremonies and practices are an important kind of activity that are healthy for team cultures. A code freeze and promotion might be a single click, but the milestone markes a change in phase in a team’s workflow and it’s important not to lose that sense of co-ordination

Is reference counting a solution to DB Garbage Collection?

Phillip Voyle — Fri, 22 Aug 2025 08:58:06 +0000

For the past 4 months I've been working on a side project to develop an object database from scratch, and chronicling my work and learnings on my YouTube channel. To date I've managed to implement an on disk B-Tree mechanism with caching to improve read performance and a way to separate different transactions into different files so that history is preserved
Things are starting to work pretty good, but I'm quickly getting to the point where I'm going to have to start thinking about how to compress old transactions once there are no longer relevant references to remove any blocks that are now garbage

How is garbage created?

In the object database I've been working on, garbage is created when a B-Tree node in an older transaction is superceded by a newer version of that node either by adding or deleting an entry to that node, sometimes also incorporating merges and splits of nodes depending on the operation. The nodes of the current transaction are allowed to be overwritten during the transaction and so are not currently able to be superceded during the transaction, however older transactions are treated as immutable. I partly wanted to do this so that it was optional to collect and compress older transactions, but for now without a collection mechanism, history retention is mandatory, and over time the garbage builds up. I'd like to start addressing that factor soon

How should garbage be detected

In most typical cases a B-Tree node will have exactly one parent, however when a node is superceded by another, that node's children will temporarily have two parents. There might be several references from stale parent nodes until those nodes are collected, but I'm anticipating those nodes might stay active while there are read-only operations occurring on previous transactions and where those transactions have the consistent-read transaction mode. When a node is superceded, one option would be to immediately move the parent, but this will add overhead to whichever operation is in progress. One option I'm considering is to stage node supercessions into a buffer that can be scavenged by the garbage collector. The collector would increment the reference count on a set of referred nodes, and then decrement the reference count on the stale node, and when said node reached zero references, it could be added to the collector list. This allows two things to occur: the collection process will be incremental, and transactions won't have to wait for it

How should garbage be collected?

For now the nodes are referenced via a file and offset pointer with a typical small transaction confined to one file. I decided on this layout because in the beginning I was anticipating that I would want to restructure or remove files to reclaim disk space after nodes were cleaned up, but the more I think about it the more I think that the block files may need to be compacted. One way to reuse the space is to add a free list to the data store and allow new blocks to be allocated from the pool of previously allocated and freed blocks. This is a pretty good technique to ensure that space isn't used inefficiently but has the drawback that once data is allocated, it's very difficult for applications outside of the data store to reclaim the space. One mechanism I'm considering for improving this situation is to resize old files when their blocks are collected. Unfortunately this means that those blocks will need to have all of their free space at the end of the block. One way to ensure this might be to map block file entries to an offset using a per file index. This would incur an additional overhead and it's unclear to me whether this will be worth the payoff of reduced size. Probably this is something I should measure, and perhaps I can make that a setting that may be optional too

Thanks for reading, and if you're interested in what I've been up to you can find my work here at https://github.com/PhillipVoyle/objectdb, and my youtube channel here https://www.youtube.com/@phillip-voyle

Forem: Phillip Voyle