Forem: Kieran Bond

This is a well-written, technical post. It also features some excellent external links, making it highly recommended reading for everyone developing!

Kieran Bond — Tue, 15 Jul 2025 15:17:36 +0000

Isaac Hagoel

Jun 17 '24

Are Sync Engines The Future of Web Applications?

#webdev #svelte #javascript #programming

233

Comments 25

15 min read

I'm investigating the pain points people are having with conducting user interviews/research! Help me out and fill out my survey? https://tally.so/r/wop2BX Thanks in advance!

Kieran Bond — Thu, 08 May 2025 19:36:32 +0000

Why you want cross-cutting product teams

Kieran Bond — Tue, 03 Sep 2024 14:24:46 +0000

I've recently been reading/listening to 'The Lean Startup' and 'The Making of a Manager' (Disclaimer: Associate links!), so this may be the impact of thinking about startups a lot and how they function. However, I think that in more 'established' companies, we do things inefficiently and don't ever try to get better. Startups are great at getting ahead and disrupting because they need to be impactful instead of stagnant.

In traditional 'enterprise' software companies, you can almost guarantee that everybody works solely within their area of expertise: their ‘silo of expertise’. They are isolated from each other. Engineers work exclusively on engineering tasks, salespeople focus solely on sales, etc. Each department operates separately, leading to a lack of collaboration. For example, in developing a product, engineers write the code without involving designers until much later, when they ‘throw it over the fence’ to the other ‘silo’. The ‘silo’s’ never really interact or collaborate, and nothing is done promptly.

We should strive for something more akin to startups. Cross-functional teams designed around a product to focus on that one product and not their 'silo of expertise'. Let's call a simplified version of this structure the 'Product Silo' (aka, a cross-cutting product team).

What it means to have a cross-cutting product team
Why would I want a cross-cutting product team?
Why don’t companies adopt a cross-cutting product team structure?
The unsolved problems

What it means to have a cross-cutting product team

So what? What does it mean to have a cross-cutting team? Just another name for the team building the product, right?

Nah. It's foundational; it's cutting down the barriers. It's a way of working.

When you're in a silo of expertise, your team is other experts in your domain. In a cross-functional team, your team is only the people needed for the product to succeed. Rapport and collaboration are different - we are all biased toward working better with our team than with generic colleagues.

There's no throwing shit over the fence and blaming the salespeople now, you're one team. It's like the shift-left movement of DevOps; you become one team with one focus.

Instead of putting in a request to design to mock up the UI flow, you sit with them and build it together. Instead of sales selling every product the company makes, they sit with the product owner and engineers to have deep product knowledge and know all about the upcoming features they can convince prospective clients they want.

So, what does it mean to have a cross-cutting product team? It means having a team of experts dedicated to one product.

Who's in this supposed team, then?

Everybody who needs to be to make and sell the best damn product you can.

The non-exhaustive list, if you need somewhere to start:

Product owner
Product engineers
Product support
Sales
Marketing
Design

Why would I want a cross-cutting product team?

Great question.

You’ll have a product that users want, with fewer issues.

But you don't just get that automatically by forming the team. The team gets there (eventually) for multiple reasons.

Feedback loops are shorter.

You'll get the right product in the hands of the right people quicker.

It's easier to get direct user feedback to the correct people because they're the teammates of the ones who hear it first - Sales, product support and the product owner. They can pass a note over as they're on the call; it's that quick. There's no barrier to break, no having to go through multiple team leads or jump through hoops that make you give up. Tap them on the shoulder (as they probably sit next to you now) and tell them. Better yet, take them out to clients with you - and no hoops to jump through to get that approved.

Sales can get all the metrics/insights they need to help sell the product directly from the engineers - or even be given hands-on ability to get the data because they're a trusted team member. They can hear from the team what's in the pipeline and are ready to start telling clients about it, and they can say to the product owner right after a call what the clients are screaming for before they can make the sale.

Support can work with the product owner to ensure they have the proper domain knowledge to help customers. Work with engineering to eliminate the biggest problems that keep occurring or get playbooks in place that make the resolutions quicker.

Marketing, design, and engineering work side-by-side to build the best damn-looking product out there - and the iteration is tenfold quicker because there are no scheduling issues.

Quicker feedback loops lead to teams being more efficient and more agile. The whole team can pivot and shift focus on what's most essential to the product without any roadblocks or re-alignments needed across multiple teams - it just happens.

Deep knowledge base

Everybody on the team will have a deep knowledge base about the product; they'll be able to chat your ear off about it.

What this translates to in value, however, is:

No bus factors in knowledge
A team of product experts, people who can knowledge-share with others and answer questions
A team of people who think differently can contribute to solving a complex problem - instead of lots of people who think like engineers (for example)

Cross-cutting expertise

Let's take T-shaped engineers to another level. Let's build a T-shaped team.

Okay, maybe not—that's too much of an ask. But we can get a team of experts with basic cross-domain understanding instead.

When you have a team of people with varying expertise, you'll find that everybody starts to be able to speak the same language. That's because they all develop foundational skills for each expertise they interact with.

Suddenly, sales can have a more technical discussion with the CTO they're trying to convince to be a client.

Marketing can use the domain lingo with an understanding that entices users while collaborating with design to provide consistency across their advertisements and products.

Engineers can better support users and experience dogfooding the product and support pressures while cross-training their support peers to have a deeper understanding of the technical details that enable them to help users better. They can join sales on a call with a prospective client and demo the product without blundering and putting them off.

Fewer distractions

What slows down a lot of delivery? Distractions and context switching.

What causes distractions and context switching?

Needing to interact with other people, particularly those outside of the team.
Different projects/products requiring your input

Bring the people into the team and the inherent trust circle, and put their sole focus on one product.

By having one sole product to focus on, there's no real need to context switch. You don't need to learn about another domain or product regularly so that you can contribute your expertise to it; you just contribute to the one you're now an expert in.

You become an expert in the two things that matter and need execution: The product and your skillset. You're efficient when an expert, so keep the expert's experts.

Being in the circle of trust of the team also has its advantages; there's less back-and-forth. For example, the engineering team can teach sales how to use the analytics tool and only need to teach two people - then they can be trusted to have full access and not break it. They don't need to distract engineering ever again for this data.

Better business understanding

When working in silos of expertise, it's effortless to fall into the trap of focusing solely on your 'craft'. For example, engineers may chase 'perfect code' instead of recognising the cost of refactoring because they're focused exclusively on producing the perfection of their craft.

By contrast, a cross-cutting product team has a broader view of the business—it becomes easier to see and understand the product's impacts on the company.

As a team, you're all closer to the impact of decisions, and you can see the pressures every team member has to face. You'll hear the sales experts talking about how they still need to find £0.5m to hit revenue targets for the product, and you can help them get on top of that because now these targets hit closer to home. If your product doesn't hit its goals, you're closer to the chopping block than when you are merely in a silo of expertise.

A significant problem existing in ‘silo of expertise’ companies is that everybody is disconnected from focusing on making the business/product profitable. It’s not their responsibility, and they have no direct impact on the bottom line (or so they believe).

Why don’t companies adopt a cross-cutting product team structure?

Why isn't everybody doing it if it’s so great ‘KB’? Great question!

Inertia

Inertia is required to make an idea come to life. Once the ball gets rolling, it’s much easier to get buy-in from others.

Gathering inertia in a company requires political capital, especially for a change like this, which is probably controversial, ‘unknown’, and challenges egos.

‘Unknown’ ideas are feared as they can cause disruption, making it hard to get buy-in from people with the capital to spare.

Cost of change

Rome wasn’t built in a day, and there’s no Trojan horse.

Changing tact across a whole org is costly, and many companies don’t like experiments—especially when they take a lot of time to prove.

Cultural barriers

Some cultures don’t care for teamwork and are more finger-pointy. Cross-cutting product teams require collaboration, trust, and selflessness to work.

Everybody shifts away from the comfort zones of their silos/departments and has to learn to work closely with other experts. They’re no longer ‘the’ expert. Some will struggle with this shift from focusing on being top-dog, particularly those in top-down cultures.

Good enough is good enough.

Sometimes, chasing perfection isn’t worth it, and companies are happy with ‘good enough’.

Why risk changing something that already works(ish)?

The unsolved problems

I think cross-cutting product teams are more effective than silos of expertise. But look, perfect doesn't exist. There are problems with every approach.

My unsolved issues are either unsolved because I don't know the answer or because I'm not sure I'm right.

Cost

These teams are going to cost more to run than traditional silos of expertise, in a sense.

'One salesperson per product?! We're going to need 5 salespeople per client!!'

On paper, you're going to want more people. However, there are tradeoffs.

While you may need more of each role, like the above sales example, you will have more efficient teams. Does this mean you can run leaner overall as an organisation? Possibly!

You may also reduce labour hours. If you've got experts focusing on problems (such as a member of support dealing with a client outage), they can find the solution quicker than a non-expert. That reduced labour time could add up to less cost overall.

Multiple teams for one product

Extensive products may be too big for one team to handle. It then becomes tricky, as you don't want large teams - it'll likely fall back into silos of expertise.

The question is: Do you need multiple teams?

You're trying to do too much if you answered yes. You don't. You should make smaller, more focused products - split your 'big' product into multiple, more defined products.

If Amazon's two-pizza team size is accurate, then most AWS products are owned by a team of 10 or fewer people. If they can build things like S3, Lambda, etc., with that few people - why can't you?

Splitting products up will cost money, so it’s probably a significant deterrent for companies that already have their products tied together.

Team growth is more challenging.

How do you grow your team of experts professionally? Being mentored by experts is a big part of learning to perfect your craft. Your team will be filled with experts but not with the expertise the mentee needs.

More senior teams are needed.

It is harder to grow teams professionally, so you probably need more senior teams. Hiring juniors is now less effective (fewer mentors), and thus:

The teams cost more to hire and run.
Juniors are not coming up through the ranks, making the team less sustainable in the long term.

Summary

Cross-cutting product teams are a strategy you can use to change how work works in your company. Move the focus away from expertise and to the product - align the right people to the work, leading to better outcomes from a leaner team.

I'll add a cheeky disclaimer here, too. I've never fully experienced working in a cross-cutting team, so some of this is hypothetical, but I have tasted the forbidden fruit. It works.

However, I'm just an engineer trying to do things right and efficiently. I'm probably deluded and drinking the Kool-Aid.

Measure twice, cut once

Kieran Bond — Thu, 01 Aug 2024 14:42:50 +0000

While renovating my house, my partner's dad guided and assisted me. It was my first real foray into DIY, but he had plenty of experience. The apprentice meets the master.

One thing drilled into my head throughout the renovation was that you go faster when you make fewer mistakes, which resonates generally in life. One of the easiest ways to avoid mistakes is to 'measure twice, cut once' — a popular saying taught to apprentice tradesmen, I'm sure.

We should take this approach with software: Make fewer mistakes by measuring. Too often, we make assumptions and act on them before validating them - these acts are mistakes because they lead to wasted effort.

Photo by Loren Biser on Unsplash

When should we measure?
How do we measure?
Counter Arguments

When should we measure?

To measure, or not to measure?

If you're making decisions based on assumptions or incomplete information, it's time to pause and measure. Rushing into action without proper validation can lead to costly mistakes. If you can't measure yet, the best decision might be to do nothing.

We're wired to believe that inaction is terrible; spoiler - it's not.

Let's take a look at two examples:

The trading system

You're building a system for trading stocks. It has two components: a front end (UI) and a ‘backend’ that accomplishes the work for the UI.

Your UI needs to allow users to create requests to trade stocks. They range from wanting to create one to one thousand requests at a time but average ten (according to product owners!). You already have the ability to create these requests on your backend, but you need to trigger this functionality one request at a time. To improve the user experience, we’ve been tasked with creating a UI that allows users to create many requests to trade at once.

Engineers are discussing creating new functionality on the backend to allow the creation of multiple trades simultaneously, with two purposes:

Simplifying the work for the engineers trying to 'bulk' create trades on the UI.
Reduce performance concerns of calling the create functionality thousands of times.

So, what do you do?

I understand the instant urge to say, 'Create the ‘bulk create’ functionality on the backend, easy.' I've felt that urge; it's an easy answer and sounds like it solves the problems.

Stop. Do nothing. Breathe and measure.

Before you commit to any action, validate your hypothesis. You will save time, prevent technical debt, and provide facts about system use and user behaviours. It's time to start adding measuring to your software development processes.

How do we measure?

First, we need to start fresh and remove both bias and assumption. We need to be analytical here and remove emotion.

Different hypotheses require different methods of measuring - Users might need surveys, and systems might need specific tooling.

Refrain from making assumptions about what to measure. Find evidence that the metric will answer your question.

Let’s work through the above example. First, let’s reiterate what we’re trying to solve:

Simplify the work for the engineers trying to 'bulk' create trades.
Reduce performance concerns of calling the create functionality thousands of times.

There are two assumptions here, so let’s actively call them out. Assumption one : Enabling users to create bulk trades if there’s a specific endpoint is simpler for engineers.

Assumption two : Designated bulk create functionality will be more performant (latency and throughput) than calling the single create functionality individually each time.

If we don’t highlight these assumptions more factually, it’s easy to get lost in what we are trying to analyse - we are discovering the hypothesis’.

There are four assumptions. Did you spot the third and fourth?

Assumption three : Users create bulk orders regularly.

Assumption four : Users create large (1000+) quantities of orders in bulk.

Without assumptions three and four, one and two cannot be accurate, and without assumption three, four cannot be true. We should understand this pattern of dependency to avoid over-investigating when not needed. Let’s investigate assumption three first, as it’s the root of all others.

Assumptions three and four

We must understand user behaviours and measure how often orders are created singularly or in bulk - assumption three. Then we can investigate assumption four - How many orders are made per ‘bulk’?

If you have an observability system connected to the API, it should be as simple as recording every time each endpoint is used and which user it relates to. You can then aggregate orders over a short period, grouping by user, to see how many orders are being created in bulk. If you have access to observability on the system producing these ‘bulk’ orders, you can more accurately observe there instead.

If assumption three, that users create bulk orders, is satisfied, and assumption four is satisfied (the quantitative number is now confirmed), we can justify spending time examining the remaining assumptions.

Assumption two

Next, we need to determine if there’s acceptable latency at the expected throughput— assumption two. Thanks to assumption four, we should know the expected throughput. So, the question we need to answer is: What’s acceptable latency?

It’s subjective and depends on many factors. The easy answer would be ‘whatever the product owner decides’, and that’s ok if they have that answer. Otherwise, it’s complex and requires a lot of research.

Some things to consider might be:

Does the user care about individual requests succeeding? Or is it only about the whole bulk succeeding?
How time-critical is this functionality? Are users blocked from doing further things until it is complete and they have visual confirmation?
Where does the latency requirement sit? Is it in the confirmation for users in the UI (the whole process), or is it just in the sending of the request to another system (a sub-process)?

In this example, latency isn’t too high on our priorities as it requires manual (other) user input on another downstream system, and our user is not blocked waiting for that confirmation. This means we can essentially throw away the need to satisfy this assumption.

In another world, where we have a latency requirement, this comes down to observability and performance testing. You need to set up scenarios that stress test the system to see if it meets the determined latency requirement. The great thing about that is that you will then be able to performance test your system at will because you built the ability to when proving this out.

Assumption one

Our last assumption ( assumption one ) to review is that enabling users to create bulk trades through a specific endpoint is simpler for engineers. Engineering productivity is really hard to measure, so maybe that is the case, or maybe it’s not—you’ll need to do your own measuring/study here, I’m afraid.

Where does that leave us?

Assumption three - Users create bulk orders regularly ✅

Assumption four - Users create large (1000+) quantities of orders in bulk ✅

Assumption two - Designated bulk create functionality will be more performant (latency and throughput) than calling the single create functionality individually each time ❌

Assumption one - Enabling users to create bulk trades if there’s a specific endpoint is simpler for engineers. ❓

Depending on which way assumption one swings, you need to make a decision. Here are my thoughts:

If assumption one turns out not to be true, then just stop caring. You’ve measured and determined it’s not worth the effort. Win.

If assumption one is true, then you still need to weigh up the cost. It costs to build the required functionality to ‘simplify’ the work. Does that cost more than the (technical debt) cost of leaving it as it is?

Weighing up technical debt cost against engineering cost is hard and probably subjective. I don’t know how to measure that, but I’m sure somebody on the internet has something less subjective than gut feeling.

Pitfalls to avoid

Before you get going on measuring everything everywhere, there are some pitfalls to be aware of.

Measuring only once. Look at the title; it doesn’t say measure one. It says to measure twice—in fact, measure continuously. Don’t just measure twice and call it done; keep measuring after the cut and see what you can learn. The secondary impact of all this measuring is the learning.

Measuring for perfection. Sometimes, it’s just not worth measuring absolutely everything. Do you need to invest time measuring productivity when working in a specific codebase to know if it’s worth refactoring to something more manageable? That’s a big investment upfront, and it's hard to measure the impact - especially on something as subjective as code. How do you measure the effectiveness of the action? My point is that it’s okay to trust your gut and let your brain do that automatic thing where it tells you the answer without spending brain cycles deliberating it. Previous experiences do still count as long as we make sure to take lessons from them.

A subset of measuring for perfection is spending too long deciding what to measure or how to measure. You’ll reach analysis paralysis, get nowhere, and just waste time. Make sure it’s going to have a worthwhile impact if you’re going to spend a long time deliberating it.

Measuring the wrong thing. In counter to not measuring for perfection, don’t measure the wrong thing. This isn’t the worst thing in the world if you still learn from it and can use the measurements, but it’s still a waste of time. Do spend some time trying to make sure you’re measuring the correct metrics!

Counter Arguments

Measuring may take longer than implementing (I can't measure it, I need to build the tooling to measure it, or it's very complex to measure).

Yes, it might. But with it in place, you'll be able to measure more things and objectively discuss other pieces in the future. It has a further impact than ‘just now’.
Yes, it might. But you won't build something unneeded and won't have to remove it / have the tech debt of maintaining it.
If it's too complex, you may be trying to measure too much at once or measuring the wrong thing.

My previous experience tells me that I am right in my assumptions. It’s a waste of time to measure it all first.

How did you find out that you were right last time? You measured.
Where’s the harm in proving it out? Having definitive evidence is always better when staring down the barrel.
There are great learning opportunities for everybody involved in measuring - it’s not just about you being right.

Summary

Stop and measure once in a while. Commit to knowledge before action to make informed decisions - It’s easier to account for when things go wrong, and hopefully, things should go wrong less frequently because you’re using real data to form your actions. If it’s too expensive to measure, then take no action. Measure once the expense of measuring is less than that of not.

Figuring out what to measure is hard and involves confronting our biases and assumptions, but it’s worthwhile. Data is hard to argue with. You’ll save yourself a lot of time and work in the long run to make better products for your users, which is what we’re here for at the end of the day.

The conclusion to the example

So what happened in the end? We did nothing. It wasn’t worth measuring because we had nothing causing us to need to measure - we didn’t have users complaining, and the engineers could still easily build their functionality. So, we didn’t measure and took no action.

The ‘issue’ just went away.

Maximising one-to-one effectiveness

Kieran Bond — Fri, 02 Feb 2024 15:33:56 +0000

Photo by Priscilla Du Preez 🇨🇦 on Unsplash

As I've grown in my career, I've realised that one-to-ones are a critical aspect. You can accelerate your career and capabilities with effective one-to-ones, but making one-to-ones productive is challenging and ever-changing.

There are a few different types of one-to-one meetings. I'm going to focus on the kind of one-to-one you have with your team lead - often used or seen as a 'reporting' meeting. People usually tell their boss exactly what they've been up to since the last one, hoping to portray themselves as productive - but that's different from what your one-to-ones should be about. Using it as a reporting meeting is an antipattern and not worth wasting time doing.

The reality is that one-to-ones should be a helpful and productive use of time - they’re short bursts of focus from your team lead, which you can maximise. Do so by keeping them about these three things:

You.
Your career.
Your relationship with your team lead.

Topics of focus
One-to-one environments
Frequency of one-to-ones

Topics of focus

As I said above, you should aim to have productive one-to-ones, and the most productive things to focus on are you, your career, and your relationship with your team lead.

Having a rapport with your team lead allows you to build trust, which goes both ways. You trust them; they trust you. With trust, you can have more open and frank conversations and don't spend time dancing around trying to avoid offending or upsetting. It allows for more truthful insights from both ways, expanding how you can help each other.

So, what do you talk about in your one-to-one? In a non-definitive order of importance:

Goals - career and personal
Things going on with and around you
Feedback
Opportunities
Frustrations
Recent achievements
The business - context, goals, things happening
The team

Goals - career and personal

Having goals is essential for both your career and your self-growth. Goals help give direction (in and out of work). These can be as simple as 'Getting to the gym every day' or as stretched and ambiguous as 'Accelerating development across the org'. You're good if you have achievable objectives that you both agree to and feel are worthwhile pursuing. Talk about creating new goals, progress towards current goals, and what support you need to achieve your goals.

When looking to create new (career-oriented) goals, think about your key areas of growth - where is there the opportunity to grow, and what areas are worthwhile growing in? Ensure they’re measurable and achievable goals, so you know how well it’s going. It’s also worthwhile having the goal be something you’re intrinsically motivated to achieve, as otherwise, it’s very easy to let it slip.

For example, technical writing may be a worthwhile skill in your job - perhaps write regular blog posts to practise and improve.

Things going on with and around you

It's good to keep your team lead in the loop about your life - if you feel comfortable sharing it.

If you keep your team lead in the know, they can offer you personal support or support from the company and provide air cover if needed.

An example of things happening with you could be something as simple as your kid having trouble sleeping recently or as complex as a medical diagnosis. It doesn’t have to be something big. Just keep them in the loop - otherwise, you risk an assumption being made when it starts visibly affecting you.

Keeping your team lead in the loop will also contribute towards a trusting relationship as they’ll be more empathetic towards you.

Feedback

A big one! Feedback is hard for everybody, but remember that the intention is to improve. Sometimes, asking for advice or perceiving feedback as advice will yield better results; advice feels less formal and can change the tone perceived.

Feedback is another topic heavily driven by goals but less so than opportunities. As I said above, your team lead must be helping you grow; offering feedback is a big part of that.

Feedback can be as simple as something your team lead has observed, or it can be an opportunity for you to ask for specific feedback, such as asking them to observe you in a meeting coming up so that they can offer targeted feedback - if one of your goals centres around involvement in meetings, for example.

Feedback/advice is a significant topic in itself and is tricky. A better relationship with your team lead might offer more insight, as they may feel more comfortable giving you direct feedback. Ideally, you don't need this, but as we're all human, having that relationship there will make it easier for everybody involved.

Opportunities

Goals really help drive this conversation. If your team lead has their ear to the ground (hopefully they will!), they'll know about things happening around the company that may align with your goals and can get you involved. You can then start discussing these together, as a critical team-lead responsibility is ensuring you grow professionally and personally - even if it means transferring you to another team offering better goal alignment.

An example would be that you want to learn more about cloud infrastructure, and a role has just opened up in the newly founded platform team. Your team lead heard about this vacancy and has suggested you go for it to get closer to your goals.

Frustrations

We all get frustrated; it's inevitable. Some more than others (I'm pretty bad at this!). Letting out your frustration is important as otherwise, it builds up and becomes resentment. Venting some of this frustration to your team lead can be productive if it's something they can help with. However, keep this from eating up your one-to-ones, as it can turn into a negative feedback loop.

Try to vent to your team lead privately instead of publicly, as otherwise, you risk demoralising those around you, again causing a negative feedback loop and making the team unpleasant.

Don’t linger on venting frustrations for too long if you can. Remember what you aim to achieve in your one-to-one so you can refocus the session.

Recent achievements

As much as your team lead may try, it can be easy for them to miss everything of worth that you do - especially if you're part of a big team.

Your one-to-ones are an excellent opportunity to highlight your recent achievements, ideally ones with tangible outcomes. Highlighting achievements helps ensure that when your team lead gives feedback or considers performance reviews, they have a better picture to paint.

The business - context, goals, things happening

Akin to opportunities for you, your team lead likely has more context around decisions made and what's happening where.

One-to-ones are an opportune time to get more clued up on anything you're unsure about within the business, primarily how it feeds into your growth and goals. This is also the ideal situation to ask questions you’re uncomfortable asking or may be riskier in a public forum.

The team

Talking about the team in a one-to-one is tricky and is only sometimes a good use of time - as the one-to-one should be focused on you and your team lead.

However, if your relationship with your team lead is ‘high trust’, you can offer more genuine feedback about your team members - your team lead may even expect this as your seniority increases. As mentioned above, feedback is a significant topic and probably warrants its own post - especially giving feedback.

If you have less granular feedback about team members to offer or the relationship isn’t ‘high trust’, then keep feedback high level here rather than getting into the details - it takes less time and reduces the chance of putting your foot in it. Some high-level things to discuss can include the team's ways of working or milestones being worked towards.

Don’t forget - this is also a great chance to praise your team members!

One-to-one environments

Where your one-to-one takes place can impact the effectiveness of your one-to-one as it affects the tone/atmosphere from the get-go. Sometimes, it's even worth choosing the environment per meeting based on what you're looking to get out of the session.

I'm not a fan of choosing a meeting room as it makes the session feel very formal and feels like a meeting with 'management', which always puts me on edge. If I need a laptop or notepad, I'll choose a more open and communal space with somewhere to sit, and we'll chat over a coffee.

If you're unsure, going for a short walk is a great default. The exercise and fresh air are good for you and will make you feel more engaged. Also, the lowered formality of walks themselves, coupled with walking side-to-side instead of sitting face-to-face, may make you feel more at ease and make any long pauses less uncomfortable.

Frequency of one-to-ones

How often should you have a one-to-one with your team lead? That's up to you and them to discover.

Start more frequently, like once per week, and work backwards from there. I like the interval of once a fortnight as enough time passes between catch-ups that things worthwhile discussing happen, yielding a productive session without being so far apart that things get missed.

Even if you have a scheduled one-to-one, you can cancel it. As a rule of thumb, it’s better to organise more frequent one-to-ones and cancel when not wanted - just make sure to avoid this becoming a habit, however.

Conversely, you can organise extra one-to-ones ad-hoc, given you both can find time. If something has come up that you want to ensure you discuss, then put a meeting in. Don't just sit on it until the next one-to-one if it's important.

In Summary

Everybody has different one-to-ones, with different goals and ideals for productivity. Don't just follow what I've said here; take a deep look at what you want to get from them and tailor it to your needs. The time allocated for the one-to-one is your time, so you should aim to maximise it.

Treating a one-to-one as a formality and not planning ahead (or at least having an informal agenda) to get the most from it isn’t worthwhile for either of you attending. You’re missing out on opportunities if you don’t maximise it. You have your team lead's full attention; use it.

So, try customising the format/style until you think they're successful. Mix up the environment, too; it has a more significant impact than you think.

Have any tips for having more effective one-to-ones? Let me know; I'm always looking to improve mine.

Succeeding with the 3 Musketeers pattern

Kieran Bond — Fri, 19 Jan 2024 22:14:23 +0000

Photo by Recha Oktaviani on Unsplash

Why use the 3 Musketeers?
What is the 3 Musketeers pattern?
When can you use the 3 Musketeers?
An example of using the 3 Musketeers pattern

Why use the 3 Musketeers?

The 3 Musketeers pattern has, to me, three (funnily enough!) primary reasons to use it:

Consistency

All environments you use it in should(!) behave the same. Developer machines, CI/CD pipelines, etc. Debugging issues that arise, such as in your pipeline, is considerably more accessible when it behaves identically locally. Rather than pushing code up and waiting for your pipeline to tell you it's incorrect, you can test it out locally first in an identical manner and save yourself some time.

Ease of use

One command to rule them all - Make doThing. No worrying about every developer fully understanding the madness under the hood. Your onboarding Engineers can get straight into the thick of it, and your team can easily embrace your latest addition.

Speed

Development is much quicker when you work consistently across the whole team, ideally org. When you have a pattern to follow for new commands, there's no thinking about exactly how to implement it - you follow the pattern. Following the pattern speeds up others, too; they don't have to learn a new toolchain for every new command.

What is the 3 Musketeers pattern?

I won't do a better job than this website, but I'll take a go at summarising here for those wanting a TL; DR.

'3 Musketeers' is a method that aims to keep your development repeatable and consistent by using three tools: (GNU) Make, Docker, and Docker Compose.

Using Make as an orchestrator and entry-point, you can use docker/compose to provide the environment where you then execute your code.

I’ve also recently come across a replacement to Make (as this isn’t Make’s real purpose) called Just. It seems to focus more on why we use Make in the 3 Musketeers pattern, so maybe it will replace it one day.

When can you use the 3 Musketeers?

So now that we understand what the pattern is and the benefits of it, when should we use it?

All the time. (Mostly). The 3 Musketeers is best when used for commands or functional tasks.

Its primary use case is for making your build pipelines simpler to follow and easy to develop locally. Any time you have a step that runs in your CI/CD pipeline, you should define it as a Make command and follow the pattern. Creating and updating the step behaves the same locally as your pipeline, so it should be a quick iterative process.

Another great scenario is whenever you need to run services in conjunction - such as for local testing. Have a microservice architecture and want to run all the services simultaneously to develop a new feature? Use the musketeers.

Onboarding a new Engineer to the team can be tricky and convoluted. Why not set up the environment using the 3 Musketeers? You can host most services (such as Databases) in a container, spin up the service they will work on, and then seed it with a bash script - all from one command.

Any other tooling scenarios are great candidates, too. Seeding, resetting, uploading, testing, you name it.

When not to use it

Sometimes, it’s really easy to get lost in the weeds with a new pattern and try to use it everywhere. Looking at you, design patterns.

While I highly recommend the 3 Musketeers for many things, don’t use it for everything. That’s an anti-pattern.

It's a lot less useful when it’s just one or two people working on a project or there’s a project with not many repeated ‘commands’.

The 3 Musketeers is also not great if all it’s doing is overcomplicating things. If all you’re doing is adding another place to run commands from, maybe don’t add them. For example - If you’re just in a Javascript environment (where you likely use Yarn/Node/NPM) and all of your commands for building, testing, etc., are already in a ‘package.json’ file, then unless you plan to add more automation across the project outside of your javascript I would just leave it be. You’re just adding confusion and another place that needs updating.

An example of using the 3 Musketeers pattern

We primarily use the pattern for our build pipelines, as it enables running and testing them locally before committing and pushing code up to the repository.

Let's review an example command for running tests on a .NET project in the build pipeline. I’ll omit the pipeline file to keep this as minimal as possible, but I’m afraid a small amount of prerequisite knowledge around Docker/compose will be required, and you’ll need to install Make and Docker if you want to do a hands-on follow along.

Here's a gist link for those who want to see the example pieced together. Here’s an example .NET project which you can follow along with.

So, the premise for the example is this: 'I want to run the unit tests for my .NET project in a CI/CD pipeline and locally as part of my development cycle.'. A very common scenario.

First, we need to figure out how to run our unit tests. It's very straightforward with .NET, luckily. In a realistic scenario, we would add extra things, such as static analysis, but I want to keep this simple enough. In my team, we would usually add this in by extrapolating the command into a Bash script.

So, to run unit tests in dotnet, we simply need to run this command in a terminal (with the .NET SDK installed):

dotnet test

That’s straightforward. It works on my machine; it's all good, right? Commit, push, watch it run…

Oh, wait, our build pipeline doesn’t know the dotnet command. It doesn’t have the .NET SDK installed, and I can’t install dependencies on the agent. Hmm. Now what? Dockerise it!

So, let’s use Docker to provide a consistent environment. Microsoft offers an official Docker image with the .NET SDK, which we’ll use. We’ll put it in a ‘docker-compose.yaml’ file like so:

version: "3.8"

services:
    dotnet:
        image: mcr.microsoft.com/dotnet/sdk:latest
        working_dir: /app/app/src
        volumes:
            - .:/app

So, we’ve added the ability to spin up a container with .NET installed and mounted our source code to the container when it runs.

Now, we could just call it a day there and, in our pipeline, run this command:

docker-compose run --rm dotnet dotnet test

But let’s simplify it for everybody as there’s a bit to unpack there, and a big advantage of Make is simplicity.

So, let’s create our Make file. I’ll call it ‘Makefile’. Here’s the first draft:

test:
    echo "Running Make test"
    docker compose run --rm dotnet dotnet test

Now, all we have to do in our terminal and pipeline is run the following:

make test

Isn’t that so much simpler? You can add that to your team documentation, where it will likely never change. If you had added the previous command, what happens when you want to change the behaviour? You might need to extend your test command to perform static code analysis, but then you’ll have to update your docs and pipeline and ensure your team knows they must run the commands the new way.

Once your Makefile expands to contain other commands, particularly similar ones, you might want to tidy up and follow some best practices. I’ve expanded the Makefile in the linked gist to include a build command and a slight refactor to improve the file. I’ve also extended it to utilise a bash script so you can see how to utilise it in a similar scenario.

In my team, we utilise Bash scripts alongside the 3 Musketeers. We use Bash scripts because most Docker images can execute them (maybe with a tweak), we want to do several things in a command, and we are comfortable with Bash.

I suspect a more traditional use of the pattern is to chain multiple Make commands together instead. The expanded gist does both of these.

In Summary

You might not need the 3 Musketeers; it has its downsides - including added complexity in small projects. But, if you have yet to use it or want some order to your chaos, I recommend trying it.

You don't need to use it for everything - we don't, but I strongly suggest it for your CI/CD pipelines at the minimum and anything automated that is repeatedly needed in your local development process.

Integration tests are king

Kieran Bond — Thu, 18 Jan 2024 11:22:29 +0000

Stop writing so many unit tests and join me on the dark side. Focus on integration testing instead of unit testing, and bask in the saved time and superiority.

Originally posted on my Substack here.

A caveat before we get into the meat of it:

Most of the test suites I work in are for Web APIs and deployed systems - especially recently, so I am biased. Writing integration tests for Web APIs is more accessible than other ‘systems’. When building other systems, such as libraries, consider ‘big unit’ tests but know that unit tests do still have their place.

Test suite composure
Why integration tests are king
The downfalls of integration testing
What makes an excellent integration test

Test suite composure

The ‘Testing Pyramid’ concept advises that most automated tests you create are the quickest to write and execute - often considered your unit tests. You have fewer tests as you move up the pyramid, but they are more integrated with the whole service. The three tiers from top to bottom are Unit testing, Integration testing, and E2E testing. Most people will refer to the testing pyramid as why you should have more unit tests than anything else. However, there are other options.

Unit tests are, generally, very targeted tests. They’ll often test a specific function with a particular input, expecting an output that exhibits a behaviour - or they should. This style of unit testing is a ‘small unit’ test.

These ‘small unit’ tests are perfect for critical behaviours/functions but quickly become cumbersome for comprehensive coverage. Most functional changes lead to you updating tests because these tests struggle to avoid implementation details, so at which point, why bother with them? You’ve fallen into an anti-pattern. Consider something better if you find yourself in this situation. Enter ‘large unit’ tests and integration tests.

One model called the ‘Testing Diamond’ or ‘Testing Honeycomb’ is more powerful (see the photo at the top!). Spotify highlights it as particularly great for testing Microservices. The main difference between the diamond and the pyramid is that Integration testing is where you focus most of your tests.

As a de facto, I believe the testing diamond model to be better than the testing pyramid - even outside microservices. The critical reason the diamond is better is that it moves you away from implementation details and you go ‘bigger’; you move to behaviour testing at the most controlled and significant scale, interacting with the users’ boundaries with the system.

What is integration testing?

The key focus in integration testing is to check how the system-under-test (SUT) behaves with the systems it integrates with. For example, my service may interact with another service - I want to test some behaviours dependent on this other service without it being there. If I rely on the service being there, I may end up with flaky tests that randomly fail when it is offline or when it has been broken. Creating a stub you can control and inspect for the other service is standard - a fake upstream system.

Integration testing allows us to test user flow through our system within our control bounds - at a high level. We can focus on how they interact with the system, from input to output, thus allowing us to determine if the behaviours we have built are correct without knowing implementation details. These tests have the added benefit of enabling us to assert there have been no side effects on other systems, as we usually have control over the ‘integrated’ systems via stubs/mocks.

Why integration tests are king

Integration testing reduces the ability to involve implementation details in your testing. They focus a lot more on input and output and thus are much less prone to failure and are easier to keep up-to-date.

You see, you’re doomed to fail when your tests involve implementation details. Implementation details require constant upkeep and lead to biased tests. Whenever your related feature changes, you need to update your tests, and it’s effortless to ‘cheat’ with your test and do the easiest thing (such as faking behaviour) rather than testing a flow thoroughly.

By doing tests at a larger scale, you don’t mock out everything that makes the function behave the way it does - you get a true reflection of the behaviour in your system. You write fewer mocks, and so there are fewer mocks to maintain. You write tests against genuine entry points to your system from a user perspective and assert against the same output your user would receive.

Another bonus to writing tests at the integration level is that they are generally ‘cleaner’. There’s rarely much setup because a stub already has defined behaviour and doesn’t need setting up for specific tests, so it’s simply the case of calling the entry point to your service and asserting on the result. I’ve had some tests be as small as two simple lines at this level.

If you’re writing behaviour tests at this scale, you’ll ideally need fewer of these tests than you do ‘small unit’ tests. This can be easier to manage than the (probably) thousands of ‘small unit’ tests you write in every project. It’s also quicker to get off the ground.

A great, overlooked advantage of integration testing is that you have service stubs. You need them for your tests, but have you considered that you can also host them in lower-release environments?

Hosted stubs allow you to have development environments that don’t have dependencies on other teams or systems. You can release more quickly into these environments and perform any manual testing you fancy in a controlled manner. This control is also great for introducing chaos engineering or performance testing. You’ve got no risk of breaking other systems, can alter your stubs to suit the situation, don’t need to talk to other teams when experimenting and have control of the observation on these dependent systems to ensure they’re not causing the trouble you’re investigating in higher environments.

The downfalls of integration testing

As always, it’s not all roses. When is it? If you find some silver bullets, let me know.

Integration tests can, and often are, slower to run. You need to start systems before testing, which is much more work than exercising a small unit. As the test suite grows, the execution time does too. Not that unit testing is exempt from that, however. I’m keen to discover whether this becomes negligible at a big enough test suite scale; as mentioned above, you don’t need as many integration tests as you do unit tests. The most considerable slowdown in integration tests is the startup process.

The beauty of being a ‘big unit’ is also a downfall. You’re not testing things at such a granular level; thus, it’s easier for tiny bugs to slip through the net. Arguably, they may not be bugs if they’re not affecting the overall behaviour - or your tests don’t cover enough behaviours.

Sometimes, depending on how your suite is made to run, it can be harder to debug/write integration tests - especially without clear thought on the Developer Experience (DevXP) when building the framework. The difficulty usually comes from running the SUT in a docker container to emulate a genuine service (dependent on your architecture, etc.). You’ll either need tools to attach debugging to this container or spend the time ensuring you can run it locally when testing. This ability is essential; otherwise, writing tests can be slow, bugs can be hard to investigate, and you’ll be limited in your ability to use TDD.

The most considerable downside of integration testing is that you now need system-level stubs. In most testing frameworks, creating mocks for ‘small unit’ tests is simple; thus, you don’t need to invest much time in getting them working for your tests. However, creating system-level stubs takes more time - often considerably more.

These stubs are more complex, and ideally, you want them to replicate the other systems’ behaviours, which means they take time to build - even if simplified. You can mitigate this by making an integration test framework one of your top priorities when starting a project - you can then evolve it over time with your system as needed.

Another downside to needing a stub - how do you know it behaves the same as the system it’s pretending to be? You can quickly instil false confidence in your system if you’re not careful. Mitigate this using E2E tests in your suite, potentially even having some that validate your stubs.

One problem you can sometimes experience when creating integration tests for a Web API is static typing. If you build your test suite in a separate repository, you may need help referencing the static types you use in the API. It would be best to keep the tests colocated to make them easier to work with as part of the development process; otherwise, people will not contribute.

What makes an excellent integration test

Short and sweet.

I’ve seen tests be as small as two lines (alongside some setup used for all integration tests). Keeping your test small and focused on a target behaviour is essential - making it readable and easy to comprehend. You can take a quick look at the test and understand the exact behaviour it is focused on and if it’s even being tested correctly (very useful for peer reviews). Make the call to your user interface and validate that the output/behaviour is seen correctly. That’s it.

Try to do most of the data seeding in your framework before running any tests. Ideally, the SUT is already configured before any tests are run to behave more like a real system - The closer to realism, the better. How often does your system get called without any data set up, really?

How I like to create them

I like to try writing integration tests first as part of TDD (and often skip unit tests altogether), so next time you’ve got a new feature or bug to write - give that a go. Before creating your new feature, define your behaviour in the integration test, specify the user interface in the test, and assert the output from your interface. Then, write your feature. The idea here is you’re focusing on the interactivity that matters - the users. If you concentrate on that up front, I find it makes the implementation more straightforward and is an easy point to start from.

I like using the Arrange, Act, Assert methodology in my tests to keep them focused and clear - as I do in my unit testing (see this post for more).

Once that test passes, you can refactor as much as you like. You can then decide what might also be worthwhile unit tests to write. Don’t forget that you can always delete tests - you can use them to reassure yourself without adding them permanently to the test suite.

An example

Suppose a trading system that depends on the market’s stock prices (market data). When an ‘order’ (trade) is created in this system, it wants to annotate the order with the current market price alongside the price it was executed at - this might be done to see the ‘improvement’ that the seller or buyer has achieved, aka profit.

So, the behaviour I want to test is: An order is annotated with market data prices, and the user receives the data correctly in the output.

public async Task CreateOrder_Annotates_With_MarketData_Touch()
{
    // Arrange
    var orderRequest = CreateOrderRequest(price: 42);

    // Act
    var order = await Client.PostCreateOrder(orderRequest);    

    // Assert
    order.TouchPrice.Should().Be(MarketDataStub.AskTouchPrice);
}

You would probably need a few unit tests to cover the same scenario, as it’s likely to encompass considerable parts of the system. However, you do need a stub to represent the market data integration, and depending on how you integrate with that system, this can be simple or tricky to create. In the example above, there’s a presumption the market data stub has already seeded the trading system.

You would want one (or many) unit tests to verify that market data updates are fed correctly through the system from ingress; you would want another (or more) to confirm that an order is annotated with market data. You would want at least one more to ensure the API returns an order object correctly.

Because that’s a lot of tests to write, I generally prefer integration tests when using TDD, as I think it helps me move quicker and just as accurately without compromise.

In Summary

If you want to feel closer to the authentic experience of using your service, write more integration/‘big unit’ tests. Do them first with TDD, and you’ll make the system better for users.

Try replacing some of your unit test suites with integration tests - you’ll hopefully see reduced coupling (to implementation behaviour) in your tests and a significant reduction in the number of tests that need to be written.

If you’re not creating a web API, you can still write larger unit tests focused on functional/behaviour flow. Try moving some code towards that instead of small, focused units.

Just don’t forget that other tests are still necessary; a well-rounded suite helps avoid the pitfalls of each test type.

Strong opinions, loosely held

Kieran Bond — Tue, 02 Jan 2024 15:39:38 +0000

Repeat after me: Strong opinions, loosely held. This is your mantra.

Forming a strong opinion is essential, and it's equally important to keep an open mind and listen to others. It’s a pivotal strategy for growing as an Engineer and pushing forward decisions. Form an argument you can back, but be willing to accept someone else's opinion if it's better, or respectfully disagree and commit.

A fundamental rule before we dive in: Ensure you believe any opinion you take and are not using it to play devil’s advocate - you need to believe it enough to execute it, as you may have the top opinion. Playing devil’s advocate generally causes more resentment and wastes time than productive discussion. It has a place, but it isn't worthwhile 90% of the time.

Credit where credit is due: I learned this from one of my team leads, who liked to repeat it often - and it's helped me a lot. Hopefully, it's helped my team, too, as we all try to follow the practice.

Why it’s essential to have an opinion
Traps to avoid
Generating an opinion on short notice

Why it's essential to have an opinion

Opinions are easy to come by, and just like assholes - everybody has one! You can form one in minutes. You have no excuse not to have one, and we'll discuss how to generate one later in this post.

The most helpful thing about having an opinion is that they are great for starting discussions. Even inaccurate opinions can do this well - sometimes better.

It's easy to face analysis paralysis when staring down the barrel and figuring out where to start. Enter the opinion, the conversation starter. The fire starter.

One of the quickest ways to form ideas when collaborating is to propose an idea. Just watch how quickly it will get picked apart and improved on. It's the same with opinions; defending an opinion will generate the most momentum. You want a 'defendable' opinion because it provokes thought and discussion and can lead to divergent thinking. As long as you genuinely believe in your opinion, it’s okay not to be 100% sold on it. You need some genuine belief to defend it for this provocation and creativity and see which way the conversation takes your opinion - it’s perfectly fine to defect to somebody else’s side after debating.

It seems borderline human nature for people to want to prove why your opinion is wrong, so give them one they have to work for. You'll end up with a much better decision that has been validated and discussed in detail as to why it's better, with actual data points - and you'll probably have garnered agreement from the team during this discussion.

Opinion essentials

When you want to have a strong opinion on something and back it in a discussion, I think it’s important to hit this checklist:

Enough believe that if you have to run with the idea, you’re happy to.
Empirical evidence over anecdotal evidence, if possible.
Deep enough understanding to carry thoughtful discussion

If you can’t tick these off, maybe you shouldn’t argue about it. You’ll probably just be wasting people’s time.

Traps to avoid

Don't just pick an opinion from the internet. Come to an informed opinion that you understand. If you don't understand the opinion, you can't defend it and thus can't trigger a good discussion - in which case, you're just wasting time and causing aggravation.

For less experienced engineers, it can be daunting to go into a meeting and strongly voice an opinion that goes against your seniors. Have courage. Don’t back down immediately because they’re more ‘senior’ than you. You’ll develop quite a few skills doing this. In a high-trust environment, this is a great way to grow - you’re not just learning from repetition but through understanding, which is fundamental. Listen to their arguments against your opinions carefully; hopefully, you can learn a lot! You may get proven wrong sometimes, but through having the discussions, you’ll get much better insight. Believe in yourself; you may teach them something new, too!

Know when to back down; don't be stubborn. If you've caused some good discussion, you don't need to keep returning to your opinion, especially if you've been proven wrong enough - start rolling with the leading opinion and continue the conversation. Once the discussion has progressed enough, bringing up the opinion restarts the clock and slows additional progress - so just let it go.

If this is an opinion you can't let go of and it determines a decision, consider agreeing to disagree and commit. Everybody on the team needs to be on board with the disagree and commit methodology before using it. Otherwise, it won’t play out well, and people will feel burnt. It can also be worth discussing if some method of veto is in place - For example, is the tech lead allowed to do so to ensure a terrible mistake isn’t made? Be careful with this, as it can easily lead to resentment, but consider putting something in place. Another thing might be that everybody is allowed one veto (ever), so you can ensure it’s only used when truly needed.

Disagree and commit: vote on which way to go when the team is butting heads and at an impasse. Sometimes an agreement just can’t be come to, so if you've said your piece, but the team has decided to go another way, then suck it up and roll with the team. You won't win every battle, but you can win the war as a team - you need some trust. Worst case? You're mistaken, and the team/everybody learns a lesson. Again, consider a veto method to ensure that truly is the worst case.

In low trust, low maturity, or toxic environments, it can be risky to follow the mantra due to the chance of backlash from others. Work with your team to build trust and maturity in discussions, or consider changing jobs/teams instead! Don’t risk it all to follow the practice; pick your timing. Sometimes, it’s not always worth the fight in these environments.

Generating an opinion on short notice

I just said not to pick an opinion from the internet, but maybe you should. Sort of.

If you've got a meeting coming up, and you know the agenda but aren't informed enough to get involved in the discussion, then you have two options:

Don't go.
Get an opinion quickly.

Both are good choices. If you don't go, you save some time and avoid wasting others. If you go with an opinion, you can actively contribute and make it worthwhile.

How to

So you've chosen option two - Get an opinion quickly. Ideally, you've got at least 15 minutes. Anything less than that timespan, and you're going to struggle. How do we do it?

Here are my tips:

Ask your colleagues (all of them) and put something together that is a mash-up of the opinions and that you can get behind and understand.
Search the vast resource known as the internet. Put together a few higher-trust sources or popular views and use those as your backbone.
Do some mind-mapping and quickly come to something - Ensure you have facts backing it up, though! Combine this with the tip above to get a strong base opinion.

As I’ve said a few times, make sure you can get behind this opinion - truly. You may end up executing it. It can be hard on short notice, but it’s crucial as otherwise you risk wasting everybody’s (including your) time. Devils advocates are a waste of brainpower.

In Summary

Always have an opinion for the meeting you're heading into, or don't go. It takes little time to form an argument, especially with practice.

You should be willing to carry your opinion high and drive discussion with it, even willing to execute it. However, you'll help everybody if you're ready to back down when defeated (or disagree and commit). Just don’t play devil’s advocate.

There’s a lot of nuance around this subject, so I hope you’ll form your strong opinion on it and maybe not agree with it all! I do think, however, that opinions are what drive forward you and your team, so I encourage everybody to follow the strong opinions loosely held adage and bring that into every meeting. Try it out, and get your team into it, too.

The cost of refactoring

Kieran Bond — Fri, 08 Dec 2023 14:17:10 +0000

Originally posted on my Substack here.

Most engineers have actively refactored code they're responsible for, and a lot think there's no cost to refactoring - but there is. Let's explore and understand, what is it?

Many teams like to live by the 'boy scouting' mantra - leave an area cleaner than you found. Thus, refactoring is widespread in software engineering teams - but at what cost?

There's always a cost. Everything done, every decision made, has tradeoffs. It's essential to remember this when making decisions - whether architecture decisions, when and what to refactor, work prioritisation, or even just delaying replying to a message from somebody until later.

Photo by Tim van der Kuip on Unsplash

When is a good time to refactor?
What are the costs of refactoring?
What is the cost of NOT refactoring?
How to refactor well

When is a good time to refactor?

Deciding when to refactor is subjective - and relies heavily on context. You have to make an informed decision where you have weighed the tradeoffs (costs) in refactoring something, and the cost is worth the price.

I can't make that decision for you, but I can inform you when there are good scenarios to consider refactoring.

The two most prominent ticks to go ahead with refactoring, in my mind, are:

The team is hurting from an earlier decision (such as architecture), and refactoring can remove or reduce the pain.
There is a business case (user requirement) to be working in the area that is giving your team pain.

Many engineers often make it apparent that they want to refactor an area. Generally, engineers are more vocal about this than anything and are persistent about it - but this doesn't mean it's an actual pain point that needs to be addressed, and that's an easy trap to fall into.

One way to ensure you/the team (truly) feel the pain of an area is to choose a priority area to refactor and provide reasons for it. If there's even data to back it up, then it's a pain point - not to say you need this data, but it helps validate the concern.

Before you go off and do any refactoring, ensure the team is on board - there's nothing worse than all the cost and no reward because the team disagrees the area needs refactoring.

Dedicate time to refactoring, or do it ad hoc?

I'm not a fan of etching out dedicated time for refactoring - putting that out there now.

The best time to refactor is ad hoc when you meet both criteria above. Having a user need gives you a reason to be in the area and feel the pain, which means you have work to do in the area - so you can refactor as part of business-as-usual (BAU) work without putting dedicated time aside, reducing one cost of refactoring and optimising time utilisation.

What are the costs of refactoring?

Many engineers find refactoring code rejuvenating work, so they vocalise this desire and overplay the importance of it. We must ensure we're pragmatic and consider the tradeoffs before committing.

Time is such a costly thing. When you are refactoring, what are you not doing? Revenue (features, infra, etc.) generating work. I know it's not black and white; it never is, but that's an easy way to look at it here. Time spent refactoring is time spent not building new features - but it's more than that. When you make a change (refactor), somebody has to use their time to review that change, and then somebody has to spend time ensuring no behaviours have changed or bad ones have been introduced. You should use behaviour-driven tests to help with the last one, but that only slightly reduces costs.

Knowledge is lost, and confusion is added when we refactor - this is prone to being glossed over because we usually focus on the benefits of the refactoring. In the long run, your change may bring more clarity to a codebase, but your refactor breaks the mental map others have in the short term. Time needs to be expended to familiarise/learn the new codebase - this mental map that has been eradicated is costly to replace and essential. Without a mental map, everything in this area takes longer. If a new, severe bug is found and needs to be fixed, then there are potentially fewer people who understand the codebase and can quickly execute the changes required to fix the bug.

Unknown unknowns. What was known is now unknown. Akin to the mental map lost in the above point, you suddenly don't know what you don't know about the code section. As systems grow, bugs are discovered (as is almost inevitable) and often classified as 'won't fix'. These are known knowns. Your refactor may have fixed or hidden them deeper - but we don't know that now. We need to determine if they exist or are gone and if new bugs will appear. Time is required to discover these things (if you're lucky to find them). Tests can help, particularly behaviour-driven tests, but we only catch some bugs.

What is the cost of NOT refactoring?

It's not all doom and gloom. Refactoring is almost inevitable in software; it's a constant, like the speed of light (and it seems to come up as often, too!), and there is usually a good reason it's wanted. It can be expensive not to refactor.

The three most significant costs of not refactoring:

Bus factor remains a risk.
Technical debt persists.
Morale steadily declines.

All three costs go hand in hand. Technical debt can lead to poor 'working conditions', declining morale and people moving on from the project/company, increasing the bus factor and creating a higher risk.

One trick here is that refactoring may not altogether remove the technical debt, but it can still alleviate morale (temporarily) - reducing chances of a high bus factor. Sometimes, taking a more minor cost and performing a smaller refactor is the better tactical choice. Smaller cost, smaller reward.

The excitement fallacy

One of the worst reasons to refactor is because 'it's exciting' or 'energising' work. You're at most going to increase morale minutely but incur all the costs - not a worthy tradeoff.

Most reasons behind refactoring are emotional, but this is the one to avoid. Pain and data are your friends, not excitement. Find other ways to sate the desire for exciting work - do personal development and try out a new technology (as a POC, not production work!).

How to refactor well

So, given we know when and why - how do you refactor excellently?

Small increments - just like good old-fashioned trunk-based development recommends (we all know how important trunk-based is). Small, concise, targeted changes are ideal because they're easy/quick for somebody to review. When you make small changes, it's easier to determine the scope of their impact.

Highly focused refactors are essential. If you try to bite off more than you can chew, you'll fix nothing and break everything. Remember, the most considerable cost of refactoring is time. If you are highly focused, you reduce the scope and thus time investment. You can choose the higher priority refactor targets, and once they have been completed, you can re-evaluate. Things move fast, and having small targets means you can pivot more successfully.

Use (behavioural) tests to minimise the risk/cost of introducing unknown unknowns - another cost. You should ensure you have tests in place before you touch the code. Use TDD to guide you here. Understand the behaviours expected in the part of the system you are modifying, put tests in place, and then execute the refactor.

Another way of mitigating this pain is to pair/mob on the refactor. You'll also reduce the cost of knowledge loss because more people will be close to the changes and understand the new code.

In summary

Refactoring isn't as perfect as it's made out to be, and we should be conscious of the cost before doing it. It’s important work, but it’s not always important enough.

So, next time you want to do some refactoring, pause and take your time to find reasoning and break down the work with a focus on small deliverables in the highest priority areas.

Unit Testing - Behaviours and critical units

Kieran Bond — Tue, 05 Dec 2023 20:16:57 +0000

Originally posted on my substack.

How I like to write automated tests
Critical Unit testing
Why I think what I think

How I like to write automated tests

Preface: I don't always work this way. It takes patience and discipline; my natural behaviour is to be gung-ho and write the implementation first. It's a constant battle and situation dependant. This methodology may not work for you, but it works for me.

My workflow looks something like this:

Picking a specific user behaviour to implement.
Thinking about how this behaviour is interacted with - What does the API look like?
Defining the API for the behaviour in code - Focusing on the inputs and outputs specifically.
Write this into a test scenario - using the AAA (Arrange, Act, Assert) pattern.
Write the bare minimum code required to compile/run the test - ensuring it fails.
Make the test pass - using hardcoded values, the bare minimum
Refactor/clean - constantly checking the test still passes, still only writing the bare minimum needed
Repeat for different scenarios related to the behaviour; if required - specific input scenarios may need to be tested.

This workflow is close to the classic red/green TDD testing pattern if not precisely that.

It works well for me, but I think it is essential to be incredibly fussy about focusing on a particular behaviour that is well-defined and singular. Concentrating on precise behaviours ensures that the test is focused and has minimum code. This makes it easier to reason about, as well as to maintain.

I like to ensure my tests avoid as much implementation detail as possible so that they are easily understood and maintained. Any time you leak implementation details into a test, you add tight coupling to the implementation and stray further from a focus on behaviours.

I also like to ensure that any API I design only provides access to functional behaviours. If the API is not for a specific behaviour, it is trying to do too much and needs to be split into multiple, more focused interfaces. It can sometimes be easier to think about user behaviours - what exactly are they trying to achieve when interacting with this API?

Advantages of this methodology

You only write the code you need.
A focused test is easier to debug when your implementation has failed to make it pass.
Your API is more straightforward to reason about - A focus on behaviours leads to simple APIs with simple tests. There should not be any unknown or unexpected behaviours when using it.
Your tests should catch regressions in behaviour.
Your tests are loosely coupled to implementations; thus, the implementation is easy to refactor.

Disadvantages of this methodology

It requires discipline and time to avoid leaking implementation details into your tests.
Using it can feel slow and unnatural (like TDD often does).
Using it is more challenging in 'established' codebases.
It can require more planning/design up front.
It can struggle when being used for fixing bugs and writing regression tests for those bugs.
It is not great for 'negative' tests - ensuring something did not happen (I generally disagree with this test style).

Critical Unit testing

Sometimes, we have things that are hard to test from an API perspective - such as a complicated algorithm.

Often, these are 'units' that are critical to the system. Think of a part of your system that needs to be performant; this is likely a good example.

In these cases, I lean more towards 'typical' unit testing - Plaster it in tests that make you confident it works correctly. With the caveat: only do this if the unit is genuinely critical!

Covering a 'unit' in many tests has disadvantages - it's harder to maintain and often harder to reason about. You will likely leak implementation details. This is why it's only reasonable if it's a very critical unit and complex to test behaviorally.

The advantage of covering the 'unit' in tests is that it's easier to test the complex scenarios and branches, easier to catch regressions, and gives you high confidence.

Confidence matters most here - after all, that's why we write tests. We want to feel confident about what we have implemented, changed, broken.

Why I think what I think

How did I get to this style of automated testing? What makes me think this is the right way for me?

TDD

To me, the essence of TDD is:

Design and define an API.
Write a test against the API.
Write the bare minimum code required to make your test pass.
Repeat.

The simplicity and focus of TDD leads to well-designed software.

I highly recommend reading 'Test Driven Development: By Example', written by Kent Beck. This book helped shape how I use TDD. It's a bit basic, but so is TDD.

Ian Cooper

Ian Cooper is great. He transformed how I looked at unit testing and TDD, which helped me enjoy the TDD process.

I highly recommend this talk by Ian:

In Summary

Give this ‘methodology’ a go. Experiment with it; try it in a pairing session.

The key point is to focus on the behaviours of the system and user. That’s the whole reason we’re building these things, right?

Living with an 'Emerging Architecture' philosophy

Kieran Bond — Thu, 26 Oct 2023 14:43:05 +0000

How we define 'Emerging Architecture'
How we got here and why we needed to
How we use this philosophy
Advantages of an 'Emerging Architecture'
Disadvantages of 'Emerging Architecture'

How we define 'Emerging Architecture'

'Emerging Architecture', as my team uses the term, is: 'Designing the system as minimally as possible, only changing the design when completely required for a user requirement. Keep the system simple, with no over-designing for the current needs.'

This philosophy means we only change the system's design when we have a reason, with zero over-engineering (in theory). The need for change can only be driven by user requirements so that we keep a clear focus on what's important.

How we got here and why we needed to

Our team has been using this design philosophy for around ~6 months at the time of writing, but it took some hardships to find it was suitable for us.

Around the beginning of 2023, we started on a project codenamed 'Bifrost'. The aim of Bifrost is simple: make the access and developer experience of using specific ancient but central systems considerably better. It takes way too long for teams to integrate with this system, and we want to enhance the quality and speed (of development) of the many teams/systems wanting to use it.

Whilst simple in theory, there are a lot of challenges - primarily with the upstream system. We created a 'Tracer Bullet' to try and crack some of the challenges early.

We struggled. We were trying to do a big design up front, and we had yet to agree on anything. We also disagreed on the tracer bullet's purpose; the combining two factors led to failure. There were too many unknowns we wanted to discover and too many ways we could attack the problem. Analysis paralysis then kicks in, too.

So, how do you move forward? We regrouped. The team had been working in fragments, and the solution was to mob together and design something that works for the known situation. We agreed that the best thing was to build something simple that worked for the current user needs and that moving forward, we should keep this approach - an 'Emerging Architecture'.

Once we agreed on this simple design, we also did some 'divergent thinking' exercises to develop a guiding design that our solution may look like at the end of our milestones. I'm not sure how helpful this part was, but it did allow everybody to collaborate, build interpersonal relations, and stretch our architecture design legs.

How we use this philosophy

We do all our planning as a team, which is JIT (Just-In-Time) with a particular use case focus. When planning, we consider if there needs to be system design changes - the answer is almost always no. If yes, then we work together to ensure we only make the most minimal change in design required - absolutely zero over-engineering allowed. By doing this designing together, we will collectively ensure that no over-designing happens.

Our use case suits 'Emerging Architecture' exceptionally well, as we're building an API gateway with success milestones containing expected user count/load.

Our milestones allow us to design iteratively, as we only need to develop with the following user milestone in mind. We can run performance/load tests against our API to see how we fare against our milestone. If our tests indicate that we aren't ready for the next milestone, then we plan the design (and work) that will get us there. Once implemented, we can rerun the tests and prove our design is good for the milestone we're working towards.

Here’s an example: Our first milestone in the team is to enable 100 users at the same time (concurrently). We know that the milestone after that is ~3,000 users concurrently.

Our design is built with 100 users in mind. Our session management is all done in memory, in one running service instance. We purposefully chose only to have one instance of our service running because we think it will handle this milestone. We suspect it won’t work for the next milestone, and for the milestones after, we’re almost certain it won’t scale.

We’ve run load/performance testing and fixed bugs that arose in the session management area, bugs that would go away if we changed our design to handle the next few milestones, as we are confident this is the area that doesn’t scale. However, we know it’s easier to hit this milestone with this design. It takes less time to build with this design and gives us more time to focus on everything else. For this milestone, we are also building the rest of the system. We know there’s no point focusing on scale if we don’t have a working system first, so we make our lives easier and choose our battles. Scale is a complex problem down the road; our problem now is a usable system.

Advantages of an 'Emerging Architecture'

Caveat: We are still in the midst of delivering our project using this philosophy, so we have yet to discover everything great and not-so-great about it.

So far, the 'Emerging Architecture' philosophy we've been using is working well for us. Here are some advantages that I think have come from following it:

It creates space that allows you and the team to dedicate time to the most important things that your users need (features).
Lack of over-engineering leads to less refactoring when you inevitably go too far in the wrong direction.
It's easy to pivot and change direction as you become more practised.
Your design is more flexible, making tackling unknowns easier.
Iterative design allows for iterative feedback.
Iterative feedback (and design) means you can solve the challenging problems individually when they appear rather than all simultaneously.
It's easier to catch the problems in your design if only small parts change at any time. You can be more confident that a solution change will work with fewer moving parts.

Disadvantages of an 'Emerging Architecture'

Greater chance of system design rework - The changes that get you to your next milestone may not be the correct change for the milestone after. Sometimes, certain choices don’t scale as well as others but are better at lower user counts.
Greater chance of feature rework - If you design for more significant user numbers early, it can affect how a feature is implemented. You may choose to do something in-memory because of the scale requirement but later need to move to a separate service for higher throughput/user counts - leading to other changes, such as how you get data into a system. You may have to rework multiple features between milestones.
You must be mindful of costly/irreversible design decisions, or you can find yourself in a challenging situation.

In Summary

'Emerging Architecture' is a philosophy that we follow. It has advantages and disadvantages, but it's working for us, and the trade-offs seem worth it. We're keen to see where it takes us and hope we make the right decisions - and when we don't, at least we'll learn some things on the way.

Let me know what you think in the comments below. Have you tried this philosophy? How did it work for you?

Introducing TinYard - Your next .NET IoC framework

Kieran Bond — Fri, 11 Dec 2020 20:48:22 +0000

TinYard

TinYard Repository. Source Code and DLL's available here!

What is TinYard?
Why use it?
How do I use it?
Inspiration
Contributing

What is TinYard?

TinYard is an IoC framework written in C# for .NET that brings more features than your bog-standard IoC-container.

TinYard aims to improve your code by providing an easy-to-use IoC container and more through its powerful but simple extensibility. The framework provides a few tools to help you create awesome things, such as:

Dependency Injection (DI)
Event System
MVC Suite
Easy Plug and Play Extensions

You can use it in your projects by adding the .dll as a dependency, or by adding the source code into your project. It's available both as binaries from the Github repository releases, or you can find it on NuGet. One of the great things about TinYard is that almost any .NET project can use it as it's compiled for .NET Standard 2.0.

Why use it?

Use TinYard because it's a straightforward and simple IoC-container that lets you do more, when you want to. It's modular and gives you only what you want from it, when you want it.

Due to its modularity, it's a tool that allows you to add whatever you want to it to speed up and enhance your workflow.

Aside from that.. to help build something awesome with me. Whether that's helping by informing me of bugs, requesting features, adding to it or just simply using it and advocating for it!

How do I use it?

Have a look at the ReadMe! I've spent some time working on the ReadMe in the hope that it is useful.
There's some examples written up and linked to in the repository too!

Inspiration

TinYard was an inspired product. I was encouraged by wanting to further understand the framework that I use at work and in personal projects at a deeper level than simply how to use it. I wanted to understand the design reasons behind it, as well as how to do the cool bits like DI.

The framework that inspired me is Robotlegs-Sharp. A framework built on top of an IoC container known as SwiftSuspenders-Sharp that aims to provide an easy to use MVC framework. Whilst I didn't use the original Robotlegs, I do use the version that was ported to C# by some past colleagues and mentors regularly linked above.

Contributing

If you would like to contribute, whether that's by letting me know of features you want or bugs you encounter or even going as far as adding directly to the codebase then please do! There's a contribution guideline on the repository, have a look at that or just open up an issue if there's features wanted or bugs encountered.

Any contribution at all would be highly appreciated! And, on the off chance that the project receives a sponsor or donation, you might find a chunk comes your way.

Final Thoughts

Let me know what you think about TinYard.
Code look good? Code look bad? Project seem pointless? I'll listen to it!

Forem: Kieran Bond

This is a well-written, technical post. It also features some excellent external links, making it highly recommended reading for everyone developing!

Are Sync Engines The Future of Web Applications?

I'm investigating the pain points people are having with conducting user interviews/research! Help me out and fill out my survey? https://tally.so/r/wop2BX Thanks in advance!

Why you want cross-cutting product teams

Contents

What it means to have a cross-cutting product team

Who's in this supposed team, then?

Why would I want a cross-cutting product team?

Feedback loops are shorter.

Deep knowledge base

Cross-cutting expertise

Fewer distractions

Better business understanding

Why don’t companies adopt a cross-cutting product team structure?

Inertia

Cost of change

Cultural barriers

Good enough is good enough.

The unsolved problems

Cost

Multiple teams for one product

Team growth is more challenging.

More senior teams are needed.

Summary

Measure twice, cut once

Contents

When should we measure?

The trading system

How do we measure?

Assumptions three and four

Assumption two

Assumption one

Pitfalls to avoid

Counter Arguments

Summary

The conclusion to the example

Maximising one-to-one effectiveness

Contents

Topics of focus

Goals - career and personal

Things going on with and around you

Feedback

Opportunities

Frustrations

Recent achievements

The business - context, goals, things happening

The team

One-to-one environments

Frequency of one-to-ones

In Summary

Succeeding with the 3 Musketeers pattern

Contents

Why use the 3 Musketeers?

Consistency

Ease of use

Speed

What is the 3 Musketeers pattern?

When can you use the 3 Musketeers?

When not to use it

An example of using the 3 Musketeers pattern

In Summary

Integration tests are king

Contents

Test suite composure

What is integration testing?

Why integration tests are king

The downfalls of integration testing

What makes an excellent integration test

How I like to create them

An example

In Summary

Strong opinions, loosely held

Contents

Why it's essential to have an opinion

Opinion essentials

Traps to avoid

Generating an opinion on short notice

How to

In Summary