Forem: Jeroen De Dauw

Clean Comparisons

Jeroen De Dauw — Mon, 18 Sep 2023 08:42:00 +0000

In programming, readability is just as crucial as functionality. One area that often gets overlooked is the way we write comparisons. Using ambiguous comparisons can make your code harder to understand and lead to unexpected bugs. In this post, we'll look at how to make your comparisons in PHP and JavaScript both readable and reliable.

Readable PHP Comparisons

Beyond being a dynamic language, PHP has some weird and unexpected behavior when casting values. For instance, if you have a string with a zero, it will evaluate to false if you do !$string.

So, in PHP, you are incentivized to do explicit comparisons not just for readability but also for correctness.

Check	Ambiguous form	Clear replacement
Is array empty?	!$list	$list === []
Is array empty?	empty($list)	$list === []
Is string empty?	!$string	$string === ''
Is int zero?	!$int	$int === 0
Is null?	!$typeOrNull	$typeOrNull === null
Scalar equals?	$a == $b	$a === $b
Array key exists?	isset($list[$a])	array_key_exists($a, $list)
Is null or false?	!$mixed	$mixed === null \|\| $mixed === false

Readable JavaScript Comparisons

Check	Ambiguous form	Clear replacement
Is array empty?	!arr.length	arr.length === 0
Is string empty?	!str	str === ''
Is number zero?	!num	num === 0
Is null?	!val	val === null
Is equal?	a == b	a === b

More Comparisons

What did I miss? Do you disagree with any of the above? Let me know in the comments!

Advice for Junior Developers

Jeroen De Dauw — Tue, 11 Oct 2022 12:11:33 +0000

Over the 15+ years of my development career, I have learned several things that significantly increase my effectiveness. In this post, I share those learnings with you.

Structure:

Foundational Advice -- Important context and motivation for what follows
Technical Advice -- The main course
Recommended Reading -- Links to high-quality books and blogs that are great for getting started

This blog post mentions and links valuable concepts that you can explore further as you see fit.

Foundational Advice for Juniors

1. Code is not the Point

As developers, we like writing code. Most of us want to be given a nice unambiguous task. A fun technical puzzle to solve without paying attention to the rest of the world.

Put reasonable effort into making sure that you are solving the right problem. To quote Peter Drucker: There is nothing so useless as doing efficiently that which should not be done at all. Gather feedback early and often, typically by continuous delivery to real users. Be Agile.

Software development is expensive, with the vast majority of the effort of real-world projects typically going into maintenance. Combine this with the goal being user/business outcomes, the best code is often no code. To quote Bill Gates: "Measuring programming progress by lines of code is like measuring aircraft building progress by weight."

2. Software Design Matters

During the first 5 years of my development career, I thought that software design is for software architects or other people with special roles. I was focused on "getting things done", and saw software design and practices such as writing tests, as a distraction at best. My code worked, and I was getting a lot of things done. Or so I thought.

Then I read Clean Code, by Robert C. Martin. This book motivates caring about software design and contains examples and many technical heuristics. The central takeaway of the book is: "The only way to go fast is to go well". In other words, if you make a mess, it will slow you down. See also: TradableQualityHypothesis, DesignStaminaHypothesis

Learning how to write well-designed clean code of course takes time and effort. And when you start, you will be slower and make mistakes. Simple is not Easy.

3. Use the BEST Practices

Writing tests tends to be beneficial. There are exceptions, but most of the time, it makes a lot of sense to write automated tests. Writing tests is an example of a best practice.

If you are new to writing tests, just follow the best practice and write tests for everything. When starting, blindly following the best practice will be better than following your own underdeveloped judgment. Over time you will learn how to write tests effectively, and be able to tell the difference between you have messed up, and situations where writing a test is not worth it. You will also start to understand the value tests bring on a more visceral level, by having experienced the decrease in debugging sessions and the worry-free refactoring enabled by your tests. After developing your judgment, you will be able to transcend the best practice.

This advice applies to best practices in any area that you are a junior in. Automated tests are just an example.

One big gotcha is that it is not easy to tell the difference between a sensible best practice and something nonsensical or even counterproductive. This is made more complicated by most existing code being a mess, and by most developers, including "experienced" and "senior" ones, not knowing software design basics. This makes having a good mentor extremely valuable. Barring that, one piece of advice based on my own experiences is to be wary of best practices specific to the community of your language or framework. Look for evergreen advice that has been around for decades.

Technical Advice for Juniors

Our focus will be on technical topics. Many other areas are important, such as health, happiness, career, and soft skills. Knowing how to avoid a technical pitfall won't help you if you are sleep deprived and working on the wrong problem for a toxic boss that underpays you.

4. Write Tests

Write automated tests. Perhaps write tests before the code, such as via Test Driven Development (TDD). This makes it easy to verify your code is correct in a repeatable manner, thus saving you from much manual retresting and from debugging sessions.

You think test-first is difficult? Try debug-after.

Perhaps even more importantly, tests give you the safety net to refactor your code. And continuous refactoring is needed to keep your code clean. Without a reliable test suite, it is all the more likely that your code will rot.

Writing tests is difficult if the design of your code is poor, such as when using inheritance for code reuse, or when using static functions. If on the other hand, you have SOLID classes, with no global dependencies, then writing nice tests is not so difficult.

Test design matters because poorly written tests will slow you down. Avoid binding your tests to implementation details of the code under test or to the structure of the system. Avoid overusing Mocks and write better Test Doubles.

5. Do Not Use Inheritance For Code Reuse

This is one of those best practices that bring to mind the "Use the Best Practices" section. My advice: do not use inheritance for code reuse at all when you are starting out. It is rarely the right call and can do a lot of harm. Favor composition over inheritance.

6. Write Object-Oriented code

Write SOLID code that is not STUPID. There is so much value in understanding these principles and anti-patterns.

Actually create objects. Classes with only static methods are not OO. Try to avoid using static code altogether.

7. Write Functional Code

(Functional programming is not to be confused with imperative structural programming.)

This point is not about fully switching to a functional language. You can benefit from using a functional style in your OO language. Minimize state, especially mutable state, and do one thing in your functions. See also: functional core, imperative shell.

8. Use Informed Duplication

Copy-pasting big chunks of code to multiple places is almost always unwise. Any self-respecting developer soon learns this and starts to follow some form of Don't Repeat Yourself (DRY). Unfortunately, the well-intended pursuit of DRY often leads to overengineering and accidental complexity. This is where the counterpart of DRY comes in: Write Everything Twice (WET). The idea behind WET is to only deduplicate on the third occurrence of duplication.

For a more in-depth look at the costs and benefits of deduplication, see The Fallacy of DRY.

9. Types, Names and Comments

Try to write self-documenting code and avoid comments.

Every time you write a comment, you should grimace and feel the failure of your ability of expression. -- Robert C. Martin

Comments are dangerous because they can lie. The code can change without the comment being updated. New code can be added right under the comment. The comment might have been wrong or inaccurate in the first place. When this happens, the comment not only becomes useless, it becomes misleading.

To write self-documenting code:

Do one thing in your functions
- By doing a single thing in a function, you can give it a clear name
- Feel the need to explain what different sections of your function do by adding comments? Instead, extract each section into its own well-named function.
- "Extract till you drop": if you can meaningfully extract a function, then you probably should. Don't be afraid of small functions.
- Command Query Separation
- Similar to the Single Responsibility Principle for classes (The S in SOLID)
Minimize state
Use types. Combined with a test suite that executes the code, you can rely on the types telling the truth.
Avoid mixed types. Avoid parameters or return values that can be an integer, a boolean, or a string. This naturally happens if you write focused functions that only do one thing.
Write tests. A well-written and comprehensive test suite shows you how the production code can be used, and how it behaves.

Clean Code by Robert C. Martin has some good rules of thumb about naming and comments.

Bonus links

Tell Don't Ask -- Encapsulation best practice
Law of Demeter -- Coupling best practice
Domain Driven Design -- A sizeable toolbox. More advanced, good to first learn the basics.
Object Calisthenics -- Rules that restrict what you can do in programming. Nice for learning how to do things differently
Pair Programming -- A great way to learn from each other
Code katas -- Simple programming, great for practicing a specific technique or skill such as Test Driven Development

About Jeroen

Jeroen De Dauw is CEO of Professional Wiki, which provides wiki hosting services. He occasionally writes on his software design blog. You can follow Jeroen on Twitter.

Applications as Frameworks

Jeroen De Dauw — Mon, 04 Mar 2019 07:04:25 +0000

What are the differences between building on top of a framework and building on top of an application? How does using an application as a framework cause problems, and how can these problems be avoided? That is what this post is all about.

Decoupled web application

In your typical web application, the code handles a request and returns a response. Let's assume we are using a web framework to handle common tasks such as routing. Let's also assume that we think framework binding has a high cost and decouple our application from it. The flow of control would look like this:

Execution starts with the framework. For PHP frameworks this will be in a file like public/index.php. The framework then bootstraps itself and does a bunch of stuff. It's safe to assume this stuff will include routing, and often it also includes things like dependency construction and error handling.

After the framework did the tasks you want it to do, it hands control over to your application. Your application does a bunch of application and domain logic and interacts with persistence. It likely uses a number of libraries, especially for infrastructure tasks like logging and database access. Even so, control stays with the application. (The key difference between frameworks and libraries is that you control/call libraries while frameworks control/call you.) Your application might also be calling the framework and use it as a library. Again, control stays with the application.

Finally when the application is done, it hands some kinda of result back to the framework. The framework then does another bunch of stuff, like template rendering and translations. In case of a web framework it then spits out a HTTP response and execution ends.

An application like this keeps you in control of what happens, making it easier to change things. This style also makes it easy to decouple from the framework. There are only two points where you need to decouple.

My post Implementing The Clean Architecture outlines one architectural approach that leads to this kind of application.

Frameworks vs Applications

Let's compare how frameworks and applications differ when they are used as a foundation for an/another application.

Frameworks don't do stuff on their own. There is no application or domain logic. There is no set of existing web pages or API endpoints with their own structure and behavior. This is all defined by your application when using a framework. When building on top of an application that acts as a framework, you'll need to deal with existing structure and behavior. You'll need to insert your own stuff, change existing behavior in certain situations and prevent default behavior altogether in others.

I know that there are "frameworks" that do provide their own stuff out of the box. (Example: web shop framework.) While they might not be a full application on their own, for the purpose of this blog post they are the same as an application that gets used as a framework.

Plugins and Extensions

There is nothing inherently bad about building things on top of an application. Plugins and extensions are a very useful pattern. A plugin that interacts with a single plugin point can decouple itself when appropriate and is in control over itself. And for extensions that use many extension points of the application yet are shallow/small, framework decoupling might not make sense.

This post is about using applications as framework foundation for sizable sets of code which are applications in their own right.

Applications as Frameworks

Let's imagine we have an application that is used on some site for some use case. We'll call this application FrameworkApp, since we'll use it as framework for another application that powers another site.

When building our application on top of FrameworkApp, we'll need to register new behavior and modify existing behavior. To make this possible, FrameworkApp needs to provide the appropriate extension points. Often these take the form of abstract classes or even systems, though the exact nature of the extension points is not important for our purposes.

This leads to a very different flow of control. Rather than calling us once, the FrameworkApp calls each extension point our application handles.

The diagram is showing just 6 extension points, though there can be 100s.

When visualized like this, it becomes easy to see how decoupling from the framework becomes next to impossible. Even if you manage to avoid coupling to framework code in your application, its whole structure is still defined by the framework. This means you are very limited in what you can do in your application and need to understand the framework to effectively develop the application. Framework coupling causes more issues than that, though a comprehensive overview of those is out of scope for this post.

An OOP Solution

Favor composition over inheritance -- OOP principle

Using an application as a framework is very similar to using inheritance for code reuse.

Just like with the application that is build on top of the app that acts as framework, the subclass might not be in control and be invoked many times from the base class. This is especially the case when using the Template Method Pattern and when having a deep inheritance hierarchy. The flow of control can bounce all over the place and decoupling the subclass from the classes up the hierarchy becomes all but impossible.

You can avoid this classical inheritance mess by using composition. Which suggests one way to move away from using an application as a framework or avoid doing so altogether: stop treating the framework as a base class. If there is code to share, use composition. This way you stay in control, can decouple easier and avoid The Fallacy of DRY.

Just like with class hierarchies you can always slap on an extra level.

Thanks to Raz Shuty for proofreading and making some suggestions.

Applications as Frameworks was originally posted on my blog, where you can find many similar posts.

Base Libraries Should be Stable

Jeroen De Dauw — Thu, 30 Aug 2018 16:20:35 +0000

In this post I go over the principles that govern package (library) design and one specific issue I have come across several times.

Robert C Martin has proposed six Package Principles. Personally I find the (linked) description on Wikipedia rather confusing, especially if you do not already understand the principles. Here is my take on library design using short and simple points:

Libraries should

cover a single responsibility
provide a clean and well segregated interface to their users
hide/encapsulate implementation details
not have cyclic dependencies
not depend on less stable libraries

The rest of this post focuses on the importance of the last point.

Stable Base Libraries

I am assuming usage of a package manager such as Composer (PHP) or NPM. This means that each library defines its dependencies (if any) and the version ranges of those that it is compatible with. Libraries have releases and these follow semantic versioning. It also means that libraries are ultimately consumed by applications that define their dependencies in the same way.

Consider the scenario where you have various applications that all consume a base library.

If this library contains a lot of concrete code, it will not be very stable and it will now and then be needed or desired to make a major release. A major release is one that can contain breaking changes and therefore does not automatically get used by consumers of the library. These consumers instead need to manually update the version range of the library they are compatible with and possibly adjust their code to work with the breaking changes.

In this scenario that is not a problem. Suppose the library is at version 1.2 and that all applications use version 1.x. If breaking changes are made in the library, these will need to be released as version 2.0. Due to the versioning system the consuming applications can upgrade at their leisure. There is of course some cost to making a major release. The consumers still do need to spend some time on the upgrade, especially if they are affected by the breaking changes. Still, this is typically easy to deal with and is a normal part of the development process.

Things change drastically when we have an unstable library that is used by other libraries, even if those libraries themselves are unstable.

In such a scenario making breaking changes to the base library are very costly. Let's assume we make a breaking change to the base library and that we want to use it in our application. What do we actually need to do to get there?

Make a release of the base library (Library A)
Update library B to specify it is compatible with the new version of A and make a new release of B
Update library C to specify it is compatible with the new version of A and make a new release of C
Update library D to specify it is compatible with the new version of A and make a new release of D
Update our application to specify it is compatible with the new version of A

In other words, we need to make a new release of EVERY library that uses our base library and that is also used by our application. This can be very painful, and is a big waste of time if these intermediate libraries where not actually affected by the breaking change to begin with. This means that it is costly, and generally a bad idea, to have libraries depend on another library that is not very stable.

Stability can be achieved in several ways. If the package is very abstract, and we assume good design, it will be stable. Think of a package providing a logging interface, such as psr/log. It can also be approached by a combination of following the package principles and taking care to avoid breaking changes.

Conclusion: Keep the package principles in mind, and avoid depending on unstable libraries in your own libraries where possible.

Iterators and Generators in PHP

Jeroen De Dauw — Wed, 18 Oct 2017 07:27:09 +0000

In this post I demonstrate an effective way to create iterators and generators in PHP and provide an example of a scenario in which using them makes sense.

Generators have been around since PHP 5.5, and iterators have been around since the Planck epoch. Even so, a lot of PHP developers do not know how to use them well and cannot recognize situations in which they are helpful. In this blog post I share insights I have gained over the years, that when sharing, always got an interested response from colleague developers. The post goes beyond the basics, provides a real world example, and includes a few tips and tricks. To not leave out those unfamiliar with Iterators the post starts with the "What are Iterators" section, which you can safely skip if you can already answer that question.

What are Iterators

PHP has an Iterator interface that you can implement to represent a collection. You can loop over an instance of an Iterator just like you can loop over an array:

function doStuff(Iterator $things) {
    foreach ($things as $thing) { /* ... */ }
}

Why would you bother implementing an Iterator subclass rather than just using an array? Let's look at an example.

Imagine you have a directory with a bunch of text files. One of the files contains an ASCII NyanCat (~=[,,_,,]:3). It is the task of our code to find which file the NyanCat is hiding in.

We can get all the files by doing a glob( $path . '*.txt' ) and we can get the contents for a file with a file_get_contents. We could just have a foreach going over the glob result that does the file_get_contents. Luckily we realize this would violate separation of concerns and make the "does this file contain NyanCat" logic hard to test since it will be bound to the filesystem access code. Hence we create a function that gets the contents of the files, and ones with our logic in it:

function getContentsOfTextFiles(): array {
    // glob and file_get_contents
}

function findTextWithNyanCat(array $texts) {
    foreach ($texts as $text) { if ( /* ... */ ) { /* ... */ } }
}

function findNyanCat() {
    findTextWithNyanCat(getContentsOfTextFiles());
}

While this approach is decoupled, a big drawback is that now we need to fetch the contents of all files and keep all of that in memory before we even start executing any of our logic. If NyanCat is hiding in the first file, we'll have fetched the contents of all others for nothing. We can avoid this by using an Iterator, as they can fetch their values on demand: they are lazy.

class TextFileIterator implements Iterator {
    /* ... */
    public function current() {
        // return file_get_contents
    }
    /* ... */
}

function findTextWithNyanCat(Iterator $texts) {
    foreach ($texts as $text) { if ( /* ... */ ) { /* ... */ } }
}

function findNyanCat() {
    findTextWithNyanCat(new TextFileIterator());
}

Our TextFileIterator gives us a nice place to put all the filesystem code, while to the outside just looking like a collection of texts. The function housing our logic, findTextWithNyanCat, does not know that the text comes from the filesystem. This means that if you decide to get texts from the database, you could just create a new DatabaseTextBlobIterator and pass it to the logic function without making any changes to the latter. Similarly, when testing the logic function, you can give it an ArrayIterator.

function testFindTextWithNyanCat() {
    /* ... */
    findTextWithNyanCat(new ArrayIterator(['test text', '~=[,,_,,]:3']));
    /* ... */
}

I wrote more about basic Iterator functionality in Lazy iterators in PHP and Python and Some fun with iterators. I also blogged about a library that provides some (Wikidata specific) iterators and a CLI tool build around an Iterator. For more on how generators work, see the off-site post Generators in PHP.

PHP's collection type hierarchy

Let's start by looking at PHP's type hierarchy for collections as of PHP 7.1. These are the core types that I think are most important:

At the very top we have iterable, the supertype of both array and Traversable. If you are not familiar with this type or are using a version of PHP older than 7.1, don't worry, we don't need it for the rest of this blog post.

Iterator is the subtype of Traversable, and the same goes for IteratorAggregate. The standard library iterator_ functions such as iterator_to_array all take a Traversable. This is important since it means you can give them an IteratorAggregate, even though it is not an Iterator. Later on in this post we'll get back to what exactly an IteratorAggregate is and why it is useful.

Finally we have Generator, which is a subtype of Iterator. That means all functions that accept an Iterator can be given a Generator, and, by extension, that you can use generators in combination with the Iterator classes in the Standard PHP Library such as LimitIterator and CachingIterator.

IteratorAggregate + Generator = <3

Generators are a nice and easy way to create iterators. Often you'll only loop over them once, and not have any problem. However beware that generators create iterators that are not rewindable, which means that if you loop over them more than once, you'll get an exception.

Imagine the scenario where you pass in a generator to a service that accepts an instance of Traversable:

$aGenerator = function() { /* ... yield ... */ };
$aService->doStuff($aGenerator());</pre>

<pre class="lang:php decode:true">public function doStuff(Traversable $things) {
    foreach ($things as $thing) { /* ... */ }
}

The service class in which doStuff resides does not know it is getting a Generator, it just knows it is getting a Traversable. When working on this class, it is entirely reasonable to iterate though $things a second time.

public function doStuff(Traversable $things) {
    foreach ($things as $thing) { /* ... */ }
    foreach ($things as $thing) { /* ... */ } // Boom if Generator!
}

This blows up if the provided $things is a Generator, because generators are non-rewindable. Note that it does not matter how you iterate through the value. Calling iterator_to_array with $things has the exact same result as using it in a foreach loop. Most, if not all, generators I have written, do not use resources or state that inherently prevents them from being rewindable. So the double-iteration issue can be unexpected and seemingly silly.

There is a simple and easy way to get around it though. This is where IteratorAggregate comes in. Classes implementing IteratorAggregate must implement the getIterator() method, which returns a Traversable. Creating one of these is extremely trivial:

class AwesomeWords implements \IteratorAggregate {
    public function getIterator() {
        yield 'So';
        yield 'Much';
        yield 'Such';
    }
}

If you call getIterator, you'll get a Generator instance, just like you'd expect. However, normally you never call this method. Instead you use the IteratorAggregate just as if it was an Iterator, by passing it to code that expects a Traversable. (This is also why usually you want to accept Traversable and not just Iterator.) We can now call our service that loops over the $things twice without any problem:

$aService->doStuff(new AwesomeWords()); // no boom!

By using IteratorAggregate we did not just solve the non-rewindable problem, we also found a good way to share our code. Sometimes it makes sense to use the code of a Generator in multiple classes, and sometimes it makes sense to have dedicated tests for the Generator. In both cases having a dedicated class and file to put it in is very helpful, and a lot nicer than exposing the generator via some public static function.

For cases where it does not make sense to share a Generator and you want to keep it entirely private, you might need to deal with the non-rewindable problem. For those cases you can use my Rewindable Generator library, which allows making your generators rewindable by wrapping their creation function:

$aGenerator = function() { /* ... yield ... */ };
$aService->doStuff(new RewindableGenerator($aGenerator));

A real-world example

A few months ago I refactored some code part of the Wikimedia Deutschland fundraising codebase. This code gets the filesystem paths of email templates by looking in a set of specified directories.

private function getMailTemplatesOnDisk( array $mailTemplatePaths ): array {
    $mailTemplatesOnDisk = [];

    foreach ( $mailTemplatePaths as $path ) {
        $mailFilesInFolder = glob( $path . '/Mail_*' );
        array_walk( $mailFilesInFolder, function( & $filename ) {
            $filename = basename( $filename ); // this would cause problems w/ mail templates in sub-folders
        } );
        $mailTemplatesOnDisk = array_merge( $mailTemplatesOnDisk, $mailFilesInFolder );
    }

    return $mailTemplatesOnDisk;
}

This code made the class bound to the filesystem, which made it hard to test. In fact, this code was not tested. Furthermore, this code irked me, since I like code to be on the functional side. The array_walk mutates its by-reference variable and the assignment at the end of the loop mutates the return variable.

This was refactored using the awesome IteratorAggregate + Generator combo:

class MailTemplateFilenameTraversable implements \IteratorAggregate {
    public function __construct( array $mailTemplatePaths ) {
        $this->mailTemplatePaths = $mailTemplatePaths;
    }

    public function getIterator() {
        foreach ( $this->mailTemplatePaths as $path ) {
            foreach ( glob( $path . '/Mail_*' ) as $fileName ) {
                yield basename( $fileName );
            }
        }
    }
}

Much easier to read/understand code, no state mutation whatsoever, good separation of concerns, easier testing and reusability of this collection building code elsewhere.

See also: Use cases for PHP generators (off-site post).

Tips and Tricks

Generators can yield key value pairs:

yield "Iterators" => "are useful";
yield "Generators" => "are awesome";
// [ "Iterators" => "are useful", "Generators" => "are awesome" ]

You can use yield in PHPUnit data providers. You can yield from an iterable.

yield from [1, 2, 3];
yield from new ArrayIterator([4, 5]);
// 1, 2, 3, 4, 5

// Flattens iterable[] into Generator
foreach ($collections as $collection) {
    yield from $collection;
}

Thanks for Leszek Manicki and Jan Dittrich for reviewing this blog post.

Originally posted on my blog as Introduction to Iterators and Generators in PHP.

Yield in PHPUnit data providers

Jeroen De Dauw — Wed, 11 Oct 2017 03:09:10 +0000

Initially I started creating a general post about PHP Generators, a feature introduced in PHP 5.5. However since I keep failing to come up with good examples for some cool ways to use Generators, I decided to do this mini post focusing on one such cool usage.

PHPUnit data providers

A commonly used PHPUnit feature is data providers. In a data provider you specify a list of argument lists, and the test methods that use the data provider get called once for each argument list.

Often data providers are created with an array variable in which the argument lists get stuffed. Example (including poor naming):

/**
 * @dataProvider provideUserInfo
 */
function testSomeStuff( string $userName, int $userAge ) {}

function provideUserInfo() {
    $return = [];

    $return[] = [ 'Such Name', 42 ];
    $return[] = [ 'Very Name', 23 ];
    $return['Named parameter set'] = [ 'So Name', 1337 ];

    return $return;
}

The not so nice thing here is that you have a variable (explicit state) and you modify it (mutable state). A more functional approach is to just return an array that holds the argument lists directly. However if your argument list creation is more complex than in this example, requiring state, this might not work. And when such state is required, you end up with more complexity and a higher chance that the $return variable will bite you.

Using yield

What you might not have realized is that data providers do not need to return an array. They need to return an iterable, so they can also return an Iterator, and by extension, a Generator. This means you can write the above data provider as follows:

function provideUserInfo() {
    yield [ 'Such Name', 42 ];
    yield [ 'Very Name', 23 ];
    yield 'Named parameter set' => [ 'So Name', 1337 ];
}

No explicit state to be seen!

Stay tuned for more generator goodness if I can overcome my own laziness (hint hint :))

Originally posted on my blog as Yield in PHPUnit data providers.

The Fallacy of DRY

Jeroen De Dauw — Sat, 16 Sep 2017 17:36:14 +0000

Originally posted on my blog as The Fallacy of DRY.

DRY, standing for Don’t Repeat Yourself, is a well-known design principle in the software development world.

It is not uncommon for removal of duplication to take center stage via mantras such as “Repetition is the root of all evil”. Yet while duplication is often bad, the well intended pursuit of DRY often leads people astray. To see why, let’s take a step back and look at what we want to achieve by removing duplication.

The Goal of Software

First and foremost, software exists to fulfill a purpose. Your client, which can be your employer, is paying money because they want the software to provide value. As a developer it is your job to provide this value as effectively as possible. This includes tasks beyond writing code to do whatever your client specifies, and might best be done by not writing any code. The creation of code is expensive. Maintenance of code and extension of legacy code is even more so.

Since creation and maintenance of software is expensive, the quality of a developers work (when just looking at the code) can be measured in how quickly functionality is delivered in a satisfactory manner, and how easy to maintain and extend the system is afterwards. Many design discussions arise about trade-offs between those two measures. The DRY principle mainly situates itself in the latter category: reducing maintenance costs. Unfortunately applying DRY blindly often leads to increased maintenance costs.

The Good Side of DRY

So how does DRY help us reduce maintenance costs? If code is duplicated, and it needs to be changed, you will need to find all places where it is duplicated and apply the change. This is (obviously) more difficult than modifying one place, and more error prone. You can forget about one place where the change needs to be applied, you can accidentally apply it differently in one location, or you can modify code that happens to the same at present but should nevertheless not be changed due to conceptual differences (more on this later). This is also known as Shotgun Surgery. Duplicated code tends to also obscure the structure and intent of your code, making it harder to understand and modify. And finally, it conveys a sense of carelessness and lack of responsibility, which begets more carelessness.

Everyone that has been in the industry for a little while has come across horrid procedural code, or perhaps pretend-OO code, where copy-paste was apparently the favorite hammer of its creators. Such programmers indeed should heed DRY, cause what they are producing suffers from the issues we just went over. So where is The Fallacy of DRY?

The Fallacy of DRY

Since removal of duplication is a means towards more maintainable code, we should only remove duplication if that removal makes the code more maintainable.

If you are reading this, presumably you are not a copy-and-paste programmer. Almost no one I ever worked with is. Once you know how to create well designed OO applications (ie by knowing the SOLID principles), are writing tests, etc, the code you create will be very different from the work of a copy-paste-programmer. Even when adhering to the SOLID principles (to the extend that it makes sense) there might still be duplication that should be removed.The catch here is that this duplication will be mixed together with duplication that should stay, since removing it makes the code less maintainable. Hence trying to remove all duplication is likely to be counter productive.

Costs of Unification

How can removing duplication make code less maintainable? If the costs of unification outweigh the costs of duplication, then we should stick with duplication. We’ve already gone over some of the costs of duplication, such as the need for Shotgun Surgery. So let’s now have a look at the costs of unification.

The first cost is added complexity. If you have two classes with a little bit of common code, you can extract this common code into a service, or if you are a masochist extract it into a base class. In both cases you got rid of the duplication by introducing a new class. While doing this you might reduce the total complexity by not having the duplication, and such extracting might make sense in the first place for instance to avoid a Single Responsibility Principle violation. Still, if the only reason for the extraction is reducing duplication, ask yourself if you are reducing the overall complexity or adding to it.

Another cost is coupling. If you have two classes with some common code, they can be fully independent. If you extract the common code into a service, both classes will now depend upon this service. This means that if you make a change to the service, you will need to pay attention to both classes using the service, and make sure they do not break. This is especially a problem if the service ends up being extended to do more things, though that is more of a SOLID issue. I’ll skip going of the results of code reuse via inheritance to avoid suicidal (or homicidal) thoughts in myself and my readers.

DRY = Coupling
– A slide at DDDEU 2017

The coupling increases the need for communication. This is especially true in the large, when talking about unifying code between components or application, and when different teams end up depending on the same shared code. In such a situation it becomes very important that it is clear to everyone what exactly is expected from a piece of code, and making changes is often slow and costly due to the communication needed to make sure they work for everyone.

Another result of unification is that code can no longer evolve separately. If we have our two classes with some common code, and in the first a small behavior change is needed in this code, this change is easy to make. If you are dealing with a common service, you might do something such as adding a flag. That might even be the best thing to do, though it is likely to be harmful design wise. Either way, you start down the path of corrupting your service, which now turned into a frog in a pot of water that is being heated. If you unified your code, this is another point at which to ask yourself if that is still the best trade-off, or if some duplication might be easier to maintain.

You might be able to represent two different concepts with the same bit of code. This is problematic not only because different concepts need to be able to evolve individually, it’s also misleading to have only a single representation in the code, which effectively hides that you are dealing with two different concepts. This is another point that gains importance the bigger the scope of reuse. Domain Driven Design has a strategic pattern called Bounded Contexts, which is about the separation of code that represents different (sub)domains. Generally speaking it is good to avoid sharing code between Bounded Contexts. You can find a concrete example of using the same code for two different concepts in my blog post on Implementing the Clean Architecture, in the section “Lesson learned: bounded contexts”.

DRY is for one Bounded Context
– Eric Evans in Good Design is Imperfect Design

Conclusion

Duplication itself does not matter. We care about code being easy (cheap) to modify without introducing regressions. Therefore we want simple code that is easy to understand. Pursuing removal of duplication as an end-goal rather than looking at the costs and benefits tends to result in a more complex codebase, with higher coupling, higher communication needs, inferior design and misleading code.

Forem: Jeroen De Dauw

Clean Comparisons

Readable PHP Comparisons

Readable JavaScript Comparisons

More Comparisons

Advice for Junior Developers

Foundational Advice for Juniors

1. Code is not the Point

2. Software Design Matters

3. Use the BEST Practices

Technical Advice for Juniors

4. Write Tests

5. Do Not Use Inheritance For Code Reuse

6. Write Object-Oriented code

7. Write Functional Code

8. Use Informed Duplication

9. Types, Names and Comments

Recommended Reading for Juniors

Books

Blogs

Bonus links

About Jeroen

Applications as Frameworks

Decoupled web application

Frameworks vs Applications

Plugins and Extensions

Applications as Frameworks

An OOP Solution

Base Libraries Should be Stable

Stable Base Libraries

See also

Iterators and Generators in PHP

What are Iterators

PHP's collection type hierarchy

IteratorAggregate + Generator = <3

A real-world example

Tips and Tricks

Yield in PHPUnit data providers

PHPUnit data providers

Using yield

The Fallacy of DRY

The Goal of Software

The Good Side of DRY

The Fallacy of DRY

Costs of Unification

Conclusion