Forem: Piotr Gaczkowski

Ego-less Development: The Philosophy of Better Code

Piotr Gaczkowski — Wed, 14 Oct 2020 15:37:35 +0000

What is the first thing that comes to mind when you hear the word "philosophy"? Probably an image of ancient bearded guys, walking down the beach reading books, writing books, thinking about books, discussing ideas that have nothing to do with the real world.

The whole idea seemed disconnected from reality. People were trying to survive: growing crops, producing food, manufacturing tools, and doing all sorts of practical things. And those guys were walking, talking and thinking about how we got there and where we're headed.

But the first impression is deceptive. Philosophers focused on things that are familiar and that are close to ordinary life, like problems of logic, problems of ethics, social and political constructs. So if you ever catch yourself thinking about these concepts, you can call yourself a philosopher. It's not limited to abstract and metaphysical things. It also covers things that can happen in everyday life and in tech.

Of course, philosophy isn't the first thing that comes to mind when we talk about programming. But if we look a bit deeper, we'll notice that its concepts are relevant for the tech industry too. So it's not a surprise that a lot of people who code take interest in it.

The Definition of Ego

So what is ego-less development? To understand the concept we first need to define ego.

The concept of ego is more complex and multilateral and there are different definitions of what it is. Basically, the ego means self; it is what makes us - us, the driving force that makes us feel special in our own understanding and perspective.

Everyone has a different view on their own "self", the one that fits in their understanding of life and the world.

Another common interpretation of the ego is putting our own interests above others. It's a survival mechanism. Even though we live in a modern world, our basic needs are still driven by this instinct. We need to survive, we need to reproduce, we need to provide. Sometimes it makes us think that we need to take advantage of the others to reach our goals. This kind of ego makes us selfish.

We can also understand ego as an inner critic or our internal feedback mechanism that tells us that we are going great things or bad things.

What unites these interpretations is that ego is focused on the person itself, not on the others. And this can cause problems in different aspects of life, including working in tech.

Problems Caused by Ego

There are several ways that the ego can complicate software development.

Terrible APIs

When an API designer has a lot of knowledge and the knowledge is paired with huge ego, curious things may happen. Using all the fancy design patterns "others should know about" may make the API so complicated noone beside the author would be able to use it. And if they won't use it, they will probably create their own solutions thus duplicating the work.

That's the happy path, at least. Even worse scenario is when the team uses the terrible API in a way that causes bugs. Instead of moving the product forward, everybody is suddenly debugging.

"Not Invented Here" Syndrome

Recreating something that's already there because we know how to do it better. For example, the proliferation of different Linux distributions. Sometimes there are valid reasons to create a new software distribution and sometimes we do it because we disagree with the others and want to live with the consequences of our own choice.

Ego Clashes

When developers try to highlight their own contributions at the cost of teamwork and/or try to protect themselves against the doings of others. It might look like a panic mode and can harm the company, the team, and the product as the problems accumulate.

(Entrenched) Code Ownership

We hear a lot that we should strive for code ownership and developers should be responsible for the code they produce. If a code has an author, then they can make sure that the code is of a good enough quality for a release. That's the good side of code ownership.

But every medal has two sides. The negative side of code ownership manifests in the reluctance to accept a pull request even though it is sensible. It can happen because of personal believes and protectiveness or because the solution is simply novel.

In fact, whenever someone says "We've always done it this way" it's a clear indicator of an ego play, a sign that someone feels insulted and doesn't have a better way to defend themselves.

What Philosophy is All About

You can skip this part if you're only looking for solutions to the problems outlined above. This part is for those interested in different approaches to ego throught the culture.

Let's get back to philosophy for a moment. Nowadays, with the rising popularity of AI and ML, it became more relevant as there are different problems related to ethics.

Stoicism

One of the philosophies I want to mention here is stoicism. Ancient Greek and Romans practiced it. It recently got the second wave of interest thanks to the popularity among many public figures in tech and marketing. They see it as the answer to many of our problems and that it fits our cultural challenges.

In my understanding, the main focus of stoicism is identifying what is and what isn't under my control. Imagine waking up today and seeing that it is cloudy outside. You are asking yourself whether it is going to rain or not, what happens if it does, how frustrated you'd be if it ruins your clothes. There is plenty of overthinking and you focus on the problems.

The stoic approach to this situation would be putting on your waterproof clothes and taking an umbrella. This way you are taking control of the situation.

When it comes to the perception of ego there is also this feeling of being in control.

When someone calls you a poor excuse for an engineer you might feel offended. The natural thing to do would be to fight back, to argue and say "No, I'm not, I know what I'm doing, it's perfectly fine!". That's the loss of control.

Stoics' approach would be to think in different terms. Ask yourself whether it is an immediate danger to our well-being and does it affect our well-being in a long-term perspective. Is it true? Is the person trustworthy? Does this opinion count? Can it hurt our status, our position? Or maybe it hurts the offender even more by showing that this person lost control and we kept our cool.

Stoicism helps us to listen to our ego but question these assumptions within our heads. Identify the real threats from the perceived ones. Is it a tiger in the bushes or is it just a striped rock?

Soli Deo Gloria

There is another philosophical way of thinking that has an interesting approach to the ego. Various Christian thinkers used two similar terms to express it.

Soli Deo Gloria (glory to God alone) which means that we shouldn’t take credit for everything that we do. We are the part of some bigger environment and we should always consider this environment and give the credit due.

For example, I've heard that Johann Sebastian Bach started signing his works with the initials SDG. When he started doing that, his work improved. He was no longer thinking about how he can show his genius to the audience, but how he helps the audience by doing his work.

Jesuits used similar approach with their Ad Maiorem Dei Gloriam (which means for the greater glory of God). And we can see how it works when we look at the process of building a cathedral.

Imagine going back in time a few hundred years and talking to a builder laying bricks on the construction site. We ask them about what you are doing. One builder replies "Oh, don't even ask. I'm laying bricks the whole day in the heat, in the rain, in the wind, in the snow, the work is endless. We'll never finish during my lifetime. People show us no gratitude and are always yelling at us. But I have to do this because I have a family and I need to support them."

Then another says "We are building a cathedral, which is going to be the most spectacular thing you've ever seen. It will be magnificent! I won't live long enough to see it finished but believe me, the future generations will think that we did extraordinary work. I love it."

So what's the difference in these two approaches? The first approach is ego-driven. It revolves around what that person can get from this work. The second approach considers a bigger picture.

Zen

Not forget to mention Buddhism and Zen philosophies and the belief systems of the East in general. In Western culture, people usually like to practice individualism. We believe that we are special, chosen ones to do great things, that is "I" who should be the most important person in the room. In Eastern culture, the focus is more on the collective, on how a person can contribute to the community, their family, their town, their country.

In Buddhism, there is the focus on feeling and being a part of a greater universe. It's not how I can approach things as an individual, but how I can make myself disappear and become a part of the universe.

And the final philosophy worth mentioning is DevOps. DevOps has different meanings and different definitions, but for me, it's something akin to philosophy or to culture, the way we do things.

Look at closing the gap between the client and development, short release cycles, iterative approach to design and reduction of waste. All these things have something in common. They come together because to achieve them we need to build a better communication framework.

We need to be a part of one collective. We can't separate clients and developers. We are building something we can all benefit from. With open communication and shared goals we come to the most important requirement - that we put our ego aside. We have to stop thinking about personal benefits for a moment and think about how we can be more involved and be part of a bigger ecosystem.

How Ego Disrupts DevOps

You know the problems that come with ego, so how they affect the DevOps practitioners?

Inadequate APIs become a source of waste. When you have to look up documentation instead of focusing on more important things, there are more chances to mess things up. And once you've messed it up and the things are in production, the cost of mistakes increases. So APIs should be easy to use and hide the complexities inside, not the other way around.

"Not Invented Here" syndrome is an obvious source of waste. Instead of focusing on delivering a good product, you try to prove somebody else that you are better and more capable.

Ego clashes also present different sources of waste. Everyone on the team focuses on making their part of the job feel more important. That wastes energy and time put into product creation. It may as well lead to decay of a culture, which means problems with efficiency and collaboration.

Entrenched code ownership can lead to "Not Invented Here" syndrome and reduce your bus factor to one. The bus factor is an idea, that illustrates code ownership limitations. How many people have to get hit by a metaphorical bus until there is nobody who knows how your project works? If there is only one person who has all the information and they get hit by a metaphorical bus (which usually means a change of employer), you are left with nothing.

Cultural Tools for Ego-less Development

First of all, you need to create a culture. There are 3 pillars of a strong team culture: safety, vulnerability, and a shared future.

A great culture emerges when people know they are safe, when people share their vulnerabilities, their problems and challenges with each other and when they share some future that they know will get better with time.

Introduce Formal Processes

Unlike bureaucracy, formal processes can help people feel less lost. It should be a logical extension of the culture you are cultivating. You don't need to force it, but there should be a natural progress.

Code Familiarity

Another cultural tool is doing code reviews. People are getting familiar with the code they are producing and everyone knows what's happening beyond the features they are working on.

And pair programming is a good opportunity to leave the ego aside and focus on how to improve the product.

Once everything is familiar and once everyone knows what they are doing, it makes it much easier to work on the product.

Software Tools

I focused on examples in Python because it's the language that is closest to my heart.

Linters (PyLint)

Check your code to see whether it contains any known bugs. If you can see them by reading the code, you can focus on the code quality instead of searching for mistakes.

Code formatters (YAPF, Black)

It takes code and formats it according to the given standard to maintain consistency.

Aux tools (PyAnnotate and MyPy)

They help you create better interfaces, check APIs, and find errors before your clients do.

Git Hooks

You can run it whenever you do an action in GitHub or run them locally. They would check all the tests even before you push the main repository. After that, the continuous integration process would check the changes and finally your peers will get a request for the code review.

Automated Build Scripts (Makefiles)

You can reduce your Continuous Integration/Continuous Delivery pipeline to calling make build, make deploy, make test, make whatever. This way, you can reproduce the same steps on any machine - developer machines, QA machines, CI workers.

Opinionated Frameworks

It's hard to make your team understand why they need to use this opinionated framework instead of a different oneBut once you do, you'll be happy that someone has already made some choices for you and you don’t have to sweat about it.

Behavior-Driven Development

Instead of writing tests to determine if the code behaves correctly, first try to decide with your client what is it that they are trying to achieve with this piece of code. BDD also encourages communication between different stakeholders and brings everyone on board.

Conclusion

These are some of my observations about the ego in play in software development. Where possible I also proposed some solutions and workarounds that could make it easier for the entire teams to own the codebase.

But each of us has probably some ego-related horror stories as well. Share your own in the comments!

Images courtesy of:

Command Line Tools for Developers

Piotr Gaczkowski — Tue, 25 Aug 2020 14:17:15 +0000

In today's online world, the battle to attract users continues to rage on, with app makers preferring either mobile or web apps. Desktop applications are becoming less and less relevant. Moreover, they also tend to be nothing more than rich clients to web apps—Electron being the popular platform of choice.

Does this mean we will soon abandon the desktop as a platform? No, of course not, I wouldn't say that. Besides, while GUI apps seem to have been stalling recently, there is a segment of desktop apps that continues to grow.

Have you ever seen any movie featuring hackers? More often than not, these people are shown working in front of monitors displaying some sort of terminal (usually with a dark background and light foreground). This terminal, in turn, tends to be flooded with passing characters that apparently have some meaning to the person watching them.

Such representations of hackers in action are often mocked by professional developers, and there are even some programs that simulate various "hacker" effects, just for fun.

However, in the real world, command line tools aren't used for their entertainment value.

Why We Still Use Command Line Interface Tools

This article focuses on the practical side of using command line interface (CLI) tools. Knowing CLI commands and using quality tools can make you more productive and can also open doors to various approaches to automation that are far more practical with textual interfaces than with GUI apps.

You can get better at doing repetitive tasks in GUI, to the point that your multiple clicks are heard as a single long one. The problem is, this still won't beat the efficiency of a specialized script. What's more, performing the same operations manually introduces both an added cognitive load and the increased possibility of human error. As usual, we rely on computers to handle tasks humans may find boring, repetitive, or overwhelming.

It is worth knowing that a terminal tool can offer several types of interfaces. There are non-interactive ones like ls, which simply take the parameters and provides the output. There are interactive or semi-interactive interfaces most often found in package managers. ("Are you sure you want to proceed with the installation from unverified source?") Then, there are textual user interfaces (TUIs), which are interactive GUI apps designed to fit the limitations of a terminal. Probably the most famous one is Midnight Commander (mc), a clone of extremely popular (in the 90s) Norton Commander.

Essential Command Line Tools

If you want to become a console dweller, you need to equip yourself with a minimum set command line developer tools—the bare essentials. Things you most definitely can't live without are an interactive shell (aim for something modern with convenient tab-completion) and a text editor.

Now, I will mention the UNIX philosophy, which is often the foundation behind design decisions made by the tool's authors, whether consciously or not. Some of the key points can be summed up as follows:

Treat everything as a file.
Do only one thing, but do it well.
Read from standard input, write to standard output, and communicate errors to a standard error stream.
When succeeded, return code 0. A non-zero value means an error (which can be specified by the exact return code).
Allow for command chaining and scripting.

Shell

The first thing you see when opening a terminal is a shell. This is the part that makes the interaction between the user and the machine possible. It interprets your commands, splits them into program names and arguments, and executes all shell commands you throw at it.

Historically, there have been many different kinds of shells. Among the most popular ones were csh (C Shell) and various implementations of the Bourne Shell (usually known simply as sh). Bourne Shell got extended into Korn Shell which also gained some traction and is still being used by its enthusiasts. Csh is currently the default shell on some BSD systems, while almost all other UNIX-like operating systems prefer some kind of a Bourne Shell. Linux distributions tend to favor bash while Mac OS X comes with zsh as the default choice.

There are other possibilities out there, but they are far less popular, except Microsoft PowerShell on Windows systems. PowerShell is inspired in part by the interactive UNIX shells such as zsh and in part by the .NET runtime. Instead of treating everything as text, a concept common in UNIX world, it allows for object-oriented manipulation of data.

Even though Microsoft PowerShell is quite popular in the Windows realm, many programs with UNIX origins (most notable being Git, Autotools, or Make) tend to prefer some variation of Bourne Shell. Because of this, projects such as msys (bundled with Git for Windows), Cygwin, or Microsoft's recent WSL were born. If you want a Linux-like feeling on Windows, MSys is the best choice here. If you want a full-featured Linux environment able to run standard Linux binaries, then WSL is the way to go. For something in between—UNIX API but compiled as a Windows executable (only use it when you actually know why you need this)—Cygwin is the answer.

Editor

Once you get acquainted with your shell, you will want to pick up some useful skills. As most of the coding work revolves around writing text (code, READMEs, commit messages), a good knowledge of interactive text editors is essential. There are many to choose from, and since an editor is one of the most necessary tools for any developer, there are probably just as many opinions on which editor is best.

The most popular text editors can be separated into two basic groups: Simple text editors and programmable text editors.

Both can be great for writing code, but, as the name suggests, the programmable ones offer the ability to shape and customize the editor to perfectly suit your needs. This comes at a price, though, as they also tend to have a steeper learning curve and may require more time to set up.

BASIC TEXT EDITORS

Among the simple text editors, GNU Nano is the most widespread. Actually, it is a clone of the pico editor, so if one is not available on your system, you can try the other. Another, more modern, alternative to both is the micro editor. If you want something simple and extensible at the same time, this one is a good place to start.

PROGRAMMABLE TEXT EDITORS

Many developers rely on programmable editors from different camps, such as Vim and GNU Emacs. Both editors can run in the console or in GUI mode, and both had an impact on the key bindings found in other software. They both offer not only an API but also actual programming languages built-in. Emacs focuses on LISP and Vim uses its own VimL, but it also offers interfaces to other popular scripting languages (like Lua, Perl, Python, or Ruby). A more recent approach to Vim, called Neovim, is also worth mentioning, as it is starting to get a serious following.

It may be somewhat confusing, but there is also an editor called vi which is a predecessor of Vim (which, incidentally, stands for "Vi improved"). It is much simpler than Vim, but if you have enough confidence to write in Vim, it should not be a challenge to you if you find yourself needing to use vi.

Since pico/GNU Nano and vi/Vim are usually preinstalled on various systems, it is a good idea to at least grasp their basics (quitting Vim is a notoriously hard problem for beginners). This way, if you need to edit something on a remote machine, you will be ready regardless of what editor is already there. On your private device, feel free to use any editor you find the most comfortable.

DEFAULT SYSTEM EDITOR

One last thing to note is that your system may have what is called a default editor.

The $EDITOR environment variable points to the default editor and in Bourne-compatible shells (sh, bash, ksh, zsh) you can see it by entering echo $EDITOR. If the value differs from your personal choice, you can set it yourself by adding export EDITOR=my-awesome-editor to your shell's runtime configuration (~/.profile, ~./bashrc, ~/.zshrc, and so on).

Other programs, such as version control systems and mail clients, will use this editor when they need longer text input.

Multiplexer

As soon as you start doing serious stuff in CLI, you will encounter the limitation of being able to keep only one application open at any given time. When coding, you may want to edit the code, execute it, fix the mistakes, and execute again. When looking for a bug, you may want to list logs and see what gets logged when you send a request to the server. Typically, this would either mean switching between the two applications constantly or opening several terminal windows.

This is where a terminal multiplexer can help you. When speaking of multiplexers, some people immediately assume the topic to be of GNU Screen. It was the first widespread tool of its kind and is still very popular today (often being installed by default). Its modern replacement is tmux which, unsurprisingly, stands for "terminal multiplexer."

These two allow you to have more than one window open in a given terminal session and switch between those sessions freely. They allow you to split the windows into panes, which helps running several applications at the same time and observing their output in real time (without switching any windows). Also, they work in a client-server mode, which means you can detach them at any given time and come back later to continue the work just where you left off. This last feature led to Screen's popularity when people wanted persistent IRC sessions.

For most use cases, GNU Screen or tmux should be great for you, but if for some reason you would consider them to be too heavy on resources, there are also lighter alternatives. There's dtach/atach and there's abduco. They are limited in scope on purpose but can perform their respective duties well.

Package Manager

At this point, you may start thinking about getting all the aforementioned software installed on your machine. One problem is that each of the tools has different installation instructions. Sometimes, you need to download sources and compile them yourself, sometimes you get the self-contained binary, and sometimes you get what is called a binary package, which usually means an executable compressed together with some metadata.

To ease the process of installing software, operating systems creators came with a concept of package managers. Put simply, a package manager is like an app store for CLI and desktop apps. It precedes actual app stores by some decades. The problem is that almost every system has its own package manager. Debian, Ubuntu, and derived GNU/Linux distributions use APT, Red Hat-based distributions prefer yum or DNF, other Linux distros have more exotic means of installing software and so does different BSD clones. Besides built-in package managers, there are also user-installed ones like Chocolatey for MS Windows and Homebrew for Mac OS X/macOS. When you want to write instructions on how to install your program, you may end up writing cases for each of those systems. Seems like a bit too much, doesn't it?

Fortunately the last of the mentioned systems, Homebrew, may be the most portable one. It works on Mac, on Linux, and it even works on WSL if you want to have a similar user experience on Microsoft Windows.

So, besides portability, what else can Homebrew offer? First of all, it does not interfere with the system packages, so everything you install resides on a separate layer to the operating system. Besides, no root permissions are usually needed to install packages. You can, therefore, have system packages which are stable and tested but at the same time check their newer versions without sacrificing the stability of the system.

If you wanted to test the editors, I mentioned earlier that all you need to do on a system with either Homebrew is to run this command:

brew install emacs micro nano vim neovim.

The Shiny Stuff

What we have already discussed is undoubtedly useful for work. But there are also applications that, while not necessary, still bring comfort to everyday life. You may not need them, but it is always worth to know them.

Interactive Filter

Searching the command history can be tedious. While both bash and zsh feature Ctrl+R keybinding, it only shows one substitution at a time. What is more, you need to enter the exact text that you used before. Since this is quite a common operation, once you start using the command line, it looks like a fine place for improvement.

Interactive filters, like fzy, percol, peco or fzf help you with filtering long lines of text. This can either be the aforementioned command history, all the lines of code in a project directory, or a list of filenames generated by find .. The general idea here is to present you first with all the lines available and then rely on fuzzy finding algorithms to filter out everything that doesn't match.

For example, binding Ctrl+R to fzf shows you a list of the most recent commands, which you can navigate up and down using arrows, or you can type git to only show commands that feature Git somewhere inside. Personally, when I work with a shell that does not have an interactive filter, I feel suddenly a little bit lost. This feature is really compelling!

Plus, you can make your interactive filter available inside your programmable text editor. This way, you will have unified searching capabilities between your shell and your editor.

Interactive Navigator

Facebook PathPicker was a great help when I was working mostly with C++ projects. The error log generated by the compiler can get pretty big and pretty nasty, and the ability to find the actual paths inside that log was a productivity boon.

In any given text file, or the content of your screen when used with tmux, fpp filters everything but the file paths. It then presents a UI where you can select one or more of those paths and run a command with them. The most common response would be to open the files in an editor, of course, which is the default action.

Git UI

Chances are at least one of the projects you work on uses Git as a version control system. While being entirely powerful, the Git CLI is not the pinnacle of excellent user experience. To save you some stress reading through all the options in the Git help $SUBCOMMAND, I recommend that you check out tig. It offers a nice console UI for the operations that benefit from it, like log or blame.

Another tool that aims to help GIt users is fac, which is an acronym for Fix All Conflicts. As you might have guessed, it comes in handy when you run into conflicts while doing merges or rebases. It's an alternative to other merge tools like vimdiff.

File Manager

There was a time in the 90s when everybody wanted a two-pane file manager. The trend started with Norton Commander. Many others followed the same path, but the one that still sees a stable user base is Midnight Commander. The most obvious use case is using mc to manipulate local files, but it's also very useful when working with remote machines.

Like most command-line programs, it's very lightweight, so there is no problem running it over ssh and thanks to supporting FTP and FISH protocols, you can have a local file system visible in one pane and the remote one in the other—a convenient feature when you want to avoid typing or copying files names as arguments to scp.

CLI Tools Just for Fun

"All work and no play makes Jack a dull boy," they say. There are a lot of programs, command line and otherwise, that only serve your amusement. The Rogue video game falls into this category. It even gave name to the whole genre of games! Other popular toys are fortune and cowsay, which can make your day a bit less dull if you use them somewhere in your CI scripts, for example.

But for some of us, the main appeal of using a console in the first place is to feel like a hacker in the movies. No More Secrets and Hollywood Hacker represent this group well. Try it when somebody's watching you work, and your hacker cred is certain to rise!

Command Line in Practice

So, what is so appealing about the command line that offsets the hours spent learning how to use the shell, the editor, and all the switches of various apps? The short answer is productivity, which comes from two things:

One is that when you are presented with only a terminal window and nothing more, you can focus more intensely, as there is not much to distract you. No notifications popping up, no ads, no pictures of pretty kittens. Just you and your goal.
The second thing is automation. You can put several frequently combined actions in a script and call it later as a whole instead of typing them all by hand each time. You can quickly get back to a particularly complex command you once wrote by searching through your shell's history. Basically, you can record and replay anything, and the code is available as a documentation of what you did.

The ability to add aliases also contributes to the gains. For example, I find myself often crafting commits in Git by updating the same one until it's perfect (for the moment). Once I stage the desired files, I run git carmh. Don't try to look it up in the manual, as it is my private alias meaning commit --amend --reuse-message=HEAD. It saves some typing for sure.

Thing is, people get bored repeating the same actions over and over, and boredom reduces focus. This can lead to mistakes and errors. The only way to avoid them is not to interlace high-focus and low-focus actions. Writing code is high-focus and reviewing a commit message and contents is high-focus, but when you need to repeat several mechanical clicks here and there to get to the stage of commit review, chances are your focus is lowered. Command line isn't, of course, free of such mechanical activities, but thanks to automation, you can avoid most of them.

Further Explorations

You may already have been aware of some or all command line tools mentioned in this article. You may have learned something new and useful while reading it. If so, excellent---my aim here was not to offer a comprehensive overview and comparison of different tools, but to demonstrate a few crucial tools that I have found helpful in my daily work, in the hopes that you might find some of them useful, too.

There are far more interesting command line programs out there, and if you are interested in them, I recommend checking the Awesome Shell curated list of some of the best command line tools available today.

Most of the GUI apps have their terminal counterpart. That includes web browsers, email clients, chat clients (IRC, Slack, XMPP), PIM suites, or spreadsheets. If you know of any good programs that I haven't mentioned, please bring them up in comments.

Originally published at https://www.toptal.com/software/best-command-line-tools

A master list of Pluralsight courses that will jumpstart your career

Piotr Gaczkowski — Thu, 20 Aug 2020 18:46:29 +0000

One of the biggest mistakes in life that you can do is stop learning. With the rapid technological development and progress, new trends emerge just as fast as the information gets outdated. If you want to stay relevant in your career, you need to keep up to date. And it doesn't apply solely for tech professionals - almost every industry is changing as you read this.

This is why I am such a huge fan of Pluralsight (Disclosure: I get commissions for purchases made through links in this article). For the past two years, I've been using it both to learn and create. And I've been very happy with it - they select the best speakers and high-quality courses, you can get it at the price of a Netflix Premium account and the community support is outstanding. .

Some of my peers asked for recommendations, so I've combined my favorite courses into one list. But first of all, I wanted to clarify one thing: Pluralsight offers more than just technical courses. This is a popular misconception, but you can find courses on a career change, marketing, leadership among the others.

Want to learn Product Marketing? You're covered with this course by Jillian Kaplan.

Not a marketing person? Maybe Product Management is your thing? Here is another good one by Jillian Kaplan.

There's something for the writing community as well. This course by Alan Ackmann highlights the importance of writing for job hunting.

One of my favorite authors on the platform is Jason Alba. If you're searching for a job, his course on Job Search and Career might be just what you need.

Want to learn how and why UX Research can help you create intuitive and delightful experiences? Krispian Emert has all the answers.

Maybe UX Design is more suitable for you? Learn what UX designers do, how they do it, and how you can get started in the field from Kurt Krumme.

Speaking of design, there's an entire path dedicated to Photoshop. Even if you're an expert, you may still be surprised by some of its contents.

When you already know design, the next step might be to create a working Website from scratch. Craig Shoemaker will be your guide on this journey.

Want to go deeper? Why not try creating stylesheets and adding style rules using different types of CSS selectors with Susan Simkins.

Interested in animation? Try making some cute moving things with Rachel Nabors.

Once you handle the basics you may want to find some frameworks. On the frontend, you can choose between the most popular ones: Angular by Deborah Kurata or React by Samer Buna.

Frontend is not your cup of tea? Well, maybe backend with Python and Django with code sensei Reindert-Jan Ekker then?

Having a Web application ready you'll want to deploy it somewhere. Check out whether Docker or Kubernetes is a way to go by Nigel Poulton.

While we're at it, before you start growing, it may help to get some DevOps Fundamentals. Learn with Richard Seroter.

If you decide to go with Kubernetes, Anthony Nocentino will help you get there fast.

Now, you'll probably want to put your work in the cloud. But which one?

GCPcloud? Howard Dierking explains the fundamentals.

AWS cloud? Justin Menga shares how to keep the budget tight.

Azure? Matt Milner is your guy.

You may also consider some Domain-Driven Design (architecture design, nor graphic or UX design) to make sure your application will be easier to develop. Check out this course co-authored by Steve Smith and Julie Lerman.

To keep track of the changes, you'll probably use Git. Paolo Perrotta has something to show you to make you better equipped.

If you're more of a GUI person, this course by Xavier Morera will help as well.

And Git is also the tool used by many Open Source projects, so why not consider contributing on GitHub with Kamran Ayub as your guide?

Web applications are not your cup of tea? Maybe some good old C++ is a better choice, Kate Gregory knows all about it.

Or maybe you've heard Elon Musk and Sam Altman talking about AI and want to get a piece of the action? Computer Vision by Xavier Morera is your first step to building a self-driving car (even if it will be a LEGO one).

And here are some courses by yours truly.

If you like them, click on the Follow button to be notified when something new and exciting pops up (and to bring me some joy). This also applies to all the other great authors sharing their knowledge. Rating and followers mean the world to us!

So many great courses for around the price of a Netflix Premium account! And until the 28th of August, they are available with a 33% discount.

Any other great courses that should make it on this list?

Why you might need Ansible and not even know it

Piotr Gaczkowski — Wed, 05 Aug 2020 17:00:52 +0000

Do you want to start using Ansible? Are you already using it, but coming up against challenges? Even if you don’t fall into either category, don’t stop reading. I’m going to show you why you might actually need Ansible and how to best take advantage of it.

Ansible’s catchphrase is “simple IT automation.” This is a pretty accurate description of what it does. In its most popular mode of operation (there are several), you describe the desired state of your machines, and Ansible manipulates them to achieve this state. At this point, you may be thinking “yeah, but we’ve already got shell scripts for that.” Ansible, though, offers several advantages over good old shell scripts.

First of all, the playbook, which describes the desired state, is declarative and written in YAML. Using a playbook means you don’t need to handle the error control and condition checking yourself. It also means no actions will be taken if the state is already satisfied (e.g. apt-get won’t run if an nginx package is already installed).

But this is only part of the story.

The other thing that makes Ansible so powerful is the use of modules. Instead of relying on many third-party applications (sed, grep, jq, useradd, parted, etc.) and parsing their output, you can focus solely on the state itself. This means that regardless of the underlying userspace programs (useradd, adduser, Busybox, BSD, or GNU variants), you can just specify a universal task as follows:

– name: Create the operator user
  user:
    name: operator
    createhome: yes
    groups: wheel
    shell: /bin/sh

In the same way, you would encapsulate parts of your script into separate libraries, Ansible embraces the concept of roles. Roles describe particular states of a machine, together with possible variables, configuration files, or templates. They’re aptly named, and the machine can use roles like docker, nginx, and python, for example. Each of these may then be tested in isolation and reused among all of your projects. They also can encapsulate more abstract concepts like the ansible-hardening role from OpenStack, which makes a target a little bit harder to break.

And the other cool thing? To run Ansible, you only need Python 2.6+ on your target machines and an open SSH connection. There’s no need to install anything else! This means you’re probably ready to start using Ansible right away! Prepare your control machine by running brew install ansible or pip install -user ansible and follow me.

Deployment as Code

Forget about the README located in the project’s root directory. It usually contains all the tedious details on how to deploy to staging or production. With Ansible, the documentation is in the code. It’s testable, it’s reusable, and it can be run by anybody, as long as the person has access to the target machines. It also forms the perfect basis for a Continuous Deployment (CD) pipeline.
Making sure your app is always deployable can also help avert disaster. Instead of figuring out what to do when your servers go down, you simply edit the inventory file and make a fresh deployment to a new set of machines.

Make Sure Your Team Is Ready for DevOps

When the deployment is a part of the codebase, it has to evolve with the codebase. Make sure all of your team is DevOps-enabled and understands how to use Ansible. The best way to do this is to provide a Vagrant environment so each dev can test the deployment process locally.

It’s essential that you test your deployment code in the same way you test your application. Each change to an application that may be relevant to deployment needs to be reflected in changes to Ansible files. For example, if you add new files that must be copied to the server, make sure there’s a corresponding task. If an application starts logging in a particular directory, make sure Ansible creates that directory and sets appropriate permissions.

It’s Easy to Make It Work, Harder to Make It Maintainable

Ansible has its downsides, as well. These aren’t necessarily related to the tool itself, but they do crop up occasionally. Even though Ansible has well-documented best practices, those don’t always help you achieve a single goal. This can lead to the creation of complicated solutions when simple ones would suffice.

Prefer One-Time Deployments

Even though one of the features Ansible advertises is idempotence, it’s still quite easy to write a playbook that won’t work correctly. For example, how do you update service in an idempotent way? You can’t, it’s self-contradictory, which means you have to sacrifice one thing to save another (an update, that is).

There are two concepts that can help with this problem: disposable infrastructure and immutable infrastructure. The two are pretty similar from a deployment point of view. The former assumes that a machine can be easily disposed of at any point in time after a successful deployment. It may be reconfigured in the future, but there’s nothing stopping you from taking it down at any time. The latter also requires that a machine never changes its configuration after it is provisioned.

Both assume your application is located beyond a load balancer (or a reverse proxy). Such a load balancer could be hosted either externally or internally, independent of the rest of the infrastructure. Services that make up your application are registered in the load balancer. The backend configuration is updated dynamically as services come and go. If you want to host the load balancer yourself, confd or consul-template can help with dynamic reconfiguration.

Use and Reuse Roles

Questions like when and how to use roles or what aspects should be configurable don’t have single, straightforward answers. In my experience, it’s best to think of the various use cases for a particular machine and then to encapsulate those use cases in separate roles that can not only be tested in isolation but can also be reused for other projects. Such reuse also leads to better code quality thanks to a larger testing base.

A Galaxy of Possibilities

Ansible Galaxy provides lots of modules we can use. This is similar to Docker Hub or NPM, where you can search for the relevant parts and use them in your project. They are all written adhering to one standard, meaning they should be easily reusable. Unfortunately, this isn’t always the case.

More than once in my career, I’ve happened upon a module that declared compatibility with Debian but was only ever tested on Ubuntu. This may not be a problem if you can choose your base OS. It may mean some additional work in porting it if you want to make use of existing infrastructure.

Version pinning is another problem. We all know that software evolves and that backward compatibility is rarely observed. When modules use several dependencies, it’s crucial that every such dependency is described with an exact version tag. This way, we can avoid the problem of installing a package in the latest version that’s no longer compatible with another part of the system.

When talking about pinned versions, there’s no avoiding the subject of bit rot. Unused software decays. Referenced hyperlinks can become obsolete, releases can be taken down, hosting services may cease operation. Even if a module uses a pinned version, it may become unusable if it’s not regularly tested and updated as needed, which leads us to our next topic.

Testing Is Hard — and Time-Consuming

Ansible usually operates on system services. While it’s possible to test some of its features in containers (e.g., with Docker), this approach will generally fail. Docker can’t test all the kernel operations or systemd calls, because they don’t exist in its scope. To properly test Ansible playbooks, you need VMs. Did you notice the plural? Good, because it’s not enough to test on just one VM.

The most basic test setup should use a clean VM, run the playbook, check the results, and then run the playbook again to check for idempotency issues. But this only gives you limited information. You still don’t know if the playbook will actually deploy in production. The target machine won’t necessarily be a clean VM (unless you’re already using disposable infrastructure).

To mitigate this risk, it may be a good idea to have a separate VM that could serve as “long-term memory.” This VM, in contrast, would not be cleaned after every test deployment but would allow changes to accumulate over time.

Summary

The best way to document code is in the code itself. Considering this simple statement leads us to a logical conclusion — the best deployment documentation is the deployment code. There are many tools to help achieve this, with Ansible being one of them. Personally, I prefer it to Chef or Puppet, but I’ve yet to try Salt or StackStorm.

As with every tool I’ve encountered in my professional life, it also has its downsides. Knowing them up front should help you avoid some of the problems I’ve stumbled upon. Hopefully, learning from my experience will save you time and frustration in your own work.

Originally published at https://www.iamondemand.com on August 10, 2017.