Forem: Sven Hettwer

Don`t try to sell chaos to the management!

Sven Hettwer — Mon, 09 Jul 2018 03:42:24 +0000

I recently visited a meetup in Düsseldorf where Russ Miles was talking about chaos engineering, its techniques, its mindset and the technology that support this awesome way of exploring weaknesses in systems. After the talk ended, it was clear to see that the attending developers were fascinated about these concepts, as was I. There was so much motivation to start testing and experimenting with software systems, it was awesome!

The next day, when I was thinking about the things I’d learned, I came up with some questions that I could not answer to myself. Fortunately, Russ is a person interested in sharing knowledge and experience. That’s why he offers that his audience should contact him as soon as questions come to their mind. So that is what I did.

Sven: Hey Russ! I was thinking about your talk last evening. I’m curious how you would explain the worth of chaos engineering or things like Game-Days, where the developers are locked out of business for a whole day, to the management? All these things improve the software quality, but this is nothing you can make money with in a direct way.

Russ: Hey! So firstly, a “Game Day” can be a lot shorter than a day. In my experience it’s a few hours at most and fits well into the usual ceremonies around your agile process and preferably around retrospectives. So it can be achieved with a minimally increased time investment in comparison to the process you may already be running. Second, when I explain chaos engineering to managers, I use the term “continuous limited scope disaster recovery”. I also point out that software resilience is directly related to money, in most cases, and that this is a proactive, lightweight approach to avoid costly outages.

Sven: Makes sense! But in my experience, even those arguments don’t convince everybody. Some may argue that they never had an outage ever so why should they invest?

Russ: Usually, there has been an outage in the near-past and so people are familiar with the potential costs of not investing into resilient software. If that’s the case, I ask about that outage and ask what they do to avoid that and other outages in the future. I also ask how they’re now building up confidence in their systems again. Another important point I mention is that for a small investment they can get big returns. They just have to add a little more discipline through resilience engineering practices.

If they haven’t experienced an outage themselves in their own systems, I bet they have been impacted from an outage in someone else’s system that they depend on. So I use that as a “you wouldn’t want to be them” perspective. In addition, I also have a growing catalogue of production failure stories that I could tell if people are still unconvinced.

Sven: You mentioned that things like Game-Days fit well into agile processes. So would you say that an agile process is a prerequisite to efficient chaos engineering?

Russ: Oh no, not exactly. I’d say that any business-critical system of software development and delivery will gain from improved resilience. If you’re agile, then you might have the sorts of learning loops in place that are critical already, but chaos engineering does not require an agile process, microservices or cloud native. It’s much more broadly applicable than those things. They sometimes help, but they’re not a prerequisite.

And just another remark on arguing for chaos engineering: As a customer, we’ve all experienced an outage at some point. So even if people have never been in touch with a system outage in their business, they know how an outage feels.

Sven: Maybe that’s a good point to start from, if you have to argue for these things with someone. Something like “What would you think if you would be on the couch to watch a movie but the service is down? And now imagine the mood of your customers with your product if it’s unavailable.”

Russ, that made it a lot clearer for me! Thank you for your time and your thoughts!

Russ: No problem at all!

For me, the most important outcome of this conversation is:

Don’t try to sell chaos to the management!

Better call it “continuous limited scope disaster recovery” and point out that their revenue stream is directly linked to their software. If the software is unstable, your revenue stream is unstable. And if this is not enough to convince the skeptics, show them how they would feel when they are hit by a system outage.

There are also a few other dos and don’ts Russ is pointing out in his article “Chaos Engineering for the Business - 6 Tips for how to explain Chaos Engineering to non-technical stakeholders”.

Authors: Sven Hettwer, Russ Miles

This article is originally published at medium.com/@SvenHettwer

Arch Linux for Devs

Sven Hettwer — Sat, 23 Jun 2018 08:58:49 +0000

This report is about the experience I've made with Arch Linux as the operating system for a developers workstation. You'll be introduced into the concepts of Arch Linux, followed by a introduction into the main tasks such as package installation and OS maintenance. At the end, I'll discuss why I think that Arch Linux is a great OS for developers, and finish with a conclusion.

Introduction

First of all, I'd like to avoid the typical OS comparison statements like "Linux is better than Windows", "Mac is better than Linux" or "Windows is better than Mac". If the operating system suites your personal requirements, it is the right choice for you. There is, as in so many aspects in life, no absolute "better" choice.

Please note that this article is nevertheless based on my personal opinion and may vary from your or others opinions.

What is Arch Linux?

Arch Linux is a extremely lightweight, fully customizable, rolling release Linux distribution which is driven by its community and has been written from scratch. It follows the KISS principle which results in the fact that there is no overhead in the operating system such as a graphical or a commandline installation dialog nor is a configuration tool provided by the distribution itself available. Therefore Arch Linux is targeted on the more experienced Linux user. The simplicity of Arch Linux allows you to build your system exactly as you need it without a reinstall to obtain a new major OS version ever, due to the rolling release attempt. That gives you the opportunity to run your first Arch Linux installation theoretically forever, if you maintain it well.
Different to other distributions, Arch Linux does not ship with a default desktop installation, a default file system or a default tool set. It is just an operating system. Nothing more and nothing less. That's simplicity, isn't it?

There are various guides in the official wiki that help you to setup your system as desired. There are also projects providing distributions based on Arch. One of them is Anarchy Linux, which I'm personally using. It's basically a Arch Linux with an installation dialog. It's still possible to customize your installation by aborting the installer and do things manually or to uncheck some packages you don't want to be installed.

Package management

The package manager of Arch Linux is called pacman. It's a CLI tool allowing you to install, remove and update the packages on your system. May the packages be self build, downloaded from the official repositories or from the AUR (Arch User Repository), where users are able to provide self maintained packages, pacman is your tool to manage them. Let's have a look at the most important commands:

pacman -Syu: Update all packages after synchronizing the package repository database
pacman -S <package-name> Install a package
pacman -R <package-name> Remove a package
pacman -Qdt Query the package database and find all packages that are not longer required.

The only slightly uncomfortable task is to install a package from the AUR. These packages have to be build, before you're able to install them. This requires the following steps:

Download the sources
cd /path/to/the/sources
Check all files carefully, because AUR repos could potentially contain anything.
makepkg -si

Another uncomfortable task is to keep your AUR packages up to date, because pacman is not able to do that. So you would have to check the versions of your AUR packages on a regular basis and reinstall them if required.
But don't be scared! AUR helper tools to the rescue! These tools are capable of managing the official repositories as well as the AUR. I've recently chosen aurman as my AUR helper, because it is well maintained, well documented, contains a lot of functionality and simply works well. It is also relatively new as its AUR entry has been created at 2018-03-20 22:31. Have a look at the aurman AUR page for more details.
To install aurman on a fresh Arch Linux, just install it manually from the AUR as mentioned earlier. If you're interested in some more details, just visit the Arch User Respository wiki entry.

Because aurman has a native pacman integration, you can just use it as you would use pacman but with AUR support.

Quick edit on 2018-12-28:
Polygamma has stopped aurman development:
I am not willing to continue developing aurman for public use. The GitHub issues are closed, I am not interested in feedback, feature requests or bug reports anymore. aurman is going to be developed for myself only from now on, I suggest migrating to yay. source
For more information, please have a look at the README on github.

I migrated to yay and it works great. Thx to @maxdevjs for pointing out that this blog still recommends aurman. Happy new year everybody!

Available packages

One of my biggest concerns has been, that I could be unable to find all the software packages that I need for my daily work or that the packages may be outdated, because Arch Linux is using its own packaging system. As of today, all packages I require where available via the official repositories or the AUR. This also includes software as the IntelliJ Ultimate IDEA. Contradicting to what I once thought, the Arch Linux packages are absolutely up to date, if not bleeding edge.

Maintaining your system

Software updates

As for many other operating systems the most usual maintenance process is installing updates. As mentioned earlier, this can be done easily with one command pacman -Syu or aurman -Syu in case that you've chosen aurman. If you install your updates once a week, it wont take long. Currently my updates take ~10 minutes. This excludes the updates where I have to update IntelliJ IDEA. This takes 5-10 min. extra.
Before you start an update, I highly recommend to check archlinux.org for any known issues. As mentioned, Arch Linux is a bleeding edge rolling release distribution. So it might be that some updates require your attention more than others. As of today, I've had only one update that required manual interaction. The instructions to solve this issue (js52 update) have been well documented on archlinux.org.

Backups and Restore

Having backups of your files is one of the most important rules when you work with computers. I think nearly everybody has had a situation where data was lost because of insufficient backup. So, I assume that this lesson has been learned and therefore I'll skip that chapter. The more interesting part, that I want to talk about is creating a backup of your operating system so that you're able to restore your system in case of a dramatic system failure. As always, you've many possibilities to achieve that. Let's discuss some scenarios and possible solutions.

Unstable system

Let's pretend, you've updated your system. Now your system is throwing error messages around and is slower as it should be. Identifying the root cause can easily become a tedious task if the update was not only performed for one package but for 10 or 20. So, it would be great to roll back your system, right? No Problem! If you've chosen BTRFS as your file system, you're able to create snapshots of your entire filesystem. These snapshots can be restored If you've booted into a restore medium or a rescue partition. Please note that a btrfs snapshot is technically **not* a backup. It is just a copy on write shadow of your original device.*
Well, to be fair, BTRFS is not Arch Linux exclusive. But it leverages the rolling release concept as any other software package does. Therefore you'll most likely receive bug fixes and improvements earlier than with other distributions.

Fatal system crash

Worst case scenario! HDD/SSD damage. Because you've created backups of your files, it's not that bad, right? You just have to setup your system again, restore your files from backup and everything is fine. Nevertheless, installing all of your required packages and setting up your individual configurations may consume some time. Therefore I recommend to backup these information as well, because what is an operating system more than a bunch of packages and configuration, right? Here are the tools I use:

etckeeper: Creates a git repository for your /etc folder including fully automated commits after every pacman operation due to lifecycle hooks.
packup: A simple tool to export the list of installed packages. It also provides a installation functionality, but I would go with aurman here as well.
dotfiles: A tool to keep track of your dotfiles across multiple systems.

So a recovery of a system using these tools is really simple: Install Arch Linux, install all packages from your exported package list, restore your config, restore your dotfiles, done!

More information

As mentioned earlier, there are tons of information in the official wiki. This also includes a article about system maintainance. Here you'll find a lot of tools and tasks to keep your system healthy and stable. Please note that not all of the mentioned tasks are required to keep your system stable. Just pick what you think is important to you.

Why I think Arch Linux is great for developers

Many of the mentioned aspects in this article are achievable with other distributions as well, but there are some points that make Arch Linux the ideal choice for developers from my point of view.
As developers, we know about software and technology. We know how software works, we know how these technical things come together, we know how to treat software systems. And if we don't know, we're able to learn quickly, because software is our daily business and passion. These aspects in combination with the simplicity and flexibility of Arch Linux is a great combination with a lot of potential to make your day to day work as efficient as possible.

As a rolling release distribution, you've always an up to date system without fearing the end of a LTS period. The software packages are also up to date if not bleeding edge. That means, you don't have to wait until a maintainer decides that a new package version is stable and good to go. You're in charge to maintain your system and customize it as you desire while the maintainance effort is similar to other distributions. You decide, if you want a fully fledged modern desktop environment with all the nice looking effects, or if you want a minimal or CLI driven system unleashing the full computation power of your machine. And if things break? Well, then you know where to search for the issue, because you've setup your system and therefore you know whats running on it.
All this in combination with the active Arch Linux community and the detailed documentation in the official wiki makes it a great choice for developers in my opinion.

Conclusion

As mentioned in the beginning:
If the operating system suites your personal requirements, it is the right choice for you.
In my case, this is Arch Linux. Not because it brings a new stunning feature or a life changing enhancement to the world of Linux but it brings some different concepts that make sense and improve the way the system works and feels from a users perspective.

I want my system to be as efficient as possible without missing the comfort of a desktop environment. I also want to be able to work with up to date software. I want my system to be stable and reliable and I want to understand what's going on on it. All of this and nothing can be found with Arch Linux.
With Arch Linux you have a simple, fully customizable, community driven rolling release operating system with a massive ecosystem of software packages and documentation. You’ve everything available that is available in other distributions, but with Arch Linux, you’ve the choice to use it or not. Nevertheless Arch Linux is not for beginners. You should have a solid understanding of Linux systems, before you start. But if you gained that knowledge, you'll be able to use its full potential.

This article is originally published at labs.consol.de at 2018-06-18

Database testing with Citrus

Sven Hettwer — Sat, 23 Jun 2018 08:53:37 +0000

Database communication is an essential part of many applications, when persistent data storage is required. May it be orders, customer data, product recommendations or product information, if persistent storage is in place, the data contains a certain business value. Therefore it's important that your software handles your persistent storage the right way.

In this blog post you'll learn how to test your database communication using Citrus.

Why would you test your database with Citrus?

Testing database communication at all is not an easy task. There are a lot of things to consider if you want to use a test database and work with it in an effective, automated way.

You have to deploy a test database.
You have costs for additional hardware, licence fees, etc.
You have to setup your test schemas.
You have to ensure that the test schema matches the prod schema and vise versa.
You have to add comples sample data to your schema to test complex use cases.
Testing your software offline is impossible.
Testing your software in case of a database failure is nearly impossible.
etc.

You'll find way more challenges to cope depending on your individual infrastructure.

We thought that all this has to come to an end, so we developed Citrus JDBC!

Requirements

To follow this tutorial, your system should fulfill the following requirements:

git
maven
Java 8
An IDE of your choice

Introduction

With the upcoming Release of the citrus framework 2.7.4, you'll be able to verify the database communication of your application without setting up a test database server or even schemas!

The Citrus JDBC integration consists of two essential parts, the Citrus-JDBC-Server and the Citrus-JDBC-Driver.

To be able to use the Citrus JDBC integration, you just have to do two things:
1) Exchange the productive JDBC Driver of your application with the Citrus-JDBC-Driver in the test environment.
2) Add the Citrus-JDBC-Server to your test project by adding the citrus-jdbc dependency.

The driver will be available on maven central under com.consol.citrus.citrus-db-driver while you can find the dependency in the following snippet.

<dependency>
  <groupId>com.consol.citrus</groupId>
  <artifactId>citrus-jdbc</artifactId>
  <version>${citrus.version}</version>
  <scope>test</scope>
</dependency>

Currently Citrus JDBC supports the following features.

Validation of:
- Connections
- Statements
- Prepared Statements
- Callable Statements
- Transactions
Simulating database failure
Simulating timeouts
Mocking result sets on requests

For detailed information, please have a look at the citrus user guide.

Please note, that it's also possible to obtain snapshot releases of the documentation as well as nightly builds of Citrus and it's components from the following repository:

<repository>
    <id>consol-labs-snapshots</id>
    <url>http://labs.consol.de/maven/snapshots-repository/</url>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
    <releases>
      <enabled>false</enabled>
    </releases>
</repository>

Example

Because this post is about a snapshot version of Citrus, some examples and outputs may look slightly differently, when the version is finally released.
As for many other use cases, the citrus-examples project contains examples concerning the jdbc implementation. So in the first step, please clone the citrus-examples project.

git clone https://github.com/christophd/citrus-samples.git

If you are new to the citrus-examples project, please have a look at the Running the examples guide.

Assumed that you performed the examples setup, let's execute the samples!

mvn clean verify -f citrus-samples/sample-jdbc/pom.xml -Dembedded

[...]

CITRUS TEST RESULTS

ExecuteQueryIT.testCreateTable ................................. SUCCESS
ExecuteQueryIT.testDelete ...................................... SUCCESS
ExecuteQueryIT.testDropTable ................................... SUCCESS
ExecuteQueryIT.testSelect ...................................... SUCCESS
TodoListIT.selectTodoEntry ..................................... SUCCESS
TodoListIT.testAddTodoEntry .................................... SUCCESS
TodoListIT.testException ....................................... SUCCESS
TodoListIT.testIndexPage ....................................... SUCCESS

TOTAL: 8
SKIPPED:       0 (0.0%)
FAILED:        0 (0.0%)
SUCCESS:       8 (100.0%)

------------------------------------------------------------------------

Now that you've executed your first database integration tests, lets have a look on how they're written.
If you're completely new to Citrus, it might be helpful to start with the setup guides on citrusframework.org to understand the basic concepts before you proceed.

This test performs a GET request to our SUT API, asking for the list of todo entries. Afterwards, when the SELECT statement hits the database, a result set is generated and sent to the SUT as query result. Finally the receive step of the http client verifies, that the SUT produces the correct data representation - here in XHTML.

public class TodoListIT extends TestNGCitrusTestDesigner {

    @Autowired
    private JdbcServer jdbcServer;

    @Autowired
    private HttpClient todoClient;

    @Test
    @CitrusTest
    public void selectTodoEntry(){
        variable("todoName", "citrus:concat('todo_', citrus:randomNumber(4))");
        variable("todoDescription", "Description: ${todoName}");

        http()
            .client(todoClient)
            .send()
            .get("/todolist")
            .fork(true)
            .accept("text/html");

        receive(jdbcServer)
            .message(JdbcMessage.execute("SELECT id, title, description FROM todo_entries"));

        send(jdbcServer)
            .messageType(MessageType.JSON)
            .message(JdbcMessage.success().dataSet("[ {" +
                    "\"id\": \"" + UUID.randomUUID().toString() + "\"," +
                    "\"title\": \"${todoName}\"," +
                    "\"description\": \"${todoDescription}\"," +
                    "\"done\": \"false\"" +
                    "} ]"));

        http()
            .client(todoClient)
            .receive()
            .response(HttpStatus.OK)
            .messageType(MessageType.XHTML)
            .xpath("(//xh:li[@class='list-group-item']/xh:span)[last()]", "${todoName}");
    }
}

As you can see in the highlighted sections, you're able to configure the database behavior, including the returned result sets, using the receive and send actions of the JDBC-Server endpoint. This empowers you to test your system without additional test-database deployments, test-schemas or other operative overhead.

You may ask yourself, where the todoClient and the jdbcServer are coming from. These components are specified in a spring configuration class.

@Configuration
public class EndpointConfig {

    @Bean
    public HttpClient todoClient() {
        return CitrusEndpoints.http()
                .client()
                .requestUrl("http://localhost:8080")
                .build();
    }

    @Bean
    public JdbcServer jdbcServer() {
        return CitrusEndpoints.jdbc()
                .server()
                .host("localhost")
                .databaseName("testdb")
                .port(3306)
                .timeout(10000L)
                .autoStart(true)
                .build();
    }
}

Citrus-JDBC Features

Now that you know how these kind of tests are setup, lets talk about the feature set you receive by adding the citrus-jdbc dependency to your project. Due to the fact that all tests are setup similarly, I'll leave the redundant part out of the examples and focus on the feature related parts.
All features are also available in the Citrus XML DSL. You can find the related examples in the Citrus examples project.

Simulating database failures

One of the core concepts of Citrus JDBC is the distinction between a success and an error concerning database operations. This concept makes it incredibly easy to simulate a database failure after a certain message has been received by the jdbcServer.

receive(jdbcServer)
        .message(JdbcMessage.execute("@startsWith(" +
                "'INSERT INTO todo_entries (id, title, description, done) " +
                "VALUES (?, ?, ?, ?)')@"));

send(jdbcServer)
        .message(JdbcMessage.error().exception("Something went wrong"));

After the message has been sent from the Citrus-JDBC-Server to the Citrus-JDBC-Driver, the SUT receives the database failure as an SQLException and is then in charge to handle it. Afterwards you're able to validate the behavior of your SUT by using other Citrus components such as a Citrus HTTP Client.

Prepared/Callable Statements

Even if prepared statements and callable statements are for different database operations and use cases, they share a similar workflow from the API perspective.

connection.prepareStatement("INSERT INTO todo_entries (id, title, description, done) VALUES (?, ?, ?, ?)");
connection.prepareCall("{CALL limitedToDoList(?)}");

Both kinds of statements are created from a JDBC connection with a SQL command expression as String parameter. Because of that, we're able to verify them in the same way as regular statements. The only difference between prepared and callable statements compared to regular statements is, that the parameters for the concrete call are attached to the query string for validation purposes.

http()
        .client(todoClient)
        .send()
        .get("api/todolist/1")
        .fork(true);

receive(jdbcServer)
        .message(JdbcMessage.execute("{CALL limitedToDoList(?)} - (1)"));

send(jdbcServer)
        .messageType(MessageType.JSON)
        .message(JdbcMessage.success().dataSet("[ {" +
                "\"id\": \"${todoId}\"," +
                "\"title\": \"${todoName}\"," +
                "\"description\": \"${todoDescription}\"," +
                "\"done\": \"false\"" +
                "} ]"));

As you can see, the API call path parameter api/todolist/1 is reflected within the statement execution validation "{CALL limitedToDoList(?)} - (1)". Multiple parameters would look as following "{CALL limitedToDoList(?, ?, ?)} - (1,2,3)". Of course you're also able to work with the regular Citrus matcher as always.

Transaction handling

When it comes to complex modifications of your database, transactions are commonly used. Citrus is able to verify the behavior of your system under test concerning start, commit and rollback actions of transactions. The verification of transactions has to be enabled in the server Citrus-JDBC-Server configuration.

@Configuration
public class EndpointConfig {
    @Bean
    public JdbcServer jdbcServer() {
        return CitrusEndpoints.jdbc()
                .server()
                .host("localhost")
                .databaseName("testdb")
                .port(3306)
                .timeout(10000L)
                .autoStart(true)
                .autoTransactionHandling(false)
                .build();
    }
}

With a JdbcServer endpoint configured that way, you are able to state and verify the transactional behavior of your SUT within your test cases.

receive(jdbcServer)
        .message(JdbcMessage.startTransaction());

receive(jdbcServer)
        .message(JdbcMessage.execute("@startsWith('INSERT INTO todo_entries (id, title, description, done) VALUES (?, ?, ?, ?)')@"));

send(jdbcServer)
        .message(JdbcMessage.success().rowsUpdated(1));

receive(jdbcServer)
        .message(JdbcMessage.commitTransaction());

Because of the ease of creating database failures, you could also cause failed transactions and check whether your SUT performs a proper rollback.

receive(jdbcServer)
        .message(JdbcMessage.startTransaction());

receive(jdbcServer)
        .message(JdbcMessage.execute("@startsWith('INSERT INTO todo_entries (id, title, description, done) VALUES (?, ?, ?, ?)')@"));

send(jdbcServer)
        .message(JdbcMessage.error().exception("Could not execute something"));

receive(jdbcServer)
        .message(JdbcMessage.rollbackTransaction());

Even the absolute worst case, a rollback failure, could be simulated with nothing more than another send statement.

receive(jdbcServer)
        .message(JdbcMessage.startTransaction());

receive(jdbcServer)
        .message(JdbcMessage.execute("@startsWith('INSERT INTO todo_entries (id, title, description, done) VALUES (?, ?, ?, ?)')@"));

send(jdbcServer)
        .message(JdbcMessage.error().exception("Could not execute something"));

receive(jdbcServer)
        .message(JdbcMessage.rollbackTransaction());

send(jdbcServer)
        .message(JdbcMessage.error().exception("Could not perform database rollback"));

Timeout simulation

Simulating timeouts with a real database can be tricky if not impossible to achieve. With Citrus, you just configure your server to validate connection statements but leave them out of the test.

Here is a server configuration enforcing connection validation.

@Bean
public JdbcServer jdbcServer() {
    return CitrusEndpoints.jdbc()
            .server()
            .host("localhost")
            .databaseName("testdb")
            .port(3306)
            .timeout(1000L)
            .autoStart(true)
            .autoConnect(false)
            .build();
}

The following test does not accept the connection attempt from the driver because the receive instruction for the server is missing. Therefore the connection will timeout, because no answer will be sent to the SUT. Nevertheless the Citrus HTTP client expects that the SUT responds as expected.

http()
    .client(todoClient)
    .send()
    .post("/todolist")
    .fork(true)
    .contentType("application/x-www-form-urlencoded")
    .payload("title=${todoName}&description=${todoDescription}");

http()
    .client(todoClient)
    .receive()
    .response(HttpStatus.I_AM_A_TEAPOT);

Advanced validations

There are some advanced validation options to verify the very basic behavior of your SUT concerning database operations like:

Opening and closing connection
Creating and closing statements
Performing transactions (seen in the previous section)

Because most modern application use some kind of framework for their database operations, one as a developer is not responsible for managing these basic JDBC driver interactions, but in case you want to be sure, you're absolutely able to! As for transactions, connection and statement validation has to be activated via configuration of the JdbcServer endpoint.

@Configuration
public class EndpointConfig {
    @Bean
    public JdbcServer jdbcServer() {
        return CitrusEndpoints.jdbc()
                .server()
                .host("localhost")
                .databaseName("testdb")
                .port(3306)
                .timeout(10000L)
                .autoStart(true)
                .autoConnect(false)
                .autoCreateStatement(false)
                .build();
    }
}

Now the JdbcServer instance will validate the connection related operations including statement creation.

receive(jdbcServer)
        .message(JdbcMessage.openConnection());

receive(jdbcServer)
        .message(JdbcMessage.createCallableStatement("{CALL limitedToDoList(?)}"));

receive(jdbcServer)
        .message(JdbcMessage.execute("{CALL limitedToDoList(?)} - (1)"));

send(jdbcServer)
        .messageType(MessageType.JSON)
        .message(JdbcMessage.success().dataSet("[]"));

receive(jdbcServer)
        .message(JdbcMessage.closeStatement());

receive(jdbcServer)
        .message(JdbcMessage.closeConnection());

Summary

In this blog post you have executed your first citrus database tests, learned how these tests are setup, how to verify your database communication using the features of citrus-jdbc including low level JDBC driver related operations. This empowers you to get rid of most of your test database deployments by using Citrus. If you are interested in further readings, I highly recommend you to visit the following links.

This article is originally published at labs.consol.de at 2018-03-01

Automated debugging with git

Sven Hettwer — Sat, 23 Jun 2018 08:40:33 +0000

Imagine your're working on a bigger feature in a complex piece of software. Your implementation is complete, all tests in scope turned green and you push your changes for integration testing. Then, some integration tests from a completely different module fail and you have no clue which change may have caused this. Now you start analyzing the issue. Probing your commits by hand would end up in a very tedious process for sure. Thankfully git can do all the work for you, while you enjoy a cup of coffee.

The high-level command git bisect allows you to automatically run a specified test procedure, while it's crawling through your commit history to find the bad revision.

Requirements

To do the hands-on part, the following setup is required on your local machine:

Git
Java 8
Maven
Bash (Cygwin on Windows)

What is git bisect?

The official description of the git-bisect documentation states:
Use binary search to find the commit that introduced a bug.

So what does that mean?

Let's say you have a bunch of commits, and you know that, at some point, your software was okay. Now, your software is broken, which means, that you introduced a bug somewhere during your development. What git bisect does is, it divides the revision graph into a good part and a bad part by testing specific commits chosen by a binary search. Based on the result of the tests, git navigates towards the broken commit. After a few iterations, git will be able to identify the revision that introduced the issue.

The basic git bisect workflow

The workflow of git bisect consists of two major steps. At first, you have to specify a range of revisions, limited by a good and a bad revision. These revisions can be referenced as revision hash, tag or any other git revision selector. At second, the bisect process starts and git automatically performs a checkout of the first revision to test. After passing the result of the test to back to git, by executing git bisect good in case of a success or git bisect bad in case of a failure, the next revision will be chosen. The second step will be repeated until the first broken revision has been found.

Manual git bisect

Now that you know about the idea and the workflow, let's continue with the hands-on. At first, checkout the repository from GitHub containing the example project. As you can see, the repository contains three folders:

cake-factory: A sample spring boot web service containing some issues to find
simple-bisect: A reference how to manually use git-bisect <- This is what we'll do now
automated-bisect: A reference how to use automated bisect including a sample script

Now let's try to build the cake-factory by executing

mvn -f ./cake-factory/pom.xml clean install

You'll recognize, that the build was not successful, because of a failed Citrus integration test. Now, let's find out, where we introduced the bug.

As shown in the workflow, we've to setup the bisect scenario first. Therefore we have to find a good and a bad revision to define the borders of the analysis. Both revisions are easy to find. The good revision is most likely the revision, you branched from. The bad revision is HEAD, due to the fact that your tests are failing now. Nevertheless, for our example we'll use two prepared tags stable and broken from the repository.

git bisect start # start the bisect procedure
git bisect bad broken # label the *bad* commit
git bisect good stable # label the *good* commit

Or if you want to use the shorter version:

git bisect start broken stable # start the procedure and label the *good* and *bad* commit

Now, that your bisect has been started, git has chosen a commit to test and provides the commit message as well as an estimation of the required iterations.

By not later than now, you should have figured out a test strategy to identify the root cause of your problem. In our case a integration test failed. Therefore we're concentrating our debugging efforts on integration tests. The setup of the cake-factory allows us to skip unit tests and start right away with the integration tests, by setting the property skip.unit.test=true.

mvn clean verify -f ./cake-factory/pom.xml -Dskip.unit.tests=true

In this revision, everything is okay and the build was successful. Now we have to report this result to git

git bisect good # tells git that the current revision is okay
#If the test would have been failed, you would have used 'git bisect bad'

Based on our reporting, git has chosen the next revision to check.

Now you have to repeat the testing and reporting procedure. After some iterations, you'll find the following commit causing the problem.

Note the revision hash of the broken commit and don't forget to close your bisect session.

git bisect reset

If you look into the code of that revision, you'll find out that the default cake, delivered by the cake services, has been changed to vanilla as stated in the commit message, but the integration test has not been adapted.

Automated git bisect

As you may have recognized, the manual execution of the bisect steps produces some overhead and is a repetitive task. This increases, if you work in more complex projects with multiple modules involved. Thankfully git bisect contains a sub command that allows us to execute a specific command or a scripted test procedure with automated reporting of the result. This does not only reduce the required amount of time but also the possibility of mistakes while repeating the steps by hand.

git bisect run <command to execute>

To report the result of the test to git, your command or script has to fulfill the following contract:
If the test was successful, the executed command delivers the return code 0, otherwise a return code other that 0 is returned.
Due to the fact that you just have to fulfill this contract, you could also do some more advanced testing logic using python, or even java. Nevertheless I would suggest to be pragmatic and keep the test simple.

Example

Let us return to practice. If not already done during the previous section, please checkout the repository from GitHub containing the example project. In the "automated-bisect" folder, you'll find a findBug.sh bash script, that contains the test procedure from the "Manual git bisect" section as code.

#!/bin/bash

mvn clean verify -f ./cake-factory/pom.xml -Dskip.unit.tests=true

To execute the automated test now, just prepare a bisect environment and pass the script to git bisect run.

git bisect start broken stable
git bisect run sh automated-bisect/findBug.sh

Alternatively, if you have a one line command as in findBug.sh, you could also pass the command directly to git bisect run as long as the passed command fulfills the contract.

git bisect start broken stable
git bisect run mvn clean verify -f ./cake-factory/pom.xml -Dskip.unit.tests=true

When the process has been finished, git will show you the first bad commit as known from the manual part.

Tips and Tricks

About git bisect and TDD

If you're following the TDD life cycle to develop your software, you may have some troubles using git-bisect, because your software alternates between a erroneous and a functional sate from a software test perspective. As soon as you specified a new test, your software becomes unstable. If you would execute a simple mvn clean install in such a situation, git bisect would not find the revision containing the root cause of your problem, because it is confused by the failed builds caused by your TDD tests. This can also be shown with our cake-factory example, by removing skip.unit.tests=true from the maven instruction.

git bisect start broken stable
git bisect run mvn clean verify -f ./cake-factory/pom.xml

Now git bisect will react on every single unit test that is failing. This leads to a result as well, but it has nothing to do with the root cause of the issue we were looking for.

Setup you test scenario wisely

As you have seen in the section "About git bisect and TDD", a test scenario which is not restrictive enough, returns misleading results. Therefore it's important to create a test scenario, that suites your situation. Here are some bullet points that might help you avoiding pitfalls.

Keep your tests small Test only what's necessary and relates directly to the issue. This will lead to the correct result and reduces the amount of time for your tests to run.
Know your dependencies If the failing test depends on a different module, that has changed during development, keep in mind that you have to rebuild the dependency in every iteration before starting the test.
Automate your tests If you do things more than once, automate them! Don't execute your tests manually. It's a repetitive task that consumes a lot of time. Script you test and let git bisect run do the rest.
Use common "bad" and "good" candidates Don't waste your time on thinking about the most efficient borders for your analysis. Because git bisect run will do the job for you, one iteration more or less is not important at all.
- bad candidate: mostly HEAD
- good candidates:
- The revision you branched from
- The last released version of your software
- The last revision of your branch that was build successfully. (E.g. last nightly)

Summary

As you have seen, git bisect is a powerful tool to find broken commits within your git history. You're able to define a range of revisions that will be checked automatically. If you'd like to learn more about git bisect, please visit the following links:

This article is originally published at labs.consol.de at 2018-01-12

Traveling through the Arch — Vol. 4

Sven Hettwer — Wed, 01 Nov 2017 12:58:18 +0000

My Desktop — Background picture by ChaseOnTwoWheels

Hey folks! It has been a while since I was able to blog. But know, let’s get together for the final episode of “Traveling through the Arch”. This time, we’ll have a look at system customization, system maintenance and a résumé about my experience with Arch Linux™ so far.

System customization

I took me some time to find out how to talk about system customization, because it’s custom and depends on your requirements. Therefore there is not “the ultimate system” I can recommend to you. At this point it’s about you to find out what kind of system you want to build and what you really need and want. So instead of giving recommendations, let me just point out the General recommendations section of the Arch documentation. It’s a good spot to start from. From that point you could build everything from a minimal installation up to a fully fledged desktop environment.

System maintenance

System maintenance was one of my biggest concerns when I started with Arch Linux™. Because the software is distributed in a rolling release it could absolutely be possible to install an update that crashes your system. To prevent those situations, I imagined that I would have to read a lot of articles and patch notes to determine whether a particular upgrade would destabilize my system. Luckily, this is not the case. But nevertheless you better secure your system to be able to solve such a situation as fast as possible.

There is a big article in the Arch Linux™ wiki talking about that topic. Basically it all comes down to one command.

pacman -Syu

S Synchronizes your system with the remote repository
y Updating the package database
u Filtering the package list for outdated packages. Those are the packages you want to update

But that’s not everything you should do. I’ve installed several additional programs to help me maintaining my system stability.

packup A simple terminal based tool to backup your pacman list of installed packages. It’s not only possible to save but also to restore from a given list of packages. So if everything goes wrong, you still have your list of installed packages to start a new installation or try a recovery.
pakbak A tool to backup your pacman database. This is similar to packup but stores much more data. In addition there is a pakbak.service specification so that this tool creates a backup whenever your system starts. Unfortunately the scripts contain some bugs and the service definition is not 100% complete. Therefore I experienced some problems using this tool. Another unfortunate fact is, that the maintainer does not react to my issues so far. Nevertheless it’s a cool attempt and if the maintainer does not answer my requests in the near future, I’ll think about forking that project and maintain it myself.
etckeeper A tool using a scm to safe your configs before and after installing packages via pacman. So you’re always able to switch to the last running config by just checking out the correct version.
btrfs subvolume snapshots of my home This allows me to track the changes of my home directory without storing tons of data on external drives. In case I did some stupid things with my data, I just recover from my last valid snapshot. There are some tools creating snapshots of the desired subvolume based on certain events. One more popular solution is Snapper but the btrfs developer are working on their own solution called autosnap which is currently not part of the upstream. Nevertheless this does not secure your data to 100%, because it stays at the same drive. A incremental chronological backup system is also required if you ask me.

Even if it comes to system maintenance, you’ve a ton of options and possibilities. You don’t have to use a certain backup solution someone created for a given problem. Just set everything up as you want and as you think it’s useful.

Résumé

Now that I work Arch Linux™ on a daily basis for more than a month, I’m absolutely happy that I decided to switch over from Xubuntu. I’ve learned incredibly much about Linux and how everything works together, I’ve a truly minimal system where I can feel that it runs faster than a Xubuntu would.

The System is totally stable and I feel that I could really restore it, if something breaks. This is something that I’ve never experienced with all the distributions I’ve used. All software packages I need are available from the official repositories or from the AUR. In addition, I’ve the feeling that I’m really free to setup my system as I want without missing up to date software.

So in case you have not tried Arch Linux™ until know, I would like to recommend it to you. One told me once “If you make the switch, you’ll never switch back” . Sir, you’ve been right!

This article is originally published at medium.com/@SvenHettwer

Traveling through the Arch — Vol. 3

Sven Hettwer — Wed, 11 Oct 2017 11:01:26 +0000

Welcome back to the third issue of “Traveling though the Arch”. The last post was about a Anarchy Linux installation using the installer from arch-anywhere.org. This time, we’ll install a basic Arch Linux™ by our own hands without any predefined installer.

Requirements

To follow this guide, I would like to recommend the following setup:

Virtual machine (VirtualBox, KVM, etc.)
A downloaded Arch Linux™ iso image to install the OS from.
A web browser to read the instructions. I guess you‘ll have one when you’re reading this article ;-)
10–60 minutes of time depending on the level of detail you’re looking for

Introduction

At first, let me point out, that I’ll not cover every step of the installation in this article but only some of the more interesting parts. You can find the whole installation process in my Arch-Installation GitHub repository, where every command is commented. I separated the installation into three major steps just as the official installation guide does.

Creating a bootable USB Stick Just in case you want to install it on a real machine
Installing a very basic Arch Linux™
Installing everything else on top Will be covered in the next episode

This structure will be represented in the GitHub repository as well.

Creating a bootable USB Stick

Under Linux, there is nothing easier than that!

Disclaimer!

If you use dd, please be careful! You can easily erase your disks, if you set of to the wrong device.

dd bs=4M if=/path/to/archlinux.iso of=/dev/sdX status=progress && sync

dd A tool to copy files byte by byte.
bs=4M Telling dd to read/write 4M byte at a time.
if Stands for i nput f ile.
of Stands for o utput f ile.
status=progress Limits the output of the output to transfer statistics.
&& An logical and operator. Within the context of this command: If dd was successful, please also execute sync.
sync Tells the system to write cached data to persistent storage.

If you’re not familiar with dd or the way that Linux handles devices, please use a tool like the Startup Disk Creator or one of the other numerous programs to create bootable USB Sticks.

If you’re starting from Windows, I’d recommend the LinuxLive USB Creator.

Installing a very basic Arch Linux™

Arch Linux™ — Boot screen

After booting your system from the medium of your choice, the boot screen will show up as in any other distribution. But After deciding to Boot Arch Linux (x86_64) you’ll be dropped into a the live system with nothing more than a command prompt.

Arch Linux live shell

Partitioning and encryption

After performing some basic configuration, I prepared my partitions. I used the tool cfdisk (just because I know how it works) and created two partitions. One partition for /boot, with round about 200MB of space and the rest of the disk for everything else.

If you’re curious what /boot is:

Referring to the Filesystem Hirarchy Standards (FHS), the /boot folder holds the static files of the boot loader. It is allowed to store the kernel there as well.

The only reason, why I setup a different partition for my boot loader and kernel is, that my main partition will be encrypted and therefore a decryption by the kernel is required before user-mode programs can be loaded.

So now that I’ve setup my partitions, it’s time to encrypt the device!

Therefore I used the tool cryptsetup, which is a tool that is able to perform a plain dm-crypt or a LUKS encryption. dm-crypt is the cryptographic device mapper module where LUKS stands for Linux Unified key Setup, which extends the plain encryption of dm-crypt by an additional meta information header easing the identification of the encryption algorithm, finding user data within the encrypted data blob and makes some additional cryptographic features possible.

cryptsetup -y -v luksFormat /dev/sdaX

cryptsetup Tool to encrypt devices
-y Asks for passphrase verification
-v Verbose
luksFormat Encrypt with LUKS header
/dev/sdaX The device to be encrypted

Encryption is done!

Setting up the File system

After opening the encrypted device, we’re now able to write our file system.

mkfs.btrfs /dev/mapper/cryptroot

mkfs A tool to create Linux file systems
mkfs.btrfs Creates a btrfs file system
/dev/mapper/cryptroot The opened/mapped device

mkfs.ext2 /dev/sda1

mkfs.ext2 Creates a ext2 file system
/dev/sda1 The device to create the file system on. In this case, this is my /boot partition

Setting up a file system is not that hard. Choosing your file system however is very hard. There are multiple file systems out there with many different features. Because I don’t want to start comparing all the pros and cons of the different systems, let me just point out why I’ve chosen ext2 for my boot partition and btrfs for everything else. At first: It was a recommendation. But that’s not enough reason to do so. Therefore I did some research.

ext2 is what one would call a very basic file system nowadays. It was meant to solve various issues of ext. But due to its simplicity it’s ideal for boot partitions, USB-Sticks and basically every storage, where journaling is not required.

btrfs on the other hand is a file system that hits its stable state initially in 2014. It’s a copy-on-write file system with a vast amount of features including self-healing mechanisms, file system snapshots, subvolumes and much much more. Especially because of it’s snapshot mechanism it’s very easy to create file system backups without copying tons of data to an external drive.

Setting up initramfs

This section clarifies what bugged me the most when installing Linux systems with encrypted devices… until now.

The keyboard layout while setting up the passphrase was German (due to my choices during the installation process) but while booting the system, the layout switches back to English!

This led me to so many moments full of frustration but now, they are gone!

And unfortunately it could have been so easy avoiding the frustration at all.

There is a file on your Arch Linux™ machine called /etc/mkinitcpio.conf.

This is a configuration file for (who would have guessed) a program called mkinitcpio which is able to create initramfs archives. The most important part of this configuration file is the definition of the HOOKS variable. HOOKS specifies the composition and behavior of the initramfs. One of this glorious hooks is called keymap. And this is what keymap does:

Adds the specified keymap(s) from /etc/vconsole.conf to the initramfs.

Loads the specified keymap(s) from /etc/vconsole.conf during early userspace.

Source: https://wiki.archlinux.org/index.php?title=Mkinitcpio&oldid=492088

mkinitcpio -p linux

Done!

This means, that the keyboard layout of my choice is loaded within the initramfs. This empowers me to use as many special characters in my passphrase as I want!

Summary

One told me once: Setting up an Arch Linux™is no witchcraft.

Now that I did it, I can only agree. I took some hours to find out only some of the details I wanted to know. One could spend hundreds of hours more depending on the level of detail you’re looking for. So the learning curve is steep and one will have to do some parts of the installation twice but at the end, you’ve a system which is absolutely transparent to you. And if I forget what I did once while I performed the installation, I can just look it up in the repository. At my current point it feels like there is not a single situation where the system could receives critical damage to the software that I’m unable to repair.

This article is originally published at medium.com/@SvenHettwer

Traveling through the Arch — Vol. 2

Sven Hettwer — Wed, 04 Oct 2017 11:01:01 +0000

Here we are again! In the last post, I summarized what I know about Arch Linux™ and its ecosystem. This time I’ll go through the installation using the Arch Linux™ distribution Anarchy Linux form https://arch-anywhere.org.

Let’s start up the (virtual) machines!

Installation

After telling the system to boot the live environment and choosing to install Arch Linux™, one is asked to answer some basic questions well known from other installers like preferred language and keyboard layout in a shell based dialog. After answering that, it’s time to partition your drive. In the first attempt, I’ll try the Auto partition encrypted LVM option, because I’ve a encrypted LVM on all my machines and will definitely use some kind of encryption with Arch Linux™, too. Choosing the partition size, choose to create a swap partition, typing in the encryption passphrase for the LVM.

Encryption passphrases… These little nasty things…

Well, if you have not guessed it yet, I’m from Germany and here in Germany, we’ve a different keyboard layout (qwertz). So using a passphrase that contains special characters led me into some very annoying situations when I realized, that the installer absolutely supported the German keyboard settings, but the prompt when booting your system after the fresh installation, does not! There we’ve the american layout and the special characters are located in different places. So I added a special character to my test password to try out, what’s going on with Arch Linux™ concerning this problem.

The partitioning has been finished. Time to choose your kernel. I’ll stay with the base kernel for no special reason. After that I was choosing bash, grub, the networkmanager, the 32 bit repositories in addition to the 64 bit, enabled DHCP, disabled wifi utils, excluded PPPoE DSL connections (2017 u know?) and excluded os-prober because I’ve no multi boot system here. Typing my choices into this article took way longer that choosing them via the dialog.

Now I’m at the point of choosing my desktop environment.

I’ll go with the default Xfce4 Light Desktop, just as always. But wait! There is something called Arch Anywhere Xfce, which is some kind of a custom Xfce4 from the Anarchy Linux developer. I’ll try this one out!

Going to the next screen…

Screenshot of the installation process — Detected VirtualBox installation

So the installer is clever enough to detect that I’m using a VirtualBox installation and will therefore install the guest utils package? Awesome!

After choosing the last few components for the installation, the installer asks me if I want to install all of the packages I’ve chosen. For sure I will and there we go…

A few minutes later…

All packages have been installed. I choose a name for the computer, set the root password and created a test user with root privileges. Rebooted the system… aaaaaand…

O’rly? That was not what I expected…

Okay so the ever so promising installer left me with a broken system after the fresh installation. Honestly, I’m not a into the deeps Linux expert so that I would exactly know what went wrong and what I have to do, to solve the problem. Therefore I’ll try to install the system again with the exact same setup.

A few more minutes later…

Okay… it seems like it worked this time…

So what could be the reason for that? I’ll come back to this point later on. Now let us enjoy the magnificence of Arch that everybody told me about!

Ah! Before I forget: The LVM passphrase prompt comes with an English keyboard layout, while the booted OS loads the default keyboard setup you’ve chosen during the installation. So remember kids: Before you setup your LVM passphrase, change the layout of your keyboard to English!

Screenshot after the first time booting successfully into Arch Linux™

Mission complete! Clicked a little bit around to discover what’s behind all the fancy symbols. Yep! Feels like a Xfce4 with some nice extras like the drop down shell. I’ll keep that one in mind. But for some reason I’m not satisfied with what I found here. And the reason is not Arch Linux™, it’s not the custom Xfce4 (which IMHO brings an awesome look and feel with it) but the installation of the System. So let’s get back and try to find some reasons why the first installation was broken after reboot.

Research about the broken installation

Due to the fact, that I’m unable to reconstruct the technical environment, the only thing I could do now is guessing. But even if I’m not a Linux kernel hacker, the installation process is nothing more than a program. I can read programs! So, lets visit the Anarchy Linux github repository. With as of today round about 2k commits and 41 contributors, happily developing.

Contributions of Anarchy Linux

Now lets find the place where the setup of the hard drive is made. A quick look into the arch-installer.sh showed up that prepare_divers is what we’re looking for and this is contained in configure_devices.sh.

Before I go any further into details, I want to make clear that I appreciate the work of everyone who is providing free/open software to the world! I even more appreciate if someone offers her or his work to everyone free of charge! This is why the free software/open source community has grown so much and is so strong! So many thanks to the Anarchy Linux team for your work and the effort you put into your project! I really much appreciate it!

As of today, the configure_devices.sh script is 850 lines long, whereas the auto_encrypt method, which has been called after my set of choices, is 111 lines of that. The maximum indentation I’ve found in that file is 10. So navigating the code and finding what may have caused the troubles isn’t that easy, if you haven’t seen the code before. So after 5 Minutes scrolling the code I just left it. Googling the issue was not helpful too, because I could not find out what has exactly been done by the installation scripts.

Conclusion

I think the main question now is: Would I recommend the arch-anywhere.org installer? The answer is, as ever so often: It depends.

If you’re a technical or Linux newbie, I wouldn’t recommend Arch Linux at all, start with Mint, Ubuntu or something like that.

If you know how Linux basically works and you’re looking for a distribution you can grow with, use the arch-anywhere.org and start to hack your distribution to your heart’s content.

If you know Linux quite well and just want to get rid off the extra ballast coming with the more end user oriented distributions and start with bleeding edge technology, write the installation by yourself and avoid any predefined installer.

IMHO the benefits of doing the installation manually are huge.

You know exactly what you configured or installed on your system. So if a problem occurs, you know the parameters of every command you’ve set during installation so troubleshooting is way easier as if you would use a tool that selects the parameter due to user choices.
A predefined installer is never able to provide all the options to you that the software comes with. E.g. the Anarchy Linux installer is not able to set user home encryption on top of the LVM encryption.
Your system is truly minimal. You don’t know if an installer is choosing some additional packages and if you want to find out how the installer exactly works, you’ll have to bring a lot of time with you.
You learn a lot! You’ll never learn as much as doing things with your own hands. This includes running into pitfalls, some level of frustration and pain, but at the end you will succeed and get a deep understanding of how things work together.

In the previous post, I said, that installing everything by my own hands is not state of the art. And depending on the use case, this is still true. A non technical user would never be able to setup an Arch Linux™ out of the box. But this is not the use case of the distribution. It wants to be simple (not meaning user friendly), efficient and lightweight whereas the target audience is focused on people with technical know how.

So contrary to my statement from the last article, I’ll setup my Arch Linux™ by my own hands!

So lets start to explore strange new commands, to seek out for new packages to use and new setup options to choose in the next Episode of “Traveling through the Arch”.

This article is originally published at medium.com/@SvenHettwer

Traveling through the Arch — Vol. 1

Sven Hettwer — Wed, 27 Sep 2017 11:01:01 +0000

In this series of blog posts, I want to travel with you through my experiences with Arch Linux™ from the very beginning to the (hopefully) happy ending.

As mentioned in my first blog post, I’ll join a new company next week. Because they allowed me to choose my OS, I’ll go for a Linux distribution. I use Linux on my PC, my Netbook, my Smartphone, etc. So for me it’s a obvious choice. The only question I ask myself every time, if I want to install a Linux on new hardware is: What distribution should I use?

This time, I’ll try Arch Linux™. No clue what Arch Linux™ is? No Problem! I’ll try to give you a little overview of what I found out!

Currently I’m mostly using Debian GNU/Linux, Ubuntu or one of its derivatives. Both of them are easy to install and run on nearly every system. If a more stable system is required where up to date technology is not a must have, I go with Debian GNU/Linux. For development, I prefer Ubuntu or its derivatives, because the packages are kind of up to date. Nevertheless Ubuntu is a very heavy distribution consuming a lot of memory for it’s Unity Desktop (and Xubuntu with Xfce isn’t much better), it is shipped with a lot of software I’ll never need and every version has a expiration date (even the LTS versions). This is where Arch Linux comes into play.

Arch Linux™ is a very lightweight distribution which allows you to setup your system just as you want it to be. You’re able to choose what runs on your computer package by package. To be fair: You could also achieve that, if you use a minimal Debian GNU/Linux installation and do the rest of the setup manually.

Speaking of doing things manually…

The installation guide of Arch Linux™ is a step by step tutorial of how to setup a operating system with nothing more than a bootable medium and your own hands. If you ask me, this is not the way it works nowadays. Even though that I’m a software developer, I don’t want to set up all the little bits and pieces for my OS installation. Time is valuable! When should I write Blog posts, if I have to set up my OS manually first?

So I had to find something smarter. A friend of mine recommended Arch anywhere to me. This project provides a minimal installer for an Arch Linux™ based distribution called Anarchy with the option of choosing additional packages from the official Arch Linux™ repositories. So I’ll give it a go!

An additional advantage of Arch Linux™ is, that it’s a rolling release distribution. That means that you’ll never have an outdated OS as long as you perform software updates. There is nothing like a support limitation for your OS version. Every update is shipped to every installation out there. This is also interesting in terms of bleeding edge technology. You’ll never have the problem to obtain up to date packages for your OS, because there is no version number limiting the support or compatibility.

But where light is there is also shadow!

Working with bleeding edge technology harbours the risk of crashing your system because of unstable packages or updated. And this is one of the biggest concerns I’ve with Arch Linux™. I want an OS that is stable in any situation. For me it would be a nightmare to loose one day of work, because my OS crashed by some incompatibilities. So the only way to minimize the risk of breaking your system, is to update frequently and visit the official website and forums to find showstopper before you update. This will require at least a bit of time and effort to keep yourself up to date.

Another point I’m concerned about is that Arch Linux™ is using its own package manager called pacman. So if there is no official support for the software you’re looking for, you may have some trouble. Even though there is the Arch User Repository (AUR) containing a lot of community-driven packages, I’m not sure if I’ll be able to find everything I need there.

Enough of theory! Let’s start with the “hands on” part in the next Episode of “Traveling through the Arch”.

You want to read more about Arch Linux™?

Just visit their website for more details!

This article is originally published at medium.com/@SvenHettwer

Why I started!

Sven Hettwer — Tue, 26 Sep 2017 18:21:31 +0000

Well, I think it would be a good start for my blog if I explain why I started.

It may sound easy to start with a blog post but it actually cost me four or five attempts to formulate these first words.

I think the main reason for me is a pragmatic one. By the end of this week, I leave the company I worked the last few years for. While I was working there, I was an active member of the software developer community, because I love to share knowledge. I gave tech talks about various topics like Reactive programming with ReactiveX or clean code, I wrote blog posts about many things worth to share and I created a clean code community to teach others. I would have liked to do some more community work but daily business needs to be done, too.

So because I’m leaving I was looking for a possibility to keep in contact with my old colleagues and continue sharing what I’ve learned and what I work with. And here we are!

I’ll try to write as much as I can and hope, that you’ll find it interesting.

This article is originally published at medium.com/@SvenHettwer