Forem: Jesper Reiche

Dedicated or Shared Scrum Master

Jesper Reiche — Mon, 13 Apr 2020 06:54:38 +0000

When implementing the agile way of working, companies are left with a decision on whether to have dedicated scrum master or let the role be handled by one of the existing resources in the team.

I have worked in teams with both dedicated and shared resources for the scrum master role and I have formed a very clear preference. The question should also go beyond just that distinction. Because "shared" resource is not just one final decision. It is naturally followed by which shared resource.

I will try to give my observations on a lot of these constellations as some of them has caused quite a bit of frustration for both myself and others in the teams I have been part of.

Dedicated scrum master

Let's start with the dedicated role. The upside of having dedicated resources is of course that they can be better suited for the role. They can have more experience and be better at controlling the process. They should be able to know what works and what doesn't.

The upside of this should be a better run project, less strain on other resources in the project and as a result hereof more work delivered faster. Right!?

The reasoning is sound but in my experience it is a case of

"In theory - everything works in practice".

In the intersection between theory and practice some of the magic from the theory seems to get lost and replaced by - for lack of better word - "anti-magic".

Running a dedicated scrum master has in all instances where I experienced it, given problems. A bit like setting the cart ahead of the horse.

The scrum master and the agile ways of working is hopefully set in place for the project to run smoother and reach its goal most efficiently while still handling corrections along the way.

In the instances where I have been in projects with dedicated scrum masters it has skewed to process over outcome, which isn't really surprising if you think about it. If you have a resource only dedicated to the task of being scrum master, then this resource has 37-40 hours of work to fill with scrum-related tasks.

Tasks that may be very important seen from project management point of view, but not very important in isolation seen from delivery point of view. Yes - if it works as it is supposed to then hopefully it is a giant enabler for the project and hence delivery. But what I usually see is that it ends up being somewhat the opposite.

The scrum process ends up taking on a life on its own and developers and other resources in the team tries to push back as much as possible have time to do "actual" work, while the scrum master will advocate for following the process.

Project management is of course very important. You need control and overview on what can be delivered and what is being worked on in order to do prioritisation. It just should not end up eating away at the actual delivery it self.

A burndown-chart is rarely a delivery in itself.

One way of using dedicated scrum masters is to have them span multiple teams as that acts as a way of filling up their calendar and keeps them focussed on only the minimum required input to keep the project running smoothly.

Shared scrum master

The shared scrum master role avoids a lot of the pitfalls from the dedicated resource, but still has some things to take into consideration. Not the least a decision on what role to take on the additional responsibility.

Let's start with the positives of having a shared resource. The less time you have for an assignment, the more focused your approach becomes. You end up having to focus only on the things that really changes something. So for the Scrum Master role it ends up being only the things that management requires and then leaves as much of the daily/weekly running to the team.

This in my opinion results in a much more lean way of running scrum. It cuts of the fat and only leaves the things that really make a difference in the project.

There are of course some pitfalls. Given that the resource is shared, then either of the two roles this person has, runs the risk of being de-prioritized. Having and executing two roles can be a challenge and it also sets some boundaries on the use of the Scrum Master which may not be there with a dedicated resource.

Having a dedicated resource means that the Scrum Master can go after and run with a lot of blockers for the team members and they can concentrate only on their un-blocked tasks. A shared resource won't have the same capacity for support and hence requires team members to carry a higher load. Some teams thrive in this, others struggle.

Sharing Role

So let's get back to discussing who should then have the resposibility of Scrum Master, if it is to be a shared role.

The end goal of running the "Scrum Process" hopefully is to deliver more, faster and being able to course-correct along the way. Noone would ever run a process just because everyone else does it, without really having decided what to get out of it or why it was implemented - right!?

In order to deliver what is advertised the process needs to be as slim as possible. The end goal is to deliver - right. We need to be able to communicate progress and estimations on what is doable within a given timeframe. That is to keep management happy and give them the possibility to make informed decisions about the project along the way.

We also need to make room for the people working in the project to actually get things done. They should not be bugged down in meetings and exercises just because the process prescribes them. They should deliver clear value.

The reason for reestablishing those points is that I think that they are important when it comes to deciding on which role to take on the responsibility.

If we go pure management/business side, then we run the risk of having someone micromanage and hold process over outcome.

If we go pure developer/executing role, then we run the risk of not having any control at all and hence no ability to monitor or course correct along the way.

Hence my recommendation would be to have someone that straddles both sides carry the extra load. I have seen good business analysts successfully take on the role. I could imagine experienced Tech Leads could do so as well. And perhaps even business oriented architects, depending on their workload in the project.

In short someone that can see the issues that scrum process tries to solve from both sides and keep the process itself as slim as possible to ensure maximum benefit to all parties in the project.

That is as far as I understand actually the core of the agile ways of working. Not the individual parts of the process but the philosophy of getting maximum outcome with as little overhead as possible, while still maintaining ability to change course along the way.

Building .Net Core API for easy database switch

Jesper Reiche — Fri, 24 Jan 2020 21:10:19 +0000

Building a solution from the start to be able to switch database can be a very smart move. In some cases it could be argued that too much software is built for re-usability and with layers of abstraction that are never actually used for anything. But in some cases it makes a lot of sense to make sure that you build your solution in a way that would afford you a good number of options going forward.

So I will explain how you can easily build your .Net Core solutions to enable easy switch of database, but also a little story of a case where I was actually very glad that I had build my solution with this option in mind.

The startup

I am currently working on a startup in the health care sector. We defined the product as requiring an app for both iOS and Android. This pointed me towards React Native as I already had knowledge with React. But it also required a backend API which in theory could be built in just about anything. I am a long time .Net developer as I have worked with .Net since 2006, but I also had an itch towards trying a new programming language.

So my thoughts circled around building this new backend API in Python for instance, as that was one of the languages I wanted to learn. But building something new from scratch in a technology that is unknown and with a deadline, is no easy feat, so Python was not the way to go.

I also wanted something that did not narrow my options for hosting too much, but was still familiar enough for me to be proficient and get something out of the door quickly. My decision ended up being .Net Core with a Microsoft SQL database underneath, handled by Entity Framework via code first. But keeping to my promise of not narrowing my hosting options too much, I wanted to make it easy for me to change the underlying database from MSSQL to something else.

This effort proved useful although the choice of database provider I ended up with was not what I had predicted, but the theory of having to replace MSSQL proved correct.

The code

Enabling the database to be switched altogether has actually never been easier than with Entity Framework and.Net Core. Basically I have the following structure in my API solution, ignoring test projects:

Solution
 - DataModel
    - Context
    - Entities
 - MsSqlDatabase
    - Migrations
 - MySqlDatabase
    - Migrations
 - WebApi
    - Dtos
    - Controllers
    - ....

So basically all my classes for code first generation reside in the DataModel project. This is just a basic class library apart from one thing. It also houses the context. This is the interface that enables the easy switching of database providers, which we will get to in a few seconds.

Then I have two Database projects, one is MSSQL and the other is MySQL as my initial theory was that if I was to switch then it would probably be to MySQL (spoiler alert: it wasn't). And then finally I have the actual WebApi project that exposes the Api and hosts the Startup.cs file we will look at later.

But lets start with the context file I promised earlier:

public interface IEfContext
{
    DbSet<User> Users { get; set; }
    DbSet<Program> Programs { get; set; }
    DbSet<Exercise> Exercises { get; set; }
    ...

    void SaveChanges();

    Task<int> SaveChangesAsync();
}

This is basically the interface that enables us to easily switch providers. By having the entire context described in this interface we can use the brilliant dependency injection capabilities in .Net Core to easily switch between actual databases. Having the two SaveChanges methods exposed via this interface also enables us to override the "normal" SaveChanges method and set some properties on save automatically, which I have also talked more about here: Override SaveChanges

For this project I automatically set CreatedTms and UpdatedTms using this method. And I have created an DataEntityBase class with these properties that I can use for all entities that I want this information present on:

public abstract class DataEntityBase
{
    public int Id { get; set; }
    public DateTime CreatedTime { get; set; }
    public DateTime UpdateTime { get; set; }
}

If we then look inside the MsSqlDatabase project we find two files that are not auto-created:

MsSqlDatabase
 - MsSqlEfContext.cs
 - MsSqlEfContextFactory.cs

Let us start by looking at MsSqlEfContext:

public class MsSqlEfContext : DbContext, IEfContext
{
    public MsSqlEfContext(DbContextOptions<MsSqlEfContext> options)
        : base(options)
        {
        }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<UserProgram>()
            .HasKey(k => new { k.UserId, k.ProgramId });


        base.OnModelCreating(modelBuilder);
    }

    public override int SaveChanges()
    {
        ContextHelper.UpdateTimestamps(ChangeTracker);

        return base.SaveChanges();
    }



    public override Task<int> SaveChangesAsync(CancellationToken cancellationToken = default(CancellationToken))
    {
        ContextHelper.UpdateTimestamps(ChangeTracker);

        return base.SaveChangesAsync(cancellationToken);
    }

    public DbSet<User> Users { get; set; }
    public DbSet<Program> Programs { get; set; }
    public DbSet<Exercise> Exercises { get; set; }
    ...

    void IEfContext.SaveChanges()
    {
        this.SaveChanges();
    }

    async Task<int> IEfContext.SaveChangesAsync()
    {
        return await this.SaveChangesAsync();
    }
}

Unsurprisingly this MsSqlEfContext implements our IEfContext interface, which gives our DbSets and then the SaveChanges and SaveChangesAsync methods. As explained before, the primary reason for having these is that we want to be able to call SaveChanges on our IEfContext which we will be passing around, but also to be able to set some properties automatically on save. In order to obtain this ability to call SaveChanges on our IEfContext and still override the "normal" SaveChanges for the DbContext, we need to do some juggling as you can see from the code.

It also has a little logic to handle many-to-many relationships for a couple of our tables. This is handled in the OnModelCreating method.

Finally in order to be able to have ef migrations work from inside the project folder, we need the following. At least this class was needed at the time of me creating this project - it may have changed since. But as the saying goes - if it ain't broke - don't fix it.

public class MsSqlEfContextFactory : IDesignTimeDbContextFactory<MsSqlEfContext>
    {
        public MsSqlEfContext CreateDbContext(string[] args)
        {
            var optionsBuilder = new DbContextOptionsBuilder<MsSqlEfContext>();
            optionsBuilder.UseSqlServer<MsSqlEfContext>("Server=(localdb)\\mssqllocaldb;Database=myDb;Trusted_Connection=True;ConnectRetryCount=0;");

            return new MsSqlEfContext(optionsBuilder.Options);
        }
    }

With this class in place we have everything we need in order to be able to call "dotnet ef migrations add initial" and be on our way with Entity Framework code first migrations.

The only thing we need to be on our way with our MSSQL implementation is to wire it up in the Startup class and then pass around the IEfContext where ever it is needed. First in the Startup.cs file we need to add the following two lines inside the ConfigureServices method:

// To set MsSqlEfContext as the implementing class of our context
services.AddScoped<IEfContext, MsSqlEfContext>();

// To set SqlServer as the database and use our MsSqlEfContext - it then fetches the connection string from my appSettings class
services.AddDbContext<MsSqlEfContext>(options => options.UseSqlServer(appSettings.MsSqlConnectionString));

Having done this we can piggyback on the dependency injection offered in .Net Core and use the IEfContext inside the classes where it is needed. This is just a random example copied from a service in our solution but it shows the concept:

public class ExerciseService : IExerciseService
    {
        private readonly IEfContext _dbContext;
        private readonly IServiceProvider _serviceProvider;

        // IEfContext will be instantiated and handled by .Net Core dependency injection
        // The life-cycle will depend on how the service is added AddScoped is recommended
        // as shown in Startup.cs
        public ExerciseService(IEfContext dbContext, IServiceProvider serviceProvider)
        {
            _dbContext = dbContext;
            _serviceProvider = serviceProvider;
        }

        public IEnumerable<Exercise> GetExercises(int userId, int trainingProgramId)
        {
            var trainingProgram = _dbContext.TrainingPrograms.Where(x => x.UserId == userId && x.Id == trainingProgramId).FirstOrDefault();
            if (trainingProgram != null)
                return _dbContext.TrainingPrograms.Where(x => x.UserId == userId && x.Id == trainingProgramId).FirstOrDefault().Exercises;
            else
                throw new ArgumentException("Invalid trainingProgramId");
        }

        public Exercise AddNewExercise(Exercise exercise, int userId)
        {
            var trainingProgram = _dbContext.TrainingPrograms.Where(x => x.UserId == userId && x.Id == exercise.TrainingProgramId)
                    .Include(ex => ex.Exercises)
                    .FirstOrDefault();
            if (trainingProgram == null)
                throw new ArgumentException("Invalid training program for user");

            // We just create empty Exercise and add to context to create Id
            exercise.Index = trainingProgram.Exercises.Count > 0 ? trainingProgram.Exercises.Max(x => x.Index) + 1 : 0;

            trainingProgram.Exercises.Add(exercise);

            // SaveChanges on our IEfContext interface
            _dbContext.SaveChanges();

            return exercise;
        }
   }

This pretty much sums up the total implementation. But what it did not touch upon was changing the underlying database provider. Because this implementation worked a treat for very long with the underlying MSSQL database. But as I had anticipated the world changed.

The change

Relying on MSSQL did exactly what was planned; it got us up and running as quickly as possible. I hosted the backend at AspHostPortal, deployed easily from Visual Studio Publish method and that worked fine for internal testing.

But as we matured, both in terms of product offering and product maturity we needed to expand our abilities for scaling. This meant looking into more scalable offerings for hosting, especially to be able to handle storage of a lot of images and videos. Looking at the cheapest options also made it rather clear that change of database from MSSQL to PostgreSQL would keep a lot more doors open.

So finally it was time to put the theory to the test. How hard would it be to actually change database provider. I had as earlier mentioned created a MySqlEfDatabase project, as I thought that would be the other option we might use - but never really come around to using it. But at least the entire project was built on the assumption that we would meet this crossroad.

The new code

First of all I created the new project to hold PostgreSql. Then I added the Postgres Nuget package to that project (Npgsql.EntityFrameworkCore.PostgreSQL). The project follows the exact same structure as the MsSqlDatabase project explained earlier. The PostgreSqlEfContext class is apart from name even an exact copy from its sibling MsSqlEfContext.

This actually led me down a road to consider whether I needed two different projects at all. The theory was that I could potentially just wire them to use either .UseSqlServer or .UseNpgsql, but apart from that use the exact same implementation of IEfContext and hence not have two projects: MsSqlDatabase and PostgreSqlDatabase - but only just a generic Database project.

However looking at the migration files that is automatically created by entity framework it becomes apparent that they are distinct for the underlying database provider and hence trying to make a generic would probably lead to all sorts of strange behavior. Then a little duplication is much preferred.

The only thing in the PostgreSqlDatabase project that is different is hence the Nuget package for Postgres and then the PostgreSqlEfContextFactory class that looks like this:

public class PostgreSqlEfContextFactory : IDesignTimeDbContextFactory<PostgreSqlEfContext>
    {

        public PostgreSqlEfContext CreateDbContext(string[] args)
        {
            var optionsBuilder = new DbContextOptionsBuilder<PostgreSqlEfContext>();
            optionsBuilder.UseNpgsql<PostgreSqlEfContext>("User ID =user;Password=password;Server=server;Port=5432;Database=postgres;Integrated Security=true;Pooling=true;");

            return new PostgreSqlEfContext(optionsBuilder.Options);
        }
    }

Then creation of our Postgres database is as easy as calling: "dotnet ef migrations add initial" and then "dotnet ef database update" and voila! We have a Postgres database with the exact same tables and relations as we had in MSSQL.

In order to wire up Postgres we of course also need to add the Postgres Nuget package to the WebApi project. But then finally we just need to change two lines of code in the Startup.cs class. I have commented out the old ones here for easier reading:

// To set MsSqlEfContext as the implementing class of our context
// First comment out the MSSQL
// services.AddScoped<IEfContext, MsSqlEfContext>();

// Then add the Postgres specific
services.AddScoped<IEfContext, PostgreSqlEfContext>();

// To set SqlServer as the database and use our MsSqlEfContext - it then fetches the connection string from my appSettings class
// First comment out the MSSQL
//services.AddDbContext<MsSqlEfContext>(options => options.UseSqlServer(appSettings.MsSqlConnectionString));

// Then add the Postgres specific
services.AddDbContext<PostgreSqlEfContext>(opt =>
                opt.UseNpgsql(appSettings.PostgreSqlConnection));

And that is it! We just switched from one database provider to another in a smooth and painless way. A lot of it comes from .Net Core's clever dependency injection combined with the strengths of Entity Framework, we just took full advantage of it and combined it with some good old interfaces.

I hope you enjoyed this little walk through of building a .Net Core API for easy database switch. A further advantage of having switched to Postgres is also easier development of .Net Core in Ubuntu, which I have become extremely fond of. And of course also in OSx. But much more of that at another time.

Harmful management

Jesper Reiche — Tue, 21 Jan 2020 17:19:18 +0000

Harmful management is not a very widely used term. I even considered coining the term "Iatrogenic management" but left it - at least for now - even though iatrogenic is a perfect term to describe the problem at hand. Iatrogenic is a term taken from health care and medicine. It refers to the adverse condition of a patient resulting from the treatment of a health care professional. So in other words; action done by health care professional resulting in harm to the patient. This can then of course be put into several categories from unintentional accident to more or less intentional malice.

I have "stolen" the term to use as a vehicle to describe the situation of managers applying actions where none should have been. It is a tough conundrum because managers, especially senior managers are often in their position especially because of their ability to make decisions and take action. Hence the difficulty in letting go and do nothing in pressured situations.

Harmful management appears at a lot of different points in time but usually when the manager comes under pressure. This could for instance be around project deadlines. When a project comes under pressure because of upcoming deadline or perhaps the manager has come under pressure for showing progress, then the manager turns to the one tool in the toolbox that worked so well in the past: Action (Read: manage)

The manager will call in the leads of the project and express the worry. The usual outcome of this is wanting to put more people on the project. More people equals more work done and most importantly signals action. The manager can then go back to the people who had put pressure on him and show action. Extra people was allocated to the project, so correcting measures was taken.

This of course brings us to the good old saying:

What one developer can do in a week, two developers can do in two weeks

Adding more people is rarely the right correcting last minute measure. Yes, it may be an option very early in the process, if those people are to be permanently on the project, but as correcting measures once pressure builds and deadlines approach - very rarely so.

This is where the "iatrogenic" or "harmful management" part kicks in. The manager has acted in a way that is actually harmful to the project. Not only will the project not be done faster, it may actually be done even later than if no people were added. But that will be dismissed with a variation of the phrase: "but at least we really tried and took every measure - we even assigned more people to the project".

The manager has used the get-out-of-jail-free card and no one will question whether putting more people on the project actually helped or not. And even less so question whether it not only did not help but actually made the situation even worse.

Harmful management is more common than you would think. Once you start seeing it you cannot "unsee". And this is not only related to project management or even the corporate world. I have seen it in a wide variety of sports as well. People who have made a brilliant career out of being able to make decisions generally have a very hard time stepping back and acknowledge that the best decision is to not do anything.

Hence harmful management.

Ensuring code quality for API’s with Request Manager API Pattern

Jesper Reiche — Wed, 15 Jan 2020 20:02:04 +0000

Coding guidelines and code reviews are good ways to ensure code quality and enforce structure. But they also have their limitations. When time is in short demand and the lead developer/architect have no chance of keeping up with the new code being developed, the architecture, structure and thereby quality of the code-base declines. This makes it more prone to errors, which in turn are harder to fix. Furthermore random code structure also makes it hard for new/other developers to comprehend code in new business areas and understand the basics. They do not understand the business area and they do not understand the structure and therefore have no point of reference.

Therefore setting and keeping code structure and guidelines is a very smart move. This Request Manager API Pattern helps with exactly this.

But how do we go about doing this in a way that is sustainable even in a hectic project with a tight deadline(As they all are). Well enter generics and base classes. With clever use of those and a little help from general structuring of code, we can actually make sure that the code fulfills the needs we have – no matter who implements it – and further more make it easier to review and understand for new/other developers.

The story and origination of this pattern

So while I will lay claims to having come up with this pattern for the project in which I used it, I won't claim that I am the only one to come up with something like it or that everything came out of thin air in my head. Part of this pattern is actually taken from another pattern a very good architect and old mentor of mine used to poll a database that acted as "worker queue" for an old airline system. That pattern planted the seed of using generics and abstract methods to "force" a good implementation on the implementers. But the distinct implementation and use of these tools in this pattern came from my head and the problem I was asked to solve.

The project was a rather big one with 10-12 developers when we were most. A combination of permanent employees and freelancers (whih I also was) and also spread out across different countries. What quickly became clear to me was that we had to bring some structure to both the layers of code but also actual implementations, to not drown in code reviews and differnt implementations.

The biggest road block was that everyone was already started on their implementation of the API they had the responsibility for. So asking a lot of people to re-implement and change their code was a very difficult decision to make. But I am very glad that I did and as you can read at the end of this it also payed of in an unexpected and fun way.

I will introduce the Request Manager API Pattern with a simple scenario and try my best to explain it with both text and code. The entire repository is free and hosted on BitBucket so you can clone it and play aroud with it. I will provide the link at the end.

Setting the scene

For this example we have a scenario where we are building a SaaS product. This specific product has several business specific API endpoints that each has a number of operations/methods.

Across all methods/operations we have common things that needs to be validated, primarily to do with authorization in one way or another. These are things that all implementers(developers) need to take into account and therefore a prime candidate for general implementation.

Furthermore, apart from the general validation of general properties, it is required that all operations/methods validate the incoming request with their own business specific validation for the operation/method.

The actual validation of this, we cannot generalize, as it is request-specific. But we can “force” the developer to actually validate an incoming request before working on it. And further more also make the developers implement handling of invalid and potentially “warning” requests.

Creating request and response base classes

To do so, we will make good use of inheritance and generics. First we need base classes for requests and responses.

public abstract class RequestBase
{
        // This is empty for now but could have properties shared among all             
}

/// <summary>
/// Base class for all responses
/// </summary>
public abstract class ResponseBase
{
  /// <summary>
  /// Error code for the specific error
  /// </summary>
  public int ErrorCode { get; set; }
  /// <summary>
  /// Error level
  /// </summary>
  public ErrorLevelType ErrorLevel { get; set; }
  /// <summary>
  /// Error message
  /// </summary>
  public string ErrorMessage { get; set; }
}

In the example included I have actually made two levels of base classes, as I have RequestBase and ResponseBase that are at the top of the hierarchy and then in this case; ExternalRequestBase and ExternalResponseBase just below.

public class ExternalRequestBase : RequestBase
{
    public int ApplicationId { get; set; }
}

public class ExternalResponseBase : ResponseBase
{
  // This just inherits the properties from ResponseBase in this example
}

This opens up the possibility of using the same pattern for internal request/response objects or whatever else you would want to distinguish on. But they could absolutely be omitted.

Having base classes for request/response objects is needed in order to have a generic processor class that can handle processing of all classes that inherits from those.

Creating validation interfaces

I our example we needed to have some general validation of all incoming requests and then some custom validation for each specific operation.

In other words an interface for a generic validator, that needs to take a request-class as input and then return a validation result. Using this generic approach enables us to use the same structure for the general validation as for the specific validation.

/// <summary>
/// Interface for request validators
/// </summary>
/// <typeparam name="Req"></typeparam>
public interface IRequestValidator<Req> where Req : RequestBase
{
  IInputValidationResult Validate(Req request);
}

And the return value interface:

public interface IInputValidationResult
{
    ValidationResultType ValidationResult { get; set; }
}

The ValidationResultType here is just an enum with Success, Warning, Error options.

The IInputValidationResult can be implemented by any specific business area, if extra return values are needed, but we also implement a default that can be used by everyone out of the box.

public class InputValidationResult : IInputValidationResult
{
  public ValidationResultType ValidationResult { get; set; }
}

One of my main issues with explaining the use of this pattern to other developers probably revolves around this area, where you can create your own implementation of the IInputValidationResult and in turn have your own properties available inside the HandleValid, HandleInvalid and HandWarningRequest as we will get to.

But this can be used to pass information from the validator to the specific request manager. We used it if we had heavy validation-logic that ran through structures with information we might as well save for the actual processing inside the request manager.

Creating main generic request manager

So speaking of which, let us look at the IRequestManager interface, as we have all the building blocks in place:

public interface IRequestManager<Req, Resp> where Req : RequestBase where Resp : ResponseBase
{
  Resp ProcessRequest(Req request);
}

This is a very basic interface and all it does is expose a "ProcessRequest" method that takes a "Req" of type RequestBase and returns a "Resp" which is of type ResponseBase. The reason for having an interface here is that we can inject different IRequestManagers and always just call "ProcessRequest".

But this is only part of the magic, the rest happens inside the RequestManagerBase class, which is the biggest part of this pattern:

/// <summary>
/// Base class for all RequestManagers. Handles the flow with processing of requests including uniform way of validation
/// </summary>
/// <typeparam name="Req">The request type</typeparam>
/// <typeparam name="Resp">The response type</typeparam>
/// <typeparam name="InpValRes">The input validation type that is returned from validation of request type</typeparam>
public abstract class RequestManagerBase<Req, Resp, InpValRes> : IRequestManager<Req, Resp> where Req : RequestBase where Resp : ResponseBase where InpValRes : IInputValidationResult
{
  private readonly IRequestValidator<RequestBase> _baseValidator;
  public RequestManagerBase(IRequestValidator<RequestBase> baseRequestValidator)
  {
    _baseValidator = baseRequestValidator;
  }
  public Resp ProcessRequest(Req request)
  {
    Resp result = null;

    // Validate BaseRequest
    IInputValidationResult baseValidationResult = _baseValidator.Validate(request);

    // If BaseRequest is valid - then validate specific request
    if (baseValidationResult.ValidationResult == ValidationResultType.Success)
    {
      // GetInputRequestValidator from implementing class
      IRequestValidator<Req> validator = GetInputRequestValidator(request);

      // Validate with validator from above
      InpValRes validationResult = (InpValRes)validator.Validate(request);

      // Call corresponding method based on result
      // With this pattern we have this code only once!
      switch (validationResult.ValidationResult)
      {
        case ValidationResultType.Success:
          result = HandleValidRequest(request, validationResult);
          break;
        case ValidationResultType.Warning:
          result = HandleWarningRequest(request, validationResult);
          break;
        case ValidationResultType.Error:
          result = HandleInvalidRequest(request, validationResult);
          break;
        default:
          throw new ArgumentOutOfRangeException("Unknown ValidationResult");
      }
    }
    else
    {
      // Should we handle it here in the base manager or send it back to the implementing class as HandleInvalidRequest ?
      // I think we should handle it here
      // TODO - Decorate response from BaseRequestValidator with enough information to set these properties below with valid
      // information across all interfaces
      result = HandleInvalidBaseRequest(baseValidationResult);
    }

    // TODO Log outgoing request - perhaps? 

    return result;
  }

  private Resp HandleInvalidBaseRequest(IInputValidationResult failedBaseValidation)
  {
    // TODO we may have an issue if the request class suddenly needs parameters in the constructor
    // In that case we may need to "push" the instantiation to the implementer class
    Resp result = (Resp)Activator.CreateInstance(typeof(Resp));
    result.ErrorCode = 1;
    result.ErrorMessage = "Validation of request failed";
    result.ErrorLevel = ErrorLevelType.Error;
    return result;
  }

  protected abstract IRequestValidator<Req> GetInputRequestValidator(Req request);

  protected abstract Resp HandleInvalidRequest(Req request, InpValRes failedValidation);

  protected abstract Resp HandleWarningRequest(Req request, InpValRes warningValidation);

  protected abstract Resp HandleValidRequest(Req request, InpValRes successfulValidation);
}

The RequestManagerBase class is abstract as it is not supposed to be instantiated, but only used as base class for actual implementations. It inherits the IRequestManager interface which we created earlier. In the constructor it takes an IRequestValidator which is used as the baseValidator, meaning it will be used to validate the common properties across all requests.

From the IRequestManager interface it gets its only public method, being ProcessRequest. This is where the main logic of this request manager API pattern resides. The logic is explained in comments inside the code itself, but basically it first validates the properties from the BaseRequest and if valid then it asks the implementing class through the abstract method GetInputRequestValidator for the validator that should be used to validate the incoming request. Once this is returned it will then call this validator and ask it to validate the request. Depending on what the validator returns it will then call either of the abstract methods HandleValidRequest, HandleWarningRequest or HandleInvalidRequest.

Finally we have the four abstract methods described in description above. By way of being abstract we force the implementing class to implement these methods. We will look at an example of what this looks like once you create a class that inherits from the RequestManagerBase class. But this concludes the RequestManagerBase class.

Implementing specific request manager

I won't go into detail with the specific implementations of the request and response classes as that it too simple to show. But you can of course clone the repository (link is at the end of the post) and see for yourself.

But I will just show the simple implementation of a IRequestValidator that is being used in the unit tests of the project.

public class MockRequestValidator : IRequestValidator<MockExternalRequest>
    {
        public IInputValidationResult Validate(MockExternalRequest request)
        {
            if (request != null)
            {
                if (request.MyInputString.Equals("Success"))
                    return new InputValidationResult() { ValidationResult = ValidationResultType.Success };
                else if (request.MyInputString.Equals("Warning"))
                    return new InputValidationResult { ValidationResult = ValidationResultType.Warning };
                else
                    return new InputValidationResult() { ValidationResult = ValidationResultType.Error };
            }
            else
                return new InputValidationResult() { ValidationResult = ValidationResultType.Error };
        }
    }

As you can see this implementation is very basic and oly to show the functionality. What is important is that depending on InputValidationResult either HandleValidRequest, HandleInvalidRequest or HandleWarningRequest will be called on the implementing request manager.

So let us look at the request manager from the unit testing:

public class MockRequestManager : RequestManagerBase<MockExternalRequest, MockExternalResponse, InputValidationResult>
    {
        private IRequestValidator<MockExternalRequest> _requestValidator;

        public MockRequestManager(IRequestValidator<RequestBase> baseRequestValidator, IRequestValidator<MockExternalRequest> requestValidator) : base(baseRequestValidator)
        {
            _requestValidator = requestValidator;
        }

        protected override IRequestValidator<MockExternalRequest> GetInputRequestValidator(MockExternalRequest request)
        {
            return _requestValidator;
        }

        protected override MockExternalResponse HandleInvalidRequest(MockExternalRequest request, InputValidationResult failedValidation)
        {
            MockExternalResponse response = new MockExternalResponse
            {
                ErrorCode = 2,
                ErrorMessage = "Invalid MockExternalRequest",
                ErrorLevel = ErrorLevelType.Error
            };

            return response;
        }

        protected override MockExternalResponse HandleValidRequest(MockExternalRequest request, InputValidationResult successfulValidation)
        {
            MockExternalResponse response = new MockExternalResponse
            {
                ErrorCode = 0,
                ErrorLevel = ErrorLevelType.None,
                MyOutputString = "Success"
            };

            return response;
        }

        protected override MockExternalResponse HandleWarningRequest(MockExternalRequest request, InputValidationResult warningValidation)
        {
            MockExternalResponse response = new MockExternalResponse
            {
                ErrorCode = 3,
                ErrorLevel = ErrorLevelType.Warning,
                MyOutputString = "Warning"
            };

            return response;
        }
    }

As you can see in the first line, we inherit from RequestManagerBase and in doing so also specifies which actual request and response classes to be used and also which validator. By doing so all the protected override methods will automatically be generated and are ready to be filled with logic required for handling the specific requests.

This saves a lot of time for developers and it avoids everyone having to make branching logic around validation in their own API's we have everything centralized and can hence ensure that validation is done and developers can fill the methods with logic depending on whether the request was valid, invalid or warning.

Praise from security

The full power of using this API request manager pattern can of course be a little hard to see with such simple examples. But I can finish of with a little anecdote from the project where I actually introduced this pattern. I think it initially took a little time for all to see the advantages of this pattern. It does require some extra work around creation of classes - but not too much in my opinion.

But one of the real eye openings for the other developers and a big victory for my part, was when we had to have our project security approved. Everyone feared this as it usually took very long and we were under a lot of time pressure. As part of this security assesment there was a dreaded code review to be performed which almost always caused a lot of hassle and additional changes.

The funny thing happened that, after I walked the security approver through the code base, which I had taken responsibility for structuring apart from also implementing the API request manager pattern, he actually ended up not just approving our solution but actually praising this exact pattern highly. He could clearly see the advantages of ensuring that all requests were validated by "forcing" developers to think about all the scenarios and also making it a lot easier for code reviews to spot potential loop-holes.

And that is how the department I was atteached as a freelancer ended up being applauded and given praise from the security department.

It essentially boils down to this:

Easy to do the right thing - hard to do the wrong thing

We make it easy for developers to do the right thing and hard to do the wrong thing. And in my opinion we do so without taking initiative and freedom away from the developer.

Potential improvements

This pattern is of course by no means perfect and can probably be enhanced in a number of ways. I have just listed a few things that have come to mind while watching this pattern in use with a medium project with 5-10 developers attached.

RequestManagerBase could take a special IRequest*Base*Validator to further ensure that the BaseRequest is validated by the right validator. With current implementation it would be possible to use a "wrong" validator for BaseRequest - but it would be easy to spot on code review at least.
Originally I had planned for HandleValidRequest method to only take the request as input, HandleInvalidRequest to only take validation result and HandleWarningRequest was the only one who took both request and validation result as parameters. This looked a lot cleaner, but over time it just hindered the scenarios where you needed for instance a lot of work in the validator which could be re-used in the HandleValidRequest method - then suddenly we had no way of passing that. Hence I ended up giving all methods both parameters.

This concludes my introduction to the Request Manager API Pattern. I hope you find it understandable and potentialy useful. I have included link to the public repository on BitBucket, so have fun with it.

You can clone use and play around with the code from here: https://bitbucket.org/jmreiche/requestmanager/src/master/

Feel free to reach out to me with questions or suggestions for improvements.

Automated two app's from one React Native Expo repository

Jesper Reiche — Mon, 30 Dec 2019 15:17:14 +0000

In the following I will show how it is possible to have two app's from one React Native Expo repository with a little help from Python and a bit of Powershell.

While building an app for a startup I am currently working on, I ran into the issue of having to support two different apps from the same repository. In our case it is a matter of having one app for "professionals" and one app for "patients". A lot of the screens are exactly the same for either app and in order to foster re-use and not having to create a whole new repo as a copy of the other, I wanted to be able to start and build two individual apps from the same repository.

This proved to be quite difficult.

I may have been bad at searching or explaining when I asked for help on reddit etc. But the bottom line was that I was unable to find someone who could tell my how it could be done. So with the uneasy feeling of actually having to do something myself, I started experimenting to see whether it could be done. (Spoiler alert: It could)

First part of it has to do with app.json. My theory which ended up being correct, was that by changing "name" and "slug" then it was possible to actually have two different apps build and released, first of all to Expo. The following two app.json files will create two independent apps in Expo, even though they are built from the exact same code base:

App 1

{
  "name": "App 1",
  "expo": {
    "sdkVersion": "33.0.0",
    "name": "App 1",
    "slug": "app1",
    "version": "0.0.22",
    "description": "",
    "privacy": "unlisted",
    "orientation": "portrait",
    "icon": "./src/icons/App1_iOS_1024.png",
    "ios": {
      "supportsTablet": true,
      "bundleIdentifier": "me.identifier.app1"
    }
  }
}

App 2

{
  "name": "App 2",
  "expo": {
    "sdkVersion": "33.0.0",
    "name": "App 2",
    "slug": "app2",
    "version": "0.0.22",
    "description": "",
    "privacy": "unlisted",
    "orientation": "portrait",
    "icon": "./src/icons/App2_iOS_1024.png",
    "ios": {
      "supportsTablet": true,
      "bundleIdentifier": "me.identifier.app2"
    }
  }
}

Since I have included the "ios" settings I might as well give the spoiler; that not only does this work when publishing to Expo, it is possible to build for iOS and have individual separate apps in the AppStore as well.

Separation in code

Now that I proved that we can build two individual apps, then we need to separate the entry/startpage of the apps. At least that was the scenario I needed. I had to present one type of user with a specific startpage for their needs and another for the other type. This stretched for both on-boarding and later running of the two "separate" apps. Each type needed their distinct description of how to use the app etc.

In my React Native app I am using react-navigation and needed a way to present different startup screen depending on whether I started the "patient" app or the "professional" app.

This Stack Navigator works in a very simple way in that the first screen configured will be used as the startup screen. Therefore the simple way to control whether the user starts in the flow of the "patient" or the "professional" is to have either screen appear as the first one. So commenting out "patient" screen will give "professional" app and uncommenting will give "patient" app.

I have since speculated whether this was the best way to solve it as there, as far as I remember, also is an "initialStartupScreen" setting or something like it. But this worked and will be 100% handled in code (explained later), so I am not that worried for now.

export default createAppContainer(
  createStackNavigator(
    {
//      UserTypeLoginScreen: { screen: UserTypeLoginScreen },
      Main: {
        screen: mainBottomNavigation,
        navigationOptions: ({ navigation }) => ({
          header: null
        })
      },

This could be handled manually, but would be annoying and error prone to run through a number of manual steps and comment/uncomment different lines of code each time the other type of app needed to be build or run. Hence I build a small python script to take care of the AppNavigation.js file and make sure that depending on the app being build/run it will comment/uncomment the line that controls which screen to present at startup. It also comments/uncomments the import of that screen as well.

import shutil
import os
import sys
# Get type of app - which one are we trying to build/run
appType = sys.argv[1]

# Remove .expo as it can cause problems and is easily re-created
shutil.rmtree('./.expo')

# Create temporary file
file = "./src/helpers/AppNavigation.js"
shutil.move(file, file+"~")

# Not the prettiest but it works
# Find lines in AppNavigation.js that mentions "UserTypeLoginScreen"
# Comment them if they are not commented and should be, given the "appType"
# so we do not double-comment if the same type is started twice
destination = open(file, "w")
source = open(file+"~", "r")
for line in source:
    if "UserTypeLoginScreen" in line:
        if "patient" in appType:
            line = line.replace("//", "")
        else:
            if not "//" in line:
                line = "//" + line
    destination.write(line)
source.close()

destination.close()

os.remove(file+"~")

So just to quickly walk you through it. First of all it looks for an argument to decide whether we are currently trying to start the "patient" or "professional" app. I will later show how this script is called.

Then it removes the .expo folder as it sometimes gave issues when switching from running one app to the other. And as it only takes a few seconds to re-create I just remove it here and have it be re-created.

Then it creates a temporary file from the AppNavigation.js file and comments/un-comments the correct lines and finally removes the temporary file.

Putting it all together

Now finally I needed to create some small scripts to link it all together and be able to start or build either app. I decided to create them as Powershell scripts and decided on 4 scripts for now:

Patient run
Patient build iOS
Professional run
Professional build iOS

Here is an example of "Patient run" script:

copy-item -path ./src/app_type/patient/app.json -destination .

python ./app_prepare.py patient

npm start

First it copies the app.json file to the same directory as this Powershell script is located as it is located along side the app.json file that is being used to run the app.

Then it calls the python script that we created above. It call it with "patient" as parameter. This is then used as "appType" in the python script.

And finally it calls npm start. This is defined to run "expo start -c" in the package.json file.

If we then look at the Powershell script for building iOS .ipa package:

copy-item -path ./src/app_type/patient/app.json -destination .

python ./app_prepare.py patient

expo build:ios

The first two lines are exactly the same, the only difference is that in the last line it calls "expo build:ios" instead.

I will not include them here, but of course for the "professional" app it just copies the app.json file from the "professional" directory where it is stored and calls the python script with "professional" instead.

This then enables me to run or build the app I want by just calling the corresponding Powershell script. And voila - 2 apps from the same repository.

So finally just a few pros and cons as this of course may not be the right solution for anyone:

Pros

Re-use of components

Cons

Bigger release packages as they can carry unsused screens

Feel free to get in touch if you have any questions to the above or if you have any ideas to optimize it even further.

Is agile killing time for reflection

Jesper Reiche — Tue, 26 Nov 2019 20:26:11 +0000

"Agile" is all over every workplace from municipalities over manufactoring to of course software development. Everyone has caught the "bug" and everyone talks about agile as the best thing since sliced bread. Agile is of course a framework that somewhat encourages you to take the things you like and get value from and leave out the rest. The movement in of itself is not trying to push some magic pill on you - there are just a lot of people who did not get that memo, and talks with the same objectivity about agile as your newly-turned vegan friend talks about your 500 gram steak.

But this piece is actually not trying to bash agile (or vegans for that matter) it is more a stream of thoughts and reflections on what agile has been interpreted to mean and where the implementation has perhaps gone a bit too far.

Defining "Agile"

As almost everyone has their own interpretation of agile ways of working I will start by describing what I lay in the meaning of agile.

Agile seen from my perspective, is about quick iterations and quick learnings. It is about shortening the feedback loop and course correcting as quickly as possible along the way. It acts as the anti-thesis of "waterfall" or is often put up against "waterfall". Waterfall is painted as the process of making sure you lock yourselves into a room and cut all communication with the outside world - especially your customers - because by doing waterfall you have obliged to have tunnel vision and only deliver what was originally planned, and by no means change anything along the way. Change along the way is patented by "Agile"! (Irony may occur)

Reflection - anyone?

Seeing Agile in this light of quick iterations and quick learnings, my concern becomes visible. Because while I do really like the aspect of quick iterations and feedback i feel like it has somewhat killed off thoughtful time for reflection. It is as though thinking has been totally replaced by doing. As if people has totally lost the value of actual thoughtful reflection on a problem.

I get it that you cannot have a team full of developers sitting for hours reflecting over their work, but that does not mean that nobody should spent time reflecting over whether the project is moving in the right direction. Whether the ideas that are being quickly iterated makes sense at all.

Quick iterations are good if you have an idea about what you want to do, but you are unsure about the actual implementation. They are also very good if you are trying to keep up with a competitor that is already ahead. But in my opnion it is not always the best way forward for all situation.

Faster horse

As the old quote by Henry Ford goes:

If I had asked people what they wanted, they would have said faster horses.

If you attack a problem head on and iterate quickly you may end up doing the faster horse instead of taking a step back and seeing whether horses is really the future or not. This approah may be right, depending on what you want to achieve.

In my experience "Agile" has been a way to shift analyses work to development - meaning that it is better to have the developers try something out instead of spending the effort of analysing the problem properly. This can of course cut both ways. Analyses paralysis is a real thing and should not be ignored. In some organisations the bar had perhaps swung to hard in the direction of analysis and you ended up with these multi-year projects only inquiring actual feedback at the big-bang launch.

This is of course not what I am advocating getting back to. I fully acknowledge the value of agile ways of working and the iterative approach. I have just met a lot of overemphasis on the efficiency of this applied across all functions in an organisation. It has absolutely earned its place and merit, but it should not be over-done and kill time for reflection for the functions that actually require that.

Even though our society does its best to distract us in any way possible I actually still believe that there is merit and value in the good ol' time for reflection.

Keep header with nested createBottomNavigator

Jesper Reiche — Wed, 13 Nov 2019 19:25:33 +0000

Every once in a while you into these strange annoying issues when working with React Native. For a lot of things it is really good and you really feel and are productive. But then these little annoyances just sneak up on you out of the blue.

This issue with createBottomNavigator and the header was one of these for me. I have spent waaay to long on this issue. Tried to solve it many times but then just ended up throwing in the towel time and time again. I have searched all over the Internet and only found people with somewhat the same issue - but never someone who actually had a solution for it. But today I finally managed to solve it and thought I might as well spent the time waiting for the build to iOS to finish, with writing up this little piece on my findings.

The issue:

I have a stackNavigator that has all my screen defined and acts as my "main" navigator. But I wanted to have my first screen be presented with a bottom tab navigator. Implementing this was not that hard as I just created a "createBottomTabNavigator" with the two screen I wanted to show and then defined small icons and texts for these small tabs. This worked out of the box. But what did not work was that I suddenly had my header-bar in the top of the screen not showing any text. Usually this is controlled via the "title" property, but this was totally ignored and I had no text at all in the header.

I tried setting the title all sorts of places in my "createBottomNavigator", but nothing seemed to work. I tried changing the formatting if the text for some reason was written with the same color as the background. But still to no avail.

Trials:

Third or fourth time I want back to this bug I found a way to set the header - but then only to a static value with no way to change it depending on the tab selected. I did this by setting the header in my main stackNavigator like this:

export default createAppContainer(
  createStackNavigator(
    {
      UserTypeLoginScreen: { screen: UserTypeLoginScreen },
      Main: { screen: mainBottomNavigation,
        navigationOptions: ({ navigation }) => ({
          title: "Title here"
        })},
      Login: { screen: LoginScreen },

I also then tried to use the navigation state, which did not work as apparently navigation state is not present in "createBottomTabNavigator", so this would not even run:

export default createAppContainer(
  createStackNavigator(
    {
      UserTypeLoginScreen: { screen: UserTypeLoginScreen },
      Main: { screen: mainBottomNavigation,
        navigationOptions: ({ navigation }) => ({
          title: `${navigation.state.params.title}`
        })},
      Login: { screen: LoginScreen },

Reading a bit more on React Navigation I found a site where someone suggested creating "StackNavigators" as these had "navigation state". My initial trial with this however only got me in a mess where I ended up having 2 headers. My originalle formatted one but without any text as title and then one beneath it without formatting but now with text.

Solution:

I actually rolled back my changes as I abandoned them since they only made my problems worse. Researching a bit more I however found a suggestion on setting the header to null in my "main" stackNavigator. I though this might enable the underlying bottomTabNavigator to overwrite and present the header text. But setting it to null removed the header all together.

Remembering that I had a way to create 2 headers I now formed a solution in my head. If I could "hide" the normal header from the "main" stackNavigator once going into the bottomTabNavigator, then I could perhaps style the header to look exactly the same and then end up with a solution to my problem. And low and behold. After re-creating my discarded code on "stackNavigators" inside the "bottomTabNavigator" and styling the headers to look the same as my "main" "stackNavigator" then it finally worked!

Below is the code first for my "createBottomTabNavigator" with its independent "createStackNavigator" for each page to hold their own navigation state and hence header. And after that is the code from my "main" "createStackNavigator" where I hide the header so that each of the nested "stackNavigators" can show their own equally formatted headers. And then everything works!

const ProgramListStack = createStackNavigator(
  {
    ProgramListScreen: {
      screen: ProgramListScreen
    }
  },
  {
    defaultNavigationOptions: {
      headerStyle: {
        backgroundColor: "#4EC3B2"
      },
      headerTintColor: "#fff",
    }
  }
);

const PatientListStack = createStackNavigator(
  {
    PatientListScreen: {
      screen: PatientListScreen
    }
  },
  {
    defaultNavigationOptions: {
      headerStyle: {
        backgroundColor: "#4EC3B2"
      },
      headerTintColor: "#fff",
    }
  }
);

export const mainBottomNavigation = createBottomTabNavigator(
  {
    ProgramListScreen: {
      screen: ProgramListStack,
      navigationOptions: ({ navigation }) => ({
        title: `Programs`
      })
    },
    PatientListScreen: {
      screen: PatientListStack,
      navigationOptions: ({ navigation }) => ({
        title: `Patients`
      })
    }
  },
  {
    defaultNavigationOptions: ({ navigation }) => ({
      // Removed for brevity
  }
);

export default createAppContainer(
  createStackNavigator(
    {
      Main: {
        screen: mainBottomNavigation,
        navigationOptions: ({ navigation }) => ({
          // Set to null to hide header and have the bottomNavigator display its own  
          header: null
        })
      },
      Login: { screen: LoginScreen },
      // Removed for brevity
    },
    {
      defaultNavigationOptions: {
        headerStyle: {
          backgroundColor: "#4EC3B2"
        },
        headerTintColor: "#fff",
      }
    }
  )
);

Software Events vs. Real Life Events

Jesper Reiche — Mon, 05 Aug 2019 18:15:47 +0000

Ignoring the fact that the word “event” has a special meaning in software development I will proceed to use it in its broader definition as “something that happens”.

Building software to solve a “business problem” can be daunting enough and leave all sorts of cracks in the relationship between IT and Business. But if those cracks were not visible or wide enough from the initial design phase, then start to introduce changes to business requirements and watch the Grand Canyon form between the departments.

Why is change such a hard thing to handle? And especially within software?

One of the big issues is in the title; Software events vs. real life events. Software events are binary of nature. Everything needs to be specified or left out by deliberate choice. Software is built to handle and encompass real life events of the business area or problem it is built to solve. A lot of analysis, decisions and compromises will have gone before the final design of the software. These can be more or less correct (there is no such thing as the correct solution) but nevertheless they are written in code and hence, as mentioned earlier, binary.

They are built on assumptions and analysis of how the world works in relation to the problem the software is built to solve. These assumptions can be more or less correct and even their correctness evolves over time, as the reality on which they are based, changes.

Real life can operate inside binary rules. It can even seem like real life follows these rules. Right up until the point, where they do not. Then they just change.

Small changes are usually okay. You can add a little more information here and there. Perhaps even remove some. The big clash with software happens when the change in real life goes against a “rule” on which the software is built.

When designing software you are always making compromises and opt outs in order to reach the final design. These compromises and opt outs are based on your knowledge about the reality in which your solutions operates. Some of these will be based on “hard” facts and other will be based on “history” or “experience” and lastly some will also be based on assumptions.

Assumptions could sound like the “worst” kind as it seems like the least researched or analysed. But assumptions are not bad per se. If you know that you are basing a decision on an assumption, then you also know that it may change in the future. This hopefully leads to an implementation based on that assumption which is designed for change.

History, experienced, and hard facts are actually potentially worse, as they are things that will be designed as “truths”. Meaning that the compromises surrounding those will probably be more rigid and not built for potential change in the future. This can give serious issues once they do actually change.

The problem with real life events vs. software events is that real life is always fluid. What is reality today will change tomorrow. Software design needs to take this into account and not only think about it but also design for it. D. L. Parnas even proposed having “potential change” as the main driver behind software design. This of course needs to be countered with another important point from a historical figure:

“Premature optimization is the root of all evil” – Donald Knuth

But it is an important point that the ability to change should have high priority in software design. We cannot predict the future, but we can try to make sure we are ready to adapt. The real test of software architects and developers is how good they are at hitting a happy medium of changeability and “hard truths”. Sometimes we architects and developers can be a little too “binary” in our thinking and force limitations on end-users, that seen from the business or end-users perspective does not give much value. We can be a little “too clever” at times.

Of course it is never easy to strike the “happy medium”. We will have times where we end up in the “too rigid” and we will have times where we end up “too changeable”(costing too much to develop). But hopefully with experience and enough humbleness and hindsight to acknowledge own mistakes we can get better and ensure that we improve over time.

Chain of responsibility design pattern for response handling

Jesper Reiche — Tue, 09 Apr 2019 17:40:50 +0000

As the name suggests Chain of responsibility pattern creates/describes a chain of responsibility. This chain can be static/hardcoded or it can be dynamic and loaded or changed at run-time. It is in many ways a more object oriented cleaner way to implement if/else or switch statement. With the additional benefit of it being able to be dynamically loaded.

The easy way to describe it would be to consider a scenario of first day in high school. Everyone sits in a circle and the teacher sends a bag with nametags around. The task is easy everyone takes the nametag with their name on it and then passes the bag along again. In this scenario, your responsibility is your nametag, which you grab, and then you pass along to the next link in the chain.

It could also be a scenario where your teacher asks you to put your names on a list of courses that you wanted to attend. She passes along a few lists with courses and you put your name on the lists (courses) you want, and then, passes the lists along to the next in line.

The dynamic loading equivalent of this scenario would be your teacher asking all who wants to attend philosophy raise their hand and then pass the list to the first in line, which should enter their name and then pass it along to the next one with their hand raised.

It is as simple as that.

However, for this to make sense in a programming scenario and add clarity and cleanliness instead of overhead and complexity, we need to make good use of generics. I want to explain how we can use it as a clean way of handling responses from for instance API calls.

A prerequisite for doing this in a clean way will be base classes for request/response classes. This is required in order to implement a ResponseChainHandlerBase-class that can be used across all requests/responses. Then we can centralize the generic part of the handling and only create concrete implementations to handle the specific cases.

First, we will create an interface for response-chain handlers.

This interface will, as shown, use our RequestBase and ResponseBase as generics and have two methods. CallAndHandleResponse that takes a request, the request-funtion and a retry-function as parameters. More on these last two func-parameters later. Then it has SetNextHandler medthod, which as the name suggests, sets the next handler in the chain.

With this in place, we can implement the abstract base class that all response-chain handlers has to inherit from. The implementation of the ResponseChainHandlerBase will be as shown below, with further explanation underneath it.

From top of the class, we have a request function (CallFunc), which as the name suggests, is the function that will be called with the request and return the response. Then we have the SetNextHandler that sets the next handler in the chain.

Then there is the public CallAndHandleResponse – which is the “public” entry point of calling our handlers. As you can see from the implementation it calls the function from the method body, with the request (also from method body) and then it calls the function via CallFunc passing along the retryFunc from method body. The reason for having this is that we want the response handlers to be able to re-try a call. But, the response-handler should not be allowed to decide whether re-try is allowed or not. This is the responsibility of the class that uses the response chain. Therefore we have this retryAllowedFunc passed along as parameter.

If response is not null then it calls the HandleResponse on the implementing class and if there are more classes after this, then it will call the next in line afterwards.

Finally there is the abstract HandleResponse method that the implementing class will have to implement and enrich.

With our interface and base class in order, we will turn our attention to the implementation of a concrete response handler.

The specific use case where this came into play was a system doing calculations. This system sat on top of a stateless calculator and managed the flow. Depending on input for the calculator, there were scenarios, where the calculator would return error if calculation was done with insurance included. In these cases the end-user would then be required to un-select insurance – if that was allowed for the product – and then calculate again to get a valid quote.

Therefore, there was a need to build some logic that could handle this retry automatically. As there are many other “add-on” services related to the calculations, I also knew that this would be a good candidate for a general pattern for how response-handling should be done.

Hence, the chain of responsibility pattern looked like a good fit. We could have a number of handlers that each had their own narrow responsibility and we could then apply them as needed either by themselves or in combination.

So now, with a bit of context, let us look at a concrete implementation of the InsuranceResponseChainHandler:

The logic of this handler is very simple. It looks at the ErrorCode returned and if it is one of the ErrorCodes that are insurance-related, then it will try to de-select insurance. However, as some products does not allow this, we call the retryAllowedFunc passed into the method. If allowed we call the CallFunc method on the ResponseChainHandlerBase. With this, our InsuranceResponseChainHandler is complete. It can re-try if ErrorCode returned is one of the ones specific to insurance. This list of ErrorCodes can of course be fetched from config, set-up dynamically depending on request-type etc. Nevertheless, for our use this private readonly HashSet was sufficient.

Now let us look at how this chain handler pattern is actually used.

We will start with the Factory that creates the chain. This could be done with other dependency-injection patterns, but for this scenario, we use factories.

The most interesting part for explaining the chain handler pattern is the part to do with _responseChainHandler. In this factory we instantiate the InsuranceResponseChainHandler class that we created above. However, reading through the comments below I also show how we could easily add more handlers after this one. As you will probably also have noticed we could dynamically have built this chain depending on the type of request or whatever else might determine the correct “chain”.

Having now injected the response chain handler into our “Manager”, which shortly described for this piece, is the class that manages the flow around calculations, we can call it with just a single line of code:

The parameters supplied to the CallAndHandleResponse are in order of appearance:

The internalRequest – in this case a “calculation-request”
(_calculateInternalEngine.CallCalculateInternal) The “business engine” class that actually calls the “calculator”. (This is our CallFunc from earlier)
(ValidInternalRetryRequest) A method residing in the “Manager” that decides whether retry is allowed. In our scenario, it will validate whether the product we are calculating upon actually allows IncludeInsurance to be set to false and then returns true or false. (This is our retryAllowedFunc from earlier)

With this, the response chain handler pattern is complete. We have created a dynamic and extendable way of handling responses from either internal API’s, external API’s or a way for handling responses from internal validators or classes – the use-cases are vast.

Happy coding and “architecturing“

Three ways to solve a problem

Jesper Reiche — Wed, 20 Mar 2019 08:54:41 +0000

In software development and perhaps more broadly, in creative endeavors in general, rarely does one solve a problem that has only one possible solution. Mostly there are “more ways than one to skin a cat”.

There is the “wrong way” which causes more problems than it really solves. This is usually where the newcomer goes wrong. Lacking experience with both the environment and tools, the inexperienced is fighting an uphill battle understanding both the problem and the consequences of a proposed solution. The fun part is that in an environment with good testing and feedback, this can actually be one of the quickest way to get problems solved.

Fail fast.

Then there is the striving for the unobtainable “perfect solution” to a problem. This is usually the “lightly trained” or “slightly experienced” Achilles heel. Having spent enough time in the trade to see that there are “bad” solutions, the pursuit now goes for the perfect “good” solution. This leads to analysis-paralysis and in software development usually over engineering. This will usually be the slowest and the most expensive solution/way to solve a problem.

The third way is the “least bad” solution. This is where experience makes it mark. Having enough experience to identify “really bad” solutions, but also the confidence and full overview of the problem, to not chase a non-obtainable perfect solution.

The sweet spot is usually the “least wrong” solution.

The one which solves the problem in a good way while not closing too many doors, but on the other hand not trying to keep all thinkable doors open (over engineering). This is of course in no way an easy feat. Trying to find the “least wrong” solution can be as exposed for over engineering and analysis paralysis as the “perfect solution”.

Even more annoying is the fact that you will probably only really be able to see a clear difference over time. But hopefully over time and with experience you will also be able to recognize where you created a solution that fits either of the three above scenarios and hence progress towards “less wrong” solutions.

Leading with empathy

Jesper Reiche — Tue, 26 Feb 2019 15:21:16 +0000

Good leadership/management takes empathy and compassion for the people you lead. You can lead without it, it may even be today’s norm – but you cannot be a good leader or manager without it.

If you lead without empathy or compassion you will end up seeing people as resources that can just be replaced and manipulated in any way you like and then bluntly discarded. You may even end up having an entire department dedicated to the extraction of resources from humans, and in some evil post-apocalyptic World, you may even end up calling it Human Resource Management – oh wait….

Leading with empathy has sort of become the anti-norm. Leaders, companies and perhaps even society has moved in a direction where leaders and managers are expected to keep a “professional distance” from the people they lead. What has pushed on in this direction is hard to say with certainty. But I can see one potential explanation.

Who benefits?

As any good investigator you can always revert to the good old; “who benefits?” With employees and managers distancing from each other, who benefits? From an employee’s standpoint I do not see much gain. Even if you do not like your manager it would be a gain if your manager liked you and felt compassion and empathy. Being fired by someone who cares for you must, all things being equal, be better than being fired by someone who does not. So from an employee’s/subordinates point of view I cannot see many benefits.

Where I can see “benefits” is from the manager/leaders point of view. From their point it is easier to manage, criticize and fire someone if they are distant from them. Compare kicking someone out of a chatroom to asking someone in person to leave the room. The first is easy, the latter a lot harder.

But this is a feature and not a bug.

It should be hard and uncomfortable to move people around, criticize them and not the least fire them. They are real people. They do have families. They do have people that care for them and they do have people that rely on them.
This does not mean that you should never fire someone or that your company has suddenly turned into a charity. Firing people is part of your job as a manager. It can be the right solution for both the company and the employee. But it should be hard. Because it does have consequences.

Skin in the game

If you as a manager does not have skin in the game and does not feel the uncomfortable sting of having to confront someone with the fact that you have to let them go, then it becomes too inconsequential and hence too easy.

You are paid to deliver results, make sure that things go smoothly and that your subordinates are in the best position to do their job efficiently. If you do not have skin in the game, care for your subordinates and feel compassion, then it becomes mechanical and false. You can go a long way with force and pressure of rank, but it will never be optimal and it should never be the default.

I have seen a lot of managers lead with “brute force”. It almost seems like the default. But it is too “easy” and it is too short-sighted. No one have ever delivered their best work with a gun to their head. Perhaps their most important work – in keeping them alive – but never their best work.

People are social creatures. We thrive with relations and will go a long way for people we care for and people who care for us. Managers that try to keep a “professional distance” rob themselves, the employee and the company of great work.

Great work happens when the employee finds meaning in the task in relation to the overall mission. Plus the manager and the company cares for the employee, so he/she feels valuable and feels trust. Being seen as a simple (Human) “resource” does none of this.

Of course caring for your employees and developing strong relationships with them also makes it harder to make difficult decisions. But it should be difficult. You are dealing with another human being, a colleague and hopefully a friend. They should not necessarily be your best friends, but you should feel empathy towards them and care for them.

If they do not perform then you should try to make it work, but if it does not, then you should fire them, both for your and their sake. Help them find other work if you can – you care for them – remember?

By forming better and closer relationships with their subordinates a lot of manager are required to make their job a little harder. But in my book that is a necessary evil that makes both themselves, their employees and the company better of – so what are we waiting for?

The push for new and shiny solutions to old known problems

Jesper Reiche — Tue, 29 Jan 2019 07:15:16 +0000

As a species we have always been attracted to novelty. This has probably served us well for things with short feedback loops as we quickly learnt whether the “new and shiny” thing actually helped us or harmed us.

In software development the JavaScript community has been notorious for “new and shiny” frameworks. This has - for good reason - been mocked over and over again. But a little more concerning is the fact that I see a lot of the same worrying tendencies creeping in(or surfacing) in larger organisations and software projects outside the JavaScript domain. Sectors or organisations where caution, conservatism and stability is the norm. But for some reason a perhaps mistaken attempt at “conservatism” ends up being anything but.

I have discussed this with many other people or observed the pattern in discussions and talks with others which have only confirmed and backed the picture I see.

My main concern is that I see many projects and organisations make decisions on technology based on what is the latest trend from the top 5-10 technology companies. There seems to be a tendency towards “new and shiny” solutions to problems where objectively a much more simple solution would have been better. It seems like many architects and developers look to solutions from Facebook, Amazon and Google and come to the conclusion that; “if only we use the same technology, structure and architecture, then we are home safe. If it works for Facebook, Amazon and Google then it will surely be the right scale-able approach for us as well.”

This “lazy” way of approaching technology and software can not only end up causing many problems initially but also end up with a worse result than sticking to basics. This is of course not to say that mimicking giants or looking at established and proven solutions to find inspiration is never the right approach but many people seem to miss a vital step.

Which problem are we trying to solve

The key to finding good and relevant inspiration from others is to first of all find out what problem are you trying to solve. This can be top-level, looking at the entire solution, or it can be zoomed in on any smaller specific part of the solution.

“If you do not know where you are going, any road will take you there”

Once you have figured out which problem (or problems) your solution is trying to solve, then you can go looking at whether this problem has been solved before (which it probably has).

But, there is a very important point to have in mind when looking at others solutions. And this is where I think a lot of people and organizations put too little emphasis. First you had to figure out what problems you were trying to solve. I do not think many people go wrong here. But when looking at others solutions you also need to do some complex discovery or thinking. Because in order for their solutions to be a good fit for yours, you need to know or figure out; what was the problem that they were trying to solve when they arrived at their solution.

I have encountered people recommending NoSQL databases because they are so smart, that you can just put any data into them without it being structured. And that is smart. If you are working with unstructured data. If you are working with perfectly fine structured data then all you will gain is probably complexity and performance issues. Plus the fact that in the cases I heard them proposed none of the developers on the teams had any experience with the technology. So you can add that cost and complexity on top of it. That just does not add up.

As said earlier the most worrying part about this is that the people recommending these solutions are not necessarily young eager hackers, but actually experienced architects working in large organizations, where I would have expected a lot more push-back and conservatism.

Why is this happening?

So the big question is of course; why is this happening?

We could call it incompetence and end it with that. But that would be rather ignorant and unhelpful.

I think there are a few underlying explanations as to why these decisions are made. I am pretty convinced that people make these decisions under a presumption of them being right, and perhaps even a “safe” choice. As said earlier; “if it works for Google or Facebook then it should also work for us”.

So hence part of the problem becomes as highlighted before, that this lazy thinking can actually cause more harm than good. It is probably not a “safe” choice it is probably a wrong and dangerous choice if taken lightly.

But what I also found when talking to people about similar experiences is that there can be other factors driving this trend. A recent example was a larger Danish government project where a larger management consultancy firm had been advisers and was pushing really hard for Amazon-based infrastructure and solutions. That is not necessarily bad, as it could be the right solution.

But in this case they were for instance pushing Amazons DynamoDB really hard, as it was the future and could hold just about anything. Again true. But that is just not a core issue of the problem they were trying to solve. All their data was well structured – almost as structured as it gets. This was government data in the real-estate sphere. It is scrutinized, audited and very well organized. To further complicate matters, none of the people who was going to be on the team had any experience with DynamoDB, so everyone had to be sent off on weeks of courses. Again adding cost and complexity.

What drives these kinds of decisions? Should you put on your tinfoil hat and claim that in the case above, the management consultancy firm may have had economic interests in pushing a certain technology? Perhaps. But then why did the people working for the government not provide enough push-back? Could be that if they had the ability to dissect that the decision was bad, then they had not needed the management consultancy firm in the first place. Or perhaps a very bad case of sunk cost fallacy where an expensive bill for a management consultancy firm is hard to swallow if you go against their recommendations.

I do not know the final answer to the question, but I find the trend a little worrisome and sad. It seems like the software developments version of the “Hedonic treadmill” and not really progress at all.

In my book simplicity always wins. You win by subtracting complexity – not by adding it. Start with the simplest possible solution to the problem and see where it gets you. Simple makes it cheaper. Simple solutions makes it easier to implement, makes it easier to test, faster to ship and hence faster to get feedback. Once you have this feedback, whether from unit tests, Proof of Concept or user tests, then you can decide to add complexity if your simple solution proves too slow or too rudimentary.

Always start with the simple solution.