Forem: Morten Hau Lyng

Creating a single-file executable CLI with ASP.NET Core 3.0

Morten Hau Lyng — Thu, 17 Oct 2019 10:36:27 +0000

A new experimental feature in .NET Core is the System.CommandLine NuGet package(s). These make it possible for developers to create CLI (Command Line Interface) projects using the console application template from .NET Core.

This is interesting since it enables you to expose existing business logic class libraries via a CLI.

Working with System.CommandLine

When working with System.CommandLine there are multiple approaches

DragonFruit

One approach is to use an extra NuGet Package called System.CommandLine.DragonFruit, this package allows you to make strongly typed input arguments for your Main class, afterwards the package will generate appropriate commands, options and arguments for running the application as a CLI. This is however in my opinion not the best solution as it generates input arguments as options instead of Arguments forcing user to explicitly declare argument names.

Method first

Method first relies on reflection to find the appropriate method inside the Application and create an invocation for it. I'm not a fan of this solution and will use no further time explaining it. Please see the System.CommandLine wiki for more info.

Syntax first

Syntax first is a declarative approach where you define your commands, options and arguments in code. Furthermore the invocation of the given commands are also declared in code. This is the approach we will be using for this demonstration.

Show me the code

To emulate the scenario where you have existing Business Logic you want to expose through a CLI we have implemented the class Calculator. It is a very simple static calculator class containing four methods (add, subtract, multiply and divide). See the calculator code here:

public static class Calculator
{
    public static double Add(double first, double second)
    {
        return first + second;
    }

    public static double Subtract(double first, double second)
    {
        return first - second;
    }

    public static double Multiply(double first, double second)
    {
        return first * second;
    }

    public static double Divide(double first, double second)
    {
        return first / second;
    }
}

To get started with the CLI project run the following command to scaffold you Console Application

dotnet new console -o Calculator

Add the necessary NuGet packages with the following commands (Please note that as of writing this, these are experimental features which is why we include an alpha version)

dotnet add package System.CommandLine.Experimental --version 0.3.0-alpha.19405.1

To make it console friendly a wrapper class is implemented to handle the console output.

public static class ConsoleCalculator
{
    public static void Add(double first, double second) {
        Console.WriteLine($"Add called with arguments, first: {first}; second: {second}");
        Console.WriteLine($"Result: {Calculator.Add(first, second)}");
    }

    public static void Subtract(double first, double second) {
        Console.WriteLine($"Subtract called with arguments, first: {first}; second: {second}");
        Console.WriteLine($"Result: {Calculator.Subtract(first, second)}");
    }

    public static void Multiply(double first, double second) {
        Console.WriteLine($"Multiply called with arguments, first: {first}; second: {second}");
        Console.WriteLine($"Result: {Calculator.Multiply(first, second)}");
    }

    public static void Divide(double first, double second) {
        Console.WriteLine($"Divide called with arguments, first: {first}; second: {second}");
        Console.WriteLine($"Result: {Calculator.Divide(first, second)}");
    }
}

All this is all fine and well but, there's nothing new here. For the System.CommandLine magic to shine through we need to take a look at our Program.cs.

class Program
{
    static int Main(string[] args)
    {
        // Define the 'add' command
        var addCommand = new Command("add")
        {
            Description = "Adds <first> to <second>"
        };

        // Define the arguments for the add command
        // Please note that these argument names should match the input variables in the method
        addCommand.AddArgument(new Argument<double>("first"));
        addCommand.AddArgument(new Argument<double>("second"));

        // Define command handler for command
        // The parsed arguments are forwarded to the Add method of the ConsoleCalculator
        addCommand.Handler = CommandHandler.Create<double, double>(ConsoleCalculator.Add);

        // Add a shorthand alias
        addCommand.AddAlias("a");

        // Define the 'sub' command
        var subComamnd = new Command("sub")
        {
            Description = "Subtracts <second> from <first>"
        };

        // Define the arguments for the sub command
        // Please note that these argument names should match the input variables in the method
        subComamnd.AddArgument(new Argument<double>("first"));
        subComamnd.AddArgument(new Argument<double>("second"));

        // Define command handler for command
        // The parsed arguments are forwarded to the Subtract method of the ConsoleCalculator
        subComamnd.Handler = CommandHandler.Create<double, double>(ConsoleCalculator.Subtract);

        // Add a shorthand alias
        subComamnd.AddAlias("s");

        // Define the 'multiply' command
        var multiplyComamnd = new Command("multiply")
        {
            Description = "Multiplies <first> with <second>"
        };

        // Define the arguments for the multiply command
        // Please note that these argument names should match the input variables in the method
        multiplyComamnd.AddArgument(new Argument<double>("first"));
        multiplyComamnd.AddArgument(new Argument<double>("second"));

        // Define command handler for command
        // The parsed arguments are forwarded to the Multiply method of the ConsoleCalculator
        multiplyComamnd.Handler = CommandHandler.Create<double, double>(ConsoleCalculator.Multiply);

        // Add a shorthand alias
        multiplyComamnd.AddAlias("m");

        // Define the 'divide' command
        var divideComamnd = new Command("divide")
        {
            Description = "Divides <first> with <second>"
        };

        // Define the arguments for the divide command
        // Please note that these argument names should match the input variables in the method
        divideComamnd.AddArgument(new Argument<double>("first"));
        divideComamnd.AddArgument(new Argument<double>("second"));

        // Define command handler for command
        // The parsed arguments are forwarded to the Divide method of the ConsoleCalculator
        divideComamnd.Handler = CommandHandler.Create<double, double>(ConsoleCalculator.Divide);

        // Add a shorthand alias
        divideComamnd.AddAlias("d");

        // A RootCommand is required, we define it below and define our commands as subcommands
        var rootCommand = new RootCommand
        {
            addCommand,
            subComamnd,
            multiplyComamnd,
            divideComamnd
        };

        // Invoke the root command async and return the result
        return rootCommand.InvokeAsync(args).Result;
    }
}

That's actually it. now we can build the project using the following

dotnet build

After that please go into the destination folder of the build. If your setup is similar to mine it'll be

<project-folder>\bin\Debug\netcoreapp3.0\win10-x64

Now you can test the application by running the following

.\Calculator.exe --help

which produces the following output:

Usage:
  Calculator [options] [command]
Options:
  --version    Display version information
Commands:
  a, add <first> <second>         Adds <first> to <second>
  s, sub <first> <second>         Subtracts <second> from <first>
  m, multiply <first> <second>    Multiplies <first> with <second>
  d, divide <first> <second>      Divides <first> with <second>

Let's do a simple test by adding two doubles, you can do this by running the following

.\Calculator.exe a 1.2 2.3

Which gives the output

Add called with arguments, first: 1,2; second: 2,3
Result: 3,5

Publish Single file

To enable a true cli experience we would like to ship our application as a single executable file.
This has been made possible in .NET Core 3.0 - please modify your .csproj file to look like the following

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <RuntimeIdentifier>win10-x64</RuntimeIdentifier>
    <PublishSingleFile>true</PublishSingleFile>
    <PublishTrimmed>true</PublishTrimmed>
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="System.CommandLine.Experimental" Version="0.3.0-alpha.19405.1" />
  </ItemGroup>

</Project>

The new properties will be described below:

PublishSingleFile: When true a single platform-specific single-file executable. This is the reason for setting <RuntimeIdentifier> property. See single-file bundler design document

PublishTrimmed: When true the compiler analyses the IL code and removes unused assemblies to optimize the size of the final artifact. Note! If your application uses Reflection or related dynamic features might break as the compiler cannot resolve the dynamic features. This can be prevented by introducing a list for the linker of DLL's it cannot remove. See the release announcement for .NET Core 3.0 or the documentation for the IL Linker.

PublishReadyToRun: This enables R2R (Ready To Run) compilation which is a kind of AOT (Ahead OF Time) compilation. This introduces restrictions in regard to cross-platform compilation and architecture. Please see the release announcement for .NET Core 3.0.

Now when you publish your project using

dotnet publish -c Release

You will get a single exe file that contains your application. It would seem that the first time you run this application it extracts the necessary dll's to the temporary file storage as to be able to use them.

A closer look at trimming

Let's take a look at how effective the trimming actually is, first i set the property to false in my .csproj file and then run

dotnet publish -c Release

I get a single-file executable that hasn't been trimmed. The size of this file is 67,787KB or roughly 67.7MB.
Now lets set the property to true again and publish the file anew.

After publishing with the property set to true, the size of our executable is 36,713KB or roughly 36.7MB, so the trimming actually almost halved the size of our executable - neat!

Conclusion

In my opinion this is an awesome development enabling users to expose business logic via a CLI. I especially see it as being extremely powerful in a DevOps setup where you can build customs utilities to manage stuff like database seeding, migration and so on. It is still experimental though, the code might change a lot before it's released.

All source code from this post can be found on Github.

gRPC in ASP.NET Core 3.0

Morten Hau Lyng — Thu, 03 Oct 2019 08:25:48 +0000

Alongside .Net Core 3.0 comes a welcome feature to the ASP.NET Core framework, more precisely the support for using gRPC (Google Remote Procedure Calls).

gRPC is a RPC (Remote Procedure Calls) framework, and was made open source by Google back in 2015, so as you might have noticed the technology isn't new. But it's new in ASP.NET Core. The communication is done over the http2 protocol (see a brilliant introduction to http2 here).

Why is it cool?

Firstly; it's fast, very fast. It relies on binary serialization instead of a human readable JSON format, it makes the package size considerably smaller reducing the network overhead.

Secondly; it has a idiomatic service definition language, meaning that when you have specified your services you can generate server stubs or clients in 10 different programming languages. This offers immense flexibility in systems with a microservice architecture because all of your services doesn't have to be written in the same language/framework as long as it adheres to the specification.

Thirdly; the framework supports bidirectional streaming over http2, meaning that the client can stream its request to the server and the server can stream responses to its clients. This enables processing of the received data while the transfer is still going, i.e. you might need to upload a lot of images to the server at once, but would like the server to compress and save the images as it receives them.

It is "often" used in microservice architectures as the communication framework of choice in between services, often an API Gateway is implemented to mitigate communication with end clients via a RESTful JSON API. This is in part due to the lacking http2 support on client machines.

<TL;DR>

It has a lot of nifty features, is very flexible, allows rapid prototyping and is blazing fast compared to RESTful APIs using JSON.

Show me the code

After downloading and installing the .NET Core 3.0.1 SDK a new project template will be available through dotnet new,

We will scaffold our gRPC service with the following CLI command:

dotnet new grpc -o grpc-blog

This will scaffold an ASP.NET Core web project for you with a folder called Protos. This is the idiomatic service definition mentioned above.

Let's take a look at the service definition that was generated. This is a so called protobuf (Protocol Buffers) definition:

syntax = "proto3";

option csharp_namespace = "grpc_blog";

package Greet;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply);
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings.
message HelloReply {
  string message = 1;
}

This is a simple example of a service called GreeterService. The service has one method called SayHello. The method receives a HelloRequest containing a name to greet, and returns a HelloReply greeting the name specified in the request.
Please note that all the fields in the Request and the Reply is numbered. This is to ensure that the object is deserialized correctly. The numbers must be unique and shouldn't be changed once a service is in use. If a field is removed the number dies with it. New fields should also have new unique numbers.

Fields can be made made required, optional and repeated. Repeated fields can be repeated any number of times including zero i.e. it is used for collections of objects.

When you build this project, a stub for the service is generated. This stub is used to implement the actual service. Let's take a look at the default implementation of our GreeterService.

public class GreeterService : Greeter.GreeterBase
{
    private readonly ILogger<GreeterService> _logger;
    public GreeterService(ILogger<GreeterService> logger)
    {
        _logger = logger;
    }

    public override Task<HelloReply> SayHello(HelloRequest request, ServerCallContext context)
    {
        return Task.FromResult(new HelloReply
        {
            Message = "Hello " + request.Name
        });
    }
}

As you can see the service inherits Greeter.GreeterBase. This is the Service stub that was generated during build. The stub is generated from the proto file we just took a look at. In this case it has made a service stub and the corresponding Requests and Responses.

Working with streams

As I mentioned earlier the gRPC framework allows for bidirectional streaming of requests/responses. The default example doesn't include any streaming operations, lets make one.
First lets modify our proto file to include a streaming request. In this case we will add a new method StreamHello that streams the response as a stream of charts instead of a string.

I've modified my proto file to include a method that streams the response, and added a new response type containing a single character instead of the entire message.

syntax = "proto3";

option csharp_namespace = "grpc_blog";

package Greet;

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply);
  // New endpoint
  rpc StreamHello (HelloRequest) returns (stream HelloStreamReply);
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}
// New response message type
message HelloStreamReply {
  string character = 1;
}

After modifying our service to include a streaming endpoint it looks like this:

public class GreeterService : Greeter.GreeterBase
{
    private readonly ILogger<GreeterService> _logger;
    public GreeterService(ILogger<GreeterService> logger)
    {
        _logger = logger;
    }

    public override Task<HelloReply> SayHello(HelloRequest request, ServerCallContext context)
    {
        return Task.FromResult(new HelloReply
        {
            Message = "Hello " + request.Name
        });
    }

    public override async Task StreamHello(HelloRequest request, IServerStreamWriter<HelloStreamReply> responseStream, ServerCallContext context)
    {
        var message = $"Hello {request.Name}";
        foreach (var c in message)
        {
            var reply = new HelloStreamReply { Character = c.ToString() };
            await responseStream.WriteAsync(reply);
        }
    }
}

The functionality in the stream endpoint StreamHello is rather pointless, it only serves to be a simple example of the stream implementation.

Enabling gRPC in Startup

To enable the gRPC endpoints we need to add a few things to our Startup class. The scaffolded project already contains these lines, but I'll highlight them here in case you need to include a gRPC service in an existing web project.

Firstly we need to modify our ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
{
    // ADding tbe gRPC library to the service registration
    services.AddGrpc();
}

Second we need to modify our Configure method:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }

    app.UseRouting();

    // Configure the endpoints for gRPC
    app.UseEndpoints(endpoints =>
    {
        endpoints.MapGrpcService<GreeterService>();

        endpoints.MapGet("/", async context =>
        {
            await context.Response.WriteAsync("Communication with gRPC endpoints must be made through a gRPC client. To learn how to create a client, visit: https://go.microsoft.com/fwlink/?linkid=2086909");
        });
    });
}

The client side of things

A client for your gRPC service can be implemented in many different languages/frameworks. A list of languages can be found on the gRPC homepage grpc.io

For simplicity reasons we will create our client as a dotnet core console app. We do that by running the following:

dotnet new console -o client
cd client
dotnet add package Grpc.Net.Client
dotnet add package Google.Protobuf
dotnet add package Grpc.Tools

This scaffolds an ASP.NET Core console application with the required nuget packages for making gRPC communication.

We need our proto definition to ensure the client, we generate adhere to the service definition. We create a folder in our client application called Protos and copy the proto definition from earlier into this new folder.

After copying the proto file we need to add the following xml to our project file in the client project:

<ItemGroup>
  <Protobuf Include="Protos\greet.proto" GrpcServices="Client" />
</ItemGroup>

Now when we build our client project the Greeter namespace stubs will be generated as we did in our service project. After generating the client part of the library we added some business logic to our Program.cs. See below:

public class Program
{
    static async Task Main(string[] args)
    {
        Console.WriteLine("Calling SayHello Endpoint with name: 'World'");

        // The gRPC endpoint set up in our service.
        var channel = GrpcChannel.ForAddress("https://localhost:5001");
        // Create a client using the channel
        var client = new Greeter.GreeterClient(channel);

        // Call the SayHello endpoint
        var helloReply = client.SayHello(new HelloRequest { Name = "World" });
        Console.WriteLine($"Response from server, '{helloReply.Message}'");

        Console.WriteLine("Calling StreamHello endpoint with name: 'John Doe'");
        // Calling the StreamHello ednpoint and constructing the stream
        using (var streamResponse = client.StreamHello(new HelloRequest { Name = "John Doe" })) 
        {
            Console.WriteLine("Response stream listed here with index of receiving order");
            int i = 1;
            // Keep moving next and print the received character untill end of stream
            while(await streamResponse.ResponseStream.MoveNext(cancellationToken: CancellationToken.None))
            {
                Console.WriteLine($"[{i:D2}] {streamResponse.ResponseStream.Current.Character}");
                i++;
            }
        }
        Console.WriteLine("Closing stream resposne");

        // Prevent early exit
        Console.WriteLine("Press any key to exit...");
        Console.ReadKey();
    }
}

Running the code produces the following output:

Calling SayHello Endpoint with name: 'World'                      
Response from server, 'Hello World'
Calling StreamHello endpoint with name: 'John Doe'
Response stream listed here with index of receiving order
[01] H
[02] e
[03] l
[04] l
[05] o
[06]
[07] J
[08] o
[09] h
[10] n
[11]
[12] D
[13] o
[14] e
Closing stream resposne
Press any key to exit...

Conclusion

This has been a brief look into what it takes to get gRPC up and running in ASP.NET Core 3.0 - there's still a lot more to learn.
The protobuf definition language contains a lot of small caveats (reserved field numbers, field number size allocation, etc...) and neat tricks to precisely specify the behavior of the service.

Furthermore the client and server stubs can be generated in a lot of different languages (C++, C#, Ruby, Go, node.js, PHP and many more...).

All source code used in this blog post can be found on GitHub.

Inspiration/Further reading

Articles, guides and tutorials i read before writing this blog post: