Forem: elfalem

Tale of .NET Core and the JSON SQLite Extension

elfalem — Fri, 09 Apr 2021 16:59:05 +0000

I was working on a .NET Core application and was getting to the stage where data needed to be persisted. I wanted to use SQLite because it's great as an application file format. Some of the data would be stored using JSON. Although SQLite doesn't have built-in functionality for working with JSON, it allows the use of extensions. This is a very powerful feature allowing anyone to customize SQLite to their needs. Things like JSON support (JSON1) and full-text search (FTS3 and FTS4) are implemented in this way.

Although loadable extensions can be statically linked at compile time, they can also be dynamically linked at run time. The latter is very beneficial because it means you don't have to re-compile your own version of SQLite to include an extension and ship it with your application. Instead, you can leverage an existing SQLite instance and load the necessary extensions at run time.

In the case of .NET, there is the System.Data.SQLite.Core NuGet package, the official database and ADO.NET provider. This instance of SQLite doesn't have JSON support enabled by default but I should be able to load the extension as mentioned above. However there were some unexpected challenges and thus began my journey into getting a SQLite extension to load in a .NET core application on a Linux machine.

Initial Attempt

I was working on a Linux Mint machine with .NET SDK version 2.2. After referencing the necessary packages, the first step was to create a connection to the database.



dotnet new console
dotnet add package System.Data.SQLite.Core
dotnet add package Dapper



using(var connection =  new SQLiteConnection("Data Source=:memory:"))
{
    connection.Open();
    var result = connection.QueryFirst<string>("select SQLITE_VERSION() AS Version");
    Console.WriteLine(result);
}

This part was straightforward. I was able to connect to an in-memory database and successfully execute a query through Dapper.

The next step was loading the JSON1 extension. This is what the internet says to do to load an extension:



using(var connection =  new SQLiteConnection("Data Source=:memory:"))
{
    connection.Open();
    connection.LoadExtension("SQLite.Interop.dll", "sqlite3_json_init");
    var result = connection.QueryFirst<string>("select json(' { \"this\" : \"is\", \"a\": [ \"test\" ] } ')");
    Console.WriteLine(result);
}

The above actually works on Windows (which I'll discuss at the end) but it fails on Linux. It shows an error indicating that the DLL is not found:



Unhandled Exception: System.Data.SQLite.SQLiteException: SQL logic error
The specified module could not be found.

Unfortunately I wasn't able to find a lot of information on the net to help with this issue.

Structure of the NuGet package and SQLite

I started looking into how the NuGet package and SQLite work in hopes of uncovering something that resolves the issue. SQLite is written in C. The NuGet package allows us to interface with it by calling the C functions through C#. Since the C code is native, it's specific to different operating systems and architectures. The NuGet package contains the native SQLite code compiled to different platforms.

Actually, if you directly download System.Data.SQLite.Core from NuGet and extract it (.nupkg files are just archives), there isn't a lot of code. If you open up System.Data.SQLite.Core.nuspec in a text editor you will find that it's a simple wrapper for other packages. It conditionally references various other packages based on the version of .NET framework encountered. For instance, the package Stub.System.Data.SQLite.Core.NetStandard is referenced for .NET Core. This package has a much larger size (3.72 MB vs 11.44 KB for the wrapper) and is actually what contains the code.

Downloading and extracting this package, you'll find a lib/ folder with the C# DLL as well as the runtimes/ folder with platform specific native code as mentioned above.



runtimes/linux-x64/native/SQLite.Interop.dll
runtimes/win-x64/native/SQLite.Interop.dll
runtimes/osx-x64/native/SQLite.Interop.dll
runtimes/win-x86/native/SQLite.Interop.dll

You can inspect DLLs with nm on Linux. For example: nm --dynamic runtimes/linux-x64/native/SQLite.Interop.dll shows all the symbols in the DLL. This is one way to examine what functions exist. A portion of the output for this command looks like this:



0000000000017028 T sqlite3_hard_heap_limit64
0000000000114ea2 T sqlite3_index_column_info_interop
00000000000d3cc4 T sqlite3_initialize
00000000000d6240 T sqlite3_interrupt
0000000000142a7f T sqlite3Json1Init
0000000000142c9d T sqlite3_json_init
00000000000d2533 T sqlite3_keyword_check
00000000000d2528 T sqlite3_keyword_count
00000000000d24ba T sqlite3_keyword_name
00000000000d556d T sqlite3_last_insert_rowid
0000000000114679 T sqlite3_last_insert_rowid_interop
00000000000d3ca1 T sqlite3_libversion
00000000000d3cae T sqlite3_libversion_number
00000000000d764f T sqlite3_limit
0000000000095810 T sqlite3_load_extension
000000000001b695 T sqlite3_log
000000000001746c T sqlite3_malloc
00000000000174a4 T sqlite3_malloc64

In the above snippet, we find functions that are part of the core SQLite code such as sqlite3_initialize and sqlite3_load_extension. But we also find sqlite3_json_init which is the entry point of the JSON1 loadable extension. It looks like the DLL is sourced from the SQLite Amalgamation. The SQLite Amalgamation is a single file containing all of SQLite proper as well as about half a dozen loadable extensions that are maintained by the SQLite team. Although the extensions are not enabled during compilation, having them bundled makes it easier to distribute and enable them at runtime.

Putting all this together, the structure of the NuGet package is as follows:

How SQLite Loads an Extension at Runtime

Using a SQLite loadable extension at runtime entails dynamically linking to the extension code. Whether it's done through C# code (connection.LoadExtension() in the above snippet), the command-line SQLite shell (.load), or other means, ultimately the process involves searching for a DLL and loading it into memory.

I came across a Microsoft Docs page which indicates that SQLite doesn't leverage the .NET Core logic for finding native libraries. It directly calls the platform API to load the DLL. The way to influence this is to modify environment variables (PATH on Windows, LD_LIBRARY_PATH or DYLD_LIBRARY_PATH on Linux). There is sample code linked from the Doc that finds native libraries for the current runtime in a referenced NuGet package. The code notes that on Windows, PATH could be modified with the path of found native libraries after the application starts. However for Linux, the environment variable must be modified before running the application which is cumbersome.

Instead of changing environment variables to modify the DLL search locations, I wanted to see if we can somehow provide an absolute path for the DLL location. Looking at the source for the NuGet package it's clear that, as the Microsoft Docs page noted, the C# code just invokes the native code (sqlite3_load_extension()) directly. Therefore I looked at the SQLite source code. SQLite has the concept of VFS which is a portability layer at the bottom of the SQLite architecture. There are implementations of this abstraction for specific operating systems. Windows and Linux implementations are os_win.c and os_unix.c respectively. The extension loading code eventually relies on VFS implementations to load DLLs. On Linux this is handled by dlopen and it's LoadLibraryW on Windows. I later rediscovered this same information after reading the bottom (section 7) of a SQLite documentation page.

Focusing on dlopen(), the documentation shows that it accepts a full file path. This is great because I can obtain the location of the platform specific native library using the same approach as the Microsoft sample code from earlier. However instead of attempting to modify environment variables, I can provide the direct path to connection.LoadExtension() and load the DLL. Since I'm on a Linux machine, I started by directly hard-coding the path to the native Linux library. If it works, I should be able to easily generalize it to work on other platforms.

I changed the code to the following and ran dotnet publish.



using(var connection =  new SQLiteConnection("Data Source=:memory:"))
{
    connection.Open();
    connection.LoadExtension("/<my-project-path>/bin/Debug/netcoreapp2.2/publish/runtimes/linux-x64/native/SQLite.Interop.dll", "sqlite3_json_init");
    var result = connection.QueryFirst<string>("select json(' { \"this\" : \"is\", \"a\": [ \"test\" ] } ')");
    Console.WriteLine(result);
}

I then ran the published DLL from the /bin/Debug/netcoreapp2.2/publish/ folder and it worked. The query that includes the json() function provided by the extension ran successfully and I got the expected result.

I didn't realize it at the time but it becomes important that I ran the DLL from the publish/ folder and not the one that's one level up at /bin/Debug/netcoreapp2.2/.

Confusing Observations

At this point I thought I had a good solution for loading the DLL. I started making other unrelated changes to my program. But when I tried to run it with dotnet run, it wouldn't work. The program abruptly exits when it reaches the line that loads the extension. I tried running the published DLL as before, and that works. But using dotnet run does not. There are no exceptions or errors of any kind that's outputted. Even when debugging (with justMyCode set to false in .vscode/launch.json) there are no signs that anything is wrong. It even indicated an exit code of 0 (i.e. successful termination of program). Yet the program was quitting abruptly. I was very confused and posted a question on Stack Overflow.

Unfortunately I didn't get any answers. One thing I eventually tried was using newer versions of the .NET framework since I was on version 2.2. I changed the project target to 3.1 and tried dotnet run. I got the same behavior of the program terminating without errors when it got to loading the extension. I executed the DLL (from the /bin/Debug/netcoreapp3.1/ folder) directly and this time it terminated in the same place but I got a segmentation fault error. I was more confused because I expected this to work (it seemed like it was working when using version 2.2). I changed the target framework to 5.0 and tried again. I got the same results as 3.1. Then I changed it back to 2.2 and I started getting segmentation fault there as well when executing the DLL directly. This was really puzzling because it was working earlier. We'll get to it later but this was because this time I was executing the DLL from /bin/Debug/netcoreapp2.2/ instead of /bin/Debug/netcoreapp2.2/publish/. However, I didn't realize it at the time.

Segmentation Fault

I turned my attention towards investigating the segmentation fault I was seeing. It means there is improper memory access occurring somewhere. However this can't happen in managed code so it was likely coming from the native C code for SQLite. I used gdb to debug and find the offending line. The following is a snippet of the call stack after my program crashes.



Thread 1 "dotnet" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007fffec630c04 in sqlite3_mutex_enter (p=0x555555859f28) at ../core/sqlite3.c:25905
#2  0x00007fffec6f05c9 in createFunctionApi (db=0x5555559a43c8, zFunc=0x7fffec773d60 "json", nArg=1, enc=2099201, p=0x7fffec99a3cc <aFunc.54932+12>, xSFunc=0x7fffec75ae78 <jsonRemoveFunc>, 
    xStep=0x0, xFinal=0x0, xValue=0x0, xInverse=0x0, xDestroy=0x0) at ../core/sqlite3.c:162077
#3  0x00007fffec6f0708 in sqlite3_create_function (db=0x5555559a43c8, zFunc=0x7fffec773d60 "json", nArg=1, enc=2099201, p=0x7fffec99a3cc <aFunc.54932+12>, 
    xSFunc=0x7fffec75ae78 <jsonRemoveFunc>, xStep=0x0, xFinal=0x0) at ../core/sqlite3.c:162117
#4  0x00007fffec75cb33 in sqlite3Json1Init (db=0x5555559a43c8) at ../ext/json1.c:2591
#5  0x00007fffec75ccbd in sqlite3_json_init (db=0x5555559a43c8, pzErrMsg=0x7fffffffc670, pApi=0x7fffecd215c0 <sqlite3Apis>) at ../ext/json1.c:2623
#6  0x00007fffeca366ae in sqlite3LoadExtension (db=0x5555559a43c8, 
    zFile=0x7fff5807dd98 "/<my-project-path>/bin/Debug/netcoreapp2.2/publish/runtimes/linux-x64/native/SQLite.Interop.dll", zProc=0x7fff5807de60 "sqlite3_json_init", 
    pzErrMsg=0x7fffffffc7c8) at ../core/sqlite3.c:124451
#7  0x00007fffeca36850 in sqlite3_load_extension (db=0x5555559a43c8, 
    zFile=0x7fff5807dd98 "/<my-project-path>/bin/Debug/netcoreapp2.2/publish/runtimes/linux-x64/native/SQLite.Interop.dll", zProc=0x7fff5807de60 "sqlite3_json_init", 
    pzErrMsg=0x7fffffffc7c8) at ../core/sqlite3.c:124484
#8  0x00007fff7d01ca34 in ?? ()
#9  0x4164657a696c6169 in ?? ()
#10 0x0000000094cf77ea in ?? ()
#11 0x00007ffff66f6c38 in ?? () from /usr/share/dotnet/shared/Microsoft.NETCore.App/2.2.8/libcoreclr.so
#12 0x00007fffffffcbb8 in ?? ()

As expected the issue was in SQLite code. I tried looking at the code where the problem was occurring but nothing stood out. Searching the web for segmentation faults related to SQLite I came across this forum post. The answer pointed to multiple instances of SQLite being the issue. Although the forum post was about a python application, I started wondering if there were multiple instances being created in my application as well. It seemed unlikely because I wasn't using threads or doing anything fancy. It was a simple console application. Perhaps something was lingering from earlier runs of my application? I tried restarting VS Code and the entire machine. I tried using an actual file instead of an in-memory database. I tried explicitly enabling extension loading (connection.EnableExtensions(true)) even though it was already enabled. I looked at various compile and connection setup flags in SQLite. None of this helped.

Looking up information about the function where the error is occurring (sqlite3_mutex_enter), I came across a Stack Overflow question about access violation (i.e. segmentation fault) in python. The accepted answer talks about multiple modules which apparently are DLLs on Windows. The solution in this case was invoking sqlite3_initialize() in the SQLite code for the second module. Although it was a different language, it seemed like the same issue I was facing. And it seems to align with the eariler forum post about multiple instances of SQLite being the issue.

Given the above information, I wanted to see if I can reach into the native library and invoke sqlite3_initialize(). This was quite straightforward. I added the following method signature that corresponds to the C function I wanted to invoke and decorated it with DllImport attribute specifying the full path to the DLL.



[DllImport("/<my-project-path>/bin/Debug/netcoreapp2.2/publish/runtimes/linux-x64/native/SQLite.Interop.dll")]
static extern void sqlite3_initialize();

Then I added a call to this method before loading the extension.



using(var connection =  new SQLiteConnection("Data Source=:memory:"))
{
    connection.Open();
    sqlite3_initialize();
    connection.LoadExtension("/<my-project-path>/bin/Debug/netcoreapp2.2/publish/runtimes/linux-x64/native/SQLite.Interop.dll", "sqlite3_json_init");
    var result = connection.QueryFirst<string>("select json(' { \"this\" : \"is\", \"a\": [ \"test\" ] } ')");
    Console.WriteLine(result);
}

After the above changes, the program did not terminate abruptly or lead to a segmentation fault. The JSON extension was loaded and the SQL query that made use of syntax provided by the extension executed successfully.

It was working at this point. But the fix was a workaround. It was clear that there were two different DLLs being loaded. As we saw earlier SQLite.Interop.dll is made from the SQLite Amalgamation. Therefore when the extension is loaded, a second full instance of SQLite was also being loaded. The first instance would be what is referenced directly by the NuGet package and used to establish the database connection and execute queries. I wanted to inspect the location of this first instance. To do so, I printed the location of the assembly that contains the SQLiteConnection class.



Console.WriteLine(typeof(SQLiteConnection).Assembly.Location);

and got the output:



/home/<my-user>/.nuget/packages/stub.system.data.sqlite.core.netstandard/1.0.113.2/lib/netstandard2.0/System.Data.SQLite.dll

This was surprising because I thought it would be referencing the package from the publish/ folder. I changed my code to refer to the same absolute path as above when loading the extension. I also removed the call to sqlite3_initialize():



using(var connection =  new SQLiteConnection("Data Source=:memory:"))
{
    connection.Open();
    Console.WriteLine(typeof(SQLiteConnection).Assembly.Location);
    connection.LoadExtension("/home/<my-user>/.nuget/packages/stub.system.data.sqlite.core.netstandard/1.0.113.2/runtimes/linux-x64/native/SQLite.Interop.dll", "sqlite3_json_init");
    var result = connection.QueryFirst<string>("select json(' { \"this\" : \"is\", \"a\": [ \"test\" ] } ')");
    Console.WriteLine(result);
}

The above worked. The same exact DLL was being referenced so there weren't multiple instances of SQLite.

At this point, I remembered that publishing my program creates a DLL in the publish/ folder. Ever since I encountered the segmentation fault, I was executing the DLL that was one level up. When I executed the DLL inside the publish/ folder, I saw that the path of the assembly containing SQLiteConnection was the following and the program was terminated with a segmentation fault error.



/<my-project-path>/bin/Debug/netcoreapp2.2/publish/System.Data.SQLite.dll

It now made sense why the program worked when I initially hard-coded the full path of the DLL. I executed it from the publish/ folder. Therefore, the NuGet package and the LoadExtension call were referencing the same native library. It meant only one instance of SQLite existed and hence there wasn't an issue. This observation raised a couple of questions.

Questions About .NET SDK

The first question was why there are two seemingly identical DLLs created. Ever since I first worked with .NET Core I've noticed that when publishing a program Foo, both /bin/Debug/netcoreapp2.2/Foo.dll and /bin/Debug/netcoreapp2.2/publish/Foo.dll get created but didn't figure out why. It was clear that the one in the publish/ folder is part of the end result of the publishing process. However, why does the same DLL exist one level up? There were quite a few Stack Overflow questions that asked about this (among other things) but frustratingly none of their answers addressed this particular question. I finally came across one question that directly answered it. I now understand that /bin/Debug/netcoreapp2.2/ is used as the output location when building. The files are then copied to /bin/Debug/netcoreapp2.2/publish/ when publishing. This would have been less confusing if Microsoft defaulted to a different folder as the output location of build artifacts, perhaps /bin/Debug/netcoreapp2.2/build/.

The second question was why when executing the DLL from the build location, it looks for packages in /home/<my-user>/.nuget/packages/ (trivia: this is called a "probing directory". In this context, probing is the process of locating assemblies by the runtime). I was able to find the answer in the documentation for dotnet build. It states that:

For executable projects targeting versions earlier than .NET Core 3.0, library dependencies from NuGet are typically NOT copied to the output folder. They're resolved from the NuGet global packages folder at run time.

So this was a change in newer versions of the framework. I was able to confirm this by outputting the location of the assembly containing SQLiteConnection for the different versions.

Version	Build output DLL	Publish output DLL
2.2	`/home/<my-user>/.nuget/packages/stub.system.data.sqlite.core.netstandard/1.0.113.2/lib/netstandard2.0/System.Data.SQLite.dll`	`/<my-project-path>/bin/Debug/netcoreapp2.2/publish/System.Data.SQLite.dll`
3.1	`/<my-project-path>/bin/Debug/netcoreapp3.1/System.Data.SQLite.dll`	`/<my-project-path>/bin/Debug/netcoreapp3.1/publish/System.Data.SQLite.dll`
5.0	`/<my-project-path>/bin/Debug/netcoreapp5.0/System.Data.SQLite.dll`	`/<my-project-path>/bin/Debug/netcoreapp5.0/publish/System.Data.SQLite.dll`

As can be seen above, newer versions of the framework load dependencies from a location relative to the executing program whether the program was built or published. This makes it easier to consistently locate a native library for my use case.

Another unresolved item was why directly executing the DLL resulted a segmentation fault error but using dotnet run abruptly ends the program with no indication as to the cause. I was curious as to what actually prints the error on the terminal in the first case. In Linux, a segmentation fault is one of the many signals that can be sent from the kernel or between processes. According to this question, if the SIGSEGV signal (i.e. a segmentation fault) is not handled by the program, the program is terminated and the the parent process (i.e. the shell) prints out the message to it's (not the program's, which is already terminated) standard error.

So in the context of my program, it's the Linux shell that's telling me that there's a segmentation fault. I wanted to know how dotnet run executes programs. Looking at the .NET SDK source code, it starts a new process to execute the program. It also reads the standard output and standard error streams from the process it creates. However, I did not see logic dealing with handling signals such as SIGSEGV. I've opened an issue for this.

Behavior on Windows

To gain a broader understanding of items discussed above, I also looked at the behavior on a Windows machine. I used VS Code and not Visual Studio so that that doesn't become a factor. I looked into why simply specifying the DLL name without a path (e.g. SQLite.Interop.dll) works on Windows. This is explained by the search order used when loading DLLs. The first thing that the system checks is if the DLL with the same module name is already in memory. If so, then the already loaded DLL is used and the search stops. This seems to be what's happening in this case. When the application creates the database connection, it's using the native library (since it's an amalgamation containing the core SQLite code as well as the loadable extension). Therefore, the DLL is already loaded by the time we invoke LoadExtension().

To test this further, I made a copy of the DLL in the same folder and renamed it to SQLite.Interop_copy.dll. Then I modified the code to load this renamed DLL (again without specifying the path) and it was not able to find it. The search order is then to look for the DLL in various directories, and lastly in paths listed in the PATH environment variable. The location of the renamed DLL doesn't match any of these and thus it's not found. If I modify the PATH environment variable and add the location, then the renamed DLL is found. This made it possible to see the Windows DLL search heuristic in action.

I also wanted to see if Windows will report a segmentation fault. Using the bash shell within VS Code, I got a segementation fault while using dotnet run as well as when directly executing the DLL. I then switched to using PowerShell in which case the program halted abruptly for both ways of executing it. So it looks like the behavior differs based on the shell I'm using when it comes to Windows.

Lastly, I wanted to see how .NET handles segmentation faults within C# code. I didn't find a way to intentionally cause this with managed code so I had to switch to an unsafe context and write unmanaged code. I declared an integer variable and assign it's memory address to a pointer. I then shift the pointer by an arbitrary value and attempt to access that address as an integer.



unsafe
{
    int foo = 70;
    int *pointer = &foo;
    pointer += 500000000;
    Console.WriteLine((int)*pointer);
}

The above results in the exception: Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. This indicates that there is a difference in how improper memory access is handled when it occurs in unmanaged code as opposed to a native library (which technically is still unmanaged code). If anyone has an explanation of why this is, I'd like to know.

Lessons Learned

This journey was a diversion from the project I was working on but a lot of lessons were learned in the process. I learned more about the .NET build and publish steps and why there are duplicate DLL files in the output directory. I also learned about the the types of things that could be different between .NET SDK versions and the importance of ensuring installed versions are up-to-date. I have a better understanding of how DLLs are loaded in both Linux and Windows. I gained insight into the inner workings of SQLite and the NuGet packages that interface with it to allow it's usage in .NET. Diving into the source code of a very successful project and a complex one like that of a database seems daunting, but SQLite has excellent documentation and it was a rewarding experience.

Furthermore, I got to use gdb to debug a segmentation fault and also learned about how signals in general work in Linux. This was also a good experience at putting together information from various sources and different programming languages to solve a problem. It also demonstrated the effectiveness of using different phrases when searching for solutions. Searching for "segmentation fault" and "access violation" provided different pieces to the puzzle. Finally, this was a practice in perseverance. Not giving up easily when feeling lost is difficult. But at the same time, you gain a powerful sense of accomplishment when things make sense in the end. This image is an apt illustration:

Image: George Broussard

Alternative view engine for ASP.NET Core: Passing data from controller

elfalem — Fri, 29 Jan 2021 17:21:40 +0000

Part III in this series looked at how view templates are dynamically compiled including the processing of simple expressions (like 1 + 1) embedded in the templates. However, view templates are useful when we can pass values to them to be shown in the variable areas of the template. In this final post of the series, we'll demonstrate how values added to the ViewData in the controller can be passed to the view template and rendered in the final output.

In the Razor view engine, accessing a value from ViewData looks like this:

Hello @@ViewData["name"]

For our mustache syntax, we want a more readable syntax like this:

Hello {{name}}

For the above to work name must exist as some sort of variable in the compiled template and have an assigned value.

You can find the full contents of the source files discussed below on this GitHub gist.

Identify Replaceable Values

The first step is knowing all the replaceable values in the view template. Since we allow expressions in our mustache syntax, we need a way to parse C# expressions and identify the variables. As we are already compiling the view template, we can use diagnostic messages from the compilation process to indirectly obtain the list of replaceable values.

In StacheTemplateCompiler.cs, add the following method:

private static List<string> GetMissingProperties(CSharpCompilation compilation)
{
  var missingProperties = compilation.GetDiagnostics().Where(d => d.Id.Equals("CS0103"))
            .Select(d => d.GetMessage()).Distinct()
            .Select(m => m.Replace("The name '", string.Empty)
            .Replace("' does not exist in the current context", string.Empty));

  return missingProperties.ToList();
}

We're using the Roslyn compiler for the heavy lifting. Any names it reports as not existing in the current context are replaceable values in the view template.

Adjust Compiled Template

Once we know this list, we can create the class document a second time and declare the replaceable values as properties. In StacheTemplateCompiler.cs, insert the following into the Compile() method right before loading the assembly:

var missingProperties = GetMissingProperties(compilation);
if(missingProperties.Any())
{
    var secondClassDocument = CreateClassDocument(parsedResult, templateClassName, missingProperties);
    var secondSyntaxTree = CSharpSyntaxTree.ParseText(secondClassDocument);
    compilation = compilation.ReplaceSyntaxTree(syntaxTree, secondSyntaxTree);
}

Additionally modify CreateClassDocument() to accept a list of strings as the last paramater named properties (optional, with default value of null). Insert the following code right after the creation of the StringBuilder object.

if(properties != null)
{
    foreach(var property in properties){
          result.AppendLine($"public dynamic {property} {{get; set;}}");
    }
}

We declare each replaceable value as a dynamic property. Although we have been using strings in the examples, values sent through ViewData from the controller could have other types. Therefore treating it as a dynamic property gives us the flexibility to accept all types.

To use the dynamic keyword, we need to include references to additional assemblies when creating the compilation. Update the beginning of the CreateCompilation() method as follows:

var assemblyLocations = new List<string>{
    // system runtime assembly
    typeof(object).Assembly.Location,
    // the following are needed for dynamic keyword support
    Assembly.Load(new AssemblyName("System.Linq.Expressions")).Location,
    Assembly.Load(new AssemblyName("Microsoft.CSharp")).Location,
    Assembly.Load(new AssemblyName("System.Runtime")).Location,
    Assembly.Load(new AssemblyName("netstandard")).Location
};

var references = assemblyLocations.Select(location => MetadataReference.CreateFromFile(location));

Pass Values to View Template

At this point, we have code to successfully complile a view template while including C# properties for each replaceable value in the template. The last step is to set the properties with the the actual values provided from ViewData when rendering the view template.

In StacheView.cs, update the RenderAsync() method to set the properties using Reflection before invoking the Execute() method on the compiled instance:

var properties = instance.GetType().GetRuntimeProperties()
  .Where((property) =>
    {
      return property.GetIndexParameters().Length == 0 &&
          property.SetMethod != null &&
          !property.SetMethod.IsStatic;
});

foreach(var prop in properties){
    if(context.ViewData.ContainsKey(prop.Name)){
        prop.SetValue(instance, context.ViewData[prop.Name]);
    }
}

To test the changes, we will modify the Bar action in HomeController.cs we have been using to the following:

public IActionResult Bar(){
    ViewData["name"] = "Jonathan";

    return View();
}

And modify the view template Bar.stache as follows:

Hello {{name}}, your name has {{name.Length}} characters.

If you now run the application (dotnet run) and navigate to https://localhost:5001/home/bar, you should see:

Hello Jonathan, your name has 8 characters.

We have successfuly passed a value from the controller to the view template.

Summary

In this series, we created an alternative view engine with its own template syntax. We parsed the template, created a compilation, and used it to render values supplied by the controller. Hopefully this illustrated the modular nature of ASP.NET and its ability to accept third party view engines.

If you're interested in other alternatives, this Stack Overflow answer compares various view engines. Also check out NVue, my experimental attempt at a view engine based on the Vue.js template syntax.

Alternative view engine for ASP.NET Core: Compiling templates

elfalem — Mon, 18 May 2020 15:53:59 +0000

The last post in this series looked at parsing the template syntax. It also promised the next post will cover supporting inline C# expressions. And that's exactly what we'll do. Be sure to check out the previous posts in the series so that it's easier to follow this one.

In the Razor view engine, you can use the @ symbol to transition from markup into writing plain C#. We're going to do something similar where we treat what's inside mustache tags as C# code. For example:

Your Message: {{1 + 1}}

should output:

Your Message: 2

We'll work with a simple arithmetic expression in this post but we'll still explore the core mechanism through which view engines work. You can find the full contents of the source files discussed below on this GitHub gist.

General Approach

At the end of the day, the .NET runtime expects to run code in one of the languages that it supports. So we have to find a way to get from our template syntax to C# code that can be executed. The general approach that's used in such cases is to dynamically construct a C# class from the view template. Once we have a C# representation of the template, we can use APIs in Microsoft.CodeAnalysis and System.Reflection namespaces to dynamically compile, load, and run it. If you look at the source code for the Razor view engine, it's essentially using this same approach.

View Template to C# Class

As shown in the last post, we can parse the view template to generate a DocumentNode. We can use the structured information in DocumentNode to create a C# class.
For example, given the parsed template of Your Message: {{1 + 1}} the following class can be generated:

using System.Text;
namespace StacheTemplateNamespace{
    public class Bar
    {
        private StringBuilder _output = new StringBuilder();

        public string Execute()
        {

_output.Append("Your Message: ");
_output.Append(1 + 1);

            return _output.ToString();
        }
    }
}

Note that we give the class the same name as the view template. It also has a method named Execute() that computes the contents of the template and outputs a string.

Create a new class Stache/StacheTemplateCompiler.cs and add the following method:

    private static string CreateClassDocument(DocumentNode parsedResult, string templateClassName)
    {
      var result = new StringBuilder();
      result.AppendLine($@"
using System.Text;
namespace StacheTemplateNamespace{{
    public class {templateClassName}
    {{
        private StringBuilder _output = new StringBuilder();");

      result.AppendLine(@"
        public string Execute()
        {
            ");

      foreach(var node in parsedResult.Nodes){
        switch(node){
          case TextNode textNode:
            result.Append("_output.Append(\"");
            result.Append(textNode.Value);
            result.AppendLine("\");");
          break;
          case ExpressionNode expNode:
            result.Append("_output.Append(");
            result.Append(expNode.Value);
            result.AppendLine(");");
          break;
        }
      }

      result.AppendLine(@"
            return _output.ToString();
        }
    }
}
            ");

      return result.ToString();

    }

Since DocumentNode contains information about whether a parsed node is a TextNode or ExpressionNode, CreateClassDocument() can render the node value as a string or expression accordingly.

Creating an Assembly

After creating the C# class in string form, it has to be compiled before it can be executed by the .NET runtime. This compiled code is referred to as an assembly. In StacheTemplateCompiler.cs, add the following method:

internal static Assembly Compile(DocumentNode parsedResult, string templateClassName){
  var classDocument = CreateClassDocument(parsedResult, templateClassName);
  // to be continued...
}

This Compile() method first makes use of the method we defined earlier to generate the class document as a string. Next we take that string and parse it using Microsoft.CodeAnalysis to generate a syntax tree:

var syntaxTree = CSharpSyntaxTree.ParseText(classDocument);

Note that this is analogous to the tokenization and parsing that we performed in the last post. But this time with C# code using Microsoft tooling.

The next step is to compile the parsed syntax to machine code (IL code). This is achieved by what Microsoft calls a compilation. A compilation represents everything needed to compile some code. It includes a name for the assembly, references to other assemblies, and of course the code we want to compile. Add the following method to StacheTemplateCompiler.cs:

private static CSharpCompilation CreateCompilation(SyntaxTree syntaxTree){
    var systemRuntimeAssemblyLocation = typeof(object).Assembly.Location;

    var references = new List<MetadataReference>{
      MetadataReference.CreateFromFile(systemRuntimeAssemblyLocation)
    };

    var compilationOptions = new CSharpCompilationOptions(
        OutputKind.DynamicallyLinkedLibrary);

    var assemblyName = System.IO.Path.GetRandomFileName();

    var compilation = CSharpCompilation.Create(assemblyName, options: compilationOptions, 
        references: references);
    compilation = compilation.AddSyntaxTrees(syntaxTree);

    return compilation;
}

This method prepares the compilation. You can see that we're making a reference to the assembly containing the System.Object class (System.Runtime.dll). We want the output to be a DLL (as opposed to EXE). We also picked a random file path for the assembly name since it's not that important for our case. Finally, we add our parsed C# class. We can then invoke this method in Compile():

var compilation = CreateCompilation(syntaxTree);

We now have all the ingredients that we need to compile the class and create an assembly. We do that in the following method (also in StacheTemplateCompiler.cs):

private static Assembly LoadAssembly(CSharpCompilation compilation){
    using (var assemblyStream = new MemoryStream())
    {
        var result = compilation.Emit(assemblyStream, null);
        if (!result.Success)
        {
            throw new Exception(
                string.Join(" ", result.Diagnostics.Select(
                    d => $"{d.Id}-{d.GetMessage()}\n"))
            );
        }

        assemblyStream.Seek(0, SeekOrigin.Begin);
        return Assembly.Load(assemblyStream.ToArray(), null);
    }
}

This method actually compiles the class and emits the IL code. The result is written to computer memory and loaded to the same domain as our running application. We now have a dynamically created class we can instantiate and whose methods we can invoke! Overall the Compile() method looks like this:

internal static Assembly Compile(DocumentNode parsedResult, string templateClassName){
  var classDocument = CreateClassDocument(parsedResult, templateClassName);
  var syntaxTree = CSharpSyntaxTree.ParseText(classDocument);
  var compilation = CreateCompilation(syntaxTree);
  var assembly = LoadAssembly(compilation);
  return assembly;
}

Using the Assembly

We need to make some modifications to the code discussed in the previous posts to use the assembly. For the templateClassName parameter in StacheTemplateCompiler.Compile(), we will use the view name. This requires changing StachViewEngine.cs slightly to include the view name when constructing the StacheView object:

// in StachViewEngine.FindView() method
return ViewEngineResult.Found(viewName, new StacheView(possibleViewLocation, viewName));

// in StachViewEngine.GetView() method
var viewNameWithExtension = viewPath.Substring(viewPath.LastIndexOf('/')+1);
var viewName = viewNameWithExtension.Substring(0,
    viewNameWithExtension.IndexOf(ViewExtension));
return ViewEngineResult.Found(viewPath, new StacheView(appRelativePath, viewName));

We also modify StacheView.cs to accept the template name in the constructor and store it as an instance variable.

public StacheView(string path, string templateName){
  Path = path;
  TemplateName = templateName;
}

public string TemplateName {get; private set;}

Next we replace the logic in StacheView.RenderAsync() that looped through the document node with a call to StacheTemplateCompiler.Compile() we created in this post:

// in StacheView.RenderAsync()
var assembly = StacheTemplateCompiler.Compile(document, TemplateName);
var instance = assembly.CreateInstance($"StacheTemplateNamespace.{TemplateName}");
var processedOutput = instance.GetType().GetMethod("Execute").Invoke(instance, null);

return context.Writer.WriteAsync(processedOutput.ToString());

The above code compiles the document node and obtains an assembly. It then instantiates the C# class we dynamically generated and invokes the Execute() method on it.

To test our changes, update the contents of Bar.stache to the following:

<h2>Your Message: {{1 + 1}}</h2>

Then running the app (dotnet run) and navigating to https://localhost:5001/Home/Bar you should see:

Your Message: 2

Other simple expressions should work as well. For example,

<h2>Your Message: {{"foobar".Length}}</h2>

<h2>Your Message: {{5 == 4}}</h2>

Summary

As shown above, view engines depend on the ability to dynamically compile template syntax into code that can be executed by .NET. A lot of the Roslyn compiler's functionality is exposed as APIs (Microsoft.CodeAnalysis namespace) which greatly facilitates the analysis of C# (and VB) code. Although it's great that we can show the results of simple expressions, view engines are immensely more useful if they can make use of variable values supplied by the controller. Stay tuned for the next post in which we examine how we can use the values added to the ViewData in the compiled view template.

Alternative view engine for ASP.NET Core: Parsing template syntax

elfalem — Sat, 05 Oct 2019 13:24:35 +0000

In my last post, we looked at how we can hook into the ASP.NET Core architecture to create a custom view engine. Although we had a working view engine, it was quite rudimentary. One of the limitations mentioned was that there is only one variable named Message that we can bind to. In this post, we'll look at how we can use as many variables as we need for a given template. Be sure to check out the last post if you haven't done so already as it will make it easier to follow this one.

As it currently stands, our view rendering logic is using direct text replacement to look for {{Message}}:

var processedOutput = template.Replace("{{Message}}", context.ViewData["Message"]?.ToString());
return context.Writer.WriteAsync(processedOutput);

In order to build a more robust view engine, we will need to more intelligently analyze the template syntax by tokenizing and parsing it.

Tokenization and Parsing

Tokenization (or lexical analysis) is the process by which we identify tokens from the view template. Tokens are the specific components that are the building blocks of our view template syntax. Suppose that we're tokenizing the following arithmetic expression:

3 * -4 + 55

The specific tokens would be: 3, *, -, 4, +, 55. We can also assign categories to tokens. In this case, 3, 4, and 55 would be operands or numbers and *, - and + would be operators.

After tokenization the next part of the process is parsing. Parsing takes the tokens and generates some sort of data structure (such as a parse tree or abstract syntax tree) that represents the syntactic structure. Parsing the above arithmetic expression would result a tree as follows:

Having a structure like the above tree allows us to see the relationship between tokens so that we can process it appropriately whether that is evaluating an arithmetic expression or rendering a view from a template.

Looking at our mustache syntax template, we can tokenize and parse it in a similar fashion.

Original template:

Your Message: {{Message}}

The tokens from that template are: Your Message: (string), {{ (open braces), Message (string), }} (close braces).

In terms of parsing, since our mustache syntax is not quite hierarchical, a tree structure is not ideal. Instead we can use a simple list structure with two types of elements: text (that we render directly) and expression (that we treat as a variable and replace with a value). Here it is represented as a table:

-----------------------------
|Type       | Value         |                
-----------------------------
|text       | Your Message: |
|expression |{{Message}}    |
-----------------------------

Implementing a parser

Now that we have some idea of how we can tokenize and parse our template syntax, we'll turn our attention to how we can actually implement it in code. Depending on the template syntax that you're using, it may be possible to use an existing parser. For example, if the syntax is similar to HTML, you can perhaps use Html Agility Pack. Alternatively, if it's mustache or Liquid syntax, you can use fluid parser. If there is no existing parser for the syntax you are using, then you would have to create it. Although we can probably use the fluid parser for our case, we're going to build our own.

Creating a parser could be a very complex process but there are tools and libraries to facilitate it. One such library is Superpower, a parser construction toolkit. Superpower simplifies the process of creating tokenizers and parsers. The first step is to install the NuGet package into our app from the terminal:

dotnet add package Superpower

Once installed, create a new file Stache/StacheParser.cs. In addition to the StacheParser class, I'm going to include other small classes in the same file. Feel free to create separate files for these if that makes better organizational sense to you. You can find the full contents in this GitHub gist.

We'll first create an enum to represent our token categories.

public enum Tokens
{
  String,
  OpenBraces,
  CloseBraces
}

Next, we'll create a tokenizer in the StacheParser class.

public static Tokenizer<Tokens> Tokenizer  = new TokenizerBuilder<Tokens>()
  .Match(Span.EqualTo("{{"), Tokens.OpenBraces)
  .Match(Span.EqualTo("}}"), Tokens.CloseBraces)
  .Match(Span.MatchedBy(Character.ExceptIn('{','}')).Many(), Tokens.String)
  .Build();

Basically we read the input and attempt to match a specific set of characters that map to a token. The Match statements will be tried in order until a match is found. Note that the above tokenizer will run into issues if there are single braces present in the template (e.g. {{foo{bar}}). We will not address this limitation in this post as it involves creating more advanced tokenizers as opposed to using the building blocks provided by Superpower.

The next step is to create a parser. Let's create a class called ParseNode to represent parsed elements. It will be an abstract class that's derived by three other classes: TextNode for literal text, ExpressionNode for a mustache expression that we'll need to replace, and DocumentNode for a top level structure to hold a list of the other two nodes.

public abstract class ParseNode 
{
}

public class TextNode : ParseNode
{
  public string Value { get; set; }
}

public class ExpressionNode : ParseNode
{
  public string Value { get; set; }
}

public class DocumentNode : ParseNode
{
  public List<ParseNode> Nodes { get; set; }
}

Superpower is a parser combinator so it allows us to combine simpler parsers to create a more complex one. In our case we will create a simple parser for text literals in the StacheParser class.

private readonly static TokenListParser<Tokens, ParseNode> LiteralParser =
  from str in Token.EqualTo(Tokens.String)
  select (ParseNode)new TextNode {
    Value = str.ToStringValue()
  };

The above parser will consume tokens that are of type String and returns a TextNode.

We will create another parser for mustache expressions.

private readonly static TokenListParser<Tokens, ParseNode> ExpressionParser = 
  from ob in Token.EqualTo(Tokens.OpenBraces)
  from str in Token.EqualTo(Tokens.String)
  from cb in Token.EqualTo(Tokens.CloseBraces)
  select (ParseNode)new ExpressionNode {
    Value = str.ToStringValue()
  };

This parser will consume three tokens in order: an open brace followed by a string followed by a close brace. It will return an ExpressionNode with the value of the String token since that represents the variable we need to replace.

Putting it all together, we have the combined parser.

public readonly static TokenListParser<Tokens, ParseNode> MainParser =
  from nodes in ExpressionParser.Or(LiteralParser).Many()
  select (ParseNode)new DocumentNode {
    Nodes = nodes.ToList()
  };

The combined parser looks for tokens to match either the expression parser or the literal parser. The Many() tells it to match as many times as possible so that we consume all the tokens and process the entire template. It returns a DocumentNode with a list of the nodes returned from the other two parsers. We now have our tokenizer and parser that we can use to render views!

Using our parser

In StachView.cs, we can replace the contents of RenderAsync() with the following:

var template = File.ReadAllText(Path);

var tokens = StacheParser.Tokenizer.Tokenize(template);
var parsedResult = StacheParser.MainParser.TryParse(tokens);

var processedOutput = new StringBuilder();

if(parsedResult.HasValue){
  var document = (DocumentNode)parsedResult.Value;

  foreach(var node in document.Nodes){
    switch(node){
      case TextNode textNode:
        processedOutput.Append(textNode.Value);
      break;
      case ExpressionNode expNode:
        processedOutput.Append(context.ViewData[expNode.Value]?.ToString());              
      break;
    }
  }
}else{
  throw new Exception(parsedResult.ErrorMessage);
}

return context.Writer.WriteAsync(processedOutput.ToString());

In the above code, we run the template through the tokenizer. The resulting tokens are then parsed giving us a DocumentNode structure. We then process each parsed node and directly write text nodes or find variable values in ViewData for expression nodes.

We are now able to use any variable in our view template and bind to it in our controller.

For example, we can change our template to:

<h2>{{header}}</h2>

<p>{{content}}</p>

<small>{{footer}}</small>

and the Bar action in HomeController.cs to:

public IActionResult Bar(){
    ViewData["header"] = "Hello World!";
    ViewData["content"] = "Greetings from a mustache template.";
    ViewData["footer"] = "Powered by ASP.NET Core.";

    return View();
}

Then running the app (dotnet run) and navigating to https://localhost:5001/Home/Bar you should see:

Hello World!

Greetings from a mustache template.
Powered by ASP.NET Core.

Summary

In this post we explored the process of building a more flexible view engine that can tokenize and parse the template syntax. We now have gotten rid of one of the limitations mentioned in the first post. Using the Superpower library we can design a template syntax that suits our needs. Stay tuned for the next post in which we'll explore supporting inline CSharp expressions such as {{1 + 1}}.

Cover photo by Paul Skorupskas via Unsplash

Alternative view engine for ASP.NET Core: Hooking into the ASP.NET Core architecture

elfalem — Fri, 09 Aug 2019 16:40:15 +0000

ASP.NET Core is a great open-source framework for building web apps. One of its advantages is its modular architecture, which makes it possible to replace many of the default components with alternative ones. In this post, we'll look at how we can create and use a new view engine with ASP.NET.

Creating a Razor View

Let's first create an ASP.NET Core MVC app with a Razor view. You'll need the .NET Core SDK. Using the terminal on your OS or in your favorite IDE, such as VS Code (a great, cross-platform IDE), create a project called Foo and initialize it with the default template for an MVC app.

mkdir foo
cd foo/
dotnet new mvc

You can now run the app with dotnet run and navigate to https://localhost:5001/ to access the home page displaying a welcome message.

Now we'll add an action named Bar to the Home controller as well a corresponding Razor view.

Edit HomeController.cs to add the following:

public IActionResult Bar(){
    ViewData["Message"] = "Hello World!";

    return View();
}

Create Views/Home/Bar.cshtml and add the following:

Your Message: @ViewData["Message"]

If you stop and run the app again, you should see the text Your Message: Hello World! when accessing https://localhost:5001/Home/Bar

An alternative view engine

As we saw above, the Razor view engine that comes with ASP.NET uses markup files with the extension .cshtml. The Razor syntax uses the @ symbol to transition to C# and evaluate expressions. As an alternative, what if we use the mustache syntax for markup files? We can use braces to denote variables that will need to be treated as C# expressions. In our case, the above Bar view will look like this in the new template system:

Your Message: {{Message}}

We'll use a similar convention as Razor for locating views - i.e. Views/[controller]/[action] and utilize the file extension .stache. Let's see how we can implement the Stache View Engine.

Creating a view engine

A view engine needs to implement two interfaces: IViewEngine and IView from the namespace Microsoft.AspNetCore.Mvc.ViewEngines. In our project, create a top level directory named Stache containing the files StacheViewEngine.cs and StacheView.cs. We'll walk through the key parts of these two interfaces below. You can find the full contents of the two files on this GitHub gist.

IViewEngine

IViewEngine requires implementing two methods:

public ViewEngineResult GetView (string executingFilePath, string viewPath, bool isMainPage);

public ViewEngineResult FindView (ActionContext context, string viewName, bool isMainPage);

Both methods return a ViewEngineResult which encapsulates ViewEngineResult.Found, indicating an instance of the view was found, and ViewEngineResult.NotFound, indicating that the view was not found and containing the list of locations searched.

It's helpful to discuss how a view can be referenced when it's returned from a controller to understand the roles these two methods play in the process of rendering a view. In our Bar controller, we just write return View(); which references the view with the same name as the action (i.e. Bar). We could have also directly named the view: return View("Bar");. Alternatively it's possible to explicitly provide the full path to a view file: return View("~/Views/Home/Bar.stache");. In all cases, the framework first invokes the GetView() method. If a view is not found, it will then invoke FindView().

In cases where the view path is explicitly given, we expect GetView() to return an instance of that view. In cases where we only have the view name, we fallback to FindView() to search various paths that we've defined by convention to be possible locations where a view could exist.

In our implementation of GetView(), we first check if viewPath is an actual file path or just the name of the action. We'll get the latter if the view is referenced by name only. In that case, we'll return ViewEngineResult.NotFound so the call can proceed to FindView().

If it's a file path, we'll pass it to a private method GetAbsolutePath() to ensure it's well formatted. executingFilePath will only have a value if there is another view that's currently executing (e.g. the parent view when processing a Razor partial view). If it has a value, it will get combined with the view path to generate a path relative to the parent executing view. Once we have the path, we check if a template exists at that path and return appropriately.

public ViewEngineResult GetView(string executingFilePath, string viewPath, bool isMainPage)
{
  if(string.IsNullOrEmpty(viewPath) || !viewPath.EndsWith(ViewExtension, StringComparison.OrdinalIgnoreCase)){
    return ViewEngineResult.NotFound(viewPath, Enumerable.Empty<string>());  
  }

  var appRelativePath = GetAbsolutePath(executingFilePath, viewPath);

  if(File.Exists(appRelativePath)){
    return ViewEngineResult.Found(viewPath, new StacheView(appRelativePath));
  }

  return ViewEngineResult.NotFound(viewPath, new List<string>{ appRelativePath});
}

For our implementation of FindView(), we're provided the view name as well as the action context from which we can get the controller name. Based on our convention for locating views, we'll plug in those two values into each of the _viewLocationFormats (e.g. Views/[controller]/[action].stache) and check for a template file until one is found. If none are found, we return ViewEngineResult.NotFound with all the paths we checked.

public ViewEngineResult FindView(ActionContext context, string viewName, bool isMainPage)
{
  if(context.ActionDescriptor.RouteValues.TryGetValue("controller", out var controllerName)){
    var checkedLocations = new List<string>();

    foreach(var locationFormat in _viewLocationFormats){
      var possibleViewLocation = string.Format(locationFormat, viewName, controllerName);

      if(File.Exists(possibleViewLocation)){
        return ViewEngineResult.Found(viewName, new StacheView(possibleViewLocation));
      }
      checkedLocations.Add(possibleViewLocation);
    }

    return ViewEngineResult.NotFound(viewName, checkedLocations);
  }
  throw new Exception("Controller route value not found.");
}

IView

IView requires implementing the Path property and the RenderAsync() method. Path is the location of the view template. We populate the Path property when instantiating the view in GetView() or FindView().

The RenderAsync method is responsible for rendering the view given a ViewContext and is where the magic happens. In this grossly oversimplified example, we just look for the string {{Message}} in the template and replace it with the value provided in the ViewData. In a more fleshed out version, this is where your custom syntax parser would do it's work.

public Task RenderAsync(ViewContext context)
{
  var template = File.ReadAllText(Path);

  var processedOutput = template.Replace("{{Message}}", context.ViewData["Message"]?.ToString());

  return context.Writer.WriteAsync(processedOutput);
}

Using the alternative view engine

Let's create an alternative Bar view using the new Stache template syntax. Create the file Views/Home/Bar.stache with the contents:

Your Message: {{Message}}

The last step is to tell MVC to use the new view engine. In Startup.cs modify the MVC configuration to be:

services.AddMvc()
    .SetCompatibilityVersion(CompatibilityVersion.Version_2_2)
    .AddViewOptions(options => {
        options.ViewEngines.Insert(0, new StacheViewEngine());
    });

Note that we're adding the Stache view engine as the first item in the list. This will ensure that the Stache template is used if both .cshtml and .stache markup files are found for a given view. If you'd like to default to Razor views instead, you can add Stache to the end of the list: options.ViewEngines.Add(new StacheViewEngine());.

If you run the app and navigate to the Bar action https://localhost:5001/Home/Bar, you should now see the view rendered with Stache!

Summary

In the above example, we saw that it's relatively simple to plug in an alternative view engine in place of Razor. It illustrates the possibility of using a variety of markup templates, even simultaneously, with ASP.NET.

Although we have the foundation on which can build upon, this rudimentary example is quite limited:

There is only one variable named Message that we can bind to. We should be able to use as many variables as needed for a given template.
We need support for more advanced expressions instead of just variables (e.g. inline expressions such as {{1 + 1}}), loops to iterate through lists, and if-else conditions for control flow.
We're missing some of the more advanced concepts of Razor such as layouts, partials and tag helpers.

It will be interesting to explore some of these missing features and develop our alternative view engine to implement them. Perhaps we'll do that in future posts.

Thanks to @sohjsolwin for providing feedback on this post.
Also a special shout-out to the blog entry "Creating a New View Engine in ASP.NET Core" by Dave Paquette for putting together some of the key puzzle pieces.
Cover photo by Eryk via Unsplash