Forem: Babu Annamalai

How to create unique file names in bash shell

Babu Annamalai — Wed, 12 Apr 2023 08:39:05 +0000

For debugging purposes, we had to profile a python program using cProfile which will create a .prof file with the profiled data. Also we wanted to create unique file names for each run using a bash shell.

We had couple of options:

Use Unix epoch time which is a number and can be suffixed to a file to make it unique.
Use uuidgen to create a GUID. Since it is for dev/debugging purposes, we wanted to shorten it and use only the last part of the GUID. I am quite aware that just using the last part may not be always unique, it is for some dev/debugging so it is okay for our purposes.

We will run through to how to generate a unique file name using both the methods below:

In the below script, we are using $(date +"%s") which provide the current date time as an epoch. The output is something like test-1681287886.prof

PROFILE_FILE=test-$(date +"%s").prof
echo ${PROFILE_FILE}

with the below script, we using a combination of uuidgen and piping it to cut to get the last part of GUID. The output is like test-5F4C3E649454.prof. With regards to the args for cut, -d "-" argument refers to using - as the delimiter and -f5 argument refers to getting the 5th field from the array after splitting it

PROFILE_FILE=test-$(uuidgen | cut -d "-" -f5).prof
echo ${PROFILE_FILE}

VitePress fenced code block syntax highlighting quirks with .NET or other languages.

Babu Annamalai — Wed, 12 Oct 2022 02:41:29 +0000

This post was originally published at https://mysticmind.dev/vitepress-fenced-code-block-syntax-highlighting-quirks-with-net-or-other-languages

VitePress uses Shiki for syntax highlighting which is a fantastic library. Shiki automatically fetches language grammar from VSCode so the language names etc for the fenced code block comes from it.

For .NET, the language names captured are csharp and fsharp by default. This means that the syntax highlighting will work only if you use csharp or fsharp. But idiomatic usage use the file extensions cs and fs for syntax highlighting so this tends to be an issue. I did send a PR to Shiki to resolve this. But there are many moving parts i.e. Shiki will need to do a release with the fix then VitePress picking it up and in turn do a release.

If you are stuck, not only with .NET languages or any other language as well, there is a easy fix under your control. Edit your VitePress config.js file and add the following:

import { BUNDLED_LANGUAGES } from 'shiki'

// Include `cs` as alias for csharp
BUNDLED_LANGUAGES
  .find(lang => lang.id === 'csharp').aliases.push('cs');

// Include `fs` as alias for fsharp
BUNDLED_LANGUAGES
  .find(lang => lang.id === 'fsharp').aliases.push('fs');

A fun little poem on frontend JS frameworks

Babu Annamalai — Fri, 08 Apr 2022 16:54:49 +0000

I took the popular JS frontend frameworks in The state of JS 2021 and wanted to write a little fun poem including most of them. Here is my take below:

Let your mind be Lit with Vue, Stimulus and Ember to Vite your way with Svelte in an Angular vortex. Alpine in the Solid woods will make you Preact but don't React. Eventually everything is a Remix. Eagerly waiting for what is Next. :-)

Note that there is no implied meaning to how the poem is structured.

.NET build tool using Bullseye and SimpleExec

Babu Annamalai — Wed, 23 Feb 2022 06:28:08 +0000

This post was originally published at https://mysticmind.dev/dotnet-build-tool-using-bullseye-and-simpleexec

NAnt was quite a popular build tool used by projects of all sizes targeting .NET Framework on Windows. I have written many complex build systems using NAnt in the past. But XML based DSL was quite clunky to use and maintain. Besides, it was always a daunting task to explain to a new dev on the team. With .NET Core/.NET becoming a cross-platform framework, CAKE and FAKE gained a lot of adoption providing a C# and F# based DSL for build tasks accordingly.

I personally advocate Bullseye in combination with SimpleExec as a build tool for .NET projects due to its sheer simplicity by providing a bare bones API to define the tasks and their dependent tasks in plain C# code. Both these libraries are build by Adam Ralph. It does not enforce any specific model or structure to write your build tasks. When someone runs through the build script, it is fairly straight forward to understand whats going on. We use this build tool setup for Marten for both local dev and CI build tasks and has worked well so far. Also many OSS projects have adopted it.

We will take a simple sample .NET project available in GitHub which has the following setup and then attempt to add a build script for it:

src/WebApp folder contains a .NET 6 web app with a Razor page to serve the index. The project uses minimal API.
src/WebApp/static/css/site.css contains the css file for the web app which is referenced in the Razor page
package.json at the root folder which has dev dependencies to esbuild and defines 2 scripts as below:
- minify - esbuild --minify ./src/WebApp/static/css/site.css --outdir=./src/WebApp/wwwroot/css
- watch - esbuild --watch ./src/WebApp/static/css/site.css --outdir=./src/WebApp/wwwroot/css
ESBuild is used to minify the static assets i.e. css files from src/WebApp/static/css and adds it under wwwroot/css

Now let us look at adding a build script using Bullseye and SimpleExec:

Add a build folder at the top level
Add a new console project under the build folder as below

dotnet new console

Add Bullseye and SimpleExec NuGet packages to build.csproj

dotnet add package Bullseye
dotnet add package SimpleExec

Open Program.cs to add the build targets in your favourite editor and add the following code (uses .NET 6 minimal API). The inline comments should be self explanatory.

using static Bullseye.Targets;
using static SimpleExec.Command;

// Create a build target for npm install to make esbuild available for use
Target("npm:install", () => Run("npm","install"));

// Create build target esbuild:dist target which will minify
// CSS under static/css and add it to wwwroot/css folder
// This build target has a depedency on `npm:install` build target

Target("esbuild:dist", DependsOn("npm:install"), () => Run("npm","run minify"));

// Create build target esbuild:dist target which will allow to
// watch for changes in dev env
// This build target has a depedency on `npm:install` build target
Target("esbuild:watch", DependsOn("npm:install"), () => Run("npm","run watch"));

// Create a build target Clean the web project output from previous build
Target("clean", () => Run("dotnet","clean src/WebApp/WebApp.csproj"));

// Create a build target to compile the web project
// This build target depends on `clean` build target
Target("compile", DependsOn("clean"), () => Run("dotnet","build ./src/WebApp/WebApp.csproj -c Release"));

// Create a build target to watch for changes in dev
Target("watch", () => Run("dotnet", "watch --project ./src/WebApp/WebApp.csproj"));

// Create a default target
// This target depends on `esbuild:dist` and `compile` build targets
Target("default", DependsOn("esbuild:dist", "compile"));

// run the build target requested and exit
await RunTargetsAndExitAsync(args);

At the top level folder, let us create build.sh with executable permission as below to be used in *nix environments to run a the build

#!/usr/bin/env bash

set -euo pipefail

dotnet run --project build/build.csproj -c Release -- "$@"

Create build.cmd with the following content to be used in Windows commandline

@echo Off
dotnet run --project build/build.csproj -- %*

Create build.ps1 with the following content to be used from PowerShell prompt

$ErrorActionPreference = "Stop";
dotnet run --project build/build.csproj -- $args

To run the default build target, we use the below:

./build.sh

You will see an output as below:

...
...
Time Elapsed 00:00:04.88
build: compile: Succeeded (5.66 s)
build: default: Succeeded
build: ──────────────────────────────────────
build: Target        Outcome    Duration
build: ────────────  ─────────  ─────────────
build: npm:install   Succeeded  868 ms  10.1%
build: esbuild:dist  Succeeded  206 ms  2.4%
build: clean         Succeeded  1.82 s  21.3%
build: compile       Succeeded  5.66 s  66.1%
build: default       Succeeded
build: ──────────────────────────────────────
build: Succeeded (default) (8.55 s)

To run any other build target, pass the target name as the parameter to the build file

# In this example, `esbuild:dist` is the build target
./build.sh esbuild:dist

You will see an output as below:

...
...
⚡ Done in 4ms
build: esbuild:dist: Succeeded (184 ms)
build: ──────────────────────────────────────
build: Target        Outcome    Duration
build: ────────────  ─────────  ─────────────
build: npm:install   Succeeded  1.58 s  89.6%
build: esbuild:dist  Succeeded  184 ms  10.4%
build: ──────────────────────────────────────
build: Succeeded (esbuild:dist) (1.76 s)

If you want to make changes with hot reload in dev, you would run the .NET project with watch using build script ./build.sh watch and also the ESBuild task for CSS ./build.sh esbuild:watch on two terminal windows and work with it. Build tool shortens the typing required to run things.

Bullseye generates a nice coloured output on the terminal and also adapts the colour profile according to the CI environments such as GitHub, GitLab etc.

If you are looking at a more full fledged build tool in action, you can take a look at the build tool setup for Marten here.

Happy coding :-)

GitHub action to publish .NET packages to NuGet

Babu Annamalai — Wed, 23 Feb 2022 06:19:57 +0000

This post was originally published at https://mysticmind.dev/github-action-to-publish-net-packages-to-nuget

I have been a co-maintainer of .NET OSS library Marten and largely involved in taking care of the DevOps stuff pertaining to CI pipeline, build and docs system. In the early days, the build was driven using a RAKE build script written by Jeremy Miller, our lead dev and creator of Marten. NuGet packages were published from the local dev machine after confirming that we had bumped up the versions of the packages in preparation for the release. Apparently, this being a manual step, we publish the NuGet packages but sometimes tend to forget tagging and creation of release in GitHub (you know how an OSS author life is around managing day job and doing open source work).

We were also using AppVeyor for CI during the early periods of the library and I managed to add a step at the end of the CI pipeline to do a NuGet publish, when a tag was created in GitHub. This completely removed any human error or any inadvertent omissions due to lack of time etc. This worked well for us other than the occasional NuGet publish failures due to expired API key. We had to jump into AppVeyor dashboard to see what was going on and fix things. Eventually, we migrated all the CI builds to use GitHub actions so that we can lookup things all in one-place without logging into different systems. This is a huge convenience and time saver for us. We settled on doing a manual GitHub action trigger (user invoked) especially for publishing NuGet packages rather than keep it automated so that we can inspect and keep an eye on the NuGet publish as it happens after we tag and add a release in GitHub.

Following is a sample of the GitHub action YAML script for publishing package to NuGet with detailed comments adapted from the one we use with Marten :

name: NuGet Manual Publish

# workflow_dispatch is used to manually invoke the GH action
on: [workflow_dispatch]

env:
  # Setting the required env flags
  config: Release
  DOTNET_CLI_TELEMETRY_OPTOUT: 1
  DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
  dotnet_core_version: 6.0.x

jobs:
  publish_job:
    # CI running on linux
    runs-on: ubuntu-latest

    steps:
      # This step clones the source code to the CI build machine
      - name: Checkout code
        uses: actions/checkout@v2

      # This step installs the .NET SDK
      - name: Install .NET 6
        uses: actions/setup-dotnet@v1
        with:
          dotnet-version: ${{ env.dotnet_core_version }}

      # Run dotnet pack to create the nupkg file for the project and store in artifacts folder
      - name: Run Pack
        run: dotnet pack src/proj/proj.csproj -o ./artifacts --configuration ${{ env.config }}
        shell: bash

      # Find all the created nupkg files and publish it to NuGet, we use the wonderful swiss-army knife capability `find -exec` to find and then execute an action on it.
      # NUGET_DEPLOY_KEY is the NuGet API key stored in the repo GH action secrets
      # If you also publish symbol packages, find the snupkg files and publish it to NuGet
      - name: Publish to NuGet
        run: |
          find . -name '*.nupkg' -exec dotnet nuget push "{}" -s https://api.nuget.org/v3/index.json -k ${{ secrets.NUGET_DEPLOY_KEY }} --skip-duplicate \;
          find . -name '*.snupkg' -exec dotnet nuget push "{}" -s https://api.nuget.org/v3/index.json -k ${{ secrets.NUGET_DEPLOY_KEY }} \;
        shell: bash

You can take look at the current version of GH action used for publishing NuGet packages in Marten here.

Stay tuned, I will be writing more posts detailing the build scripts, CI build systems (GitHub Actions, Azure Pipelines, AppVeyor), matrix builds which we use to run integration tests on various versions of Postgres based on my experiences building and maintaining these for Marten. Also will ramble on various other topics on Marten, Postgres, jsonb, Python, building desktop apps with Electron/Tauri, node.js, Tailwind CSS, alpine.js etc.

Good day :-)