Forem: Javier Salcedo

Writing a "Hello, Triangle!" with Metal 4 and exclusively C++

Javier Salcedo — Mon, 09 Feb 2026 22:39:25 +0000

This article was initially published here

This article will serve as both a tutorial for those who don't want to have to deal with ObjC, and as a demonstration of how I work with my own custom C++ build system "Dedalo".

The build system

Dedalo is a strongly opinionated C++ build system, inspired by Swift and Zig.
The goal is to have a build system that doesn't require anything other than a C++ compiler and C++ code to work, no need to install anything, nor to learn a different bespoke language.

You can install Dedalo by copying the binary and cpp file in the appropriate paths (or using make install), but you can also distribute it with your program, so you don't need to worry about versioning. It's just a single cpp file with no dependencies other than the STL. The only thing needed to build a C++ project with Dedalo is a C++ compiler that supports C++20.

Getting started

Create a new directory and place dedalo.cpp in there.
Compile it like this clang++ -std=c++20 -O3 dedalo.cpp -o ddl
1. If you want logs, add -DENABLE_LOGS
Initialise the project like this ./ddl init

You should now have a file structure like this:

├── build
│   ├── cache
│   │   └── lto
│   └── obj
├── build.cpp
├── ddl
├── dedalo.cpp
├── lib
└── src
    └── main.cpp

If you run it like this ./ddl run you should get a "Hello, World!" message.

Sorting out the dependencies

GLFW

To keep it simple, I'll be using GLFW for the window management. Please refer to their website to see how to install it.
Since it's a dynamic system library (in my case in /usr/local/include/GLFW), adding it to the project is very easy:

// build.cpp
void build( Project* project, const MainArgvSlice args )
{
    assert( project );

    *project = Project(
    {
        .name = "metal-4-only-cpp",
        .dependencies =
        {
            { "glfw" }
        }
    });
}

Apple Frameworks

The official metal-cpp headers are incomplete, but there're workarounds. Just get this repo and copy it into ./lib:

├── dedalo.cpp
├── ddl
├── build.cpp
├── lib
│   ├── AppKit
│   ├── Foundation
│   ├── Metal
│   ├── MetalFX
│   ├── MetalKit
│   └── QuartzCore
└── src
    └── main.hpp

Now just add them to the build.cpp, and disable the gnu-anonymous-struct and nested-anon-types warnings:

// build.cpp
*project = Project(
{
    .name = "metal-4-only-cpp",
    .dependencies =
    {
        { "glfw" }
    },
    .frameworks =
    {
        { "AppKit" }, { "Foundation" }, { "QuartzCore" }, { "Metal" }
    },
    .common_compiler_flags =
    {
        // metal-cpp needs these
        "Wno-gnu-anonymous-struct",
        "Wno-nested-anon-types"
    },
});

Hello, Triangle!

From now on, I'll use some custom pieces of code that I won't explain here but you can find in the utils.hpp file, like a defer macro or renaming some types.

For brevity, I'm also going to assume no errors can happen, so I won't check or assert results.

After initialising the Metal device and creating the window, I'll be following this article from Apple, but using C++ instead of ObjC.

Creating the window

This is the same as with any other GLFW app. Just remember to define GFW_EXPOSE_NATIVE_COCOA:

#define GLFW_INCLUDE_NONE
#include <GLFW/glfw3.h>
#define GFW_EXPOSE_NATIVE_COCOA
#include <GLFW/glfw3native.h>

#include "utils.hpp"

static constexpr u32 WIN_WIDTH  = 800;
static constexpr u32 WIN_HEIGHT = 600;

i32 main( i32 argc, char* argv[] )
{
    glfwInit();

    glfwWindowHint( GLFW_CLIENT_API, GLFW_NO_API ); // Because GLFW wasn't designed to work with Metal
    GLFWwindow* glfw_window = glfwCreateWindow( WIN_WIDTH, WIN_HEIGHT, "Metal Only C++", nullptr, nullptr );
    defer( glfwDestroyWindow( glfw_window ) );

    while( !glfwWindowShouldClose( glfw_window ) )
    {
        glfwPollEvents();
        // ...
    }
}

If you run the program now you should get an empty window:

Initialising Metal

Creating the Metal device and layer

We need the device to issue commands, and the layer to connect with the GLFW window.

// ...
#define MTL_PRIVATE_IMPLEMENTATION
#define NS_PRIVATE_IMPLEMENTATION
#include "Metal/Metal.hpp"

#define CA_PRIVATE_IMPLEMENTATION
#include "QuartzCore/QuartzCore.hpp"
// ...
static constexpr auto MAX_FRAMES_IN_FLIGHT = 3u;
static constexpr auto PIXEL_FORMAT         = MTL::PixelFormatBGRA8Unorm_sRGB;
// ...
    MTL::Device*    device      = MTL::CreateSystemDefaultDevice();
    CA::MetalLayer* metal_layer = CA::MetalLayer::layer();
    {
        metal_layer->setDevice( device );
        metal_layer->setPixelFormat( PIXEL_FORMAT );
        metal_layer->setMaximumDrawableCount( MAX_FRAMES_IN_FLIGHT );
    }
    // ...
    // Create the GLFW window
    // ...
    {
        auto* metal_window = reinterpret_cast< NS::Window* >( glfwGetCocoaWindow( glfw_window ) );
        NS::View* view = metal_window->contentView();
        view->setLayer( metal_layer );
        view->setWantsLayer( true );
    }
// ...

Set up the GLFW window to work with Metal

Now that we have both the window and the Metal device/layer, we can link them together so Metal knows where to present.
To do that, we need to get the view of the GLFW window's underlying Cocoa window:

auto* metal_window = reinterpret_cast< NS::Window* >( glfwGetCocoaWindow( glfw_window ) );
NS::View* view = metal_window->contentView();
view->setLayer( metal_layer );
view->setWantsLayer( true );

Command Queue, Buffer and Allocators

Metal 4 changes a bit how commands work:

You can watch this video to learn more.

In Metal 3, command buffers were created from their queues, but in Metal 4 you can create them independently and commit multiple of them at once to a queue later.
They can also be reused, which is why it's a single member variable of the renderer and not a temporary object created per frame like on Metal 3.

Metal 4 also introduces command allocators, that manage the memory necessary for the encoded commands.
While the buffer can be reused, each frame needs its own allocator, so we'll make an array of 3, 1 for each frame in flight.

// In the Renderer struct/class
MTL4::CommandQueue*  cmd_queue  = nullptr;
MTL4::CommandBuffer* cmd_buf    = nullptr;
Array< MTL4::CommandAllocator*, MAX_FRAMES_IN_FLIGHT > cmd_allocators;

// In the initialisation:
cmd_queue = device->newMTL4CommandQueue();
cmd_buf   = device->newCommandBuffer();
for( auto& alloc: cmd_allocators )
{
    alloc = device->newCommandAllocator();
}

Frame Synchronisation and Presentation

Each frame, before getting the appropriate per-frame objects (vertex buffer, command allocator, ...), we need to make sure the GPU has finished work with at least one of the frames in-flight.
To do that we'll use a Shared Event, an object used to synchronise operations between the CPU and the GPU (like Vulkan Fences).

We need to keep the event itself, and also a count of all the frames rendered so far:

// In the Renderer struct/class
MTL::SharedEvent*    frame_available_shared_event = nullptr;
size_t               frame_num                    = 0;

The even can be created like this:

frame_available_shared_event = device->newSharedEvent();
frame_available_shared_event->setSignaledValue( 0 );

Shared Events work this way:

The GPU sets a predetermined value (signalling) once the work is done. We do this on the command queue after we've finished encoding command buffers:

// ... encoding command buffers ...
// ... presenting drawable ...
cmd_queue->signalEvent( frame_available_shared_event, frame_num )
frame_num++;
// Done for this frame

The CPU waits until the event has that value. Because we have 3 frames in flight, we wait for the one 3 frames ago to be done. We do this before we start encoding commands.

frame_available_shared_event->waitUntilSignaledValue(
    frame_num - MAX_FRAMES_IN_FLIGHT,
    timeout_in_ms );

// Now we can safely access this frame's triple-buffered resources
const u8 frame_idx = frame_num % MAX_FRAMES_IN_FLIGHT;
MTL4::CommandAllocator* cmd_alloc = cmd_allocators[ frame_idx ];
// ...

Before we can present the results of a frame, we first need to get the next drawable, and the queue will have to explicitly synchronise with it. This is a bit more verbose than Metal 3, where you could just encode a presentDrawable command directly on the command buffer.
But it's not that complex either.

MTL::Drawable* surface = metal_layer->nextDrawable();

// According to the documentation you *must* do it in this exact order
cmd_queue->wait( surface );
cmd_queue->commit( &cmd_buf, 1 )
cmd_queue->signalDrawable( surface );
surface->present();

If you run it now it would crash because we haven't set up the command buffer.
Just so we can get something on the screen, we'll set it up with no actual commands encoded.
The command buffer(s) frame workflow is pretty straightforward, you begin it, encode commands in it, and end it:

cmd_buf->beginCommandBuffer( cmd_alloc );
// We'll encode commands here later...
cmd_buf->endCommandBuffer();

Now, if you compile and run it, you should get a fully opaque black window:

Encoding commands

Render pass and encoder

A Render Pass is essentially a group of commands that share the same outputs (attachments).
You can configure a bunch of settings and fixed function operations for each of the attachments, but let's get started by clearing the surface with a given color (magenta in this example):

// First we create the pass descriptor...
auto* pass_desc = MTL4::RenderPassDescriptor::alloc()->init();
defer( pass_desc->release() );

// ...then we setup its first color attachment
MTL::RenderPassColorAttachmentDescriptor* color_attachment = pass_desc->colorAttachments()->object( 0 );
color_attachment->setTexture( surface->texture() );
color_attachment->setClearColor( MTL::ClearColor::Make( 1.0, 0.0 , 1.0, 1.0 ) );

We also need to tell Metal what to do before (load) and after (store) it starts rendering to this attachment.
We just want to clear it to the given color and then store it to be used in future passes, but there're other options if you need to preserve the previous contents, or if you don't care about them:

color_attachment->setLoadAction ( MTL::LoadActionClear  );
color_attachment->setStoreAction( MTL::StoreActionStore );

Finally we need to encode the pass into the command buffer. To do that we'll need Command Encoders. They configure the render pipeline, and set up resources for draw calls.
From the docs:

The most important difference with Metal 4 encoders is that they don’t have methods that bind individual buffers, textures, and heaps. Instead, you configure the resource bindings in an argument table and then bind that table to one or more pipeline stages with a command encoder

There're more differences specific for render command encoders, but I won't get into those here because they're not relevant for a simple "Hello, Triangle!".

We'll create the encoder and leave it empty for now:

MTL4::RenderCommandEncoder* encoder = cmd_buf->renderCommandEncoder( pass_desc );
defer( encoder->endEncoding() );

Once we're done with that, we need to end the command buffer, commit it to the queue, and present it; like we already did in the "Frame Synchronisation" section.

If you run it now you should get a magenta window:

The Pipeline and Shaders

The Metal 4 Compiler

Metal 4 introduces a new class that can compile both shaders and PSOs: MTL4::Compiler.
We only need it when we build the PSO, so (for this example) it doesn't need to be a member or anything:

// In the initialisation:
MTL4::Compiler* compiler;
{
    auto* compiler_desc = MTL4::CompilerDescriptor::alloc()->init();
    defer( compiler_desc->release() );

    compiler = device->newCompiler( compiler_desc, nullptr );
}

The Shaders and Library

For now we'll hardcode the vertex positions and colors in the shader.

// src/shaders/triangle.metal

#include <metal_stdlib>

using namespace metal;

constant float4 positions[3] =
{
    {  0.0,  0.5, 0.0, 1.0 },
    {  0.5, -0.5, 0.0, 1.0 },
    { -0.5, -0.5, 0.0, 1.0 }
};

constant float4 colors[3] =
{
    { 1, 0, 0, 1 },
    { 0, 1, 0, 1 },
    { 0, 0, 1, 1 }
};

struct VertexOut
{
    float4 position [[position]];
    float4 color;
};

vertex
VertexOut vertex_shader( uint id [[vertex_id]] )
{
    return
    {
        .position = positions[ id ],
        .color    = colors[ id ]
    };
}

fragment
float4 fragment_shader( const VertexOut in [[stage_in]] )
{
    return in.color;
}

You can compile it like this:

# compile individual shader
$ xcrun metal -c src/shaders/triangle.metal -o build/bin/shaders/triangle.air
# link into a library
$ xcrun metal -o /build/bin/shaders/shaders.metallib build/bin/shaders/*.air

I added it to the build.cpp as a post build script.

You could also use the new MTL4::Compiler we created before to compile the shaders on runtime (learn more here), but I prefer to compile them offline.

A shader library contains the project's shaders, now it's the time to create one. Many tutorials use the "default library", but that requires using Xcode bundles and other complications that I'm actively trying to avoid here.

// In the Renderer struct/class
MTL::Library* shader_lib = nullptr;
// In the initialisation:
shader_lib = device->newLibrary( ns_str( "build/bin/shaders/shaders.metallib" ), nullptr );

On Metal 3 you could now create MTL::Function objects from the library to pass them to the PSO descriptor.
Metal 4 does things slightly differently. The new PSO descriptor requires MTL4::LibraryFunctionDescriptor that have the library as a member:

// Metal 3
auto* vert_shader = shader_lib->newFunction( ns_str( "vertex_shader" ) );
auto* frag_shader = shader_lib->newFunction( ns_str( "fragment_shader" ) );

// Metal 4
auto* vertex_fun = MTL4::LibraryFunctionDescriptor::alloc()->init();
vertex_fun->setLibrary( shader_lib );
vertex_fun->setName( ns_str( "vertex_shader" ) );

auto* fragment_fun = MTL4::LibraryFunctionDescriptor::alloc()->init();
fragment_fun->setLibrary( shader_lib );
fragment_fun->setName( ns_str( "fragment_shader" ) );

Metal 3 was nicer to use imo, but I'm guessing the reasoning behind the new system is so everything is done at once by the MTL4::Compiler when it creates the PSO.

The PSO

We now have everything we need to create the PSO.
It's pretty straightforward, and the only difference from Metal 3 is that we now do it through the compiler instead of through the device:

// In the Renderer struct/class:
MTL::RenderPipelineState* pso = nullptr;

// In the initialisation:
auto* desc = MTL4::RenderPipelineDescriptor::alloc()->init();
defer( desc->release() );
desc->setLabel( ns_str( "Hello Triangle PSO" ) );
desc->colorAttachments()->object( 0 )->setPixelFormat( PIXEL_FORMAT );
desc->setVertexFunctionDescriptor( vertex_fun );
desc->setFragmentFunctionDescriptor( fragment_fun );

pso = compiler->newRenderPipelineState( desc, (MTL4::CompilerTaskOptions*)nullptr, (NS::Error**)nullptr );

You can also provide a MTL4::CompilerTaskOptions with an array of archives that can be used to speed up compilation, but I'm skipping that for this example.

DRAW!

We are ready!
Head back to the command encoder, set the PSO, and add the drawcall:

encoder->setRenderPipelineState( pso );
encoder->drawPrimitives( MTL::PrimitiveTypeTriangle, 0, 3 );

I also changed the clear color to a calmer grey.

If you now run it you should get something like this:

BONUS: Using a proper Vertex Buffer

So far we've used a hardcoded vertex buffer in the shader.

An important caveat is that Metal 4 treats vertex buffers as any other buffer. There's no encoder->setVertexBuffer().

We'll need 3 buffers, one for each frame in flight, although we could use just 1 for this example because they won't change.
But before we get there I need to explain 2 concepts: Argument Tables and Residency Sets.

Argument Tables

Argument Tables are new (at least as something exposed to the programmer) in Metal 4, and are essentially a list of the resource binding points that an encoder needs per stage (they can be reused and shared across encoders). For bindless, they'll probably have just 1 buffer.
For this, all we need is a single table with 1 buffer (the vertex buffer):

// In the Renderer struct/class
MTL4::ArgumentTable* arg_table  = nullptr;

// In the initialisation:
{
    // Personally, I'm not a fan of allocating a *descriptor*, but wcyd?
    auto* desc = MTL4::ArgumentTableDescriptor::alloc();
    defer( desc->release() );
    desc->setMaxBufferBindCount( 1 );
    // For the sake of simplicity I'm completely ignoring error management
    arg_table = device->newArgumentTable( desc, /* error */ nullptr );
}

Residency Sets

To make sure a resource is available for the GPU before using it you can manually make it resident (useResource and useHeap), but that has an overhead that builds up quickly.
Residency sets on the other hand can make them all resident at the same time, with less cost.
You can populate/update them at any point, and then you can attach them to a command buffer or the whole queue.
The driver will then make all the resources resident at once when you call commit on the command buffer (unless you explicitly request ahead-of-time residency).
It's a pretty neat feature, you can learn more about it here.

For now, let's simply create a long-term (meaning resources in this set will stay resident for the whole lifetime of the application) set:

// In the Renderer struct/class
MTL::ResidencySet* residency_set = nullptr;

// In the initialisation:
auto* desc = MTL::ResidencySetDescriptor::alloc();
residency_set = device->newResidencySet( desc, nullptr );
desc->release();

After adding resources to the set (which we'll do in a minute), we need to commit it, and add it to the command queue.
We should also add the Metal layer's residency set, which is managed by AppKit.

// In the initialisation:
residency_set->commit();
cmd_queue->addResidencySet( residency_set );
cmd_queue->addResidencySet( metal_layer->residencySet() );

Creating the Vertex Buffer(s)

Now that we have the argument table and residence set, we can create the vertex buffers.
As I mentioned before, Metal 4 doesn't have a setVertexBuffer(), they are treated as regular buffers.
We'll create them with the Shared storage mode so we can write to them directly from the CPU.
We also need to add them to the residency set so the driver can make sure everything is available before it starts drawing.

WARNING
Because for this example these buffers won't change, I'll just populate and add them to the residency set and argument table during initialisation. However, in a normal use case we would do it before the draw call, and/or manage the argument table and residency set differently.

static constexpr u8 VERTEX_BUFFER_BINDING_IDX = 0;

struct Vertex
{
    vec4 position;
    vec4 color;
};

static constexpr Array< Vertex, 3 > triangle_vertices
{
    Vertex{ .position = {  0.0,  0.5, 0.0, 1.0 }, .color = { 1, 0 ,0, 1 } },
    Vertex{ .position = {  0.5, -0.5, 0.0, 1.0 }, .color = { 0, 1, 0, 1 } },
    Vertex{ .position = { -0.5, -0.5, 0.0, 1.0 }, .color = { 0, 0, 1, 1 } }
};

// In the Renderer struct/class
Array< MTL::Buffer*, MAX_FRAMES_IN_FLIGHT > vertex_buffers;

// In the initialisation
for( u8 i = 0; i < vertex_buffers.size(); ++i )
{
    vertex_buffers[i] = device->newBuffer(
        /* length  */ sizeof( Vertex ) * triangle_vertices.size(),
        /* options */ MTL::ResourceStorageModeShared );

    const String name = fmt( "Vertex Buffer for frame #{}", i );
    vertex_buffers[i]->setLabel( ns_str( name.c_str() )  );

    // Populate the vertex buffer
    memcpy(
        vertex_buffers[i]->contents(),
        triangle_vertices.data(),
        sizeof( Vertex ) * triangle_vertices.size() );

    residency_set->addAllocation( vertex_buffers[i] );
    arg_table->setAddress( vertex_buffers[i]->gpuAddress(), VERTEX_BUFFER_BINDING_IDX );
}

Then, each frame before encoding the draw call, we set the argument table:

// Renderer::render_frame()
MTL4::RenderCommandEncoder* encoder = cmd_buf->renderCommandEncoder( pass_desc );
// ...
encoder->setArgumentTable( arg_table, MTL::RenderStageVertex );
// ...
encoder->drawPrimitives( MTL::PrimitiveTypeTriangle, 0, 3 );

Finally, we make the appropriate changes to the vertex shader:

// These two could be defined in a header file included by both the Renderer and the shader
static constant uint VERTEX_BUFFER_BINDING_IDX = 0;
struct VertexIn
{
    float4 position;
    float4 color;
};

struct VertexOut
{
    float4 position [[ position ]];
    float4 color;
};

vertex
VertexOut vertex_shader(
    uint id [[ vertex_id ]],
    constant VertexIn* vertices [[ buffer( VERTEX_BUFFER_BINDING_IDX ) ]] )
{
    return{
        vertices[ id ].position,
        vertices[ id ].color };
}

I ❤️ Offline

Javier Salcedo — Mon, 01 Sep 2025 22:31:40 +0000

Let me start by saying this blogpost doesn’t come from a place of hate. I don’t hate online tools, I don’t hate media streaming. Many times they’re just more convenient, and I use them a lot (I even had a Stadia back in the day).
But I just love not needing it. That feeling of independence, of not having to ask a server for permission to access your own stuff, is why I went back to buy physical media.
It’s faster to access, it doesn’t require having a power and memory hungry program like a modern web browser constantly open, and it works even when I’m on a plane, or somewhere with spotty or slow internet.
The main problem is that you need to know what you'll need in advance.

However, sometimes what you need is mainly online, or simply my terminally online brain struggles to conceive that you don't need a web browser for that. So it always feels really nice to discover new tools that allow you to access some resources offline.
Here are some of my favourites.

Notes and TODOs

The weird thing is that we accepted online-first or even online-only note taking apps.
I used to be a huge fan of Trello and later Notion, but their online-first nature ended up getting in the way.
Nowadays I just use a very simple system of templated Markdown files. I'm even considering trying out Org-mode (outside emacs, I'm a vim type of guy).

Documentation

Man pages

Good old man pages. That's it. They nailed it back in the 70s.
The navigation could be improved, but they're fast and exhaustive. They were created in a time before the Internet after all (why, they even predate C!)
Depending on your platform, there might be GUI apps to navigate them a bit more comfortably, like Yelp on Linux, or Man Reader on macOS, but TUI pagers like less or even vim do a good enough job imho.

I also really like the cppman project, which allows you to download and navigate cppreference in a similar manner.

Kiwix

I recently discovered Kiwix, which is meant for offline Wikipedia access but also has a bunch of documentation for APIs, tools and languages like Vulkan, CMake, Python, etc.
You can even download the whole of StackOverflow (80GB at the time of writing).

macOS' 1st party apps

If you're on macOS, Apple’s own Developer app allows you to download the excellent educational videos from past WWDC.
Xcode also includes an offline version of all the Apple frameworks’ documentation.

Blogs & Websites

I use GoodLinks to save and synchronise articles across all my devices using iCloud.
It’s a paid app but it’s absolutely worth it.
I used to be one of the dozens of Pocket fans, until Mozilla killed it. Same with Omnivore. 🥀🪦

For other pages that might not translate so well to the “Reader View”, I use the “Reading List” in Safari (or lately on Orion). It allows you to download the whole page as it is.
I'm not sure if there's a similar feature for Chromium or Firefox, but you can always print the page as a PDF!

One must imagine Sisyphus writing a new JS framework

Javier Salcedo — Fri, 27 Sep 2024 19:51:33 +0000

We inherited incredibly complex systems built by the greatest minds of previous generations.

With the fatal arrogance of a teenager that got their heart broken for the first time and thinks no one understands them or has ever felt what they feel, we saw their limitations, pitfalls and compromises, and attribuited them to incompetence or even malice. And promptly dedicated our efforts to reinvent the wheel, because we knew better.

We added layer upon layer of complexity, and each time a new model, theory or framework failed us, rather than reevaluating and stepping back, we added a new one. A patch to fix the uninteded consequences of the previous patch... ad aeternum.

And a new bureaucratic class of scrum masters, and agile coaches, and productivity gurus was born to manage all of that.

And then things started to break down.

What used to be done fast and cheaply now takes orders or magnitude longer, with much bigger teams, and somehow it feels lower quality.
We have much greater resources at our disposal and yet we can't seem to achieve things that we already did decades ago.
And in an schizophrenic combination of learned helplessness and god complex we deceive ourselves, coming up with rationalizations about how, actually, it can't and it shouldn't be done like that.

We're doing great!

Our latest self-analysis shows an improvement of N%!

Each year/version we're sweeping the floor with the last!

The old ways aren't feasible nowadays! It was unsafe! Systems are too complex now!

(Also, when so many people suffer of impostor syndrome, something is not right)

But people start to notice anyway, and then new factions started to emerge.

Reject modernity! Embrace C99 and Emacs!

No! That wasn't really applying the methodology of my ambiguous book! We need just another layer of abstraction!

The End is NOT nigh

They're all wrong. I say we're actually in a golden age of computing.

At some point, each generation thinks they have it uniquely difficult. Even Socrates (allegedly) believed the youths of his time were proof of the unique decay of his society.

True, there're issues. Shitty code and bloated, buggy projects are everywhere nowadays, but so is knowledge. It's in our hands to build something better.

We have access to computers orders of magnitudes more powerful than the mainframes of the olden days, and dirt cheap.
In fact, so cheap, I've seen homeless people in the park with laptops.

You can install a light weight Linux distro (free of course) on almost any old piece of junk with a CPU and you'll get a perfectly good programming machine.

You can choose from probably hundreds of programming languages. Or you can use top quality C/C++ compilers for free (that wasn't a thing back in the day!).

There's also an incredibly wide selection of free and powerful tools.
We even have lengthy discussions about what free editor or IDE is best.

You can host your code for free and even allow others to review and modify it in places like GitHub or GitLab.
They even have CI/CD tools for free as well!
And you can use it to read other people's code and learn from it. The source code of Doom is available for you to look around!

And, thanks to the modern Internet we have free and instant access to programming books, talks, conferences, classes and much more!

Harvard publishes CS lectures for free on YouTube
So does the University of Utah about Computer Graphics
You have courses on a lot of different languages and technologies in W3Schools for free!
I've also heard there're hidden websites in the dark corners of the interwebs where you can get free pdf copies of books...

We have free online communities full of people happy to discuss and teach.

"But I'm a single little dev. What change can I make?"

You're not alone!

Technology has democratised the production of many things. Empowering tiny teams to compete with (and even beat) multinationals.
We're seeing the same phenomenon all over the place:

Record companies produce bland and repetitive songs from cloned singers.
But nowadays anyone can produce an album in their bedrooms. You just need a laptop and a cheap sound card (and maybe not even that).
And places like Soundcloud are full of awful music from amateurs as proof of that.
But some of those amateurs improve, and grow, and now you have fresh, original, and incredibly niche bands that would've never been able to make a living with their music in years past.

The film and TV industries are producing increasingly mediocre and boring movies and series.
But some nerdy kids with a laptop webcam have built multimedia empires because now anyone can have access to cheap recording equipment and video editing software.

The same happened with video games thanks to projects like Unity or Blender.

We even have more people writing and selling books than ever before!

And YOU can do the same for software.

Big companies are the main ones producing all that bloated buggy code. They grew too big and fat, with tons of legacy code. Now they move slowly and can't take any risks.
The democratisation of code opened the flood gates and a lot of shitty code and shitty devs poured in, but so did a lot of talented devs that just need some experience. And experienced ones that just needed a push.

Get inspired!

Don't like having to use gigantic game engines to make games? Here's a whole suite of beautifully simple libraries with bindings for a lot of languages: raylib
VisualStudio is too slow and bloated? Write your own IDE! 10x
Debugging C++ is painfully slow? How about a super fast, tiny and portable debugger? remedybg?
You crave the simplicity of C, but miss modern features and a nice build system? You got Zig
Or maybe you want a language that's more geared towards game dev, and includes a lot of the commonly use libraries? Odin
Websites are too complex nowadays? Why not trying to make one using only text? Monospace Web
You don't like the licensing of your OS? BUILD YOUR OWN! Linux

These were all built (or at least created) by single devs, passionate for their craft. And there are many other examples.
I personally like the collection of the Handmade Network.

I love the concept of "recreational programming" pushed by Tsoding.
And people like ThePrimeagen, and Low Level Learning not only demystify a lot of the scary parts of software, they also have a contagious passion about it.

Now, some of these people are very opinionated about the right way to write software. I tend to agree with them, but that's not the point of this article.
Forget about the right way. Just build stuff! Build crappy software. Refine it, tweak it, learn from it, improve. Find something you're excited about, or a piece of software you don't like, and write code.
Don't let the doomsayers scare you, but remember to put love and care into it.

One must imagine Sisyphus doing a (free) operating system (just for fun).

Simple "Infinite" Grid Shader

Javier Salcedo — Mon, 03 Jun 2024 20:25:12 +0000

I tried to follow a "recipe" for an infinite grid shader, but after hitting a bug and not being able to fix it because I didn't understand the code, I decided to write it from scratch myself so I could understand it properly.

In the process I found better implementations than the original one I was following, and better than my own.

In the end I came up with a different, simpler approach, although more limited in some aspects, that ticks all my boxes at the moment.

You can see the code here.
Here's a detailed step-by-step explanation of how it works.

⚠️ DISCLAMER 1:
The goal of this article is just to document and organise my thought process while I was trying to understand the code behind another implementation.
If you want something robust / production-worthy, you should read this instead.

⚠️ DISCLAIMER 2:
I'm using MSL (Metal Shading Language) because I like it and because my toy renderer uses Metal as the graphics API anyway, but it should be pretty straightforward to translate it to GLSL or HLSL.

⚠️ DISCLAIMER 3:
This article will only focus on the shader part, but the setup in the host (CPU) side should be very simple regardless of API:

Set everything up to render a quad (2 triangles)

Enable depth testing

Disable face culling (we want to be able to see it from bellow too)

Because it's transparent it should be rendered after the opaque objects

1. Finite version

1.1 Vertex Shader



static constant auto grid_size = 1.0f;
static constant float4 positions[4]{
    { -0.5, 0.0,  0.5, 1.0 },
    {  0.5, 0.0,  0.5, 1.0 },
    { -0.5, 0.0, -0.5, 1.0 },
    {  0.5, 0.0, -0.5, 1.0 } };

struct VertexOut
{
    float4 position [[ position ]] [[ invariant ]];
    float2 coords;
};

vertex
VertexOut main(
    uint id [[vertex_id]],
    float4x4 view_proj [[buffer(N)]])
{
    auto world_pos = positions[id];
    world_pos.xyz *= grid_size;

    return{
        .position = view_proj * world_pos,
        .coords = world_pos.xz };
}

This will create a quad of size 1x1 centred at the origin.

If we output the normalised coordinates in the fragment shader...



using FragmentIn = VertexOut;

fragment
float4 main(FragmentIn frag [[stage_in]])
{
    auto c = float4(frag.coords/grid_size + 0.5, 0.0, 1.0);
    return c;
}

...you'll get something like this:

1.2 Fragment Shader

1.2.0 Modulo

⚠️ IMPORTANT in case you're using MSL or HLSL

GLSL defines its modulo function as x - y * floor(x/y)
which produces this type of repeating pattern:

However, both MSL and HLSL define it as x - y * trunc(x/y),
which produces this:

Since I'm using MSL, I'll add a template function that mimics GLSL's mod:



template<typename T, typename U>
constexpr T mod(T x, U y) { return x - y * floor( x / y ); }

For HLSL you'll have to rely on macros:



#define mod(x,y) ((x) - (y) * floor((x)/(y)))

EDIT: From HLSL 2021 you can also use templates.

1.2.1 Draw the cells

We'll subdivide the grid into 2 types of cells:

Big 1x1m cells
Small 0.1x0.1 (sub)cells To make it clearer to see, I'm increasing the plane size to 2x2m ```C++

static constant auto grid_size = 2.0f;

static constant auto cell_size = 1.0f;
static constant auto half_cell_size = cell_size * 0.5f;

static constant auto subcell_size = 0.1f;
static constant auto half_subcell_size = subcell_size * 0.5f;


We start by calculating the coordinates inside the cells and subcells.

1. First, displace the plane coordinates so the world's origin is in a corner rather than in the middle of the (sub)cell.
2. Then get the coordinates inside the (sub)cell
```C++


auto cell_coords    = mod( frag.coords + half_cell_size,    cell_size    );
auto subcell_coords = mod( frag.coords + half_subcell_size, subcell_size );

If we normalise and output this as a colour, we'll get something like this:

Cell UVs	Subcell UVs

Next we calculate the distances in U and V to the (sub)cell's edge.
The coordinates we calculated before are in the [0, (sub)cell_size] range so:

First, we transform them into the [-(sub)cell_size/2, (sub)cell_size/2] range so the center of the (sub)cell has the (0,0) coordinates.
Then, the distance to the edge is simply the absolute of these new coordinates



auto distance_to_cell    = abs( cell_coords    - half_cell_size    );
auto distance_to_subcell = abs( subcell_coords - half_subcell_size );

Now it's time to draw the lines.
Comparing that distance to the (sub)cell line thickness, we determine if we should draw the line or not:



static constant auto cell_line_thickness    = 0.01f;
static constant auto subcell_line_thickness = 0.001f;

static constant auto cell_colour    = float4( 0.75, 0.75, 0.75, 0.5 );
static constant auto subcell_colour = float4(  0.5,  0.5,  0.5, 0.5 );

...

// In the fragment shader
auto color = float4(0);
if ( any( distance_to_subcell < subcell_line_thickness * 0.5 ) ) color = subcell_color;
if ( any( distance_to_cell    < cell_line_thickness    * 0.5 ) ) color = cell_color;

return color;

The line thicknesses are halved because only half the line is within a given (sub)cell, as the other half is in the neighbouring (sub)cell

However, this has obvious issues (I made the plane opaque and the lines yellow to make it clearer):

1.2.2 Get uniform lines

So the problem here is that, due to perspective, the coordinates can change drastically from one fragment to another, meaning that the exact coordinates that fall within the line might be skipped from one fragment to the next.

An easy solution is to increase the line width by how much the coordinates change across fragments.
Thankfully, we get a very handy tool to get that change: partial derivatives.
If you don't know what they are or how they work, I recommend reading this article.



auto d = fwidth(frag.coords);
auto adjusted_cell_line_thickness    = 0.5 * ( cell_line_thickness    + d );
auto adjusted_subcell_line_thickness = 0.5 * ( subcell_line_thickness + d );

auto color = float4(0);
if ( any( distance_to_subcell < adjusted_subcell_line_thickness ) ) color = subcell_color;
if ( any( distance_to_cell    < adjusted_cell_line_thickness ) )    color = cell_color;

return color;

And this is how it looks with dimensions 100x100m and the actual colour:

1.3 Fade out

This already looks pretty good, but you can see issues in the distance: mainly aliasing and Moiré patterns.
There's a lot of literature about how to fix these kind of issues, but I decided to take a different approach...

...just fade out the grid around the camera so you can't see them!

First, we need the camera position.
Let's make the necessary changes in the vertex shader:



struct VertexIn
{
    float4x4 view_proj;
    float4 camera_pos;
};

struct VertexOut
{
    float4 position [[ position ]] [[ invariant ]];
    float3 camera_pos [[ flat ]];
    float2 coords;
};

vertex
VertexOut main(
    uint id [[vertex_id]],
    constant VertexIn& vert_in [[buffer(N)]])
{
    auto world_pos = positions[id];
    world_pos.xyz *= grid_size;

    return {
        .position   = vert_in.view_proj * world_pos,
        .camera_pos = vert_in.camera_pos,
        .coords     = world_pos.xz };
}

Note the [[ flat ]]: this is an attribute that disables interpolation, so we have the same value for all fragments.

Of course, in the host you'll need to add the camera position to the buffer too.

After that we calculate the horizontal distance to the camera, and use that to interpolate:



static const auto max_fade_distance = 25.0f;

...

// In the fragment shader
float opacity_falloff;
{
    auto distance_to_camera = length(frag.coords - frag.camera_pos.xz);
    opacity_falloff = smoothstep(1.0, 0.0, distance_to_camera / max_fade_distance);
}

return color * opacity_falloff;

I've found that a 25m fade radius works pretty well at a camera height of 1m, but it's too small when the camera is very high, and too big if it's very low.
So the fade radius will now be adjusted by height, keeping that 1:25 ratio.



static constant auto height_to_fade_distance_ratio = 25.0f;

...

// In the fragment shader
auto fade_distance = abs(frag.camera_pos.y) * height_to_fade_distance_ratio;
opacity_falloff = smoothstep(1.0, 0.0, distance_to_camera / fade_distance);

However, if the camera gets very close to the plane, that radius will approach zero, so I added a minimum radius.
And, if the camera gets very high, you can see the shape of the quad, so I added a maximum.



static constant auto min_fade_distance = grid_size * 0.05f;
static constant auto max_fade_distance = grid_size * 0.5f;

...

// In the fragment shader
auto fade_distance = abs(frag.camera_pos.y) * height_to_fade_distance_ratio;
{
    fade_distance = max(fade_distance, min_fade_distance);
    fade_distance = min(fade_distance, max_fade_distance);
}
opacity_falloff = smoothstep(1.0, 0.0, distance_to_camera / fade_distance);

Et voilà!

2. Make it "infinite"

We're almost done, but at the moment you can (eventually) move outside of the plane.
Thankfully, it has a very easy solution, just move the plane and the UVs with the camera. Think of it as a moving treadmill.

In the vertex shader:



auto world_pos = positions[id];
world_pos.xyz *= grid_size;
world_pos.xz  += vert_in.camera_pos.xz;
}

An important detail is to ignore the Y component of the camera position. We only want the grid to move horizontally, it shouldn't change when you move vertically.

3. Conclusions and future work

Even though there're objectively better approaches, this solution has scratched my brain itch (for now) and provides a good enough gizmo for my purposes.

The process of reinventing the wheel was a bit frustrating at times, but I'm glad I did. I wouldn't be happy having a copy-pasted piece of code that I don't understand and thus can't debug.
I learn best by doing so it was a great exercise.

Also, the process of writing this article forced me to dissect the code to be able to provide step by step examples. That uncovered a series of bugs and misunderstandings in my renderer and the shader itself. As a result, the version explained here is significantly different from the first one I wrote.

In terms of future work, there's still a bunch of stuff to be done:

Fix the fade to black in the distance
Actually filter the grid instead of using that hacky fade
Colour the X/Z axis
Add a Y axis at 0,0?

However, at the time of writing this, neither sound very fun, and I have a loong list of other features that do.
At the end of the day, this is a toy project and its sole purpose is to have fun while I experiment and learn.

So I consider it done (for now).

📚 References

🌄 Rendering a Screen-space Skybox

Javier Salcedo — Thu, 27 Apr 2023 22:18:09 +0000

NOTE: This won't be an article that goes very deep into details, it's more a way to document how I implemented the skybox in my renderer. I use Metal, but it should translate to other APIs almost 1-to-1.

When I decided to implement a skybox on my renderer my first impulse was to follow the ol' reliable LearnOpenGL article on cubemaps, which essentially consists on rendering a giant cube around the camera.

It's a simple and very intuitive approach but I wanted something a wee bit fancier.
Also, this technique requires uploading a lot of data (all the cube's vertices, a new view matrix so it "stays in place" as you move the camera, etc) which wouldn't be necessary on a screen-space approach.

📺 Screen-space implementation

For those unfamiliar with the term, screen-space basically means that we render onto a plane (normally a quad or a giant triangle) that covers the whole screen.
If you want to be super efficient you can use a giant triangle but I opted for a quad because it was simpler to set up (I can hardcode the output vertex positions as the corners of the normalised device coordinates space at the near plane).

The fragment shader will sample the cube texture with the view direction in world space.
To get it, I simply multiply the vertex position by the inverse view and projection matrices and let the GPU interpolate it.

Normally, in vertex shader we go from object to world, to camera, and finally to clip/device space, but in this case we already have the device coordinates and we need the world's.



// So instead of doing:
device_space_pos = projection * view * model * objec_space_pos;
// We need the inverse transformation, which implies multiplying by the inverse
// matrices, in reverse order:
world_space_pos = inverse(view) * inverse(projection) * device_space_pos;

🚀 Performance tip: Inverting a matrix is an expensive operation, but we can take a shortcut.
Orthonormal matrices (matrices that don't change the relative position between points) have the convenient property that their inverse is the same as its transpose, which is way easier to compute.
Thankfully, the view matrix is orthonormal, so we can save some work there. However, the same doesn't apply to the projection matrix and it has to be inverted "properly".
ℹ️ You could precompute these inverse matrices on the CPU instead of re-calculating them for every vertex, but since there're only 4 vertices (potentially 3), the cost of uploading and binding the extra matrices would probably be higher.

If we render the skybox the first thing in the pass, that'd be it. The skybox will "clear" the render target and everything else would go on top (just remember to disable depth write).
However I wanted to go further and optimise it a bit more.

🥱 Avoiding work

I got the idea from this reply to one of my tweets:

It makes a lot of sense!

My test scene only has 3 objects at the moment: the floor, a Stanford Bunny and a Utah Teapot, but a "real" environment will have a lot of things on the screen, and the skybox will take only a small percentage of the render target. Why would we want to calculate the skybox for every single fragment if most of them will get overwritten anyway? And keep in mind that we are doing 2 expensive operations here: inverting a matrix (vertex stage) and sampling a cubemap (fragment stage).
A stencil is the perfect fit.

In this case we don't need anything fancy, we just need to know if that fragment is "free". So what I did was:

Move the skybox draw call to the end of the scene pass.
For every scene object, increment the stencil when a fragment is drawn.
When rendering the skybox, only draw the fragments that still have a stencil of 0.

This is how my stencil looks at the end of the pass:

The skybox fragment shader will only run for the pixels in black, saving a lot of computing.

To achieve this in the code I had to:

Change the Z buffer format to include stencil
Create 2 separated stencil states. One for the scene rendering, and another for the skybox. ```swift

mainStencilDesc.depthStencilPassOperation = .incrementClamp
mainStencilDesc.stencilCompareFunction = .always

// Only render the skybox for the fragments that have the same stencil value as the reference
skyboxStencilDesc.stencilCompareFunction = .equal

2.1 BONUS: Disable depth writing and testing for the skybox. It's not 100% necessary because the quad will be so close to the camera that it'll always pass, but it has a cost that we can easily avoid.
```swift


skyboxDSDesc.frontFaceStencil       = skyboxStencilDesc
skyboxDSDesc.isDepthWriteEnabled    = false
skyboxDSDesc.depthCompareFunction   = .always

Swap the states and set the reference value ```swift

...
// All other scene draw call encoding
...
cmdEncoder.setDepthStencilState(skyboxDepthStencilState)
cmdEncoder.setStencilReferenceValue(0);
// Encode the skybox draw call
...


Et voilà! 

![Demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5fve9r5180rfs8gzywun.gif)

---

## 🧑‍🏫 Conclusions
### 😄 The good
- Pretty simple to implement.
- Ridiculously small bandwidth cost: This approach only requires us to upload the cubemap and 4 vertices (3 if we used a triangle) that don't really need to carry any information. We could even just upload a point and generate the quad in the geometry stage. The cubemap can (and will) be used for other techniques so it's not really an extra cost and it can stay bound, it uses the same view and projection matrices so we don't need to bind any, and it uses the same attachments as the rest of the scene so it can stay in the same pass.
- The most expensive part (cubemap sampling) is only done where it's stricly necessary.

### 😭 The bad
- Not as "intuitive" as the giant cube approach?

### 👹 The ugly
- I spent more time than what I'm proud to admit debugging a "lense distortion" and doubting my math and graphics knowledge. Turns out it was caused by me "inverting" the projection matrix by transposing it, despite it not being orthonormal.
- Managing the depth-stencil state can be annoying in some graphic APIs?

Rendering Shadows in Real-Time Applications 2: Shadow Maps

Javier Salcedo — Sun, 05 Feb 2023 21:06:23 +0000

In the previous article, I gave a basic explanation on how static shadows can be done in real-time applications.
Now it's the turn for dynamic shadows.

This article will only cover the basics but, since it lays the foundations for more complex techniques, it'll be a bit longer and more in depth.

Dynamic Shadows

All dynamic shadow maps are essentially the same technique, simply changing the number of maps, the positioning of the lights’ views and the blending between maps (if there’re more than 1), but they have different use cases and greatly vary in complexity.

They also share the same limiting fact: each shadow-casting light source will need to render and apply its own shadow map.
This greatly limits the number of shadow-casting lights present in a scene, since each one implies rendering the scene [1,N] times from different viewpoints. This makes them a good fit for other techniques such as bindless rendering.

2D Shadow Maps

The main problem on rasterised real-time applications regarding shadows is that, at a given fragment, we don’t know anything else about the rest of the object or the scene so initially it’s not possible to know if the fragment is illuminated or not.

A smart way to bypass this limitation of information is to check against what the light “sees”:
If a fragment is visible from the camera, but not from the light, it means something else is in between the light and the fragment, so (for that particular light) it’s shaded.
But, how do we know that?

We can easily transform the fragment’s position to the light’s coordinate space if we create a view matrix for the light. The z/depth of that transformed fragment would be the depth from the light if nothing was in between the two.
If we now sample the light’s depth buffer at that fragment’s coordinates and it’s smaller we know something else got rendered on top of it, so it’s shaded.

Of course, this means we need to render the scene from the light’s perspective first, so we can get the depth buffer. This is the reason why this technique can get very expensive very quickly if we have too many shadow-casting lights.

This is a perfect technique for spot lights like flashlights, headlights or floodlights but it also works for directional lights, as long as the covered area is small.

The general steps to follow are:

Create a new texture and bind it as the depth attachment of a new render pass
Bind the light matrix as the view and a perspective or orthogonal matrix as the projection (it’ll be perspective or orthogonal depending on the type of light).
Render the scene. We can skip non-shadow-casting objects and the fragment stage, since we only want the depth buffer. (Keep in mind some objects like cloud billboards might need to have back-face culling disabled or flipped)
Bind light’s view and projection matrices (you can pre-multiply them) as/inside a buffer for the vertex stage.
Bind the now filled depth buffer as a regular texture for the fragment stage.
Render the scene normally with the following additions to the shaders:

In the vertex shader, the output now must include the transformed position of the vertex.

vertexOut.PositionInLightSpace = lightViewProj * object.model * vertex.position

The fragment shader on the other hand will use the interpolated position in light clip space to sample the shadow map.
First, the position needs to be normalised. That requires:

Transforming it into device space by dividing it by the w component (perspective divide)
Transform it into the [0,1] range (this will depend on the projection matrix and device coordinate space chosen)

Finally, the x and y components can be used to sample the shadow map, and the z component can be used to compare agains that result. If the fragment depth (z component) is greater than the shadow map’s it means the fragment was not visible from the light, and thus it’s in the shade.

float ComputeShadow(float4 position, texture2d<float> shadowMap)
{
    constexpr sampler smp(min_filter::linear,
                          mag_filter::linear,
                          s_address::clamp_to_border,
                          t_address::clamp_to_border);

    // Transform from clip to device coordinate system.
    // Orthographic projection matrices don't need this.
    auto lightCoords = position / position.w;

    // Normalise the coordinates.
    // Depends on the projection matrix and device coordinate
    // system.
    lightCoords.xy = (lightCoords.xy + 1) * 0.5;

    auto closestDepth = shadowMap.sample(smp, lightCoords.xy).x;
    auto currentDepth = lightCoords.z;

    return currentDepth > closestDepth ? 1 : 0;
}

Problems

As it is, this presents an issue known as shadow acne.

Shadow acne happens when a single texel of the shadow map covers multiple fragments at the same time. This means they all will compare against a single depth value. That means the surface’s orientation relative to the light can cause some fragments to have different depths for the same shadow map texel, causing the distinct shadow patters.

We can solve this pushing the shadow map’s depth away from the light by adding a bias to closestDepth but, if pushed too far, the shadow can appear “detached” from the object. We call that peter-panning.

A hardcoded bias however is never going to work for all cases, since it depends on the relative orientation of a fragment with the light. For that reason, multiple techniques to adapt the depth bias to the geometry exist, from simply adjust it according to the angle between the light and normal vectors, to more complex techniques like Receiver Plane Depth Bias (Isidoro J.R, 2006[4]) that uses derivatives to calculate how much the depth changes in the neighbouring fragments, or Adaptative Depth Bias (Ehm et al., 2014[5]).

Thankfully nowadays APIs provide ways to adapt the depth bias automatically, depending on the gradient of the primitives.
For example, in Metal these values perform quite well:

commandEncoder.setDepthBias(1, slopeScale: 3, clamp: 1/128)

This is applied while rendering the shadow map, so no bias has to be added to the fragment shader.

Conclusion

This technique as presented here produces hard shadows. Soft shadows techniques require sampling multiple times the shadow map, which of course will add a certain performance cost depending on the complexity of the filtering done.
This will be covered in future articles.

The main disadvantage of this technique however is that the greater the area lit by the light source, the bigger the shadow map should be. Otherwise, we’ll miss small or far away objects, and the rest of the shadows will lose sharpness.
It’s easy to see how this can become a big issue in terms of memory and bandwidth.
This is particularly bad for directional lights like the Sun when the scene is big. In that case, it’s better to use Cascaded Shadow Maps.

I’d choose this technique for the player's in a first person game, headlights of vehicles, or other spot light sources like floodlights or searchlights.
For directional lights I’d avoid this as a rule of thumb, but it could be used as the sun if the scene is relatively small (or the player’s view distance is limited), or maybe as an approximation for a big area light.

📚 References

[3] Lengyel, E. (2019), Foundations of Game Engine Development: Rendering, Terathon Software LLC
[4] Isidoro J. R. (2006), Shadow Mapping GPU-based Tips and Techniques, GDC 2006
[5] Ehm, A., Ederer, A., Klein, A., & Nischwitz, A. (2015). Adaptive depth bias for soft shadows

📷 Photo by Brie Odom-Mabey on Unsplash

Rendering Shadows in Real-Time Applications 1: Static Shadows

Javier Salcedo — Thu, 02 Feb 2023 21:24:58 +0000

Rendering shadows on Real Time Applications presents distinct challenges compared to their offline counterparts. On Offline ray/path-traced applications, shadows come “for free” as part of the rendering algorithm but for real time we normally need to rely on physically inaccurate, multi-pass techniques.

On this series of articles, I'll explain different ways we can achieve this.

I would classify the most common techniques in the following way, depending on use case and type of light:

Static
- Baked Light Maps
Dynamic
- Shadow Maps
  - Directional and spot lights
    - Single
    - Cascaded
  - Point Lights
  - Cube
- Stencil Shadows

In any case, all these techniques involve generating one or multiple textures with the shadow information that will be sampled in the lighting pass to determine if a fragment is in shadow or not.

Let's start with static shadows.

Baked Light Maps

If we know for sure that the light sources and the shadow-casting objects are not going to change, we can use the power and simplicity of our offline cousins and create a baked light map.
This would have the extra benefit of providing high quality lighting information as well, although we could bake just the shadows if we want to.

To bake a light map we ray/path trace the scene from each light, adding the contribution of each ray to one or many textures that map every static object of the scene, and will be stored on disk after we’re finished.
Later, when we render the scene, for each static object, we simply sample the texture(s).

This of course would require us to organise the scene into a hierarchy that allows not only the ray/path tracing algorithm to work, but also to map each static surface to a unique region of the texture, and in a way that the texture coordinates can be retrieved during rendering.
This technique will provide the highest quality and most physically correct shadows as we’re simulating the light rays bounces around the scene, but won’t be applied to moving or changing objects, which can create visually incoherent issues if not handled properly.

This technique is also the cheapest at runtime, since it only requires sampling a single texture.

It could also be combined with other real time techniques to be able to use dynamic light sources such as flashlights or vehicle headlights, or to add shadows for moving objects.

I’d choose this technique if:

The environment and main shadow-casting light sources are static.
You want the highest quality and most realistic shadows possible.
You want to reduce the number of real-time shadow-casting light sources while keeping the shadows they cast on the environment.
And/Or you want a high number of shadow-casting lights

An example of an ideal game for this technique would be a photo-realistic first person walking simulator in a space ship.

I’d avoid it if:

You have a changing environment:
- Trees moving with the wind
- Destructible structures
- User-built structures
The shadow-casting light sources change color, position or intensity over time:
- Day-night cycle
- Flickering or switchable lights

An example of a worst case game for this technique would be a cartoonish real time strategy game with a day-night cycle.

📚 References

“Lightmapping: Getting Started” (Unity docs) - https://docs.unity3d.com/2018.2/Documentation/Manual/Lightmapping.html
Battlefield 4 example: “What is Baked Lighting? (Lightmap)”: https://www.youtube.com/watch?v=eGmXDOGjKPA

📷 Photo by Maria Ionova on Unsplash

Replicating Swift's Enumeration Associated Values in C++

Javier Salcedo — Wed, 21 Dec 2022 21:44:52 +0000

Lately I'm working on a Metal engine as a hobby project.
I work with Metal in my day job and it has become my favourite graphics API in terms of developer experience.
At home I wanted the full "Apple Experience" so I decided to use Swift instead of the usual suspects (C/C++, Rust...).

I've never worked with Swift before so it's all new. Coming from C++ I feel a bit constrained from time to time (I still heavily dislike the way structs and classes work), but so far I'd describe it as a "comfy" language. It's easy to work with, the syntax is simple (most of the time), the tooling is pretty good, and it has a lot of syntax sugar.

One example of syntax sugar that I love is they way you can add arbitrary values to enums, officially called Associated Values.
I love it because it allows me to simplify APIs and reduce boiler plate a lot.

Let's see a simplified example from my project:

let PI: Float = 3.1416
let TAU = 2 * PI

struct Vec3 { let x, y, z : Float } // Imagine there's vector math here

enum Axis
{
    case X, Y, Z
    case CUSTOM(Vec3)
}

struct Transform
{
    // ...
    func rotate_around(axis: Axis, radians: Float)
    {
        var the_axis: Vec3
        switch axis
        {
            case .X: the_axis = Vec3(x:1, y:0, z:0)
            case .Y: the_axis = Vec3(x:0, y:0, z:0)
            case .Z: the_axis = Vec3(x:0, y:0, z:1)
            case .CUSTOM(let a): the_axis = a
        }

        print("Rotating \(radians) radians around the(\(the_axis.x), \(the_axis.y). \(the_axis.z)) axis")
        // Do math stuff with the axis
    }
    // ...
}

let t = Transform(/* ctr arguments*/)

t.rotate_around(axis: .X, radians: 0.25 * TAU)
t.rotate_around(axis: .Y, radians: 0.5  * TAU)
t.rotate_around(axis: .Z, radians: 0.75 * TAU)

let my_axis = Vec3(x: 0, y: 0.3333, z: 0.6667)
t.rotate_around(axis: .CUSTOM(my_axis), radians: TAU)

The output of which is:

Rotating 1.5708 radians around the(1.0, 0.0. 0.0) axis
Rotating 3.1416 radians around the(0.0, 0.0. 0.0) axis
Rotating 4.7124 radians around the(0.0, 0.0. 1.0) axis
Rotating 6.2832 radians around the(0.0, 0.3333. 0.6667) axis

Look at how nice and simple the implementation of rotate_around is! ❤️

It was just a matter of time until I encountered something at work that could use a feature like this. And that moment came today.
At work we use C++, and enums there are not this complex, so APIs like this can get a lot of boilerplate to get them working.
But thankfully, this line in Swift's docs made me realise there's a way to get a similar thing working on C++:

Enumerations similar to these are known as discriminated unions, tagged unions, or variants in other programming languages.

Variants!
So I went on and drafted a quick C++ alternative to my Swift code:

#include <iostream>
#include <variant>

constexpr float PI = 3.1416f;
constexpr float TAU = 2 * PI;

struct Vec3 { float x, y, z; }; // Imagine there's vector math here

enum class PredefinedAxis
{
    X, Y, Z
};
using CustomAxis = Vec3;
using Axis = std::variant<PredefinedAxis, CustomAxis>;

struct Transform
{
    // ...
    void rotate_around_axis(Axis axis, float radians)
    {
        Vec3 the_axis;
        if (auto* predefined = std::get_if<PredefinedAxis>(&axis))
        {
            switch (*predefined)
            {
                case PredefinedAxis::X: the_axis = Vec3{.x=1, .y=0, .z=0}; break;
                case PredefinedAxis::Y: the_axis = Vec3{.x=0, .y=1, .z=0}; break;
                case PredefinedAxis::Z: the_axis = Vec3{.x=0, .y=0, .z=1}; break;
            }
        }
        else if (auto* custom = std::get_if<CustomAxis>(&axis))
        {
            the_axis = *custom;
        }
        else
        {
            // We shouldn't get here!!
        }

        std::cout << "Rotating " << radians << " radians around the (" << the_axis.x << ", " << the_axis.y << ", " << the_axis.z << ") axis" << std::endl;
        // Do math stuff with the axis
    }
    // ...
};

auto main() -> int
{
    auto t = Transform{/* ctr arguments */};

    t.rotate_around_axis(PredefinedAxis::X, 0.25 * TAU);
    t.rotate_around_axis(PredefinedAxis::Y, 0.5  * TAU);
    t.rotate_around_axis(PredefinedAxis::Z, 0.75 * TAU);

    auto my_axis = Vec3{.x = 0, .y = 0.3333, .z = 0.6667};
    t.rotate_around_axis(my_axis, TAU);
}

Which outputs:

Rotating 1.5708 radians around the (1, 0, 0) axis
Rotating 3.1416 radians around the (0, 1, 0) axis
Rotating 4.7124 radians around the (0, 0, 1) axis
Rotating 6.2832 radians around the (0, 0.3333, 0.6667) axis

Of course, this being C++ it's more verbose and it feels a bit hacky (especially the using part), but it works!
The implementation of rotate_around_axis is messier, but I actually like the use of it way better!

I look forward to learning more cool features from Swift that I can copy in C++!

Pet projects shouldn't be optimal

Javier Salcedo — Mon, 23 May 2022 20:16:03 +0000

Over the years I started noticing a pattern in my life. It goes somewhat like this:

Find new hobby / skill / technology.
Start working on it and notice progress.
Become obsessed with it, read articles, watch videos, listen to podcasts, etc.
Start worrying about the optimal way to get results or the best practices.
It becomes a chore and I end up not working on it anymore.

I first noticed it with powerlifting.
I started going to the gym with my friends, having a good time and becoming really passionate about it.
After a couple of years I decided to join a club and start competing.
A year later I was working out for hours every day by myself, following strict and hard programs to add every pound I could to my maxs, and barely going out because I could feel my performance dropping if I skipped the diet.
In the end I left the club and powerlifting entirely. It took me more than a year to start truly enjoying it again.
However, it didn't click in my head. I blamed the long sessions, or the strict diet, or the insufficient results I was getting.

And of course, a similar thing happened when I started my first rendering engine.
I had a bit of experience with OpenGL but I decided to go all in with C++ and Vulkan.
I got pretty far pretty quickly, but eventually the time and mental cost of dealing with such low level tools led me to give up.

But then, a while I go I saw a tweet (sadly I haven't been able to find it again) about how JavaScript of all languages is actually used in computer graphics research.

Anyone who has worked with computer graphics knows that performance is everything. C++ is king, with Rust trying to make a breaktrhough. So this was mindblowing to me. JavaScript?? For real time graphics?? Really??

The reasoning however was quite simple: When you are doing research, trying to develop new techniques, you don't really care about performance. You care about relative performance compared to other techniques.
What is vital though is development speed. How quickly you can iterate and add new features. And languages like JavaScript or Python are perfect for that.

And I believe the same applies to hobby projects (as long as the point is not precisely to get the most optimised solution you can, obviously).

So forget about finding the perfect language, API or framework for your pet projects. Focus on choosing one that you enjoy working with, and start coding.

Photo by Annie Spratt on Unsplash

My best daily dev tool

Javier Salcedo — Sat, 21 Aug 2021 17:29:37 +0000

I like to travel and work light. When I'm choosing tools, languages or frameworks, I like to be able to work with minimal setups. The less things I need to use or learn, the better.

And that's why my favourite tool that I use on a day to day basis is my tablet.
It allows me to merge a lot of other things into a single device that I can carry around and takes almost no space.

📓 Notebook

This is most basic tool for any programmer. Period.

If you are not taking notes of everything already I don't know what are you waiting for, go get yourself a cheap notebook and a pen right now.

But if you're looking for a cleaner and more flexible alternative, a tablet + pen combo is the best.
It can do all a regular notebook can + allowing you to organise, clean up and backup your notes (and you can also use multiple colours, add pics, etc).

I personally use GoodNotes, and I really like the fact that it also has a desktop app.

🖊 Whiteboard

Sadly, with whiteboards size matters, but it's still very useful to make quick diagrams and schemes, while being able to erase and/or modify them, use multiple colour markers, etc.

There are also apps that allow you to share and present the whiteboard with others, so you can still have the whiteboard collaborative experience.

Definitely a great feature during this last year and probably a must for any remote programmer.

📚 eBook

As programmers, we always strive for expanding our knowledge.

That means that we read a lot, and we can end up with a lot of books that we might need to consult.
But carry them around just in case is not viable.
Unless you have them digitalised.

I still prefer paper books though, specially for technical books.
I simply enjoy reading them more, and I like seeing my little library growing.

However this is all about optimising and convenience.
And for that there's simply nothing better than having your books accessible from anywhere, all in one (digital) place.
That's why nowadays I consider my paper books more like collection items or a treat.

It also makes searching for specific stuff much faster and eBook versions of books tend to be much much cheaper.

📝 BONUS: Annotating papers

I guess this could go with the eBook, but papers are normally PDFs, so you're not really replacing anything.

In my line of work, it's not unusual to read research papers fairly often, and being able to take notes directly over them makes it much more convenient and enjoyable.

Before having my iPad I used to print the ones I was more interested on. So I guess it's also a good move for the environment. 🍃

Conclusion

I use a relatively old iPad with a 1st gen Apple pencil, but obviously you don't need to get into the (expensive) Apple ecosystem. Any tablet compatible with a digital pen would work just as good (if not better).

If I were to buy one right now I would probably consider getting an e-ink device (the recently announced PineNote looks promising.
You give up colours, but they are generally cheaper and better for your eyes and they have less distractions such as games, social media apps, streaming services, etc.

At the end of the day the best tool is the one you take the most value from, and I use my iPad constantly both for my work as for my personal projects.

So if you're looking for a tool to improve your development experience and productivity, before you spend money on a fancy keyboard, laptop, IDE license, etc; consider getting one of these.

💥 EXTREME documenting: My workflow

Javier Salcedo — Sun, 18 Jul 2021 21:38:05 +0000

Writing documentation is one of those things we all despise.
It's boring, it's tedious and takes way too much time that we could be using to build a new feature, or to fix another bug.

However, we all know too how important is to have proper documentation (and if you don't, just wait until you have to work on an old, abandoned part of the code whose authors are long gone from the company).

(Dealing with legacy code is a whole other topic, but I always recommend this book if you are interested in learning more.)

In general, my advise is to write down EVERYTHING (hence the click-baity title) and filter it later if necessary. Act like if you were going to forget everything tomorrow (because you will).

Note taking

The good old reliable. What can I say?

I normally take notes by hand during my research but, after months and months, it's easy to lose them and, even if you find the appropriate ones, you will probably not understand them anymore, since you are not in the same mental space and they are likely to lack any structure.

My first iteration to solve this was to buy an iPad + iPencil. That way I have my notes synced and centralised, and I can rearrange / organise them.
But of course this is still less than ideal and it still means that if I leave the company that knowledge leaves with me.

Recently I added Notion to the mix, but I'll write another article about that.

Leverage your wiki

If a notebook is your personal knowledge hub, the wiki is your team's (or even the whole company's). It's your shared hive brain.
Every team should have one (and if you don't, I strongly suggest you to set one up asap).

It doesn't really matter if it's Confluence, Notion, or a Git repo with a lot of Markdown files (I've even read articles recommending adding a Markdown file for each source file, in the same directory), the important part is to have it easily accessible and modifiable by all team members.

Equally important is to keep it updated and reviewed!
I like apps like Notion, but a great advantage of simpler formats like Markdown is that you can easily integrate documenting into your usual code review pipeline.

Level up your ticket game

My next step into documenting fanaticism was a while after fixing a particularly complex bug. I had to go back to fix a new related issue and I found that I no longer remembered the inner workings of that system. The main issue was that I found the solution of that bug on an article online, and I didn't remember where.

So I decided to start writing down every source of information I used, not only my own thoughts/ideas.

But that turned out to be a lot of information, so I needed a better organising system.

My solution was to create a Notion template for a ticket/bug with all the fields that I'm likely to need in the future.

Hoarding browser tabs / links

We've all been there, tens of open tabs, with StackOverflow posts, blog articles, YouTube videos...

It can get hard to navigate, even overwhelming. And on top of that, what happens to all of that when you're done? Normally just closed and forgotten.

That's why nowadays I put them into their own list on Notion.
That way I can safely close the tabs once I read them and I know where to find them when I need them again, be it next hour or next month.
And it keeps my browser lean and tidy too.

Logs

Logs are your friends. No matter how much loggin are you doing, you should probably do more. Period.

I keep any related / suspicious logging message saved too, that way, if the bug comes back from the dead, I can check if the logs have changed.

Ticket postmortem

We already do this for whole features / products, time to go down to the ticket level.

All the information above can be a lot to process weeks/months down the line, so a quick summary of what you did to fix/implement it and why can save you a lot of headaches.

Diagrams

Maybe it's because I'm a very visual person but for me, there's nothing better than a flow/class diagram to get me to understand how a piece of code works/is structured.

Code can sometimes be a labyrinth and diagrams are your map. A map that can be improved and updated by and passed to other team members.

A lot of people like to use UML diagrams with things like PlantUML, that allows you to create diagrams through a text file that can be easily added to your version control system of choice.

But I personally prefer Diagrams.net (previously known as Draw.io) because you don't need to learn a whole new syntax and you can rearrange and color code it as you please. Its save file format is basically xml so it is git-friendly too.

Trick yourself into writing

Now to the hardes part, actually doing it.

We don't like to write documentation for a reason, and a super detailed and strict new system is not going to make it any easier (specially if imposed from upper management).

So the most important thing, more important than the system itself, is to build the habit of documenting our work.

For me, documenting is a pretty mindless chore, since I already take lots of notes during normal development. All I have left to do is recopilate and organise it. So I choose the most boring and lowest energy moment of my day to do it: the afternoon slump.

At that moment of the day my mental energy is non-existent and any attempt at actual thinking/reasoning will be fruitless, so copying notes into a database is easy enough to do and ramps me up into more mentally demanding work.

📷 Pics

Header: Photo by Roberto Nickson on Unsplash
Manual note taking: Photo by Green Chameleon on Unsplash
Level up your ticket game: Photo by Geeky Shots on Unsplash
Hoarding browser tabs / links: Reddit
Logs: Photo by Radek Grzybowski on Unsplash
Trick yourself: Photo by Clint Bustrillos on Unsplash

In defence of daily stand-ups

Javier Salcedo — Sat, 01 May 2021 17:41:51 +0000

Photo by Dylan Gillis on Unsplash

I've seen lately a lot of hate towards dailies and I want to add my 2 cents on what I think is the real issue behind.

A common critique I've heard is that they are distractions and they are a waste of time.

Daily stand-ups, meetings that should last about 10 minutes, are a waste of time?
I think we have a hint on what can be going on here.

I've seen and heard of dailies that take too long. My personal record is attending one that lasted over an hour, but the average seems to be around 20-30 minutes.

No wonder people think like that!

But how did we get here?
Let's remember what a daily should be:

Less than 9 attendees
Each person speaks for ~1 minute, saying:
- What did they do yesterday
- What will they do today
- If they need any help

The goal is to keep the individuals aware of the state and progress of the project and the rest of the team, and ask for / offer help if needed. Nothing more.

That wouldn't even take 15 min on the worst cases, so what happened?

In my opinion, we lost our way. And they became nothing more than an opportunity to brag about how smart we are, explaining in detail how we implemented that new tricky feature, or how we fixed that elusive bug. And the sad truth is no one cares.

Without seeing the code and the appropriate context and references, no one can follow a complex (and incomplete) explanation, and people just stop paying attention, simply waiting for their turn to do the same.

That way, when everybody leaves the meeting (bored to death), no one really knows what the rest are doing and they feel (rightly so) that they just lost a valuable part of their day with nothing in return.

Dailies are useful, Agile methodologies are useful.
But only if implemented properly (and if they're actually appropriate for the specific project).
They are mere tools. No one would complain that a screwdriver is a useless tool while holding it backwards (and/or trying to use it with a nail).

So let's go back to the basics, and Keep It Simple (Stupid).