go.mod replace: fallthrough

William Antonelli — Sat, 08 Jun 2019 11:59:33 +0000

How do you use `replace` in `go.mod` when you want to replace a pkg that an imported pkg uses?

The Problem

Project uses Lib
Lib uses SubLib

Project
  Lib
    SubLib

You want to replace SubLib with a local version that you're working on to see the result in Project.

The Answer

Open go.mod in Project to use the replace directive.

// go.mod

require(
  github.com/example/lib
  // `lib` imports github.com/sublib/sublib
)

replace github.com/sublib/sublib => /local/dir/sublib

Profit!

May my hours of suffering help another.

When Files Are Better Than a DB

William Antonelli — Wed, 29 May 2019 13:30:40 +0000

That's a lot of data!

While creating a roguelike game in golang I got to the point of needing to save the objects. Naturally my first thought was to use a database, and of course I didn't want to use something heavy and external, but rather light and embeddable, so I chose storm, a toolkit for bbolt. bbolt is a key/value datastore that writes to disk. But when one Screen's worth of objects is a few hundred MB, and you have 240 Screens in the world, well... the query results tend to get mangled after saving just 2-3 Screens. I had very weird results wherein the last 12% to 40% of the results from one Screen query weren't being processed correctly. Combine this with the query taking a few seconds due to the DB easily growing by 300MB for each Screen added, and it becomes apparent that a different method is needed.

The world is flat!

Enter the flat-file database. Instead of one giant datastore, I decided that each Screen would be a named folder, and inside there would be a separate file for each object. This yielded much better speeds for the program. For this method I used scribble. This project uses JSON as the file structure, but I don't need the files to be human-readable so I modified it to use encoding/binary. There is a library that was created to be able to use encoding/binary as a drop-in replacement of encoding/json by implementing the Marshaler and Unmarshaler interfaces: github.com/kelindar/binary

By using encoding/binary instead of encoding/json the file sizes are reduced and there's also a bit of a performance increase.

Obviously I need to tune the program to not use so much space, but I like to test things under a high load to see how it performs, and in this case it paid off.

My version of Scribble can be found here:
https://github.com/nanohard/scribble

Forem: William Antonelli