Forem: Hajime Hoshi

Compiling a Go program into a native binary for Nintendo Switch™

Hajime Hoshi — Mon, 03 Jan 2022 08:35:41 +0000

This article has been moved to ebiten.org due to some complicated business reasons.

This is an English translation of my article in Japanese.

tl;dr

Previously, we compiled a Go program into a WebAssembly and then converted it into C++ files to make it run on Nintendo Switch. Now, I have succeeded in compiling a Go program into a native binary for Nintendo Switch, and also running a game there. I replaced system calls with C function calls using the -overlay option. Also, I have developed a new package Hitsumabushi to generate JSON content for this.

Caution

This article and the open-source projects in this article are based only on publicly available information. Hajime is responsible for this article's content. Please do not ask Nintendo about this article.

Background

I have been developing a 2D game engine called Ebiten in my spare time. I have succeeded in porting this to Nintendo Switch and the Nintendo Switch version of "Bear's Restaurant" was released in 2021.

The method was to compile a Go program into a WebAssembly (Wasm) binary and then convert it to C++ files. See the presentation slides from GoConference 2021 Autumn for more details. The advantages were low uncertainty, low maintenance cost, and high portability. Once I developed the tool, its maintenance cost was pretty small as Wasm's specification is stable. On the other hand, the disadvantages were bad performance and long compiling time. Not only that performance was worse than native, but GC also suspended the game due to a single thread.

Compiling a Go program into a native binary for Nintendo Switch without using Wasm was quite uncertain and a rocky road. Of course, Go doesn't support Nintendo Switch officially. And naturally, Nintendo Switch's source code and binary formats are not open. Even if I hit an issue, it'd be possible that there would not be any clues to help me solve it. However, if I knew that I were to succeed, performance would be better than ever, and compiling speed would be as fast as Go. So I thought it was worth a shot and have been doing some experiments intermittently for one year.

Strategy

The strategy is basically to replace system calls with C function calls in the runtime and the standard library. The system calls part is OS-dependent, and if I replace it with something portable, Go should work everywhere in theory. It seems pretty easy, doesn't it? Well, it was a lot more challenging than I expected...

The graphic below describes what I had to do. The left side is a structure of a structural overview of standard Go compiling. System calls work on specific systems and of course, this doesn't work on Nintendo Switch. So I had to replace them with standard C function calls like the right side.

Replacing system calls with C function calls

And, there is another action item to adjust the binary format that the Go compiler generates to fit with Nintendo Switch. So in summary, the action items were as follows:

Replacing system calls with standard C function and/or pthread function calls
Adjust the ELF format that the Go compiler generates.

For replacing system calls, of course, system calls do not correspond one-to-one with C functions. And, there are too many system calls to implement. So, I replaced system calls one by one by finding which ones refused to work on an actual Nintendo Switch device.

The Go compiler can generate only formats that the Go compiler officially supports. For example, when a target is Linux, the format is ELF. Can Nintendo Switch support ELF? make a long story short, yes, I managed it. I won't describe the details about 2. here ¹.

What I have to do is create a .a file via the Go compiler with GOOS=linux GOARCH=arm64 and -buildmode=c-archive, and then link it with other object files and libraries via Nintendo Switch compiler. The reason why I don't use -buildmode=default is that there are some items I have to do around an entry point. IMO, in general, it is more portable to depend on the platform for an entry point.

System calls are defined basically in the standard library, especially runtime and syscall packages. So, how did I rewrite them? In this project, I adopted the -overlay option.

Hitsumabushi - rewriting the runtime with the `-overlay` option

go build's -overlay is an option that overwrites Go files to be compiled. I overwrote Go files in the runtime with this option. This is the official document's explanation:



-overlay file
    read a JSON config file that provides an overlay for build operations.
    The file is a JSON struct with a single field, named 'Replace', that
    maps each disk file path (a string) to its backing file path, so that
    a build will run as if the disk file path exists with the contents
    given by the backing file paths, or as if the disk file path does not
    exist if its backing file path is empty. Support for the -overlay flag
    has some limitations: importantly, cgo files included from outside the
    include path must be in the same directory as the Go package they are
    included from, and overlays will not appear when binaries and tests are
    run through go run and go test respectively.

This is the format to give -overlay:



{
  "Replace": {
    "/usr/local/go/src/runtime/os_linux.go": "/home/hajimehoshi/my_os_linux.go"
  }
}

If you build a Go program with this, os_linux.go's content in runtime is replaced with my_os_linux.go's. Pretty handy, isn’t it?

Managing this JSON file as it is is not portable. A location where Go is installed depends on environments, and then the target files' locations vary. Plus, you very rarely have to replace the full contents of a file, and in most cases, it is enough to replace some functions. As such, it is troublesome to update source files to match each Go version update.

So, I developed a new package to generate a JSON for this project. This is Hitsumabushi (ひつまぶし) ². I adopted this name because I wanted a name ending with 'bushi' as a play on libc (ree-boo-shee (りぶしー) in Japanese pronunciation), because this is one of the primary things that Hitsumabushi deals with. There was another candidate I was considering, Katsuobushi (かつおぶし) ³, but I won’t get into that...

hajimehoshi / hitsumabushi

Run Go programs (almost) everywhere

Hitsumabushi is a very simple package defining an API like this:



// GenOverlayJSON generates JSON content that can be passed
// to -overlay based on the given options, or returns an error
// when an error occurs.
//
// There are some options like specifying command arguments
// and specifying the number of CPU.
func GenOverlayJSON(options ...Option) ([]byte, error)

Implementation of Hitsumabushi

I have created an original patch format for Hitsumabushi that looks like this:



//--from
func getRandomData(r []byte) {
    if startupRandomData != nil {
        n := copy(r, startupRandomData)
        extendRandom(r, n)
        return
    }
    fd := open(&urandom_dev[0], 0 /* O_RDONLY */, 0)
    n := read(fd, unsafe.Pointer(&r[0]), int32(len(r)))
    closefd(fd)
    extendRandom(r, int(n))
}
//--to
// Use getRandomData in os_plan9.go.

//go:nosplit
func getRandomData(r []byte) {
    // inspired by wyrand see hash32.go for detail
    t := nanotime()
    v := getg().m.procid ^ uint64(t)

    for len(r) > 0 {
        v ^= 0xa0761d6478bd642f
        v *= 0xe7037ed1a0b428db
        size := 8
        if len(r) < 8 {
            size = len(r)
        }
        for i := 0; i < size; i++ {
            r[i] = byte(v >> (8 * i))
        }
        r = r[size:]
        v = v>>32 | v<<32
    }
}

The part after //--from and the part after //--to represent a replacing source and a target respectively. The reason why I invented my simple format is that the existing patch formats don't assume to be modified by a human being. In the above example, Linux's getRandomData implementation is replaced with Plan 9's. Linux's getRandomData uses /dev/urandom and this is not potable⁴. This patch format saves some amount of work to manage the differences I want to replace. Of course, the cost to keep up with the Go version updates doesn't become zero even with this, but it should help a lot.

Hitsumabushi creates modified files with this format and puts them in a temporary directory. It uses the files as the content of JSON (the replacing source file names).

Note that Hitsumabushi rewrites the standard library and the runtime, and the Go compiler is not the target to rewrite. In other words, the regular Go compiler is used as is.

The replacements by Hitsumabushi are only the standard C function calls and pthread function calls. It never deals with platform-specific APIs⁵. So, ideally, Hitsumabushi should enable a Go program to run on any platform, regardless of whether or not the Go compiler originally supports it.

Replacements

Calling C functions from `runtime`

It is not an easy task to call a C function from runtime. In a usual Go program, you can call a C function easily with Cgo. However, runtime cannot use Cgo. Using Cgo means to depend on runtime/cgo, and runtime/cgo depends on runtime, so this would be a circular dependency.

To get straight to the point, libcCall makes it possible to call a C function from runtime. Some environments like GOOS=darwin already do this.

In addition, various compiler directives are required.

//go:nosplit: Skips an overflow in the stack.
//go:cgo_unsafe_args: Treats Go arguments as C arguments.
//go:linkname: Treats something defined in another package as if it was defined in this package. Or, it treats something defined in this package as if it was defined in another package. It ignores whether the symbol is exported or not. Very useful!
//go:cgo_import_static: Static-links a C function and makes it possible to treat the symbol value in Go.

Let's see an actual example. To call the write system call from runtime, a function called write1 is defined on the Go side.



// An excerpt from runtime/stubs2.go in Go 1.17.5

//go:noescape
func write1(fd uintptr, p unsafe.Pointer, n int32) int32



// An excerpt from runtime/sys_linux_arm64.s in Go 1.17.5

TEXT runtime·write1(SB),NOSPLIT|NOFRAME,$0-28
    MOVD    fd+0(FP), R0
    MOVD    p+8(FP), R1
    MOVW    n+16(FP), R2
    MOVD    $SYS_write, R8
    SVC
    MOVW    R0, ret+24(FP)
    RET

In the case of 64bit ARM, SVC is used to invoke a system call.

Let's replace this with a C function call by libcCall and compiler directives.



// An excerpt from runtime/stubs2.go after Hitsumabushi's replacement

//go:nosplit
//go:cgo_unsafe_args
func write1(fd uintptr, p unsafe.Pointer, n int32) int32 {
    return libcCall(unsafe.Pointer(abi.FuncPCABI0(write1_trampoline)), unsafe.Pointer(&fd))
}
func write1_trampoline(fd uintptr, p unsafe.Pointer, n int32) int32



// An excerpt from runtime/os_linux.go after Hitsumabushi's replacement

//go:linkname c_write1 c_write1
//go:cgo_import_static c_write1
var c_write1 byte



// An excerpt from runtime/sys_linux_arm64.s after Hitsumabushi's replacement

TEXT runtime·write1_trampoline(SB),NOSPLIT,$0-28
    MOVD    8(R0), R1   // p
    MOVW    16(R0), R2  // n
    MOVD    0(R0), R0   // fd
    BL  c_write1(SB)
    RET



// An excerpt from runtime/cgo/gcc_linux_arm64.c after Hitsumabushi's replacement

int32_t c_write1(uintptr_t fd, void *p, int32_t n) {
  static pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER;
  int32_t ret = 0;
  pthread_mutex_lock(&m);
  switch (fd) {
  case 1:
    ret = fwrite(p, 1, n, stdout);
    fflush(stdout);
    break;
  case 2:
    ret = fwrite(p, 1, n, stderr);
    fflush(stderr);
    break;
  default:
    fprintf(stderr, "syscall write(%lu, %p, %d) is not implemented\n", fd, p, n);
    break;
  }
  pthread_mutex_unlock(&m);
  return ret;
}

By the way, libcCall is not defined on GOOS=linux. I had to rewrite //go:build in runtime/sys_libc.go properly.

If you forcibly call a C function using assembly without libcCall, a C stack will be on the current Goroutine's stack. Then, you might find very mysterious errors. I don't recommend invoking a C function without libcCall.

Ignoring signals

Hitsumabushi ignores all signals. For example, sigaltstack and sigprocmask in runtime are empty. There are standard C functions that deal with signals, but they are not implemented in some environments.

As a side effect, accessing a nil pointer caused SEGV, and recover-ing it became impossible. A program dies without panic messages, even. This is inconvenient to some extent, but we have to put in the effort to avoid this issue in production environments.

Implementing a pseudo file system

Even when a Go program does nothing, the runtime might access the file system. On Linux, apparently these files are read from the runtime:

/proc/self/auxv (Information about e.g. a page size)
/sys/kernel/mm/transparent_hugepage/hpage_pmd_size (Huge Page Size)

I hand-crafted some content for both. For example, I used 0 for Huge Page Size since it worked. For the implementation, see Hitsumabushi's c_open.

For writing files, I implemented only a standard output and a standard error. Both just use fprintf. Without them, even println doesn't work. I decided not to implement reading and writing other files for now. For the implementation, see Hitsumabushi's c_write1.

Implementing a pseudo memory system

In Go's heap memory management, mmap system call is the bottom layer on Linux. Go manages virtual memory allocated there. munmap is called for unused regions.

There are 4 states of a heap memory region and these states transition as in the diagram below. When the state is 'Ready', the region is available.

The state transition diagram of Go's memory

Go specifies an address in virtual memory and uses an allocated memory region with the address. However, there is no standard C function to allocate memory with a specific address. That's unfortunate.

There are some platforms where it is impossible to allocate memory with a specific address: Plan 9 and Wasm. Hitsumabushi referred to them and implemented a 'corner-cutting' memory system. It referred to the Wasm version in particular, which is the simplest implementation. I won't describe the details here, but basically, the implementation is as shown in the following list. For an actual source, see Hitsumabushi's mem_linux.go.

sysAlloc: Calls sysReserve and sysMap.
sysMap: Increments the total size record of heap memory.
sysFree: Decrements the total size record of heap memory.
sysReserve: Calls calloc.
The other functions do nothing.

As you can see, there is a call of calloc but no call of free. It is impossible to free a part of a region allocated by calloc. This means that memory usage is monotonically increased. Originally, the method to make an Ebiten application work on Nintendo Switch was to convert Go to C++ via Wasm, and memory usage was also monotonically increased there⁶. It didn’t end up making things worse, at the very least, so I’ve compromised with this solution so far, but I would like to fix this in the future…

Implementing pseudo `futex`

futex is the bottom layer of the part that handles sleeping and waking up threads. Of course, the standard C functions and pthread functions cannot invoke futex directly. So, I had to mimic the behavior of futex with pthread. Originally, pthread itself is implemented with futex, so I had to do the opposite thing.

There are two ways to use futex via Go.

futexsleep(uint32 *addr, uint32 val): Makes the thread sleep when addr is val.
futexwake(uint32 *addr): Wakes up the thread which sleeps with addr.

In Hitsumabushi, I added a simple implementation like this. For an actual source, see Hitsumabushi's pseudo_futex.



// A pseudo code
pseudo_futex(void* uaddr, int32_t val) {
  static pthread_cond_t cond; // A condition variable

  switch (mode) {
  case sleep:
    if (*uaddr == val) {
      cond_wait(&cond); // Sleep
    }
    break;
  case wake:
    cond_broadcast(&cond); // Wake up all the threads sleeping with cond.
    break;
  }
}

When wake is called, it will wake up not only the necessary threads, but all the threads. If you want to wake up only necessary threads, you would need to manage multiple condition variables for each uaddr, which would be cumbersome. Such unnecessary waking up is called spurious wakeup. This is explicitly expected in Go source code, so this is not problematic. However, performance might be degraded.

Adjusting the number of CPU cores

The number of CPU cores is determined by the result of the sched_getaffinity system call. There is no corresponding standard C function, so I gave Hitsumabushi an option to specify the number of cores to GenOverlayJSON. For the actual source, see Hitsumabushi's c_sched_getaffinity

There were some environments where an application froze with 2 or more CPU cores specified. It's because a thread could use only one core by default. Thus, I had to call pthread_setaffinity_np explicitly. In Hitsumabushi, I added a hack to call pthread_setaffinity_np just after pthread_create. For the actual source, see Hitsumabushi's overlay.go. As an aside, it was quite hard to find this solution. I can’t tell you how happy I was to finally solve this conundrum.

Entry point

Hitsumabushi is assumed to be used with -buildmode=c-archive. The generated file is a C library, and even main is not called. If you want to call main, you have to define a C function and call main explicitly inside. Calling main explicitly does not make sense usually, but I think it is practical for c-archive.



package main

import "C"

//export GoMain

func GoMain() {

    main()

}



// Call the entry point in Go in the entry point in C.

int main() {

  GoMain();

  return 0;

}

Results

I managed to get a game called "Innovation 2007" working on an actual Nintendo Switch device. Controller support, touch inputting, and audio all work perfectly. Innovation 2007 uses most of Ebiten's features, so I'm sure other games would work as well.
Compiling speed became much faster. Before this solution, it took 5 to 10 minutes to full-build a C++ project, but now it only takes less than 10 seconds. This is awesome!
Suspensions by GC seem to have disappeared.
I now have to update whenever a new version of Go is released. This is an acceptable compromise to me. From my past experiments, I don’t expect any major changes anyway.

Remarks

This is a side note, but the implementation of Go's runtime has ample accumulation of knowledge about modern OSes and is very insightful. I think it can teach you a significant amount about computer science. That said, it can be quite daunting to read it without a purpose, so I recommend doing so with some sort of modification project in mind.

Thanks to the near-success of this project, the method I presented in the Go Conference is now becoming outdated. This was inevitable, obviously, but it still makes me feel a little sad to see that hard work go obsolete.

Future works

I'll continue polishing this so that a proper game can be released for Nintendo Switch. As I described first, there is a high level of uncertainty in this project. Until a game is released, I cannot anticipate what kind of issues will occur, and I always have to be on high alert. Even in the worst case scenario, however, I know we can continue to release the game with the help of go2cpp, which is reassuring. Still, with all the hard work I’ve put into this already, I really want to release a game with Hitsumabushi and see it achieve some actual results.

Acknowledgments

Thanks to the kind folks over in the PySpa community for all their technical advice. I’d also like to express my gratitude to Daigo, President of Odencat Inc., who kindly uses Ebiten for Nintendo Switch. Thank you very much.

Happy new year!

It's due to complicated business reasons. ↩
Hitsumabushi is Japanese food. ↩
Katsuobushi is yet another Japanese food. ↩
There is another solution, making a pseudo /dev/urandom file, but I didn't adopt this. There is no other good way than using a platform-specific API. ↩
The main reason is portability, but there is also another compelling reason: I wouldn't be able to make it open-source if it used a platform-specific API. ↩
To be exact, about 2G of memory was allocated first and was used without additional allocations. ↩

GopherJS vs WebAssembly for Go

Hajime Hoshi — Fri, 15 Jun 2018 19:49:51 +0000

EDIT: For the latest of GopherJS/Wasm comparison, see Wasm benchmark result

Hi all!

This article describes about my experiment of the new WebAssembly port of Go. WebAssembly port is now available on the master branch of Go, and you'd need to compile Go yourself.

tl;dr

I have created GopherWasm, an agnostic WebAssembly wrapper that works both on GopherJS and WebAssembly port.
Performance of GopherJS and WebAssembly depends on browsers. GopherJS is faster than WebAssembly on some environments, and slower on other environments. For Ebiten 'sprite' example, (GopherJS on Chrome) > (WebAssembly on Firefox) > (GopherJS on Firefox) > (WebAssembly on Chrome) with 5000 sprites.

Go on browsers

Running Go applications on web browsers must be awesome. Needless to say, Go is an awesome language. I don't discuss how good Go is in this article :-)

There is a transpiler from Go to JavaScript - GopherJS by Richard Musiol. This also enables Go programs to run both on browsers and Node.js. You can use all the features of Go. The compilation result is reasonably readable JavaScript. The performance is so-so due to some overhead. To emulate Go behaviors precisely, GopherJS adds some overhead like boundary check of index access to slices. Instead of emulating Go behavior by JavaScript, executing binaries like WebAssembly on browsers seems much more efficient.

WebAssembly is a performance-wise format compared to JavaScript. WebAssembly is supported by most of modern browsers. WebAssembly is a low-level language as the name says, and it is expected that WebAssembly binary is generated from other languages. Actually C, C++ and Rust already support WebAssembly port.

The latest Go version 1.11 supports WebAssembly port by Richard Musiol, the same author of GopherJS. Now Go 1.11 is on the way releasing, but you can test WebAssembly APIs with the latest Go by compiling yourself. Your compiled program for WebAssembly is available both on browsers and Node.js. You can use full features of Go including goroutines. You can call any JavaScript functions from Go, and you can pass Go function as a JavaScript callback. The API is defined at syscall/js package. The environment variables for WebAssembly are GOOS=js and GOARCH=wasm. As WebAssembly is performance-wise format, this should be faster than GopherJS, right? Unfortunately, this was not true. I'll describe this later.

Ebiten

Ebiten is a dead simple 2D game library by me. This is basically an OpenGL wrapper. This works on browsers with WebGL by GopherJS, and actually you can see some examples work on the website and the jsgo playground by Dave Brophy. Recently (actually today!) I fixed Ebiten (master branch) to accept WebAssembly compilation of the latest Go compiler except for the audio part. Thus, Ebiten now works both on GopherJS and WebAssembly!

Port GopherJS library to WebAssembly

As I said, Ebiten can already work with GopherJS. GopherJS's API is similar to WebAssembly, but different. For example, the counterpart of js.Object of GopherJS is js.Value of syscall/js.

Then, how can I write libraries to accept both GopherJS and WebAssembly? Of course it is easily possible to write similar duplicated code, but isn't there a more elegant way?

I've created GopherWasm, an agnostic WebAssembly wrapper. If you use GopherWasm, your library automatically works both on GopherJS and WebAssembly port! GopherWasm API is almost same as syscall/js. The only one difference is js.ValueOf accepts []float32 or other slices in GopherWasm, not in syscall/js. I have already filed to fix syscall/js.ValueOf to accept such slices, so the situation might change in near future.

Performance comparison

I've compared the performances between GopherJS and WebAssembly port with my Ebiten example 'sprites'.

By pressing left or right arrow keys, you can change the number of sprites and see how FPS (frames per second) changes.

On my MacBook Pro 2014, I took very rough measurements by showing 5000 sprites:

GopherJS on Chrome:     55-60 FPS
GopherJS on Firefox:    20-25 FPS
WebAssembly on Chrome:  15-20 FPS
WebAssembly on Firefox: 40-45 FPS

Chrome: Version 67.0.3396.87 (Official Build) (64-bit)
Firefox: 60.0.2 (64-bit)
Ebiten: 460c47a9ebaa21bcce730a460a7f87fa6cbe56ed
Go: 534ddf741f6a5fc38fb0bb3e3547d3231c51a7be

This is a very interesting result. Before this experiment, I thought WebAssembly should always be faster than GopherJS. However, the result depended on browsers. For 5000 sprites, the result was (GopherJS on Chrome) > (WebAssembly on Firefox) > (GopherJS on Firefox) > (WebAssembly on Chrome). I guess optimization way is different among browsers.

I took rough profile and it looks like allocation (runtime.mallocgc) was the heaviest task on WebAssembly. This is different tendency from GopherJS. I'm not sure the details how objects are allocated on WebAssembly, but at least WebAssembly requires different optimization from GopherJS.

I plan to do optimization to keep 60 FPS as much as possible. Stay tuned!

Binary size comparison

-rw-r--r--    1 hajimehoshi  staff  7310436 Jun 16 05:23 sprites.js
-rw-r--r--    1 hajimehoshi  staff   278394 Jun 16 05:23 sprites.js.map
-rwxr-xr-x    1 hajimehoshi  staff  8303883 Jun 16 04:03 sprites.wasm

It looks like WebAssembly binary is slightly bigger.

Appendix - How to do experiments

Install Ebiten and other libraries

go get -u github.com/hajimehoshi/ebiten/...
go get -u github.com/hajimehoshi/gopherwasm
go get -u github.com/gopherjs/gopherjs

Get the latest Go and compile it

cd
git clone https://go.googlesource.com/go go-code
cd go-code/src

# Compile Go. ./all.bash is also fine if you want to run tests.
./make.bash

Compile an Ebiten example for WebAssembly

cd /path/to/your/wasm/project

# Compile 'sprites' example for WebAssembly
GOOS=js GOARCH=wasm ~/go-code/bin/go build -tags=example -o sprites.wasm github.com/hajimehoshi/ebiten/examples/sprites

# Copy wasm_exec.js
cp ~/go-code/misc/wasm/wasm_exec.js .

Prepare an HTML file to run the wasm file. This file is based on ~/go-code/misc/wasm/index.html.

<!DOCTYPE html>
<script src="wasm_exec.js"></script>
<script>
// Polyfill
if (!WebAssembly.instantiateStreaming) {
  WebAssembly.instantiateStreaming = async (resp, importObject) => {
    const source = await (await resp).arrayBuffer();
    return await WebAssembly.instantiate(source, importObject);
  };
}

const go = new Go();
WebAssembly.instantiateStreaming(fetch("sprites.wasm"), go.importObject).then(result => {
  go.run(result.instance);
});
</script>

Run an HTTP server as you like.

Run GopherJS server

gopherjs serve --tags=example

Then access http://localhost:8080/github.com/hajimehoshi/ebiten/examples/sprites/ to see the example.

Go Packages we developed for our games

Hajime Hoshi — Sat, 17 Feb 2018 06:54:08 +0000

We have released some games in Go like Clock of Atonement and Bluebird of Happiness. Unfortunately, we haven't made them open-source yet for some business reasons. However, we're now splitting the code into some public packages for games gradually so that Go game developers can utilize them, and game developing in Go would get more popular. In this article, I'd like to list the public Go packages we developed and used for the games!

`github.com/hajimehoshi/ebiten`

A dead simple 2D game library. Basically this is an OpenGL wrapper. This works on multiple platforms, not only desktop but also mobiles and even web browsers by GopherJS! Actually we were able to develop the games on desktops and browsers, and released for mobiles.

Ebiten (海老天) means a tempura of shrimp, which is one of my favorite Japanese food :-)

`github.com/hajimehoshi/go-mp3`

An MP3 decoder in pure Go. This is a port of PDMP3, an MP3 decoder in Public Domain license. As I just ported this automatically without understanding the spec, I am still struggling to understand what is going on in my lib :-)

The motivation I developed MP3 decoder instead of other decoder like Ogg is that we wanted to make games work on browsers. Now MP3 is supported by most modern browsers but others are not supported well (e.g. Ogg). We could utilize the browser-native decoder on browsers instead of MP3 decoder in Go for performance. MP3 had patent problems, but the patents were expired in 2017.

`github.com/hajimehoshi/oto`

A lower level audio lib. The great thing of this is portability: this works on Windows, macOS, Linux, Android, iOS and Web browsers. This just offers an io.Writer and you can play any sound by writing bytes to the writer. With go-mp3, you can play an MP3 file with io.Copy (Example)!

func run() error {
    f, err := os.Open("classic.mp3")
    if err != nil {
        return err
    }
    defer f.Close()

    // Decode MP3 stream and the result is an `io.Reader`
    d, err := mp3.NewDecoder(f)
    if err != nil {
        return err
    }
    defer d.Close()

    // Create an oto's player, which is an `io.Writer`
    p, err := oto.NewPlayer(d.SampleRate(), 2, 2, 8192)
    if err != nil {
        return err
    }
    defer p.Close()

    // With `io.Copy`, you can play the decoded MP3 on your machine!
    if _, err := io.Copy(p, d); err != nil {
        return err
    }
    return nil
}

Oto (音) means 'sound' in Japanese.

`github.com/hajimehoshi/file2byteslice`

Yet another tool to embed binary to Go file, just like go-bindata. file2bytesslice is dead simple compared to other similar packages. This just converts one binary to one byte slice literal. That's it! The motivation I developed this is that there was an incident: go-bindata's original author deleted their GitHub account and another person used the same GitHub account again, and I thought the repository is no longer trusty. Thus, I tried to develop it myself.

Usage of file2byteslice:
  -compress
        use gzip compression
  -input string
        input filename
  -output string
        output filename
  -package string
        package name (default "main")
  -var string
        variable name (default "_")

`github.com/hajimehoshi/go-mplusbitmap`

A package to offer font.Face object of M+ Bitmap font for 8/16 bitty style Japanese texts. font.Face is a semi-standard interface to represent a font face, and there is a TrueType parser in Go to make font.Face instances. You can draw the text with Ebiten's text package very easily! Of course, this can be used with (semi-)standard rendering package (Example).

const text = `Hello, World!

こんにちは世界!`

func run() error {
    const (
        ox = 16
        oy = 16
    )

    dst := image.NewRGBA(image.Rect(0, 0, 320, 240))

    // Initialize the destination image dst by filling with white.
    draw.Draw(dst, dst.Bounds(), image.NewUniform(color.White), image.ZP, draw.Src)

    f := mplusbitmap.Gothic12r

    // golang.org/x/image/font's Drawer draws a text on a specified image.
    d := font.Drawer{
        Dst:  dst,
        Src:  image.NewUniform(color.Black),
        Face: f,
        Dot:  fixed.P(ox, oy),
    }

    for _, l := range strings.Split(text, "\n") {
        d.DrawString(l)
        d.Dot.X = fixed.I(ox)
        d.Dot.Y += f.Metrics().Height
    }

    // Now the text is drawn to the image dst.

    return nil
}

`github.com/hajimehoshi/chinesegamefonts`

This offers (compressed) TTF binaries of Noto CJK fonts. This package's fonts are different from original Noto CJK fonts in some points.

The fonts are TrueType, not OpenType since there is no actually-usable Go package to parse OpenType fonts so far (golang.org/x/image/font/sfnt is under development).
Many glyphs are removed from the originals' and chinesegamefonts fonts include only common CJK glyphs to reduce file size. The original file sizes are 46.1 MB (both Simplified and Traditional), and the reduced ones are 4.4 MB (Simplified) and 7.6 MB (Traditional)!

Our games don't have Chinese support so far, but we're now implementing that with this font package.

Besides above packges, we are now developing packages e.g. for user interface. Stay tuned!

New Ebiten package inpututil

Hajime Hoshi — Mon, 12 Feb 2018 15:14:53 +0000

Hi all,

This is the very first post for Ebiten, my own game library in Go. I didn't have any places to announce the recent updates of Ebiten until now, and I started that here now!

I'd like to announce that I have introduced a new package, inpututil! With this package, for example, you can detect the key is pressed just in the current frame (IsKeyJustPressed). This is a very common idiom, and the package would help you do such things.

Enjoy!

摩訶般若波羅蜜多心経

Hajime Hoshi — Wed, 15 Nov 2017 14:22:48 +0000

観自在菩薩　行深般若波羅蜜多時　照見五蘊皆空　度一切苦厄
舎利子　色不異空　空不異色　色即是空　空即是色　受想行識亦復如是
舎利子　是諸法空相　不生不滅　不垢不浄　不増不減
是故空中無色　無受想行識　無眼耳鼻舌身意　無色声香味触法
無眼界　乃至無意識界　無無明亦　無無明尽　乃至無老死　亦無老死尽
無苦集滅道　無智亦無得　以無所得故　菩提薩埵　依般若波羅蜜多故
心無罣礙　無罣礙故　無有恐怖　遠離一切顛倒夢想　究竟涅槃
三世諸仏　依般若波羅蜜多故　得阿耨多羅三藐三菩提
故知般若波羅蜜多　是大神呪　是大明呪　是無上呪　是無等等呪
能除一切苦　真実不虚　故説般若波羅蜜多呪　即説呪曰
羯諦　羯諦　波羅羯諦　波羅僧羯諦　菩提薩婆訶
般若心経

Oreo

Hajime Hoshi — Wed, 15 Nov 2017 06:31:04 +0000

Oreo

Forem: Hajime Hoshi

Compiling a Go program into a native binary for Nintendo Switch™

This article has been moved to ebiten.org due to some complicated business reasons.

tl;dr

Caution

Background

Strategy

Hitsumabushi - rewriting the runtime with the -overlay option

hajimehoshi / hitsumabushi

Run Go programs (almost) everywhere

Implementation of Hitsumabushi

Replacements

Calling C functions from runtime

Ignoring signals

Implementing a pseudo file system

Implementing a pseudo memory system

Implementing pseudo futex

Adjusting the number of CPU cores

Entry point

Results

Remarks

Future works

Acknowledgments

GopherJS vs WebAssembly for Go

tl;dr

Go on browsers

Ebiten

Port GopherJS library to WebAssembly

Performance comparison

Binary size comparison

Appendix - How to do experiments

Install Ebiten and other libraries

Get the latest Go and compile it

Compile an Ebiten example for WebAssembly

Run GopherJS server

Go Packages we developed for our games

github.com/hajimehoshi/ebiten

github.com/hajimehoshi/go-mp3

github.com/hajimehoshi/oto

github.com/hajimehoshi/file2byteslice

github.com/hajimehoshi/go-mplusbitmap

github.com/hajimehoshi/chinesegamefonts

New Ebiten package inpututil

摩訶般若波羅蜜多心経

Oreo

Hitsumabushi - rewriting the runtime with the `-overlay` option

Calling C functions from `runtime`

Implementing pseudo `futex`

`github.com/hajimehoshi/ebiten`

`github.com/hajimehoshi/go-mp3`

`github.com/hajimehoshi/oto`

`github.com/hajimehoshi/file2byteslice`

`github.com/hajimehoshi/go-mplusbitmap`

`github.com/hajimehoshi/chinesegamefonts`