Forem: gerwert

Calling Swift from Go

gerwert — Wed, 30 Jan 2019 20:03:30 +0000

Both Swift and Go are modern compiled languages. They have their own typical use cases, strengths and weaknesses. Would it not be great to combine strengths of both languages in a single project?

One possible use case is when you want to use Cocoa API's (native to macOS) such as NSUserNotificationCenter in an existing Golang code base. This can be done via Objective-C as intermediate language. But I very much prefer Swift over Objective-C.

This "best of both worlds approach" has some caveats of course, but we'll come to that later. First the fun part.

A library as a bridge

There is no direct interoperability between Swift and Go. But what we can do, is using a generic way to link binaries written in arbitrary languages: shared libraries. Or more specific in this case: dynamic libraries.

Prerequisite is that both languages support the same calling convention. Both Golang and Swift support creating and linking to libraries that use the cdecl calling convention, which originates from the C programming language. If we build a Swift library that exports functions with cdecl calling convention, we should be able to call those functions from Go.

We have to make sure to explicitly export functions with the cdecl convention in Swift. Calling Swift functions in a "normal" Swift library from Go is not possible, because Swift and Golang have their own (and different) calling conventions.

But even if both languages would use the same calling convention, we would need some way of converting argument and return types from one language to the other. For example, a string in Golang is completely different from a String in Swift. So we would not be able to simply pass strings between Swift and Golang without some kind of layer in between.

In our case that layer is a dynamic C library. Thus, we'll use C as lingua franca, which is common when bridging programming languages.

Now, let's look at the steps needed create a C library from Swift code that can be called from Go.

Creating a Swift library

Let's create a very simple Swift library, that exports a single function that prints a string. First step is initialise our project using Swift Package manager:

$ swift package init --type library

Now we can create a Swift file that contains our function: ./Sources/go-swift/go_swift.swift.

In this file we declare one function: sayHello. This function
accepts a pointer to a C string as the only argument:

@_cdecl("sayHello") // export to C as `sayHello`
public func sayHello(namePtr: UnsafePointer<CChar>?) {    
    // Creates a new string by copying the null-terminated UTF-8 data (C String) 
    // referenced by the given pointer.
    let name = String(cString: namePtr!)
    print("Welcome, \(name)!")
}

To make this Swift function accessible from C, we have to add a @ _cdecl attribute to it.
@_cdecl is an undocumented and unsupported attribute (so might not be available anymore in future Swift versions), which gives a Swift function a "C name" in the resulting library. It tells the compiler not to mangle the function name (as it does by default), ensuring our function name is exported as if it had been written in C.

Also notice that we can't use the build-in Swift String as type for our argument, because that is not a C compatible type. Therefore we're dealing with raw pointers to C-style null-terminated strings: UnsafePointer<CChar>?.

Call library from Go

After building the swift library via $ swift build, we can try to call the sayHello function in that library.

To call our function from Go, we need cgo. Cgo acts as a bridge between C and Go.

But before we can link the Swift library to our Go program using cgo, we need to create a C header file: ./include/go_swift.h. Otherwise our Go program is not aware of the C function signature of sayHello.

Since we only have a single function, our header file can consist of a single line:

void sayHello(char* name);

With $ swiftc -emit-objc-header <swift file> you can generate an Objective-C header automatically. But as far as I'm aware, there is no automatic generation for plain C header. So we'll have to create the C
header file manually.

For simple functions this is relatively straightforward. For more complex functions you might need (lots of) trial and error. I didn't go as far for example to test if it's possible to pass a pointer to a function as parameter.

After defining the header, it's time to write a small Go program:

package main

/*
#cgo CFLAGS: -I./include
#cgo LDFLAGS: -L.build/debug/ -lgo_swift
#include <stdlib.h>
#include "go_swift.h"
*/
import "C"
import "unsafe"

func main() {
    // Copy Go string to C string
    cstr := C.CString("Bob")

    // Call Swift function `sayHello`
    C.sayHello(cstr)

    // The C string is allocated in the C heap using malloc.
    // Therefore memory must be freed when we're done
    C.free(unsafe.Pointer(cstr))
}

In our Go file, we import the C package and declare a preamble. From the cgo documentation:

To use cgo write normal Go code that imports a pseudo-package "C". The Go code can then refer to types such as C.size_t, variables such as C.stdout, or functions such as C.putchar.

If the import of "C" is immediately preceded by a comment, that comment, called the preamble, is used as a header when compiling the C parts of the package. For example:

In the preamble, we need to set some linker flags:

cgo CFLAGS: -I./include: set path to header files
cgo LDFLAGS: -L.build/debug/ -lgo_swift sets the location in which to look for library files, and which libraries to link to. go_swift is library the name from Package.swift. The library location is path that contains the output from the Swift build process.

Finally, we can build an run our Go program:

$ go build -o hello
$ ./hello
Welcome, Bob!

This works fine! But, as mentioned before, there are several issues with this approach, compared to "normal" development in either Golang or Swift.

Memory management

Both Swift and Go have automatic memory management, such as memory allocation and garbage collection. Therefore, normally when writing code in Swift or Go we don't need to worry about things like memory leaks and buffer overflows.

In order for automatic memory management to work, a runtime needs to be fully in charge of all memory allocated and to know the location of all pointers. That means Swift's runtime can't manage pointers created by Go's runtime, and visa versa.

Because of this, there are restrictions when passing pointers via a shared library. Go pointers passed to C code must be pinned for the duration of the C call, and C code must stop storing the Go pointer when the C function returns. If you don't strictly adhere to these and other rules, things can become messy quickly.

Bottom line is you must be careful and cautious about memory management. That's a big disadvantage, because you'll lose the convenience and safety of automatic memory management when using functions from a shared library. The type name UnsafePointer we have to use in the Swift code says it all.

Performance

Using C as an intermediate language means we need to convert Go types to C types, C types to Swift types, and back again. This, amongst other things, causes overhead that will impact application performance. For simple programs this might not be a problem at all. But when performance matters, there is always a better solution than bridging languages via a shared library.

Cross-platform support

While Go is available on many platforms, Swift is not. By linking a Go program to a Swift library you'll loose the broad platform support that Golang offers.

So, is this a good idea?

"Yes", because it's fun (!) and bridges a gap between Golang and native MacOS frameworks. But also "No", because, because calling Swift from Go creates issues that would not be there when using a single language.

In other words, whether or not choosing the Golang/Swift route is for you, depends on your project and if there is another way to achieve what you want. For example, MacOS APIs can be called using Objective-C as intermediate language instead.

But, that aside, it is something definitely worth experimenting with.

Repository with example code available on Github.

The other way around:
- Writing Apps in Go and Swift
- Calling Go Functions from Other Languages

On building a small cross-platform CLI tool in C, Go & Swift

gerwert — Fri, 25 Jan 2019 13:03:28 +0000

A great thing about being a programmer is that if you need specific, customised, tooling, you can just write it yourself. Often there are existing options, but of course it’s a lot more fun to write your own tools, especially when you have room for some experimentation.

This is how I came to write a simple tool.. 3 times, in different languages.

The experiment

My goal was to write a very simple command line tool that can generate one-time passwords compatible with Google Authenticator. Google authenticator uses the Time-based One-Time Password algorithm (TOTP) to generate codes. Instead of writing my own implementation, I wanted to use an existing TOTP library, since there already are many good ones.

Essentially, all I want my tool to do, is accept a secret as single input, then call an exiting TOTP library to generate a code, and print the generated access code to the standard output.

The question I was asking myself was: suppose I would like to use the tool on several platforms (Mac, Windows, Ubuntu), and would like to

distribute the tool amongst a small group of — not necessarily technical — people (e.g. colleagues), what programming language would be the most pragmatic/viable/fun option?

Of course you can look at this question from many angles. Let’s focus on building and distributing the tool. Then, these were my “should have” requirements:

It should be possible to distribute the tool as single executable that works “out of the box”, meaning the user does not have to install dependencies like runtimes, frameworks, libraries, etc.
With the same code base (but possibly different toolchains) it should be possible to produce builds for multiple platforms.

Language choice

I wanted to create a binary for this specific experiment, that’s why I did not consider interpreted languages like Node.js, Ruby, and Python for this specific tool. Although, of course in general these languages would all make perfectly viable options to use for writing a cross-platform command line tool.

There is also a disadvantage to those languages, being that the end user needs to have a runtime (e.g. Node.js) installed. Although many platforms come with common runtimes pre-installed, the user might need to install a different version. That is not always a trivial task for non-technical users.

(I’m aware that there are tools to compile interpreted languages to stand-alone executables, but that feels a bit like cheating here).

In the end, my choice was to experiment with C , Go and Swift.

I decided to stay in my “programming language comfort zone”, because learning a new language was not part of my experiment. Therefore I did not experiment with (in opinion) very interesting other languages, such as Rust , which I will try out in the future (feel free to leave a comment with your Rust experiences). Also good to note maybe: for this experiment I considered C++ overkill (or actually maybe, my C++ knowledge is just lacking).

What I learned

C

Typically, executables build with C are linked dynamically. That means that end users need to install dependencies (linked libraries) in order to run the tool. That's definitely not ideal.
There are ways around this, but these all come with some disadvantages:
Static linking: create single binary that will hold all required binary code. But, that requires that all libraries that you use (for example a TOTP library) support static linking. This is definitely not always the case. Furthermore, Appledoes not support statically linked binaries on Mac OS X.
Distribute linked dynamic libraries with your application. This means that for every target OS you’ll have to pre-build all linked libraries, make sure these libraries can be found by the executable (e.g. changing rpath on macOS), and bundle them with the app. In other words you need compile and bundle .dll (Windows), .dylib (macOS) or .so (Linux) files with your app.
C does not have a runtime that needs to be bundled with the application. Therefore the resulting executable is quite small. The only dependency (dynamic library) is the C standard library libc, which is by default available on the OS'es I would like to target.
Building a single C code base on different platforms can be a pain. I generally prefer to use the “default”, or most widely supported, build chain for a platform. In my opinion that is Visual Studio on Windows, Xcode on Mac (or GCC on Mac command line) and GCC on Linux. But that means that for every platform you need to install and setup a completely different build environment (project file, build scripts, etc).
Compiling dependencies from source for multiple platforms is hard. Like I mentioned above, setting up build chain for your own code on different platforms can be difficult already. It is even more difficult to compile third party libraries from source for multiple platforms. Some are relatively easy to work with cross-platform, but others are a real pain because they lack support for, or documentation on, cross-platform building.

Executables build by Golang are statically linked by default. This means users don’t have to install any dependencies, and you don’t need to distribute dynamic libraries with your application. For a small command line application, the only thing you need to distribute is the executable.
Unfortunately, because of the static linking, the resulting executable is relatively large. That is because a Go binary includes the Go runtime, so that the end user does not need to have Go installed. (but, as Dotan Nahum, points out, there are ways to trim down some fat)
Go is available as a binary distribution on all target platforms I was interested in. That makes setting up a build environment and building on these platforms painless.
A great thing about Go, is that you can easily compile for multiple platforms on one machine.

Swift

It is recommended to statically link to the Swift standard library, so that the resulting executable is not bound to the specific version of Swift that it was built with. This results in a large binary (more than 10 MB for a simple tool). The need for static linking is due to Swift lacking ABI stability. That is on the roadmap to be solved in a future Swift version though. (In comparison, Objective-C does have ABI stability by the way).
Cross-platform support has not yet matured. You can compile a Swift program on both on Mac and Linux (there is no official Windows release yet), but the cross-platform build system — Swift Package Manager (SPM) — is not nearly as mature as Xcode on MacOS. Furthermore, many libraries that are available on CocoaPods or Carthage (MacOS only) don’t support SPM (cross-platform) yet.

Conclusion

When it comes to building cross-platform and distributing the tool, Go gave me the best developer experience.

Thanks to the default static linking, it is easy to create a single executable for distribution.

Building a Go program on different platforms is also really easy. There is no need for writing platform specific build scripts, or using platform dependent tool chains.

Downside is that the resulting executable is relatively large (several megabytes), but in my situation that was not a real issue.

Next up is C. Writing in C always gives me a pleasant sense of control and a feeling of freedom due to the lack of constraints from a runtime. Of course the downside to this is that you can easily shoot yourself in the foot. But the biggest problem here was that there is no single toolchain for building that works just as flawlessly cross-platform as Go.

And finally, Swift. While I really like Swift as a language, and I would only consider Swift as obvious choice when writing command line tools specifically for macOS. Swift is too much of a “moving target” for me. That has several implications, an important one is that it is not straightforward to use Swift on other platforms. Another issue for me is that Windows is not yet officially supported.

As a final note: I wanted to share my experiences, but in the end, what language suits you best comes down to personal preference and current state of languages. Next year might be different.