Skip to main content

Command Palette

Search for a command to run...

I tried to put Go inside Python. A Unix socket won.

Five abandoned Python-Go bridges, then the Unix socket that shipped.

Updated
9 min readView as Markdown
I tried to put Go inside Python. A Unix socket won.

There's a dead branch in the locatr repo called feat/experiment-wasip1. On it, every string that crosses the border between Python and Go gets folded into a single unsigned 64-bit integer: pointer in the top half, length in the bottom.

const MASK uint64 = (1 << 32) - 1

func copyBufferToMemory(buffer []byte) uint64 {
	bufferPtr := &buffer[0]
	unsafePtr := uintptr(unsafe.Pointer(bufferPtr))

	ptr := uint32(unsafePtr)
	size := uint32(len(buffer))

	return (uint64(ptr) << uint64(32)) | uint64(size)
}

I wrote that in August 2024, four days after locatr's initial commit. This post follows five abandoned ways to let Python call a Go library, then the Unix socket that shipped.

The library and the gap

locatr takes a plain-English description of a UI element ("search bar", "the checkout button") and returns a working selector for it. Under the hood it injects JavaScript to minify the page's DOM, hands that to an LLM, gets back an element id, and maps it to a locator your automation driver can click. Two capabilities make the whole thing go: a live browser it can evaluate JavaScript in, and an HTTPS connection to an LLM API. Keep those two in mind; they decide everything that follows.

I wrote locatr's first commit on August 15, 2024, at Vertexcover. The experiments in this post are mine: the initial architecture, the wasm branches, the gRPC layer, and the first IPC implementation. Teammates later hardened the IPC path into what ships today, and one of them took a swing at the FFI route I'll get to near the end.

The gap was visible from day one, and so was the ambition. The library is Go because our stack is Go. The people who want it write Python, because browser automation lives where Playwright, Selenium, and Appium users are. The README from that first day already lists the plan under Future Plans: "compile the core BaseLocatr to wasm and use wasmer for using Locatr in multiple languages." The plan was in the repository before the experiments began.

Attempt 1: the browser target, without a browser

Go's oldest WebAssembly target is GOOS=js GOARCH=wasm, built for running inside a JavaScript host. Its escape hatch, syscall/js, lets Go reach out and touch JS values. Two days after the initial commit I had locatr compiling for it.

The problem is in the target's name. syscall/js assumes a JavaScript engine on the other side; our callers have a Python interpreter. Running that artifact from Python means embedding Node or simulating enough of a JS host to fool the Go runtime, which is a strange dependency chain for a library whose pitch is "no extra moving parts."

That experiment left a fossil: commit 97d62d0 checks main.wasm straight into git, all 4,150,936 bytes of it. The artifact includes the Go runtime, garbage collector and scheduler included. Four megabytes for a locator library, committed to version control at 1:55 in the afternoon.

Attempt 2: WASI Preview 1 meets the sandbox

WASI was the version of the dream with a standard behind it: compile to wasip1 (stock Go grew the target in 1.21), get a portable sandboxed binary, and run it from any language with a wasm runtime. The result should have been one artifact callable outside a JavaScript host.

Preview 1's system interface covers files, clocks, arguments, environment variables, randomness, stdio. No sockets. No browser, obviously. Read the locatr requirements again: an HTTPS line to an LLM and a live browser to evaluate JavaScript in. The two capabilities the library cannot exist without are precisely the ones the sandbox does not have.

So the sandbox gets inverted. The guest imports the missing capabilities from the host:

//go:wasmimport env wasiEvaluateJs
func wasiEvaluateJs(jsStrPtr uint64) uint64

//go:wasmimport env wasiHttpPost
func wasiHttpPost(urlPtr uint64, headersPtr uint64, bodyPtr uint64) uint64

Python implements both imports. wasiEvaluateJs calls page.evaluate(...) on the caller's Playwright page. wasiHttpPost calls requests.post(...). The LLM client inside the Go code grew an injectable transport, because net/http has nothing to dial with under wasip1.

Then comes the ABI. The functions exposed by this prototype accept numeric values, not Go strings or structs. Every string needs the dance: Python calls the module's exported malloc, writes UTF-8 bytes into linear memory through a uint8_view, packs pointer and length into the u64 from the top of this post, and passes that. Go unpacks it and reads its own memory byte by byte through unsafe.Pointer arithmetic. Results travel the same way in reverse, with Python responsible for calling free.

def _to_wasm_string(self, string: str) -> int:
    string_bytes = string.encode("utf-8")
    length = len(string_bytes)
    ptr = self._wasm_malloc(length)
    memory = self._wasm_memory.uint8_view(ptr)
    memory[:length] = string_bytes
    return (ptr << 32) | length

The commit log from August 19 reads like a man bargaining with an instruction set: accept combined ptr_size instead of 2 seperate ptr and size, then pass pointer for locatr instead of global. The latter replaced a global singleton with InitLocatr returning:

uint64(uintptr(unsafe.Pointer(app)))

That is a Go heap pointer handed to Python as a session handle, with nothing on the Go side telling the garbage collector the object is still owed to someone.

Two more walls, briefly. The //export directives that make InitLocatr and GetLocatorStr visible to the host are a TinyGo convention; stock Go had no way to export a function to a wasm host at all until go:wasmexport landed in Go 1.24, in February 2025, six months after this branch. And the Python-side runtime was wasmer-python, whose last release predates Python 3.12 entirely, with the Cranelift compiler shipped as a separate wheel.

The honest part: it ran. Strings crossed, the LLM answered, a locator came back. You can also see exactly where the belief gave out, because the branch froze mid-shortcut. GetValidLocator, the step that's supposed to test candidate selectors against the real DOM, ends in return locators[0] with the actual querySelectorAll loop sitting above it, commented out.

Every future feature had the same price sheet: one more hand-rolled host function, one more u64-packing ceremony on both sides of the wall. August 19 was the branch's last commit.

WIT described the boundary I had hand-built

Everything ugly about that branch has a first-class answer in WASI Preview 2's component model. Interfaces are declared in WIT, a typed IDL, and I know because I wrote one:

interface host {
    evaluate-js: func(code: string) -> result<string, string>;
}

interface guest {
    init-locator: func(config: llm-config) -> result<locatr, string>;

    resource locatr {
      get-locator-str: func(user-req: string) -> result<string, string>;
    }
}

world locatr {
    import host;
    export guest;
}

WIT gives the boundary strings, typed errors, and a locatr resource with methods. The wasip1 branch had represented that resource by smuggling a raw pointer through a u64. The world also declares the capability contract explicitly: this component imports evaluate-js from whoever hosts it. It is the hand-built architecture written down as an interface.

The catch, in August 2024: Go could not emit components. The mature bindings lived in Rust, so the prototype did too. It used wit-bindgen 0.30 on the guest side and wasmtime's Python package on the host. Its entire business logic was a stub returning Ok("Dummy locator string"). The interface worked, but adopting it meant rewriting a working Go library in Rust for a cleaner boundary.

That prototype directory still exists on my disk. It has a .git folder and zero commits.

gRPC, and the FFI branch

Between the wasm attempts and the thing that shipped, two more doors got opened. On August 27 I added a gRPC server with response caching: it worked, and it also meant a proto toolchain, codegen on both sides, and service lifecycle management for what is conceptually a local function call. It got parked. That November a teammate opened a branch titled wip: ffi, the classic route: compile Go to a C shared library, load it from Python over ctypes. Two garbage collectors sharing a border crossing, a per-OS build matrix, and cgo in the loop. It never merged.

The Unix socket that shipped

The commit that ended the experiments is dated October 22, 2024: refactor plugin logic and add experimental ipc support. Teammates then did the unglamorous work that makes something real: separating the IPC layer, adding cached plugin support, wiring in Selenium and Appium.

The shipped design is almost aggressively boring. Python keeps the browser, because it already has one: whatever Playwright or Selenium session the user's script owns. The Go side runs as a separate process listening on a Unix domain socket, and connects to that same browser over CDP. Messages are JSON with a 4-byte big-endian length prefix and a 3-byte version handshake. A map from client id to locatr instance replaces the pointer-as-handle trick.

The Go process keeps real net/http and a real CDP client, so neither capability has to be imported through a wall. The interface is length-prefixed JSON, with encoding/json as the ABI on the Go side. The operating system supplies the process boundary. Distribution becomes one static Go binary per target OS, using the project's existing build tooling. The socket won because it put no artificial boundary between the library and the capabilities it needed.

Would I take the detour again?

The toolchain has changed since the experiment. go:wasmexport shipped in Go 1.24, and TinyGo supports both wasip1 and wasip2. A new attempt would not start with exactly the same compiler limitations.

The sharper lesson is architectural. locatr needs a browser and a network line. Once the guest imports "evaluate JavaScript in the caller's browser" and delegates HTTPS back to the host, much of the intended isolation has moved outside the module. For this library, a subprocess on a socket gave up little isolation that the WASI prototype had actually retained.

The branches are all still up: feat/experiment-wasip1, with the pointer-packing, the commented-out validator, and the 4 MB wasm binary in its history.


locatr is open source at github.com/vertexcover-io/locatr. Samrid Pandit is a backend engineer at CrowdVolt . Reach him at [email protected].