--- title: "Mounting Remote Disks Over Plain HTTP: Contributing to fs-relay" description: How Muayyad Alsadi's fs-relay mounts remote folders over HTTP 10x faster than s3fs, and the upstream work that made SQLite locks solid for v0.2.0. summary: A reflection on 20 years of Linux network mounts, why Muayyad Alsadi's fs-relay beats s3fs and NFS by 10x, and the upstream work on SQLite locking and v0.2.0. date: 2026-11-08T00:00:00.000Z heroImage: src: /_astro/hero.DPw8SuwK.webp width: 1360 height: 768 format: webp tags: - linux - fuse - go - networking - sqlite - open-source categories: - The Linux Lounge - Technology Bites published: true featured: true draft: false author: Jad href: /posts/mounting-remote-directories-over-http/ slug: mounting-remote-directories-over-http --- In 2004, setting up NFS across two beige desktop boxes in my room felt like inviting a thunderstorm indoors. You edited `/etc/exports`, restarted `portmap` and `rpcbind`, crossed your fingers, and waited for the whole desktop to freeze the moment someone tripped over an Ethernet cable. Anyone who grew up in [those early Linux days](/posts/open-source-ai-rebellion/) knows the specific dread of a dead network mount. A terminal tab running `ls` would enter an unkillable kernel sleep (`D` state). Even `kill -9` bounced off it. You either lived with a haunted directory tree or reached for the physical reset button. Two decades later, our workloads moved to cloud instances and container sandboxes, yet the friction of touching files across a wire never really disappeared. We traded manual NFS exports for `rsync` loops, background sync daemons, or `sshfs`. When you work from the Middle East, where [latency is a daily tax](/posts/latency-is-political-when-you-live-far-from-the-center/) on every round trip, chatty remote filesystems punish every command you type. At the end of October, [Muayyad Alsadi](https://github.com/muayyad-alsadi/) ([@muayyadalsadi](https://x.com/muayyadalsadi)) started building a clean-sheet answer to this problem called [fs-relay](/project/fs-relay/). Over the past week, I had the pleasure of contributing upstream fixes across locking, startup safety, shutdown, and self-update for today's `v0.2.0` release. This is the story of how Muayyad's design makes `fs-relay` ten times faster than the usual suspects, and what went into making multi-mount SQLite and atomic self-updates solid. What's in this post: - [Why traditional remote mounts struggle](#why-traditional-remote-mounts-struggle) - [How Muayyad made fs-relay 10x faster](#how-muayyad-made-fs-relay-10x-faster) - [Upstream work toward v0.2.0](#upstream-work-toward-v020) - [Try fs-relay v0.2.0](#try-fs-relay-v020) ## Why Traditional Remote Mounts Struggle When you want multiple machines or [coding agents](/posts/my-mcp-stack/) to share a live workspace without copying gigabytes back and forth, you hit three walls almost immediately: 1. Root privileges and firewall ports: Classic NFS and SMB expect root daemons, kernel modules, and open ports that are painful to expose across networks. 2. Round-trip chatty metadata: A simple `tar -xf` or project build issues thousands of `lookup`, `getattr`, `open`, `read`, and `close` calls. Sending each call as a separate network round trip turns a fast link into molasses. 3. Broken advisory locks: Most network mounts either ignore POSIX byte-range locks or fake them locally. Point two processes at the same SQLite database and you get silent corruption. ## How Muayyad Made fs-relay 10x Faster Muayyad designed `fs-relay` around a strict constraint: five Go packages, zero CGO, rootless Linux FUSE v3 (`fusermount3`) on the client, and standard HTTP and WebSockets on the wire. Instead of JSON or reflection-heavy codecs, `fs-relay` encodes frames with FlatBuffers (a recurring lesson for anyone who has watched [serialization overhead dominate Go hot paths](/posts/go-json-v2-explained/)). But the real speed comes from how the mount avoids round trips altogether: - Read-only opens take no server file handle. The client resolves `open`, `flush`, and `release` locally and reads by inode. - Directory listings (`readdir`) return entry attributes in the same call, then batch-prefetch up to 32 small regular files (`<= 64 KiB`, up to 1 MiB total) via `reads` and up to 32 child directories via `readdirs`. - Writes buffer contiguously up to 1 MiB on the mount before flushing, while large transfers bypass framing overhead over a raw socket backed by Linux `splice` pipes. - Instead of polling, the mount listens on one WebSocket or chunked `join` stream for server-pushed cache evictions and 2-second heartbeats. The numbers in the repository benchmarks speak for themselves. Unpacking `ansible.tar` (27 MiB across 5,868 files) takes 203.56 seconds on `s3fs` (mounted over `rustfs`) and 107.89 seconds on `fuse-nfs`. On `fs-relay`, the same unpack finishes in 15.47 seconds. Even more surprising, running DuckDB queries over Parquet files mounted through `fs-relay` beat DuckDB reading the exact same files directly from `rustfs` over native S3. ## Upstream Work Toward v0.2.0 Muayyad's code and `SPEC.md` are clear enough that jumping in was easy. The first contribution ([`9071dbd`](https://github.com/muayyad-alsadi/fs-relay/commit/9071dbd)) proposed dual licensing under MIT and the Waqf Digital Public License (`WaqfDPL-1.0`), which Muayyad adopted so the project stays free and open in perpetuity. From there, the work covered four practical engineering problems. ### 1. Server-Authoritative POSIX Locks and SQLite Coherence To run SQLite across two remote mounts without corruption, both mounts must agree on byte-range locks (`getlk`, `setlk`, `setlkw`) and `flock`. [PR #3](https://github.com/muayyad-alsadi/fs-relay/pull/3) added the server-side lock table in `internal/local/lock.go`. A classic POSIX lock belongs to a `(PID, inode)` pair, which breaks across machines when two hosts reuse the same PID. In `fs-relay`, a lock holder is the mount's HMAC-signed client ID paired with the kernel lock owner. Because SQLite's rollback journal hammers three exact byte offsets at `0x40000000`, the lock table tracks dedicated fast-path counters for the pending byte, the reserved byte, and the 510-byte shared range, spilling arbitrary ranges onto an overflow list only when needed. Getting the lock table working was only half the battle. Enabling a dedicated `go test -race` job in CI ([PR #25](https://github.com/muayyad-alsadi/fs-relay/pull/25)) and running multi-writer SQLite stress tests exposed subtle distributed races, fixed across [PR #24](https://github.com/muayyad-alsadi/fs-relay/pull/24), [PR #30](https://github.com/muayyad-alsadi/fs-relay/pull/30), [PR #33](https://github.com/muayyad-alsadi/fs-relay/pull/33), and [PR #34](https://github.com/muayyad-alsadi/fs-relay/pull/34): - Draining write-behind before unlock: If mount A writes a transaction and unlocks before its buffered bytes reach the server, mount B acquires the lock and reads stale bytes. `fs-relay-bind` now drains dirty buffers before sending `F_UNLCK` and drops cached attributes and prefetched file bodies the moment a lock grant arrives. - Closing wake-up races: When a writer downgrades to a shared read lock or a waiter receives `EAGAIN`, registering the waiter under the lock mutex ensures coalesced broadcast wake-ups never miss a blocked reader. - Guarding late grants: If a file handle closes while a lock request is still in flight, the client immediately releases the late grant so orphaned locks never wedge the namespace. We verified this on Linux 6.12 with SQLite 3.46.1 in rollback journal mode: four writers split across two mounts committed 200 rows concurrently, and `PRAGMA integrity_check` returned `ok` on both mounts. (`WAL` mode stays out of scope because it relies on an `-shm` file mapped into shared memory via `mmap` on a single kernel, which cannot be shared across separate machines.) ### 2. Startup Validation and Single-Server Data Locks Early on, a typo on the command line or a second `fs-relay` process pointing at the same data directory could cause confusing state. Across [PR #16](https://github.com/muayyad-alsadi/fs-relay/pull/16), [PR #17](https://github.com/muayyad-alsadi/fs-relay/pull/17), [PR #18](https://github.com/muayyad-alsadi/fs-relay/pull/18), [PR #20](https://github.com/muayyad-alsadi/fs-relay/pull/20), [PR #29](https://github.com/muayyad-alsadi/fs-relay/pull/29), and [PR #35](https://github.com/muayyad-alsadi/fs-relay/pull/35), we added non-blocking `-token-file` secret loading, default `http://` URL schemes, pre-mount directory validation (`-force-existing-mnt`), and an exclusive `.fs-relay.lock` file lock on the data root hardened against symlink traversal. ### 3. Bounded Unmount on Interrupts Anyone who has pressed `Ctrl+C` on a FUSE daemon while a shell is still sitting inside the mounted folder knows how easily shutdown can hang. In [PR #19](https://github.com/muayyad-alsadi/fs-relay/pull/19), [PR #37](https://github.com/muayyad-alsadi/fs-relay/pull/37), [PR #38](https://github.com/muayyad-alsadi/fs-relay/pull/38), and [PR #39](https://github.com/muayyad-alsadi/fs-relay/pull/39), we tightened CLI argument validation and reworked signal shutdown in `fs-relay-bind`. On `SIGINT` or `SIGTERM`, `fs-relay-bind` attempts a clean unmount bounded by a 2-second timeout. If open files or a stalled network flush keep the mount busy, it falls back to an interruptible lazy detach (`fusermount3 -uz` or `MNT_DETACH`) and exits non-zero with a clear message instead of hanging the terminal. ### 4. Two-Phase Atomic Self-Update in v0.2.0 For `v0.2.0` ([PR #40](https://github.com/muayyad-alsadi/fs-relay/pull/40)), we added `update`, `version`, and `help` subcommands to both `fs-relay` and `fs-relay-bind`, alongside a 12-hour cached background release check (`FS_RELAY_NO_UPDATE_CHECK` disables it). Updating two companion binaries (`fs-relay` and `fs-relay-bind`) in place has sharp edges. If the first binary swaps and the second fails, the machine is left running mismatched versions. Our updater verifies the release tarball against the GitHub SHA-256 digest (falling back to `checksums.txt`), preserves existing file permissions (such as `0700`), stages both replacement binaries in exclusive temporary files in the target directory, and rolls back the primary binary automatically if the companion commit fails. ## Try fs-relay v0.2.0 There is genuine satisfaction in contributing to software that refuses unnecessary layers. You do not need root daemons, kernel modules, or heavyweight cluster coordinators to mount a directory across machines. Standard HTTP, careful FUSE caching, and server-side POSIX locks get the job done in five small packages. Starting a share, mounting it on a client, and updating both binaries in place takes three commands: ```bash fs-relay -listen 127.0.0.1:8080 -data ./data -token-file ./token.txt fs-relay-bind -url 127.0.0.1:8080 -ns demo -token-file ./token.txt /mnt/demo fs-relay update ``` You can grab the `v0.2.0` binaries or read the design contract in [Muayyad Alsadi's fs-relay repository](https://github.com/muayyad-alsadi/fs-relay), or check the summary card on the [project page](/project/fs-relay/). Have you tried replacing `sshfs` or `rsync` loops with user-space FUSE over HTTP in your own workflows, or are you still nursing old NFS mounts?