Deploy Headscale as the non-HTTP TCP connectivity layer (replaces mesh0) #35

Open

opened 2026-08-06 05:58:17 +00:00 by agent.odin · 1 comment

agent.odin commented

2026-08-06 05:58:17 +00:00

Collaborator

Stable ID: FM-HEADSCALE
Source: owner ruling 2026-08-06 (#32 mesh decommission)
Disposition: open — spike, gated on one container check
Blocked by: nothing

Decision

Headscale replaces mesh0 as the non-HTTP TCP connectivity layer. Every host
gets a stable 100.x.y.z address that bypasses Cloudflare entirely.

Resolves: SSH discovery without CF (#32 Q2), monitoring scraping path, sops
key trust transport, mobile client connectivity (the original mesh0 failure).

Why Headscale

headscale is in nixpkgs; services.headscale module exists
Tailscale clients (services.tailscale) on each host point at the self-hosted
login server — no SaaS dependency
DERP relays handle NAT traversal where direct hole-punching fails (the
property mesh0 lacked — it died on mobile)
MagicDNS gives each host a stable name, not just an address
Arbitrary TCP tunnels (SSH, scraping, agents) — not limited to HTTP(S)

Scope

Headscale control server. Deploy on bifrost (or a small VPS).
services.headscale.enable, config: server_url, base_domain,
DERP map, listen addresses. One flake module.
Per-host tailscale client. services.tailscale.enable +
extraUpFlags = ["--login-server" "https://headscale.et0.pw"] on each
fleet host. Pre-auth keys minted via headscale CLI.
just ssh update. Today resolves via tofu output. Adds headscale
address as the primary path, tofu as fallback. Known_hosts survives —
stable address, same host key.
DNS. headscale.et0.pw for the control server. MagicDNS for host names
(optional — 100.x.y.z addresses work directly).
Mobile. Tailscale app on phone, pointed at custom login server. This is
the connectivity that mesh0 failed to provide.

The one check that gates everything

Does /dev/net/tun exist in a vpsFree container?

tailscaled needs either /dev/net/tun (kernel networking) or userspace
networking mode (--tun=userspace-networking). The container may not expose
the tun device — same class of constraint that killed nested KVM
(migration plan §Phase 3.4).

If tun exists: standard deployment. Half a day.
If not: userspace networking works but is slower and can't route traffic
for other hosts (subnet router mode unavailable). May be acceptable for
SSH/scraping but limits the overlay's utility.

Five-minute check on a running container: ls -la /dev/net/tun.

Non-scope

Replacing HTTP(S) routing. CF + Caddy stays for browser-facing services.
Headscale is for non-HTTP TCP only.
Authelia/OIDC. Separate concern (#33). Headscale authenticates devices;
Authelia authenticates users.

Acceptance

/dev/net/tun availability checked on a vpsFree container
services.headscale deployed and serving pre-auth keys
At least two hosts joined (bifrost + one server) with stable addresses
just ssh works over headscale address
Mobile client connected
DERP relay verified (simulate NAT: connect two hosts behind NAT, confirm
traffic flows)
just check green

Provenance

Owner ruling 2026-08-06 during mesh decommission (#32) SSH discovery
discussion. Freenet evaluated and ruled out (content routing, not TCP tunnel).
Headscale is the known-quantity overlay that solves the discovery problem
mesh0 was meant to solve, with mobile properties mesh0 lacked.

Stable ID: FM-HEADSCALE Source: owner ruling 2026-08-06 (#32 mesh decommission) Disposition: open — spike, gated on one container check Blocked by: nothing ## Decision Headscale replaces mesh0 as the non-HTTP TCP connectivity layer. Every host gets a stable `100.x.y.z` address that bypasses Cloudflare entirely. Resolves: SSH discovery without CF (#32 Q2), monitoring scraping path, sops key trust transport, mobile client connectivity (the original mesh0 failure). ## Why Headscale - `headscale` is in nixpkgs; `services.headscale` module exists - Tailscale clients (`services.tailscale`) on each host point at the self-hosted login server — no SaaS dependency - DERP relays handle NAT traversal where direct hole-punching fails (the property mesh0 lacked — it died on mobile) - MagicDNS gives each host a stable name, not just an address - Arbitrary TCP tunnels (SSH, scraping, agents) — not limited to HTTP(S) ## Scope 1. **Headscale control server.** Deploy on bifrost (or a small VPS). `services.headscale.enable`, config: `server_url`, `base_domain`, DERP map, listen addresses. One flake module. 2. **Per-host tailscale client.** `services.tailscale.enable` + `extraUpFlags = ["--login-server" "https://headscale.et0.pw"]` on each fleet host. Pre-auth keys minted via headscale CLI. 3. **`just ssh` update.** Today resolves via tofu output. Adds headscale address as the primary path, tofu as fallback. Known_hosts survives — stable address, same host key. 4. **DNS.** `headscale.et0.pw` for the control server. MagicDNS for host names (optional — `100.x.y.z` addresses work directly). 5. **Mobile.** Tailscale app on phone, pointed at custom login server. This is the connectivity that mesh0 failed to provide. ## The one check that gates everything **Does `/dev/net/tun` exist in a vpsFree container?** tailscaled needs either `/dev/net/tun` (kernel networking) or userspace networking mode (`--tun=userspace-networking`). The container may not expose the tun device — same class of constraint that killed nested KVM (migration plan §Phase 3.4). - **If tun exists:** standard deployment. Half a day. - **If not:** userspace networking works but is slower and can't route traffic for other hosts (subnet router mode unavailable). May be acceptable for SSH/scraping but limits the overlay's utility. Five-minute check on a running container: `ls -la /dev/net/tun`. ## Non-scope - **Replacing HTTP(S) routing.** CF + Caddy stays for browser-facing services. Headscale is for non-HTTP TCP only. - **Authelia/OIDC.** Separate concern (#33). Headscale authenticates devices; Authelia authenticates users. ## Acceptance - [ ] `/dev/net/tun` availability checked on a vpsFree container - [ ] `services.headscale` deployed and serving pre-auth keys - [ ] At least two hosts joined (bifrost + one server) with stable addresses - [ ] `just ssh` works over headscale address - [ ] Mobile client connected - [ ] DERP relay verified (simulate NAT: connect two hosts behind NAT, confirm traffic flows) - [ ] `just check` green ## Provenance Owner ruling 2026-08-06 during mesh decommission (#32) SSH discovery discussion. Freenet evaluated and ruled out (content routing, not TCP tunnel). Headscale is the known-quantity overlay that solves the discovery problem mesh0 was meant to solve, with mobile properties mesh0 lacked.

agent.teyla commented

2026-08-17 18:44:36 +00:00

Grooming state: Superseded
Evidence: lar.ad/flake-ops#239 (Epic: Deploy vegr overlay (Nebula) — Phase 1 v4-safety, ratified 2026-08-17). Impl pick swapped Headscale → Nebula; hostile-mobile explicitly out of scope; heimdallr as lighthouse re-ratified (see vegr R6 and this repo's #165 provenance).
Body edited in place to prepend the supersession banner and updated Decision; historical Headscale rationale preserved beneath per grill rule 5 (preserve historical decisions as provenance).
Ticket disposition: stays open until vegr epic #239 ships. FM-HEADSCALE stable ID stays owned here.

**Grooming state:** Superseded **Evidence:** [lar.ad/flake-ops#239](https://jo.et0.pw/lar.ad/flake-ops/issues/239) (`Epic: Deploy vegr overlay (Nebula) — Phase 1 v4-safety`, ratified 2026-08-17). Impl pick swapped Headscale → Nebula; hostile-mobile explicitly out of scope; heimdallr as lighthouse re-ratified (see `vegr R6` and this repo's [#165](https://jo.et0.pw/lar.ad/flake-ops/issues/165) provenance). **Body edited in place** to prepend the supersession banner and updated Decision; historical Headscale rationale preserved beneath per grill rule 5 (preserve historical decisions as provenance). **Ticket disposition:** stays open until vegr epic #239 ships. FM-HEADSCALE stable ID stays owned here.

Commenting is not possible because the repository is archived.