env-builder

Companion service to JupyterHub. Phases 2 + 3 of the per-notebook conda env paradigm (architectural decisions live in docs/notebook-envs-phase-0-decisions.md).

What it does

Receives manifest YAML from the Hub, runs micromamba to resolve a pinned explicit lock and install it on the shared /workspace volume, writes the lock + status back to Supabase's notebook_environments table. Stateless except for an in-memory job queue.

Phase 2 produced the lock only. Phase 3 extends this service with end-to-end env-on-disk lifecycle: per-notebook envs land at /workspace/<workspace_id>/<notebook_id>/.env/ with atomic move semantics (failed builds never displace the live env), shared package cache via hardlinks (100 notebooks sharing pandas → one copy on disk), and on-disk audit trail at .env-meta/.

Lifecycle

Hub: POST /api/notebooks/:id/environment/build
  └─ HMAC-signed POST → env-builder /jobs
       └─ enqueue
       └─ worker:
            ├─ DB: build_status='solving'
            ├─ micromamba create --prefix /workspace/<ws>/<id>/.env.tmp/ ...
            ├─ micromamba env export --explicit → lock body
            ├─ DB: build_status='building'  (callback fires here)
            ├─ atomic swap:
            │     · rm -rf .env.previous
            │     · mv .env → .env.previous (rollback target, Phase 6)
            │     · mv .env.tmp → .env
            ├─ write .env-meta/{manifest.yaml, lock.yaml, build.log}
            └─ DB: build_status='ready', lock_yaml=<...>, last_build_*=now

On-disk layout (per Phase 0 D4)

/workspace/
├── .mamba-cache/                     # global pkg cache, hardlink source
└── <workspace_id>/
    └── <notebook_id>/
        ├── *.ipynb                   # user files
        ├── .env/                     # current usable env
        ├── .env.previous/            # last good env (Phase 6 rollback)
        └── .env-meta/
            ├── manifest.yaml
            ├── lock.yaml
            └── build.log

Endpoints

HMAC contract

The Hub computes:

sig = hex(hmac.sha256(ENV_BUILDER_SHARED_SECRET, raw_body))

and sends header X-Node-Signature: <sig>. env-builder recomputes and compares with hmac.compare_digest. Mismatch → 401.

Required environment variables

Optional knobs

Railway volume setup (Phase 3 requirement)

Both this service and the JupyterHub service MUST mount the SAME Railway volume at /workspace. Without it, env-builder writes to ephemeral container storage and the Hub-side kernel spawner can't find the env on disk.

In the Railway dashboard:

1. Create a volume named node-workspace (or any name; consistency matters across services).
2. Attach it to the JupyterHub service at mount path /workspace.
3. Attach the SAME volume to this env-builder service at the same mount path /workspace.
4. Set ENV_BUILDER_WORKSPACE_VOLUME_ROOT=/workspace (or override).

Local development

docker build -t node-env-builder .
docker run --rm -p 8000:8000 \
  -e SUPABASE_URL=... \
  -e SUPABASE_SERVICE_ROLE_KEY=... \
  -e ENV_BUILDER_SHARED_SECRET=$(openssl rand -hex 32) \
  node-env-builder

Deployment (Railway)

railway.json wires Railway to build from this Dockerfile and run the ASGI server via uvicorn. After deploy, set the env vars above in the service dashboard. The Hub side reads ENV_BUILDER_URL (the public Railway URL of this service) + the same ENV_BUILDER_SHARED_SECRET.

What it doesn't do (yet)

• Build log streaming via SSE: Phase 5 nicety, currently the Hub polls the DB for the truncated tail.
• Persistent job queue: Phase 7. Today the queue is in process memory; service crash loses in-flight jobs, the user retries.
• Per-workspace quotas / GC: Phase 7.
• Multi-version env history: Phase 6. Today only .env/ (current) and .env.previous/ (last good) survive; older builds are evicted on the next successful build.
• Per-notebook env owner UID coordination: Phase 7. Today envs are written as root inside the env-builder container, mode 755 so the kernel reads but can't __pycache__-write inside .env/. Python tolerates this gracefully.

What it deliberately does

• Verifies HMAC on every write request: even on Railway's internal network — defense-in-depth against future leaks of the URL.
• Times out long solves: 10-minute cap. micromamba on a sane manifest resolves in seconds; anything longer is a pathological dependency graph or a stuck network call.
• Truncates build logs: full log goes to disk (Phase 3); DB column keeps the last 100 KB, which is what the UI shows.
• Classifies failure reasons: solver_conflict, channel_unreachable, timeout, runtime_error. The full text is in build_log; the tag is for queryability + Phase 7 telemetry.