Branches & Pull Requests — Operational Runbook

Architecture Overview

User UI (checkout branch)
    │
    ▼
notebooks.current_branch_id  ◄── updated by checkout API
    │
    ▼
JupyterHub pre_spawn_hook
    │  ┌── 1. Query Supabase for current_branch_id
    │  ├── 2. POST /api/sdk/internal/mint-kernel-token {branchId: ...}
    │  └── 3. Inject NODE_BRANCH_ID + STORELY_BRANCH_ID env vars
    │
    ▼
Kernel JWT carries `br` claim
    │
    ▼
SDK API handlers read `auth.claims.br`
    │
    ├── br=default  ──▶  dataset_rows (canonical)
    └── br=feature   ──▶  dataset_branch_rows (overlay)

Common Operations

Force-remint a kernel token (without Hub restart)

If a user changes branches but doesn't restart their kernel, the JWT still carries the old branch. The fix is to restart the named server:

# Via JupyterHub admin API
curl -X DELETE \
  "https://<hub>/hub/api/users/<username>/servers/<server-name>" \
  -H "Authorization: token <JUPYTERHUB_API_TOKEN>"
# The next spawn will re-run pre_spawn_hook and mint a fresh JWT.

Check a kernel's active branch

# From the notebook itself (SDK)
from node.transforms import Dataset
ds = Dataset.get("any-dataset")
print(ds.current_branch)  # reads from /whoami response

# Or directly check the JWT's br claim
import jwt, os
claims = jwt.decode(os.environ["NODE_API_TOKEN"], options={"verify_signature": False})
print(claims.get("br", "(default branch)"))

Troubleshooting

Problem: SDK writes go to the wrong branch

Symptoms: Data appears in dataset_rows instead of dataset_branch_rows (or vice versa).

Root cause: Stale JWT. The kernel was spawned before the user checked out the branch, so the JWT's br claim is missing or points to the old branch.

Fix:

1. Restart the named server (forces re-spawn → fresh JWT).
2. Verify: GET /api/sdk/v1/whoami → check branchId field.

Problem: 409 BRANCH_NOT_ACTIVE on SDK read/write

Symptoms: SDK calls fail with BRANCH_NOT_ACTIVE.

Root cause: The branch was archived or merged while the kernel was still running.

Fix:

1. User should checkout a different (active) branch in the UI.
2. Restart the named server.
3. If the branch was archived accidentally:

   UPDATE workspace_branches
   SET state = 'active'
   WHERE id = '<branch-uuid>';
   

Problem: Orphan overlay rows after PR merge

Symptoms: dataset_branch_rows still has rows for a merged branch after the PR merge completed.

Root cause: The merge_pull_request RPC copies overlay rows to dataset_rows but does NOT delete the overlay originals immediately (the reaper handles cleanup).

Reaper behavior:

• Migration 20260657_branch_reapers.sql defines the reaper.
• It sweeps dataset_branch_rows for branches in terminal states (merged, archived) older than a configurable window (default 7d).
• The reaper is idempotent — safe to re-run.

Manual cleanup (if reaper hasn't run):

DELETE FROM dataset_branch_rows
WHERE branch_id IN (
  SELECT id FROM workspace_branches
  WHERE state IN ('merged', 'archived')
    AND updated_at < now() - interval '7 days'
);

Problem: Hung PR (state=open but branch archived)

Symptoms: PR visible in the UI with state open, but the source branch has state archived.

Diagnosis:

SELECT pr.id, pr.title, pr.state AS pr_state,
       b.name AS branch_name, b.state AS branch_state
FROM pull_requests pr
JOIN workspace_branches b ON b.id = pr.head_branch_id
WHERE pr.state = 'open'
  AND b.state != 'active';

Fix: Close the PR manually:

UPDATE pull_requests
SET state = 'closed', closed_at = now()
WHERE id = '<pr-uuid>';

Diagnostic SQL Queries

List all active branches with overlay row counts

SELECT
  b.id, b.name, b.state, b.is_default,
  COUNT(dbr.id) AS overlay_rows
FROM workspace_branches b
LEFT JOIN dataset_branch_rows dbr ON dbr.branch_id = b.id
WHERE b.state = 'active'
GROUP BY b.id
ORDER BY overlay_rows DESC;

Find transactions with branch mismatches

SELECT t.id, t.branch_id, t.workspace_id,
       b.workspace_id AS branch_workspace_id,
       t.branch_id = b.id AS branch_match
FROM sdk_dataset_transactions t
LEFT JOIN workspace_branches b ON b.id = t.branch_id
WHERE t.branch_id IS NOT NULL
  AND b.workspace_id != t.workspace_id;

Check kernel token freshness

-- Notebooks whose current_branch_id differs from what the last
-- spawned kernel would have received (indicates stale JWT)
SELECT n.id AS notebook_id,
       n.current_branch_id,
       n.last_opened_at
FROM notebooks n
WHERE n.current_branch_id IS NOT NULL
  AND n.last_opened_at < n.updated_at;

Environment Variables Reference

Deployment Checklist

When deploying branch-aware changes:

1. Vercel (Next.js): Deploy handler changes first. They are backward-compatible — legacy JWTs without br still work.
2. JupyterHub (Railway): Deploy the updated jupyterhub_config.py. New kernels will start receiving branch-enriched JWTs.
3. Existing kernels: NOT affected until restarted. They continue operating in branchless (default-branch) mode.
4. Signing key rotation: If NODE_SDK_SIGNING_KEY changes, ALL active kernels must be restarted (tokens become invalid).