ml-runner

Node's classical-ML data plane. Runs the heavy lifting (snapshot materialization, model training, prediction, drift detection) on behalf of the Node workers..

The Node side never holds Supabase Storage credentials directly; it mints signed URLs and hands them to this service. Postgres reads are scoped to the surgical ml_runner_reader role created by migration 20260610_ml_snapshot_lifecycle.sql.

Status

PR4.2.1 — service skeleton only:

• GET /health (no auth)
• Bearer-token auth scaffolding for protected routes
• Configuration + JSON logging + optional Sentry

POST /materialize lands in PR4.2.5.

Local dev

# 1. From repo root, ensure ML buckets exist on your Supabase project.
npm run ml:ensure-buckets

# 2. Set env in the repo .env.local:
#    ML_RUNNER_TOKEN=<32+ chars; openssl rand -hex 32>
#    DATABASE_URL_READER=postgresql://ml_runner_reader:<pwd>@<host>:5432/postgres

# 3. Start the service in Docker (uses repo-level docker-compose.yml).
docker-compose up ml-runner

# 4. Smoke test.
curl -fsS http://localhost:8000/health
# {"status":"ok","version":"0.1.0","active_jobs":0,"max_concurrent":4}

Local dev without Docker

cd services/ml-runner
python -m venv .venv && source .venv/bin/activate    # or .venv\Scripts\activate on Windows
pip install -r requirements-dev.txt
ML_RUNNER_TOKEN=dev-token-1234567890-abcdef \
  uvicorn app.main:app --reload --port 8000

Tests

cd services/ml-runner
pip install -r requirements-dev.txt
pytest -v
ruff check app tests

Endpoints

GET /health (public)

Returns service status. Used by Docker/Railway healthchecks and the Node bridge for connectivity probes.

POST /materialize (PR4.2.5)

Auth: Authorization: Bearer <ML_RUNNER_TOKEN>. Streams SSE events while a Postgres dataset_rows cursor is dumped to a Parquet file and PUT to a Supabase Storage signed URL. Updates the row in ml_dataset_snapshots with its final lifecycle state.

Configuration

Every setting comes from environment variables (validated at boot via app/config.py). See services/ml-runner/.env.example for the full list once landed; the contract today: