Published

Dashboards Reliability Smoke Test — Fase 1

Víctor Obregón
Víctor Obregón
0

Connect any source, model it as an ontology, transform it, and operationalize it, analytics, automation and machine learning, under one governed, self-hostable roof. --- Most teams stitch the...

Dashboards Reliability Smoke Test — Fase 1

Manual verification that the Fase 1 reliability floor (1.A–1.E) holds end to end. Run in a dev environment with a dashboard that has at least one widget bound to a working dataset. Keep DevTools open (Network, Console) and, if available, the Prometheus/Sentry views.

All six cases must pass before shipping Fase 1.

1 — Widget crash is isolated (WidgetErrorBoundary)

Setup: open a dashboard with 3+ widgets. Temporarily add throw new Error('boom') at the top of WidgetChart (behind if (widget.id === '<one-id>')) to force one widget to crash. Reload.

Expected:

  • The crashing widget shows the red-accented fallback: icon, "Widget error", Retry and Copy ID buttons, monospace Error ID underneath.
  • The widget header (title + Remove button) stays visible so the user can still delete the broken widget.
  • Other widgets in the canvas keep rendering normally.
  • Clicking Copy ID shows "Copied" for ~1.8s and puts the UUID in the clipboard.
  • Clicking Retry re-mounts the children; if you undo the throw, the widget recovers without a full reload.
  • Prometheus: dashboard_client_errors_total{scope="widget",widget_type="<type>",source_kind="<kind>"} increments.
  • Sentry: a DashboardError[widget] event exists with the same Error ID as the copied UUID.

2 — Backend 500 triggers one retry, then surfaces the error

Setup: in DevTools Network, override /api/dashboards/query with a 500 response (right-click → "Block request URL" doesn't work here; use a proxy or temporarily return new Response(null, { status: 500 }) in the route handler).

Expected:

  • The widget shows the loading skeleton, then the "Query error" placeholder with the server message as hint.
  • Network tab: two POSTs to /api/dashboards/query within ~600ms of each other (the second is the automatic retry with jittered backoff).
  • Prometheus: dashboard_query_errors_total{error_type="server"} increments.
  • Removing the 500 override and clicking around to re-trigger the fetch recovers normally (no stuck state).

3 — 15 s timeout fires cleanly

Setup: in a proxy / MSW mock, make /api/dashboards/query hang indefinitely.

Expected:

  • Widget shows loading skeleton for 15 s.
  • After 15 s, it flips to the error placeholder with message "Query timed out after 15s" — no infinite spinner.
  • Network tab: request is marked as cancelled/aborted at ~15 s; no third or fourth request (we do not retry timeouts).
  • Prometheus: dashboard_query_errors_total{error_type="timeout"} increments.
  • Switching to a different dashboard and back shows normal behaviour (no leaked pending request, no memory leak in DevTools Memory).

4 — Save race condition is prevented

Setup: in the Builder, make a small edit (rename a widget). The Save button lights up in unsaved state. Open Network tab.

Expected (double-click):

  • Double-click Save rapidly → exactly one PUT to /api/projects fires. The second click is swallowed by the saveStatus === 'saving' lock.
  • While saving, the Save button shows the spinner and is disabled.
  • On success: button transitions to green "Saved" for 2 s, then back to neutral "Saved".

Expected (Ctrl+S spam):

  • Hold Ctrl+S for a second (fires the keydown ~20× depending on repeat rate) → only one PUT fires.

Expected (create modal doubled-click):

  • In the "New dashboard" modal, spam-click Save → exactly one POST to /api/projects.
  • Retrying after a failure reuses the same Idempotency-Key header (check the request headers — both attempts carry the same UUID).

5 — Network failure surfaces human message, doesn't lose work

Setup: in the Builder, make an edit. Before clicking Save, disable the network (DevTools → Network tab → "Offline").

Expected:

  • Click Save → banner appears next to the Save button with "Network issue — check your connection and try again."
  • Save button flips to red "Retry" state.
  • The unsaved changes are still in the local state (widget positions, names, bindings intact).
  • Re-enable network → click Retry → save succeeds → banner dismisses.
  • Prometheus: dashboard_save_errors_total{failure_mode="network"} incremented.

Same flow in the create modal:

  • The modal stays open on failure with the red banner.
  • Input values are preserved.
  • Clicking "Retry" (button text changes from Save to Retry after an error) re-attempts with the same idempotency key.

6 — Error ID round-trip: UI → Sentry / logs

Setup: force a crash in any widget (case 1) OR trigger a malformed response from the backend (case 2 but with {"success": true, "rows": "not-an-array"}).

Expected:

  • Error ID displayed in the fallback UI is a valid UUID.
  • Copy ID puts it in the clipboard.
  • Server logs (pino, search for component: 'api-dashboards-telemetry') contain a record with matching errorId, including widgetType, sourceKind, dashboardId, scope.
  • Sentry: event with matching errorId tag, grouped under DashboardError[widget] or DashboardError[builder].
  • The source_kind tag on the Sentry event matches the widget's actual data path (ontology_backed if the widget is bound to an object_type-backed dataset, raw_dataset otherwise).

Coverage note — the ontology differentiator

Every metric emitted across these cases carries source_kind. Before shipping, eyeball one canonical dashboard with both an object_type widget and a dataset widget, trigger a success on each, and confirm Prometheus has two distinct rows:

dashboard_widget_renders_total{status="success", source_kind="ontology_backed", widget_type="…"} 1
dashboard_widget_renders_total{status="success", source_kind="raw_dataset",    widget_type="…"} 1

That split is the foundation for measuring adoption of the Node differentiator in Fase 5 onwards. If the split breaks, stop and fix before moving on.