Published

lib/transformations — JSONata sandboxed engine

Connect any source, model it as an ontology, transform it, and operationalize it, analytics, automation and machine learning, under one governed, self-hostable roof. --- Most teams stitch the...

lib/transformations — JSONata sandboxed engine

Fase 5 — Motor real de expresiones para computed mappings y submission.conditions[]. Reemplaza los stubs que antes retornaban el literal ƒ(${expr}).

Arquitectura en 30 segundos

 save (POST/PUT)                 runtime (wrapper executor)
 ───────────────                 ──────────────────────────
 validateActionTypeExpressions   buildInputContext
     │                              │
     ├─ compile(expr, scope)  ←─────┤
     │    · parser JSONata            │
     │    · AST analysis              │
     │    · inference (best-effort)   │
     │    · cache in-memory           │
     ↓                              ↓
  expression_cache            evaluate(compiled, context, opts)
  (persiste en action_types)      · runWithTimeout 100ms
                                   · assertInputSize 1MB
                                   · assertOutputSize 256KB
                                   · classify: success | error | timeout
                                expression_trace
                                (persiste en action_type_executions)

Los 7 archivos

FileResponsabilidad
types.tsContratos compartidos (ExpressionScope, CompileResult, EvaluateResult, SourceSchema).
sandbox.tsrunWithTimeout + assert(Input/Output)Size + ExpressionTimeoutError / ExpressionSizeError.
context.tsbuildInputContext / buildOutputContext / buildConditionContext — whitelist frozen.
type-inference.tsAST walker + tabla de 40+ built-ins JSONata tipados. inferType best-effort.
compiler.tscompile() — parse + validate + cache. Never throws. Cap MAX_COMPLEXITY=500.
engine.tsevaluate() — orquesta sandbox + JSONata. Classifica en success/error/timeout.
validate-config.tsvalidateActionTypeExpressions() — bulk validator usado en save.
monaco-jsonata.tsRegistración del lenguaje en Monaco + completion provider + temas node.

Barrel: @/lib/transformations exporta todo lo público.

Contratos never-throws

// compile — devuelve discriminated union
const result = compile({ expression, scope, sourceSchema });
if (result.status === 'broken') return 400;
// result._compiled listo para evaluate()

// evaluate — clasifica en 3 buckets
const outcome = await evaluate({ compiled, context, opts });
switch (outcome.status) {
  case 'success': use(outcome.value); break;
  case 'error':   log(outcome.error); break;
  case 'timeout': bail(); break;
}

Sandboxing — las 3 capas

CapaCapCuándo
Complexity500 AST nodosSave-time (compile)
Input size1MBRuntime (pre-eval)
Timeout100ms runtime / 1000ms previewRuntime (durante eval)
Output size256KBRuntime (post-eval)

JSONata es un DSL cerrado — no tiene eval, Function, require, process, global. No necesitamos vm2/isolated-vm. La única barrera real es el timeout (cooperativo via Promise.race).

Contexts whitelist

ScopeNamespaces
inputupstream, params, user{id,role,email}, workspace{id}, now
outputitem, index, response, user, workspace, now
conditionuser, workspace, params, upstream, now

Todo Object.freeze recursivo. JSONata no puede mutar nada ni acceder fuera del whitelist.

Type inference (best-effort)

  • Literales: number | string | boolean | null
  • Binarias: +-*/% → number; =/!=/<=/>=/and/or/in → boolean; & → string
  • Built-ins: tabla de 40+ (ver FUNCTION_TYPES en type-inference.ts)
  • Paths: contra SourceSchema.namespaces[ns][field] si se pasa; sino unknown

Mismatch con targetTypesoft warning, nunca hard fail. Runtime aplica coerción donde posible.

Integración end-to-end

ConsumerLlamadas
POST/PUT /api/actiontypesvalidateActionTypeExpressions() → persiste expression_cache
Wrapper executorcompile() + evaluate() per-row en applyInputMapping / applyOutputMapping
Guard (canExecuteActionType)compile() + evaluate() per-condition tras RPC OK
/expressions/validatecompile() headless
/expressions/previewcompile() + evaluate() contra sample
UI editor Monacocompile() debounced para live validation

Misma librería en save, runtime, UI, headless — zero drift entre qué valida el servidor y qué autocompleta el editor.

Caches

  • compiler.tsMap<(scope,expression), CompiledExpression>. Per-process, no LRU. Reset via invalidateCompilerCache(). En un run con N items que comparten la misma expresión: 1 parse + N evals. En serverless cold-start: 1 parse per request.

Observabilidad

  • Save-time events: expression.validated / expression.validation_failed (phases canónicas).
  • Runtime events: expression.evaluated / expression.failed (agregado summary por run).
  • Trace column: action_type_executions.expression_trace — per-eval detail.
  • Cache column: action_types.expression_cache — per-action snapshot.
  • Health block: /api/observability/healthexpressions (24h stats + top slowest + broken count).

Queries SQL útiles

-- Top action_types con más evals en 24h
SELECT action_type_id, SUM((expression_trace->>'evalCount')::int) AS evals
FROM action_type_executions
WHERE expression_trace IS NOT NULL
  AND created_at > NOW() - INTERVAL '24 hours'
GROUP BY 1 ORDER BY evals DESC LIMIT 10;

-- Runs con timeouts en la última hora
SELECT id, action_type_id,
       (expression_trace->>'timeoutCount')::int AS timeouts,
       (expression_trace->>'totalMs')::int     AS total_ms
FROM action_type_executions
WHERE (expression_trace->>'timeoutCount')::int > 0
  AND created_at > NOW() - INTERVAL '1 hour';

-- Action_types con expresiones inválidas
SELECT id, name, (expression_cache->>'brokenCount')::int AS broken
FROM action_types
WHERE (expression_cache->>'brokenCount')::int > 0;

Lo que NO hace Fase 5

  • Approval flows (requireApproval stub) → Fase 6.
  • Execution chains coordinados → Fase 6.
  • Runtime isolation con worker_threads — cooperative timeout es suficiente dado MAX_COMPLEXITY + input caps.
  • UI de conditions editorsubmission.conditions[] hoy se edita vía API bruta. La UI Monaco-powered para conditions queda como follow-up menor.
  • Streaming tier eval tracking — Tier 2 (streaming batches) hoy no registra en expression_trace (solo Tier 1 inline). Extensión trivial para post-5.