Unified Event Schema Documentation

Overview

This document describes the unified event schemas used across the webhook processing system. All webhook events follow a consistent structure to enable reliable processing, observability, and debugging.

Base Event Structure

All webhook events share a common base structure:

interface BaseWebhookEvent {
  // Event Identification
  id: string;                    // Unique event identifier
  type: string;                  // Event type (e.g., "orders/create")
  source: 'shopify' | 'slack';   // Event source system
  
  // Timing
  timestamp: number;             // Unix timestamp (ms) when event was received
  processedAt?: number;          // Unix timestamp (ms) when event was processed
  
  // Source Metadata
  shop?: string;                 // Shopify shop domain (Shopify events only)
  workspaceId?: string;          // Slack workspace ID (Slack events only)
  
  // Payload
  payload: Record<string, any>;  // Original webhook payload
  
  // Processing Metadata
  attemptNumber?: number;        // Current retry attempt (1-indexed)
  idempotencyKey?: string;       // Unique key for deduplication
}

Shopify Events

Schema

interface ShopifyWebhookEvent extends BaseWebhookEvent {
  source: 'shopify';
  type: string;                  // Shopify topic (e.g., "orders/create", "products/update")
  shop: string;                  // Shop domain (e.g., "example.myshopify.com")
  payload: {
    id: number;                  // Resource ID
    [key: string]: any;          // Resource-specific fields
  };
  apiVersion?: string;           // Shopify API version
  webhookId?: string;            // Shopify webhook ID
}

Supported Event Types

Event Type	Description	Payload Shape
`orders/create`	New order created	`{ id, customer, line_items, total_price, ... }`
`orders/updated`	Order modified	`{ id, customer, line_items, total_price, ... }`
`orders/cancelled`	Order cancelled	`{ id, cancel_reason, cancelled_at, ... }`
`orders/fulfilled`	Order fulfilled	`{ id, fulfillment_status, ... }`
`products/create`	New product added	`{ id, title, variants, images, ... }`
`products/update`	Product modified	`{ id, title, variants, images, ... }`
`products/delete`	Product deleted	`{ id }`
`customers/create`	New customer registered	`{ id, email, first_name, last_name, ... }`
`customers/update`	Customer updated	`{ id, email, first_name, last_name, ... }`
`customers/data_request`	GDPR data request	`{ customer: { id, email }, orders_requested, ... }`
`customers/redact`	GDPR data deletion	`{ customer: { id, email }, orders_to_redact, ... }`
`shop/redact`	Shop deletion (48hr notice)	`{ shop_id, shop_domain, ... }`

Example Shopify Event

{
  "id": "evt_shopify_1234567890",
  "type": "orders/create",
  "source": "shopify",
  "timestamp": 1698412800000,
  "shop": "example.myshopify.com",
  "payload": {
    "id": 5678901234,
    "email": "customer@example.com",
    "created_at": "2024-10-27T12:00:00Z",
    "total_price": "99.99",
    "currency": "USD",
    "line_items": [
      {
        "id": 123456,
        "title": "Product Name",
        "quantity": 2,
        "price": "49.99"
      }
    ],
    "customer": {
      "id": 987654321,
      "email": "customer@example.com",
      "first_name": "John",
      "last_name": "Doe"
    }
  },
  "apiVersion": "2024-10",
  "idempotencyKey": "example.myshopify.com-orders/create-5678901234"
}

Slack Events

Schema

interface SlackWebhookEvent extends BaseWebhookEvent {
  source: 'slack';
  type: string;                  // Slack event type (e.g., "message", "app_mention")
  workspaceId: string;           // Slack team/workspace ID
  payload: {
    type: string;                // Event type (matches outer type)
    event: Record<string, any>;  // Event-specific data
    event_id: string;            // Unique event ID from Slack
    event_time: number;          // Unix timestamp from Slack
  };
  channelId?: string;            // Channel where event occurred
  userId?: string;               // User who triggered event
}

Supported Event Types

Event Type	Description	Payload Shape
`message`	Message posted	`{ type, channel, user, text, ts, ... }`
`app_mention`	Bot mentioned	`{ type, channel, user, text, ts, ... }`
`reaction_added`	Reaction added to message	`{ type, user, reaction, item: { channel, ts }, ... }`
`reaction_removed`	Reaction removed	`{ type, user, reaction, item: { channel, ts }, ... }`
`channel_created`	New channel created	`{ type, channel: { id, name, created }, ... }`
`channel_deleted`	Channel deleted	`{ type, channel }`
`channel_archive`	Channel archived	`{ type, channel, user }`
`channel_unarchive`	Channel unarchived	`{ type, channel, user }`
`member_joined_channel`	User joined channel	`{ type, user, channel, team, ... }`
`member_left_channel`	User left channel	`{ type, user, channel, team, ... }`
`team_join`	New user joined workspace	`{ type, user: { id, name, real_name, ... } }`

Example Slack Event

{
  "id": "evt_slack_1234567890",
  "type": "message",
  "source": "slack",
  "timestamp": 1698412800000,
  "workspaceId": "T01234567",
  "channelId": "C01234567",
  "userId": "U01234567",
  "payload": {
    "token": "...",
    "team_id": "T01234567",
    "api_app_id": "A01234567",
    "event": {
      "type": "message",
      "channel": "C01234567",
      "user": "U01234567",
      "text": "Hello, world!",
      "ts": "1698412800.123456",
      "event_ts": "1698412800.123456",
      "channel_type": "channel"
    },
    "type": "event_callback",
    "event_id": "Ev01234567",
    "event_time": 1698412800
  },
  "idempotencyKey": "T01234567-Ev01234567"
}

Queue Job Structure

When events are enqueued in BullMQ, they follow this structure:

interface QueueJob {
  id: string;                    // Job ID (usually event.id)
  name: string;                  // Job name (usually event.type)
  data: BaseWebhookEvent;        // The event data
  opts: {
    attempts: number;            // Max retry attempts
    backoff: {
      type: 'exponential';
      delay: number;             // Base delay in ms
    };
    removeOnComplete: {
      age: number;               // Keep completed jobs for N seconds
      count: number;             // Keep max N completed jobs
    };
    removeOnFail: {
      age: number;               // Keep failed jobs for N seconds
      count: number;             // Keep max N failed jobs
    };
  };
}

Idempotency Keys

Idempotency keys prevent duplicate processing of the same event:

Shopify

Format: {shop}-{topic}-{resourceId}
Example: example.myshopify.com-orders/create-5678901234
TTL: 24 hours

Slack

Format: {teamId}-{eventId}
Example: T01234567-Ev01234567
TTL: 24 hours

Error Handling

When events fail processing, they are enriched with error metadata:

interface FailedEvent extends BaseWebhookEvent {
  error: {
    message: string;             // Error message
    stack?: string;              // Stack trace
    code?: string;               // Error code
    timestamp: number;           // When error occurred
    attemptNumber: number;       // Which attempt failed
  };
  failedAt: number;              // Final failure timestamp
  retriedCount: number;          // Total retry attempts made
}

Data Retention

State	Retention	Max Count
Completed	1 hour	100 jobs
Failed	24 hours	1,000 jobs
Active	Until processed	N/A
Waiting	Until processed	N/A

Validation

All events are validated before processing:

Required Fields

id: Must be non-empty string
type: Must be non-empty string
source: Must be 'shopify' or 'slack'
timestamp: Must be valid Unix timestamp
payload: Must be non-null object

Source-Specific Requirements

Shopify:

shop: Required, must match *.myshopify.com
payload.id: Required for resource events

Slack:

workspaceId: Required, must start with 'T'
payload.event_id: Required, must start with 'Ev'

Event Lifecycle

Usage Examples

TypeScript Type Guards

import { BaseWebhookEvent } from '@/types/webhooks';

function isShopifyEvent(event: BaseWebhookEvent): event is ShopifyWebhookEvent {
  return event.source === 'shopify';
}

function isSlackEvent(event: BaseWebhookEvent): event is SlackWebhookEvent {
  return event.source === 'slack';
}

// Usage
const event: BaseWebhookEvent = getEvent();
if (isShopifyEvent(event)) {
  console.log(event.shop); // TypeScript knows this exists
}

Creating Events

import { v4 as uuidv4 } from 'uuid';

function createShopifyEvent(topic: string, shop: string, payload: any): ShopifyWebhookEvent {
  return {
    id: `evt_shopify_${uuidv4()}`,
    type: topic,
    source: 'shopify',
    timestamp: Date.now(),
    shop,
    payload,
    idempotencyKey: `${shop}-${topic}-${payload.id}`,
  };
}

Validating Events

function validateEvent(event: BaseWebhookEvent): boolean {
  if (!event.id || !event.type || !event.source || !event.timestamp) {
    return false;
  }

  if (event.source === 'shopify' && !event.shop) {
    return false;
  }

  if (event.source === 'slack' && !event.workspaceId) {
    return false;
  }

  return true;
}

Migration Guide

If you have existing webhook handlers, migrate to the unified schema:

Before

// Old handler
async function handleWebhook(body: any) {
  console.log('Processing webhook:', body);
  // Process directly
}

After

// New handler
import { BaseWebhookEvent } from '@/types/webhooks';
import { getShopifyWebhooksQueue } from '@/lib/queues/definitions';

async function handleWebhook(body: any, headers: any) {
  const event: ShopifyWebhookEvent = {
    id: `evt_shopify_${Date.now()}`,
    type: headers['x-shopify-topic'],
    source: 'shopify',
    timestamp: Date.now(),
    shop: headers['x-shopify-shop-domain'],
    payload: body,
    idempotencyKey: `${headers['x-shopify-shop-domain']}-${headers['x-shopify-topic']}-${body.id}`,
  };

  const queue = getShopifyWebhooksQueue();
  await queue.add(event.type, event, {
    jobId: event.id,
    attempts: 3,
  });
}

Reference Implementation

See the following files for complete examples:

lib/workers/shopify-webhook-worker.ts - Shopify event processing
lib/workers/slack-webhook-worker.ts - Slack event processing
app/api/shopify/webhooks/route.ts - Shopify webhook ingestion
app/api/integrations/slack/webhooks/route.ts - Slack webhook ingestion
lib/queues/idempotency.ts - Deduplication logic

Changelog

Version	Date	Changes
1.0.0	2024-10-27	Initial unified schema documentation