Your events, pushed and signed.

Section 01 · the problem

Polling is a tax on everyone who integrates with you.

Every workspace that wants to react to Omazy in their own systems, a CRM row on a new lead, a Slack ping when an agent publishes, a spreadsheet of closed sessions, faces the same choice: poll our API on a timer, or wait for us to push.

Polling is the worse deal on both sides. The integrator writes cursor bookkeeping and eats latency equal to their poll interval. We serve a constant baseline of requests that mostly return nothing. And the interesting events, the ones worth reacting to, are rare enough that a one-minute poll is 99 percent waste and still a minute late.

So webhooks. The design constraint that shaped everything else: a pushed event has to be provably ours. Anyone on the internet can POST JSON at your endpoint claiming a session just closed. Without a signature, a webhook receiver is a public, unauthenticated write path into your business logic. That is why the signature is not a feature of this system; it is the system.

Section 02 · architecture

One event bus. Two consumers. No second vocabulary.

Workspace automations already listened to a stream of domain events. Rather than build a parallel emitter for webhooks, the webhooks service registers as a second sink on the same publisher. An event is emitted once and reaches both an in-platform automation and an out-of-platform endpoint.

flowchart LR
  subgraph CORE["domain"]
    EV["session.opened
agent.published
kb.updated · …"]
  end
  EV --> PUB["automations.Publisher"]
  PUB --> AUT["workspace automations
(in-platform reaction)"]
  PUB -->|"EventSink"| WH["webhooks.DispatchEvent
(out-of-platform push)"]
  WH --> Q{"active subs
for this event?"}
  Q -->|"none"| STOP["return, no work"]
  Q -->|"n"| GO["n goroutines · one POST each"]
  GO --> EP["your HTTPS endpoint"]
  GO -.-> LOG[("webhook_deliveries")]

Section 03 · the event registry

Where the event names come from.

There is no webhook-specific event vocabulary. The platform's domain-event registry is a single closed set of constants in the automations package, and the webhook engine subscribes against that same set. What a workspace can trigger an automation on is exactly what it can receive a webhook for.

flowchart LR
  REG["internal/automations/registry.go
EventKind · 6 domain events"]
  REG -->|"producers call Publish"| PUB["automations.Publisher"]
  PUB --> A["automations engine"]
  PUB --> WH["webhooks.DispatchEvent"]
  REG -.->|"redeclared, not imported"| WREG["internal/webhooks/registry.go
same 6 + webhook.test"]
  WREG -->|"validates subscriptions"| WH
  subgraph OTHER["separate vocabularies · deliberately not on this bus"]
    TEL["webingest · browser telemetry"]
    RT["realtime · WebSocket room events"]
  end

Status, plainly stated: the registry, the publisher, the sink, and the delivery engine are all shipped and wired. The producer call sites are the remaining step: the domains that own these facts do not yet call Publisher.Publish in their hot paths, which the automations package documents as a follow-up. Until they do, the ping is the only thing that reaches a registered endpoint, and any automation on the same six events is idle for the same reason. Because both consumers hang off one publisher, wiring a producer once lights up both at the same time.

Section 04 · the envelope

Four fields, the same shape every time.

Every delivery is a flat JSON object with four keys: the event name, the workspace it belongs to, a per-event data payload, and the send timestamp. The test ping uses the identical shape, so your parser exercises the same code path for a ping as for a real event.

POST https://your-endpoint.example/hooks
Content-Type: application/json
User-Agent: Omazy-Webhooks/1.0
X-Omazy-Signature: sha256=52c9141de7b8cd0e30f0604ddbee1df47e54ef7f27f9fdac377086691fa0d4c4

{"event":"webhook.test","workspace_id":"0e4b758f-39a0-4923-bc5e-2b6524add7a1","data":{"message":"This is a test ping from Omazy."},"sent_at":"2026-07-31T01:45:35.404112686Z"}

sent_at carries nanosecond precision because Go writes RFC 3339 that way. Treat it as an opaque string when verifying. Any parse-and-reformat round trip will change the bytes and break the signature.

The subscribable catalog

There is a seventh kind, webhook.test, that is intentionally not subscribable. It is emitted only by the explicit ping button, and the ping fires regardless of which events a webhook is subscribed to, so you can validate a brand-new endpoint before wiring it to anything real. A ping is also honoured on an inactive webhook, because "does my endpoint work" is a question you ask before you turn it on.

Section 05 · signing

HMAC over the exact bytes on the wire.

The envelope is marshalled once. That byte slice is what gets signed and what gets sent, in that order, with nothing in between. The signature travels in X-Omazy-Signature as the algorithm name, an equals sign, and lowercase hex.

// internal/webhooks/service.go
// sign returns "sha256=<hex>" where hex is HMAC-SHA256(secret, body).
func sign(secret string, body []byte) string {
    mac := hmac.New(sha256.New, []byte(secret))
    mac.Write(body)
    return "sha256=" + hex.EncodeToString(mac.Sum(nil))
}

// ...and at delivery time, over the SAME bytes that go on the wire:
req.Header.Set("Content-Type", "application/json")
req.Header.Set("User-Agent", "Omazy-Webhooks/1.0")
req.Header.Set("X-Omazy-Signature", sign(w.Secret, body))

flowchart TB
  A["envelope marshalled once
body := json.Marshal(payload)"] --> B["sig = HMAC-SHA256(secret, body)"]
  B --> C["X-Omazy-Signature: sha256=hex"]
  C --> D["POST · same body bytes"]
  D --> E["your server reads the RAW body"]
  E --> F["recompute HMAC with your copy of the secret"]
  F --> G{"constant-time equal?"}
  G -->|"no"| H["401 · drop it"]
  G -->|"yes"| I["trust the payload, do the work"]

Section 06 · verifying, for real

Fifteen lines, three rules.

Recompute the HMAC with your copy of the secret, over the raw request body, and compare the whole header value in constant time. Everything that goes wrong is a violation of one of those three.

Pipedream, copy-paste

Paste this as a code step. Pipedream renders a masked input on the step where the secret goes, which avoids the most common first failure (see section 06).

import crypto from "crypto";

export default defineComponent({
  props: {
    secret: {
      type: "string",
      label: "Omazy webhook signing secret",
      description: "The full whsec_... value, prefix included",
      secret: true,
    },
  },
  async run({ steps }) {
    const secret = this.secret;
    if (!secret) throw new Error("signing secret is not set on this step");

    const event = steps.trigger.event;

    // Prefer the raw body. JSON.stringify of the parsed object is a fallback,
    // not a guarantee: it only matches while key order and separators survive.
    const raw =
      typeof event.bodyRaw === "string" && event.bodyRaw.length
        ? event.bodyRaw
        : JSON.stringify(event.body);

    const expected =
      "sha256=" +
      crypto.createHmac("sha256", secret).update(raw, "utf8").digest("hex");

    const got = event.headers["x-omazy-signature"] ?? "";

    const ok =
      expected.length === got.length &&
      crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(got));

    if (!ok) {
      throw new Error(
        `bad signature
  expected:    ${expected}
  received:    ${got}
  usedRawBody: ${event.bodyRaw != null}
  bodyLen:     ${Buffer.byteLength(raw)} (compare to content-length)`
      );
    }

    return { verified: true, event: event.body.event };
  },
});

Note the error message. It prints both signatures plus the body length, so the moment it does fail you can compare bodyLen against the content-length header and immediately tell whether the key is wrong or the body reconstruction is.

Go and Python

func verify(secret string, body []byte, header string) bool {
    mac := hmac.New(sha256.New, []byte(secret))
    mac.Write(body)
    want := "sha256=" + hex.EncodeToString(mac.Sum(nil))
    return hmac.Equal([]byte(want), []byte(header))
}

import hmac, hashlib

def verify(secret: str, body: bytes, header: str) -> bool:
    want = "sha256=" + hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(want, header)

# Flask: use request.get_data(), NOT request.json
# FastAPI: use await request.body(), NOT the parsed model

In every framework, the trap is the same: the convenient accessor gives you the parsed body. Reach for the raw one. In Express that means express.raw() or capturing req.rawBody in the JSON parser's verify hook, and in Next.js route handlers it means await req.text() before any json() call, because the body stream can only be read once.

A test vector you can run offline

Before pointing anything at a live endpoint, check your implementation against a fixed triple. The secret here is a throwaway, not a real one.

secret     whsec_4a1f0c9b7e2d5a8c3f6b9d0e1a2c4b6d8e0f1a3c5b7d9e0f2a4c6b8d0e1f3a5c

body       {"event":"webhook.test","workspace_id":"0e4b758f-39a0-4923-bc5e-2b6524add7a1",
            "data":{"message":"This is a test ping from Omazy."},
            "sent_at":"2026-07-31T01:45:35.404112686Z"}
           (174 bytes, no whitespace between tokens)

signature  sha256=4c49aedadf7de2875d9e448a818bf51fb39c59a632a8b3ff15c9e01c574621a3

If your code returns 355cf1ee… you stripped the whsec_ prefix.
If it returns df763d6b… you hex-decoded the secret instead of using the string.

Section 07 · two failures worth naming

Both were on the receiving side. Both looked like our bug.

Every integration we have watched hit one of these two. Neither is exotic, and neither produces an error message that points at the actual cause.

The transferable lesson: a signature mismatch is not one failure, it is four, and they are indistinguishable from the outside. Wrong key, wrong bytes, wrong comparison, missing config. A verifier that just returns false is a verifier you will debug by guessing. Print the expected value, the received value, and the body length on failure, and all four become one glance.

Section 08 · retries, rate limiting, and giving up

Your server sets the pace, and we stop when you stop answering.

A delivery engine that ignores backpressure is a denial-of-service tool pointed at your own customers. Three rules keep it polite: retry only what is worth retrying, wait as long as the receiver asks, and stop entirely when an endpoint has been dead for a while.

flowchart TB
  P["POST attempt n"] --> C{"outcome"}
  C -->|"2xx"| OK["success · streak reset to 0"]
  C -->|"other 4xx"| PERM["permanent · refused, not retried"]
  C -->|"408 · 429 · 5xx · transport error"| R{"attempt < 3?"}
  R -->|"no"| EX["exhausted"]
  R -->|"yes"| W{"Retry-After header?"}
  W -->|"present, ≤ 30s"| WA["wait exactly that long"]
  W -->|"present, > 30s"| EX
  W -->|"absent"| WB["wait 2s, then 4s"]
  WA --> P
  WB --> P
  PERM --> EX
  EX --> S["consecutive_failures + 1"]
  S --> D{"streak ≥ 20?"}
  D -->|"no"| KEEP["stays active"]
  D -->|"yes"| OFF["is_active = false
disabled_at · disabled_reason"]

What counts as worth retrying

Resending identical bytes can only help if the receiver's answer might change. A 500 might; a 401 will not. So the classification is small and deliberate, and the two codes a rate limiter actually speaks are both on the retryable side.

// internal/webhooks/service.go
// 429 and 503 are the two codes a rate limiter actually speaks, and both are
// retryable by definition: they mean "later", not "no". Every other 4xx is a
// refusal of these exact bytes, and a 401 will not start accepting an
// unchanged signature however many times we resend it.
func classify(code int) outcome {
    switch {
    case code == http.StatusRequestTimeout,  // 408
         code == http.StatusTooManyRequests: // 429
        return outcomeRetryable
    case code >= 500:
        return outcomeRetryable
    default:
        return outcomePermanent
    }
}

Which rate-limit headers we respect

Exactly one: the standard Retry-After, in both forms RFC 9110 permits. When it is present and within the cap, it replaces our backoff entirely. That ordering is the whole point of reading it: a rate-limited endpoint knows when its window resets, and we are guessing.

// Both RFC 9110 forms are accepted: delta-seconds and an HTTP-date.
Retry-After: 120
Retry-After: Wed, 21 Oct 2026 07:28:00 GMT

// A past date means "you may retry now" and clamps to zero.
// A negative or unparseable value is ignored, and the exponential
// fallback (2s, then 4s) applies instead.
//
// X-RateLimit-Remaining / X-RateLimit-Reset are NOT read. They are a
// convention, not a standard: Reset is a unix timestamp for some vendors,
// a delta in seconds for others, and milliseconds for a few. Guessing
// wrong turns a one-second pause into a one-day one.

When we stop sending altogether

Twenty consecutive exhausted deliveries auto-disable the endpoint. The next event is never dispatched to it, because the dispatcher's lookup filters on is_active, so a dead URL stops costing anyone anything. The threshold is deliberately large: a webhook is somebody's integration, and killing it over a brief outage is worse than dropping a handful of events. At three attempts each, twenty is sixty failed POSTs.

The rest of the contract

The one remaining "not yet" is deliberate. Replay protection needs the timestamp inside the signature rather than beside it, which changes what receivers hash and therefore breaks every verifier written against today's contract. That is a versioned envelope change, not a patch, and shipping it badly is worse than not having it.