Ingester Deployment

Deploy the standalone ingester and queue worker. Provider callbacks should target the ingester service, not the Next.js app URL.

Raw markdown

Endpoints

POST /events/ses — SES/SNS sending lifecycle notifications for delivered, bounced, complained, opened, and clicked events.
POST /events/inbound — inbound MIME provider notifications. The payload must include event_id and either raw_mime, raw_mime_base64, or raw_mime_url.
POST /jobs/poll, /jobs/scheduled-emails, /jobs/webhooks, /jobs/domain-verify — internal job endpoints protected by a required production INGESTER_JOB_TOKEN.

Set INGESTER_INBOUND_TOKEN to require Authorization: Bearer <token> on inbound MIME notifications. Production ingesters reject /events/inbound requests when the token is missing.

For SES receipt-rule receiving, set SES_INBOUND_SNS_TOPIC_ARN and S3_BUCKET_NAME or SES_INBOUND_BUCKET_NAME on the app and ingester, subscribe that SNS topic to /events/inbound/ses-s3, and grant SES permission to write raw MIME objects to the bucket. Hosted-style receiving uses these values to create SES receipt rules when receiving is enabled for a domain.

SES delivery feedback

Delivery, bounce, complaint, and delivery-delay metrics require SES configuration sets to publish events to the ingester SNS topic. Set SES_EVENTS_SNS_TOPIC_ARN on both the app and ingester services, subscribe that topic to https://events.yourdomain.com/events/ses, then run:

bash

bun run deliverability:preflight -- --domain example.com --json --strict
bun run deliverability:preflight -- --repair --domain example.com --json --strict

The preflight/repair report lists the previous and resulting ses_configuration_set_name, database write-back status, and SES event-destination state. Start with one verified domain, prove a new validation send creates a user-scoped email_events row, then repair the verified-domain batch.

Inbound payload

json

{
  "provider": "ses-receiving",
  "event_id": "provider-event-id",
  "message_id": "provider-message-id",
  "recipients": ["support@inbound.example.com"],
  "raw_mime_base64": "...",
  "metadata": { "receipt_rule": "opensend-inbound" }
}

The ingester stores sanitized provider metadata in inbound_provider_events, parses MIME headers/body/attachments, resolves the recipient domain/route to one tenant, validates OpenSend reply tokens for conversation threading, uploads attachment bodies through OpenSend storage, inserts received_emails, writes an internal durable email_events row of type received, and then evaluates matching forwarding rules without deleting or hiding the stored message.

Terminal outcomes are recorded for malformed MIME, missing or ambiguous receiving domain, oversized messages, attachment storage failure, and duplicate provider events. Raw MIME bodies and secrets are not written to logs or provider metadata.

Reply-domain setup

For threaded replies, run the app and ingester with the same OPENSEND_REPLY_TOKEN_SECRET, enable receiving on the sending domain, and point that domain's inbound provider notifications at POST /events/inbound. Outbound sends from receiving-enabled domains get generated reply addresses and headers; inbound messages with invalid or cross-tenant tokens stay unmatched for the resolved recipient-domain tenant instead of being attached to another tenant's thread.