Skip to content

Publish opted-in form submissions to Kafka through Bus API - #1

Merged
jmgasper merged 1 commit into
developfrom
kafka
Sep 27, 2026
Merged

jmgasper merged 1 commit into
developfrom
kafka

Conversation

@jmgasper

Copy link
Copy Markdown
Contributor

Form submissions can now opt in to downstream processing by including "kafka": true (or the exact string "true") alongside version and answers. After the submission commits, the service publishes its validated answers, form key/version, receipt identity, timestamp, member ID, and source page to form.submitted using the shared tc-bus-api-wrapper and Auth0 M2M configuration used by neighboring v6 services.

  • Omitted/false flags preserve existing behavior. Successful HTTP receipts are unchanged.
  • A separate SubmissionEvent delivery receipt preserves immutable submissions, serializes concurrent retries, and avoids republishing acknowledged deliveries. Failed delivery returns 503; retrying the same body/key resumes it, including after form retirement.
  • Includes OpenAPI, API/operations documentation, example environment settings, and pinned wrapper/core-library dependencies. The shared library requires Git subdependencies to be enabled in pnpm; both Git dependencies are pinned to commits.

Deployment requires migration 20260928010000_submission_events, BUSAPI_URL pointing to the API /v6 base, and outbound AUTH0_URL, AUTH0_AUDIENCE, AUTH0_CLIENT_ID, and AUTH0_CLIENT_SECRET. ECS must expose these settings to the runtime task. There is no background retry worker; callers retry failed requests. Consumers must deduplicate by submissionId because a lost acknowledgement or crash between Bus API acceptance and receipt commit can cause redelivery.

Validation: frozen-lockfile install, Prisma validation and migration against disposable PostgreSQL 17, lint, build, typecheck, 36 unit tests, and 20 PostgreSQL integration tests all passed. Tests cover event contents, true/string-true and false/omitted flags, rejection paths, commit-before-publication, concurrent retries, flag conflicts, and delivery failure/retry after retirement. The compiled service also initialized the real wrapper successfully; outbound event calls in automated tests are mocked, so no shared Kafka messages were emitted.

@jmgasper
jmgasper merged commit 99cd6e4 into develop Sep 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant