Repository navigation
feat(im): Feishu and DingTalk chat Bots, proactive notifications, web-confirmed writes and data retention - #1114
Open
newexploreor wants to merge 34 commits into
Open
newexploreor wants to merge 34 commits into
newexploreor wants to merge 34 commits into
Conversation
added 30 commits
November 18, 2026 03:28
Single-branch plan for Feishu and DingTalk on top of the WeCom Bot stack. Version 3 rebases on origin/main@e89cda9 (migrations now 036/037) and fixes the implementation contracts: database-time leases with a dialect helper, physical enabled/version columns, lock order config -> connection -> interaction/delivery, retry terminal states, disable semantics, separate interaction and notification status enums (send_unknown), shared tenant confirmation for Feishu and DingTalk, and the maintenance window order.
Introduce the Channel/Conn/PushSender contract and route the existing WeCom long connection through it without changing user-visible behavior. - channel.go: inbound event, reply route, outbound message and SendResult types; optional progress and card capabilities. - channel_wecom.go: WeCom adapter wrapping dial/stream/cards; send errors are classified by evidence (not written / written without receipt / platform rejection). - inbound.go: common validation and enqueue shared by all adapters. - socket.request records the send phase while keeping error messages and context unwrapping. - notification.New injects CLIProvider.Verify (signed bootstrap only) and a classified wecom-cli push into the WeCom channel. - Fake channel test covers receive, duplicate redelivery, app mismatch, rejection, unknown outcome with reconnect, and reconnect after drop.
Separate module exercising larksuite/oapi-sdk-go v3.13.0 and dingtalk-stream-sdk-go v0.9.1 under Go 1.26.11. Feishu Start returns on context cancel; the DingTalk StreamClient panics with 'send on closed channel' after a pong timeout followed by a late frame, which is why the DingTalk adapter will run its own stream loop.
TestConnectionReconnectsAfterLocalClose incremented a plain int from httptest handler goroutines, which go test -race reported on main as well. Use atomic.Int64; the every-second-attempt rejection and the two-reconnect assertion are unchanged.
Document the probe steps, the panic stack (client.go:165 'send on closed channel' in a processLoop goroutine), the source lines that close the channels while reader and pong goroutines still send, the other rejected StreamClient behaviors, and the requirements for the stage F connection layer. The probe stays a separate diagnostics module and fails once the defect stops reproducing.
Stages B-D of docs/proposals/enterprise-im.md; they change the same repo and service code paths and land together. Stage B - configuration scope and credentials - NotificationBot gains channel, write approval, config/credential versions and tenant state; IdentityScope/ReceiveSlot/LeaseKey keep WeCom keys byte-identical and scope new channels by configuration. - notification_bots physical columns (channel, external_app_id, tenant_key, unique receive_slot, enabled, interaction_enabled, versions); saves are CAS on config_version, retry MySQL deadlock victims, and keep one receiving configuration per application. - internal/credcrypt: AES-256-GCM keyring from AGENTFLOW_CREDENTIAL_KEYS, lf1:<kid>: envelope bound to configuration and channel; legacy plaintext stays readable; -credential-action status|reencrypt (re-entrant). - dbNowUS dialect clock; leases and send windows are *_us columns written and compared on the database clock. - bot_native_sessions mapping written before the CLI starts; a failed write stops the turn. - AGENTFLOW_BOT_WORKERS_ENABLED maintenance switch; migrations 036/037 backfill physical columns, project snapshots and WeCom binding owners (BINARY identifier comparisons; verified twice on MariaDB 10.11). Stage C - fencing and delivery state machine - bot_connections epoch per owner; every interaction write locks config -> connection -> interaction and checks owner, epoch and live lease; zero rows is ErrBotLeaseLost. - ready -> sending -> replied | reply_failed | reply_unknown with a fixed send key, short send lease, 15s lease headroom and 10s call timeout. Not-delivered retries 3x (5s/30s/2m) then fails; unknown retries with the same key inside first send + 55 minutes only for idempotent channels, else reply_unknown. Expired routes fail with route_expired. - Disable keeps ready replies, cancels queued/running and marks sending unknown; delete cancels unsent work, marks in-flight unknown and revokes the configuration's bindings. Notifications gain send_unknown. - Manual confirm/resend APIs with action-key de-duplication; resend is a new generation and key. WeCom fallback send only after proven non-delivery. Stage D - bindings and read-only policy - Challenges carry config, channel and masked sender; confirmation must match the configuration; link issuance is rate limited. - botExecutionUser re-reads configuration, binding and project role. - Read-only exec mode: platform writes hidden and refused, external MCP writes and workspace tools refused, drafts never executed; new Bots default to read-only, legacy WeCom keeps auto. - Natural-language Bot answers no longer need a terminal tool call.
Stage E of docs/proposals/enterprise-im.md (R1 Feishu private-chat loop). - channel_feishu.go: Verify obtains a tenant access token (enterprise query is informational); Connect runs the official long connection (larksuite/oapi-sdk-go v3.13.0, auto reconnect off, service reconnects) and acknowledges an event only after persistence within 2.5s, returning 500 for storage errors so Feishu redelivers; Reply uses Message Reply with the persisted send key as uuid. Transport and platform codes are classified (dial failure / rate limit / token: not delivered; other transport: unknown; other codes: rejected). - Tenant confirmation: the first trusted event records a candidate without bumping config_version and answers with a fixed notice; a Bot manager locks it (CAS on configVersion) via /tenant/confirm; locking does not reconnect; other tenants are dropped; conflicts can be rediscovered. - Inbound: private text only, group events counted and ignored, non-text and over-8000-character input get a notice without reaching the model. - Bot management UI: channel picker (DingTalk shown as in development), write approval for WeCom, credential migration hint, enterprise confirmation step with the platform query, record-page confirm/resend actions. Bind page shows channel, Bot, project and masked sender. - OpenAPI for Bot create/update/delete, tenant and manual actions; docs/im-channels.md (setup, minimum Feishu permissions, maintenance window, R1 acceptance checklist); CHANGELOG notes the WeCom read-only default, fallback-send and disable behavior changes; Go notices bundle.
…migration Found by the MySQL 8.4.13 rehearsal and the multi-process fault test: - Feishu ack: repository calls take no context, so a blocked database held the event callback past Feishu's 3s window. The sink now runs in its own goroutine and the callback answers 500 at 2.5s; the redelivery is stored once and processed once (fault test F2). - 037: the WeCom owner backfill and report selected a non-aggregated column, rejected under ONLY_FULL_GROUP_BY (MySQL 8.4 default, error 1055). - Bot natural answers: only plain text from a successful run is accepted. Protocol JSON whose drafts were filtered during parsing (for example "将创建需求") previously passed as the reply; it now reruns the terminal step like any draft attempt. Matrix test covers read-only and legacy auto Bots, progress-only, empty, protocol JSON, failed CLI and redaction. - internal/botinteraction/feishu_fault_mysql_test.go: real MySQL, receiver child processes killed with SIGKILL, simulated Feishu platform with one-hour uuid de-duplication. F1 duplicate delivery, F2 blocked storage, F3 long-connection drop, F4 crash during the model turn (lease takeover), F5 crash with the reply in flight, F6 lost reply response. - docs/im-channels.md: rollback, fault recovery and the natural-answer behavior change; CHANGELOG entry.
Fixed acceptance commit 57c5450; automated results; MySQL 8.4.13 upgrade and rollback rehearsal with legacy data written by origin/main; natural-answer behavior matrix with baseline comparison; fault tests F1-F6; R1-01..R1-13 status by real-environment criteria with the evidence collected so far and what each item still needs.
The page heading, command palette entry and English copy still described WeCom-only notification channels, which read as if Feishu were missing. The channel picker itself is inside Add channel (step 1).
compose.yaml lists the platform environment explicitly, so AGENTFLOW_CREDENTIAL_KEYS never reached the container and saving a Bot secret failed with credential_key_missing even when .env set it. AGENTFLOW_PUBLIC_URL (chat binding links) and AGENTFLOW_BOT_WORKERS_ENABLED were missing as well. - compose.yaml passes the three variables; the public URL falls back to LOOPFORGE_PUBLIC_URL. - scripts/configure.sh generates a 32-byte key for new .env files and adds one to existing .env files that lack it; an existing key is never replaced. - test-install.py covers generation, upgrade of an old .env, idempotence and the compose wiring.
R1 accepted by yowucai on 2026-11-18. Real-tenant pass: R1-01, R1-02, R1-05; rehearsal pass: R1-13; remaining items not run individually and waived by the acceptance owner, with automated and fault-test evidence.
scripts/check-public.py rejects internal identifiers in tracked files.
Stage F of docs/proposals/enterprise-im.md (R2 DingTalk private-chat loop). - channel_dingtalk.go: own Stream connection layer on gorilla/websocket; only payload frames and the chatbot callback model of dingtalk-stream-sdk-go v0.9.1 are used. One reader, serialized writes, no channel closed under a sender; WebSocket ping every 60s with a read deadline; context-aware dial; no reconnect inside the adapter (the service reconnects with backoff); SYSTEM ping answered, disconnect reconnects; bot callbacks acknowledged after persistence within 2.5s, 500 on storage errors, bounded concurrency. - Verify via /v1.0/oauth2/accessToken; endpoint via connections/open with a no-redirect, bounded HTTP client. - Events: senderStaffId identity, sender/bot corp must match, robotCode checked when present; session webhook accepted only for https://oapi.dingtalk.com/robot/sendBySession. - Session webhook sealed with the credential keyring, bound to the interaction (AAD), cleared on replied, route_expired and Bot delete; expiry in ms -> route_expires_at_us. No idempotency key: unknown outcomes stop at reply_unknown. - Enterprise scope: administrator-entered CorpId; the first matching trusted event locks it (config_version+1, attributed to that admin), another enterprise is a sticky conflict until corrected or rediscovered. - API tenantKey (tenant_required, immutable once locked); UI CorpId field and automatic confirmation step; channel page copy. - Tests: event mapping, stream ping/ack/timeout/disconnect/late-frame/ close/cancel (-race), webhook classification and no redirects, service loop, repo tenant/seal, HTTP save, webtest; multi-process MySQL fault test D1-D5 (duplicate, blocked storage, silent gateway + late frame, crash in send + manual resend, expired webhook).
Fixed acceptance commit f9c94e0, automated results, fault tests D1-D5 and the R2-01..R2-12 real-environment checklist (not run: no DingTalk tenant yet), including the facts to record during the pilot.
DingTalk and Feishu replies are plain text, so the Markdown link syntax was shown verbatim and the auto-linker kept the closing bracket in the URL, which made the binding page reject the token. Put the URL on its own line for these channels and make the page extract the 64-hex token tolerantly. Page title no longer says WeCom.
Stage G, part 1. Feishu (im/v1/messages to open_id, send key as uuid) and DingTalk (robot oToMessages/batchSend, cached app token) implement PushSender; notification delivery routes both through it, also for notification-only configurations. Targets are members bound in a private chat with the Bot. Saving a target first sends a trial message and only a confirmed delivery records target_verified_at on the binding; revocation or re-binding clears it. Deliveries persist send_key, generation and the database start time before the network call: Feishu retries an unknown outcome with the same key inside 55 minutes, DingTalk stops at send_unknown. Admins can confirm receipt or resend (new generation and key), and a target switch with queued notifications requires an explicit cancel or migrate choice.
Stage G, part 2. When a DingTalk session webhook expired before the reply went out (reply_failed / route_expired), an administrator can switch the saved reply to a proactive private message to the original sender. The transaction re-checks the configuration, the failed state and that the sender's binding still belongs to the same platform user with a successful trial send; the HTTP layer also re-checks project permission. The reply goes out as a new generation with a new key via PushSender and the switch, actor and reason are recorded. Without a verified sender the records page offers a trial send to that member instead (POST /api/notification-bots/:id/targets/verify), which leaves the notification recipient unchanged.
Stage G, part 3. WeCom Bots get an optional confirm write mode. The turn runs with platform writes blocked; drafts are frozen in bot_pending_drafts (random ID, parameter hash, 30-minute expiry) and the reply carries a summary plus a /bot-approve.html#draft=<id> link. Only the bound platform user can open or confirm a draft. Confirmation re-checks the configuration, the sender's binding, project permission, expiry and the frozen parameters, then moves the draft from pending to executing in one locked statement, so a repeated click or a second tab returns the current state instead of executing again. Execution reuses resolveAssistantDrafts (preflight warnings stop a draft). A draft left in executing is reported as needs_review and never re-run. Unbinding, deleting the Bot, disabling chat or leaving the confirm mode cancel pending drafts. Feishu and DingTalk stay read-only. Also adds a MySQL/MariaDB integration test for the stage G statements (push targets, two-phase pushes, target switch, push resend, draft confirmation under concurrency).
…ation Manual section 8 (recipients and trial sends, delivery rules, DingTalk push resend, WeCom web confirmation, upgrade), the stage G acceptance record with automated results and the real-environment checklist, CHANGELOG, OpenAPI entries and the implementation notes in the proposal (verification stored on bindings, no business idempotency key).
web/embed.go lists static files explicitly; add bot-approve.html and bot-approve.js and cover both Bot pages in the static route test.
Stage H, part 1. An hourly cleanup (with the binding-request cleanup, batches of 1000, at most ten per category and run) applies two horizons to terminal records, counted from when they finished: - body (default 90 days): message, reply and sealed route of interactions, rendered notification content, frozen write drafts and the native CLI logs of Bot turns are removed; rows stay and can no longer be sent or executed again (body_purged_at, content_source purged); - record (default 180 days): the rows are deleted. Records waiting for someone (queued, running, ready, sending, reply_unknown, pending, awaiting_authorization, send_unknown, executing, needs_review) are never touched. Pending drafts past expiry become expired. AGENTFLOW_BOT_BODY_RETENTION_DAYS and AGENTFLOW_BOT_RECORD_RETENTION_DAYS adjust the horizons; 0 disables one. Native logs are removed only where a finished, due bot_native_sessions mapping points: file names equal to the requested or reported session ID in the user's own CLI home (CodeBuddy, Claude Code; Codex rollouts by thread ID), without following symbolic links and skipping sessions still shared with a running or newer mapping. Each mapping records removed, missing, error or unsupported with the relative paths; mappings are deleted after the record horizon only once their log is settled.
compose.yaml passes AGENTFLOW_BOT_BODY_RETENTION_DAYS and AGENTFLOW_BOT_RECORD_RETENTION_DAYS (90/180 by default); the install test checks every Bot variable is wired; deployment.md lists the chat Bot settings.
Conflicts: go.mod/go.sum (both dependency sets, go mod tidy), internal/repo/models.go (Bot tables plus gitHTTPCredentialRow), web/embed.go (bot-approve files plus kimi-code.png) and the Go notice bundle (regenerated with scripts/collect-go-notices.py --write).
Manual: retention settings, upgrade step to confirm them before the first cleanup, section 9 (retention rules, native log cleanup and monitoring) and section 10 (rollback matrix). Final test record with the automated results on the merged code, the MySQL 8.4 upgrade and rollback rehearsal of the final build (first retention run included) and the IM-01..IM-12 traceability per channel. OpenAPI gains the remaining Bot notification routes and the purge markers; CHANGELOG and the proposal note the retention behaviour. The stage G record now carries the MySQL 8.4 rerun.
Feishu and DingTalk Bots can now choose the 网页确认 write mode, so members can create requirements and dispatch or start SOPs from a private chat: the turn returns an operation summary and a confirmation link, and the bound user confirms on the platform page, where the draft runs at most once with the same re-checks as on WeCom. The link stands on its own line in these plain-text channels. Automatic writes stay WeCom only; saving auto on Feishu or DingTalk is refused.
added 4 commits
November 20, 2026 02:53
Requirements created through web confirmation used to get the project default SOP silently. The chat summary now names the SOP each new requirement will be bound to, and the confirmation page lists the project's published SOPs so the user can switch it and choose to start the SOP right after creation. Only SOPs offered on the page are accepted; the frozen draft is unchanged. The confirm-mode prompt tells the model to set workflowId when the user names an SOP.
Waiting and failed node notifications of Bots with chat enabled are now sent as action cards on Feishu (interactive message, buttons carry the action and delivery) and DingTalk (createAndDeliver with the template ID an administrator configures, STREAM callbacks, IM_ROBOT space). Without a DingTalk template the notification stays plain text. Clicks arrive on the receiving connection (Feishu card.action.trigger, DingTalk /v1.0/card/instances/callback), are acknowledged inside the callback window and queued through the common inbound path. They run handleBotCardAction with the WeCom card checks (bound sender, project permission, node still in the notified state and attempt, action still offered); only the recipient's click is placed in the delivery's conversation. The result goes back to the clicker as a private message. Manual section 8.5 documents the Feishu callback and the DingTalk template contract; acceptance items C-01..C-05 and W-05..W-06 added.
…odel - Feishu replies with write drafts are now an interactive card (the Bot's explanation, the operations with their SOP, and 确认执行 / 确认并启动 SOP / 取消 buttons plus a link to the confirmation page). Clicks come back as card.action.trigger and run the same re-checks and pending -> executing transition as the page (shared executeBotDraft); only the requester can use them, the result is a private message. - Feishu and DingTalk answer group messages that @ the Bot (Feishu mentions with mentioned_type=bot, DingTalk isInAtList); other group messages are still ignored and counted. @ placeholders are removed from the text. Each group keeps its own conversation; every message runs as the sender's bound account. - The Bot form calls the model field 回答模型 and always offers 默认. - The confirm-mode prompt keeps internal field names out of replies.
The SOP dropdown of new requirements now starts with 项目默认 SOP (name), selected when the model named none, and lists the same SOPs as the platform requirement form: the project's own, shared SOPs and the generic templates (archived ones excluded). Choosing the default removes any workflowId the model set, so the requirement is bound the usual way. The chat summary marks unnamed SOPs as 项目默认.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Chat Bots now cover three enterprise IM channels instead of WeCom only, following
docs/proposals/enterprise-im.md(v3, stages A–H).网页确认, all three channels): the turn freezes write drafts and replies with a summary and a/bot-approve.html#draft=…link; only the bound user can open it, confirmation re-checks binding, project permission, expiry and parameters, and a draft runs at most once. Automatic writes stay WeCom only. New WeCom Bots default to read-only; existing ones keep automatic execution.uuid; DingTalk stops at “delivery unknown” for manual confirm or resend. Expired DingTalk replies can be resent as a proactive private message.reply_unknowninstead of guessing, manual confirm/resend on the record page, maintenance switchAGENTFLOW_BOT_WORKERS_ENABLED.AGENTFLOW_CREDENTIAL_KEYS,-credential-action status|reencrypt); Compose generates a key.0keeps).Manual:
docs/im-channels.md. Final test record:docs/testing/im-final-test-record.md.Scope
Compatibility
User-visible changes (also in
CHANGELOG.md):AGENTFLOW_BOT_BODY_RETENTION_DAYS/AGENTFLOW_BOT_RECORD_RETENTION_DAYSfirst if longer retention is needed.Upgrade needs a maintenance window (manual section 2): stop all old replicas → start the new version once with
AGENTFLOW_BOT_WORKERS_ENABLED=false(startup DDL adds columns and tables) → runmigrations/036_enterprise_im_channels.sqland037_bot_binding_ownership.sql(idempotent) → configure keys and run-credential-action reencrypt→ confirm retention settings → restart with workers enabled. Afterreencrypt, rolling back to a pre-IM build requires restoring the backup (rollback matrix in manual section 10).New dependencies:
github.com/larksuite/oapi-sdk-go/v3v3.13.0 (Feishu long connection) andgithub.com/open-dingtalk/dingtalk-stream-sdk-gov0.9.1 (only itspayload/chatbotdata types; the connection loop is an in-repo Stream client because the SDK client can crash the process, reproduction in the separate probe modulescripts/im-sdk-probe). Go notices regenerated.Validation
Run locally on the merged code (
origin/main@6f606cfmerged in, no conflicts left):Upgrade and rollback rehearsal on MySQL 8.4.13 from
origin/main@6f606cfto this branch, including the first retention run and restore-backup rollback: see the final test record, section 3.Real tenants: Feishu private chat (R1) and DingTalk private chat (R2) signed off on a staging deployment; stage G signed off without per-item evidence. Feishu/DingTalk web confirmation (W-01..W-04) not yet run in a real tenant.
Not run locally (left to CI): Go coverage gate, real-browser tests,
scan-secrets.sh, Playwright e2e,local/scripts/*.Safety