rewrite from mcp node to operations and trigger nodes over ws - #3
Merged
Merged
Conversation
The package was an MCP client: it exposed List Tools / Execute Tool and
asked users for a fully-qualified tool name. n8n ships an MCP client node
already, so it read as a duplicate of something they maintain, and none of
DataGrout's own capabilities were visible at the surface.
DataGrout's gateway speaks JSON-RPC 2.0 as a first-class transport
(POST /servers/:id/rpc, same tool surface and auth as MCP), so the node no
longer touches MCP at all. What it exposes instead are operations named for
what a workflow builder wants:
Answer -> Ask plan, run and verify a plain-language question,
returning the answer plus a certificate URL
Data -> Transform compute server-side over inline data or a
reference, emitting one item per record
Memory -> Remember store facts that outlive the execution
-> Recall answer from stored facts and what follows from them
Skill -> Run re-run previously verified work
Details worth knowing:
- DataGrout gates transports per server (enabled_protocols defaults to
[mcp]). The node recognises the refusal and enables JSON-RPC itself,
and the credential test does the same, so there is no dashboard step.
- API Token authentication joins OAuth2, selectable on the node.
- Background tasks are still collected transparently; a caller never sees
a task reference.
- Tool-level failures reported inside a 200 response now fail the node.
Live testing caught an Ask whose plan 400'd being returned as a null
answer marked verified - the one outcome this integration must not
produce.
Verified in a real n8n (Docker, package installed from a local tarball)
against the live gateway: Remember then Recall round-tripped, Transform
computed group-and-sort server-side for 1 credit with a certificate and
emitted clean rows, and a failing Ask surfaced as a node error.
BREAKING CHANGE: the DataGrout MCP node is removed. Workflows using it
must be repointed at the DataGrout node.
Starts a workflow when DataGrout pushes an event. DataGrout multiplexes JSON-RPC 2.0 over one WebSocket, so the node holds a single connection and the server pushes — nothing polls, and there is no webhook URL to paste into a dashboard. It ships in this package rather than a second one because n8n pairs an action node with its trigger (Slack + Slack Trigger, Airtable + Airtable Trigger), and because both must share the DataGrout credential — two packages registering the same credential name would collide. Zero dependencies: Node's built-in WebSocket carries the connection, with the bearer token supplied through the constructor's headers option. The trigger fails with an actionable message when that global is missing (Node < 22) rather than a ReferenceError inside a listener. Live-verified against the gateway: enable_protocol turned on `ws`, the socket negotiated the datagrout-jsonrpc.v1 subprotocol, subscribe was acked, and a server-side broadcast arrived as an emitted item. That test also caught a bug worth naming: `session.ready` is pushed on every connect, so the first version would have started the workflow each time the socket opened or reconnected. Real subscription events carry `params.subscription` and lifecycle frames do not, so that field — matched against our own subscription id — is what separates them.
DataGrout publishes every server-scoped lifecycle event to one topic, ws:<server_uuid>:events, with the event name in the envelope. So the trigger now subscribes once and filters, instead of asking the user to name a topic it could only guess at. Events replace the freeform topic field with a multi-select over the four slugs DataGrout publishes — run.completed, task.completed, task.failed, tool_call.failed. An empty selection means every event, so a slug added to DataGrout later arrives without this node needing a release. Topic survives as an option for following a single orchestration run, which still has a topic of its own. The "Test step" example now carries the real fields of whichever event is selected, so downstream nodes can be wired against the true shape. Also fixes a fault that only a long-running live test could show: the gateway closes an idle socket after 60s, and an event subscriber is idle by definition, so the connection dropped and re-subscribed every single minute — losing anything published during the gap. A JSON-RPC notification (a request with no id) is the correct keepalive: the spec forbids a reply, so it resets the idle timer without provoking a frame or a log line at either end. Verified over three minutes: three drops before, zero after. Verified in n8n 2.36.9 against the live gateway, end to end — a real prism.refract call produced run.completed, which started the workflow and recorded an execution carrying the event's real fields.
DataGrout now sends run_id as the integer runs.get accepts, with execution_id alongside as its own field. The example event and the README table carried the old shape, where run_id was an exec_ string.
The server issues a fresh subscription id per connection. The old one was kept across a reconnect, so events carrying the new id were filtered out until the new ack arrived — and permanently if that ack were ever missed or refused. Clearing it on open makes the filter accept any subscription event until the new id is known, which is the correct default. Also relabels the error log: it fires for any id-bearing error frame, not just a refused subscribe.
The trigger had no unit coverage: its lifecycle was tangled with
ITriggerFunctions and the global WebSocket, so the only way to exercise
it was to run a real n8n against the real gateway. That is how the last
two defects were found, which is too late and too slow.
The lifecycle now takes its dependencies as arguments — the socket, the
clock, the emit callback, the logger — so the tests drive it directly with
a fake socket and no network. 24 new tests pin the behaviour that only a
long live run used to reveal: session.ready never starting a workflow,
slug filtering, backoff and its reset, re-subscribing per connection,
the id-less keepalive ping, and the stale-subscription-id regression
fixed in the previous commit. Frame shapes are the ones captured from the
gateway on 2026-08-27.
It shares pure.ts rather than living in its own module because shipped
source is compiled with node10 resolution and so cannot carry `.ts`
import extensions, and without them Node cannot load a sibling import
under --experimental-strip-types. One file, one import path, no loader.
The node itself is now the adapter it should have been: read parameters,
supply a real socket, hand n8n its two callbacks.
Two examples, since a trigger with no example is hard to evaluate:
- task-result: a detached task's `task.completed` carries the result's
cache_ref, and the next node transforms that result server-side. This
is why cache_ref is on the wire.
- failures: one topic carries every event, so a workflow subscribes
once and branches on `event`. Splits task.failed from
tool_call.failed.
Re-verified live after the refactor, since it moved the runtime path: in
n8n 2.36.9 against the gateway, a real prism.refract call produced
run.completed and recorded an execution carrying it. Both examples were
imported and activated to confirm their node types resolve — they stop
only at "no credentials set", which is correct for a shipped example.
Leads with what the package does that a plain HTTP call cannot — a verified answer or an admission, memory that outlives the execution, compute that stays next to the data, pushed events — because the sharpest question a reader has is why this is a node rather than an HTTP Request, and the answer was previously the second thing on the page. Registers the trigger properly: the single event topic, what each event carries, that Task Completed brings a cache_ref the next node can hand straight to Transform, and the Test step button. Adds the examples table (all three, including the two new trigger ones), the trigger's own options, and a Compatibility section. The Node floor is now stated in context: n8n 2.x requires Node 24, so the trigger's Node 22 minimum is already satisfied rather than being a constraint a reader has to weigh. Also states plainly that the credential test and the trigger switch on the transports they need, and that both calls are idempotent.
Testing the credential POSTed to interaction/enable_protocol, which persists a new enabled_protocols list on the user's server. Three things were wrong with that. It overrode a deliberate decision. enabled_protocols is a surface control and a new server has only MCP on, so leaving JSON-RPC off is how an admin says "only MCP reaches this server". Between the credential test, the transport's self-heal and the trigger's startup, three separate paths switched it back on — silently — which made the dashboard toggle non-authoritative for anyone using this node. They could not win that argument, and nothing told them they were in one. It answered the wrong question. Someone pressing Test is asking whether their token is still good; that should not mutate their account. And it demanded rights the node otherwise never uses. If DataGrout gains scoped tokens, a token that works perfectly for asking questions would fail the credential test, telling the user their credentials are broken when they are fine. The test is now a read: tools.list over POST /rpc. The status codes line up exactly right — a bad token is 401/403 from MCPAuthPlug, an unknown Server ID is 404, and a server that has not yet enabled JSON-RPC answers HTTP 200 with a JSON-RPC error in the body, which n8n reads as a pass. That last one is the case the old design existed to solve, and it turns out not to need solving: "your transport is not switched on yet" is a server setting, not a bad credential. Auto-enabling stays where the user has actually asked for something to run — the transport's first call, and the trigger's activation — and both now log that they changed a server setting instead of doing it silently. So a transport deliberately left off stays off right up to the moment someone runs a workflow through this node. Verified: tools.list against the live gateway returns 200 with the tool list, and the action node still computes correctly end to end in n8n (31+7 grouped to lisbon=38, porto=12).
DataGrout answers an identical call repeated with nothing changing in
between with an explanation instead of running it again. Two of those
three replies are not flagged as errors, so they flowed through as though
they were the result: a Transform expecting rows handed the workflow
`{loop_detected: true, consequence_tier: "read", execution_count: 4,
action: "possible_loop"}` as its data, with nothing to indicate the call
had not run. Caught live on 2026-09-01 while re-running a test workflow.
Nothing executed, so the honest outcome is a failure. interventionNotice
recognises both unflagged shapes — the read-tier loop and the
write-confirmation gate — and payloadError surfaces them, which covers
every resource since all four already funnel through it. The
write/destructive loop was already handled: it sets isError, which
callTool throws on.
The messages read as the tail of the node's own "DataGrout: " prefix
rather than repeating the product name, and each says what to do: vary the
input, or change the arguments for a write.
Verified live at the documented threshold: the same workflow run six times
succeeded three times, then failed on the 4th, 5th and 6th with
"DataGrout: this identical call ran 4 times with nothing changing in
between, so the loop guard stopped it and no new result was produced."
AGENTS.md now records how to test against a real gateway, why inputs must
vary between runs, and the rule that no token belongs in this repo or in
CI without a secret that is skipped when absent.
AashishChakravarty
approved these changes
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.