Skip to content

slack: validate data_visualization against Slack's per-chart caps - #36

Open
josephschorr wants to merge 2 commits into
mainfrom
slack-data-viz-validation-limits
Open

josephschorr wants to merge 2 commits into
mainfrom
slack-data-viz-validation-limits

Conversation

@josephschorr

@josephschorr josephschorr commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Problem

A time-series answer ("p95 over the last 3 hours") rendered via the data_visualization Block Kit component produced a chart with far more than Slack's 20 points/categories (and >20-char timestamp labels). Slack rejected the whole message with invalid_blocks, and sender.go degraded it to plain text — so the chart silently vanished while the agent believed respond_to_user succeeded.

Root cause: validateVizSeries only checked that series and axis_config.categories were non-empty. It did not model the per-chart limits Slack actually enforces (documented in slack-go's block_data_visualization.go, verified against the live API):

  • Data must be 1–20 points, exactly one per axis_config.categories entry
  • 1–6 series (bar/area/line), unique names
  • all category/series/point labels ≤20 chars; pie 1–6 segments

A fine-grained time series blows straight past the 20-point/category cap (and 20-char timestamp labels), passes local validation, and only fails at post time — where the failure is invisible because the degrade log records just the bare invalid_blocks code.

Fix

  1. Harden validateVizSeries / validateDataViz to enforce Slack's documented caps, so an over-dense or misaligned chart becomes a precise, model-facing, correctable error (same philosophy the file already states) instead of a silent post-time degrade.
  2. Surface ResponseMetadata.Messages (Slack's per-block json-pointer reason) at all three blocks rejected degrade log sites — SlackErrorResponse.Error() returns only the bare code, so the actionable detail was being discarded.
  3. Teach the model the caps up front in the respond_to_user authoring instructions (bucket long time ranges into ≤20 points, short labels, etc.).

Tests

TDD: added failing cases to TestValidateComponents (series/category/point caps, one-point-per-category alignment, label lengths, unique series names, pie segment cap/labels — plus boundary-accept cases at 20 points / 6 series) and a unit test for the new blocksRejectionDetail helper. go test ./pkg/channels/... and go vet pass.

@vercel

vercel Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
openagentprimitives Ignored Ignored Preview Oct 23, 2026 8:50pm UTC

Request Review

A time-series answer (e.g. "p95 over the last 3 hours") produces a
data_visualization chart with far more than Slack's 20 points/categories,
and/or >20-char timestamp labels. validateVizSeries only checked that series
and axis_config.categories were non-empty, so such a chart passed local
validation, reached chat.postMessage, and was rejected whole as
invalid_blocks — which the sender degrades to plain text, dropping the chart
silently. The agent sees its respond_to_user succeed and believes it posted a
chart that never rendered.

Enforce the limits slack-go's block_data_visualization.go documents (verified
against the live API) so an over-dense/misaligned chart becomes a precise,
model-facing, correctable error instead of a post-time silent degrade:
- bar/area/line: 1–6 series, unique names; 1–20 x-axis categories; exactly one
  data point per category; category/series/point labels ≤20 chars; axis
  x_label/y_label ≤50 chars.
- pie: 1–6 segments; segment labels ≤20 chars.

Also surface Slack's ResponseMetadata.Messages (the per-block json-pointer
reason) at the three blocks-rejected degrade log sites — SlackErrorResponse's
Error() returns only the bare code, so the actionable detail was being
discarded — and teach the model the caps up front in the respond_to_user
authoring instructions.
@samkim

samkim commented Oct 23, 2026

Copy link
Copy Markdown
Member

Review: data_visualization validation is missing some of Slack's rules

The new checks in pkg/channels/channelkinds/slack/components.go cover only part of what Slack enforces. Some invalid charts still pass validation, Slack rejects them with invalid_blocks, and the whole respond_to_user reply silently drops to plain text. That is the failure this PR is meant to prevent.

Rules Slack enforces that our checks miss

  1. Point labels aren't matched to categories (components.go:426). The check only compares counts. Categories ["Mon","Tue"] with points [{label:"Mon"},{label:"Mon"}] or [{label:"Monday"},{label:"Tuesday"}] pass here and fail at Slack.
  2. Repeated or empty categories pass (components.go:408). For example, ["Mon","Mon"] or a category of "".
  3. Pie slices with a value of 0 or less, or an empty label, pass (components.go:375). Slack requires a value above 0 and a label of at least 1 character. A breakdown with a zero bucket ({"label":"Refunds","value":0}) is a realistic way to hit this.
  4. Empty series names and empty point labels pass (components.go:417).

Recommendation: use slack-go's validator

slack-go v0.30.1 (already in go.mod) has DataVisualizationBlock.Validate() in block_data_visualization.go. It already rejects repeated categories (must not contain duplicate category), labels that don't match a category (must match axis_config.categories), and repeated labels within a series. The code here is a partial copy of it and has already drifted, which goes against the repo's "prefer a library over a hand-rolled utility" rule.

Suggested change: call dv.Validate() first, wrap its error with blockErrf(loc, ...) so the model still gets a located, correctable error, and keep only the repo-specific checkBroadcast pass. That removes about 50 lines and closes items 1 and 2. Please confirm whether Validate() also covers items 3 and 4; if it doesn't, keep explicit checks for those.

Smaller items

  • Model instructions (components.go:72) say "EXACTLY one data point per category" but never say each point's label must equal its category string, or that categories must be unique. A model could reasonably label category "2026-10-01" as "Oct 1". Worth stating both.
  • Tests (components_test.go:224): there are no cases for any of the gaps above. Some fixtures are themselves invalid to Slack and pass only because an earlier check fails first. For example, the "category label over 20 chars" case uses point label "x", which also doesn't match its category. Suggested cases: label/category mismatch, repeated categories, repeated point labels, a pie value of 0 or less, and empty names/labels.
  • Comment (components.go:412) says "Slack requires exactly one data point per category, unique series names", which implies the loop enforces alignment. It only checks counts. Update it, or let it go away along with the switch to Validate().

🤖 Generated with Claude Code

The hand-rolled per-chart checks were a partial copy of slack-go's
DataVisualizationBlock.Validate and missed rules Slack enforces: point
labels must match a category, categories must be unique and non-empty,
pie values must be > 0, and names/labels must be non-empty. Call
Validate, wrap its error with the block location plus the full rule set,
and keep only the whitespace-title check and the broadcast guard locally.
State the label==category and unique-category rules in the model
instructions, and fix test fixtures that were invalid to Slack.
@josephschorr

Copy link
Copy Markdown
Member Author

Switched to dv.Validate() (448e6be). It covers all four gaps, including pie value > 0 and empty names/labels, so the only checks left in our code are the whitespace-only title and the broadcast guard. Validate's errors are wrapped with the block location and the full chart rule set. The model instructions now say that point labels must equal their category string and that categories must be unique. I fixed the invalid fixtures and added cases for each gap.

This branch had an error being deployed

1 failed (outdated) deployment
Preview — 9f23380a Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants