Skip to content

Commit 47ff76d

Browse files
ericallamericallam
authored
feat(webapp,clickhouse): return an actionable error instead of a 500 when a runs list query is too expensive (#4773)
## Summary When a runs list query is too expensive to complete, it now fails with a clear, actionable error instead of a generic 500. Previously, a runs list query that exceeded ClickHouse resource limits threw an opaque error. On the public `runs.list` API that surfaced as a retryable 500, so a customer task calling it would keep retrying a query that could never succeed. On the dashboard it rendered as a generic error page with no hint about what to do. ## Fix The ClickHouse client now tags resource-limit failures (memory, time, rows, bytes) with their error type, and the runs repository maps those to a dedicated `RunsListQueryError` (HTTP 422). - `runs.list` API returns 422 with a message telling the user to narrow their `created_at` range, plus an `x-should-retry: false` header so the SDK does not retry it. - The dashboard runs list (and the errors, scheduled, standard-task, agents, and webhooks list views) render a shared error state with the same guidance, so a too-broad time filter is recoverable by the user.
1 parent 1eda438 commit 47ff76d

17 files changed

Lines changed: 813 additions & 376 deletions

File tree

‎.server-changes/runs-list-query-limit-error.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: improvement
4+
---
5+
6+
When a runs list or runs.list API request spans too much data to complete, it now returns a clear, actionable error asking you to narrow the time range, instead of failing with a generic error.

‎apps/webapp/app/components/runs/v3/RunsListErrorState.tsx‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import { Callout } from "~/components/primitives/Callout";
2+
3+
/**
4+
* Error state for a runs list that failed to load. Shown as the `errorElement` of the deferred
5+
* runs-list data. The most common recoverable cause is a query that was too expensive over a broad
6+
* time range (see `RunsListQueryError`), so the copy guides narrowing the range; a refresh covers
7+
* transient failures. The precise reason is not shown because Remix scrubs thrown error messages in
8+
* production.
9+
*/
10+
export function RunsListErrorState() {
11+
return (
12+
<div className="flex items-center justify-center px-3 py-12">
13+
<Callout variant="error" className="max-w-fit">
14+
We couldn't load these runs. If you're filtering over a broad time range, try narrowing it,
15+
then refresh to try again.
16+
</Callout>
17+
</div>
18+
);
19+
}
20+
21+
/**
22+
* Renders nothing. Used as the `errorElement` for secondary awaits of the same runs-list promise
23+
* (e.g. the pagination controls), so a rejection is handled locally there and does not bubble to
24+
* the route error boundary. The primary awaits render {@link RunsListErrorState}.
25+
*/
26+
export function RunsListErrorStateNoop() {
27+
return null;
28+
}

‎apps/webapp/app/presenters/v3/ErrorGroupPresenter.server.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,8 @@ export class ErrorGroupPresenter extends BasePresenter {
8787
constructor(
8888
private readonly replica: PrismaClientOrTransaction,
8989
private readonly logsClickhouse: ClickHouse,
90-
private readonly clickhouse: ClickHouse
90+
private readonly clickhouse: ClickHouse,
91+
private readonly runsListClickhouse: ClickHouse
9192
) {
9293
super(undefined, replica);
9394
}
@@ -409,7 +410,7 @@ export class ErrorGroupPresenter extends BasePresenter {
409410
columns?: RunColumnsSelect;
410411
}
411412
): Promise<NextRunList | undefined> {
412-
const runListPresenter = new NextRunListPresenter(this.replica, this.clickhouse);
413+
const runListPresenter = new NextRunListPresenter(this.replica, this.runsListClickhouse);
413414

414415
const result = await runListPresenter.call(organizationId, environmentId, {
415416
userId: options.userId,

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.agents.$agentParam/route.tsx‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,11 @@ import { Paragraph } from "~/components/primitives/Paragraph";
2222
import * as Property from "~/components/primitives/PropertyTable";
2323
import { Spinner } from "~/components/primitives/Spinner";
2424
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
25+
import {
26+
RunsListErrorState,
27+
RunsListErrorStateNoop,
28+
} from "~/components/runs/v3/RunsListErrorState";
29+
import { RunsListQueryError } from "~/services/runsRepository/runsRepository.server";
2530
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
2631
import { TaskRunsTable } from "~/components/runs/v3/TaskRunsTable";
2732
import { SessionsTable } from "~/components/sessions/v1/SessionsTable";
@@ -92,10 +97,10 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
9297
const directionRaw = url.searchParams.get("direction") ?? undefined;
9398
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
9499

95-
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
96-
project.organizationId,
97-
"standard"
98-
);
100+
const [clickhouse, runsListClickhouse] = await Promise.all([
101+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "standard"),
102+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "runsList"),
103+
]);
99104

100105
const presenter = new AgentDetailPresenter($replica, clickhouse);
101106
const agent = await presenter.findAgent({
@@ -154,7 +159,7 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
154159
})
155160
.catch(() => ({ data: [], statuses: [] }) satisfies AgentActivity);
156161

157-
const runList = new NextRunListPresenter($replica, clickhouse)
162+
const runList = new NextRunListPresenter($replica, runsListClickhouse)
158163
.call(project.organizationId, environment.id, {
159164
userId,
160165
projectId: project.id,
@@ -166,7 +171,12 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
166171
direction,
167172
columns: getRunColumnsForSelect(request),
168173
})
169-
.catch(() => null);
174+
.catch((error) => {
175+
if (error instanceof RunsListQueryError) {
176+
throw error;
177+
}
178+
return null;
179+
});
170180

171181
const sessionList = new SessionListPresenter($replica, clickhouse)
172182
.call(project.organizationId, environment.id, {
@@ -341,7 +351,7 @@ export default function Page() {
341351
<>
342352
<RunsDisplayOptions sampleFilters={{ tasks: agent.slug, rootOnly: "false" }} />
343353
<Suspense fallback={null}>
344-
<TypedAwait resolve={runList} errorElement={null}>
354+
<TypedAwait resolve={runList} errorElement={<RunsListErrorStateNoop />}>
345355
{(list) => (list ? <ListPagination list={list} /> : null)}
346356
</TypedAwait>
347357
</Suspense>
@@ -395,7 +405,7 @@ function AgentContentArea({
395405
</Suspense>
396406
) : (
397407
<Suspense fallback={<TableLoading />}>
398-
<TypedAwait resolve={runList} errorElement={<TableLoading />}>
408+
<TypedAwait resolve={runList} errorElement={<RunsListErrorState />}>
399409
{(list) =>
400410
list ? (
401411
<TaskRunsTable

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.errors.$fingerprint/route.tsx‎

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ import { Spinner } from "~/components/primitives/Spinner";
5555
import { useToast } from "~/components/primitives/Toast";
5656
import TooltipPortal from "~/components/primitives/TooltipPortal";
5757
import type { TaskRunListSearchFilters } from "~/components/runs/v3/RunFilters";
58+
import { RunsListErrorState } from "~/components/runs/v3/RunsListErrorState";
5859
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
5960
import { TaskRunsTable } from "~/components/runs/v3/TaskRunsTable";
6061
import { $replica } from "~/db.server";
@@ -254,12 +255,18 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
254255
const directionRaw = url.searchParams.get("direction") ?? undefined;
255256
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
256257

257-
const [logsClickhouseClient, clickhouseClient] = await Promise.all([
258+
const [logsClickhouseClient, clickhouseClient, runsListClickhouseClient] = await Promise.all([
258259
clickhouseFactory.getClickhouseForOrganization(environment.organizationId, "logs"),
259260
clickhouseFactory.getClickhouseForOrganization(environment.organizationId, "standard"),
261+
clickhouseFactory.getClickhouseForOrganization(environment.organizationId, "runsList"),
260262
]);
261263

262-
const presenter = new ErrorGroupPresenter($replica, logsClickhouseClient, clickhouseClient);
264+
const presenter = new ErrorGroupPresenter(
265+
$replica,
266+
logsClickhouseClient,
267+
clickhouseClient,
268+
runsListClickhouseClient
269+
);
263270

264271
const detailPromise = presenter
265272
.call(project.organizationId, environment.id, {
@@ -393,16 +400,7 @@ export default function Page() {
393400
</div>
394401
}
395402
>
396-
<TypedAwait
397-
resolve={data}
398-
errorElement={
399-
<div className="flex items-center justify-center px-3 py-12">
400-
<Callout variant="error" className="max-w-fit">
401-
Unable to load error details. Please refresh the page or try again in a moment.
402-
</Callout>
403-
</div>
404-
}
405-
>
403+
<TypedAwait resolve={data} errorElement={<RunsListErrorState />}>
406404
{(result) => {
407405
if ("error" in result) {
408406
return (

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs._index/route.tsx‎

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ import {
7474
import { throwNotFound } from "~/utils/httpErrors";
7575
import { ListPagination } from "../../components/ListPagination";
7676
import { CreateBulkActionInspector } from "../resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.bulkaction";
77-
import { Callout } from "~/components/primitives/Callout";
77+
import { RunsListErrorState } from "~/components/runs/v3/RunsListErrorState";
7878
import {
7979
isRunsListLoading,
8080
RUNS_BULK_INSPECTOR_OPEN_VALUE,
@@ -208,17 +208,7 @@ export default function Page() {
208208
</div>
209209
}
210210
>
211-
<TypedAwait
212-
resolve={data}
213-
errorElement={
214-
<div className="flex items-center justify-center px-3 py-12">
215-
<Callout variant="error" className="max-w-fit">
216-
Unable to load your task runs. Please refresh the page or try again in a
217-
moment.
218-
</Callout>
219-
</div>
220-
}
221-
>
211+
<TypedAwait resolve={data} errorElement={<RunsListErrorState />}>
222212
{(list) => {
223213
return (
224214
<RunsList

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.tasks.scheduled.$taskParam/route.tsx‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,11 @@ import {
6060
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
6161
import { useToast } from "~/components/primitives/Toast";
6262
import { EnabledStatus } from "~/components/runs/v3/EnabledStatus";
63+
import {
64+
RunsListErrorState,
65+
RunsListErrorStateNoop,
66+
} from "~/components/runs/v3/RunsListErrorState";
67+
import { RunsListQueryError } from "~/services/runsRepository/runsRepository.server";
6368
import type { TaskRunListSearchFilters } from "~/components/runs/v3/RunFilters";
6469
import { ScheduleTypeIcon, scheduleTypeName } from "~/components/runs/v3/ScheduleType";
6570
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
@@ -143,10 +148,10 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
143148
const directionRaw = url.searchParams.get("direction") ?? undefined;
144149
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
145150

146-
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
147-
project.organizationId,
148-
"standard"
149-
);
151+
const [clickhouse, runsListClickhouse] = await Promise.all([
152+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "standard"),
153+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "runsList"),
154+
]);
150155

151156
const taskPresenter = new TaskDetailPresenter($replica, clickhouse);
152157
const task = await taskPresenter.findTask({
@@ -211,7 +216,7 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
211216
})
212217
.catch(() => null);
213218

214-
const runList = new NextRunListPresenter($replica, clickhouse)
219+
const runList = new NextRunListPresenter($replica, runsListClickhouse)
215220
.call(project.organizationId, environment.id, {
216221
userId,
217222
projectId: project.id,
@@ -224,7 +229,12 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
224229
includeHasAnyRuns: true,
225230
columns: getRunColumnsForSelect(request),
226231
})
227-
.catch(() => null);
232+
.catch((error) => {
233+
if (error instanceof RunsListQueryError) {
234+
throw error;
235+
}
236+
return null;
237+
});
228238

229239
return typeddefer({
230240
task,
@@ -375,14 +385,14 @@ export default function Page() {
375385
) : null}
376386
<RunsDisplayOptions sampleFilters={{ tasks: task.slug, rootOnly: "false" }} />
377387
<Suspense fallback={null}>
378-
<TypedAwait resolve={runList} errorElement={null}>
388+
<TypedAwait resolve={runList} errorElement={<RunsListErrorStateNoop />}>
379389
{(list) => (list ? <ListPagination list={list} /> : null)}
380390
</TypedAwait>
381391
</Suspense>
382392
</TitleBar>
383393
<div className="min-h-0 overflow-hidden">
384394
<Suspense fallback={<TableLoading />}>
385-
<TypedAwait resolve={runList} errorElement={<TableLoading />}>
395+
<TypedAwait resolve={runList} errorElement={<RunsListErrorState />}>
386396
{(list) =>
387397
list ? (
388398
<TaskRunsList

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.tasks.standard.$taskParam/route.tsx‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@ import {
3030
} from "~/components/primitives/Resizable";
3131
import { Spinner } from "~/components/primitives/Spinner";
3232
import { TextLink } from "~/components/primitives/TextLink";
33+
import {
34+
RunsListErrorState,
35+
RunsListErrorStateNoop,
36+
} from "~/components/runs/v3/RunsListErrorState";
37+
import { RunsListQueryError } from "~/services/runsRepository/runsRepository.server";
3338
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
3439
import {
3540
QUEUE_METRIC_COLORS,
@@ -103,10 +108,10 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
103108
const direction = directionRaw ? DirectionSchema.parse(directionRaw) : undefined;
104109
const versions = url.searchParams.getAll("versions").filter((v) => v.length > 0);
105110

106-
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
107-
project.organizationId,
108-
"standard"
109-
);
111+
const [clickhouse, runsListClickhouse] = await Promise.all([
112+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "standard"),
113+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "runsList"),
114+
]);
110115

111116
const presenter = new TaskDetailPresenter($replica, clickhouse);
112117
const task = await presenter.findTask({
@@ -153,7 +158,7 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
153158
})
154159
.catch(() => ({ data: [], statuses: [] }) satisfies TaskActivity);
155160

156-
const runList = new NextRunListPresenter($replica, clickhouse)
161+
const runList = new NextRunListPresenter($replica, runsListClickhouse)
157162
.call(project.organizationId, environment.id, {
158163
userId,
159164
projectId: project.id,
@@ -167,7 +172,12 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
167172
includeHasAnyRuns: true,
168173
columns: getRunColumnsForSelect(request),
169174
})
170-
.catch(() => null);
175+
.catch((error) => {
176+
if (error instanceof RunsListQueryError) {
177+
throw error;
178+
}
179+
return null;
180+
});
171181

172182
return typeddefer({
173183
task,
@@ -271,14 +281,14 @@ export default function Page() {
271281
) : null}
272282
<RunsDisplayOptions sampleFilters={{ tasks: task.slug, rootOnly: "false" }} />
273283
<Suspense fallback={null}>
274-
<TypedAwait resolve={runList} errorElement={null}>
284+
<TypedAwait resolve={runList} errorElement={<RunsListErrorStateNoop />}>
275285
{(list) => (list ? <ListPagination list={list} /> : null)}
276286
</TypedAwait>
277287
</Suspense>
278288
</TitleBar>
279289
<div className="min-h-0 overflow-hidden">
280290
<Suspense fallback={<TableLoading />}>
281-
<TypedAwait resolve={runList} errorElement={<TableLoading />}>
291+
<TypedAwait resolve={runList} errorElement={<RunsListErrorState />}>
282292
{(list) =>
283293
list ? (
284294
<TaskRunsList

‎apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.webhooks.$webhookParam/route.tsx‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ import {
2929
import { PulsingDot } from "~/components/primitives/PulsingDot";
3030
import { Spinner } from "~/components/primitives/Spinner";
3131
import { TabButton, TabContainer } from "~/components/primitives/Tabs";
32+
import {
33+
RunsListErrorState,
34+
RunsListErrorStateNoop,
35+
} from "~/components/runs/v3/RunsListErrorState";
36+
import { RunsListQueryError } from "~/services/runsRepository/runsRepository.server";
3237
import { TimeFilter, timeFilterFromTo } from "~/components/runs/v3/SharedFilters";
3338
import { TaskRunsTable } from "~/components/runs/v3/TaskRunsTable";
3439
import { DeliveriesTable } from "~/components/webhookDeliveries/v1/DeliveriesTable";
@@ -116,10 +121,10 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
116121
const runsDirectionRaw = url.searchParams.get("runsDirection") ?? undefined;
117122
const runsDirection = runsDirectionRaw ? DirectionSchema.parse(runsDirectionRaw) : undefined;
118123

119-
const clickhouse = await clickhouseFactory.getClickhouseForOrganization(
120-
project.organizationId,
121-
"standard"
122-
);
124+
const [clickhouse, runsListClickhouse] = await Promise.all([
125+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "standard"),
126+
clickhouseFactory.getClickhouseForOrganization(project.organizationId, "runsList"),
127+
]);
123128

124129
const presenter = new WebhookDetailPresenter($replica, clickhouse);
125130
const webhook = await presenter.findWebhook({
@@ -156,7 +161,7 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
156161
})
157162
.catch(() => ({ data: [], statuses: [] }) satisfies WebhookActivity);
158163

159-
const runList = new NextRunListPresenter($replica, clickhouse)
164+
const runList = new NextRunListPresenter($replica, runsListClickhouse)
160165
.call(project.organizationId, environment.id, {
161166
userId,
162167
projectId: project.id,
@@ -167,7 +172,12 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
167172
cursor: runsCursor,
168173
direction: runsDirection,
169174
})
170-
.catch(() => null);
175+
.catch((error) => {
176+
if (error instanceof RunsListQueryError) {
177+
throw error;
178+
}
179+
return null;
180+
});
171181

172182
const deliveriesList = presenter
173183
.listDeliveries({
@@ -329,7 +339,7 @@ export default function Page() {
329339
</Suspense>
330340
) : (
331341
<Suspense fallback={null}>
332-
<TypedAwait resolve={runList} errorElement={null}>
342+
<TypedAwait resolve={runList} errorElement={<RunsListErrorStateNoop />}>
333343
{(list) =>
334344
list ? (
335345
<ListPagination
@@ -482,7 +492,7 @@ function WebhookContentArea({
482492
</Suspense>
483493
) : (
484494
<Suspense fallback={<TableLoading />}>
485-
<TypedAwait resolve={runList} errorElement={<TableLoading />}>
495+
<TypedAwait resolve={runList} errorElement={<RunsListErrorState />}>
486496
{(list) =>
487497
list ? (
488498
<div className="h-full overflow-y-auto scrollbar-thin scrollbar-track-transparent scrollbar-thumb-charcoal-600">

0 commit comments

Comments
 (0)