Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/tanchat-send-latency.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,13 @@ Live Worker tracing on the next short reply measured 1,398 ms of wall time and 5
The HTTP send path now overlaps workflow ownership, lifecycle metadata, and run context reads after resolving the authenticated conversation identity. All three reads settle before the request database context is released. Authorization, lifecycle rejection, workflow ownership rejection, and post-preparation checks remain in place.

Structured chat_send_timing, chat_admission_timing, and chat_run_timing logs record cumulative milliseconds within each operation. They separate HTTP authentication and metadata, admission guards and preparation, model credentials and selection, usage reservation, enrichment, context preparation, first visible text, its durable save, and response completion. They do not log message text, credentials, or account identity. Admission and run logs use the message receipt ID for correlation. The first-text save is not a measurement of browser paint or stream delivery.

## Measured database round trips, October 5

The first instrumented live run measured 301 ms of HTTP authentication and metadata, 694 ms inside admission, and 1,319 ms from run start to model context. The two admission guards accounted for 555 ms. Usage reservation took 916 ms after credentials, and enrichment took another 171 ms. First text arrived 2,895 ms after model context, consistent with the recorded 3.01-second model call. The browser observed the reply after 5.935 seconds, including browser control and observation overhead.

Reservation now executes the existing protocol inside one PostgreSQL function call. It keeps the same receipt and day advisory lock keys, fresh receipt and role checks, allowance ordering, and atomic receipts, counters, and funded reservations. The function is SECURITY INVOKER and VOLATILE, so queries inside it obtain fresh snapshots after waiting for locks. A real database regression revokes admin access while reservation waits on the day lock and verifies rejection without a receipt.

Ordinary admission guards now read membership, ownership, bot lifecycle, and thread lifecycle together. Both checks before and after preparation remain fresh. Retry readiness still receives a separate lifecycle check after its asynchronous readiness wait.

[PostgreSQL function volatility](https://www.postgresql.org/docs/current/xfunc-volatility.html) explains the fresh snapshot behavior. These changes reduce application overhead without changing the model or its response quality. They do not promise near-zero provider latency.
86 changes: 86 additions & 0 deletions drizzle/migrations/0041_tanchat_usage_reservation.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
-- Run the existing allowance protocol next to its data, in one network call.
-- VOLATILE gives each query a fresh snapshot after the advisory locks resolve.
CREATE FUNCTION public.reserve_chat_run_usage(request jsonb, receipt_lock_key text)
RETURNS TABLE(outcome text, receipt_day text)
LANGUAGE plpgsql VOLATILE SECURITY INVOKER SET search_path = public
AS $$
DECLARE
conversation_id_value text := request->>'conversationId';
run_id_value text := request->>'runId';
workspace_id_value text := request->>'workspaceId';
user_id_value uuid := (request->>'userId')::uuid;
bot_id_value text := request->>'botId';
scheduled_value boolean := (request->>'scheduled')::boolean;
day_value text := request->>'day';
unlimited_value boolean := (request->>'unlimited')::boolean;
spend jsonb := request->'spend';
reservation bigint := (spend->>'reservationMicros')::bigint;
receipt chat_run_usage_receipts%ROWTYPE;
user_turns bigint;
global_turns bigint;
scheduled_turns bigint;
user_spend bigint;
global_spend bigint;
included_admin boolean := false;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended(receipt_lock_key, 0));
PERFORM pg_advisory_xact_lock(hashtextextended('chat-run-usage:' || day_value, 0));
SELECT * INTO receipt FROM chat_run_usage_receipts
WHERE conversation_id=conversation_id_value AND run_id=run_id_value;
IF FOUND THEN
IF receipt.workspace_id <> workspace_id_value OR receipt.user_id <> user_id_value
OR receipt.bot_id <> bot_id_value OR receipt.scheduled <> scheduled_value THEN
RETURN QUERY SELECT 'conflict'::text, receipt.day;
ELSE
RETURN QUERY SELECT 'duplicate'::text, receipt.day;
END IF;
RETURN;
END IF;
SELECT
COALESCE((SELECT turns FROM chat_daily_usage WHERE user_id=user_id_value::text AND day=day_value),0),
COALESCE((SELECT turns FROM chat_daily_usage WHERE user_id='__global' AND day=day_value),0),
COALESCE((SELECT turns FROM chat_scheduled_daily_usage WHERE user_id=user_id_value AND day=day_value),0),
COALESCE((SELECT SUM(billed_micros) FROM chat_funded_spend WHERE user_id=user_id_value AND day=day_value),0),
COALESCE((SELECT SUM(billed_micros) FROM chat_funded_spend WHERE day=day_value),0)
INTO user_turns,global_turns,scheduled_turns,user_spend,global_spend;
IF NOT (request->>'lift')::boolean AND user_turns >= (request->>'dailyLimit')::integer THEN
RETURN QUERY SELECT 'user'::text, day_value; RETURN;
END IF;
IF NOT (request->>'lift')::boolean AND global_turns >= 300 THEN
RETURN QUERY SELECT 'global'::text, day_value; RETURN;
END IF;
IF NOT unlimited_value AND scheduled_value AND scheduled_turns >= (request->>'scheduledLimit')::integer THEN
RETURN QUERY SELECT 'scheduled'::text, day_value; RETURN;
END IF;
IF reservation IS NOT NULL AND NOT unlimited_value THEN
SELECT EXISTS (
SELECT 1 FROM users u
LEFT JOIN role_assignments a ON a.user_id=u.id
LEFT JOIN roles r ON r.id=a.role_id
WHERE u.id=user_id_value AND
('admin'=ANY(u.capabilities::text::text[]) OR 'admin'=ANY(r.capabilities::text::text[]))
) INTO included_admin;
IF NOT included_admin AND user_spend + reservation > (spend->>'userCapMicros')::bigint THEN
RETURN QUERY SELECT 'user-spend'::text, day_value; RETURN;
END IF;
IF NOT included_admin AND global_spend + reservation > (spend->>'globalCapMicros')::bigint THEN
RETURN QUERY SELECT 'global-spend'::text, day_value; RETURN;
END IF;
END IF;
INSERT INTO chat_run_usage_receipts(conversation_id,run_id,workspace_id,user_id,bot_id,scheduled,day,created_at,attempt_id)
VALUES(conversation_id_value,run_id_value,workspace_id_value,user_id_value,bot_id_value,scheduled_value,day_value,
(request->>'now')::bigint,(request->>'attemptId')::uuid);
INSERT INTO chat_daily_usage(user_id,day,turns)
VALUES(user_id_value::text,day_value,1),('__global',day_value,1)
ON CONFLICT(user_id,day) DO UPDATE SET turns=chat_daily_usage.turns+1;
IF reservation IS NOT NULL THEN
INSERT INTO chat_funded_spend(conversation_id,run_id,user_id,day,reserved_micros,billed_micros)
VALUES(conversation_id_value,run_id_value,user_id_value,day_value,reservation,reservation);
END IF;
IF scheduled_value THEN
INSERT INTO chat_scheduled_daily_usage(user_id,day,turns) VALUES(user_id_value,day_value,1)
ON CONFLICT(user_id,day) DO UPDATE SET turns=chat_scheduled_daily_usage.turns+1;
END IF;
RETURN QUERY SELECT 'reserved'::text, day_value;
END;
$$;
7 changes: 7 additions & 0 deletions drizzle/migrations/meta/_journal.json
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,13 @@
"when": 1790963800008,
"tag": "0040_tanchat_waitlist",
"breakpoints": true
},
{
"idx": 41,
"version": "7",
"when": 1791183973089,
"tag": "0041_tanchat_usage_reservation",
"breakpoints": true
}
]
}
145 changes: 0 additions & 145 deletions harness-tests/core/run-usage-admin.test.ts

This file was deleted.

59 changes: 59 additions & 0 deletions harness-tests/pending-runtime/run-usage-admin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,62 @@ it('increments both daily counters once for concurrent reservations and duplicat
await h.db`SELECT turns FROM chat_daily_usage WHERE day='2026-10-04' AND turns=4`,
).toHaveLength(2)
})

it('rechecks role revocation after waiting for the usage lock', async () => {
const h = await conversationHarness()
await h.db`UPDATE users SET capabilities=ARRAY['admin']::capability[] WHERE id=${input.identity.userId}`
await h.db`INSERT INTO chat_funded_spend(conversation_id,run_id,user_id,day,reserved_micros,billed_micros) VALUES('main-conversation','prior',${input.identity.userId},'2026-10-04',20000000,20000000)`
const result = await h.db.begin(async (tx) => {
await tx.unsafe(
"SELECT pg_advisory_xact_lock(hashtextextended('chat-run-usage:2026-10-04',0))",
)
const pending = reserveRunUsage(input).then(
(value) => ({ value }),
(error: unknown) => ({ error }),
)
await expect
.poll(async () => {
const [row] = await tx.unsafe<{ waiting: boolean }[]>(
"SELECT EXISTS(SELECT 1 FROM pg_locks WHERE locktype='advisory' AND NOT granted) AS waiting",
)
return row.waiting
})
.toBe(true)
await tx.unsafe(
"UPDATE users SET capabilities='{}'::capability[] WHERE id=$1",
[input.identity.userId],
)
return { pending }
})
expect(await result.pending).toMatchObject({ error: { code: 'user-spend' } })
expect(
await h.db`SELECT run_id FROM chat_run_usage_receipts WHERE run_id='admin-run'`,
).toHaveLength(0)
})

it('rejects a regular account at the shared included spending cap', async () => {
const h = await conversationHarness()
await h.db`INSERT INTO users(id) VALUES('00000000-0000-4000-8000-000000000002')`
await h.db`INSERT INTO chat_funded_spend(conversation_id,run_id,user_id,day,reserved_micros,billed_micros) VALUES('main-conversation','prior','00000000-0000-4000-8000-000000000002','2026-10-04',20000000,20000000)`
await expect(reserveRunUsage(input)).rejects.toMatchObject({
code: 'global-spend',
})
})

it.each(['user', 'global', 'scheduled'] as const)(
'retains the %s turn limit for admins',
async (code) => {
const h = await conversationHarness()
await h.db`UPDATE users SET capabilities=ARRAY['admin']::capability[] WHERE id=${input.identity.userId}`
if (code === 'scheduled')
await h.db`INSERT INTO chat_scheduled_daily_usage(user_id,day,turns) VALUES(${input.identity.userId},'2026-10-04',20)`
else
await h.db`INSERT INTO chat_daily_usage(user_id,day,turns) VALUES(${code === 'user' ? input.identity.userId : '__global'},'2026-10-04',${code === 'user' ? 20 : 300})`
await expect(
reserveRunUsage({ ...input, scheduled: true }),
).rejects.toMatchObject({ code })
expect(
await h.db`SELECT run_id FROM chat_run_usage_receipts WHERE run_id='admin-run'`,
).toHaveLength(0)
},
)
5 changes: 5 additions & 0 deletions scripts/verify-tanchat-schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ try {
missing.push(`${table.name}.${column.name} must be NOT NULL`)
}
}
const [reservation] = await sql<{ ready: boolean }[]>`
SELECT to_regprocedure('public.reserve_chat_run_usage(jsonb,text)') IS NOT NULL AS ready
`
if (!reservation.ready)
missing.push('reserve_chat_run_usage function missing')
if (missing.length)
throw new Error(`Chat schema is not ready:\n${missing.join('\n')}`)
console.log(
Expand Down
Loading
Loading