From 680655efbda5db37eda4ec6ea7270af482906376 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 21:37:57 +0200 Subject: [PATCH 01/14] feat(console): split the navigation into a User View and an Admin View The sidebar was one flat list, which put "Settings" next to "Calendar" as though changing the bot's runtime configuration and looking at your own meetings were the same kind of act. They are not: one is a use of your own data, the other is an operation on a guild on behalf of everyone in it. Two named groups say which hat the reader is wearing before they click anything, and the Admin View has room to grow -- a user directory, a queue overview and reporting are all administrative and all homeless in a flat list of four. The section flag hides the heading as well as its entries. A visible "Admin View" with nothing under it would announce the existence of a section to exactly the person who may not have it. Hiding remains a courtesy and never a control: every administrative endpoint checks administrator status itself, and the worst this can do is offer somebody a page that then refuses them. Collapsed, the groups are a rule between two runs of icons rather than a heading -- but each group carries its name on the group element in both modes, so the boundary is announced rather than merely drawn. Somebody who cannot see the rule is exactly the person who most needs telling that the next icon acts on the whole guild. The settings page moves to /admin/bot-settings and /settings becomes a permanent redirect. The old address is in browser histories and in whatever anybody pasted into a chat, and a 404 there teaches people the console loses pages. --- console/app/components/AppSidebar.vue | 61 ++++++-- .../{settings.vue => admin/bot-settings.vue} | 2 +- console/app/utils/navigation.ts | 134 +++++++++++++----- console/nuxt.config.ts | 9 ++ console/test/navigation.spec.ts | 78 ++++++++-- 5 files changed, 222 insertions(+), 62 deletions(-) rename console/app/pages/{settings.vue => admin/bot-settings.vue} (99%) diff --git a/console/app/components/AppSidebar.vue b/console/app/components/AppSidebar.vue index c0c620d..4e7a766 100644 --- a/console/app/components/AppSidebar.vue +++ b/console/app/components/AppSidebar.vue @@ -7,8 +7,15 @@ * name through `aria-label` and its `title`, so a screen reader and a * hover both still say what the icon means. An icon rail whose entries * announce nothing is a rail only its author can navigate. + * + * The same rule governs the two groups. Expanded, "User View" and "Admin + * View" are visible headings; collapsed, they are a rule between two runs + * of icons -- but each group carries its name on the group element itself + * in both modes, so the boundary is announced rather than merely drawn. + * Somebody who cannot see the rule is exactly the person who most needs to + * be told that the next icon acts on the whole guild. */ -import { visibleEntries } from '~/utils/navigation' +import { visibleSections } from '~/utils/navigation' const { collapsed } = useSidebar() const session = useSession() @@ -16,7 +23,7 @@ const session = useSession() // The list and the filter both live in `~/utils/navigation`, where they // can be tested without rendering anything -- and where the note that // hiding is a courtesy rather than a control is written down. -const visible = computed(() => visibleEntries(session.value)) +const visible = computed(() => visibleSections(session.value)) diff --git a/console/app/pages/settings.vue b/console/app/pages/admin/bot-settings.vue similarity index 99% rename from console/app/pages/settings.vue rename to console/app/pages/admin/bot-settings.vue index f4b2cae..afc632b 100644 --- a/console/app/pages/settings.vue +++ b/console/app/pages/admin/bot-settings.vue @@ -40,7 +40,7 @@ import { writeSelectedGuild, } from '~/utils/settings' -useHead({ title: 'Settings' }) +useHead({ title: 'Bot Settings' }) const api = useApi() diff --git a/console/app/utils/navigation.ts b/console/app/utils/navigation.ts index 98540e1..57f2acb 100644 --- a/console/app/utils/navigation.ts +++ b/console/app/utils/navigation.ts @@ -6,47 +6,115 @@ * and a decision embedded in a template can only be tested by rendering * one. * - * **Hiding is a courtesy, never a control.** Every settings endpoint checks - * administrator status itself. If this list and the API ever disagree, the - * API is right, and the worst this can do is show somebody a section that - * then refuses them. + * **Two groups, because there are two jobs.** "User View" is what a person + * does with their own recordings; "Admin View" is what somebody does to the + * system on behalf of a guild. Those were one flat list until the admin + * side grew past a single entry, and a flat list is where "Settings" sits + * next to "Calendar" as though changing the bot's configuration and looking + * at your own meetings were the same kind of act. The grouping says which + * hat the reader is wearing before they click anything. + * + * **Hiding is a courtesy, never a control.** Every administrative endpoint + * checks administrator status itself. If this list and the API ever + * disagree, the API is right, and the worst this can do is show somebody a + * section that then refuses them. */ export interface NavEntry { to: string label: string - /** An SVG path. Inline rather than an icon dependency: four glyphs do not - * justify a package, and a self-contained path cannot go missing. */ + /** An SVG path. Inline rather than an icon dependency: a handful of + * glyphs do not justify a package, and a self-contained path cannot go + * missing. */ icon: string adminOnly?: boolean } -export const NAV_ENTRIES: readonly NavEntry[] = [ - { - to: '/', - label: 'Dashboard', - icon: 'M4 13h7V4H4v9Zm0 7h7v-5H4v5Zm9 0h7V11h-7v9Zm0-16v5h7V4h-7Z', - }, - { - to: '/recordings', - label: 'Recordings', - icon: 'M12 3a3 3 0 0 1 3 3v6a3 3 0 1 1-6 0V6a3 3 0 0 1 3-3Zm7 9a7 7 0 0 1-6 6.93V21h-2v-2.07A7 7 0 0 1 5 12h2a5 5 0 0 0 10 0h2Z', - }, - { - to: '/calendar', - label: 'Calendar', - icon: 'M7 2v2h10V2h2v2h1a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1V2h2ZM4 9v11h16V9H4Z', - }, - { - to: '/settings', - label: 'Settings', - icon: 'M12 8a4 4 0 1 0 0 8 4 4 0 0 0 0-8Zm9.4 4a7.4 7.4 0 0 0-.1-1.2l2-1.6-2-3.4-2.4 1a7.6 7.6 0 0 0-2-1.2L16.5 3h-4l-.4 2.6c-.7.3-1.4.7-2 1.2l-2.4-1-2 3.4 2 1.6a7.4 7.4 0 0 0 0 2.4l-2 1.6 2 3.4 2.4-1c.6.5 1.3.9 2 1.2l.4 2.6h4l.4-2.6c.7-.3 1.4-.7 2-1.2l2.4 1 2-3.4-2-1.6c.1-.4.1-.8.1-1.2Z', - adminOnly: true, - }, -] - -/** The entries this viewer should see. Nobody signed in sees no admin-only - * sections, which is also what an anonymous render gets. */ +/** + * A named run of entries. + * + * `adminOnly` sits on the section as well as on its entries, so a section + * whose every entry is administrative is hidden whole -- heading included. + * A visible "Admin View" heading with nothing under it would announce the + * existence of a section to exactly the person who may not have it. + */ +export interface NavSection { + label: string + entries: readonly NavEntry[] + adminOnly?: boolean +} + +export const USER_VIEW: NavSection = { + label: 'User View', + entries: [ + { + to: '/', + label: 'Dashboard', + icon: 'M4 13h7V4H4v9Zm0 7h7v-5H4v5Zm9 0h7V11h-7v9Zm0-16v5h7V4h-7Z', + }, + { + to: '/recordings', + label: 'Recordings', + icon: 'M12 3a3 3 0 0 1 3 3v6a3 3 0 1 1-6 0V6a3 3 0 0 1 3-3Zm7 9a7 7 0 0 1-6 6.93V21h-2v-2.07A7 7 0 0 1 5 12h2a5 5 0 0 0 10 0h2Z', + }, + { + to: '/calendar', + label: 'Calendar', + icon: 'M7 2v2h10V2h2v2h1a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1V2h2ZM4 9v11h16V9H4Z', + }, + ], +} + +export const ADMIN_VIEW: NavSection = { + label: 'Admin View', + adminOnly: true, + entries: [ + { + to: '/admin/bot-settings', + label: 'Bot Settings', + icon: 'M12 8a4 4 0 1 0 0 8 4 4 0 0 0 0-8Zm9.4 4a7.4 7.4 0 0 0-.1-1.2l2-1.6-2-3.4-2.4 1a7.6 7.6 0 0 0-2-1.2L16.5 3h-4l-.4 2.6c-.7.3-1.4.7-2 1.2l-2.4-1-2 3.4 2 1.6a7.4 7.4 0 0 0 0 2.4l-2 1.6 2 3.4 2.4-1c.6.5 1.3.9 2 1.2l.4 2.6h4l.4-2.6c.7-.3 1.4-.7 2-1.2l2.4 1 2-3.4-2-1.6c.1-.4.1-.8.1-1.2Z', + adminOnly: true, + }, + ], +} + +export const NAV_SECTIONS: readonly NavSection[] = [USER_VIEW, ADMIN_VIEW] + +/** + * Every entry, in the order the sidebar renders them. + * + * The flat view is kept because "what does this console have pages for" is + * a question with one answer, and answering it by walking two levels at + * every call site is how the two levels get walked slightly differently. + */ +export const NAV_ENTRIES: readonly NavEntry[] = NAV_SECTIONS.flatMap( + (section) => section.entries, +) + +/** Nobody signed in sees no admin-only sections, which is also what an + * anonymous render gets. */ +function permitted(viewer: { is_admin: boolean } | null, adminOnly?: boolean): boolean { + return !adminOnly || Boolean(viewer?.is_admin) +} + +/** + * The sections this viewer should see, each already filtered. + * + * A section left with no entries is dropped rather than rendered empty: a + * heading over nothing is a heading that says a section exists and is + * withheld, which is more than the person is owed and less than they can + * use. + */ +export function visibleSections(viewer: { is_admin: boolean } | null): NavSection[] { + return NAV_SECTIONS.filter((section) => permitted(viewer, section.adminOnly)) + .map((section) => ({ + ...section, + entries: section.entries.filter((entry) => permitted(viewer, entry.adminOnly)), + })) + .filter((section) => section.entries.length > 0) +} + +/** The entries this viewer should see, flattened out of their sections. */ export function visibleEntries(viewer: { is_admin: boolean } | null): NavEntry[] { - return NAV_ENTRIES.filter((entry) => !entry.adminOnly || Boolean(viewer?.is_admin)) + return visibleSections(viewer).flatMap((section) => section.entries) } diff --git a/console/nuxt.config.ts b/console/nuxt.config.ts index 9848fc6..f5b0b08 100644 --- a/console/nuxt.config.ts +++ b/console/nuxt.config.ts @@ -46,6 +46,15 @@ export default defineNuxtConfig({ }, }, + routeRules: { + // The bot's configuration used to live at `/settings`, before there + // was an Admin View for it to sit inside. The old address is kept as a + // permanent redirect rather than deleted: it is in browser histories + // and in whatever anybody pasted into a chat, and a 404 there teaches + // people the console loses pages. + '/settings': { redirect: { to: '/admin/bot-settings', statusCode: 301 } }, + }, + nitro: { // The container runs this as a plain Node server behind a Service. preset: 'node-server', diff --git a/console/test/navigation.spec.ts b/console/test/navigation.spec.ts index 8456c90..f46e83d 100644 --- a/console/test/navigation.spec.ts +++ b/console/test/navigation.spec.ts @@ -1,5 +1,5 @@ /** - * Which navigation entries exist, and who is offered which. + * Which navigation entries exist, who is offered which, and in which group. * * The list lives in its own module rather than inside the component for * exactly this reason: what belongs in the navigation is a decision, and a @@ -7,15 +7,30 @@ */ import { describe, expect, it } from 'vitest' -import { NAV_ENTRIES, visibleEntries } from '../app/utils/navigation' +import { + ADMIN_VIEW, + NAV_ENTRIES, + NAV_SECTIONS, + USER_VIEW, + visibleEntries, + visibleSections, +} from '../app/utils/navigation' + +const ADMIN = { is_admin: true } +const PARTICIPANT = { is_admin: false } describe('the navigation', () => { - it('offers the four sections the console has', () => { - expect(NAV_ENTRIES.map((e) => e.label)).toEqual([ + it('separates what a person does with their own recordings from what an administrator does', () => { + expect(NAV_SECTIONS.map((s) => s.label)).toEqual(['User View', 'Admin View']) + }) + + it('puts the personal sections first', () => { + // A participant who administers nothing is the common case, and their + // sections should not be below a heading they never see. + expect(USER_VIEW.entries.map((e) => e.label)).toEqual([ 'Dashboard', 'Recordings', 'Calendar', - 'Settings', ]) }) @@ -24,26 +39,63 @@ describe('the navigation', () => { // announce nothing is a rail only its author can navigate. for (const entry of NAV_ENTRIES) { expect(entry.label.trim()).not.toBe('') + expect(entry.icon.trim()).not.toBe('') } }) - it('hides settings from somebody who administers nothing', () => { - expect(visibleEntries({ is_admin: false }).map((e) => e.label)).not.toContain('Settings') + it('addresses every entry with a path of its own', () => { + const paths = NAV_ENTRIES.map((e) => e.to) + expect(new Set(paths).size).toBe(paths.length) + }) +}) + +describe('who is offered the Admin View', () => { + it('marks every entry under it administrative, not just the section', () => { + // The section flag hides the heading; the entry flags are what + // `visibleEntries` filters on. A section marked administrative whose + // entries are not would leak its entries to any caller that flattens + // first and filters second. + for (const entry of ADMIN_VIEW.entries) { + expect(entry.adminOnly).toBe(true) + } }) - it('offers settings to an administrator', () => { - expect(visibleEntries({ is_admin: true }).map((e) => e.label)).toContain('Settings') + it('offers the bot settings to an administrator', () => { + expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('Bot Settings') }) - it('hides settings when nobody is signed in', () => { - expect(visibleEntries(null).map((e) => e.label)).not.toContain('Settings') + it('hides it from somebody who administers nothing', () => { + expect(visibleSections(PARTICIPANT).map((s) => s.label)).toEqual(['User View']) + }) + + it('hides it when nobody is signed in', () => { + expect(visibleSections(null).map((s) => s.label)).toEqual(['User View']) + }) + + it('never renders a heading over an empty section', () => { + // A visible "Admin View" with nothing under it would announce the + // existence of a section to exactly the person who may not have it. + for (const viewer of [ADMIN, PARTICIPANT, null]) { + for (const section of visibleSections(viewer)) { + expect(section.entries.length).toBeGreaterThan(0) + } + } }) it('shows every non-administrative section to everyone', () => { // Hiding a section is a courtesy to the person looking at the screen, - // never a control -- so it applies to exactly the one section whose + // never a control -- so it applies to exactly the entries whose // endpoints refuse a non-administrator, and to nothing else. const forEveryone = NAV_ENTRIES.filter((e) => !e.adminOnly).map((e) => e.label) - expect(visibleEntries({ is_admin: false }).map((e) => e.label)).toEqual(forEveryone) + expect(visibleEntries(PARTICIPANT).map((e) => e.label)).toEqual(forEveryone) + }) + + it('offers an administrator everything a participant is offered, and more', () => { + const asParticipant = visibleEntries(PARTICIPANT).map((e) => e.label) + const asAdmin = visibleEntries(ADMIN).map((e) => e.label) + for (const label of asParticipant) { + expect(asAdmin).toContain(label) + } + expect(asAdmin.length).toBeGreaterThan(asParticipant.length) }) }) From 0b6fb1e68d1807137e8f0146b3cafeff4d6d3972 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 21:48:53 +0200 Subject: [PATCH 02/14] feat(console): let an administrator withdraw a person's recording consent Until now a consent could end in two ways: the person ended it themselves with `/consent revoke`, or an administrator bumped `policy_version`, which ends everybody's at once. Neither answers the case in between -- somebody left the team, or asked in a channel rather than in a slash command, or is no longer somebody this guild should be recording. What an administrator reaches for otherwise is removing the Discord role by hand. That stops the recording and leaves `revoked_at` NULL, so `/consent status` still reports consent active and re-adding the role silently resumes recording somebody who never re-consented. This closes that gap from the side that lasts. What a revocation from the console is, exactly ---------------------------------------------- It stamps `revoked_at` on the stored consent record, through `ConsentRepository.record_revocation` -- the same statement `/consent revoke` makes, unwrapped, because a second definition of what a revocation is would agree with the first right up until one of them changed. It does **not** remove the Discord role. The API process holds no Discord token and never will (Spec 13.2): a process that can decrypt every recording ever made is not one to also give the ability to act as the bot. That is enough to stop the recording -- consent is two layers, and the stored record is checked on every frame through a five second cache, and it is the layer that exists precisely because the role can be bypassed by anyone with administrator permissions in Discord. So a revocation takes effect within five seconds, mid-session. What it leaves behind is a role the person still holds, which is visible in Discord and misleading if nobody says so. It is also **not a delete**. Nothing already recorded is touched; `/audio purge` does that, it is admin-gated, and it is deliberately a separate act. Withdrawing consent is a decision about the future, and erasing a meeting a team has already read is not the same decision. Every row therefore carries how many recordings of that person the guild still holds, so the distinction is on the screen rather than in a document nobody opens. The rest -------- `active` is computed server-side from `sturnus.domain.consent`, never derived in the browser. A grant names the policy version it was given under, so a guild that has moved its `policy_version` on has consents with no `revoked_at` and no force -- and a console working that out for itself would be a second implementation of `is_consent_active`. Authorisation is in `ConsoleConsentDirectory`, not in the handlers: there is no method on it that can be called without saying who is asking, and both answer `None` for "no such guild" and for "you do not administer it" alike. The handlers answer 404 to both. A 403 would confirm that a list of who agreed to be recorded exists here, to somebody just established as having no business with it. `consent.revoked_at` records that a revocation happened and never who performed it, so `console.consent_revoked` at WARNING carries `requested_by` beside `discord_user_id` and is the only place that pair is ever written down. --- src/sturnus/console/adapters.py | 257 ++++++++++- src/sturnus/console/app.py | 6 + src/sturnus/console/ports.py | 79 ++++ src/sturnus/console/routes_consent.py | 241 +++++++++++ src/sturnus/entrypoints/api.py | 17 +- src/sturnus/observability/events.py | 14 + tests/console/conftest.py | 40 ++ tests/console/test_consent_directory.py | 545 ++++++++++++++++++++++++ tests/console/test_consent_routes.py | 409 ++++++++++++++++++ 9 files changed, 1605 insertions(+), 3 deletions(-) create mode 100644 src/sturnus/console/routes_consent.py create mode 100644 tests/console/test_consent_directory.py create mode 100644 tests/console/test_consent_routes.py diff --git a/src/sturnus/console/adapters.py b/src/sturnus/console/adapters.py index aca3b8c..fe4f86f 100644 --- a/src/sturnus/console/adapters.py +++ b/src/sturnus/console/adapters.py @@ -19,26 +19,35 @@ from __future__ import annotations +from collections.abc import Callable, Sequence +from dataclasses import dataclass from datetime import datetime, timedelta -from sqlalchemy import delete, select +from sqlalchemy import delete, func, select from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker from sturnus.application.linking import new_state from sturnus.console.ports import ( AdminDirectory, + ConsentHolder, QueueSnapshot, QueueSpeaker, RequeueOutcome, + RevocationOutcome, + SettingsStore, Track, ) +from sturnus.domain import settings +from sturnus.domain.consent import ConsentRecord, is_consent_active from sturnus.infrastructure.db.models import ( AccountLink, + Consent, ConsoleState, SessionParticipant, TranscriptionJob, ) from sturnus.infrastructure.db.models import Session as SessionRow +from sturnus.infrastructure.db.repositories import ConsentRepository from sturnus.infrastructure.db.requeue import ( SessionView, apply_requeue, @@ -321,3 +330,249 @@ def refusal_reason(view: SessionView) -> str: "There is nothing to re-queue: every recording in this session has been erased, or it " "never had any." ) + + +#: Why a revocation did nothing. Bounded literals from this file rather +#: than sentences, because they travel into a log line as `reason` -- a +#: field the observability registry admits precisely on the grounds that +#: its values are fixed literals from this repository's own source. The +#: sentences a person reads are the console's, next to the button that +#: produced them. +NO_CONSENT_ON_RECORD = "no_consent_on_record" +ALREADY_REVOKED = "already_revoked" + + +@dataclass(frozen=True) +class _ConsentRow: + """One `consent` row, with the columns the table declares NOT NULL. + + Not `ConsentRecord`: that one makes `granted_at` and `policy_version` + optional, because the *absence* of a record is one of the states it + represents. A row that was read out of the table is not absent, and + carrying the optionality forward would push a `None` check into every + caller that cannot happen. + """ + + granted_at: datetime + revoked_at: datetime | None + policy_version: str + + +class ConsoleConsentDirectory: + """Who has consented in a guild, and an administrator's power to end it. + + The authorisation is here rather than in a handler, exactly as it is + for `ConsoleTrackDirectory` and `ConsoleQueueControl`: every method + asks `AdminDirectory` whether this person administers *this* guild and + answers `None` when they do not. There is no method that can be called + without `requested_by`, so there is no filter to forget. + + **The write is `ConsentRepository.record_revocation`, unwrapped.** That + is the same statement `/consent revoke` makes -- newest row by + `granted_at`, `revoked_at` stamped rather than a new row inserted, + because the history keeps grants and a revocation modifies the grant + it revokes. A console that reimplemented it would be a second + definition of what a revocation is, and the two would agree right up + until one of them changed. + + **What this cannot do, and what follows from that.** Consent is two + layers (Spec 3.1). The Discord role is checked synchronously on every + frame with no cache; the stored record is checked on every frame + through `ConsentCache`'s five second TTL. This process holds no + Discord token (Spec 13.2) so it writes the record and leaves the role + alone -- which stops the recording within five seconds, mid-session, + because the stored record is the layer that exists precisely because + the role can be bypassed. It does not take the role away, and the + console says so rather than letting an administrator infer it. + + It also does not erase anything already recorded. `/audio purge` + does, it is admin-gated, and it is deliberately a separate act: + withdrawing consent is a decision about the future, and deleting a + meeting a team has already read is not the same decision. Every + holder therefore carries `recordings_with_audio`, so nobody has to + guess which of the two they just did. + """ + + def __init__( + self, + session_factory: async_sessionmaker[AsyncSession], + admins: AdminDirectory, + config: SettingsStore, + now: Callable[[], datetime], + ) -> None: + self._session_factory = session_factory + self._admins = admins + self._config = config + self._consents = ConsentRepository(session_factory) + self._now = now + + async def holders( + self, guild_id: int, *, requested_by: int + ) -> tuple[ConsentHolder, ...] | None: + if not await self._admins.is_admin(guild_id, requested_by): + return None + + newest = await self._newest_consent_per_person(guild_id) + if not newest: + return () + + people = tuple(newest) + names = await self._latest_display_names(guild_id, people) + held = await self._recordings_with_audio(guild_id, people) + # Read once for the whole listing rather than per person: whether + # a grant is still active depends on the guild's current policy + # version, and that is one value, not one per row. + policy = (await self._config.snapshot(guild_id)).get(settings.POLICY_VERSION, "") + + return tuple( + ConsentHolder( + discord_user_id=discord_user_id, + display_name=names.get(discord_user_id), + policy_version=row.policy_version, + granted_at=row.granted_at, + revoked_at=row.revoked_at, + # The domain's rule, not a reimplementation of it. An + # administrator must be shown the same verdict the + # recorder acts on -- including the case nobody expects, + # where a policy bump has quietly ended a consent nobody + # withdrew. + active=is_consent_active( + ConsentRecord( + granted_at=row.granted_at, + revoked_at=row.revoked_at, + policy_version=row.policy_version, + ), + policy, + ), + recordings_with_audio=held.get(discord_user_id, 0), + ) + # By id, so two page loads agree. What order a *person* + # wants to read this in is the console's decision, made in + # `~/utils/consents` where it can be tested without a + # database. + for discord_user_id, row in sorted(newest.items()) + ) + + async def revoke( + self, guild_id: int, discord_user_id: int, *, requested_by: int + ) -> RevocationOutcome | None: + if not await self._admins.is_admin(guild_id, requested_by): + return None + + # Read before writing, only so the answer can say what happened. + # `record_revocation` is idempotent and silent -- it stamps the + # newest row or does nothing -- and an administrator told + # "revoked" for somebody who never consented would believe a + # protection is in place that never was. + record = await self._consents.current(discord_user_id, guild_id) + if record is None or record.granted_at is None: + return RevocationOutcome(revoked=False, refusal=NO_CONSENT_ON_RECORD) + if record.revoked_at is not None: + return RevocationOutcome(revoked=False, refusal=ALREADY_REVOKED) + + # A grant naming a superseded `policy_version` is revoked rather + # than refused, even though it is already inactive. It is inactive + # *because of a setting*, and a setting can be set back; stamping + # `revoked_at` is the only thing that survives somebody restoring + # the old policy version. + await self._consents.record_revocation(discord_user_id, guild_id, self._now()) + return RevocationOutcome(revoked=True, refusal=None) + + async def _newest_consent_per_person(self, guild_id: int) -> dict[int, _ConsentRow]: + """The newest grant per person in this guild. + + Ordered and folded rather than a window function, and ordered by + `granted_at` descending because that is the rule + `ConsentRepository.current` applies -- the console must show the + row the recorder acts on, not a different one. `id` descending is + added as a tiebreak the repository does not have: two grants at + the same instant are not a thing that happens, and a listing whose + order the planner decides is a listing that changes between two + refreshes for no reason. + """ + async with self._session_factory() as db: + rows = ( + await db.execute( + select( + Consent.discord_user_id, + Consent.granted_at, + Consent.revoked_at, + Consent.policy_version, + ) + .where(Consent.guild_id == guild_id) + .order_by( + Consent.discord_user_id, + Consent.granted_at.desc(), + Consent.id.desc(), + ) + ) + ).all() + + newest: dict[int, _ConsentRow] = {} + for discord_user_id, granted_at, revoked_at, policy_version in rows: + newest.setdefault( + discord_user_id, + _ConsentRow(granted_at, revoked_at, policy_version), + ) + return newest + + async def _latest_display_names(self, guild_id: int, people: Sequence[int]) -> dict[int, str]: + """The name each person last appeared under in this guild. + + `consent` stores no name, and a page of eighteen-digit numbers is + not a page an administrator can act on. The most recent + `session_participant` row is the closest thing the system holds -- + the name at the time of somebody's last recorded meeting -- and it + is scoped to this guild, because a display name is per-guild and + borrowing one from another guild would put a nickname from + somewhere else next to a decision about this one. + """ + async with self._session_factory() as db: + rows = ( + await db.execute( + select( + SessionParticipant.discord_user_id, + SessionParticipant.discord_display_name, + SessionRow.started_at, + ) + .join(SessionRow, SessionRow.id == SessionParticipant.session_id) + .where( + SessionRow.guild_id == guild_id, + SessionParticipant.discord_user_id.in_(people), + ) + .order_by( + SessionParticipant.discord_user_id, + SessionRow.started_at.desc(), + ) + ) + ).all() + + names: dict[int, str] = {} + for discord_user_id, display_name, _started_at in rows: + names.setdefault(discord_user_id, display_name) + return names + + async def _recordings_with_audio(self, guild_id: int, people: Sequence[int]) -> dict[int, int]: + """How many recordings of each person this guild still holds. + + `audio_deleted_at IS NULL` is the only claim that an object is + still in the store: the retention sweep erases the object first + and stamps the row second, so a stamped row is one whose audio is + already gone. Counting stamped rows would tell an administrator + that revoking consent leaves recordings behind which were erased + weeks ago. + """ + async with self._session_factory() as db: + rows = ( + await db.execute( + select(TranscriptionJob.discord_user_id, func.count()) + .join(SessionRow, SessionRow.id == TranscriptionJob.session_id) + .where( + SessionRow.guild_id == guild_id, + TranscriptionJob.discord_user_id.in_(people), + TranscriptionJob.audio_deleted_at.is_(None), + ) + .group_by(TranscriptionJob.discord_user_id) + ) + ).all() + return {discord_user_id: held for discord_user_id, held in rows} diff --git a/src/sturnus/console/app.py b/src/sturnus/console/app.py index 91b8e0d..6504d9b 100644 --- a/src/sturnus/console/app.py +++ b/src/sturnus/console/app.py @@ -36,6 +36,7 @@ ) from sturnus.console.ports import ( AdminDirectory, + ConsentDirectory, LinkDirectory, OAuthClient, QueueControl, @@ -45,6 +46,8 @@ ) from sturnus.console.routes_audio import AUDIO_DELIVERY from sturnus.console.routes_audio import register as register_audio +from sturnus.console.routes_consent import CONSENT_DIRECTORY +from sturnus.console.routes_consent import register as register_consent from sturnus.console.routes_queue import QUEUE_CONTROL from sturnus.console.routes_queue import register as register_queue from sturnus.console.session import ( @@ -268,6 +271,7 @@ def build_api( console_origin: str, audio: AudioDelivery, queue: QueueControl, + consents: ConsentDirectory, ) -> web.Application: """Builds the application, with every collaborator injected. @@ -293,6 +297,7 @@ def build_api( app[_CONSOLE_ORIGIN] = console_origin app[AUDIO_DELIVERY] = audio app[QUEUE_CONTROL] = queue + app[CONSENT_DIRECTORY] = consents app.add_routes( [ web.get("/healthz", healthz), @@ -306,5 +311,6 @@ def build_api( routes_read.register(app) register_audio(app) register_queue(app) + register_consent(app) routes_settings.register(app) return app diff --git a/src/sturnus/console/ports.py b/src/sturnus/console/ports.py index bdf9ba7..51d2db0 100644 --- a/src/sturnus/console/ports.py +++ b/src/sturnus/console/ports.py @@ -287,3 +287,82 @@ class QueueControl(Protocol): async def status_for(self, session_id: int, *, requested_by: int) -> QueueSnapshot | None: ... async def requeue(self, session_id: int, *, requested_by: int) -> RequeueOutcome | None: ... + + +@dataclass(frozen=True) +class ConsentHolder: + """One person's standing consent in one guild, as an administrator sees it. + + The newest `consent` row for that person in that guild, which is the + same selection `ConsentRepository.current` makes and the same one the + recorder acts on -- showing an administrator an older row would show + them a decision nothing enforces. + + `active` is not `revoked_at is None`. Consent also expires when the + guild's `policy_version` moves on, because a grant names the version + it was given under (`sturnus.domain.consent.is_consent_active`). Both + states are reported separately rather than folded into one flag: "they + withdrew it" and "we changed the policy under them" are different + facts about a person and lead to different conversations. + + `recordings_with_audio` is here for one purpose, and it is not a link + to a delete button. Withdrawing consent stops future recording; it + does not erase what is already stored. An administrator who is not + shown that number would reasonably assume it does. + """ + + discord_user_id: int + #: From `session_participant`, which is the only place a name is + #: stored -- `consent` has none. `None` for somebody who consented and + #: has not yet been in a recorded session, which is exactly the state + #: a well-run guild onboards people into. + display_name: str | None + policy_version: str + granted_at: datetime + revoked_at: datetime | None + active: bool + recordings_with_audio: int + + +@dataclass(frozen=True) +class RevocationOutcome: + """What a revocation did, or why it did nothing.""" + + revoked: bool + #: Why nothing happened, as one of a fixed set of reasons. `None` when + #: something did. + refusal: str | None + + +class ConsentDirectory(Protocol): + """Who has consented in a guild, and the power to withdraw it for them. + + `requested_by` is not optional and there is no method here without it, + for the reason `TrackDirectory` and `QueueControl` have none: the + authorisation rule lives inside the call rather than in a handler that + could forget to apply it. Both methods answer `None` for "no such + guild" and for "you do not administer it" alike. + + **What a revocation from here can and cannot do.** Consent is two + layers (Spec 3.1): a Discord role, checked synchronously on every + frame, and a stored record, checked on every frame through a five + second cache. This process holds no Discord token and never will + (Spec 13.2) -- it can decrypt every recording ever made, and a process + with that reach is not one to also give the ability to act as the bot. + So it writes the record and cannot touch the role. + + That is enough to stop the recording: the stored record is the layer + that exists precisely because the role can be bypassed, and a + revocation takes effect within the cache's five seconds, mid-session. + What it leaves behind is a role the person still holds, which is + visible in Discord and misleading if nobody says so. The console says + so, in the interface, next to the button. + """ + + async def holders( + self, guild_id: int, *, requested_by: int + ) -> Sequence[ConsentHolder] | None: ... + + async def revoke( + self, guild_id: int, discord_user_id: int, *, requested_by: int + ) -> RevocationOutcome | None: ... diff --git a/src/sturnus/console/routes_consent.py b/src/sturnus/console/routes_consent.py new file mode 100644 index 0000000..b477476 --- /dev/null +++ b/src/sturnus/console/routes_consent.py @@ -0,0 +1,241 @@ +"""Who has consented in a guild, and an administrator's power to end it. + +- `GET /api/guilds/{guild_id}/consents` +- `POST /api/guilds/{guild_id}/consents/{discord_user_id}/revoke` + +**Why this exists.** Until now the only way a consent could end was the +person ending it themselves with `/consent revoke`, or an administrator +bumping `policy_version`, which ends everybody's at once. Neither answers +the case this is for: somebody left the team, or asked in a channel rather +than in a slash command, or is no longer somebody this guild should be +recording. The alternative an administrator reaches for otherwise is +removing the Discord role by hand -- which stops the recording and leaves +`revoked_at` NULL, so `/consent status` still reports consent active and +re-adding the role silently resumes recording a person who never +re-consented. + +**What a revocation from here is, exactly.** It stamps `revoked_at` on the +stored consent record. It does not remove the Discord role, because this +process holds no Discord token and never will (Spec 13.2). That is enough +to stop the recording -- the stored record is checked on every frame +through a five second cache, and it is the layer that exists precisely +because the role can be bypassed by anyone with administrator permissions +in Discord. It is not enough to make Discord *look* right, and the console +says so next to the button rather than letting somebody infer it. + +**It is not a delete.** Nothing already recorded is touched. That is a +separate decision with a separate command (`/audio purge`), and folding +the two together would mean an administrator who wanted to stop recording +somebody tomorrow had also erased a meeting their team read last week. +Every row in the listing carries how many recordings of that person the +guild still holds, so the distinction is on the screen rather than in a +document nobody opens. + +**404, never 403.** A guild this person does not administer answers +exactly as a guild that does not exist. The list is a list of people who +consented to being recorded, together with when and under which policy; +a 403 would confirm that such a list exists here, to somebody just +established as having no business with it. + +**The audit line is the whole audit.** `consent.revoked_at` records that a +revocation happened and never who performed it. So +`Event.CONSOLE_CONSENT_REVOKED` is emitted at WARNING with `requested_by` +alongside `discord_user_id`, and it is the only place the pair is ever +written down. +""" + +from __future__ import annotations + +import logging + +from aiohttp import web + +from sturnus.console.ports import ConsentDirectory, ConsentHolder, RevocationOutcome +from sturnus.observability.events import Event, log_event + +# Every reference to `sturnus.console.app` below is imported inside a +# function rather than at module scope: `app` imports this module the +# ordinary way, so a module-level import back into it would close a cycle +# while `app` is still defining the very names wanted here. The same trade +# `routes_settings` makes, for the same reason. + +log = logging.getLogger(__name__) + +#: Where the collaborator is found. Its own key rather than a parameter to +#: `register`, so `build_api` stays a one-line edit -- several agents are +#: adding sections to that function and each extra line is a merge by hand. +CONSENT_DIRECTORY: web.AppKey[ConsentDirectory] = web.AppKey("consent_directory") + +_LIST_PATH = "/api/guilds/{guild_id}/consents" +_REVOKE_PATH = "/api/guilds/{guild_id}/consents/{discord_user_id}/revoke" + +#: The one refusal, for every reason there is to refuse. See the module +#: docstring on why "no such guild" and "not yours" are one answer. +_NO_SUCH_GUILD = "no such guild" + + +def register(app: web.Application) -> None: + """Adds the consent routes to an application that already has its directory.""" + from sturnus.console.app import require_session + + app.add_routes( + [ + web.get(_LIST_PATH, require_session(list_consents)), + web.post(_REVOKE_PATH, require_session(revoke_consent)), + ] + ) + + +async def list_consents(request: web.Request) -> web.Response: + """Everyone this guild holds a consent record for.""" + viewer = _caller(request) + guild_id = _guild_id(request) + if guild_id is None: + return _no_such_guild() + + holders = await request.app[CONSENT_DIRECTORY].holders(guild_id, requested_by=viewer) + if holders is None: + return _no_such_guild() + return web.json_response( + { + "guild_id": str(guild_id), + "consents": [_holder_json(holder) for holder in holders], + }, + # It names who agreed to be recorded in a particular guild, and it + # goes stale the moment anybody runs `/consent grant`. + headers={"Cache-Control": "private, no-store"}, + ) + + +async def revoke_consent(request: web.Request) -> web.Response: + """Withdraws one person's consent on their behalf. + + A revocation that changes nothing is **409, not 400**: the request is + well formed and the person is real, and what is wrong is the state + they are already in -- which is the distinction a client needs to + decide between "fix your request" and "somebody got there first". The + reason travels with it, because a button that fails without saying why + is a bug report waiting to be filed. + """ + viewer = _caller(request) + guild_id = _guild_id(request) + subject = _subject(request) + if guild_id is None or subject is None: + return _no_such_guild() + + outcome = await request.app[CONSENT_DIRECTORY].revoke(guild_id, subject, requested_by=viewer) + if outcome is None: + return _no_such_guild() + + if not outcome.revoked: + # INFO, not WARNING: two administrators reaching for the same name + # is this feature working. The interesting line is the one below. + log_event( + log, + logging.INFO, + Event.CONSOLE_CONSENT_REVOKE_REFUSED, + "Refused a consent revocation asked for from the console", + guild_id=guild_id, + discord_user_id=subject, + requested_by=viewer, + reason=outcome.refusal, + ) + return web.json_response(_outcome_json(outcome), status=409) + + # The audit line, and the only one there will ever be: + # `consent.revoked_at` records that a revocation happened and never + # who performed it. WARNING because this is a third party acting on + # somebody else's consent, which is a heavier act than any other the + # console offers. + log_event( + log, + logging.WARNING, + Event.CONSOLE_CONSENT_REVOKED, + "An administrator withdrew a person's recording consent from the console", + guild_id=guild_id, + discord_user_id=subject, + requested_by=viewer, + ) + return web.json_response(_outcome_json(outcome)) + + +# --------------------------------------------------------------------------- +# Reading the request +# --------------------------------------------------------------------------- + + +def _guild_id(request: web.Request) -> int | None: + """The guild from the path. `None` for a segment that is not a number. + + A path segment that is not a number names no guild, which is the same + answer as naming one that does not exist -- and the same answer as + naming one this person does not administer. All three are + `_no_such_guild`. + """ + try: + return int(request.match_info["guild_id"]) + except ValueError: + return None + + +def _subject(request: web.Request) -> int | None: + """The person whose consent is being withdrawn.""" + try: + return int(request.match_info["discord_user_id"]) + except ValueError: + return None + + +def _caller(request: web.Request) -> int: + """The Discord id of the person making this request. + + Only ever reached from behind `require_session`, which is what + guarantees there is one -- `current_user` raises rather than returning + `None` if that is ever untrue, so a route registered without the + wrapper fails loudly instead of quietly acting for somebody else. + """ + from sturnus.console.app import current_user + + return current_user(request).discord_user_id + + +# --------------------------------------------------------------------------- +# Writing the response +# --------------------------------------------------------------------------- + + +def _holder_json(holder: ConsentHolder) -> dict[str, object]: + return { + # A Discord snowflake exceeds JavaScript's safe integer range, + # where a JSON number silently loses its last digits and produces + # an id that looks right and names nobody. + "discord_user_id": str(holder.discord_user_id), + "display_name": holder.display_name, + "policy_version": holder.policy_version, + "granted_at": holder.granted_at.isoformat(), + "revoked_at": None if holder.revoked_at is None else holder.revoked_at.isoformat(), + # Sent as its own field rather than left to the client to derive + # from the two above it. Whether a grant is still in force also + # depends on the guild's current `policy_version`, and a console + # that worked it out for itself would be a second implementation + # of `sturnus.domain.consent.is_consent_active` -- one that would + # agree with the recorder right up until one of them changed. + "active": holder.active, + # What revoking will *not* do, as a number. An administrator not + # shown this would reasonably assume withdrawing consent erases + # what was recorded under it. + "recordings_with_audio": holder.recordings_with_audio, + } + + +def _outcome_json(outcome: RevocationOutcome) -> dict[str, object]: + return {"revoked": outcome.revoked, "refusal": outcome.refusal} + + +def _no_such_guild() -> web.Response: + """One refusal for every reason there is to refuse. + + "No such guild" and "you do not administer that guild" are + deliberately indistinguishable; see the module docstring. + """ + return web.json_response({"error": _NO_SUCH_GUILD}, status=404) diff --git a/src/sturnus/entrypoints/api.py b/src/sturnus/entrypoints/api.py index 92ce9b7..d9d70cf 100644 --- a/src/sturnus/entrypoints/api.py +++ b/src/sturnus/entrypoints/api.py @@ -39,6 +39,7 @@ from sturnus.config import StrictSettings from sturnus.console.adapters import ( + ConsoleConsentDirectory, ConsoleLinkDirectory, ConsoleQueueControl, ConsoleStateStore, @@ -187,20 +188,32 @@ async def _run() -> None: ) admins = AdminMemberStore(session_factory) + config = ConfigStore(session_factory) + + def now() -> datetime: + """The one clock this process reads. + + Named rather than repeated as a lambda at each call site: two + collaborators that each build their own would be two clocks a test + has to pin separately, and one of them would be missed. + """ + return datetime.now(UTC) + schema_ready = False app = build_api( oauth=oauth, states=ConsoleStateStore(session_factory), links=ConsoleLinkDirectory(session_factory), admins=admins, - config=ConfigStore(session_factory), + config=config, reads=ConsoleQueries(session_factory), sessions=SessionCookie(settings.session_secret.get_secret_value(), _SESSION_LIFETIME), - now=lambda: datetime.now(UTC), + now=now, schema_ready=lambda: schema_ready, console_origin=settings.console_origin, audio=audio, queue=ConsoleQueueControl(session_factory, admins), + consents=ConsoleConsentDirectory(session_factory, admins, config, now), ) runner = web.AppRunner(app) diff --git a/src/sturnus/observability/events.py b/src/sturnus/observability/events.py index 2362979..a495fba 100644 --- a/src/sturnus/observability/events.py +++ b/src/sturnus/observability/events.py @@ -164,6 +164,20 @@ class Event(StrEnum): #: about what a valid value is. CONSOLE_SETTING_REJECTED = "console.setting_rejected" + #: An administrator withdrew somebody else's consent to be recorded. + #: **WARNING, and for a stronger reason than `console.requeue_applied` + #: has:** this is a third party acting on a person's own consent, and + #: `consent.revoked_at` records only that it happened, never who did + #: it. This line is the entire answer to "who withdrew whose consent, + #: and when" -- `guild_id`, `discord_user_id` (whose consent) and + #: `requested_by` (who withdrew it). + CONSOLE_CONSENT_REVOKED = "console.consent_revoked" + #: A revocation that changed nothing, because there was no consent on + #: record or it had already been withdrawn. INFO: two administrators + #: reaching for the same name is this feature working. `reason` says + #: which of the two it was. + CONSOLE_CONSENT_REVOKE_REFUSED = "console.consent_revoke_refused" + # -- cross-cutting ------------------------------------------------------ PROCESS_STARTING = "process.starting" SHUTDOWN_BEGIN = "shutdown.begin" diff --git a/tests/console/conftest.py b/tests/console/conftest.py index c2266ee..09cb3c7 100644 --- a/tests/console/conftest.py +++ b/tests/console/conftest.py @@ -21,11 +21,14 @@ from sturnus.console.audio import AudioDelivery from sturnus.console.ports import ( AdminDirectory, + ConsentDirectory, + ConsentHolder, LinkDirectory, OAuthClient, QueueControl, QueueSnapshot, RequeueOutcome, + RevocationOutcome, SessionReads, SettingsStore, StateStore, @@ -389,6 +392,41 @@ async def requeue(self, session_id: int, *, requested_by: int) -> RequeueOutcome return self.outcome +class FakeConsents: + """A consent directory nobody administers, until a test says otherwise. + + Defaults to answering `None` everywhere, which is what the real + directory answers for "no such guild or not yours" -- so a test with no + interest in consent gets 404s rather than a fake that quietly + authorises everything. + """ + + def __init__( + self, + holders: Sequence[ConsentHolder] | None = None, + outcome: RevocationOutcome | None = None, + ) -> None: + self.holders_by_guild = None if holders is None else tuple(holders) + self.outcome = outcome + #: Every revocation this was asked for, as (guild, subject, actor). + #: Recorded rather than merely counted: "the administrator's own id + #: reached the write, not one taken from the URL" is the property + #: the authorisation tests assert on, and it cannot be seen in a + #: response body. + self.revoked: list[tuple[int, int, int]] = [] + self.listed: list[tuple[int, int]] = [] + + async def holders(self, guild_id: int, *, requested_by: int) -> Sequence[ConsentHolder] | None: + self.listed.append((guild_id, requested_by)) + return self.holders_by_guild + + async def revoke( + self, guild_id: int, discord_user_id: int, *, requested_by: int + ) -> RevocationOutcome | None: + self.revoked.append((guild_id, discord_user_id, requested_by)) + return self.outcome + + def build_test_api( *, oauth: OAuthClient | None = None, @@ -399,6 +437,7 @@ def build_test_api( config: SettingsStore | None = None, audio: AudioDelivery | None = None, queue: QueueControl | None = None, + consents: ConsentDirectory | None = None, sessions: SessionCookie | None = None, now: Callable[[], datetime] | None = None, schema_ready: bool = True, @@ -433,6 +472,7 @@ def build_test_api( source=FakeAudioSource(), ), queue=queue or FakeQueue(), + consents=consents or FakeConsents(), sessions=sessions or SessionCookie(SECRET, timedelta(hours=12)), now=now or now_at(), schema_ready=lambda: schema_ready, diff --git a/tests/console/test_consent_directory.py b/tests/console/test_consent_directory.py new file mode 100644 index 0000000..5f3091a --- /dev/null +++ b/tests/console/test_consent_directory.py @@ -0,0 +1,545 @@ +"""The consent directory, against the real database. + +Against PostgreSQL rather than a double because everything worth pinning +here is a property of the statements: which row of several is the current +one, that a guild's listing contains only that guild's people, and that +the count of recordings still held excludes the ones the retention sweep +erased. A double would agree with whatever it was written to agree with. + +The authorisation half is here too, and it is here rather than in the +route tests on purpose: the rule lives in the adapter, so this is where a +regression would appear. A handler cannot restore a check the adapter +dropped, because the handler was never given anything to check. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta + +import pytest +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from sturnus.console.adapters import ( + ALREADY_REVOKED, + NO_CONSENT_ON_RECORD, + ConsoleConsentDirectory, +) +from sturnus.domain import settings +from sturnus.infrastructure.db.config_store import ConfigStore +from sturnus.infrastructure.db.models import ( + Base, + Consent, + Session, + SessionParticipant, + TranscriptionJob, +) + +T0 = datetime(2026, 8, 21, 12, 0, 0, tzinfo=UTC) +GUILD, OTHER_GUILD = 4711, 9999 +ANNA, BEN, CARL = 100, 200, 300 +POLICY = "2026-01" + + +@pytest.fixture +async def factory(clean_database: str) -> async_sessionmaker[AsyncSession]: + engine = create_async_engine(clean_database) + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + return async_sessionmaker(engine, expire_on_commit=False) + + +class Admins: + """The mirrored administrator membership, per guild. + + Per guild and never a bare set, because "administers something" and + "administers *this*" are the two questions this whole feature turns + on, and a double that could not tell them apart would let a test + claiming the second prove only the first. + """ + + def __init__(self, by_guild: dict[int, set[int]] | None = None) -> None: + self.by_guild = by_guild if by_guild is not None else {GUILD: {ANNA}} + + async def is_admin_anywhere(self, discord_user_id: int) -> bool: + return any(discord_user_id in members for members in self.by_guild.values()) + + async def administered_guilds(self, discord_user_id: int) -> tuple[int, ...]: + return tuple( + sorted( + guild_id + for guild_id, members in self.by_guild.items() + if discord_user_id in members + ) + ) + + async def is_admin(self, guild_id: int, discord_user_id: int) -> bool: + return discord_user_id in self.by_guild.get(guild_id, set()) + + +def directory( + factory: async_sessionmaker[AsyncSession], + *, + admins: Admins | None = None, + now: datetime = T0, +) -> ConsoleConsentDirectory: + return ConsoleConsentDirectory( + factory, + admins or Admins(), + ConfigStore(factory), + lambda: now, + ) + + +async def set_policy( + factory: async_sessionmaker[AsyncSession], + version: str = POLICY, + guild_id: int = GUILD, +) -> None: + await ConfigStore(factory).set(guild_id, settings.POLICY_VERSION, version, T0) + + +async def grant( + factory: async_sessionmaker[AsyncSession], + discord_user_id: int, + *, + guild_id: int = GUILD, + granted_at: datetime = T0, + revoked_at: datetime | None = None, + policy_version: str = POLICY, +) -> None: + """One `consent` row, written straight to the table. + + Direct inserts rather than `ConsentRepository.record_grant`: what is + under test is which of several rows the directory reads back, and + going through the writer would make it a test of two things at once -- + including a writer that cannot produce a revoked row at all. + """ + async with factory() as db: + db.add( + Consent( + discord_user_id=discord_user_id, + guild_id=guild_id, + granted_at=granted_at, + revoked_at=revoked_at, + policy_version=policy_version, + source="button", + ) + ) + await db.commit() + + +async def a_recorded_session( + factory: async_sessionmaker[AsyncSession], + *, + guild_id: int = GUILD, + started_at: datetime = T0, + people: dict[int, str] | None = None, + audio_deleted: bool = False, +) -> int: + """A closed session with one participant and one recording per person.""" + async with factory() as db: + session = Session( + guild_id=guild_id, + channel_id=555, + channel_name="meeting", + started_at=started_at, + ended_at=started_at + timedelta(hours=1), + status="documented", + ) + db.add(session) + await db.flush() + for discord_user_id, name in (people or {ANNA: "anna"}).items(): + db.add( + SessionParticipant( + session_id=session.id, + discord_user_id=discord_user_id, + discord_display_name=name, + first_seen_at=started_at, + ) + ) + db.add( + TranscriptionJob( + session_id=session.id, + discord_user_id=discord_user_id, + s3_key=f"sessions/{session.id}/speakers/{discord_user_id}.enc", + encryption_key_id="k1", + wrapped_data_key=b"wrapped", + retention_until=started_at + timedelta(days=30), + status="done", + attempts=1, + audio_deleted_at=started_at if audio_deleted else None, + ) + ) + await db.commit() + return session.id + + +# --------------------------------------------------------------------------- +# Who may ask +# --------------------------------------------------------------------------- + + +async def test_an_administrator_of_the_guild_sees_who_consented( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert [holder.discord_user_id for holder in holders] == [BEN] + + +async def test_an_administrator_of_another_guild_is_nobody_here( + factory: async_sessionmaker[AsyncSession], +) -> None: + # The rule the whole console turns on: an administrator of one guild is + # not an administrator, they are an administrator *of that guild*. + await grant(factory, BEN) + admins = Admins({OTHER_GUILD: {CARL}}) + + assert await directory(factory, admins=admins).holders(GUILD, requested_by=CARL) is None + + +async def test_a_participant_who_administers_nothing_gets_no_listing( + factory: async_sessionmaker[AsyncSession], +) -> None: + await grant(factory, BEN) + assert await directory(factory).holders(GUILD, requested_by=BEN) is None + + +async def test_a_guild_nobody_administers_answers_the_same_as_one_that_is_not_yours( + factory: async_sessionmaker[AsyncSession], +) -> None: + # A guild the bot does not serve has no administrators, so "no such + # guild" needs no separate check -- and must not have one, because a + # distinct answer is an oracle for which guilds exist. + assert await directory(factory).holders(123456, requested_by=ANNA) is None + + +async def test_a_revocation_by_somebody_who_does_not_administer_the_guild_writes_nothing( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN) + + assert await directory(factory).revoke(GUILD, BEN, requested_by=CARL) is None + + # The refusal is not merely a return value: nothing may have been + # written on the way to it. + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + assert holders is not None + assert holders[0].revoked_at is None + + +# --------------------------------------------------------------------------- +# Which row is the current one +# --------------------------------------------------------------------------- + + +async def test_the_newest_grant_is_the_one_reported( + factory: async_sessionmaker[AsyncSession], +) -> None: + """Somebody who revoked and consented again reads as consenting. + + The same selection `ConsentRepository.current` makes, and it has to + be: an administrator shown an older row would be shown a decision + nothing enforces. + """ + await set_policy(factory) + await grant( + factory, BEN, granted_at=T0 - timedelta(days=30), revoked_at=T0 - timedelta(days=20) + ) + await grant(factory, BEN, granted_at=T0 - timedelta(days=1)) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert len(holders) == 1 + assert holders[0].revoked_at is None + assert holders[0].active is True + + +async def test_a_person_appears_once_however_often_they_consented( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + for day in range(4): + await grant(factory, BEN, granted_at=T0 - timedelta(days=day)) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert [holder.discord_user_id for holder in holders] == [BEN] + + +async def test_consent_in_another_guild_is_not_this_guild_s_business( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, CARL, guild_id=OTHER_GUILD) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders == () + + +# --------------------------------------------------------------------------- +# Whether it is still in force +# --------------------------------------------------------------------------- + + +async def test_a_grant_naming_a_superseded_policy_version_is_not_active( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The case an administrator would never guess from the columns. + + `revoked_at` is NULL and the consent is over, because the guild's + policy moved on and a grant names the version it was given under. + Deriving `active` in the browser from the two dates would have + reported this person as consenting. + """ + await set_policy(factory, "2026-02") + await grant(factory, BEN, policy_version="2026-01") + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].revoked_at is None + assert holders[0].active is False + + +async def test_a_guild_with_no_policy_version_has_no_active_consent( + factory: async_sessionmaker[AsyncSession], +) -> None: + # `policy_version` is a required key with no default, so an + # unconfigured guild is a real state -- and one where nothing may be + # recorded at all. + await grant(factory, BEN) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].active is False + + +async def test_a_withdrawn_grant_is_not_active( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN, revoked_at=T0) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].active is False + + +# --------------------------------------------------------------------------- +# What the row says about the person +# --------------------------------------------------------------------------- + + +async def test_a_name_comes_from_the_person_s_most_recent_meeting( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN) + await a_recorded_session(factory, started_at=T0 - timedelta(days=9), people={BEN: "old name"}) + await a_recorded_session(factory, started_at=T0 - timedelta(days=1), people={BEN: "ben"}) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].display_name == "ben" + + +async def test_somebody_who_has_never_been_recorded_has_no_name_to_show( + factory: async_sessionmaker[AsyncSession], +) -> None: + # The state a well-run guild onboards people into: they consented and + # have not yet been in a meeting. The console shows the id and says so. + await set_policy(factory) + await grant(factory, BEN) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].display_name is None + + +async def test_a_name_is_not_borrowed_from_another_guild( + factory: async_sessionmaker[AsyncSession], +) -> None: + # A display name is per-guild. Borrowing one would put a nickname from + # somewhere else next to a decision about this guild. + await set_policy(factory) + await grant(factory, BEN) + await a_recorded_session(factory, guild_id=OTHER_GUILD, people={BEN: "elsewhere"}) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].display_name is None + + +async def test_the_recordings_still_held_are_counted( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The number that says what revoking will *not* do. + + An administrator not shown it would reasonably assume withdrawing + consent erases what was recorded under it. + """ + await set_policy(factory) + await grant(factory, BEN) + await a_recorded_session(factory, started_at=T0 - timedelta(days=2), people={BEN: "ben"}) + await a_recorded_session(factory, started_at=T0 - timedelta(days=1), people={BEN: "ben"}) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].recordings_with_audio == 2 + + +async def test_a_recording_the_sweep_erased_is_not_counted_as_still_held( + factory: async_sessionmaker[AsyncSession], +) -> None: + # `audio_deleted_at` is the only claim that an object is gone. Counting + # stamped rows would tell an administrator that revoking leaves + # recordings behind which were erased weeks ago. + await set_policy(factory) + await grant(factory, BEN) + await a_recorded_session(factory, people={BEN: "ben"}, audio_deleted=True) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].recordings_with_audio == 0 + + +async def test_recordings_from_another_guild_are_not_counted( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN) + await a_recorded_session(factory, guild_id=OTHER_GUILD, people={BEN: "ben"}) + + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + + assert holders is not None + assert holders[0].recordings_with_audio == 0 + + +# --------------------------------------------------------------------------- +# Withdrawing it +# --------------------------------------------------------------------------- + + +async def test_a_revocation_stamps_the_newest_grant( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN) + revoked_at = T0 + timedelta(hours=3) + + outcome = await directory(factory, now=revoked_at).revoke(GUILD, BEN, requested_by=ANNA) + + assert outcome is not None + assert outcome.revoked is True + holders = await directory(factory).holders(GUILD, requested_by=ANNA) + assert holders is not None + assert holders[0].revoked_at == revoked_at + assert holders[0].active is False + + +async def test_revoking_twice_says_so_rather_than_pretending( + factory: async_sessionmaker[AsyncSession], +) -> None: + await set_policy(factory) + await grant(factory, BEN, revoked_at=T0) + + outcome = await directory(factory).revoke(GUILD, BEN, requested_by=ANNA) + + assert outcome is not None + assert outcome.revoked is False + assert outcome.refusal == ALREADY_REVOKED + + +async def test_revoking_a_consent_that_was_never_given_says_so( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`record_revocation` is silent about this, and silence is a lie here. + + An administrator told "revoked" for somebody who never consented would + believe a protection is in place that never was. + """ + await set_policy(factory) + + outcome = await directory(factory).revoke(GUILD, CARL, requested_by=ANNA) + + assert outcome is not None + assert outcome.revoked is False + assert outcome.refusal == NO_CONSENT_ON_RECORD + + +async def test_a_grant_under_a_superseded_policy_is_still_revoked( + factory: async_sessionmaker[AsyncSession], +) -> None: + """Inactive is not the same as withdrawn, and only one of them lasts. + + A grant is inactive under a newer policy version *because of a + setting*, and a setting can be set back. Stamping `revoked_at` is the + only thing that survives somebody restoring the old version. + """ + await set_policy(factory, "2026-02") + await grant(factory, BEN, policy_version="2026-01") + + outcome = await directory(factory).revoke(GUILD, BEN, requested_by=ANNA) + + assert outcome is not None + assert outcome.revoked is True + + +async def test_a_revocation_does_not_reach_the_same_person_in_another_guild( + factory: async_sessionmaker[AsyncSession], +) -> None: + # Consent is per guild, so a revocation is too. An administrator of one + # guild ending somebody's consent in another would be the widest + # possible reading of "administers a guild". + await set_policy(factory) + await set_policy(factory, guild_id=OTHER_GUILD) + await grant(factory, BEN) + await grant(factory, BEN, guild_id=OTHER_GUILD) + + await directory(factory).revoke(GUILD, BEN, requested_by=ANNA) + + elsewhere = await directory(factory, admins=Admins({OTHER_GUILD: {CARL}})).holders( + OTHER_GUILD, requested_by=CARL + ) + assert elsewhere is not None + assert elsewhere[0].revoked_at is None + assert elsewhere[0].active is True + + +async def test_a_revocation_leaves_the_earlier_grants_alone( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The history keeps grants; a revocation modifies the one it revokes. + + Consent rows are kept permanently, revoked ones included, because the + record *is* the evidence that consent was given (Spec 12.4). A + revocation that rewrote the history would destroy the thing the table + exists to hold. + """ + await set_policy(factory) + await grant(factory, BEN, granted_at=T0 - timedelta(days=30)) + await grant(factory, BEN, granted_at=T0 - timedelta(days=1)) + + await directory(factory).revoke(GUILD, BEN, requested_by=ANNA) + + async with factory() as db: + rows = (await db.execute(Consent.__table__.select())).all() + assert len(rows) == 2 + assert sum(1 for row in rows if row.revoked_at is not None) == 1 diff --git a/tests/console/test_consent_routes.py b/tests/console/test_consent_routes.py new file mode 100644 index 0000000..ead11cd --- /dev/null +++ b/tests/console/test_consent_routes.py @@ -0,0 +1,409 @@ +"""Who may withdraw somebody else's consent, and what they are told about it. + +The authorisation rule itself lives in `ConsoleConsentDirectory` and is +pinned against the real database in `test_consent_directory.py`. What is +pinned here is the other half, which no adapter test can see: that each +handler passes the *signed-in* person's id as `requested_by` rather than +anything taken from the URL, that a refusal is the same refusal for every +reason there is to refuse, and that the shapes going over the wire are the +ones the console reads. + +The one behaviour worth stating twice is the audit line. `consent` records +that a revocation happened and never who performed it, so +`console.consent_revoked` is the only place the pair "who withdrew whose +consent" is ever written down. A test that let it be dropped would let the +feature ship without the thing that makes it defensible. +""" + +from __future__ import annotations + +import logging +from datetime import UTC, datetime, timedelta + +import pytest +from aiohttp import web +from aiohttp.test_utils import TestClient + +from sturnus.console.adapters import ALREADY_REVOKED, NO_CONSENT_ON_RECORD +from sturnus.console.app import SESSION_COOKIE +from sturnus.console.ports import ConsentHolder, RevocationOutcome +from sturnus.console.session import SessionCookie, SignedSession +from sturnus.observability.events import Event +from tests.console.conftest import ( + ANNA, + BEN, + GUILD, + SECRET, + T0, + AiohttpClientFactory, + FakeConsents, + build_test_api, +) + +GRANTED = datetime(2026, 8, 1, 9, 0, 0, tzinfo=UTC) + + +def token(discord_user_id: int = ANNA) -> str: + return SessionCookie(SECRET, timedelta(hours=12)).issue(SignedSession(discord_user_id), now=T0) + + +async def signed_in( + aiohttp_client: AiohttpClientFactory, app: web.Application, as_user: int = ANNA +) -> TestClient[web.Request, web.Application]: + client = await aiohttp_client(app) + client.session.cookie_jar.update_cookies({SESSION_COOKIE: token(as_user)}) + return client + + +def list_url(guild_id: int | str = GUILD) -> str: + return f"/api/guilds/{guild_id}/consents" + + +def revoke_url(guild_id: int | str = GUILD, discord_user_id: int | str = BEN) -> str: + return f"/api/guilds/{guild_id}/consents/{discord_user_id}/revoke" + + +def holder(**over: object) -> ConsentHolder: + base: dict[str, object] = { + "discord_user_id": BEN, + "display_name": "ben", + "policy_version": "2026-01", + "granted_at": GRANTED, + "revoked_at": None, + "active": True, + "recordings_with_audio": 3, + } + base.update(over) + return ConsentHolder(**base) # type: ignore[arg-type] + + +# --------------------------------------------------------------------------- +# Who may ask +# --------------------------------------------------------------------------- + + +async def test_an_administrator_sees_who_has_consented_in_their_guild( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(holders=[holder()]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.get(list_url()) + + assert response.status == 200 + body = await response.json() + assert [entry["display_name"] for entry in body["consents"]] == ["ben"] + + +async def test_the_listing_asks_on_behalf_of_the_signed_in_person( + aiohttp_client: AiohttpClientFactory, +) -> None: + """The property no response body can show. + + The whole authorisation model is that the id reaching the directory is + the one out of the signed cookie. A handler that passed anything else + would look identical from outside, right up until the day the URL + carried a user id too. + """ + consents = FakeConsents(holders=[]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents), as_user=BEN) + + await client.get(list_url()) + + assert consents.listed == [(GUILD, BEN)] + + +async def test_somebody_who_does_not_administer_the_guild_is_told_it_does_not_exist( + aiohttp_client: AiohttpClientFactory, +) -> None: + # 404 and not 403: the list names who agreed to be recorded and when, + # and a 403 would confirm such a list exists here to somebody just + # established as having no business with it. + client = await signed_in(aiohttp_client, build_test_api(consents=FakeConsents())) + + response = await client.get(list_url()) + + assert response.status == 404 + assert (await response.json())["error"] == "no such guild" + + +async def test_a_guild_id_that_is_not_a_number_is_the_same_refusal( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in( + aiohttp_client, build_test_api(consents=FakeConsents(holders=[holder()])) + ) + + response = await client.get(list_url("not-a-guild")) + + assert response.status == 404 + + +async def test_signing_out_is_the_end_of_it( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await aiohttp_client(build_test_api(consents=FakeConsents(holders=[holder()]))) + + assert (await client.get(list_url())).status == 401 + assert (await client.post(revoke_url())).status == 401 + + +# --------------------------------------------------------------------------- +# What the listing says +# --------------------------------------------------------------------------- + + +async def test_every_discord_id_travels_as_a_string( + aiohttp_client: AiohttpClientFactory, +) -> None: + # A snowflake exceeds JavaScript's safe integer range, where a JSON + # number silently loses its last digits and produces an id that looks + # right and names nobody. + big = 1234567890123456789 + consents = FakeConsents(holders=[holder(discord_user_id=big)]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + body = await (await client.get(list_url())).json() + + assert body["guild_id"] == str(GUILD) + assert body["consents"][0]["discord_user_id"] == str(big) + + +async def test_a_consent_ended_by_a_policy_bump_is_reported_as_inactive( + aiohttp_client: AiohttpClientFactory, +) -> None: + """`active` is sent rather than left to the client to work out. + + A grant names the version it was given under, so a guild that moved + its `policy_version` on has consents with no `revoked_at` and no + force. A console deriving `active` from the two dates would report + this person as consenting -- a second implementation of + `is_consent_active` that agrees with the recorder until one of them + changes. + """ + consents = FakeConsents(holders=[holder(active=False)]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + entry = (await (await client.get(list_url())).json())["consents"][0] + + assert entry["revoked_at"] is None + assert entry["active"] is False + + +async def test_a_person_with_no_recorded_meeting_yet_has_no_name( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(holders=[holder(display_name=None)]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + entry = (await (await client.get(list_url())).json())["consents"][0] + + assert entry["display_name"] is None + + +async def test_the_listing_says_how_many_recordings_survive_a_revocation( + aiohttp_client: AiohttpClientFactory, +) -> None: + # The number that says what revoking will *not* do. Without it an + # administrator would reasonably assume withdrawing consent erases + # what was recorded under it. + consents = FakeConsents(holders=[holder(recordings_with_audio=7)]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + entry = (await (await client.get(list_url())).json())["consents"][0] + + assert entry["recordings_with_audio"] == 7 + + +async def test_a_guild_where_nobody_has_consented_is_an_empty_list( + aiohttp_client: AiohttpClientFactory, +) -> None: + # Empty and not 404: the difference between "nobody here has consented" + # and "this is not your guild" is the whole point of the page. + consents = FakeConsents(holders=[]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.get(list_url()) + + assert response.status == 200 + assert (await response.json())["consents"] == [] + + +async def test_the_listing_is_never_cached( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(holders=[holder()]) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.get(list_url()) + + assert response.headers["Cache-Control"] == "private, no-store" + + +# --------------------------------------------------------------------------- +# Withdrawing it +# --------------------------------------------------------------------------- + + +async def test_an_administrator_may_withdraw_somebody_else_s_consent( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(outcome=RevocationOutcome(revoked=True, refusal=None)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.post(revoke_url()) + + assert response.status == 200 + assert (await response.json()) == {"revoked": True, "refusal": None} + + +async def test_the_revocation_names_the_signed_in_administrator_as_the_actor( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(outcome=RevocationOutcome(revoked=True, refusal=None)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents), as_user=ANNA) + + await client.post(revoke_url(discord_user_id=BEN)) + + # The subject comes from the URL and the actor comes from the cookie, + # and mixing the two up is the one mistake this endpoint can make that + # nothing else would catch. + assert consents.revoked == [(GUILD, BEN, ANNA)] + + +async def test_a_second_revocation_is_a_conflict_rather_than_a_success( + aiohttp_client: AiohttpClientFactory, +) -> None: + """409, not 400 and not 200. + + The request is well formed and the person is real; what is wrong is + the state they are already in. Answering 200 would tell an + administrator they had just achieved something that had already + happened, and 400 would send them looking for a mistake in a request + that has none. + """ + consents = FakeConsents(outcome=RevocationOutcome(revoked=False, refusal=ALREADY_REVOKED)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.post(revoke_url()) + + assert response.status == 409 + assert (await response.json())["refusal"] == ALREADY_REVOKED + + +async def test_revoking_a_consent_nobody_ever_gave_says_which_of_the_two_it_was( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(outcome=RevocationOutcome(revoked=False, refusal=NO_CONSENT_ON_RECORD)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.post(revoke_url()) + + assert response.status == 409 + assert (await response.json())["refusal"] == NO_CONSENT_ON_RECORD + + +async def test_a_revocation_in_a_guild_this_person_does_not_administer_is_a_404( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in(aiohttp_client, build_test_api(consents=FakeConsents())) + + response = await client.post(revoke_url()) + + assert response.status == 404 + + +async def test_a_subject_that_is_not_a_number_never_reaches_the_directory( + aiohttp_client: AiohttpClientFactory, +) -> None: + consents = FakeConsents(outcome=RevocationOutcome(revoked=True, refusal=None)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + response = await client.post(revoke_url(discord_user_id="nobody")) + + assert response.status == 404 + assert consents.revoked == [] + + +# --------------------------------------------------------------------------- +# The audit line +# --------------------------------------------------------------------------- + + +def _events(caplog: pytest.LogCaptureFixture, event: Event) -> list[logging.LogRecord]: + """Every record carrying one event name. + + `log_event` puts the name and the fields in `extra` under + `sturnus_event` and `sturnus_fields` rather than spreading the fields + across the record, which is what lets `scrub_fields` rebuild them from + the registry on the way out. Reading them back the same way is how + these tests stay tests of the sanctioned call shape. + """ + return [r for r in caplog.records if getattr(r, "sturnus_event", None) == str(event)] + + +def _fields(record: logging.LogRecord) -> dict[str, object]: + fields = getattr(record, "sturnus_fields", None) + assert isinstance(fields, dict) + return fields + + +async def test_a_revocation_is_logged_with_who_did_it_to_whom( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + """The only record there will ever be that this happened. + + `consent.revoked_at` says a revocation occurred and never who + performed it, so this line is the entire answer to "who withdrew whose + consent". WARNING because a third party acting on somebody else's + consent is heavier than anything else the console offers. + """ + consents = FakeConsents(outcome=RevocationOutcome(revoked=True, refusal=None)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents), as_user=ANNA) + + with caplog.at_level(logging.INFO): + await client.post(revoke_url(discord_user_id=BEN)) + + lines = _events(caplog, Event.CONSOLE_CONSENT_REVOKED) + assert len(lines) == 1 + assert lines[0].levelno == logging.WARNING + fields = _fields(lines[0]) + assert fields["guild_id"] == GUILD + assert fields["discord_user_id"] == BEN + assert fields["requested_by"] == ANNA + + +async def test_a_refused_revocation_is_logged_as_the_feature_working( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + # INFO, not WARNING: two administrators reaching for the same name is + # not an incident. `reason` is what distinguishes it from a revocation + # that did something. + consents = FakeConsents(outcome=RevocationOutcome(revoked=False, refusal=ALREADY_REVOKED)) + client = await signed_in(aiohttp_client, build_test_api(consents=consents)) + + with caplog.at_level(logging.INFO): + await client.post(revoke_url()) + + lines = _events(caplog, Event.CONSOLE_CONSENT_REVOKE_REFUSED) + assert len(lines) == 1 + assert lines[0].levelno == logging.INFO + assert _fields(lines[0])["reason"] == ALREADY_REVOKED + + +async def test_a_refusal_nobody_was_entitled_to_ask_for_is_not_an_audit_line( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + # Nothing happened and nothing was authorised, so there is nothing to + # audit. A line here would let anybody with a session fill the log with + # names of their choosing. + client = await signed_in(aiohttp_client, build_test_api(consents=FakeConsents())) + + with caplog.at_level(logging.INFO): + await client.post(revoke_url()) + + assert not _events(caplog, Event.CONSOLE_CONSENT_REVOKED) + assert not _events(caplog, Event.CONSOLE_CONSENT_REVOKE_REFUSED) From 8b585589d1f07884a87bf019d7a50f6027177380 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 21:53:49 +0200 Subject: [PATCH 03/14] feat(console): a User Settings page for withdrawing a person's consent The Admin View's second entry, and the interface half of the endpoints in the previous commit. It lists everyone a guild holds a consent record for, what policy version they agreed to, when, and whether that consent is still in force. Three things the page says out loud, because all three are ways an administrator could otherwise reasonably believe something untrue: - **The Discord role is not removed.** The API holds no Discord token, by design, and cannot change anybody's roles. Recording still stops within about five seconds, mid-session, because the stored record is what is checked on every frame -- but the role stays, and somebody who wants it gone has to remove it in Discord. - **Nothing already recorded is deleted.** The count of recordings that still hold that person's audio is on their row, and the confirmation names it. Erasing them is `/audio purge`, a separate act. - **It is written to the audit log**: who withdrew whose consent, and when. So the control is a two-step confirmation naming all three, the same pattern Bot Settings uses for a value that invalidates consent. A row that is already withdrawn gets no control at all, only the sentence saying when it happened -- the page never offers an action it knows will fail. `active` is never re-derived in the browser. A grant names the policy version it was given under, so a guild that has moved on has consents with no `revoked_at` and no force; the badge trusts the server's verdict and `revoked_at` only distinguishes "withdrawn" from "the policy version they agreed to is no longer current". Those are different facts about a person and they lead to different conversations. Every decision -- ordering, wording, whether a revoke is offered, how a refusal reads -- is in `~/utils/consents` and tested there. The ordering ranks by what the reader can still do: consent in force, then lapsed by policy version, then already withdrawn, then named people before nameless ones. Names are compared lowercased rather than through `localeCompare`, which sorts by the runtime's locale and would make the server and the browser disagree -- a hydration reshuffle for a list somebody is reading. --- console/app/pages/admin/user-settings.vue | 489 +++++++++++++++++ console/app/utils/consents.ts | 616 ++++++++++++++++++++++ console/app/utils/navigation.ts | 10 + console/test/consents.spec.ts | 565 ++++++++++++++++++++ console/test/navigation.spec.ts | 11 + 5 files changed, 1691 insertions(+) create mode 100644 console/app/pages/admin/user-settings.vue create mode 100644 console/app/utils/consents.ts create mode 100644 console/test/consents.spec.ts diff --git a/console/app/pages/admin/user-settings.vue b/console/app/pages/admin/user-settings.vue new file mode 100644 index 0000000..8304e4f --- /dev/null +++ b/console/app/pages/admin/user-settings.vue @@ -0,0 +1,489 @@ + + + diff --git a/console/app/utils/consents.ts b/console/app/utils/consents.ts new file mode 100644 index 0000000..f1871c9 --- /dev/null +++ b/console/app/utils/consents.ts @@ -0,0 +1,616 @@ +/** + * Who has consented to being recorded in a guild, and what withdrawing one + * of those consents actually does. + * + * A module rather than expressions in the page, for the same reason + * `~/utils/settings` is one: every function here is a *decision* -- which + * row comes first, whether a revoke is offered at all, what a refusal reads + * like as a sentence, how a moment is written -- and a decision embedded in + * a template can only be tested by rendering one. + * + * Three facts govern the wording of everything below, and none of them are + * softened anywhere in this file: + * + * - **Withdrawing here removes the stored consent record, not the Discord + * role.** The API process holds no Discord token by design, so it cannot + * take a role off anybody. Recording still stops, within about five + * seconds and in the middle of a running session, because the stored + * record -- not the role -- is what the packet filter checks on every + * frame. But an administrator who believes the role is gone will be wrong + * about what the member can see in Discord, and about what happens the + * next time the policy version changes. + * - **Withdrawing deletes nothing that was already recorded.** Consent + * governs what is captured from now on. The audio already on disk stays + * until somebody erases it deliberately with `/audio purge`, and a page + * that let "withdrawn" read as "removed" would be answering a data + * subject's erasure request with a lie. + * - **It is logged.** Who withdrew whose consent, and when, goes to the + * audit log. Saying so up front is fairer than letting somebody find out + * afterwards. + * + * The `active` flag is authoritative and is never re-derived here. A + * consent also stops counting when the guild's `policy_version` moves past + * the one it names, so a record can carry no revocation date and still be + * inactive. Those are two different facts about a person -- one of them + * they chose -- and this module keeps them apart everywhere. + */ +import { formatCount, formatMoment } from '~/utils/format' + +/** One person's consent, as the API describes it. */ +export interface ConsentRow { + /** A string, always. A Discord snowflake exceeds JavaScript's safe + * integer range, where a JSON number silently drops its last digits and + * produces an id that looks right and names somebody else. */ + discord_user_id: string + /** Null when Sturnus has never seen a name for them -- they consented but + * have not been in a recorded session since, so there was no occasion to + * learn one. */ + display_name: string | null + /** The guild policy version the consent names, not necessarily the one in + * force now. The difference between the two is what `active` reports. */ + policy_version: string | null + granted_at: string | null + /** Set only when somebody withdrew it. A lapsed consent has none. */ + revoked_at: string | null + /** The API's verdict, taken as given. See the module comment. */ + active: boolean + /** How many recordings the guild still holds that contain this person's + * audio. Withdrawing changes none of them. */ + recordings_with_audio: number +} + +/* -------------------------------------------------------------------- */ +/* Reading what the API sent */ +/* -------------------------------------------------------------------- */ + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** `null` stays `null`; anything else becomes the string it prints as. Ids + * and versions are strings on the wire, and a number that arrived instead + * has already lost whatever precision it was going to lose. */ +function asText(value: unknown): string | null { + if (value === null || value === undefined) return null + const text = typeof value === 'string' ? value : String(value) + return text.trim() === '' ? null : text +} + +/** A count that can be printed. Anything absent, negative or not a number + * is a defect upstream, and rendering it as "-3 recordings" would put the + * defect in front of the reader as though it were a fact about them. Zero + * is the honest floor: it says "none held", which is exactly as far as + * this console can vouch for. */ +function asCount(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return 0 + return Math.round(value) +} + +/** + * The consent rows in a payload. + * + * Accepts the `{guild_id, consents: [...]}` envelope the endpoint sends and + * a bare list, because "the consents of a guild" describes both equally and + * accepting both removes a class of failure whose only symptom is a page + * that renders empty with no error anywhere. An entry naming no user is + * dropped: there is nothing a row without an id could be revoked for. + */ +export function parseConsents(payload: unknown): ConsentRow[] { + let container: unknown = payload + if (isRecord(container) && 'consents' in container) container = container.consents + if (!Array.isArray(container)) return [] + + const rows: ConsentRow[] = [] + for (const entry of container) { + if (!isRecord(entry)) continue + const id = asText(entry.discord_user_id) ?? asText(entry.user_id) + if (!id) continue + rows.push({ + discord_user_id: id, + display_name: asText(entry.display_name), + policy_version: asText(entry.policy_version), + granted_at: asText(entry.granted_at), + revoked_at: asText(entry.revoked_at), + // `=== true` rather than truthiness: a missing flag, or the string + // "false" a careless serialiser produces, must read as "not active". + // Erring the other way would tell an administrator somebody is being + // recorded when the bot has already stopped. + active: entry.active === true, + recordings_with_audio: asCount(entry.recordings_with_audio), + }) + } + return rows +} + +/* -------------------------------------------------------------------- */ +/* Naming a person */ +/* -------------------------------------------------------------------- */ + +/** What to call somebody on screen. The whole id when there is no name -- + * never a shortened one, since snowflakes minted in the same era share + * their leading digits and a truncated id identifies a group rather than a + * person. */ +export function personLabel(row: ConsentRow): string { + return row.display_name ?? `Discord user ${row.discord_user_id}` +} + +/** + * The line under a nameless row, or `null` when there is a name. + * + * A bare snowflake where every other row has a name reads as a fault in the + * console. It is not one: consent is given in a Discord command, and a + * display name is only learned when somebody turns up in a session that was + * recorded. Saying so is the difference between "this row is broken" and + * "this person has consented and has not been in a meeting yet". + */ +export function identityNote(row: ConsentRow): string | null { + if (row.display_name) return null + return ( + 'No display name on record: this is their Discord user id. They gave consent but have not ' + + 'been in a recorded session since, so Sturnus has never had a name to learn.' + ) +} + +/* -------------------------------------------------------------------- */ +/* What state a consent is in */ +/* -------------------------------------------------------------------- */ + +/** Three states, three colours. "Withdrawn" and "the policy version they + * agreed to is no longer the current one" are different facts about a + * person -- one of them their own decision -- and rendering both as a + * single grey "inactive" would hide which. */ +export type ConsentTone = 'active' | 'superseded' | 'withdrawn' + +export interface ConsentBadge { + tone: ConsentTone + label: string + /** The long form, said in full on the row rather than hidden in a + * tooltip: the difference between the two inactive states decides what + * an administrator should do next, and nobody hovers to find that out. */ + detail: string +} + +/** + * What this row's state is, in a badge and a sentence. + * + * `active` decides, because the API says it decides. The revocation date + * only distinguishes the two ways of being inactive. A row that somehow + * arrives active *and* carrying a revocation date is contradictory data; + * the badge trusts `active` as instructed, while `revocability` below still + * withholds the button, because the API would refuse that write whatever + * this console believes. + */ +export function consentBadge(row: ConsentRow): ConsentBadge { + if (row.active) { + return { + tone: 'active', + label: 'Consent in force', + detail: + 'Sturnus records this person while a session runs. The stored record is what the audio ' + + 'filter checks on every frame — the Discord role only decides who may be asked, never ' + + 'who is captured.', + } + } + if (row.revoked_at) { + return { + tone: 'withdrawn', + label: 'Withdrawn', + detail: + `Withdrawn on ${formatMoment(row.revoked_at)}. They are not recorded, and will not be ` + + 'again until they run /consent grant themselves.', + } + } + return { + tone: 'superseded', + label: 'Policy version superseded', + detail: + `Nobody withdrew this consent — it lapsed. They agreed under policy version ` + + `${row.policy_version ?? 'an unrecorded value'}, this server's policy_version has moved on ` + + 'since, and a consent naming an old version stops counting. They are not being recorded ' + + 'until they run /consent grant again under the current version.', + } +} + +/* -------------------------------------------------------------------- */ +/* The facts on a row */ +/* -------------------------------------------------------------------- */ + +/** + * Which policy version this consent names. + * + * Shown for every row, not only the superseded ones. It is the field that + * explains a superseded row, and reading it on the active rows is how + * somebody works out what bumping `policy_version` would cost before they + * do it on the Bot Settings page. + */ +export function policyLine(row: ConsentRow): string { + if (!row.policy_version) return 'The policy version they agreed under was not recorded.' + return `Agreed under policy version ${row.policy_version}.` +} + +/** When they gave it. `formatMoment` is borrowed from `~/utils/format` + * rather than reimplemented: the console has one way of printing a moment + * -- UTC, saying so, because the server render cannot know the reader's + * zone and a second rendering would disagree with the first. */ +export function grantedLine(row: ConsentRow): string { + if (!row.granted_at) return 'When they granted it was not recorded.' + return `Granted ${formatMoment(row.granted_at)}.` +} + +/** When it was withdrawn, or `null` for a consent nobody has withdrawn. A + * lapsed consent must not borrow this line: it would read as a decision + * the person made, which is precisely what it is not. */ +export function withdrawnLine(row: ConsentRow): string | null { + if (!row.revoked_at) return null + return `Withdrawn ${formatMoment(row.revoked_at)}.` +} + +/** + * How much of this person is still on disk. + * + * On the row as well as in the confirmation, because it is the number that + * answers the question an administrator is usually really asking. Somebody + * who came here to erase a person's audio has come to the wrong page, and + * this is where they find that out. + */ +export function recordingsLine(row: ConsentRow): string { + const held = row.recordings_with_audio + if (held === 0) return 'Sturnus holds no recordings containing their audio.' + const noun = held === 1 ? 'recording' : 'recordings' + return `Sturnus still holds ${formatCount(held)} ${noun} containing their audio.` +} + +/* -------------------------------------------------------------------- */ +/* Whether a revoke may be offered */ +/* -------------------------------------------------------------------- */ + +export type Revocability = { revocable: true } | { revocable: false, reason: string } + +/** + * Whether to offer the revoke control at all. + * + * A consent already withdrawn answers 409 `already_revoked`, every time. + * Rendering the button and the refusal afterwards would be an interface + * inviting an action whose outcome it already knows -- the same rule the + * settings page follows for a required key. + * + * A *lapsed* consent is still offered, deliberately. The record is still + * there; withdrawing it removes it rather than waiting for it, which is the + * difference between "not counting today" and "gone". It matters the moment + * somebody rolls `policy_version` back to a previous value, which would + * otherwise bring every superseded consent quietly back to life. + */ +export function revocability(row: ConsentRow): Revocability { + if (row.revoked_at) { + return { + revocable: false, + reason: + `Already withdrawn on ${formatMoment(row.revoked_at)}. There is nothing left to withdraw.`, + } + } + return { revocable: true } +} + +/* -------------------------------------------------------------------- */ +/* The three things that must be said before a revoke */ +/* -------------------------------------------------------------------- */ + +/** + * The Discord role is not touched. Stated in the confirmation itself, never + * only in a footnote at the bottom of the page: an administrator who + * believes this removed the role will not go and remove it, and the member + * keeps a role that says something untrue about them. + */ +export const ROLE_STAYS_NOTE = + 'This withdraws the consent record Sturnus stores. It does not remove the Discord consent role ' + + 'from this person — the API holds no Discord token, by design, and cannot change anybody’s ' + + 'roles. Recording of them still stops within about five seconds, in the middle of a running ' + + 'session if there is one, because the stored record is what is checked on every frame. If the ' + + 'role should go too, remove it in Discord.' + +/** + * Nothing already recorded is deleted, and the count says how much that is + * for this person specifically. A general sentence about retention is easy + * to read past; "and the four recordings that already contain their audio + * stay" is not. + */ +export function recordingsKeptNote(row: ConsentRow): string { + const held = row.recordings_with_audio + const who = personLabel(row) + if (held === 0) { + return ( + `Nothing already recorded is deleted. Sturnus holds no recordings containing ${who}’s audio ` + + 'right now, so there is nothing here to erase — and erasing recordings is a separate act ' + + 'either way: /audio purge in Discord.' + ) + } + const noun = held === 1 ? 'recording that already contains' : 'recordings that already contain' + return ( + `Nothing already recorded is deleted. The ${formatCount(held)} ${noun} ${who}’s audio stay ` + + 'exactly where they are. Erasing them is a separate act: /audio purge in Discord.' + ) +} + +/** Said before the act rather than discovered after it. */ +export const AUDIT_LOG_NOTE = + 'This is written to the audit log: your Discord account, whose consent you withdrew, and when.' + +export interface RevokeConfirmation { + title: string + /** + * Three sentences, kept as three. A single paragraph carrying all of + * them is a paragraph that gets skimmed, and it would be skimmed exactly + * where the reader most needs to notice that the role and the recordings + * are not part of this. + */ + consequences: readonly string[] + confirmLabel: string +} + +/** + * The confirmation shown before a consent is withdrawn. + * + * Always shown -- there is no unattended path to this write. Withdrawing + * somebody's consent stops them being recorded in a meeting that may be + * running right now, is done on their behalf without them being asked, and + * cannot be undone from this console: only the person themselves can grant + * consent again, with `/consent grant` in Discord. + */ +export function revokeConfirmation(row: ConsentRow): RevokeConfirmation { + return { + title: `Withdraw ${personLabel(row)}’s consent?`, + consequences: [ROLE_STAYS_NOTE, recordingsKeptNote(row), AUDIT_LOG_NOTE], + confirmLabel: 'Yes, withdraw this consent', + } +} + +/* -------------------------------------------------------------------- */ +/* What the API answered */ +/* -------------------------------------------------------------------- */ + +export interface RevokeResult { + revoked: boolean + /** A machine name for the refusal (`already_revoked`, + * `no_consent_on_record`), or `null` when the write succeeded. */ + refusal: string | null +} + +/** The revoke endpoint's answer. `revoked` is read strictly: a body this + * console cannot make sense of must never be reported as a successful + * withdrawal, because the only person who would find out otherwise is the + * one still being recorded. */ +export function parseRevokeResult(payload: unknown): RevokeResult { + if (!isRecord(payload)) return { revoked: false, refusal: null } + return { revoked: payload.revoked === true, refusal: asText(payload.refusal) } +} + +/** + * A refusal, as a sentence. + * + * Both named refusals mean the same thing to the person reading the screen: + * the row they clicked is out of date, and nothing they did just now + * changed anything. They still get different words, because "somebody else + * already did this" and "Sturnus has no record of this person consenting at + * all" send an administrator to different places. + * + * The `null` case is not merely defensive. `useApi` strips the body off + * every failed request on purpose -- `ApiError` keeps the status and the + * path and nothing else, so an in-cluster hostname can never reach the + * hydration payload -- which means a 409 arrives here as a status with no + * refusal code attached. That sentence therefore has to be true of both. + */ +export function describeRefusal(refusal: string | null): string { + switch (refusal) { + case 'already_revoked': + return ( + 'This consent had already been withdrawn — by somebody else, or in another tab. Nothing ' + + 'changed just now, and nothing needed to.' + ) + case 'no_consent_on_record': + return ( + 'Sturnus holds no consent record for this person at all, so there was nothing to ' + + 'withdraw. They are not being recorded.' + ) + default: + return ( + 'Sturnus refused: there is no consent of theirs left to withdraw — it had been withdrawn ' + + 'already, or there was never a record of it. Either way they are not being recorded, ' + + 'and this row was out of date.' + ) + } +} + +export interface RevokeOutcome { + tone: 'done' | 'refused' + headline: string + detail: string +} + +/** + * The panel shown after the write, and never merely "Done". + * + * It repeats the two limits afterwards as well as before, because the + * moment somebody is most likely to believe more happened than did is the + * moment they have just watched a row change state. + */ +export function revokeOutcome(row: ConsentRow, result: RevokeResult): RevokeOutcome { + if (!result.revoked) { + return { + tone: 'refused', + headline: 'Nothing was withdrawn.', + detail: describeRefusal(result.refusal), + } + } + const held = row.recordings_with_audio + const recordings + = held === 0 + ? 'no recordings of them are held' + : `the ${formatCount(held)} ${held === 1 ? 'recording' : 'recordings'} already containing ` + + 'their audio are untouched' + return { + tone: 'done', + headline: `${personLabel(row)}’s consent is withdrawn.`, + detail: + 'Recording of them stops within about five seconds, mid-session if a meeting is running. ' + + `Their Discord consent role is unchanged, ${recordings}, and this is in the audit log. ` + + 'Only they can grant consent again, with /consent grant.', + } +} + +/* -------------------------------------------------------------------- */ +/* When the API says no */ +/* -------------------------------------------------------------------- */ + +/** `ApiError` names it `status`; a raw `$fetch` failure may name it + * `statusCode`; a request that never got a response has neither, and null + * says so rather than standing in a number that would read as an answer. */ +function statusOf(error: unknown): number | null { + if (!isRecord(error)) return null + for (const candidate of [error.status, error.statusCode]) { + if (typeof candidate === 'number' && Number.isFinite(candidate)) { + // `ApiError` uses 0 for "never reached the API", which is + // deliberately distinguishable from every real status. + return candidate === 0 ? null : candidate + } + } + return null +} + +/** + * Whether the failure means "the row you clicked is out of date". + * + * Only a 409 does, and it always does: the endpoint refuses exactly when + * there is no consent left to withdraw. That makes reloading the list the + * correct response rather than a hopeful one, which is why this is a + * decision with a name instead of a `=== 409` in the page. + */ +export function isStaleRow(error: unknown): boolean { + return statusOf(error) === 409 +} + +/** + * A failed request, in a sentence somebody can act on. + * + * Built from the status alone. `useApi` throws `ApiError`, which carries no + * body by design, so there is no server text to prefer here even if the API + * sent some -- and every sentence below therefore has to stand on its own. + */ +export function describeConsentError(error: unknown): string { + switch (statusOf(error)) { + case 401: + return 'Your session has ended. Sign in again, then retry — nothing was withdrawn.' + case 403: + return ( + 'You do not administer this server. Administrators are the members holding the role named ' + + 'by that guild’s `admin_role_id`.' + ) + case 404: + // The API answers 404 both for a guild that does not exist and for + // one the caller does not administer, on purpose: it will not confirm + // the existence of a server to somebody with no business there. So + // this sentence must cover both without guessing which. + return ( + 'Sturnus does not know this server, or you no longer administer it — it answers the same ' + + 'way to both. Reload the page; the list of servers is rebuilt from Discord.' + ) + case 409: + return describeRefusal(null) + case null: + return 'Could not reach the API. Nothing was changed; check the connection and retry.' + default: + return `Sturnus answered ${statusOf(error)}. Nothing is known about why, and nothing was changed.` + } +} + +/* -------------------------------------------------------------------- */ +/* The order the people are listed in */ +/* -------------------------------------------------------------------- */ + +/** + * Rank by what the reader can still do about a row. + * + * 0 -- consent in force: the only rows where withdrawing changes what + * happens in a meeting, including one running right now. + * 1 -- lapsed with the policy version: withdrawing is still offered and + * still removes something, but nobody is being recorded under it. + * 2 -- already withdrawn: history. There is no control on these at all, so + * they belong below everything that has one. + */ +function rank(row: ConsentRow): number { + if (row.active) return 0 + return row.revoked_at ? 2 : 1 +} + +/** + * Case-insensitive, and deliberately not `localeCompare`. + * + * `localeCompare` sorts by the runtime's locale, and this list is built + * once on the server and again in the browser. Two ICU versions disagreeing + * about where "Ö" goes is a hydration mismatch: Vue reports it in the + * console, and the reader sees the rows reshuffle themselves a moment after + * the page appears. The same trade `formatCount` makes for thousands + * separators, for the same reason. + */ +function compareNames(a: string, b: string): number { + const left = a.toLowerCase() + const right = b.toLowerCase() + if (left === right) return 0 + return left < right ? -1 : 1 +} + +/** + * Snowflakes in numeric order without ever becoming numbers. + * + * Shorter is smaller, and equal lengths compare digit by digit. Comparing + * them as plain strings would put "1000..." before "999...", and comparing + * them as numbers would round every id past the safe integer range into a + * different id. + */ +function compareIds(a: string, b: string): number { + if (a.length !== b.length) return a.length - b.length + if (a === b) return 0 + return a < b ? -1 : 1 +} + +/** + * The order the rows are listed in. + * + * Actionable first (see `rank`), then people with a name before people + * without one, then by name, then by id. + * + * Names before ids because of how the page is actually used: somebody + * arrives here having been asked about a *person*, and scans for a name. A + * snowflake is not something anybody scans for -- it is something they + * search the page for with the browser's own find -- so the nameless rows + * lose nothing by sitting at the bottom of their rank, and every named row + * above them gains. Within the nameless run the ids are in numeric order, + * which is roughly the order the accounts were created, so at least the run + * is stable and not arbitrary. + * + * Every comparison ends at the id, which is unique, so the order is total: + * two people sharing a display name never swap places between renders. + */ +export function orderConsents(rows: readonly ConsentRow[]): ConsentRow[] { + return [...rows].sort((a, b) => { + const byRank = rank(a) - rank(b) + if (byRank !== 0) return byRank + if (a.display_name && b.display_name) { + const byName = compareNames(a.display_name, b.display_name) + if (byName !== 0) return byName + } else if (a.display_name || b.display_name) { + return a.display_name ? -1 : 1 + } + return compareIds(a.discord_user_id, b.discord_user_id) + }) +} + +/** + * How many of these people are actually being recorded. + * + * The headline figure for the page: a list of forty rows where six are in + * force says something a bare row count does not, and "who can Sturnus + * record here right now" is the question an administrator came to answer. + */ +export function activeCount(rows: readonly ConsentRow[]): number { + return rows.filter((row) => row.active).length +} diff --git a/console/app/utils/navigation.ts b/console/app/utils/navigation.ts index 57f2acb..ddb22fa 100644 --- a/console/app/utils/navigation.ts +++ b/console/app/utils/navigation.ts @@ -75,6 +75,16 @@ export const ADMIN_VIEW: NavSection = { icon: 'M12 8a4 4 0 1 0 0 8 4 4 0 0 0 0-8Zm9.4 4a7.4 7.4 0 0 0-.1-1.2l2-1.6-2-3.4-2.4 1a7.6 7.6 0 0 0-2-1.2L16.5 3h-4l-.4 2.6c-.7.3-1.4.7-2 1.2l-2.4-1-2 3.4 2 1.6a7.4 7.4 0 0 0 0 2.4l-2 1.6 2 3.4 2.4-1c.6.5 1.3.9 2 1.2l.4 2.6h4l.4-2.6c.7-.3 1.4-.7 2-1.2l2.4 1 2-3.4-2-1.6c.1-.4.1-.8.1-1.2Z', adminOnly: true, }, + { + to: '/admin/user-settings', + label: 'User Settings', + // Two people rather than one: this page is about the members of a + // server, never about the account of whoever is reading it. A single + // silhouette next to "User Settings" would read as "your profile", + // which is the one thing this page is not. + icon: 'M16 11a3 3 0 1 0 0-6 3 3 0 0 0 0 6Zm-8 0a3 3 0 1 0 0-6 3 3 0 0 0 0 6Zm0 2c-2.33 0-7 1.17-7 3.5V19h14v-2.5c0-2.33-4.67-3.5-7-3.5Zm8 0c-.29 0-.62.02-.97.05 1.16.84 1.97 1.97 1.97 3.45V19h6v-2.5c0-2.33-4.67-3.5-7-3.5Z', + adminOnly: true, + }, ], } diff --git a/console/test/consents.spec.ts b/console/test/consents.spec.ts new file mode 100644 index 0000000..9d6723c --- /dev/null +++ b/console/test/consents.spec.ts @@ -0,0 +1,565 @@ +/** + * What a consent record means, and what withdrawing one actually does. + * + * All of it lives in `~/utils/consents` rather than in the page, because + * every one of these is a decision -- which row comes first, whether a + * revoke is offered at all, how a refusal reads as a sentence -- and a + * decision embedded in a template can only be tested by rendering one. + * + * The wording is asserted here on purpose, and more heavily than anywhere + * else in this console. This page performs an act on somebody else's behalf + * whose two limits are invisible: the Discord role stays, and the + * recordings stay. A test that only checked a boolean would let either of + * those quietly drop out of the confirmation, and the first person to + * notice would be a member who was told their audio had been erased. + */ +import { describe, expect, it } from 'vitest' + +import { + AUDIT_LOG_NOTE, + ROLE_STAYS_NOTE, + activeCount, + consentBadge, + describeConsentError, + describeRefusal, + grantedLine, + identityNote, + isStaleRow, + orderConsents, + parseConsents, + parseRevokeResult, + personLabel, + policyLine, + recordingsKeptNote, + recordingsLine, + revocability, + revokeConfirmation, + revokeOutcome, + withdrawnLine, + type ConsentRow, +} from '../app/utils/consents' + +/** A consent with everything harmless, so each test states only the one + * property it is actually about. */ +function row(overrides: Partial & { discord_user_id: string }): ConsentRow { + return { + display_name: null, + policy_version: '2026-01', + granted_at: '2026-08-21T12:00:00+00:00', + revoked_at: null, + active: true, + recordings_with_audio: 0, + ...overrides, + } +} + +/** What `ApiError` looks like to the functions that read a failure. */ +function failure(status: number) { + return { status, path: '/guilds/4711/consents' } +} + +describe('reading the consents payload', () => { + it('reads the rows out of the guild envelope', () => { + const parsed = parseConsents({ + guild_id: '4711', + consents: [ + { + discord_user_id: '100', + display_name: 'Anna', + policy_version: '2026-01', + granted_at: '2026-08-21T12:00:00+00:00', + revoked_at: null, + active: true, + recordings_with_audio: 3, + }, + ], + }) + expect(parsed).toHaveLength(1) + expect(parsed[0]!.display_name).toBe('Anna') + expect(parsed[0]!.recordings_with_audio).toBe(3) + }) + + it('reads the rows out of a bare list', () => { + expect(parseConsents([{ discord_user_id: '100' }]).map((r) => r.discord_user_id)).toEqual(['100']) + }) + + it('keeps an id as a string even if it arrived as a number', () => { + // A snowflake past the safe integer range has already lost its last + // digits before this function sees it. Stringifying does not undo that; + // it keeps the revoke URL well-formed so the damage surfaces as a 409 + // rather than as somebody else's consent being withdrawn. + expect(parseConsents([{ discord_user_id: 100 }])[0]!.discord_user_id).toBe('100') + }) + + it('treats a missing name as no name rather than as an empty one', () => { + expect(parseConsents([{ discord_user_id: '100' }])[0]!.display_name).toBeNull() + expect(parseConsents([{ discord_user_id: '100', display_name: ' ' }])[0]!.display_name).toBeNull() + }) + + it('treats a missing active flag as not being recorded', () => { + // Erring the other way would tell an administrator somebody is being + // recorded when the bot has already stopped. + expect(parseConsents([{ discord_user_id: '100' }])[0]!.active).toBe(false) + expect(parseConsents([{ discord_user_id: '100', active: 'false' }])[0]!.active).toBe(false) + }) + + it('never reports a negative or nonsensical recording count', () => { + // A defect upstream must not render as "-3 recordings" beside somebody's + // name, where it reads as a fact about them. + expect(parseConsents([{ discord_user_id: '1', recordings_with_audio: -3 }])[0]!.recordings_with_audio).toBe(0) + expect(parseConsents([{ discord_user_id: '1', recordings_with_audio: 'many' }])[0]!.recordings_with_audio).toBe(0) + }) + + it('drops an entry that names nobody', () => { + expect(parseConsents([{ display_name: 'Anna' }, { discord_user_id: '100' }])).toHaveLength(1) + }) + + it('yields nothing for a payload it cannot make sense of', () => { + expect(parseConsents(null)).toEqual([]) + expect(parseConsents('nonsense')).toEqual([]) + }) +}) + +describe('naming a person', () => { + it('uses the display name when there is one', () => { + expect(personLabel(row({ discord_user_id: '100', display_name: 'Anna' }))).toBe('Anna') + }) + + it('falls back to the whole id, never a shortened one', () => { + // Snowflakes from the same era share their leading digits, so a + // truncated id names a group rather than a person. + expect(personLabel(row({ discord_user_id: '1129384756123456789' }))).toContain( + '1129384756123456789', + ) + }) + + it('says plainly that a nameless row is showing an id', () => { + const note = identityNote(row({ discord_user_id: '100' }))! + expect(note).toContain('Discord user id') + }) + + it('explains why there is no name rather than looking broken', () => { + // A bare snowflake where every other row has a name reads as a fault in + // the console. It is not: consent is given in a command, and a name is + // only learned in a recorded session. + expect(identityNote(row({ discord_user_id: '100' }))!).toContain('recorded session') + }) + + it('says nothing at all when there is a name', () => { + expect(identityNote(row({ discord_user_id: '100', display_name: 'Anna' }))).toBeNull() + }) +}) + +describe('the state a consent is in', () => { + it('marks a consent that is in force', () => { + const badge = consentBadge(row({ discord_user_id: '100', active: true })) + expect(badge.tone).toBe('active') + expect(badge.label.toLowerCase()).toContain('in force') + }) + + it('takes the API at its word instead of deriving activity from the dates', () => { + // `active` is authoritative: consent also stops counting when the + // guild's policy_version moves on, which no date on this row records. + const badge = consentBadge(row({ discord_user_id: '100', active: false, revoked_at: null })) + expect(badge.tone).not.toBe('active') + }) + + it('tells a withdrawn consent apart from one whose policy version moved on', () => { + // The whole point of the badge. One of these the person decided; the + // other happened to them. Rendering both as a grey "inactive" would + // hide which, and the two need different things done about them. + const withdrawn = consentBadge( + row({ discord_user_id: '100', active: false, revoked_at: '2026-08-22T09:00:00+00:00' }), + ) + const lapsed = consentBadge(row({ discord_user_id: '101', active: false, revoked_at: null })) + expect(withdrawn.tone).toBe('withdrawn') + expect(lapsed.tone).toBe('superseded') + expect(withdrawn.tone).not.toBe(lapsed.tone) + }) + + it('says outright that a lapsed consent was not withdrawn by anybody', () => { + const badge = consentBadge(row({ discord_user_id: '100', active: false, revoked_at: null })) + expect(badge.detail.toLowerCase()).toContain('nobody withdrew') + expect(badge.detail).toContain('policy_version') + }) + + it('names the version a lapsed consent was given under', () => { + // It is the field that explains the row, and the one somebody needs + // before deciding whether bumping the version again is worth it. + const badge = consentBadge( + row({ discord_user_id: '100', active: false, revoked_at: null, policy_version: '2025-11' }), + ) + expect(badge.detail).toContain('2025-11') + }) + + it('says how a lapsed consent comes back, since the console cannot do it', () => { + const badge = consentBadge(row({ discord_user_id: '100', active: false, revoked_at: null })) + expect(badge.detail).toContain('/consent grant') + }) + + it('dates a withdrawal rather than merely reporting one', () => { + const badge = consentBadge( + row({ discord_user_id: '100', active: false, revoked_at: '2026-08-22T09:00:00+00:00' }), + ) + expect(badge.detail).toContain('22 Aug 2026') + }) +}) + +describe('the facts on a row', () => { + it('writes a moment in UTC and says so', () => { + // The console has one way of printing a moment, borrowed from + // `~/utils/format`: the server render cannot know the reader's zone, so + // a second rendering would disagree with the first and hydration would + // rewrite every timestamp on the page. + expect(grantedLine(row({ discord_user_id: '100' }))).toBe('Granted 21 Aug 2026, 12:00 UTC.') + }) + + it('admits an unrecorded grant date instead of printing a dash', () => { + expect(grantedLine(row({ discord_user_id: '100', granted_at: null })).toLowerCase()).toContain( + 'not recorded', + ) + }) + + it('gives no withdrawal line to a consent nobody withdrew', () => { + // A lapsed consent borrowing this line would read as a decision the + // person made, which is exactly what it is not. + expect(withdrawnLine(row({ discord_user_id: '100', active: false }))).toBeNull() + }) + + it('dates a withdrawal when there was one', () => { + expect( + withdrawnLine(row({ discord_user_id: '100', revoked_at: '2026-08-22T09:30:00+00:00' })), + ).toContain('22 Aug 2026, 09:30 UTC') + }) + + it('names the policy version on every row, not only the lapsed ones', () => { + expect(policyLine(row({ discord_user_id: '100', policy_version: '2026-01' }))).toContain('2026-01') + }) + + it('counts the recordings still held in singular and plural', () => { + expect(recordingsLine(row({ discord_user_id: '1', recordings_with_audio: 1 }))).toContain( + '1 recording containing', + ) + expect(recordingsLine(row({ discord_user_id: '1', recordings_with_audio: 3 }))).toContain( + '3 recordings containing', + ) + }) + + it('says none are held rather than printing a zero', () => { + expect(recordingsLine(row({ discord_user_id: '1', recordings_with_audio: 0 }))).toContain('no recordings') + }) +}) + +describe('whether a revoke is offered', () => { + it('offers it for a consent that is in force', () => { + expect(revocability(row({ discord_user_id: '100', active: true })).revocable).toBe(true) + }) + + it('still offers it for a consent whose policy version moved on', () => { + // The record is still there. Withdrawing removes it rather than waiting + // for it, which matters the moment somebody rolls policy_version back + // to a previous value and every lapsed consent comes back to life. + expect( + revocability(row({ discord_user_id: '100', active: false, revoked_at: null })).revocable, + ).toBe(true) + }) + + it('refuses to offer it for a consent already withdrawn', () => { + // The endpoint answers 409 `already_revoked`, every time. An interface + // that offers an action it knows the outcome of is worse than one that + // explains why it cannot. + const verdict = revocability( + row({ discord_user_id: '100', active: false, revoked_at: '2026-08-22T09:00:00+00:00' }), + ) + expect(verdict.revocable).toBe(false) + }) + + it('says when it was withdrawn instead of merely refusing', () => { + const verdict = revocability( + row({ discord_user_id: '100', active: false, revoked_at: '2026-08-22T09:00:00+00:00' }), + ) + expect(verdict.revocable).toBe(false) + if (!verdict.revocable) expect(verdict.reason).toContain('22 Aug 2026') + }) +}) + +describe('confirming a withdrawal', () => { + const anna = row({ discord_user_id: '100', display_name: 'Anna', recordings_with_audio: 3 }) + + it('names the person whose consent is about to go', () => { + // Two rows apart on a long page, with the same button on each, is how + // the wrong one gets clicked. + expect(revokeConfirmation(anna).title).toContain('Anna') + }) + + it('says the Discord role is not removed', () => { + // The load-bearing sentence of this page. An administrator who believes + // the role went with it will not go and remove it, and the member keeps + // a role that says something untrue about them. + const said = revokeConfirmation(anna).consequences.join(' ') + expect(said).toContain('does not remove the Discord consent role') + }) + + it('explains why the role cannot be removed rather than just that it is not', () => { + expect(ROLE_STAYS_NOTE).toContain('no Discord token') + }) + + it('says recording still stops anyway, and how soon', () => { + // Otherwise "the role stays" reads as "this did nothing". + const said = revokeConfirmation(anna).consequences.join(' ') + expect(said).toContain('five seconds') + expect(said.toLowerCase()).toContain('running session') + }) + + it('says nothing already recorded is deleted', () => { + // Letting "withdrawn" read as "erased" would answer a data subject's + // erasure request with a lie. + const said = revokeConfirmation(anna).consequences.join(' ') + expect(said).toContain('Nothing already recorded is deleted') + }) + + it('names how many recordings of that person specifically remain', () => { + // A general sentence about retention is easy to read past; "the 3 + // recordings that already contain Anna's audio stay" is not. + const said = revokeConfirmation(anna).consequences.join(' ') + expect(said).toContain('3 recordings that already contain') + expect(said).toContain('Anna') + }) + + it('names the separate act that does erase them', () => { + expect(revokeConfirmation(anna).consequences.join(' ')).toContain('/audio purge') + }) + + it('still points at the purge command for somebody with nothing on disk', () => { + const said = recordingsKeptNote(row({ discord_user_id: '100', recordings_with_audio: 0 })) + expect(said).toContain('no recordings') + expect(said).toContain('/audio purge') + }) + + it('says the withdrawal is logged, before it happens', () => { + expect(revokeConfirmation(anna).consequences).toContain(AUDIT_LOG_NOTE) + expect(AUDIT_LOG_NOTE).toContain('audit log') + }) + + it('keeps the three facts as three, not as one paragraph', () => { + // A wall of prose is skimmed exactly where the reader most needs to + // notice that the role and the recordings are not part of this. + expect(revokeConfirmation(anna).consequences).toHaveLength(3) + }) + + it('labels the button with what it does, not with "OK"', () => { + expect(revokeConfirmation(anna).confirmLabel.toLowerCase()).toContain('withdraw') + }) +}) + +describe('what the revoke endpoint answered', () => { + it('reads a successful withdrawal', () => { + expect(parseRevokeResult({ revoked: true, refusal: null })).toEqual({ + revoked: true, + refusal: null, + }) + }) + + it('reads a refusal and its reason', () => { + expect(parseRevokeResult({ revoked: false, refusal: 'already_revoked' }).refusal).toBe( + 'already_revoked', + ) + }) + + it('never reports a body it cannot make sense of as a withdrawal', () => { + // The only person who would find out otherwise is the one still being + // recorded. + expect(parseRevokeResult(null).revoked).toBe(false) + expect(parseRevokeResult({}).revoked).toBe(false) + expect(parseRevokeResult({ revoked: 'true' }).revoked).toBe(false) + }) + + it('tells the two refusals apart', () => { + // Both mean the row is out of date; they send an administrator to + // different places all the same. + expect(describeRefusal('already_revoked')).not.toBe(describeRefusal('no_consent_on_record')) + expect(describeRefusal('already_revoked').toLowerCase()).toContain('already been withdrawn') + expect(describeRefusal('no_consent_on_record').toLowerCase()).toContain('no consent record') + }) + + it('writes a refusal it has no code for so that it is true of both', () => { + // `useApi` strips the body off every failed request on purpose, so a + // 409 reaches this console as a status with no refusal attached. The + // sentence therefore has to cover either reason without guessing. + const said = describeRefusal(null).toLowerCase() + expect(said).toContain('withdrawn') + expect(said).toContain('never a record') + expect(said).toContain('not being recorded') + }) + + it('never says merely "done" after a successful withdrawal', () => { + const outcome = revokeOutcome( + row({ discord_user_id: '100', display_name: 'Anna', recordings_with_audio: 2 }), + { revoked: true, refusal: null }, + ) + expect(outcome.tone).toBe('done') + expect(outcome.headline).toContain('Anna') + }) + + it('repeats both limits after the act as well as before it', () => { + // This is the moment somebody is most likely to believe more happened + // than did: they have just watched the row change state. + const outcome = revokeOutcome( + row({ discord_user_id: '100', display_name: 'Anna', recordings_with_audio: 2 }), + { revoked: true, refusal: null }, + ) + expect(outcome.detail).toContain('Discord consent role is unchanged') + expect(outcome.detail).toContain('2 recordings') + expect(outcome.detail).toContain('audit log') + }) + + it('says who can undo it, since this console cannot', () => { + const outcome = revokeOutcome(row({ discord_user_id: '100' }), { revoked: true, refusal: null }) + expect(outcome.detail).toContain('/consent grant') + }) + + it('reports a refusal as a refusal, not as a withdrawal', () => { + const outcome = revokeOutcome(row({ discord_user_id: '100' }), { + revoked: false, + refusal: 'already_revoked', + }) + expect(outcome.tone).toBe('refused') + expect(outcome.headline.toLowerCase()).toContain('nothing was withdrawn') + }) +}) + +describe('when the API says no', () => { + it('explains the 404 without pretending to know which of the two it is', () => { + // The API answers 404 both for a guild that does not exist and for one + // the caller does not administer, on purpose. + const said = describeConsentError(failure(404)) + expect(said).toContain('no longer administer it') + expect(said.toLowerCase()).toContain('same') + }) + + it('describes a 409 as the row being out of date', () => { + expect(describeConsentError(failure(409))).toBe(describeRefusal(null)) + }) + + it('treats only a 409 as a stale row', () => { + // A 409 always means there is nothing left to withdraw, which makes + // reloading the list the correct response rather than a hopeful one. + expect(isStaleRow(failure(409))).toBe(true) + expect(isStaleRow(failure(404))).toBe(false) + expect(isStaleRow(failure(0))).toBe(false) + expect(isStaleRow(null)).toBe(false) + }) + + it('says nothing was withdrawn when the session has ended', () => { + expect(describeConsentError(failure(401))).toContain('nothing was withdrawn') + }) + + it('tells an unreachable API apart from an API that refused', () => { + // `ApiError` uses status 0 for a request that never got a response, and + // "could not reach the API" and "the API said no" need different words. + expect(describeConsentError(failure(0)).toLowerCase()).toContain('could not reach') + expect(describeConsentError(failure(500))).toContain('500') + }) + + it('says nothing was changed for a status it has never heard of', () => { + expect(describeConsentError(failure(503)).toLowerCase()).toContain('nothing was changed') + }) +}) + +describe('the order the people are listed in', () => { + it('puts the consents in force first', () => { + // They are the only rows where withdrawing changes what happens in a + // meeting, including one running right now. + const ordered = orderConsents([ + row({ discord_user_id: '2', display_name: 'Zoe', active: false, revoked_at: null }), + row({ discord_user_id: '3', display_name: 'Anna', active: true }), + ]) + expect(ordered.map((r) => r.display_name)).toEqual(['Anna', 'Zoe']) + }) + + it('puts the already-withdrawn rows last, below the lapsed ones', () => { + // A withdrawn row has no control on it at all, so it belongs below + // everything that has one. + const ordered = orderConsents([ + row({ + discord_user_id: '1', + display_name: 'Aaron', + active: false, + revoked_at: '2026-08-01T00:00:00+00:00', + }), + row({ discord_user_id: '2', display_name: 'Zoe', active: false, revoked_at: null }), + row({ discord_user_id: '3', display_name: 'Mia', active: true }), + ]) + expect(ordered.map((r) => r.display_name)).toEqual(['Mia', 'Zoe', 'Aaron']) + }) + + it('sorts the named people by name, ignoring case', () => { + const ordered = orderConsents([ + row({ discord_user_id: '1', display_name: 'zoe' }), + row({ discord_user_id: '2', display_name: 'Anna' }), + row({ discord_user_id: '3', display_name: 'mia' }), + ]) + expect(ordered.map((r) => r.display_name)).toEqual(['Anna', 'mia', 'zoe']) + }) + + it('puts the people with no name below the people with one', () => { + // Somebody arrives here having been asked about a person and scans for + // a name. Nobody scans for a snowflake -- they search the page for it. + const ordered = orderConsents([ + row({ discord_user_id: '100', display_name: null }), + row({ discord_user_id: '200', display_name: 'Zoe' }), + ]) + expect(ordered.map((r) => r.discord_user_id)).toEqual(['200', '100']) + }) + + it('orders the nameless rows numerically rather than as strings', () => { + // As plain strings "1000" sorts before "999"; as numbers a snowflake + // past the safe integer range becomes a different id entirely. + const ordered = orderConsents([ + row({ discord_user_id: '1000' }), + row({ discord_user_id: '999' }), + row({ discord_user_id: '1129384756123456789' }), + ]) + expect(ordered.map((r) => r.discord_user_id)).toEqual([ + '999', + '1000', + '1129384756123456789', + ]) + }) + + it('never leaves two people sharing a name in an arbitrary order', () => { + // Every comparison ends at the id, which is unique, so the order is + // total and the rows do not swap places between renders. + const twice = () => + orderConsents([ + row({ discord_user_id: '200', display_name: 'Anna' }), + row({ discord_user_id: '100', display_name: 'Anna' }), + ]).map((r) => r.discord_user_id) + expect(twice()).toEqual(['100', '200']) + expect(twice()).toEqual(['100', '200']) + }) + + it('leaves the payload it was given alone', () => { + const given = [row({ discord_user_id: '2' }), row({ discord_user_id: '1' })] + orderConsents(given) + expect(given.map((r) => r.discord_user_id)).toEqual(['2', '1']) + }) +}) + +describe('how many people can actually be recorded', () => { + it('counts only the consents in force', () => { + // A list of forty rows where six are in force says something a bare row + // count does not. + expect( + activeCount([ + row({ discord_user_id: '1', active: true }), + row({ discord_user_id: '2', active: false, revoked_at: null }), + row({ discord_user_id: '3', active: false, revoked_at: '2026-08-01T00:00:00+00:00' }), + ]), + ).toBe(1) + }) + + it('counts nothing in an empty guild', () => { + expect(activeCount([])).toBe(0) + }) +}) diff --git a/console/test/navigation.spec.ts b/console/test/navigation.spec.ts index f46e83d..35017c1 100644 --- a/console/test/navigation.spec.ts +++ b/console/test/navigation.spec.ts @@ -64,6 +64,17 @@ describe('who is offered the Admin View', () => { expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('Bot Settings') }) + it('offers the user settings to an administrator', () => { + expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('User Settings') + }) + + it('lists what the bot does before what the members have agreed to', () => { + // The configuration is what an administrator comes to the Admin View + // for daily; a member's consent is looked at when somebody asks about + // that one member, which is rarer and always deliberate. + expect(ADMIN_VIEW.entries.map((e) => e.label)).toEqual(['Bot Settings', 'User Settings']) + }) + it('hides it from somebody who administers nothing', () => { expect(visibleSections(PARTICIPANT).map((s) => s.label)).toEqual(['User View']) }) From f38a2e4d8a38cda3b8ecf3ad194f6238900dd76a Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:26:20 +0200 Subject: [PATCH 04/14] docs(operations): how withdrawing somebody else's consent behaves The runbook half of the console's consent revocation. Three things an administrator would otherwise reasonably believe and which are not true: that it removes the Discord role, that it deletes what was already recorded, and that a consent with no `revoked_at` is necessarily in force. It also writes down where the audit trail actually lives. `consent` has no column naming who performed a revocation -- `/consent revoke` never needed one, because the only person who could run it was the subject -- so for a third-party revocation the log line is the whole trail, and its retention is the trail's retention. That is a fact somebody should learn from the runbook rather than from needing it. --- docs/operations.md | 69 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/docs/operations.md b/docs/operations.md index 0dbf8da..8fa9ccb 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1036,6 +1036,75 @@ cannot be without giving `api` a Discord token. a recording. The console shows the track and omits the speaker rather than dropping a recording that exists. +### 6.2.6 Withdrawing somebody else's consent + +An administrator can end a member's consent from the console's **Admin +View → User Settings**, per guild. This is the third way a consent can +end, and the three are not interchangeable: + +| Way | Scope | Removes the role? | Effect on recording | +|---|---|---|---| +| `/consent revoke` in Discord | The person themselves | **Yes** | Immediate (role check is per frame, uncached) | +| Bumping `policy_version` | Everybody in the guild at once | No | Within the consent cache's 5 s TTL | +| Console → User Settings | One named person | **No** | Within the consent cache's 5 s TTL | + +**The console cannot remove the Discord role, and this is deliberate.** +`api` holds no Discord token (§6.2.2), so it writes `consent.revoked_at` +and nothing else. That is enough to stop the recording: the packet filter +checks *both* layers on every frame, and the stored record is the layer +that exists precisely because somebody with Discord's `administrator` +permission bypasses channel permissions and could speak without the role. +Recording of that person stops mid-session, within five seconds. + +What it leaves is a member who still holds the consent role. Nothing +records them, but Discord looks as though something might, and `/consent +status` will report "role assigned: yes, consent active: no". **If the +role should go too, remove it in Discord** — either by hand or by asking +the person to run `/consent revoke`, which does both. + +Note the asymmetry with removing the role *only*, which is what an +administrator would otherwise do by hand: that stops the recording and +leaves `revoked_at` NULL, so the record still reads as consent given, and +re-adding the role at any point silently resumes recording somebody who +never re-consented. Withdrawing through the console is the half that +lasts. + +**It does not delete anything already recorded.** Withdrawing consent is a +decision about the future. Erasing what was recorded under the consent +that existed at the time is `/audio purge ` in Discord (§12.3), and +it is deliberately a separate act with a separate command. The User +Settings page shows, per person, how many recordings the guild still holds +for them, so the distinction is on the screen rather than in this +document. + +**A consent record is never deleted.** `revoked_at` is stamped on the +newest grant; earlier grants keep their history. The row *is* the evidence +that consent was once given, which is what Art. 7(1) requires be +demonstrable, so a revocation modifies the grant it revokes rather than +removing it. + +**Who did it is only in the log.** `consent` has no column naming the +person who performed a revocation — `/consent revoke` never needed one, +because the only person who could run it was the subject. So the audit +trail for a third-party revocation is the log line and nothing else: + +```logql +{namespace="sturnus"} | json | sturnus_event="console.consent_revoked" +``` + +It is emitted at **WARNING** with `guild_id`, `discord_user_id` (whose +consent) and `requested_by` (who withdrew it). Retention of that line is +therefore the retention of the audit trail; if a longer one is needed, it +has to become a column, which is a migration and a change to the shared +`ConsentRepository.record_revocation`. + +**A consent that is inactive is not the same as one that was withdrawn.** +The page distinguishes them, and so should anybody reading it. A grant +naming a superseded `policy_version` has no force and a NULL `revoked_at`: +nobody withdrew anything, the guild's policy moved on under them (§6). +Restoring the old `policy_version` would bring every one of those back. +Withdrawing through the console is what survives that. + ### 6.3 Listening to a recording by hand Every automated check this system has can describe a track — its level, From 160d883af3f0eb386e3af138666e16623450ac54 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 21:56:01 +0200 Subject: [PATCH 05/14] feat(console): a guild-wide view of what the transcription queue still owes The console could already say where one session's transcription had got to, if you already knew which session to ask about. What it could not answer is the question an administrator actually starts from: is anything stuck, and if so, where. `/queue status` answers the first half in Discord; this answers both, with the sessions the totals are made of. `GET /api/guilds/{guild_id}/queue`. Everything below the authorisation is `sturnus.infrastructure.db.requeue`, and the totals are `load_status` unchanged -- the same function the `/queue status` command reads. A console that counted the jobs itself would be a second definition of "how much work is outstanding", and the two would agree until one of them changed. The one new read is `load_active_sessions`, which lives beside the others rather than in the console adapter, so both callers keep asking the same questions of the same place. **Unfinished is deliberately two conditions.** Anything that is not `documented` is obviously unfinished. But a session can reach `documented` with a `dead` job in it -- the document is written once every job is terminal, and `dead` is terminal -- so a speaker whose transcription failed permanently would vanish from the queue view at exactly the moment somebody needs to notice them. A dead job keeps its session on the list however finished the session claims to be. An `open` session with no jobs at all is included for the opposite reason: it is a recording happening right now. Two numbers travel with a caveat rather than as bare facts, because both would otherwise be read as something they are not: - `running_past_lease` is derived from an *assumed* lease, and the lease that applies is the worker's `job_lease_seconds`, which this process cannot see. So `lease_seconds` is sent with the count and the console names it -- the same caveat `/queue status` prints in Discord. - `oldest_pending_session_ended_at` is dated by the session's end, because `transcription_job` has no enqueue timestamp at all. That is within seconds of when the jobs were created, which answers "has this been sitting here for hours?" and nothing more precise. It is not the age of a re-queued job, which keeps its session's original end. The list is cut at twenty and says when it was cut. A guild that has been broken for a month has hundreds, and a page showing twenty that does not admit to cutting reads as "there are twenty". 404 for a guild this person does not administer, and the same 404 for one that does not exist: the list names when a guild met and in which channel. --- src/sturnus/console/adapters.py | 77 ++++ src/sturnus/console/app.py | 5 +- src/sturnus/console/ports.py | 73 ++++ src/sturnus/console/routes_queue.py | 120 +++++- src/sturnus/entrypoints/api.py | 2 + src/sturnus/infrastructure/db/requeue.py | 113 +++++- tests/console/conftest.py | 24 ++ tests/console/test_queue_overview.py | 443 +++++++++++++++++++++++ tests/console/test_queue_routes.py | 188 +++++++++- 9 files changed, 1039 insertions(+), 6 deletions(-) create mode 100644 tests/console/test_queue_overview.py diff --git a/src/sturnus/console/adapters.py b/src/sturnus/console/adapters.py index fe4f86f..62540e9 100644 --- a/src/sturnus/console/adapters.py +++ b/src/sturnus/console/adapters.py @@ -30,6 +30,8 @@ from sturnus.console.ports import ( AdminDirectory, ConsentHolder, + GuildQueue, + QueuedSession, QueueSnapshot, QueueSpeaker, RequeueOutcome, @@ -47,12 +49,16 @@ TranscriptionJob, ) from sturnus.infrastructure.db.models import Session as SessionRow +from sturnus.infrastructure.db.queue import DEFAULT_LEASE_SECONDS from sturnus.infrastructure.db.repositories import ConsentRepository from sturnus.infrastructure.db.requeue import ( + ActiveSession, SessionView, apply_requeue, + load_active_sessions, load_requeue_view, load_session, + load_status, ) #: How long a sign-in may take. Ten minutes is a browser round trip @@ -576,3 +582,74 @@ async def _recordings_with_audio(self, guild_id: int, people: Sequence[int]) -> ) ).all() return {discord_user_id: held for discord_user_id, held in rows} + + +class ConsoleQueueOverview: + """A guild's transcription queue, for an administrator of that guild. + + The guild-wide companion to `ConsoleQueueControl`, and built the same + way: the administrator check is part of the one call, and everything + below it is `sturnus.infrastructure.db.requeue` unchanged -- the same + `load_status` the `/queue status` command reads. A console that + counted the jobs itself would be a second definition of "how much work + is outstanding", and the two would agree until one of them changed. + + `load_active_sessions` is new machinery rather than reused, because + Discord never needed it: a slash command answers in one message and + reports totals, while a page has room to say *which* sessions the + totals are made of. It lives beside the other reads for the same + reason they do -- so both callers ask the same questions. + """ + + def __init__( + self, + session_factory: async_sessionmaker[AsyncSession], + admins: AdminDirectory, + now: Callable[[], datetime], + lease_seconds: float = DEFAULT_LEASE_SECONDS, + ) -> None: + self._session_factory = session_factory + self._admins = admins + self._now = now + #: The lease this process *assumes*. The one that actually applies + #: is `job_lease_seconds` in the worker's settings, which this + #: process cannot see -- so the number travels out with the answer + #: and the console names it rather than presenting a count derived + #: from a guess as a fact. The same caveat `/queue status` prints. + self._lease_seconds = lease_seconds + + async def for_guild(self, guild_id: int, *, requested_by: int) -> GuildQueue | None: + if not await self._admins.is_admin(guild_id, requested_by): + return None + + now = self._now() + status = await load_status(self._session_factory, guild_id, now, self._lease_seconds) + sessions, truncated = await load_active_sessions(self._session_factory, guild_id) + return GuildQueue( + pending=status.counts.get("pending", 0), + running=status.counts.get("running", 0), + done=status.counts.get("done", 0), + dead=status.counts.get("dead", 0), + running_past_lease=status.running_past_lease, + oldest_pending_session_ended_at=status.oldest_pending_session_ended_at, + closed_undocumented=status.closed_undocumented, + lease_seconds=self._lease_seconds, + sessions=tuple(_queued(session) for session in sessions), + truncated=truncated, + ) + + +def _queued(session: ActiveSession) -> QueuedSession: + return QueuedSession( + id=session.id, + channel_id=session.channel_id, + channel_name=session.channel_name, + started_at=session.started_at, + ended_at=session.ended_at, + status=session.status, + document_url=session.document_url, + pending=session.counts.get("pending", 0), + running=session.counts.get("running", 0), + done=session.counts.get("done", 0), + dead=session.counts.get("dead", 0), + ) diff --git a/src/sturnus/console/app.py b/src/sturnus/console/app.py index 6504d9b..6009375 100644 --- a/src/sturnus/console/app.py +++ b/src/sturnus/console/app.py @@ -40,6 +40,7 @@ LinkDirectory, OAuthClient, QueueControl, + QueueOverview, SessionReads, SettingsStore, StateStore, @@ -48,7 +49,7 @@ from sturnus.console.routes_audio import register as register_audio from sturnus.console.routes_consent import CONSENT_DIRECTORY from sturnus.console.routes_consent import register as register_consent -from sturnus.console.routes_queue import QUEUE_CONTROL +from sturnus.console.routes_queue import QUEUE_CONTROL, QUEUE_OVERVIEW from sturnus.console.routes_queue import register as register_queue from sturnus.console.session import ( ExpiredSession, @@ -271,6 +272,7 @@ def build_api( console_origin: str, audio: AudioDelivery, queue: QueueControl, + queues: QueueOverview, consents: ConsentDirectory, ) -> web.Application: """Builds the application, with every collaborator injected. @@ -297,6 +299,7 @@ def build_api( app[_CONSOLE_ORIGIN] = console_origin app[AUDIO_DELIVERY] = audio app[QUEUE_CONTROL] = queue + app[QUEUE_OVERVIEW] = queues app[CONSENT_DIRECTORY] = consents app.add_routes( [ diff --git a/src/sturnus/console/ports.py b/src/sturnus/console/ports.py index 51d2db0..7dcdc7e 100644 --- a/src/sturnus/console/ports.py +++ b/src/sturnus/console/ports.py @@ -366,3 +366,76 @@ async def holders( async def revoke( self, guild_id: int, discord_user_id: int, *, requested_by: int ) -> RevocationOutcome | None: ... + + +@dataclass(frozen=True) +class QueuedSession: + """One session the transcription pipeline has not finished with.""" + + id: int + channel_id: int + channel_name: str | None + started_at: datetime + ended_at: datetime | None + status: str + document_url: str | None + pending: int + running: int + done: int + dead: int + + +@dataclass(frozen=True) +class GuildQueue: + """Where a guild's transcription work stands, right now. + + The guild-wide counts and the unfinished sessions in one value, read + together, because a page that showed "3 pending" beside a list read a + moment later would occasionally show three pending jobs and no session + they could belong to. + + `running_past_lease` is the number worth reading first: a `running` + job whose lease expired is one whose worker died holding it, which no + amount of waiting fixes. + + `lease_seconds` travels with it because that count is derived from an + assumed lease, and the lease that actually applies is the *worker's* + `job_lease_seconds`. The console says which number it used rather than + presenting a derived count as a fact -- the same caveat `/queue + status` prints, for the same reason. + """ + + pending: int + running: int + done: int + dead: int + running_past_lease: int + #: When the session owning the oldest `pending` job ended. + #: `transcription_job` has no enqueue timestamp at all, and a session's + #: end is within seconds of when its jobs were created -- close enough + #: to answer "has something been sitting here for hours?". It is *not* + #: the age of a re-queued job, which keeps its session's original end, + #: and the console says so rather than calling it a job age. + oldest_pending_session_ended_at: datetime | None + #: Sessions that are closed, have no unfinished jobs, and still have no + #: document. Nothing is queued for them and nothing will happen on its + #: own. + closed_undocumented: int + lease_seconds: float + sessions: tuple[QueuedSession, ...] + #: Whether the list above was cut short. Sent so that a page showing + #: twenty sessions never reads as "there are twenty". + truncated: bool + + +class QueueOverview(Protocol): + """A guild's transcription queue, if the person asking administers it. + + The guild-wide companion to `QueueControl`, which answers about one + session. Same rule, same shape: `requested_by` is not optional, there + is no method here without it, and `None` covers "no such guild" and + "you do not administer it" alike -- because from outside those must + look the same. + """ + + async def for_guild(self, guild_id: int, *, requested_by: int) -> GuildQueue | None: ... diff --git a/src/sturnus/console/routes_queue.py b/src/sturnus/console/routes_queue.py index 6163f09..c1f03e7 100644 --- a/src/sturnus/console/routes_queue.py +++ b/src/sturnus/console/routes_queue.py @@ -1,8 +1,16 @@ -"""Re-running a session's transcription from the console, and watching it. +"""A guild's transcription queue, and re-running one session's part of it. +- `GET /api/guilds/{guild_id}/queue` - `GET /api/sessions/{session_id}/queue` - `POST /api/sessions/{session_id}/queue/requeue` +**Why the guild-wide view is here rather than in a module of its own.** +It is the same subject asked at a different scale: the per-session +endpoints answer "where has this one got to", and the guild one answers +"what is outstanding, and which sessions is it outstanding in". Splitting +them across two files would have put one authorisation rule in two places +and invited them to drift. + **Why this exists at all.** The first pass over a recording can be wrong — a model that hallucinated, a worker that died, a bug since fixed — and until now the only way to ask for another one was `/queue requeue` in @@ -42,7 +50,14 @@ from aiohttp import web -from sturnus.console.ports import QueueControl, QueueSnapshot, RequeueOutcome +from sturnus.console.ports import ( + GuildQueue, + QueueControl, + QueuedSession, + QueueOverview, + QueueSnapshot, + RequeueOutcome, +) from sturnus.observability.events import Event, log_event log = logging.getLogger(__name__) @@ -51,8 +66,45 @@ #: because it belongs to these routes and nothing else reads it. QUEUE_CONTROL = web.AppKey("queue_control", QueueControl) +#: The guild-wide overview's collaborator. A second key rather than one +#: object with both shapes, because the two answer different questions at +#: different scales and a protocol that offered both would let a handler +#: reach for the wide one where the narrow one was meant. +QUEUE_OVERVIEW: web.AppKey[QueueOverview] = web.AppKey("queue_overview") + _STATUS_PATH = "/api/sessions/{session_id}/queue" _REQUEUE_PATH = "/api/sessions/{session_id}/queue/requeue" +_GUILD_PATH = "/api/guilds/{guild_id}/queue" + + +async def guild_queue(request: web.Request) -> web.Response: + """What this guild's transcription pipeline still owes, and where. + + 404 for a guild this person does not administer, and the same 404 for + one that does not exist. The list names when a guild met, in which + channel, and how many people spoke; a 403 would confirm that such a + list exists here to somebody just established as having no business + with it. + """ + from sturnus.console.app import current_user + + viewer = current_user(request).discord_user_id + try: + guild_id = int(request.match_info["guild_id"]) + except ValueError: + # A path segment that is not a number names no guild, which is the + # same answer as naming one that does not exist. + return _no_such_guild() + + queue = await request.app[QUEUE_OVERVIEW].for_guild(guild_id, requested_by=viewer) + if queue is None: + return _no_such_guild() + return web.json_response( + _guild_queue_json(guild_id, queue), + # It names when a guild met and in which channel, and it is stale + # the moment a worker claims a job. + headers={"Cache-Control": "private, no-store"}, + ) async def queue_status(request: web.Request) -> web.Response: @@ -169,6 +221,67 @@ def _outcome_json(outcome: RequeueOutcome) -> dict[str, object]: } +def _guild_queue_json(guild_id: int, queue: GuildQueue) -> dict[str, object]: + return { + "guild_id": str(guild_id), + # The lifecycle in its own order rather than four sibling fields: + # `pending -> running -> done | dead` is how a reader finds where + # work is piling up, and a shape that spelled it out flat would + # leave that ordering to whichever client rendered it. + "counts": { + "pending": queue.pending, + "running": queue.running, + "done": queue.done, + "dead": queue.dead, + }, + "running_past_lease": queue.running_past_lease, + "oldest_pending_session_ended_at": ( + None + if queue.oldest_pending_session_ended_at is None + else queue.oldest_pending_session_ended_at.isoformat() + ), + "closed_undocumented": queue.closed_undocumented, + # Sent with the count it produced. `running_past_lease` is derived + # from an assumed lease and the one that applies is the worker's, + # which this process cannot see -- so the console names the number + # it used instead of presenting the count as a fact. + "lease_seconds": queue.lease_seconds, + "truncated": queue.truncated, + "sessions": [_queued_session_json(session) for session in queue.sessions], + } + + +def _queued_session_json(session: QueuedSession) -> dict[str, object]: + return { + # A string like every other id in this API. Session ids do not + # need it and follow anyway: two id shapes in one payload is how + # the one that matters gets parsed with the wrong one. + "id": str(session.id), + "channel_id": str(session.channel_id), + "channel_name": session.channel_name, + "started_at": session.started_at.isoformat(), + "ended_at": None if session.ended_at is None else session.ended_at.isoformat(), + "status": session.status, + "document_url": session.document_url, + "counts": { + "pending": session.pending, + "running": session.running, + "done": session.done, + "dead": session.dead, + }, + } + + +def _no_such_guild() -> web.Response: + """One refusal for every reason there is to refuse. + + "No such guild" and "you do not administer that guild" are + deliberately indistinguishable, for the reason `_no_such_session` + gives about sessions. + """ + return web.json_response({"error": "no such guild"}, status=404) + + def _no_such_session() -> web.Response: """One refusal for every reason there is to refuse. @@ -179,11 +292,12 @@ def _no_such_session() -> web.Response: def register(app: web.Application) -> None: - """Adds the queue routes to an application that already has its control.""" + """Adds the queue routes to an application that already has its collaborators.""" from sturnus.console.app import require_session app.add_routes( [ + web.get(_GUILD_PATH, require_session(guild_queue)), web.get(_STATUS_PATH, require_session(queue_status)), web.post(_REQUEUE_PATH, require_session(requeue_session)), ] diff --git a/src/sturnus/entrypoints/api.py b/src/sturnus/entrypoints/api.py index d9d70cf..0c8efbb 100644 --- a/src/sturnus/entrypoints/api.py +++ b/src/sturnus/entrypoints/api.py @@ -42,6 +42,7 @@ ConsoleConsentDirectory, ConsoleLinkDirectory, ConsoleQueueControl, + ConsoleQueueOverview, ConsoleStateStore, ConsoleTrackDirectory, ) @@ -213,6 +214,7 @@ def now() -> datetime: console_origin=settings.console_origin, audio=audio, queue=ConsoleQueueControl(session_factory, admins), + queues=ConsoleQueueOverview(session_factory, admins, now), consents=ConsoleConsentDirectory(session_factory, admins, config, now), ) diff --git a/src/sturnus/infrastructure/db/requeue.py b/src/sturnus/infrastructure/db/requeue.py index bff9114..41e3fe3 100644 --- a/src/sturnus/infrastructure/db/requeue.py +++ b/src/sturnus/infrastructure/db/requeue.py @@ -20,7 +20,7 @@ from dataclasses import dataclass from datetime import datetime, timedelta -from sqlalchemy import func, select, update +from sqlalchemy import func, or_, select, update from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker from sturnus.application.publishing import DOCUMENTED_STATUS @@ -247,6 +247,117 @@ async def load_status( ) +#: How many unfinished sessions a queue overview reports before it stops. +#: A guild that has been broken for a month has hundreds, and a page of +#: hundreds is a page nobody reads -- while the twenty newest are where +#: whatever is wrong right now actually shows. The reader is told when the +#: list was cut, so "twenty" never reads as "twenty exist". +ACTIVE_SESSION_LIMIT = 20 + + +@dataclass(frozen=True) +class ActiveSession: + """One session the pipeline has not finished with, and where its jobs are. + + `channel_name` as well as `channel_id`, unlike `SessionSummary`: that + one is rendered into Discord, where `<#id>` resolves to the channel's + current name and stays a working link. A web page has no such + rendering, so it needs the name the session opened under -- the same + name every other console view shows. + """ + + id: int + channel_id: int + channel_name: str | None + started_at: datetime + ended_at: datetime | None + status: str + document_url: str | None + #: One entry per `REPORTED_STATUSES`, zero-filled, so a caller can + #: render the lifecycle in order without checking for absent keys. + counts: dict[str, int] + + +async def load_active_sessions( + session_factory: async_sessionmaker[AsyncSession], + guild_id: int, + limit: int = ACTIVE_SESSION_LIMIT, +) -> tuple[list[ActiveSession], bool]: + """This guild's unfinished sessions, newest first, and whether there are more. + + **What counts as unfinished**, and why it is two conditions rather than + one. Anything that is not `documented` is obviously unfinished: it is + recording now, waiting for a worker, being transcribed, or stuck. But a + session can reach `documented` with a `dead` job in it -- the document + is written once every job is terminal, and `dead` is terminal -- so a + speaker whose transcription failed permanently would vanish from the + queue view at exactly the moment somebody needs to notice them. A dead + job keeps its session on this list however finished the session claims + to be. + + An `open` session with no jobs at all is included on purpose: it is a + recording in progress, which is the one thing an administrator looking + at a queue most wants confirmed. + + Guild-scoped in the statement rather than filtered afterwards, the same + rule the rest of this module follows: a `WHERE` that names the guild + cannot be forgotten, because without it the query returns nothing + rather than everything. + """ + dead_jobs = select(TranscriptionJob.session_id).where(TranscriptionJob.status == "dead") + async with session_factory() as db: + rows = ( + await db.execute( + select(Session) + .where( + Session.guild_id == guild_id, + or_(Session.status != DOCUMENTED_STATUS, Session.id.in_(dead_jobs)), + ) + # By id as well as by time, so two sessions that opened in + # the same instant do not swap places between two refreshes. + .order_by(Session.started_at.desc(), Session.id.desc()) + # One more than asked for, which is how "there are more" + # is learned without a second `COUNT(*)` over the same + # predicate -- a count that could disagree with the page it + # describes, having been taken a moment later. + .limit(limit + 1) + ) + ).scalars() + found = list(rows) + truncated = len(found) > limit + found = found[:limit] + if not found: + return [], False + + counted = await db.execute( + select(TranscriptionJob.session_id, TranscriptionJob.status, func.count()) + .where(TranscriptionJob.session_id.in_([row.id for row in found])) + .group_by(TranscriptionJob.session_id, TranscriptionJob.status) + ) + per_session: dict[int, dict[str, int]] = { + row.id: dict.fromkeys(REPORTED_STATUSES, 0) for row in found + } + for session_id, status, count in counted: + # `setdefault` rather than assignment: a status this build does + # not know about is still work somebody has to account for, and + # dropping it would make the counts silently fail to add up. + per_session[session_id][status] = per_session[session_id].get(status, 0) + int(count) + + return [ + ActiveSession( + id=row.id, + channel_id=row.channel_id, + channel_name=row.channel_name, + started_at=row.started_at, + ended_at=row.ended_at, + status=row.status, + document_url=row.document_url, + counts=per_session[row.id], + ) + for row in found + ], truncated + + async def load_session( session_factory: async_sessionmaker[AsyncSession], guild_id: int, session_id: int ) -> tuple[SessionSummary, list[JobLine], dict[int, str]] | None: diff --git a/tests/console/conftest.py b/tests/console/conftest.py index 09cb3c7..940dcf3 100644 --- a/tests/console/conftest.py +++ b/tests/console/conftest.py @@ -23,9 +23,11 @@ AdminDirectory, ConsentDirectory, ConsentHolder, + GuildQueue, LinkDirectory, OAuthClient, QueueControl, + QueueOverview, QueueSnapshot, RequeueOutcome, RevocationOutcome, @@ -392,6 +394,26 @@ async def requeue(self, session_id: int, *, requested_by: int) -> RequeueOutcome return self.outcome +class FakeQueueOverview: + """A guild queue nobody administers, until a test says otherwise. + + Defaults to `None`, which is what the real overview answers for "no + such guild or not yours" -- so a test with no interest in the queue + gets 404s rather than a fake that quietly authorises everything. + """ + + def __init__(self, queue: GuildQueue | None = None) -> None: + self.queue = queue + #: Every guild this was asked about, with who asked. The route + #: tests assert on it: "the handler passed the signed-in id, not + #: one from the URL" cannot be seen in a response body. + self.asked: list[tuple[int, int]] = [] + + async def for_guild(self, guild_id: int, *, requested_by: int) -> GuildQueue | None: + self.asked.append((guild_id, requested_by)) + return self.queue + + class FakeConsents: """A consent directory nobody administers, until a test says otherwise. @@ -437,6 +459,7 @@ def build_test_api( config: SettingsStore | None = None, audio: AudioDelivery | None = None, queue: QueueControl | None = None, + queues: QueueOverview | None = None, consents: ConsentDirectory | None = None, sessions: SessionCookie | None = None, now: Callable[[], datetime] | None = None, @@ -472,6 +495,7 @@ def build_test_api( source=FakeAudioSource(), ), queue=queue or FakeQueue(), + queues=queues or FakeQueueOverview(), consents=consents or FakeConsents(), sessions=sessions or SessionCookie(SECRET, timedelta(hours=12)), now=now or now_at(), diff --git a/tests/console/test_queue_overview.py b/tests/console/test_queue_overview.py new file mode 100644 index 0000000..31eeaeb --- /dev/null +++ b/tests/console/test_queue_overview.py @@ -0,0 +1,443 @@ +"""The guild-wide queue overview, against the real database. + +Against PostgreSQL rather than a double because the whole question is +which sessions the statement selects, and a double would select whatever +it was written to select. The one property that carries the rest is the +definition of "unfinished": it is deliberately two conditions, and the +second exists because of a case the first silently loses -- a session that +reached `documented` with a `dead` job in it, which is the exact moment +somebody needs to notice a speaker whose transcription failed for good. + +The totals are `load_status`, unchanged and unwrapped, so they are pinned +where that function is. What is pinned here is the guild scoping of the +whole answer and the per-session counts the page is made of. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta + +import pytest +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from sturnus.console.adapters import ConsoleQueueOverview +from sturnus.infrastructure.db.models import ( + Base, + Session, + SessionParticipant, + TranscriptionJob, +) +from sturnus.infrastructure.db.requeue import ACTIVE_SESSION_LIMIT, load_active_sessions + +T0 = datetime(2026, 8, 21, 12, 0, 0, tzinfo=UTC) +GUILD, OTHER_GUILD = 4711, 9999 +ANNA, BEN, CARL = 100, 200, 300 + + +@pytest.fixture +async def factory(clean_database: str) -> async_sessionmaker[AsyncSession]: + engine = create_async_engine(clean_database) + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + return async_sessionmaker(engine, expire_on_commit=False) + + +class Admins: + """The mirrored administrator membership, per guild.""" + + def __init__(self, by_guild: dict[int, set[int]] | None = None) -> None: + self.by_guild = by_guild if by_guild is not None else {GUILD: {ANNA}} + + async def is_admin_anywhere(self, discord_user_id: int) -> bool: + return any(discord_user_id in members for members in self.by_guild.values()) + + async def administered_guilds(self, discord_user_id: int) -> tuple[int, ...]: + return tuple( + sorted(g for g, members in self.by_guild.items() if discord_user_id in members) + ) + + async def is_admin(self, guild_id: int, discord_user_id: int) -> bool: + return discord_user_id in self.by_guild.get(guild_id, set()) + + +def overview( + factory: async_sessionmaker[AsyncSession], + *, + admins: Admins | None = None, + now: datetime = T0, + lease_seconds: float = 1800.0, +) -> ConsoleQueueOverview: + return ConsoleQueueOverview(factory, admins or Admins(), lambda: now, lease_seconds) + + +async def a_session( + factory: async_sessionmaker[AsyncSession], + *, + guild_id: int = GUILD, + started_at: datetime = T0, + ended_at: datetime | None = None, + status: str = "closed", + document_url: str | None = None, + channel_name: str | None = "meeting", + jobs: dict[int, str] | None = None, + claimed_at: datetime | None = None, +) -> int: + """One session with one job per speaker, written straight to the tables. + + Direct inserts rather than `RecordingService`: what is under test is + which rows the query selects, and going through the writer would make + it a test of two things at once -- and there is no writer that can + produce a `dead` job on demand. + """ + async with factory() as db: + session = Session( + guild_id=guild_id, + channel_id=555, + channel_name=channel_name, + started_at=started_at, + ended_at=ended_at if ended_at is not None else started_at + timedelta(hours=1), + status=status, + document_url=document_url, + ) + db.add(session) + await db.flush() + for discord_user_id, job_status in (jobs or {}).items(): + db.add( + SessionParticipant( + session_id=session.id, + discord_user_id=discord_user_id, + discord_display_name=f"user-{discord_user_id}", + first_seen_at=started_at, + ) + ) + db.add( + TranscriptionJob( + session_id=session.id, + discord_user_id=discord_user_id, + s3_key=f"sessions/{session.id}/speakers/{discord_user_id}.enc", + encryption_key_id="k1", + wrapped_data_key=b"wrapped", + retention_until=started_at + timedelta(days=30), + status=job_status, + attempts=1, + claimed_at=claimed_at, + ) + ) + await db.commit() + return session.id + + +# --------------------------------------------------------------------------- +# Who may ask +# --------------------------------------------------------------------------- + + +async def test_an_administrator_of_the_guild_sees_its_queue( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.pending == 1 + + +async def test_an_administrator_of_another_guild_is_nobody_here( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, jobs={BEN: "pending"}) + admins = Admins({OTHER_GUILD: {CARL}}) + + assert await overview(factory, admins=admins).for_guild(GUILD, requested_by=CARL) is None + + +async def test_a_participant_who_administers_nothing_sees_no_queue( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, jobs={BEN: "pending"}) + + assert await overview(factory).for_guild(GUILD, requested_by=BEN) is None + + +# --------------------------------------------------------------------------- +# What counts as unfinished +# --------------------------------------------------------------------------- + + +async def test_a_session_still_waiting_for_a_worker_is_in_the_list( + factory: async_sessionmaker[AsyncSession], +) -> None: + session_id = await a_session(factory, jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert [s.id for s in queue.sessions] == [session_id] + + +async def test_a_recording_in_progress_is_in_the_list( + factory: async_sessionmaker[AsyncSession], +) -> None: + """An open session has no jobs at all and belongs here anyway. + + It is a recording happening right now, which is the one thing an + administrator looking at a queue most wants confirmed. + """ + session_id = await a_session(factory, status="open", jobs={}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert [s.id for s in queue.sessions] == [session_id] + + +async def test_a_finished_session_is_not_in_the_list( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session( + factory, + status="documented", + document_url="https://outline.example/doc/1", + jobs={BEN: "done"}, + ) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.sessions == () + + +async def test_a_documented_session_with_a_dead_job_stays_in_the_list( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The case the obvious condition loses. + + The document is written once every job is terminal, and `dead` is + terminal -- so a speaker whose transcription failed permanently would + disappear from the queue view at exactly the moment somebody needs to + notice them. + """ + session_id = await a_session( + factory, + status="documented", + document_url="https://outline.example/doc/1", + jobs={BEN: "done", CARL: "dead"}, + ) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert [s.id for s in queue.sessions] == [session_id] + assert queue.sessions[0].dead == 1 + + +async def test_another_guild_s_unfinished_work_is_not_this_guild_s_business( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, guild_id=OTHER_GUILD, jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.sessions == () + assert queue.pending == 0 + + +# --------------------------------------------------------------------------- +# What each row says +# --------------------------------------------------------------------------- + + +async def test_a_session_carries_the_counts_it_is_made_of( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, jobs={ANNA: "done", BEN: "pending", CARL: "dead"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + session = queue.sessions[0] + assert (session.pending, session.running, session.done, session.dead) == (1, 0, 1, 1) + + +async def test_a_session_with_no_jobs_reports_zeroes_rather_than_nothing( + factory: async_sessionmaker[AsyncSession], +) -> None: + # A recording in progress has no jobs yet, and a row with absent + # counts would render as a gap where the numbers go. + await a_session(factory, status="open", jobs={}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + session = queue.sessions[0] + assert (session.pending, session.running, session.done, session.dead) == (0, 0, 0, 0) + + +async def test_a_session_carries_the_channel_name_it_opened_under( + factory: async_sessionmaker[AsyncSession], +) -> None: + # A web page has no `<#id>` to resolve, so it needs the stored name -- + # the same one every other console view shows. + await a_session(factory, channel_name="planning", jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.sessions[0].channel_name == "planning" + + +async def test_the_newest_session_comes_first( + factory: async_sessionmaker[AsyncSession], +) -> None: + old = await a_session(factory, started_at=T0 - timedelta(days=2), jobs={BEN: "pending"}) + new = await a_session(factory, started_at=T0 - timedelta(hours=1), jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert [s.id for s in queue.sessions] == [new, old] + + +# --------------------------------------------------------------------------- +# The totals, and the caveats that travel with them +# --------------------------------------------------------------------------- + + +async def test_an_expired_lease_is_counted_against_the_lease_that_was_assumed( + factory: async_sessionmaker[AsyncSession], +) -> None: + """A `running` job whose lease expired is one whose worker died holding it. + + No amount of waiting fixes it, which is why it is the number worth + reading first -- and why the lease it was measured against is sent + with it rather than left implicit. + """ + await a_session( + factory, + jobs={BEN: "running"}, + claimed_at=T0 - timedelta(hours=2), + ) + + queue = await overview(factory, lease_seconds=600.0).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.running_past_lease == 1 + assert queue.lease_seconds == 600.0 + + +async def test_a_job_claimed_a_moment_ago_is_not_past_its_lease( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, jobs={BEN: "running"}, claimed_at=T0 - timedelta(seconds=30)) + + queue = await overview(factory, lease_seconds=600.0).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.running_past_lease == 0 + + +async def test_a_closed_session_with_no_document_and_nothing_queued_is_counted_as_stuck( + factory: async_sessionmaker[AsyncSession], +) -> None: + # Nothing is queued for it and nothing will happen on its own, which is + # the one state that needs a person rather than patience. + await a_session(factory, status="closed", jobs={BEN: "done"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.closed_undocumented == 1 + + +async def test_the_oldest_pending_work_is_dated_by_its_session_s_end( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`transcription_job` has no enqueue timestamp at all. + + A session's end is within seconds of when its jobs were created, which + is close enough to answer "has something been sitting here for hours?" + -- the only question the figure exists for. + """ + ended = T0 - timedelta(hours=5) + await a_session( + factory, started_at=ended - timedelta(hours=1), ended_at=ended, jobs={BEN: "pending"} + ) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.oldest_pending_session_ended_at == ended + + +async def test_a_guild_with_nothing_pending_has_no_oldest_anything( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, status="documented", document_url="u", jobs={BEN: "done"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.oldest_pending_session_ended_at is None + + +# --------------------------------------------------------------------------- +# Cutting the list +# --------------------------------------------------------------------------- + + +async def test_a_long_list_is_cut_and_says_so( + factory: async_sessionmaker[AsyncSession], +) -> None: + """A guild broken for a month has hundreds, and hundreds is unreadable. + + What matters is that "twenty" never reads as "twenty exist" -- so the + cut is reported rather than merely applied. + """ + for day in range(ACTIVE_SESSION_LIMIT + 3): + await a_session(factory, started_at=T0 - timedelta(days=day), jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert len(queue.sessions) == ACTIVE_SESSION_LIMIT + assert queue.truncated is True + # The totals are not cut with the list: the counts are the guild's, + # however many sessions fit on the page. + assert queue.pending == ACTIVE_SESSION_LIMIT + 3 + + +async def test_a_list_that_fits_is_not_reported_as_cut( + factory: async_sessionmaker[AsyncSession], +) -> None: + for day in range(3): + await a_session(factory, started_at=T0 - timedelta(days=day), jobs={BEN: "pending"}) + + queue = await overview(factory).for_guild(GUILD, requested_by=ANNA) + + assert queue is not None + assert queue.truncated is False + + +async def test_exactly_the_limit_is_not_reported_as_cut( + factory: async_sessionmaker[AsyncSession], +) -> None: + # The off-by-one worth pinning: the query asks for one more than it + # will show precisely so this case answers `False`. + for day in range(ACTIVE_SESSION_LIMIT): + await a_session(factory, started_at=T0 - timedelta(days=day), jobs={BEN: "pending"}) + + sessions, truncated = await load_active_sessions(factory, GUILD) + + assert len(sessions) == ACTIVE_SESSION_LIMIT + assert truncated is False + + +async def test_a_guild_with_nothing_outstanding_returns_an_empty_list( + factory: async_sessionmaker[AsyncSession], +) -> None: + sessions, truncated = await load_active_sessions(factory, GUILD) + + assert sessions == [] + assert truncated is False diff --git a/tests/console/test_queue_routes.py b/tests/console/test_queue_routes.py index 41c6928..8fac357 100644 --- a/tests/console/test_queue_routes.py +++ b/tests/console/test_queue_routes.py @@ -23,16 +23,24 @@ from aiohttp.test_utils import TestClient from sturnus.console.app import SESSION_COOKIE -from sturnus.console.ports import QueueSnapshot, QueueSpeaker, RequeueOutcome +from sturnus.console.ports import ( + GuildQueue, + QueuedSession, + QueueSnapshot, + QueueSpeaker, + RequeueOutcome, +) from sturnus.console.session import SessionCookie, SignedSession from tests.console.conftest import ( ANNA, BEN, + GUILD, SECRET, SESSION, T0, AiohttpClientFactory, FakeQueue, + FakeQueueOverview, build_test_api, ) @@ -238,3 +246,181 @@ async def test_nothing_in_between_may_cache_a_queue_snapshot( client = await signed_in(aiohttp_client, build_test_api(queue=FakeQueue(snapshot=snapshot()))) response = await client.get(status_url()) assert response.headers["Cache-Control"] == "private, no-store" + + +# --------------------------------------------------------------------------- +# The guild-wide view: what is outstanding, and where +# --------------------------------------------------------------------------- + + +def queued(**over: object) -> QueuedSession: + base: dict[str, object] = { + "id": SESSION, + "channel_id": 555, + "channel_name": "meeting", + "started_at": T0, + "ended_at": None, + "status": "closed", + "document_url": None, + "pending": 2, + "running": 1, + "done": 0, + "dead": 0, + } + base.update(over) + return QueuedSession(**base) # type: ignore[arg-type] + + +def guild_queue(**over: object) -> GuildQueue: + base: dict[str, object] = { + "pending": 2, + "running": 1, + "done": 40, + "dead": 1, + "running_past_lease": 0, + "oldest_pending_session_ended_at": T0, + "closed_undocumented": 0, + "lease_seconds": 1800.0, + "sessions": (queued(),), + "truncated": False, + } + base.update(over) + return GuildQueue(**base) # type: ignore[arg-type] + + +def guild_url(guild_id: int | str = GUILD) -> str: + return f"/api/guilds/{guild_id}/queue" + + +async def test_an_administrator_sees_what_their_guild_still_owes( + aiohttp_client: AiohttpClientFactory, +) -> None: + overview = FakeQueueOverview(queue=guild_queue()) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + response = await client.get(guild_url()) + + assert response.status == 200 + body = await response.json() + assert body["counts"] == {"pending": 2, "running": 1, "done": 40, "dead": 1} + assert body["sessions"][0]["counts"]["pending"] == 2 + + +async def test_the_overview_asks_on_behalf_of_the_signed_in_person( + aiohttp_client: AiohttpClientFactory, +) -> None: + # The whole authorisation model is that the id reaching the overview + # comes out of the signed cookie. A handler passing anything else would + # look identical from outside. + overview = FakeQueueOverview(queue=guild_queue()) + client = await signed_in(aiohttp_client, build_test_api(queues=overview), as_user=BEN) + + await client.get(guild_url()) + + assert overview.asked == [(GUILD, BEN)] + + +async def test_a_guild_this_person_does_not_administer_does_not_exist( + aiohttp_client: AiohttpClientFactory, +) -> None: + # 404 and not 403, for the reason the session endpoints answer 404: the + # list names when a guild met and in which channel, and a 403 confirms + # such a list exists to somebody just established as having no business + # with it. + client = await signed_in(aiohttp_client, build_test_api(queues=FakeQueueOverview())) + + response = await client.get(guild_url()) + + assert response.status == 404 + assert (await response.json())["error"] == "no such guild" + + +async def test_a_guild_id_that_is_not_a_number_gets_the_same_refusal( + aiohttp_client: AiohttpClientFactory, +) -> None: + overview = FakeQueueOverview(queue=guild_queue()) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + assert (await client.get(guild_url("nope"))).status == 404 + # And never reached the overview, so a malformed path cannot be used to + # find out which guild ids are well formed. + assert overview.asked == [] + + +async def test_the_overview_needs_a_session_like_every_other_endpoint( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await aiohttp_client(build_test_api(queues=FakeQueueOverview(queue=guild_queue()))) + assert (await client.get(guild_url())).status == 401 + + +async def test_the_lease_travels_with_the_count_it_produced( + aiohttp_client: AiohttpClientFactory, +) -> None: + """`running_past_lease` is derived from an assumed lease. + + The lease that actually applies is the worker's `job_lease_seconds`, + which this process cannot see. Sending the number it used is what lets + the console name it rather than presenting a derived count as a fact -- + the same caveat `/queue status` prints in Discord. + """ + overview = FakeQueueOverview(queue=guild_queue(running_past_lease=3, lease_seconds=600.0)) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + body = await (await client.get(guild_url())).json() + + assert body["running_past_lease"] == 3 + assert body["lease_seconds"] == 600.0 + + +async def test_a_cut_list_says_that_it_was_cut( + aiohttp_client: AiohttpClientFactory, +) -> None: + # Otherwise a page showing twenty sessions reads as "there are twenty", + # which for a guild that has been broken for a month is the opposite of + # the truth. + overview = FakeQueueOverview(queue=guild_queue(truncated=True)) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + assert (await (await client.get(guild_url())).json())["truncated"] is True + + +async def test_a_guild_with_nothing_outstanding_is_an_empty_list( + aiohttp_client: AiohttpClientFactory, +) -> None: + # Empty and not 404: "everything here is finished" and "this is not + # your guild" are different answers and must look different. + overview = FakeQueueOverview( + queue=guild_queue(pending=0, running=1, sessions=(), oldest_pending_session_ended_at=None) + ) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + body = await (await client.get(guild_url())).json() + + assert body["sessions"] == [] + assert body["oldest_pending_session_ended_at"] is None + + +async def test_every_id_in_the_overview_travels_as_a_string( + aiohttp_client: AiohttpClientFactory, +) -> None: + big = 308_000_000_000_000_001 + overview = FakeQueueOverview(queue=guild_queue(sessions=(queued(channel_id=big),))) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + body = await (await client.get(guild_url())).json() + + assert body["guild_id"] == str(GUILD) + assert body["sessions"][0]["channel_id"] == str(big) + assert body["sessions"][0]["id"] == str(SESSION) + + +async def test_nothing_in_between_may_cache_the_overview( + aiohttp_client: AiohttpClientFactory, +) -> None: + overview = FakeQueueOverview(queue=guild_queue()) + client = await signed_in(aiohttp_client, build_test_api(queues=overview)) + + response = await client.get(guild_url()) + + assert response.headers["Cache-Control"] == "private, no-store" From 183cd07fe0512c5e124044e746ee80af79a59ff2 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:13:05 +0200 Subject: [PATCH 06/14] feat(console): a Queue page showing what a guild's pipeline still owes The Admin View's third entry, and the interface half of the guild-wide queue endpoint. It shows the four lifecycle counts, the three figures that mean somebody has to act, and the unfinished sessions the counts are made of. Rows are ordered by how much attention they need, then newest first. "Needs a person" -- a dead job, or a closed session with nothing queued and no document -- comes above work that is merely moving, and a dead job outranks even a written document: a session that reached `documented` with a permanently failed speaker in it looks finished from every other angle, and this page is the only place anybody finds out otherwise. Whether a session is still recording is decided by `ended_at` rather than by its status string, so an unrecognised status cannot turn a live meeting into an unexplained row of zeros. Newest first within every rank, including the stuck ones. Somebody opens this page because a team has just said "this morning's meeting has no protocol", and a list that counts backwards in one rank and forwards in another cannot be scanned at all. How long the backlog has been there is what the oldest-pending line says in words, without reordering anything. Three numbers carry their caveats in the sentence rather than in a footnote, because all three would otherwise be read as something they are not: the expired-lease count is measured against the lease this process assumed and not the worker's real one, the oldest-pending figure is dated by the session's end because `transcription_job` records no enqueue time at all, and the list is a window on the backlog rather than its size. There is no re-queue control here. Each row links to the recording page, which has one -- so whether a redo is safe stays a single decision in `plan_requeue` rather than becoming two. Polling follows `RequeuePanel`'s shape: a chain of timeouts rather than an interval, cleared on unmount, client-only. It runs while `pending` or `running` is non-zero and ignores `dead`, because a dead job never changes on its own and polling for it would wait forever on news that cannot come. --- console/app/pages/admin/queue.vue | 539 ++++++++++++++++++ console/app/utils/navigation.ts | 11 + console/app/utils/queue.ts | 916 ++++++++++++++++++++++++++++++ console/test/navigation.spec.ts | 24 +- console/test/queue.spec.ts | 776 +++++++++++++++++++++++++ 5 files changed, 2263 insertions(+), 3 deletions(-) create mode 100644 console/app/pages/admin/queue.vue create mode 100644 console/app/utils/queue.ts create mode 100644 console/test/queue.spec.ts diff --git a/console/app/pages/admin/queue.vue b/console/app/pages/admin/queue.vue new file mode 100644 index 0000000..9328d0e --- /dev/null +++ b/console/app/pages/admin/queue.vue @@ -0,0 +1,539 @@ + + + diff --git a/console/app/utils/navigation.ts b/console/app/utils/navigation.ts index ddb22fa..72f440a 100644 --- a/console/app/utils/navigation.ts +++ b/console/app/utils/navigation.ts @@ -85,6 +85,17 @@ export const ADMIN_VIEW: NavSection = { icon: 'M16 11a3 3 0 1 0 0-6 3 3 0 0 0 0 6Zm-8 0a3 3 0 1 0 0-6 3 3 0 0 0 0 6Zm0 2c-2.33 0-7 1.17-7 3.5V19h14v-2.5c0-2.33-4.67-3.5-7-3.5Zm8 0c-.29 0-.62.02-.97.05 1.16.84 1.97 1.97 1.97 3.45V19h6v-2.5c0-2.33-4.67-3.5-7-3.5Z', adminOnly: true, }, + { + to: '/admin/queue', + label: 'Queue', + // A stack of layers waiting their turn, which is what a queue of + // transcription jobs is. Deliberately not a clock or an hourglass: + // both would say "this takes time", and the reason anybody opens + // this page is that something has stopped taking time and started + // taking none. + icon: 'M12 2 2 7l10 5 10-5-10-5Zm0 20 10-5-2.5-1.25L12 19.5 4.5 15.75 2 17l10 5Zm0-5.5 10-5-2.5-1.25L12 14 4.5 10.25 2 11.5l10 5Z', + adminOnly: true, + }, ], } diff --git a/console/app/utils/queue.ts b/console/app/utils/queue.ts new file mode 100644 index 0000000..e3ac6c5 --- /dev/null +++ b/console/app/utils/queue.ts @@ -0,0 +1,916 @@ +/** + * Where a guild's transcription work stands, and which of it a person has + * to do something about. + * + * A module rather than expressions in the page, for the same reason + * `~/utils/consents` is one: every function here is a *decision* -- which + * row comes first, whether a row is waiting or stuck, what a caveat reads + * like as a sentence, whether an empty page is good news -- and a decision + * embedded in a template can only be tested by rendering one. + * + * Four facts govern the wording of everything below, and none of them are + * softened anywhere in this file. Each of them is a way this page could + * quietly lie to somebody who trusted it: + * + * - **The four lifecycle counts are guild-wide and cover all time.** They + * are not the sum of the sessions listed underneath them, and a reader + * who adds the rows up and gets a different number must find the reason + * on the page rather than conclude the page is broken. + * - **`running_past_lease` is measured against a lease this process cannot + * see.** The API assumes `lease_seconds`; the lease that actually + * applies is the *worker's* `job_lease_seconds`, in the worker's own + * environment. A job running under a raised lease is perfectly healthy + * and is still counted here, so the figure is always presented with the + * number it was measured against -- never as a fact about dead workers. + * - **The oldest pending figure is dated by a session's end, not by a + * job.** `transcription_job` has no enqueue timestamp at all. A session + * ends within seconds of its jobs being created, which is close enough + * to answer "has something been sitting here for hours?", and it is not + * close at all after a re-queue: a reset job keeps its session's + * original end and therefore reads older than it is. + * - **The list is cut.** A page showing twenty sessions reads as "there + * are twenty" unless it says otherwise, and the one thing an + * administrator wants from a backlog page is its size. + * + * Nothing here decides whether a re-queue is allowed. That decision lives + * on the recording page, once, in `~/utils/recordings` and the panel that + * uses it -- a second implementation of "when is a redo safe" is a second + * answer to it, and the two would drift. + */ +import { formatDuration } from '~/utils/duration' +import { formatCount, formatMoment } from '~/utils/format' + +/* -------------------------------------------------------------------- */ +/* What the API describes */ +/* -------------------------------------------------------------------- */ + +/** The transcription lifecycle, in the order a job moves through it. Kept + * as one shape rather than four sibling fields because that order is the + * point: it is how a reader finds where work is piling up. */ +export interface QueueCounts { + pending: number + running: number + done: number + dead: number +} + +/** + * One session the pipeline has not finished with. + * + * "Unfinished" is the API's definition and is wider than it sounds: a + * session counts as unfinished when it is not `documented` **or** when it + * carries a `dead` job. A session can reach `documented` with a + * permanently failed speaker inside it, and that speaker is exactly who + * somebody needs to notice. + */ +export interface QueuedSession { + /** A string, always. Session ids are database integers today and follow + * the snowflake rule anyway: two id shapes in one payload is how the + * one that matters gets parsed with the wrong one. */ + id: string + channel_id: string + /** Null when Sturnus has no name for the channel -- usually one deleted + * since the meeting. */ + channel_name: string | null + started_at: string + /** Null while the session is still being recorded. */ + ended_at: string | null + /** `open`, `closed` or `documented`. Deliberately a plain string: a + * fourth value added to the API must render as itself rather than + * silently as the friendliest of the three. */ + status: string + document_url: string | null + counts: QueueCounts +} + +export interface GuildQueue { + /** Null only when the payload named no guild, which the page uses to + * refuse to show one server's queue under another's heading. */ + guild_id: string | null + /** Guild-wide, across all time. See the module comment. */ + counts: QueueCounts + running_past_lease: number + /** ISO instant, or null when nothing is pending at all. */ + oldest_pending_session_ended_at: string | null + closed_undocumented: number + /** The lease `running_past_lease` was measured against, in seconds. */ + lease_seconds: number + truncated: boolean + sessions: QueuedSession[] +} + +/* -------------------------------------------------------------------- */ +/* Reading what the API sent */ +/* -------------------------------------------------------------------- */ + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** `null` stays `null`; anything else becomes the string it prints as. Ids + * are strings on the wire, and a number that arrived instead has already + * lost whatever precision it was going to lose. */ +function asText(value: unknown): string | null { + if (value === null || value === undefined) return null + const text = typeof value === 'string' ? value : String(value) + return text.trim() === '' ? null : text +} + +/** + * A count that can be printed. + * + * Anything absent, negative or not a number is a defect upstream, and + * rendering it as "-3 pending" would put the defect in front of the reader + * as though it were a fact about their server. Zero is the honest floor: + * it says "none", which is as far as this console can vouch. + */ +function asCount(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return 0 + return Math.round(value) +} + +function asCounts(value: unknown): QueueCounts { + const raw = isRecord(value) ? value : {} + return { + pending: asCount(raw.pending), + running: asCount(raw.running), + done: asCount(raw.done), + dead: asCount(raw.dead), + } +} + +/** + * The lease the past-lease figure was measured against. + * + * Zero and nonsense both become `null` rather than `0`, because the whole + * job of this number on screen is to name what the count means. "past the + * 0-second lease" would name nothing and would read as though every + * running job were overdue, which is the opposite of what it says. + */ +function asLease(value: unknown): number | null { + if (typeof value !== 'number' || !Number.isFinite(value) || value <= 0) return null + return value +} + +/** + * The queue in a payload. + * + * Always yields a well-formed value, never null. A page that has to + * distinguish "the API refused" from "the API answered something odd" + * already has the thrown `ApiError` for the first; a parser that returned + * null for the second would turn a strange payload into a blank page with + * no error anywhere, which is the failure mode hardest to report. + */ +export function parseGuildQueue(payload: unknown): GuildQueue { + const raw = isRecord(payload) ? payload : {} + const sessions = Array.isArray(raw.sessions) ? raw.sessions : [] + return { + guild_id: asText(raw.guild_id), + counts: asCounts(raw.counts), + running_past_lease: asCount(raw.running_past_lease), + oldest_pending_session_ended_at: asText(raw.oldest_pending_session_ended_at), + closed_undocumented: asCount(raw.closed_undocumented), + lease_seconds: asLease(raw.lease_seconds) ?? 0, + // `=== true` rather than truthiness: a missing flag must read as "the + // list is whole". Erring the other way would put a warning about a + // hidden backlog on every complete page, and a warning that is always + // there is one nobody reads on the day it is true. + truncated: raw.truncated === true, + sessions: sessions.flatMap((entry) => { + if (!isRecord(entry)) return [] + const id = asText(entry.id) + // A session with no id has no recording page to link to, and the + // link is the only thing this page offers per row. + if (!id) return [] + return [ + { + id, + channel_id: asText(entry.channel_id) ?? '', + channel_name: asText(entry.channel_name), + started_at: asText(entry.started_at) ?? '', + ended_at: asText(entry.ended_at), + status: asText(entry.status) ?? '', + document_url: asText(entry.document_url), + counts: asCounts(entry.counts), + }, + ] + }), + } +} + +/** Where a guild's queue is read from. The id is escaped: it is a string + * from an API, and a string allowed to contain a slash is a string + * allowed to address a different endpoint. */ +export function queuePath(guildId: string): string { + return `/guilds/${encodeURIComponent(guildId)}/queue` +} + +/* -------------------------------------------------------------------- */ +/* Naming a session */ +/* -------------------------------------------------------------------- */ + +/** What to call the channel a session happened in. The whole id when there + * is no name -- never a shortened one, since snowflakes minted in the + * same era share their leading digits, and a truncated id is something + * nobody can search a page for either. */ +export function queueChannelLabel(session: QueuedSession): string { + const name = session.channel_name?.trim() + if (name) return `#${name}` + return session.channel_id ? `Channel ${session.channel_id}` : 'An unnamed channel' +} + +/** + * The line under a row whose channel has no name, or `null` when it has + * one. + * + * A bare snowflake where every other row carries a `#name` reads as a + * fault in the console. It is not one: the name is looked up in Discord, + * and a channel deleted since the meeting has none left to look up. + * Saying so is the difference between "this row is broken" and "the + * channel this meeting happened in is gone". + */ +export function queueChannelNote(session: QueuedSession): string | null { + if (session.channel_name?.trim()) return null + return ( + 'Sturnus has no name for this channel. Channel names are read from Discord, so a channel ' + + 'deleted since the meeting leaves only its id behind — the recording itself is unaffected.' + ) +} + +/** When the recording started, always in UTC and saying so: the server + * render cannot know the reader's zone, and a second rendering in the + * browser would disagree with the first. */ +export function sessionStartLine(session: QueuedSession): string { + if (!session.started_at) return 'When this session started was not recorded.' + return `Started ${formatMoment(session.started_at)}.` +} + +/* -------------------------------------------------------------------- */ +/* What state a session is in */ +/* -------------------------------------------------------------------- */ + +/** + * Three tones, and they are about the reader rather than about the data. + * + * `alarm` means nothing will change here until a person does something. + * `watch` means the pipeline has it and the only useful act is to wait. + * `clear` means there is nothing to do and no bad news either. Rendering + * "a speaker failed for good" and "a worker is on it" in the same colour + * would hide the one distinction this page exists to draw. + */ +export type QueueTone = 'clear' | 'watch' | 'alarm' + +/** + * What kind of row this is. + * + * - `needs-person` — nothing queued will move it on. A dead job, or a + * closed session with nothing running and no document. + * - `moving` — a job is pending or running. A worker will get to it. + * - `recording` — the meeting is happening right now. It has no jobs yet + * because they are created when the recording ends, so its row of zeros + * is not an absence of work but the absence of a reason for work. + * - `finishing` — every job is done and a document exists; only the + * session's own status has not caught up. + */ +export type QueueAttention = 'needs-person' | 'moving' | 'recording' | 'finishing' + +/** + * The lifecycle, once, in the order a job moves through it. + * + * One list rather than four field names written out at each call site, so + * a session row and the summary band can never end up ordering the same + * four counts differently -- which is the one thing that would make the + * two halves of this page disagree about what it is showing. + */ +const LIFECYCLE: readonly { key: keyof QueueCounts, label: string }[] = [ + { key: 'pending', label: 'Pending' }, + { key: 'running', label: 'Running' }, + { key: 'done', label: 'Done' }, + { key: 'dead', label: 'Dead' }, +] + +/** Jobs a worker may still act on. `done` and `dead` are both terminal; + * only one of them is good news, which is why they are never counted + * together anywhere in this file. */ +function inFlight(counts: QueueCounts): number { + return counts.pending + counts.running +} + +function totalJobs(counts: QueueCounts): number { + return counts.pending + counts.running + counts.done + counts.dead +} + +/** + * Which of the four kinds of row this is. + * + * The order of the tests is the point. A dead job outranks everything, + * including a document: a session that reached `documented` with a + * permanently failed speaker looks finished from every other angle, and + * the only place anybody will find out is here. + * + * `ended_at` rather than `status === 'open'` decides whether a session is + * live, because that is the fact rather than a label for it -- and a + * status string this console does not recognise must not turn a running + * meeting into an unexplained row of zeros. + */ +export function queueAttention(session: QueuedSession): QueueAttention { + if (session.counts.dead > 0) return 'needs-person' + if (inFlight(session.counts) > 0) return 'moving' + if (!session.ended_at) return 'recording' + if (!session.document_url) return 'needs-person' + return 'finishing' +} + +export interface SessionState { + tone: QueueTone + label: string + /** The long form, said in full on the row rather than hidden behind a + * tooltip: which of these four a row is decides what to do next, and + * nobody hovers to find that out. */ + detail: string +} + +function jobs(count: number): string { + return count === 1 ? '1 job' : `${formatCount(count)} jobs` +} + +/** + * What this row's state is, in a badge and a sentence. + * + * Every sentence says what will happen next without anybody doing + * anything, because that is the question a queue page is read to answer. + * Where the answer is "nothing", it says so in those words. + */ +export function queueSessionState(session: QueuedSession): SessionState { + const { counts } = session + if (counts.dead > 0) { + const failed + = counts.dead === 1 + ? 'One speaker in this session failed for the last time' + : `${formatCount(counts.dead)} speakers in this session failed for the last time` + const document = session.document_url + ? ' A protocol was written anyway, without them, so nothing about this session looks wrong ' + + 'until somebody reads it and finds a voice missing.' + : '' + return { + tone: 'alarm', + label: + counts.dead === 1 + ? '1 speaker failed for good' + : `${formatCount(counts.dead)} speakers failed for good`, + detail: + `${failed} and will not be retried on their own.${document} Open the recording to see ` + + 'which speaker and why; re-queueing it there is the only thing that starts them again.', + } + } + + if (inFlight(counts) > 0) { + if (counts.running > 0) { + const waiting + = counts.pending > 0 ? ` ${jobs(counts.pending)} behind it are still waiting.` : '' + return { + tone: 'watch', + label: 'Being transcribed', + detail: + `A worker has ${jobs(counts.running)} from this session in hand right now.${waiting} ` + + 'Nothing needs doing unless the figures stop changing.', + } + } + return { + tone: 'watch', + label: 'Waiting for a worker', + detail: + `${jobs(counts.pending)} queued and none running. They start as soon as a worker is free; ` + + 'if this never changes, no worker is taking work at all.', + } + } + + if (!session.ended_at) { + return { + tone: 'clear', + label: 'Recording now', + detail: + 'This meeting is being recorded at this moment. It has no transcription jobs yet — they ' + + 'are created when the recording ends — so the zeros beside it mean there is nothing to ' + + 'do, not that something went missing.', + } + } + + if (!session.document_url) { + if (totalJobs(counts) === 0) { + return { + tone: 'alarm', + label: 'Nothing was ever queued', + detail: + 'The recording finished and no transcription job was ever created for it: nobody in the ' + + 'channel had consented, or the recording captured no audio. No worker has anything to ' + + 'do here and no protocol will appear on its own.', + } + } + return { + tone: 'alarm', + label: 'Closed with nothing queued', + detail: + `The recording finished, all ${jobs(totalJobs(counts))} came back done, and no protocol ` + + 'was written. Nothing is queued for this session, so nothing about it will change on its ' + + 'own — somebody has to re-queue it from the recording.', + } + } + + return { + tone: 'clear', + label: 'Waiting to be marked done', + detail: + 'Every job finished and the protocol exists. Only the session\'s own status has not caught ' + + 'up yet, which it does on its own; there is nothing to do here.', + } +} + +/** The four counts of one row, in lifecycle order and already worded. The + * page loops this rather than naming the four fields itself, so a row and + * the summary band can never end up ordering them differently. */ +export function sessionCounts(session: QueuedSession): { key: string, label: string, value: string }[] { + return LIFECYCLE.map(({ key, label }) => ({ + key, + label, + value: formatCount(session.counts[key]), + })) +} + +/* -------------------------------------------------------------------- */ +/* The order the sessions are listed in */ +/* -------------------------------------------------------------------- */ + +/** + * Rank by what the reader can still do about a row. + * + * 0 -- needs a person: nothing queued will move it on, and no amount of + * waiting changes that. These are the only rows whose position on the + * page decides whether they are seen at all. + * 1 -- moving: a worker has it or will. Second because this is what + * somebody watching a backlog drain came to watch. + * 2 -- recording now: nothing to do, and worth seeing — it is the reason a + * channel that ought to be producing work is not. + * 3 -- finishing: done in every way that matters, waiting on a flag. + */ +function attentionRank(session: QueuedSession): number { + switch (queueAttention(session)) { + case 'needs-person': + return 0 + case 'moving': + return 1 + case 'recording': + return 2 + default: + return 3 + } +} + +/** An instant as a number, or `null` when there is nothing to compare. */ +function instantOf(value: string | null): number | null { + if (!value) return null + const parsed = Date.parse(value) + return Number.isNaN(parsed) ? null : parsed +} + +/** + * Session ids in numeric order without ever becoming numbers. + * + * Shorter is smaller, and equal lengths compare digit by digit. Comparing + * them as plain strings would put "1000" after "999", and comparing them + * as numbers would round any id past the safe integer range into a + * different id -- which session ids do not reach today and channel ids do, + * so the same rule is used for both rather than two rules that differ by + * which field they are applied to. + */ +function compareIds(a: string, b: string): number { + if (a.length !== b.length) return a.length - b.length + if (a === b) return 0 + return a < b ? -1 : 1 +} + +/** + * The order the rows are listed in. + * + * Rows needing a person first (see `attentionRank`), then newest first + * inside each rank, then by id. + * + * Newest first *within every rank*, including the stuck ones, and + * deliberately not oldest-first for those. It is tempting to surface the + * longest-rotting session at the very top, and it would be the wrong page + * to do it on: somebody arrives here because a team has just said "our + * meeting from this morning has no protocol", and they scan for the + * meeting they were told about. A list that counts backwards in one rank + * and forwards in another cannot be scanned at all. How long the backlog + * has been there is a question the oldest-pending figure in the summary + * band answers, in one line, without reordering anything. + * + * A session whose `started_at` could not be parsed sorts to the end of its + * rank rather than to an arbitrary place in the middle -- it is still a + * row worth seeing, and it has no claim to a position among the ones that + * carry a real time. + * + * Every comparison ends at the id, which is unique, so the order is total: + * two sessions recorded in parallel channels never swap places between + * renders. + */ +export function orderQueueSessions(sessions: readonly QueuedSession[]): QueuedSession[] { + return [...sessions].sort((a, b) => { + const byRank = attentionRank(a) - attentionRank(b) + if (byRank !== 0) return byRank + + const left = instantOf(a.started_at) + const right = instantOf(b.started_at) + if (left !== null && right !== null) { + if (left !== right) return right - left + } else if (left !== null || right !== null) { + return left !== null ? -1 : 1 + } + + // Higher id last-created, so newest first here too, which keeps the + // tiebreak pointing the same way as the comparison above it. + return compareIds(b.id, a.id) + }) +} + +/** How many rows nothing will move without a person. The headline figure + * for the list: twelve unfinished sessions where three are stuck says + * something a bare row count does not. */ +export function needsPersonCount(sessions: readonly QueuedSession[]): number { + return sessions.filter((session) => queueAttention(session) === 'needs-person').length +} + +/** The line above the list, naming what it holds and how much of it is + * actually somebody's problem. */ +export function sessionsSummaryLine(sessions: readonly QueuedSession[]): string { + const total = sessions.length + if (total === 0) return 'No unfinished sessions are listed for this server.' + const stuck = needsPersonCount(sessions) + const noun = total === 1 ? 'session' : 'sessions' + if (stuck === 0) { + return ( + `${formatCount(total)} unfinished ${noun} here, and none of them is waiting on a person — ` + + 'every one is either being worked on or still being recorded.' + ) + } + const verb = stuck === 1 ? 'needs' : 'need' + return ( + `${formatCount(total)} unfinished ${noun} here; ${formatCount(stuck)} of them ${verb} ` + + 'somebody, because nothing queued will move them on. Those are listed first.' + ) +} + +/* -------------------------------------------------------------------- */ +/* The four lifecycle counts */ +/* -------------------------------------------------------------------- */ + +export interface LifecycleFigure { + key: string + label: string + value: string + /** What this stage means, in the reader's terms. */ + note: string + tone: QueueTone +} + +/** + * What the four counts are counting. + * + * Stated once, above the figures, and not left to be inferred. These are + * guild-wide totals over all time; the sessions below are a list of the + * unfinished ones only. A reader who adds up the rows and gets a different + * number has found the difference between the two, not a fault, and the + * page has to be the thing that tells them so. + */ +export const LIFECYCLE_SCOPE_NOTE = + 'These four count every transcription job this server has ever had, in the order a job moves ' + + 'through them: pending, then running, then done — or dead, once it has failed for the last ' + + 'time. They are totals for the whole server across all time, not a sum of the sessions listed ' + + 'below, so the two will not add up and are not meant to.' + +/** + * The four counts, in lifecycle order, with what each stage means. + * + * `dead` is the only one whose note changes with its value, because it is + * the only one where zero is worth saying out loud: "nothing has failed + * for good" is news, and an unexplained 0 next to three other numbers is + * not. + */ +export function lifecycleFigures(queue: GuildQueue): LifecycleFigure[] { + const notes: Record = { + pending: 'Queued, waiting for a worker to pick them up.', + running: 'A worker has these in hand right now.', + done: 'Transcribed successfully, counted for as long as the job row exists.', + dead: + queue.counts.dead > 0 + ? 'Failed for the last time. These are not retried on their own; each one is a speaker ' + + 'missing from a protocol until somebody re-queues their session.' + : 'Nothing in this server has failed for good.', + } + return LIFECYCLE.map(({ key, label }) => ({ + key, + label, + value: formatCount(queue.counts[key]), + note: notes[key], + tone: key === 'dead' && queue.counts.dead > 0 ? 'alarm' : 'clear', + })) +} + +/* -------------------------------------------------------------------- */ +/* The three figures somebody has to act on */ +/* -------------------------------------------------------------------- */ + +/** The assumed lease, written the way the Discord `/queue status` reply + * writes it, so the two readouts of the same number agree. */ +function leaseInWords(queue: GuildQueue): string { + if (queue.lease_seconds <= 0) return 'the lease the API assumed' + return `the ${Math.round(queue.lease_seconds)}-second lease` +} + +/** + * How many running jobs have outlived their lease, and what that is worth. + * + * The caveat is in both branches, not only the alarming one. The count is + * derived from an *assumed* lease: the API process cannot see the worker's + * `job_lease_seconds`, so a raised lease makes a healthy job look overdue + * and a lowered one hides an overdue job entirely. A zero reported without + * the caveat would read as "no worker has died", which this figure cannot + * establish. + */ +export function pastLeaseLine(queue: GuildQueue): string { + const lease = leaseInWords(queue) + const caveat = + `The lease that actually applies is the worker's own job_lease_seconds, which the API process ` + + 'cannot see, so this is measured against the lease it assumed rather than the real one.' + if (queue.running_past_lease === 0) { + return ( + `No running job has been held longer than ${lease}. ${caveat} A worker running under a ` + + 'raised lease would still be counted here, so a zero is reassuring rather than conclusive.' + ) + } + const held + = queue.running_past_lease === 1 + ? 'One running job has been held' + : `${formatCount(queue.running_past_lease)} running jobs have been held` + return ( + `${held} longer than ${lease}. ${caveat} If the worker's lease is not higher than that, the ` + + 'worker holding these died and another may already have reclaimed the job — no amount of ' + + 'waiting fixes that, which is why this is the figure to read first.' + ) +} + +/** + * Sessions that are closed, have nothing queued and still have no + * document. + * + * Nothing is pending for them, nothing is running for them, and nothing + * will start on its own. Kept separate from the lifecycle counts because + * it is not a count of jobs at all -- it is a count of meetings whose + * protocol nobody is going to get unless somebody asks for it. + */ +export function undocumentedLine(queue: GuildQueue): string { + if (queue.closed_undocumented === 0) { + return ( + 'Every closed session in this server either has its protocol or still has work queued for ' + + 'it. Nothing is sitting finished and unwritten.' + ) + } + const sessions + = queue.closed_undocumented === 1 + ? 'One closed session has' + : `${formatCount(queue.closed_undocumented)} closed sessions have` + return ( + `${sessions} no unfinished jobs left and still no protocol. Nothing is queued for them and ` + + 'nothing will start on its own, so each one waits for a person to open its recording and ' + + 'ask for the transcription again.' + ) +} + +/** An age in words, or `null` when there is nothing to measure it + * against. `now` is a parameter rather than a call to `Date.now()` inside + * this module for two reasons: a pure function is testable, and the page + * only has a clock after it has mounted -- a server render and a browser + * render a second apart would otherwise disagree about the text of the + * same paragraph, which Vue reports as a hydration mismatch. */ +function ageInWords(iso: string, now: number | null): string | null { + if (now === null) return null + const at = instantOf(iso) + if (at === null) return null + const seconds = Math.floor((now - at) / 1000) + if (seconds < 0) return null + return formatDuration(seconds) +} + +/** + * How long the oldest waiting job has been waiting -- with the two things + * that figure is not. + * + * It is not a job timestamp: `transcription_job` records no enqueue time + * at all, so this is dated by the *session's* end, which is within seconds + * of when its jobs were created. And it is not the age of a re-queued job: + * a reset job keeps its session's original end and therefore reads older + * than it is. Both are said every time the figure is shown, because a + * number labelled "oldest pending" that is quietly a different number is + * worse than no number. + */ +export function oldestPendingLine(queue: GuildQueue, now: number | null): string { + const ended = queue.oldest_pending_session_ended_at + if (!ended) { + return 'Nothing is waiting: this server has no job in pending at all.' + } + const age = ageInWords(ended, now) + const since = age === null ? '' : ` — ${age} ago` + return ( + `The oldest job still waiting belongs to a session that ended ${formatMoment(ended)}${since}. ` + + `This is dated by the session's end rather than by the job: transcription_job records no ` + + 'enqueue time at all, and a session ends within seconds of its jobs being created. A ' + + "re-queued job keeps its session's original end, so after a re-queue this reads older than " + + 'the job itself.' + ) +} + +export interface AttentionItem { + key: string + label: string + /** The figure itself, short enough to be scanned in a band of three. */ + value: string + /** The sentence, including the caveat that makes the figure honest. */ + detail: string + tone: QueueTone +} + +/** + * The three figures that mean somebody has to do something, kept apart + * from the four that merely describe the pipeline. + * + * They are shown whether or not they are zero. A row that appears only + * when it is bad news is a row whose absence has to be interpreted, and + * "there is no warning about dead workers" and "this page does not warn + * about dead workers" look identical on screen. + */ +export function attentionItems(queue: GuildQueue, now: number | null): AttentionItem[] { + return [ + { + key: 'past-lease', + label: 'Running past their lease', + value: formatCount(queue.running_past_lease), + detail: pastLeaseLine(queue), + tone: queue.running_past_lease > 0 ? 'alarm' : 'clear', + }, + { + key: 'closed-undocumented', + label: 'Closed with no protocol', + value: formatCount(queue.closed_undocumented), + detail: undocumentedLine(queue), + tone: queue.closed_undocumented > 0 ? 'alarm' : 'clear', + }, + { + key: 'oldest-pending', + label: 'Oldest job waiting', + value: queue.oldest_pending_session_ended_at + ? formatMoment(queue.oldest_pending_session_ended_at) + : 'Nothing waiting', + detail: oldestPendingLine(queue, now), + tone: queue.oldest_pending_session_ended_at ? 'watch' : 'clear', + }, + ] +} + +/* -------------------------------------------------------------------- */ +/* What the list does not show */ +/* -------------------------------------------------------------------- */ + +/** + * The notice above a list that was cut short, or `null` when it was not. + * + * The number of rows is read off the list rather than written into the + * sentence, because the server's limit is the server's to change and a + * sentence naming a number the API no longer uses is a sentence that lies + * without anybody editing it. + */ +export function truncationNotice(queue: GuildQueue): string | null { + if (!queue.truncated) return null + const shown = queue.sessions.length + const listed + = shown === 1 + ? 'Only one unfinished session is listed' + : `Only the newest ${formatCount(shown)} unfinished sessions are listed` + return ( + `${listed}; this server has more. Sturnus cuts the list rather than sending an unbounded one, ` + + 'so what is below is a window on the backlog and not its size. The four job counts above ' + + 'are guild-wide and do count all of it.' + ) +} + +/* -------------------------------------------------------------------- */ +/* Whether anything is happening at all */ +/* -------------------------------------------------------------------- */ + +/** + * Whether work is actually moving, which is what decides whether the page + * keeps polling. + * + * Read from the guild-wide `pending` and `running` rather than from the + * listed sessions, because the list is cut and the counts are not: a guild + * with a hundred pending jobs and twenty listed sessions is still moving + * even if every listed row happens to be finished. + * + * `dead` deliberately does not count. A dead job never changes again on + * its own, and polling for it would be a page that reloads for ever + * waiting for news that cannot arrive. + */ +export function isQueueMoving(queue: GuildQueue): boolean { + return inFlight(queue.counts) > 0 +} + +/** + * Whether there is genuinely nothing outstanding in this server. + * + * Stricter than "the list is empty". A guild can have no unfinished + * sessions listed and still have a dead worker holding a lease or a closed + * meeting with no protocol, and reporting that as "all clear" is exactly + * the reassurance nobody should be given. Historical `done` and `dead` + * counts are not consulted: they describe what has happened, not what is + * outstanding, and a server that once had a failure is not permanently + * unwell. + */ +export function isQueueClear(queue: GuildQueue): boolean { + return ( + queue.sessions.length === 0 + && !queue.truncated + && inFlight(queue.counts) === 0 + && queue.running_past_lease === 0 + && queue.closed_undocumented === 0 + && queue.oldest_pending_session_ended_at === null + ) +} + +/** The empty state, written as the good news it is. A queue page with + * nothing on it is the state everybody wants and the state that looks + * most like a broken page, so it says which of the two it is. */ +export const CLEAR_QUEUE_HEADING = 'Nothing is outstanding in this server' + +export const CLEAR_QUEUE_NOTE = + 'Every session has been transcribed and written up, no job is waiting or running, no worker is ' + + 'holding one past its lease, and no closed meeting is missing its protocol. There is nothing ' + + 'to do here — this page is worth coming back to when somebody says a protocol has not ' + + 'appeared.' + +/* -------------------------------------------------------------------- */ +/* When the API says no */ +/* -------------------------------------------------------------------- */ + +/** `ApiError` names it `status`; a raw `$fetch` failure may name it + * `statusCode`; a request that never got a response has neither, and null + * says so rather than standing in a number that would read as an + * answer. */ +function statusOf(error: unknown): number | null { + if (!isRecord(error)) return null + for (const candidate of [error.status, error.statusCode]) { + if (typeof candidate === 'number' && Number.isFinite(candidate)) { + // `ApiError` uses 0 for "never reached the API", which is + // deliberately distinguishable from every real status. + return candidate === 0 ? null : candidate + } + } + return null +} + +/** + * A failed request, in a sentence somebody can act on. + * + * Built from the status alone. `useApi` throws `ApiError`, which carries + * no body by design -- the API's own `{"error": "no such guild"}` never + * reaches this console -- so every sentence below has to stand on its own + * without it. + * + * Named `describeQueueError` rather than `describeError` for the same + * reason `describeConsentError` is: everything under `app/utils` is + * auto-imported into every component, and two exports sharing a name is a + * build warning and a coin toss over which one a page actually gets. + */ +export function describeQueueError(error: unknown): string { + const status = statusOf(error) + switch (status) { + case 401: + return 'Your session has ended. Sign in again to see this server’s queue.' + case 403: + return ( + 'You do not administer this server. Administrators are the members holding the role named ' + + 'by that guild’s `admin_role_id`.' + ) + case 404: + // The API answers 404 both for a guild that does not exist and for + // one the caller does not administer, on purpose: it will not + // confirm the existence of a server to somebody with no business + // there. So this sentence has to cover both without guessing which. + return ( + 'Sturnus does not know this server, or you no longer administer it — it answers the same ' + + 'way to both. Reload the page; the list of servers is rebuilt from Discord.' + ) + case null: + return 'Could not reach the API. Nothing here is out of date on purpose; check the connection and retry.' + default: + return `Sturnus answered ${status} and could not report this server’s queue. Nothing is known about why.` + } +} diff --git a/console/test/navigation.spec.ts b/console/test/navigation.spec.ts index 35017c1..9399cae 100644 --- a/console/test/navigation.spec.ts +++ b/console/test/navigation.spec.ts @@ -68,11 +68,29 @@ describe('who is offered the Admin View', () => { expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('User Settings') }) - it('lists what the bot does before what the members have agreed to', () => { + it('offers the queue to an administrator', () => { + expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('Queue') + }) + + it('hides the queue from somebody who administers nothing', () => { + // The queue endpoint answers 404 to a non-administrator -- the same + // answer it gives for a guild that does not exist -- so an entry left + // visible would offer a page that can only ever refuse them. + expect(visibleEntries(PARTICIPANT).map((e) => e.label)).not.toContain('Queue') + expect(visibleEntries(null).map((e) => e.label)).not.toContain('Queue') + }) + + it('lists the daily work first and the thing gone wrong last', () => { // The configuration is what an administrator comes to the Admin View // for daily; a member's consent is looked at when somebody asks about - // that one member, which is rarer and always deliberate. - expect(ADMIN_VIEW.entries.map((e) => e.label)).toEqual(['Bot Settings', 'User Settings']) + // that one member, which is rarer and always deliberate. The queue is + // rarer still and is never scanned -- it is opened because a protocol + // did not appear, which is a question somebody arrives holding. + expect(ADMIN_VIEW.entries.map((e) => e.label)).toEqual([ + 'Bot Settings', + 'User Settings', + 'Queue', + ]) }) it('hides it from somebody who administers nothing', () => { diff --git a/console/test/queue.spec.ts b/console/test/queue.spec.ts new file mode 100644 index 0000000..5844db6 --- /dev/null +++ b/console/test/queue.spec.ts @@ -0,0 +1,776 @@ +/** + * What a guild's transcription queue means, and what the page is allowed + * to claim about it. + * + * All of it lives in `~/utils/queue` rather than in the page, because + * every one of these is a decision -- which row comes first, whether a row + * is stuck or merely waiting, whether an empty page is good news -- and a + * decision embedded in a template can only be tested by rendering one. + * + * The wording is asserted here on purpose, and heavily. Three of the + * figures this page shows are *derived*, and each one is derived from + * something the API cannot actually see: + * + * - `running_past_lease` is measured against an assumed lease, not the + * worker's own `job_lease_seconds`. + * - `oldest_pending_session_ended_at` is a session's end standing in for a + * job's enqueue time, which does not exist in the schema at all. + * - the session list is cut, so its length is not the size of the backlog. + * + * A test that only checked the numbers would let any of those caveats + * quietly drop out of the page, and the first person to notice would be an + * administrator who restarted a healthy worker because a console told them + * it was dead. + */ +import { describe, expect, it } from 'vitest' + +import { + CLEAR_QUEUE_NOTE, + LIFECYCLE_SCOPE_NOTE, + attentionItems, + describeQueueError, + isQueueClear, + isQueueMoving, + lifecycleFigures, + needsPersonCount, + oldestPendingLine, + orderQueueSessions, + parseGuildQueue, + pastLeaseLine, + queueAttention, + queueChannelLabel, + queueChannelNote, + queuePath, + queueSessionState, + sessionCounts, + sessionStartLine, + sessionsSummaryLine, + truncationNotice, + undocumentedLine, + type GuildQueue, + type QueueCounts, + type QueuedSession, +} from '../app/utils/queue' + +/** A session with everything harmless, so each test states only the one + * property it is actually about. Closed, transcribed and written up: the + * state nothing on this page has anything to say about. */ +function session(overrides: Partial & { id: string }): QueuedSession { + return { + channel_id: '555', + channel_name: 'meeting', + started_at: '2026-08-21T12:00:00+00:00', + ended_at: '2026-08-21T13:00:00+00:00', + status: 'documented', + document_url: 'https://outline.example/doc', + counts: { pending: 0, running: 0, done: 4, dead: 0 }, + ...overrides, + } +} + +function counts(overrides: Partial = {}): QueueCounts { + return { pending: 0, running: 0, done: 0, dead: 0, ...overrides } +} + +/** A guild with nothing outstanding, for the same reason. */ +function queue(overrides: Partial = {}): GuildQueue { + return { + guild_id: '4711', + counts: counts(), + running_past_lease: 0, + oldest_pending_session_ended_at: null, + closed_undocumented: 0, + lease_seconds: 1800, + truncated: false, + sessions: [], + ...overrides, + } +} + +/** What `ApiError` looks like to the function that reads a failure. */ +function failure(status: number) { + return { status, path: '/guilds/4711/queue' } +} + +/** A fixed clock, so an age in a sentence is a fact about the fixture and + * not about when the suite happened to run. */ +const NOW = Date.parse('2026-08-21T16:20:00+00:00') + +describe('reading the queue payload', () => { + it('reads the whole envelope the endpoint sends', () => { + const parsed = parseGuildQueue({ + guild_id: '4711', + counts: { pending: 2, running: 1, done: 40, dead: 1 }, + running_past_lease: 0, + oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00', + closed_undocumented: 0, + lease_seconds: 1800.0, + truncated: false, + sessions: [ + { + id: '4711', + channel_id: '555', + channel_name: 'meeting', + started_at: '2026-08-21T12:00:00+00:00', + ended_at: '2026-08-21T13:00:00+00:00', + status: 'closed', + document_url: null, + counts: { pending: 2, running: 1, done: 0, dead: 0 }, + }, + ], + }) + expect(parsed.guild_id).toBe('4711') + expect(parsed.counts).toEqual({ pending: 2, running: 1, done: 40, dead: 1 }) + expect(parsed.lease_seconds).toBe(1800) + expect(parsed.sessions).toHaveLength(1) + expect(parsed.sessions[0]!.counts.pending).toBe(2) + expect(parsed.sessions[0]!.document_url).toBeNull() + }) + + it('keeps every id a string even if it arrived as a number', () => { + // A channel snowflake past the safe integer range has already lost its + // last digits before this function sees it. Stringifying does not undo + // that; it keeps the recording link well-formed so the damage surfaces + // as a 404 rather than as somebody opening a different meeting. + const parsed = parseGuildQueue({ sessions: [{ id: 12, channel_id: 555 }] }) + expect(parsed.sessions[0]!.id).toBe('12') + expect(parsed.sessions[0]!.channel_id).toBe('555') + }) + + it('drops a session with no id at all', () => { + // The link to its recording is the only thing a row offers, and a row + // that cannot be opened is a row that can only be misread. + expect(parseGuildQueue({ sessions: [{ channel_id: '555' }, { id: '1' }] }).sessions).toHaveLength(1) + }) + + it('treats a missing name as no name rather than as an empty one', () => { + expect(parseGuildQueue({ sessions: [{ id: '1' }] }).sessions[0]!.channel_name).toBeNull() + expect( + parseGuildQueue({ sessions: [{ id: '1', channel_name: ' ' }] }).sessions[0]!.channel_name, + ).toBeNull() + }) + + it('never reports a negative or nonsensical count', () => { + // A defect upstream must not render as "-3 pending" beside a server's + // name, where it reads as a fact about that server. + const parsed = parseGuildQueue({ + counts: { pending: -3, running: 'many', done: 2.4, dead: null }, + running_past_lease: -1, + closed_undocumented: 'lots', + }) + expect(parsed.counts).toEqual({ pending: 0, running: 0, done: 2, dead: 0 }) + expect(parsed.running_past_lease).toBe(0) + expect(parsed.closed_undocumented).toBe(0) + }) + + it('treats a missing truncation flag as a whole list', () => { + // Erring the other way would put a warning about a hidden backlog on + // every complete page, and a warning that is always there is one + // nobody reads on the day it is true. + expect(parseGuildQueue({}).truncated).toBe(false) + expect(parseGuildQueue({ truncated: 'true' }).truncated).toBe(false) + expect(parseGuildQueue({ truncated: true }).truncated).toBe(true) + }) + + it('yields a well-formed queue for a payload it cannot make sense of', () => { + // Never null: a parser that gave up would turn a strange payload into + // a blank page with no error anywhere, which is the failure mode + // hardest to report. + const parsed = parseGuildQueue('nonsense') + expect(parsed.guild_id).toBeNull() + expect(parsed.sessions).toEqual([]) + expect(parsed.counts).toEqual({ pending: 0, running: 0, done: 0, dead: 0 }) + }) + + it('escapes the guild id in the path it builds', () => { + // A string from an API allowed to contain a slash is a string allowed + // to address a different endpoint. + expect(queuePath('4711')).toBe('/guilds/4711/queue') + expect(queuePath('../guilds/1')).toBe('/guilds/..%2Fguilds%2F1/queue') + }) +}) + +describe('naming a session', () => { + it('names the channel it happened in', () => { + expect(queueChannelLabel(session({ id: '1', channel_name: 'meeting' }))).toBe('#meeting') + }) + + it('falls back to the whole channel id, never a shortened one', () => { + // Snowflakes minted in the same era share their leading digits, so a + // truncated id names a group rather than a channel. + expect( + queueChannelLabel(session({ id: '1', channel_name: null, channel_id: '1129384756123456789' })), + ).toBe('Channel 1129384756123456789') + }) + + it('says why a row has an id instead of a name', () => { + // A bare snowflake where every other row carries a #name reads as a + // fault in the console. It is not one. + const note = queueChannelNote(session({ id: '1', channel_name: null })) + expect(note).toContain('deleted since the meeting') + expect(note).toContain('the recording itself is unaffected') + }) + + it('says nothing extra about a row that has a name', () => { + expect(queueChannelNote(session({ id: '1', channel_name: 'meeting' }))).toBeNull() + }) + + it('writes the start in UTC and says which zone that is', () => { + // The server render cannot know the reader's zone, so a second + // rendering in the browser would disagree with the first. + expect(sessionStartLine(session({ id: '1' }))).toBe('Started 21 Aug 2026, 12:00 UTC.') + }) + + it('says outright when there is no start time rather than printing a dash', () => { + expect(sessionStartLine(session({ id: '1', started_at: '' }))).toContain('was not recorded') + }) + + it('lists a row’s four counts in lifecycle order', () => { + const rendered = sessionCounts( + session({ id: '1', counts: counts({ pending: 2, running: 1, done: 3, dead: 4 }) }), + ) + expect(rendered.map((c) => c.label)).toEqual(['Pending', 'Running', 'Done', 'Dead']) + expect(rendered.map((c) => c.value)).toEqual(['2', '1', '3', '4']) + }) +}) + +describe('what state a session is in', () => { + it('calls a session with a permanently failed speaker out first', () => { + const state = queueSessionState( + session({ id: '1', counts: counts({ done: 3, dead: 1 }), document_url: null }), + ) + expect(state.tone).toBe('alarm') + expect(state.label).toBe('1 speaker failed for good') + expect(state.detail).toContain('will not be retried on their own') + expect(state.detail).toContain('re-queueing it there') + }) + + it('says a documented session can still be missing a voice', () => { + // A session reaches `documented` with a dead job in it, looks finished + // from every other angle, and this page is the only place anybody + // finds out. That is the whole reason such a session is listed at all. + const state = queueSessionState( + session({ + id: '1', + status: 'documented', + document_url: 'https://outline.example/doc', + counts: counts({ done: 3, dead: 1 }), + }), + ) + expect(state.tone).toBe('alarm') + expect(state.detail).toContain('A protocol was written anyway, without them') + expect(state.detail).toContain('finds a voice missing') + }) + + it('counts the failed speakers rather than saying "some"', () => { + const state = queueSessionState(session({ id: '1', counts: counts({ dead: 3 }) })) + expect(state.label).toBe('3 speakers failed for good') + }) + + it('reads a running job as work in hand, not as a problem', () => { + const state = queueSessionState( + session({ id: '1', counts: counts({ pending: 2, running: 1 }), document_url: null }), + ) + expect(state.tone).toBe('watch') + expect(state.label).toBe('Being transcribed') + expect(state.detail).toContain('2 jobs behind it are still waiting') + expect(state.detail).toContain('unless the figures stop changing') + }) + + it('distinguishes queued-and-nobody-working from queued-and-being-worked', () => { + // Pending with nothing running is the shape of "no worker is taking + // work at all", which is a different thing to look at than a busy one. + const state = queueSessionState( + session({ id: '1', counts: counts({ pending: 2 }), document_url: null }), + ) + expect(state.label).toBe('Waiting for a worker') + expect(state.detail).toContain('none running') + expect(state.detail).toContain('no worker is taking work at all') + }) + + it('reads an open session with no jobs as a recording happening right now', () => { + // The one row of zeros on this page that is not an absence of work but + // the absence of a reason for work. Presenting it as "nothing to do" + // would hide a live meeting. + const state = queueSessionState( + session({ id: '1', status: 'open', ended_at: null, document_url: null, counts: counts() }), + ) + expect(state.tone).toBe('clear') + expect(state.label).toBe('Recording now') + expect(state.detail).toContain('being recorded at this moment') + expect(state.detail).toContain('created when the recording ends') + }) + + it('trusts the missing end time over an unrecognised status string', () => { + // A status this console does not know must not turn a running meeting + // into an unexplained row of zeros. + expect( + queueAttention( + session({ id: '1', status: 'something-new', ended_at: null, document_url: null, counts: counts() }), + ), + ).toBe('recording') + }) + + it('says nothing will happen on its own for a closed session with no protocol', () => { + const state = queueSessionState( + session({ id: '1', status: 'closed', document_url: null, counts: counts({ done: 4 }) }), + ) + expect(state.tone).toBe('alarm') + expect(state.label).toBe('Closed with nothing queued') + expect(state.detail).toContain('nothing about it will change on its own') + expect(state.detail).toContain('re-queue it from the recording') + }) + + it('separates a session that never had a job from one whose jobs all finished', () => { + // "Nobody consented" and "everything transcribed but nothing was + // written up" both look like a closed session with no protocol, and + // only one of them can be fixed by asking for the work again. + const state = queueSessionState( + session({ id: '1', status: 'closed', document_url: null, counts: counts() }), + ) + expect(state.label).toBe('Nothing was ever queued') + expect(state.detail).toContain('nobody in the channel had consented') + expect(state.detail).toContain('no protocol will appear on its own') + }) + + it('treats a written-up session waiting on its own flag as nothing to do', () => { + const state = queueSessionState( + session({ id: '1', status: 'closed', counts: counts({ done: 4 }) }), + ) + expect(state.tone).toBe('clear') + expect(state.label).toBe('Waiting to be marked done') + expect(state.detail).toContain('there is nothing to do here') + }) + + it('never leaves a row without a state', () => { + // Every combination the API can send has to land somewhere, because a + // row with an empty badge is a row whose meaning the reader invents. + for (const ended of [null, '2026-08-21T13:00:00+00:00']) { + for (const document of [null, 'https://outline.example/doc']) { + for (const c of [counts(), counts({ pending: 1 }), counts({ running: 1 }), counts({ dead: 1 })]) { + const state = queueSessionState(session({ id: '1', ended_at: ended, document_url: document, counts: c })) + expect(state.label.trim()).not.toBe('') + expect(state.detail.trim()).not.toBe('') + } + } + } + }) +}) + +describe('the order the sessions are listed in', () => { + it('puts the rows nothing will move without a person first', () => { + const ordered = orderQueueSessions([ + session({ id: '1', counts: counts({ pending: 1 }), document_url: null }), + session({ id: '2', counts: counts({ dead: 1 }) }), + ]) + expect(ordered.map((s) => s.id)).toEqual(['2', '1']) + }) + + it('ranks stuck, then moving, then recording, then merely unflagged', () => { + const ordered = orderQueueSessions([ + session({ id: 'flagging', status: 'closed', counts: counts({ done: 2 }) }), + session({ id: 'live', status: 'open', ended_at: null, document_url: null, counts: counts() }), + session({ id: 'moving', document_url: null, counts: counts({ running: 1 }) }), + session({ id: 'stuck', document_url: null, counts: counts({ dead: 1 }) }), + ]) + expect(ordered.map((s) => s.id)).toEqual(['stuck', 'moving', 'live', 'flagging']) + }) + + it('lists the newest first inside a rank, in every rank alike', () => { + // Deliberately not oldest-first for the stuck rows. Somebody arrives + // here because a team has just said "this morning's meeting has no + // protocol", and they scan for the meeting they were told about; a + // list that counts backwards in one rank and forwards in another + // cannot be scanned at all. How long the backlog has been there is + // what the oldest-pending figure answers instead. + const ordered = orderQueueSessions([ + session({ id: 'old', started_at: '2026-08-01T09:00:00+00:00', counts: counts({ dead: 1 }) }), + session({ id: 'new', started_at: '2026-08-21T09:00:00+00:00', counts: counts({ dead: 1 }) }), + ]) + expect(ordered.map((s) => s.id)).toEqual(['new', 'old']) + }) + + it('sinks a session with an unreadable start to the end of its rank', () => { + // Still a row worth seeing, and with no claim to a position among the + // rows that carry a real time. + const ordered = orderQueueSessions([ + session({ id: 'broken', started_at: 'not a date', counts: counts({ dead: 1 }) }), + session({ id: 'old', started_at: '2026-08-01T09:00:00+00:00', counts: counts({ dead: 1 }) }), + ]) + expect(ordered.map((s) => s.id)).toEqual(['old', 'broken']) + }) + + it('never leaves two sessions recorded in parallel in an arbitrary order', () => { + // Every comparison ends at the id, which is unique, so the order is + // total and the rows do not swap places between renders. + const twice = () => + orderQueueSessions([ + session({ id: '9', counts: counts({ dead: 1 }) }), + session({ id: '10', counts: counts({ dead: 1 }) }), + ]).map((s) => s.id) + expect(twice()).toEqual(['10', '9']) + expect(twice()).toEqual(['10', '9']) + }) + + it('orders tied ids numerically rather than as strings', () => { + // As plain strings "1000" sorts before "999", which would put an older + // session above a newer one at the exact moment the tiebreak is all + // there is to go on. + const ordered = orderQueueSessions([ + session({ id: '999', counts: counts({ dead: 1 }) }), + session({ id: '1000', counts: counts({ dead: 1 }) }), + ]) + expect(ordered.map((s) => s.id)).toEqual(['1000', '999']) + }) + + it('leaves the list it was given alone', () => { + const given = [session({ id: '1' }), session({ id: '2', counts: counts({ dead: 1 }) })] + orderQueueSessions(given) + expect(given.map((s) => s.id)).toEqual(['1', '2']) + }) +}) + +describe('how much of the list is somebody’s problem', () => { + it('counts only the rows nothing queued will move on', () => { + expect( + needsPersonCount([ + session({ id: '1', counts: counts({ dead: 1 }) }), + session({ id: '2', document_url: null, status: 'closed', counts: counts({ done: 1 }) }), + session({ id: '3', document_url: null, counts: counts({ running: 1 }) }), + session({ id: '4', status: 'open', ended_at: null, document_url: null, counts: counts() }), + ]), + ).toBe(2) + }) + + it('says how many rows need a person and that they are listed first', () => { + const line = sessionsSummaryLine([ + session({ id: '1', counts: counts({ dead: 1 }) }), + session({ id: '2', document_url: null, counts: counts({ running: 1 }) }), + ]) + expect(line).toContain('2 unfinished sessions here') + expect(line).toContain('1 of them needs somebody') + expect(line).toContain('Those are listed first.') + }) + + it('says outright when none of them is waiting on a person', () => { + const line = sessionsSummaryLine([session({ id: '1', document_url: null, counts: counts({ running: 1 }) })]) + expect(line).toContain('1 unfinished session here') + expect(line).toContain('none of them is waiting on a person') + }) + + it('has something to say about an empty list', () => { + expect(sessionsSummaryLine([])).toBe('No unfinished sessions are listed for this server.') + }) +}) + +describe('the four lifecycle counts', () => { + it('renders them in the order a job moves through them', () => { + const figures = lifecycleFigures(queue({ counts: counts({ pending: 2, running: 1, done: 40, dead: 1 }) })) + expect(figures.map((f) => f.label)).toEqual(['Pending', 'Running', 'Done', 'Dead']) + expect(figures.map((f) => f.value)).toEqual(['2', '1', '40', '1']) + }) + + it('says that they are guild-wide and not a sum of the list below', () => { + // A reader who adds up the rows and gets a different number has found + // the difference between the two, not a fault, and the page has to be + // the thing that tells them so. + expect(LIFECYCLE_SCOPE_NOTE).toContain('across all time') + expect(LIFECYCLE_SCOPE_NOTE).toContain('not a sum of the sessions listed') + expect(LIFECYCLE_SCOPE_NOTE).toContain('pending, then running, then done') + }) + + it('marks a dead count that is not zero, and only that one', () => { + const withDead = lifecycleFigures(queue({ counts: counts({ dead: 1, done: 9 }) })) + expect(withDead.map((f) => f.tone)).toEqual(['clear', 'clear', 'clear', 'alarm']) + const withoutDead = lifecycleFigures(queue({ counts: counts({ done: 9 }) })) + expect(withoutDead.every((f) => f.tone === 'clear')).toBe(true) + }) + + it('says what a zero in the dead column means rather than leaving it bare', () => { + const figures = lifecycleFigures(queue()) + expect(figures[3]!.note).toBe('Nothing in this server has failed for good.') + }) + + it('explains what a dead job costs somebody', () => { + const figures = lifecycleFigures(queue({ counts: counts({ dead: 2 }) })) + expect(figures[3]!.note).toContain('not retried on their own') + expect(figures[3]!.note).toContain('a speaker missing from a protocol') + }) + + it('gives every stage a note, so no figure is left to be guessed at', () => { + for (const figure of lifecycleFigures(queue())) { + expect(figure.note.trim()).not.toBe('') + } + }) +}) + +describe('the jobs running past their lease', () => { + it('names the lease the count was measured against', () => { + // The whole point. The lease that actually applies is the worker's own + // job_lease_seconds, which the API process cannot see. + const line = pastLeaseLine(queue({ running_past_lease: 2, lease_seconds: 1800 })) + expect(line).toContain('2 running jobs have been held longer than the 1800-second lease') + expect(line).toContain("the worker's own job_lease_seconds, which the API process cannot see") + expect(line).toContain('measured against the lease it assumed rather than the real one') + }) + + it('says what it means if the worker’s lease is not higher', () => { + const line = pastLeaseLine(queue({ running_past_lease: 1 })) + expect(line).toContain('One running job has been held') + expect(line).toContain('the worker holding these died') + expect(line).toContain('no amount of waiting fixes that') + }) + + it('keeps the caveat when the count is zero', () => { + // A zero reported without it would read as "no worker has died", which + // this figure cannot establish: a raised lease hides an overdue job. + const line = pastLeaseLine(queue({ running_past_lease: 0 })) + expect(line).toContain('No running job has been held longer than the 1800-second lease') + expect(line).toContain("the worker's own job_lease_seconds, which the API process cannot see") + expect(line).toContain('reassuring rather than conclusive') + }) + + it('refuses to name a lease it was not given', () => { + // "past the 0-second lease" would read as though every running job + // were overdue, which is the opposite of what the figure says. + const line = pastLeaseLine(queue({ running_past_lease: 1, lease_seconds: 0 })) + expect(line).toContain('longer than the lease the API assumed') + expect(line).not.toContain('0-second') + }) + + it('rounds a fractional lease rather than printing it', () => { + expect(pastLeaseLine(queue({ running_past_lease: 1, lease_seconds: 1800.0 }))).toContain( + '1800-second lease', + ) + }) +}) + +describe('the closed sessions with no protocol', () => { + it('says that nothing is queued and nothing will start', () => { + const line = undocumentedLine(queue({ closed_undocumented: 3 })) + expect(line).toContain('3 closed sessions have no unfinished jobs left and still no protocol') + expect(line).toContain('nothing will start on its own') + expect(line).toContain('waits for a person') + }) + + it('reads a single one as one', () => { + expect(undocumentedLine(queue({ closed_undocumented: 1 }))).toContain('One closed session has') + }) + + it('reads the zero as the good news it is', () => { + const line = undocumentedLine(queue()) + expect(line).toContain('Nothing is sitting finished and unwritten.') + }) +}) + +describe('the oldest job still waiting', () => { + it('says the figure is dated by the session’s end, not by the job', () => { + // `transcription_job` records no enqueue time at all. Calling this a + // job age would be inventing a column. + const line = oldestPendingLine( + queue({ oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00' }), + NOW, + ) + expect(line).toContain('a session that ended 21 Aug 2026, 13:00 UTC') + expect(line).toContain("dated by the session's end rather than by the job") + expect(line).toContain('transcription_job records no enqueue time at all') + }) + + it('says that a re-queued job reads older than it is', () => { + const line = oldestPendingLine( + queue({ oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00' }), + NOW, + ) + expect(line).toContain("keeps its session's original end") + expect(line).toContain('reads older than the job itself') + }) + + it('turns the instant into an age once there is a clock to compare it to', () => { + expect( + oldestPendingLine(queue({ oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00' }), NOW), + ).toContain('— 3 h 20 min ago') + }) + + it('omits the age entirely when there is no clock yet', () => { + // The server render has no reader's clock, and a paragraph whose text + // differs between the two renders is a hydration mismatch. The moment + // is shown in both; the age arrives after mounting. + const line = oldestPendingLine( + queue({ oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00' }), + null, + ) + expect(line).toContain('a session that ended 21 Aug 2026, 13:00 UTC.') + expect(line).not.toContain('ago') + }) + + it('omits an age that would be negative rather than printing one', () => { + // A clock skew between the API host and the reader's machine is not + // something to render as "-2 min ago". + expect( + oldestPendingLine( + queue({ oldest_pending_session_ended_at: '2026-08-21T17:00:00+00:00' }), + NOW, + ), + ).not.toContain('ago') + }) + + it('says nothing is waiting rather than showing a dash', () => { + expect(oldestPendingLine(queue(), NOW)).toBe( + 'Nothing is waiting: this server has no job in pending at all.', + ) + }) +}) + +describe('the three figures somebody has to act on', () => { + it('shows all three whether or not they are zero', () => { + // A row that appears only when it is bad news is a row whose absence + // has to be interpreted, and "there is no warning" and "this page does + // not warn" look identical on screen. + expect(attentionItems(queue(), NOW).map((i) => i.key)).toEqual([ + 'past-lease', + 'closed-undocumented', + 'oldest-pending', + ]) + }) + + it('puts the number worth reading first at the front', () => { + // No amount of waiting fixes a job whose worker died holding it. + expect(attentionItems(queue(), NOW)[0]!.label).toBe('Running past their lease') + }) + + it('raises the tone only on the figures that are not zero', () => { + const calm = attentionItems(queue(), NOW) + expect(calm.map((i) => i.tone)).toEqual(['clear', 'clear', 'clear']) + const loud = attentionItems( + queue({ + running_past_lease: 1, + closed_undocumented: 2, + oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00', + }), + NOW, + ) + expect(loud.map((i) => i.tone)).toEqual(['alarm', 'alarm', 'watch']) + }) + + it('gives the oldest-pending figure a value that says nothing is waiting', () => { + expect(attentionItems(queue(), NOW)[2]!.value).toBe('Nothing waiting') + }) + + it('carries the caveat into the detail of every figure', () => { + for (const item of attentionItems(queue({ running_past_lease: 1 }), NOW)) { + expect(item.detail.trim()).not.toBe('') + expect(item.label.trim()).not.toBe('') + expect(item.value.trim()).not.toBe('') + } + }) +}) + +describe('what the list does not show', () => { + it('says the list was cut and that its length is not the backlog', () => { + // Otherwise a page showing twenty sessions reads as "there are + // twenty", which is the one question a backlog page is opened with. + const notice = truncationNotice( + queue({ truncated: true, sessions: Array.from({ length: 20 }, (_, i) => session({ id: String(i) })) }), + ) + expect(notice).toContain('Only the newest 20 unfinished sessions are listed; this server has more.') + expect(notice).toContain('a window on the backlog and not its size') + expect(notice).toContain('The four job counts above are guild-wide and do count all of it.') + }) + + it('reads the number of rows off the list rather than naming the server’s limit', () => { + // The limit is the server's to change, and a sentence naming a number + // the API no longer uses is a sentence that lies without anybody + // editing it. + expect(truncationNotice(queue({ truncated: true, sessions: [session({ id: '1' })] }))).toContain( + 'Only one unfinished session is listed; this server has more.', + ) + }) + + it('says nothing at all about a list that is whole', () => { + expect(truncationNotice(queue({ sessions: [session({ id: '1' })] }))).toBeNull() + }) +}) + +describe('whether anything is happening at all', () => { + it('polls while a job is pending or running', () => { + expect(isQueueMoving(queue({ counts: counts({ pending: 1 }) }))).toBe(true) + expect(isQueueMoving(queue({ counts: counts({ running: 1 }) }))).toBe(true) + }) + + it('stops polling for a dead job, which will never change on its own', () => { + // A page that reloaded for ever waiting for news that cannot arrive is + // a load generator, not a status page. + expect(isQueueMoving(queue({ counts: counts({ dead: 3 }) }))).toBe(false) + expect(isQueueMoving(queue({ counts: counts({ done: 40 }) }))).toBe(false) + }) + + it('reads the guild-wide counts rather than the listed rows', () => { + // The list is cut and the counts are not: a guild with jobs queued is + // still moving even if every row that fitted happens to be finished. + expect( + isQueueMoving(queue({ counts: counts({ pending: 5 }), truncated: true, sessions: [session({ id: '1' })] })), + ).toBe(true) + }) + + it('calls a server clear only when all six ways of being unwell are absent', () => { + expect(isQueueClear(queue({ counts: counts({ done: 40 }) }))).toBe(true) + expect(isQueueClear(queue({ sessions: [session({ id: '1' })] }))).toBe(false) + expect(isQueueClear(queue({ counts: counts({ pending: 1 }) }))).toBe(false) + expect(isQueueClear(queue({ running_past_lease: 1 }))).toBe(false) + expect(isQueueClear(queue({ closed_undocumented: 1 }))).toBe(false) + expect(isQueueClear(queue({ truncated: true }))).toBe(false) + expect(isQueueClear(queue({ oldest_pending_session_ended_at: '2026-08-21T13:00:00+00:00' }))).toBe(false) + }) + + it('does not treat a server that once had a failure as permanently unwell', () => { + // `done` and `dead` describe what has happened, not what is + // outstanding, and a page that never went green again would stop being + // read. + expect(isQueueClear(queue({ counts: counts({ done: 40, dead: 2 }) }))).toBe(true) + }) + + it('writes the empty state as the good news it is', () => { + // A queue page with nothing on it is the state everybody wants and the + // state that looks most like a broken page. + expect(CLEAR_QUEUE_NOTE).toContain('no worker is holding one past its lease') + expect(CLEAR_QUEUE_NOTE).toContain('There is nothing to do here') + }) +}) + +describe('when the API says no', () => { + it('sends somebody back to sign in on a 401', () => { + expect(describeQueueError(failure(401))).toContain('Sign in again') + }) + + it('names where administrator status comes from on a 403', () => { + expect(describeQueueError(failure(403))).toContain('admin_role_id') + }) + + it('covers both meanings of a 404 without guessing which', () => { + // The API answers 404 for a guild that does not exist and for one the + // caller does not administer alike, on purpose: it will not confirm + // the existence of a server to somebody with no business there. + const message = describeQueueError(failure(404)) + expect(message).toContain('does not know this server, or you no longer administer it') + expect(message).toContain('answers the same way to both') + }) + + it('distinguishes a refusal from never reaching the API at all', () => { + // `ApiError` uses 0 for "never got a response", which must not print + // as "Sturnus answered 0". + expect(describeQueueError(failure(0))).toContain('Could not reach the API') + expect(describeQueueError(null)).toContain('Could not reach the API') + }) + + it('names an unexpected status rather than inventing a reason for it', () => { + expect(describeQueueError(failure(503))).toContain('Sturnus answered 503') + }) + + it('never echoes anything the failure carried with it', () => { + // `useApi` strips the body off every failed request on purpose, so an + // in-cluster hostname can never reach the hydration payload. Nothing + // here may reintroduce one from a message field either. + const leaky = { status: 500, message: 'http://sturnus-api:8080/api/guilds/4711/queue failed' } + expect(describeQueueError(leaky)).not.toContain('sturnus-api') + }) +}) From aa15d8e5386f1184e28854e4883655169880ab09 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:26:29 +0200 Subject: [PATCH 07/14] docs(operations): how to read the queue overview MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Which sessions appear on it and why "unfinished" is two conditions, plus the three figures that must not be read at face value: the expired-lease count is measured against a lease this process assumed rather than the worker's real one, the oldest-pending figure is dated by the session's end because `transcription_job` records no enqueue time at all, and the list is a window on the backlog rather than its size. Also says where the closed-and-undocumented count leads: it is what the worker's document-retry sweep normally clears, so one that stays put across refreshes is a document write that is failing, and §5 is where to look next. --- docs/operations.md | 47 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/docs/operations.md b/docs/operations.md index 8fa9ccb..dc040c3 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1105,6 +1105,53 @@ nobody withdrew anything, the guild's policy moved on under them (§6). Restoring the old `policy_version` would bring every one of those back. Withdrawing through the console is what survives that. +### 6.2.7 The queue overview + +**Admin View → Queue**, per guild: the same figures `/queue status` prints +in Discord, plus the sessions they are made of. Both read `load_status`, +so the two never disagree. + +A session is listed while it is **unfinished**, which is two conditions +rather than one: + +- its status is not `documented` — it is recording now, waiting for a + worker, being transcribed, or stuck; **or** +- it has a `dead` job. A session reaches `documented` once every job is + terminal, and `dead` is terminal, so a speaker whose transcription + failed permanently would otherwise disappear from this view at exactly + the moment somebody needs to notice them. + +A session with status `open` and no jobs at all is a recording happening +right now. It is listed on purpose. + +Three numbers need reading with their caveats, and the page prints them: + +- **`running` past its lease** — a job whose worker died holding it. No + amount of waiting fixes one; it needs `/queue requeue` or the re-queue + control on the recording page. The count is computed against an + *assumed* lease, because `api` cannot see the worker's + `job_lease_seconds`; the lease it used is shown beside the number. If + the worker's setting differs, the count is wrong in the direction the + difference points. +- **Closed and undocumented** — nothing is queued for these and nothing + will happen on its own. The worker's `retry_pending_documents` sweep is + what normally clears them; a count that stays put across several + refreshes means the document write is failing, and §5 is where to look. +- **Oldest pending** — dated by the *session's end*, not by the job. + `transcription_job` has no enqueue timestamp at all. A session's end is + within seconds of when its jobs were created, which answers "has this + been sitting here for hours?" and nothing more precise. A re-queued job + keeps its session's original end, so a redo makes this read older than + the job actually is. + +The list is cut at twenty sessions, newest first, and says so when it was +cut. The totals above it are the guild's and are never cut. + +There is no re-queue control on this page. Each row links to +`/recordings/{id}`, which carries the per-session panel — and the decision +of whether a re-queue is safe stays in one place (`plan_requeue`) rather +than being made twice. + ### 6.3 Listening to a recording by hand Every automated check this system has can describe a track — its level, From 72dea631f6da4fc8e13e600cc394c8ee06f0b85f Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:37:23 +0200 Subject: [PATCH 08/14] fix(console): stop the queue page's poll from outliving the page The loop was a chain of timeouts cleared in `onBeforeUnmount`, which reads as correct and is not: `clearTimeout` cannot stop a timer that has **already fired**, and the continuation after the `await` inside it installs a fresh one that nothing is left to cancel. Navigating away during the seconds a read is in flight therefore left the page making a guild-wide database read every five seconds for the life of the tab, invisibly. The same defect was found independently in `RequeuePanel` (#108), whose polling shape this page copied. Fixing it in two places by hand would leave a third copy waiting to be written, so the loop is extracted to `startQueuePolling` in `~/utils/queue` and the page keeps only the decision of when to start one. `alive` is checked **after every await**, which is exactly where an unmount happens without the resuming code being told, and `stop()` clears that flag as well as the pending timer. Either half alone is the bug. A failed round now ends the loop rather than retrying. The page has an error to show and a refresh control to try again with; a loop that retried on its own would turn one bad second into a request every five for as long as the tab is open. Extracted rather than fixed in place because the property is one no build, type check or render can show, and a page component cannot be asked about it without a Nuxt runtime around it. As a plain function it is seven tests with fake timers -- including the one that matters, where the stop lands *during* a request rather than between two. --- console/app/pages/admin/queue.vue | 39 ++++---- console/app/utils/queue.ts | 112 ++++++++++++++++++++++ console/test/queue.spec.ts | 153 +++++++++++++++++++++++++++++- 3 files changed, 286 insertions(+), 18 deletions(-) diff --git a/console/app/pages/admin/queue.vue b/console/app/pages/admin/queue.vue index 9328d0e..ce0f374 100644 --- a/console/app/pages/admin/queue.vue +++ b/console/app/pages/admin/queue.vue @@ -52,6 +52,7 @@ import { sessionCounts, sessionStartLine, sessionsSummaryLine, + startQueuePolling, truncationNotice, } from '~/utils/queue' import { recordingPath } from '~/utils/recordings' @@ -145,30 +146,34 @@ const moving = computed(() => Boolean(queue.value && isQueueMoving(queue.value)) * who has just pressed a button and is waiting for it, this page is a * guild-wide read that somebody leaves open. */ const POLL_MS = 5000 -let timer: ReturnType | null = null + +/** The running loop, or `null` when nothing is scheduled. + * + * The loop itself lives in `~/utils/queue`, where it can be tested with + * fake timers -- see `startQueuePolling` on the defect that makes an + * inline chain of timeouts the wrong shape. What is left here is when to + * start one and when to let it go. */ +let poll: ReturnType | null = null function stopPolling() { - if (timer !== null) { - clearTimeout(timer) - timer = null - } + poll?.stop() + poll = null } -/** Re-reads once, and only if the work has not finished in the meantime. - * Same shape as `RequeuePanel`'s loop -- a chain of timeouts rather than - * an interval -- so a slow response can never queue a second request - * behind the first. */ +/** Re-reads while there is work in flight, and stops when there is not. */ function scheduleIfMoving() { stopPolling() - // Never during a server render: there is no `clearTimeout` on the way - // out of one, so a timer started there would keep the render alive and - // would fetch for a reader who has already been sent their HTML. + // Never during a server render: a timer started there would keep the + // render alive and would fetch for a reader who already has their HTML. if (!import.meta.client) return - if (!moving.value) return - timer = setTimeout(async () => { - await reload() - scheduleIfMoving() - }, POLL_MS) + poll = startQueuePolling({ + // Re-asked each round rather than captured now, because whether there + // is anything left to watch is a fact about the data that just came + // back. + shouldContinue: () => moving.value, + run: reload, + delayMs: POLL_MS, + }) } /** Whether a re-read is in flight. `useAsyncData`'s own status settles diff --git a/console/app/utils/queue.ts b/console/app/utils/queue.ts index e3ac6c5..bdf48ea 100644 --- a/console/app/utils/queue.ts +++ b/console/app/utils/queue.ts @@ -914,3 +914,115 @@ export function describeQueueError(error: unknown): string { return `Sturnus answered ${status} and could not report this server’s queue. Nothing is known about why.` } } + +/** + * A poll that cannot outlive the thing it is polling for. + * + * Extracted from the page rather than written inline, and not for tidiness: + * the property that matters here is one no build, type check or render can + * show, and one that a page component cannot be asked about without a Nuxt + * runtime around it. Here it is an ordinary function with fake timers + * pointed at it. + * + * The defect it exists to make unrepresentable is the one `RequeuePanel` + * shipped with. A chain of timeouts is the right shape -- an interval can + * queue a second request behind a slow first -- but `clearTimeout` cannot + * stop a timer that has **already fired**, and the continuation after the + * `await` inside it installs a fresh timer that nothing is left to cancel. + * Navigating away during the seconds a poll is in flight therefore leaves a + * loop reading the database for the life of the tab, per page, invisibly. + * + * So `alive` is checked *after every await*, which is exactly where an + * unmount happens without the resuming code being told, and `stop()` sets + * it false as well as clearing the pending timer. One of those two alone + * is the bug. + * + * `shouldContinue` is re-asked each round rather than captured once, + * because whether there is anything left to watch is a fact about the data + * that just came back. + */ +/** What a timer handle is, without committing to a runtime. + * + * `setTimeout` returns a `Timeout` object under Node and a number in a + * browser, and this module is compiled with both libraries in scope + * because a page is rendered on the server and then polls in the client. + * `ReturnType` resolves to whichever overload the + * checker reaches first, which is not the same as what the call actually + * returns -- so the union is written out rather than inferred. The loop + * never inspects a handle; it stores what its own `setTimer` returned and + * hands it back to its own `clearTimer`. */ +export type QueueTimer = ReturnType | number + +export interface QueuePoll { + /** Whether another round is worth making, asked afresh each time. */ + shouldContinue: () => boolean + /** One re-read. Rejections are the caller's to handle; a rejected round + * ends the loop rather than retrying blind, because a poll that keeps + * hammering an endpoint that is failing is how a transient fault becomes + * a sustained one. */ + run: () => Promise + delayMs: number + setTimer?: (callback: () => void, ms: number) => QueueTimer + clearTimer?: (handle: QueueTimer) => void +} + +export interface QueuePollHandle { + /** Stops the loop for good. Safe to call more than once, and safe to + * call from inside the loop's own continuation. */ + stop: () => void + /** Whether the loop is still able to schedule another round. Exposed for + * the tests, which is the whole reason this is a function and not four + * lines in a component. */ + readonly alive: boolean +} + +export function startQueuePolling(poll: QueuePoll): QueuePollHandle { + const setTimer = poll.setTimer ?? setTimeout + const clearTimer = poll.clearTimer ?? clearTimeout + + let alive = true + let timer: QueueTimer | null = null + + function stop() { + alive = false + if (timer !== null) { + clearTimer(timer) + timer = null + } + } + + function schedule() { + if (!alive || !poll.shouldContinue()) return + timer = setTimer(() => { + // Cleared first: this handle has fired and can no longer be + // cancelled, so leaving it in place would make `stop()` believe it + // had cancelled something. + timer = null + if (!alive) return + poll + .run() + .then(() => { + // The check that the extracted version exists for. Between the + // timer firing and this line the component may have gone, and + // nothing tells the resuming code so. + if (!alive) return + schedule() + }) + .catch(() => { + // A failed round ends the loop. The page has an error to show + // and a refresh control to try again with; a loop that retried + // on its own would turn one bad second into a request every + // five for as long as the tab is open. + stop() + }) + }, poll.delayMs) + } + + schedule() + return { + stop, + get alive() { + return alive + }, + } +} diff --git a/console/test/queue.spec.ts b/console/test/queue.spec.ts index 5844db6..a5b7ae9 100644 --- a/console/test/queue.spec.ts +++ b/console/test/queue.spec.ts @@ -22,7 +22,7 @@ * administrator who restarted a healthy worker because a console told them * it was dead. */ -import { describe, expect, it } from 'vitest' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { CLEAR_QUEUE_NOTE, @@ -50,6 +50,7 @@ import { type GuildQueue, type QueueCounts, type QueuedSession, + startQueuePolling, } from '../app/utils/queue' /** A session with everything harmless, so each test states only the one @@ -774,3 +775,153 @@ describe('when the API says no', () => { expect(describeQueueError(leaky)).not.toContain('sturnus-api') }) }) + +describe('polling a queue that is still moving', () => { + beforeEach(() => vi.useFakeTimers()) + afterEach(() => vi.useRealTimers()) + + /** A round that resolves only when the test says so, so an unmount can be + * placed *during* a request rather than only between two of them. That + * window is where the defect this loop exists for actually lives. */ + function deferred() { + let release: () => void = () => {} + const promise = new Promise((resolve) => { + release = resolve + }) + return { promise, release } + } + + it('re-reads while there is work in flight', async () => { + let rounds = 0 + startQueuePolling({ + shouldContinue: () => true, + run: async () => { + rounds += 1 + }, + delayMs: 5000, + }) + + await vi.advanceTimersByTimeAsync(5000) + expect(rounds).toBe(1) + await vi.advanceTimersByTimeAsync(5000) + expect(rounds).toBe(2) + }) + + it('stops on its own once nothing is moving', async () => { + // Every poll is a database read across a whole guild. A page left open + // overnight on a server that finished at five o'clock must not keep + // making them. + let rounds = 0 + let moving = true + startQueuePolling({ + shouldContinue: () => moving, + run: async () => { + rounds += 1 + moving = false + }, + delayMs: 5000, + }) + + await vi.advanceTimersByTimeAsync(60_000) + + expect(rounds).toBe(1) + }) + + it('makes no further request after being stopped between two rounds', async () => { + let rounds = 0 + const handle = startQueuePolling({ + shouldContinue: () => true, + run: async () => { + rounds += 1 + }, + delayMs: 5000, + }) + + handle.stop() + await vi.advanceTimersByTimeAsync(60_000) + + expect(rounds).toBe(0) + expect(handle.alive).toBe(false) + }) + + it('makes no further request after being stopped in the middle of one', async () => { + /** + * The defect this loop was extracted to make unrepresentable. + * + * `clearTimeout` cannot stop a timer that has already fired, and the + * continuation after the `await` installs a fresh one that nothing is + * left to cancel. Navigating away during the seconds a request is in + * flight therefore left `RequeuePanel` reading the database for the + * life of the tab, invisibly. Nothing in a build, a type check or a + * render shows it. + */ + const round = deferred() + let rounds = 0 + const handle = startQueuePolling({ + shouldContinue: () => true, + run: async () => { + rounds += 1 + await round.promise + }, + delayMs: 5000, + }) + + await vi.advanceTimersByTimeAsync(5000) + expect(rounds).toBe(1) + + // The unmount lands while the first read is still outstanding. + handle.stop() + round.release() + await vi.advanceTimersByTimeAsync(60_000) + + expect(rounds).toBe(1) + }) + + it('ends the loop when a round fails rather than retrying blind', async () => { + // The page has an error to show and a refresh control to try again + // with. A loop that retried on its own would turn one bad second into + // a request every five for as long as the tab is open. + let rounds = 0 + const handle = startQueuePolling({ + shouldContinue: () => true, + run: async () => { + rounds += 1 + throw new Error('the API had a bad second') + }, + delayMs: 5000, + }) + + await vi.advanceTimersByTimeAsync(60_000) + + expect(rounds).toBe(1) + expect(handle.alive).toBe(false) + }) + + it('schedules nothing at all when there was never anything to watch', async () => { + let rounds = 0 + startQueuePolling({ + shouldContinue: () => false, + run: async () => { + rounds += 1 + }, + delayMs: 5000, + }) + + await vi.advanceTimersByTimeAsync(60_000) + + expect(rounds).toBe(0) + }) + + it('can be stopped twice without complaint', async () => { + // A page that stops the loop on unmount and again on a guild switch is + // an ordinary page, not a misuse. + const handle = startQueuePolling({ + shouldContinue: () => true, + run: async () => {}, + delayMs: 5000, + }) + + handle.stop() + expect(() => handle.stop()).not.toThrow() + }) +}) From a26f184d8da8319b31c41590d1e50d1899ab157a Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:13:22 +0200 Subject: [PATCH 09/14] feat(console): report what a guild's recording adds up to An administrator configuring Sturnus has had no way to tell whether it is working out. How often does this guild actually meet, how long do the meetings run, how many of them produced a protocol, is the transcription measuring anything at all. Every one of those is answerable from rows the system already writes, and none of them was answerable from anywhere in the product. `GET /api/guilds/{guild_id}/report`. **This reports on a guild and never on a person, and that is a boundary rather than a gap.** A report over meetings can be written two ways. One says how much a team recorded, how long its meetings run and how many people are usually in them -- facts about the guild, which is what somebody deciding whether the bot is configured sensibly needs. The other ranks named individuals by how many meetings they attended and how long they spoke, which is a different artifact entirely: in Germany and the EU a per-person readout of attendance and speaking time is a means of monitoring performance and conduct, and introducing one is a matter for a works council rather than something a console adds because the columns happen to be there. So the identities never leave the statement. The adapter counts participant rows -- per session, and `COUNT(DISTINCT ...)` across the guild -- and `sturnus.console.reporting` is handed counts rather than people. A report module holding a list of who attended is one edit away from ranking them; there is a test asserting the payload contains no `discord_user_id` and no `display_name`, and it is there to fail loudly rather than to pass quietly. Two things the numbers would otherwise be misread as: - **Months are cut in the guild's own `timezone`**, the same calendar the protocols are written in, and the payload names which zone was used. A meeting that opened at 00:30 Berlin time belongs to the month the people in it think it does; UTC would file it under the previous one and disagree with the timestamps printed in the protocol of that very meeting. An unusable value falls back to UTC, the same fallback the worker applies, and the named zone is how anyone finds out it happened. - **Null is not zero.** `speech_seconds` is nullable, `SUM` skips nulls silently, and every job predating the measurement columns has one. The number of skipped tracks travels beside the total, so a large one reads as "this figure describes part of what was recorded" rather than as "the guild was quiet". For the same reason an average over nothing is `None` and not `0`: a guild with no closed sessions has no average length, and reporting one as zero states that its meetings are instantaneous. 404 for a guild this person does not administer, and the same 404 for one that does not exist: the report describes when a team meets and how often. --- src/sturnus/console/adapters.py | 148 ++++++++++- src/sturnus/console/app.py | 6 + src/sturnus/console/ports.py | 35 ++- src/sturnus/console/reporting.py | 226 ++++++++++++++++ src/sturnus/console/routes_report.py | 94 +++++++ src/sturnus/entrypoints/api.py | 2 + tests/console/conftest.py | 24 ++ tests/console/test_guild_reports.py | 382 +++++++++++++++++++++++++++ tests/console/test_report_routes.py | 221 ++++++++++++++++ tests/console/test_reporting.py | 276 +++++++++++++++++++ 10 files changed, 1411 insertions(+), 3 deletions(-) create mode 100644 src/sturnus/console/reporting.py create mode 100644 src/sturnus/console/routes_report.py create mode 100644 tests/console/test_guild_reports.py create mode 100644 tests/console/test_report_routes.py create mode 100644 tests/console/test_reporting.py diff --git a/src/sturnus/console/adapters.py b/src/sturnus/console/adapters.py index 62540e9..749d7d7 100644 --- a/src/sturnus/console/adapters.py +++ b/src/sturnus/console/adapters.py @@ -21,16 +21,19 @@ from collections.abc import Callable, Sequence from dataclasses import dataclass -from datetime import datetime, timedelta +from datetime import UTC, datetime, timedelta, tzinfo +from zoneinfo import ZoneInfo, ZoneInfoNotFoundError -from sqlalchemy import delete, func, select +from sqlalchemy import Row, delete, func, select from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker from sturnus.application.linking import new_state +from sturnus.application.publishing import DOCUMENTED_STATUS from sturnus.console.ports import ( AdminDirectory, ConsentHolder, GuildQueue, + GuildRecording, QueuedSession, QueueSnapshot, QueueSpeaker, @@ -39,6 +42,7 @@ SettingsStore, Track, ) +from sturnus.console.reporting import RecordedSession from sturnus.domain import settings from sturnus.domain.consent import ConsentRecord, is_consent_active from sturnus.infrastructure.db.models import ( @@ -653,3 +657,143 @@ def _queued(session: ActiveSession) -> QueuedSession: done=session.counts.get("done", 0), dead=session.counts.get("dead", 0), ) + + +class ConsoleGuildReports: + """A guild's recorded sessions, counted rather than listed. + + The authorisation is here, as it is in every other directory in this + module: one `is_admin(guild_id, ...)` at the top of the one method, + and `None` for both of the reasons somebody might not get an answer. + + **What the statements deliberately do not select.** Nothing here reads + `session_participant.discord_user_id` into a value that leaves this + class. The participant rows are counted -- per session, and distinctly + across the guild -- and the identities stay in the database. That is + what keeps `sturnus.console.reporting` able to say it is about a guild + rather than about its people: a report module handed a list of who + attended is one edit away from ranking them, and a ranking of + colleagues by meeting attendance is a works-council decision rather + than a console feature. + + Three statements rather than one join, the same trade + `ConsoleQueries` makes: a join across participants and jobs multiplies + rows, and a session with five speakers and five tracks comes back + twenty-five times. + """ + + def __init__( + self, + session_factory: async_sessionmaker[AsyncSession], + admins: AdminDirectory, + config: SettingsStore, + ) -> None: + self._session_factory = session_factory + self._admins = admins + self._config = config + + async def recording_of(self, guild_id: int, *, requested_by: int) -> GuildRecording | None: + if not await self._admins.is_admin(guild_id, requested_by): + return None + + zone, zone_name = _zone( + (await self._config.snapshot(guild_id)).get(settings.TIMEZONE) or "UTC" + ) + async with self._session_factory() as db: + rows = ( + await db.execute( + select( + SessionRow.id, + SessionRow.started_at, + SessionRow.ended_at, + SessionRow.status, + ) + .where(SessionRow.guild_id == guild_id) + .order_by(SessionRow.started_at, SessionRow.id) + ) + ).all() + if not rows: + return GuildRecording((), 0, zone, zone_name) + + found = [row.id for row in rows] + people = ( + await db.execute( + select(SessionParticipant.session_id, func.count()) + .where(SessionParticipant.session_id.in_(found)) + .group_by(SessionParticipant.session_id) + ) + ).all() + # Counted in the statement rather than by collecting ids and + # taking a set: `COUNT(DISTINCT ...)` answers the question + # without any identity crossing into Python. + distinct = await db.scalar( + select(func.count(func.distinct(SessionParticipant.discord_user_id))).where( + SessionParticipant.session_id.in_(found) + ) + ) + tracks = ( + await db.execute( + select( + TranscriptionJob.session_id, + func.count(), + func.sum(TranscriptionJob.audio_seconds), + func.sum(TranscriptionJob.speech_seconds), + # Null is not zero. `SUM` skips nulls silently, so + # the number of rows it skipped is counted beside + # it -- otherwise "we never measured this" and + # "they said nothing" arrive as the same total. + func.count().filter(TranscriptionJob.speech_seconds.is_(None)), + ) + .where(TranscriptionJob.session_id.in_(found)) + .group_by(TranscriptionJob.session_id) + ) + ).all() + + attendance = {session_id: int(count) for session_id, count in people} + measured = { + session_id: (int(count), audio, speech, int(unmeasured)) + for session_id, count, audio, speech, unmeasured in tracks + } + return GuildRecording( + sessions=tuple( + _recorded(row, attendance.get(row.id, 0), measured.get(row.id)) for row in rows + ), + distinct_participants=int(distinct or 0), + zone=zone, + zone_name=zone_name, + ) + + +def _recorded( + row: Row[tuple[int, datetime, datetime | None, str]], + participants: int, + measured: tuple[int, float | None, float | None, int] | None, +) -> RecordedSession: + tracks, audio_seconds, speech_seconds, unmeasured = measured or (0, None, None, 0) + return RecordedSession( + id=row.id, + started_at=row.started_at, + ended_at=row.ended_at, + documented=row.status == DOCUMENTED_STATUS, + participants=participants, + tracks=tracks, + audio_seconds=audio_seconds, + speech_seconds=speech_seconds, + unmeasured_tracks=unmeasured, + ) + + +def _zone(name: str) -> tuple[tzinfo, str]: + """The guild's timezone, falling back to UTC on an unusable value. + + The same fallback the worker applies when writing a protocol, and for + the same reason: a report with the wrong month boundary is a smaller + loss than no report, and the value that caused it is a `/config` away + from being fixed. The name travels with the zone so the page can say + which calendar it cut the months in rather than leaving the reader to + assume theirs. + """ + try: + return ZoneInfo(name), name + except (ZoneInfoNotFoundError, ValueError): + return UTC, "UTC" diff --git a/src/sturnus/console/app.py b/src/sturnus/console/app.py index 6009375..c4b98db 100644 --- a/src/sturnus/console/app.py +++ b/src/sturnus/console/app.py @@ -37,6 +37,7 @@ from sturnus.console.ports import ( AdminDirectory, ConsentDirectory, + GuildReports, LinkDirectory, OAuthClient, QueueControl, @@ -51,6 +52,8 @@ from sturnus.console.routes_consent import register as register_consent from sturnus.console.routes_queue import QUEUE_CONTROL, QUEUE_OVERVIEW from sturnus.console.routes_queue import register as register_queue +from sturnus.console.routes_report import GUILD_REPORTS +from sturnus.console.routes_report import register as register_report from sturnus.console.session import ( ExpiredSession, InvalidSession, @@ -274,6 +277,7 @@ def build_api( queue: QueueControl, queues: QueueOverview, consents: ConsentDirectory, + reports: GuildReports, ) -> web.Application: """Builds the application, with every collaborator injected. @@ -301,6 +305,7 @@ def build_api( app[QUEUE_CONTROL] = queue app[QUEUE_OVERVIEW] = queues app[CONSENT_DIRECTORY] = consents + app[GUILD_REPORTS] = reports app.add_routes( [ web.get("/healthz", healthz), @@ -315,5 +320,6 @@ def build_api( register_audio(app) register_queue(app) register_consent(app) + register_report(app) routes_settings.register(app) return app diff --git a/src/sturnus/console/ports.py b/src/sturnus/console/ports.py index 7dcdc7e..b5581c5 100644 --- a/src/sturnus/console/ports.py +++ b/src/sturnus/console/ports.py @@ -17,9 +17,10 @@ from collections.abc import AsyncGenerator, Sequence from dataclasses import dataclass -from datetime import date, datetime +from datetime import date, datetime, tzinfo from typing import Protocol +from sturnus.console.reporting import RecordedSession from sturnus.console.statistics import AttendedSession from sturnus.infrastructure.documents.outline_oauth import ExternalIdentity @@ -439,3 +440,35 @@ class QueueOverview(Protocol): """ async def for_guild(self, guild_id: int, *, requested_by: int) -> GuildQueue | None: ... + + +@dataclass(frozen=True) +class GuildRecording: + """A guild's recorded sessions, and how many distinct people were in them. + + The two are read together and returned as one value because the second + cannot be derived from the first: `sessions` carries per-session + participant *counts*, never identities, so "how many different people + has this guild recorded" has to be counted in the statement. Keeping + the identities out of the value is the point -- see + `sturnus.console.reporting` on why this feature stops at aggregates. + """ + + sessions: tuple[RecordedSession, ...] + distinct_participants: int + #: The guild's `timezone` setting, or `UTC` when it is unset or + #: unusable. Months are cut in it, the same calendar the protocols are + #: written in. + zone: tzinfo + zone_name: str + + +class GuildReports(Protocol): + """What a guild's recording adds up to, if the person asking administers it. + + `requested_by` is not optional and there is no method here without it, + for the reason the other three directories have none. `None` covers + "no such guild" and "you do not administer it" alike. + """ + + async def recording_of(self, guild_id: int, *, requested_by: int) -> GuildRecording | None: ... diff --git a/src/sturnus/console/reporting.py b/src/sturnus/console/reporting.py new file mode 100644 index 0000000..6442f15 --- /dev/null +++ b/src/sturnus/console/reporting.py @@ -0,0 +1,226 @@ +"""What a guild's recording adds up to, computed from rows and nothing else. + +Pure functions over frozen dataclasses, separated from the SQL in +`sturnus.console.adapters` for the same reason `sturnus.console.statistics` +is separated from `sturnus.console.queries`: the shaping is where the +decisions are, and decisions that need a database to exercise are +decisions nobody exercises. + +**This module is deliberately about a guild and never about a person.** + +That restraint is the point rather than an oversight. A report over +meetings can be written two ways. One says how much a team recorded, how +long its meetings run and how many people are usually in them -- facts +about the *guild*, useful for deciding whether the bot is configured +sensibly and whether the transcription is keeping up. The other ranks +named individuals by how many meetings they attended and how long they +spoke, which is a different artifact entirely: in Germany and the EU a +per-person readout of attendance and speaking time is a means of +monitoring performance and conduct, and introducing one is a decision for +a works council rather than for a console. + +Nothing here forecloses the second. It is simply not this module, so that +switching it on is a visible, separate act rather than a field that +appeared in a payload. + +Three rules carry over from `statistics` unchanged, because they are +properties of the same columns: + +- **Null is not zero.** `audio_seconds`, `speech_seconds` and + `segment_count` are nullable, and null means nobody ever measured while + zero means somebody did and it was nothing. A total that sums null as + zero understates itself *and* hides how much of itself is missing, so + every total here is reported beside the number of tracks that had + nothing to contribute to it. +- **Every id is a string.** A Discord snowflake exceeds JavaScript's safe + integer range. +- **A session that has not ended has no length.** Answering "now minus + started_at" renders a meeting that grows every time the page is + refreshed. +""" + +from __future__ import annotations + +from collections import Counter +from collections.abc import Iterable, Sequence +from dataclasses import dataclass +from datetime import datetime, tzinfo +from typing import TypedDict + + +@dataclass(frozen=True) +class RecordedSession: + """One session of a guild, with only what a report is computed from. + + Deliberately not `AttendedSession`. That one is scoped to a person and + carries their tracks; this one is scoped to a guild and carries counts. + A report built from the other shape would have needed a viewer to + exist, which is exactly the confusion this feature must not introduce. + """ + + id: int + started_at: datetime + ended_at: datetime | None + documented: bool + #: How many people were in it, from `session_participant`. + participants: int + #: How many recordings it produced, whether or not they were measured. + tracks: int + #: Summed over the tracks that carry a measurement. `None` when not one + #: of them does -- which is a different fact from zero and is reported + #: as one. + audio_seconds: float | None + speech_seconds: float | None + #: Tracks with no `speech_seconds` at all. The size of the hole in the + #: figure above. + unmeasured_tracks: int + + @property + def duration_seconds(self) -> float | None: + """How long this ran, or `None` while it is still running.""" + if self.ended_at is None: + return None + return (self.ended_at - self.started_at).total_seconds() + + +class MonthJson(TypedDict): + """One month of a guild's recording, in the guild's own calendar.""" + + month: str + sessions: int + recorded_seconds: float + #: Sessions that produced a protocol. Against `sessions` this is the + #: pipeline's success rate as the guild experienced it. + documented: int + + +class ReportJson(TypedDict): + guild_id: str + sessions: int + documented: int + open_sessions: int + recorded_seconds: float + speech_seconds: float + unmeasured_tracks: int + tracks: int + distinct_participants: int + average_participants: float | None + largest_meeting: int | None + average_duration_seconds: float | None + longest_duration_seconds: float | None + first_session_at: str | None + last_session_at: str | None + timezone: str + months: list[MonthJson] + + +def guild_report( + sessions: Sequence[RecordedSession], + *, + guild_id: int, + distinct_participants: int, + zone: tzinfo, + zone_name: str, +) -> ReportJson: + """Everything the report page shows, from the guild's sessions. + + `distinct_participants` is passed in rather than derived, because the + only honest way to count distinct people is over every participant row + and these values carry counts rather than identities. Deriving it here + would have meant carrying the identities into this module, and a + module that holds a list of who attended is one edit away from + ranking them. + """ + closed = [session for session in sessions if session.duration_seconds is not None] + durations = [session.duration_seconds for session in closed if session.duration_seconds] + participant_counts = [session.participants for session in sessions if session.participants] + + return ReportJson( + guild_id=str(guild_id), + sessions=len(sessions), + documented=sum(1 for session in sessions if session.documented), + # Reported rather than folded into the total. A guild with four + # sessions of which one has been "recording" for three days has a + # problem, and an average length computed over the other three + # would hide it. + open_sessions=len(sessions) - len(closed), + recorded_seconds=sum(durations), + speech_seconds=sum( + session.speech_seconds for session in sessions if session.speech_seconds is not None + ), + # The size of the hole in the figure above, in the same payload as + # the figure. A total offered without it invites the reader to + # treat "we have no measurement" as "they said nothing". + unmeasured_tracks=sum(session.unmeasured_tracks for session in sessions), + tracks=sum(session.tracks for session in sessions), + distinct_participants=distinct_participants, + average_participants=_mean(participant_counts), + # How big this guild's meetings get. An aggregate about *meetings* + # rather than about the people in them. + largest_meeting=max(participant_counts, default=None), + average_duration_seconds=_mean(durations), + longest_duration_seconds=max(durations, default=None), + first_session_at=_isoformat(min((s.started_at for s in sessions), default=None)), + last_session_at=_isoformat(max((s.started_at for s in sessions), default=None)), + # Named in the payload because the months below are cut in it, and + # a month boundary is a choice: a meeting that opened at half past + # midnight in Berlin belongs to the month the people in it think it + # does, not to the previous one UTC would file it under. + timezone=zone_name, + months=months(sessions, zone), + ) + + +def months(sessions: Iterable[RecordedSession], zone: tzinfo) -> list[MonthJson]: + """One entry per month that had a session, oldest first. + + In the guild's own timezone, the same one the protocols are written in + (Spec 11). A report that bucketed by UTC would put a late-evening + meeting in the wrong month twice a year for guilds west of Greenwich + and every single time for guilds east of it -- and it would disagree + with the timestamps printed in the protocol of that very meeting. + + Months with no sessions are absent rather than zero-filled. A guild + that used the bot in March and again in November has eight empty + months between them, and a chart that draws them is a chart mostly of + nothing; a client that wants a continuous axis can fill the gaps, + knowing which months were genuinely empty. + """ + counted: Counter[str] = Counter() + documented: Counter[str] = Counter() + seconds: dict[str, float] = {} + for session in sessions: + key = session.started_at.astimezone(zone).strftime("%Y-%m") + counted[key] += 1 + if session.documented: + documented[key] += 1 + seconds[key] = seconds.get(key, 0.0) + (session.duration_seconds or 0.0) + + return [ + MonthJson( + month=key, + sessions=counted[key], + recorded_seconds=seconds[key], + documented=documented[key], + ) + # Lexicographic on `YYYY-MM` is chronological, which is the whole + # reason the key is written that way round. + for key in sorted(counted) + ] + + +def _mean(values: Sequence[float] | Sequence[int]) -> float | None: + """The average, or `None` when there is nothing to average. + + `None` rather than zero, for the reason null is not zero everywhere + else in this codebase: a guild with no closed sessions has no average + length, and reporting one as `0` states that its meetings are + instantaneous. + """ + if not values: + return None + return sum(values) / len(values) + + +def _isoformat(moment: datetime | None) -> str | None: + return None if moment is None else moment.isoformat() diff --git a/src/sturnus/console/routes_report.py b/src/sturnus/console/routes_report.py new file mode 100644 index 0000000..82c7e9b --- /dev/null +++ b/src/sturnus/console/routes_report.py @@ -0,0 +1,94 @@ +"""What a guild's recording adds up to. + +- `GET /api/guilds/{guild_id}/report` + +**Why this exists.** An administrator configuring Sturnus has no way to +tell whether it is working out. How often does this guild actually meet, +how long do its meetings run, how many of them produced a protocol, is the +transcription measuring anything at all. Every one of those is answerable +from rows the system already writes, and none of them is answerable from +anywhere in the product today. + +**What this endpoint is not, and why that is deliberate.** It reports on a +*guild*: how much was recorded, over which months, how big the meetings +were, how many distinct people the guild has recorded. It does not name a +person and it does not rank anybody. + +That is a boundary rather than a gap. A per-person readout of meeting +attendance and speaking time — "who was in the most sessions" — is a +different artifact from a usage report: in Germany and the EU it is a +means of monitoring performance and conduct, which is a matter for a works +council rather than something a console adds because the columns happen to +be there. The rows exist. Turning them into a ranking of colleagues is a +decision somebody has to take on purpose, and this endpoint is built so +that taking it is a visible act rather than a field that appeared in a +payload. + +**404, never 403**, for a guild this person does not administer — the same +answer as for a guild that does not exist. The report says when a guild +meets and how often, which is a description of a team's working week. + +**No decision is taken here.** The shaping is +`sturnus.console.reporting`, which is pure and tested without a database; +this module is the shape of one HTTP response and nothing else. +""" + +from __future__ import annotations + +from aiohttp import web + +from sturnus.console.ports import GuildReports +from sturnus.console.reporting import guild_report + +#: Where the collaborator is found. Its own key rather than a parameter to +#: `register`, so `build_api` stays a one-line edit -- several agents are +#: adding sections to that function and each extra line is a merge by hand. +GUILD_REPORTS: web.AppKey[GuildReports] = web.AppKey("guild_reports") + +_REPORT_PATH = "/api/guilds/{guild_id}/report" + + +def register(app: web.Application) -> None: + """Adds the report route to an application that already has its reports.""" + from sturnus.console.app import require_session + + app.add_routes([web.get(_REPORT_PATH, require_session(guild_report_view))]) + + +async def guild_report_view(request: web.Request) -> web.Response: + """One guild's recording, in aggregate.""" + from sturnus.console.app import current_user + + viewer = current_user(request).discord_user_id + try: + guild_id = int(request.match_info["guild_id"]) + except ValueError: + # A path segment that is not a number names no guild, which is the + # same answer as naming one that does not exist. + return _no_such_guild() + + recording = await request.app[GUILD_REPORTS].recording_of(guild_id, requested_by=viewer) + if recording is None: + return _no_such_guild() + + return web.json_response( + guild_report( + recording.sessions, + guild_id=guild_id, + distinct_participants=recording.distinct_participants, + zone=recording.zone, + zone_name=recording.zone_name, + ), + # It describes when a team meets and how often. Nothing in between + # this and the browser has any business keeping a copy. + headers={"Cache-Control": "private, no-store"}, + ) + + +def _no_such_guild() -> web.Response: + """One refusal for every reason there is to refuse. + + "No such guild" and "you do not administer that guild" are + deliberately indistinguishable; see the module docstring. + """ + return web.json_response({"error": "no such guild"}, status=404) diff --git a/src/sturnus/entrypoints/api.py b/src/sturnus/entrypoints/api.py index 0c8efbb..dcae2eb 100644 --- a/src/sturnus/entrypoints/api.py +++ b/src/sturnus/entrypoints/api.py @@ -40,6 +40,7 @@ from sturnus.config import StrictSettings from sturnus.console.adapters import ( ConsoleConsentDirectory, + ConsoleGuildReports, ConsoleLinkDirectory, ConsoleQueueControl, ConsoleQueueOverview, @@ -216,6 +217,7 @@ def now() -> datetime: queue=ConsoleQueueControl(session_factory, admins), queues=ConsoleQueueOverview(session_factory, admins, now), consents=ConsoleConsentDirectory(session_factory, admins, config, now), + reports=ConsoleGuildReports(session_factory, admins, config), ) runner = web.AppRunner(app) diff --git a/tests/console/conftest.py b/tests/console/conftest.py index 940dcf3..914811d 100644 --- a/tests/console/conftest.py +++ b/tests/console/conftest.py @@ -24,6 +24,8 @@ ConsentDirectory, ConsentHolder, GuildQueue, + GuildRecording, + GuildReports, LinkDirectory, OAuthClient, QueueControl, @@ -449,6 +451,26 @@ async def revoke( return self.outcome +class FakeReports: + """A guild nobody administers, until a test says otherwise. + + Defaults to `None`, which is what the real reports answer for "no such + guild or not yours" -- so a test with no interest in reporting gets + 404s rather than a fake that quietly authorises everything. + """ + + def __init__(self, recording: GuildRecording | None = None) -> None: + self.recording = recording + #: Every guild this was asked about, with who asked. The route + #: tests assert on it: "the handler passed the signed-in id, not + #: one from the URL" cannot be seen in a response body. + self.asked: list[tuple[int, int]] = [] + + async def recording_of(self, guild_id: int, *, requested_by: int) -> GuildRecording | None: + self.asked.append((guild_id, requested_by)) + return self.recording + + def build_test_api( *, oauth: OAuthClient | None = None, @@ -461,6 +483,7 @@ def build_test_api( queue: QueueControl | None = None, queues: QueueOverview | None = None, consents: ConsentDirectory | None = None, + reports: GuildReports | None = None, sessions: SessionCookie | None = None, now: Callable[[], datetime] | None = None, schema_ready: bool = True, @@ -497,6 +520,7 @@ def build_test_api( queue=queue or FakeQueue(), queues=queues or FakeQueueOverview(), consents=consents or FakeConsents(), + reports=reports or FakeReports(), sessions=sessions or SessionCookie(SECRET, timedelta(hours=12)), now=now or now_at(), schema_ready=lambda: schema_ready, diff --git a/tests/console/test_guild_reports.py b/tests/console/test_guild_reports.py new file mode 100644 index 0000000..5d2b161 --- /dev/null +++ b/tests/console/test_guild_reports.py @@ -0,0 +1,382 @@ +"""The report's reads, against the real database. + +Against PostgreSQL rather than a double because every property worth +pinning is a property of the statements: that the whole answer is scoped +to one guild, that `COUNT(DISTINCT ...)` counts people rather than +participations, and that `SUM` skipping nulls is accounted for rather than +silently absorbed. + +The arithmetic on top of these rows is `sturnus.console.reporting` and is +tested there without a database. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from zoneinfo import ZoneInfo + +import pytest +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from sturnus.console.adapters import ConsoleGuildReports +from sturnus.domain import settings +from sturnus.infrastructure.db.config_store import ConfigStore +from sturnus.infrastructure.db.models import ( + Base, + Session, + SessionParticipant, + TranscriptionJob, +) + +T0 = datetime(2026, 8, 21, 12, 0, 0, tzinfo=UTC) +GUILD, OTHER_GUILD = 4711, 9999 +ANNA, BEN, CARL = 100, 200, 300 + + +@pytest.fixture +async def factory(clean_database: str) -> async_sessionmaker[AsyncSession]: + engine = create_async_engine(clean_database) + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + return async_sessionmaker(engine, expire_on_commit=False) + + +class Admins: + """The mirrored administrator membership, per guild.""" + + def __init__(self, by_guild: dict[int, set[int]] | None = None) -> None: + self.by_guild = by_guild if by_guild is not None else {GUILD: {ANNA}} + + async def is_admin_anywhere(self, discord_user_id: int) -> bool: + return any(discord_user_id in members for members in self.by_guild.values()) + + async def administered_guilds(self, discord_user_id: int) -> tuple[int, ...]: + return tuple( + sorted(g for g, members in self.by_guild.items() if discord_user_id in members) + ) + + async def is_admin(self, guild_id: int, discord_user_id: int) -> bool: + return discord_user_id in self.by_guild.get(guild_id, set()) + + +def reports( + factory: async_sessionmaker[AsyncSession], admins: Admins | None = None +) -> ConsoleGuildReports: + return ConsoleGuildReports(factory, admins or Admins(), ConfigStore(factory)) + + +async def a_session( + factory: async_sessionmaker[AsyncSession], + *, + guild_id: int = GUILD, + started_at: datetime = T0, + ended_at: datetime | None = None, + status: str = "documented", + people: dict[int, str] | None = None, + speech: dict[int, float | None] | None = None, +) -> int: + """One session with participants and, optionally, measured jobs. + + `speech` is separate from `people` because the two really are: a + participant row and a `transcription_job` row are written by different + parts of the pipeline, and a person can appear in a meeting whose + recording was never measured. + """ + async with factory() as db: + session = Session( + guild_id=guild_id, + channel_id=555, + channel_name="meeting", + started_at=started_at, + ended_at=ended_at if ended_at is not None else started_at + timedelta(hours=1), + status=status, + ) + db.add(session) + await db.flush() + for discord_user_id, name in (people or {}).items(): + db.add( + SessionParticipant( + session_id=session.id, + discord_user_id=discord_user_id, + discord_display_name=name, + first_seen_at=started_at, + ) + ) + for discord_user_id, speech_seconds in (speech or {}).items(): + db.add( + TranscriptionJob( + session_id=session.id, + discord_user_id=discord_user_id, + s3_key=f"sessions/{session.id}/speakers/{discord_user_id}.enc", + encryption_key_id="k1", + wrapped_data_key=b"wrapped", + retention_until=started_at + timedelta(days=30), + status="done", + attempts=1, + audio_seconds=None if speech_seconds is None else speech_seconds * 3, + speech_seconds=speech_seconds, + ) + ) + await db.commit() + return session.id + + +# --------------------------------------------------------------------------- +# Who may ask +# --------------------------------------------------------------------------- + + +async def test_an_administrator_of_the_guild_gets_its_recording( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, people={BEN: "ben"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert len(recording.sessions) == 1 + + +async def test_an_administrator_of_another_guild_is_nobody_here( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, people={BEN: "ben"}) + admins = Admins({OTHER_GUILD: {CARL}}) + + assert await reports(factory, admins).recording_of(GUILD, requested_by=CARL) is None + + +async def test_a_participant_who_administers_nothing_gets_no_report( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, people={BEN: "ben"}) + + assert await reports(factory).recording_of(GUILD, requested_by=BEN) is None + + +# --------------------------------------------------------------------------- +# Scoped to one guild, all the way down +# --------------------------------------------------------------------------- + + +async def test_another_guild_s_sessions_are_not_in_this_guild_s_report( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, guild_id=OTHER_GUILD, people={BEN: "ben"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.sessions == () + assert recording.distinct_participants == 0 + + +async def test_somebody_in_two_guilds_is_counted_once_in_each( + factory: async_sessionmaker[AsyncSession], +) -> None: + # The distinct count is over this guild's sessions, so a person who + # meets in two guilds is one person in each rather than two in either. + await a_session(factory, people={BEN: "ben"}) + await a_session(factory, guild_id=OTHER_GUILD, people={BEN: "ben", CARL: "carl"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.distinct_participants == 1 + + +# --------------------------------------------------------------------------- +# Counting people, not participations +# --------------------------------------------------------------------------- + + +async def test_a_person_in_four_meetings_is_one_distinct_participant( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`COUNT(DISTINCT ...)` and not `COUNT(*)`. + + "How many different people has this guild recorded" is a fact about + the guild; counting participations instead would answer a question + nobody asked and answer it with a bigger number. + """ + for day in range(4): + await a_session(factory, started_at=T0 - timedelta(days=day), people={BEN: "ben"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.distinct_participants == 1 + assert len(recording.sessions) == 4 + + +async def test_each_session_carries_how_many_people_were_in_it( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, people={ANNA: "anna", BEN: "ben", CARL: "carl"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.sessions[0].participants == 3 + + +async def test_the_sessions_carry_counts_and_never_identities( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The boundary, checked on the value rather than on the payload. + + A report module handed a list of who attended is one edit away from + ranking them, so the identities are not carried out of the statement + at all. + """ + await a_session(factory, people={BEN: "ben"}, speech={BEN: 120.0}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + fields = vars(recording.sessions[0]) + assert not [name for name in fields if "user" in name or "name" in name] + + +# --------------------------------------------------------------------------- +# Null is not zero +# --------------------------------------------------------------------------- + + +async def test_a_track_nobody_measured_is_counted_as_unmeasured( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`SUM` skips nulls silently, so the skipped rows are counted beside it. + + Otherwise "we never measured this" and "they said nothing" arrive as + the same total, and every job that predates the measurement columns + quietly makes a guild look quieter than it was. + """ + await a_session(factory, people={ANNA: "anna", BEN: "ben"}, speech={ANNA: 60.0, BEN: None}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + session = recording.sessions[0] + assert session.tracks == 2 + assert session.speech_seconds == 60.0 + assert session.unmeasured_tracks == 1 + + +async def test_a_session_whose_tracks_were_all_unmeasured_has_no_total_at_all( + factory: async_sessionmaker[AsyncSession], +) -> None: + # `None` rather than `0.0`: nobody measured, which is a different fact + # from measuring and finding nothing. + await a_session(factory, people={BEN: "ben"}, speech={BEN: None}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.sessions[0].speech_seconds is None + assert recording.sessions[0].unmeasured_tracks == 1 + + +async def test_a_session_with_no_recordings_reports_no_tracks( + factory: async_sessionmaker[AsyncSession], +) -> None: + # A meeting where nobody consented produces participants and no jobs, + # and the report must not read that as a measurement of zero. + await a_session(factory, people={BEN: "ben"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.sessions[0].tracks == 0 + assert recording.sessions[0].speech_seconds is None + + +# --------------------------------------------------------------------------- +# What the session rows say +# --------------------------------------------------------------------------- + + +async def test_a_session_that_produced_a_protocol_is_marked_as_documented( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_session(factory, status="documented", people={BEN: "ben"}) + await a_session(factory, started_at=T0 + timedelta(days=1), status="closed", people={BEN: "b"}) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert [session.documented for session in recording.sessions] == [True, False] + + +async def test_a_session_still_running_comes_back_without_an_end( + factory: async_sessionmaker[AsyncSession], +) -> None: + async with factory() as db: + db.add( + Session( + guild_id=GUILD, + channel_id=555, + started_at=T0, + ended_at=None, + status="open", + ) + ) + await db.commit() + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.sessions[0].ended_at is None + assert recording.sessions[0].duration_seconds is None + + +# --------------------------------------------------------------------------- +# The guild's own calendar +# --------------------------------------------------------------------------- + + +async def test_the_guild_s_configured_timezone_is_what_the_months_are_cut_in( + factory: async_sessionmaker[AsyncSession], +) -> None: + await ConfigStore(factory).set(GUILD, settings.TIMEZONE, "Europe/Berlin", T0) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.zone == ZoneInfo("Europe/Berlin") + assert recording.zone_name == "Europe/Berlin" + + +async def test_a_guild_that_never_set_a_timezone_gets_the_configured_default( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`ConfigStore.snapshot` resolves `DEFAULTS`, so unset is not absent. + + `timezone` defaults to Europe/Berlin, which is what this deployment's + guilds actually meet in -- so the months of a guild nobody configured + are cut in the same calendar its protocols are written in, rather than + in UTC. Reaching UTC here would mean somebody set something odd, which + is the case below. + """ + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.zone_name == settings.DEFAULTS[settings.TIMEZONE] + + +async def test_an_unusable_timezone_falls_back_rather_than_failing( + factory: async_sessionmaker[AsyncSession], +) -> None: + """The same fallback the worker applies when writing a protocol. + + A report with the wrong month boundary is a smaller loss than no + report, and the value that caused it is one `/config set` from being + fixed. Naming the zone in the answer is what tells the reader it + happened. + """ + await ConfigStore(factory).set(GUILD, settings.TIMEZONE, "Mars/Olympus_Mons", T0) + + recording = await reports(factory).recording_of(GUILD, requested_by=ANNA) + + assert recording is not None + assert recording.zone_name == "UTC" diff --git a/tests/console/test_report_routes.py b/tests/console/test_report_routes.py new file mode 100644 index 0000000..f089adb --- /dev/null +++ b/tests/console/test_report_routes.py @@ -0,0 +1,221 @@ +"""Who may read a guild's report, and what it is allowed to contain. + +The arithmetic is `sturnus.console.reporting` and is pinned there without +a database. What is pinned here is the endpoint: that the id reaching the +reports comes out of the signed cookie rather than out of the URL, that a +guild somebody does not administer is indistinguishable from one that does +not exist, and -- the test worth having most -- that the payload names +nobody. + +That last one is a boundary rather than an implementation detail. A +per-person readout of meeting attendance and speaking time is a means of +monitoring performance and conduct, which is a works-council matter rather +than a field that appears in a payload because the columns were there. A +test that would fail the moment a name appeared is how it stays a decision +somebody has to take on purpose. +""" + +from __future__ import annotations + +import json +from datetime import UTC, datetime, timedelta +from zoneinfo import ZoneInfo + +from aiohttp import web +from aiohttp.test_utils import TestClient + +from sturnus.console.app import SESSION_COOKIE +from sturnus.console.ports import GuildRecording +from sturnus.console.reporting import RecordedSession +from sturnus.console.session import SessionCookie, SignedSession +from tests.console.conftest import ( + ANNA, + BEN, + GUILD, + SECRET, + T0, + AiohttpClientFactory, + FakeReports, + build_test_api, +) + +BERLIN = ZoneInfo("Europe/Berlin") + + +def token(discord_user_id: int = ANNA) -> str: + return SessionCookie(SECRET, timedelta(hours=12)).issue(SignedSession(discord_user_id), now=T0) + + +async def signed_in( + aiohttp_client: AiohttpClientFactory, app: web.Application, as_user: int = ANNA +) -> TestClient[web.Request, web.Application]: + client = await aiohttp_client(app) + client.session.cookie_jar.update_cookies({SESSION_COOKIE: token(as_user)}) + return client + + +def report_url(guild_id: int | str = GUILD) -> str: + return f"/api/guilds/{guild_id}/report" + + +def a_session(**over: object) -> RecordedSession: + base: dict[str, object] = { + "id": 1, + "started_at": T0, + "ended_at": T0 + timedelta(hours=1), + "documented": True, + "participants": 4, + "tracks": 4, + "audio_seconds": 900.0, + "speech_seconds": 300.0, + "unmeasured_tracks": 0, + } + base.update(over) + return RecordedSession(**base) # type: ignore[arg-type] + + +def recording(*sessions: RecordedSession, distinct: int = 6) -> GuildRecording: + return GuildRecording( + sessions=sessions or (a_session(),), + distinct_participants=distinct, + zone=BERLIN, + zone_name="Europe/Berlin", + ) + + +# --------------------------------------------------------------------------- +# Who may ask +# --------------------------------------------------------------------------- + + +async def test_an_administrator_sees_what_their_guild_has_recorded( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in(aiohttp_client, build_test_api(reports=FakeReports(recording()))) + + response = await client.get(report_url()) + + assert response.status == 200 + body = await response.json() + assert body["sessions"] == 1 + assert body["recorded_seconds"] == 3600 + + +async def test_the_report_is_asked_for_on_behalf_of_the_signed_in_person( + aiohttp_client: AiohttpClientFactory, +) -> None: + reports = FakeReports(recording()) + client = await signed_in(aiohttp_client, build_test_api(reports=reports), as_user=BEN) + + await client.get(report_url()) + + assert reports.asked == [(GUILD, BEN)] + + +async def test_a_guild_this_person_does_not_administer_does_not_exist( + aiohttp_client: AiohttpClientFactory, +) -> None: + # The report says when a guild meets and how often, which is a + # description of a team's working week. A 403 would confirm that such + # a description exists here. + client = await signed_in(aiohttp_client, build_test_api(reports=FakeReports())) + + response = await client.get(report_url()) + + assert response.status == 404 + assert (await response.json())["error"] == "no such guild" + + +async def test_a_guild_id_that_is_not_a_number_never_reaches_the_reports( + aiohttp_client: AiohttpClientFactory, +) -> None: + reports = FakeReports(recording()) + client = await signed_in(aiohttp_client, build_test_api(reports=reports)) + + assert (await client.get(report_url("nope"))).status == 404 + assert reports.asked == [] + + +async def test_the_report_needs_a_session_like_every_other_endpoint( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await aiohttp_client(build_test_api(reports=FakeReports(recording()))) + assert (await client.get(report_url())).status == 401 + + +# --------------------------------------------------------------------------- +# What it may contain, and what it may not +# --------------------------------------------------------------------------- + + +async def test_the_report_names_nobody( + aiohttp_client: AiohttpClientFactory, +) -> None: + """The test this endpoint exists to keep passing. + + A guild report is about a guild. The moment a Discord id or a display + name appears in it, it has become a per-person record of who attends + which meetings -- and that is a decision for a works council, not a + field somebody added because the column was already selected. + """ + reports = FakeReports(recording(a_session(participants=9), distinct=12)) + client = await signed_in(aiohttp_client, build_test_api(reports=reports)) + + raw = await (await client.get(report_url())).text() + body = json.loads(raw) + + assert body["largest_meeting"] == 9 + assert body["distinct_participants"] == 12 + # Not a check of the keys alone: a name would arrive as a value. + assert "discord_user_id" not in raw + assert "display_name" not in raw + + +async def test_the_guild_id_travels_as_a_string( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in(aiohttp_client, build_test_api(reports=FakeReports(recording()))) + + assert (await (await client.get(report_url())).json())["guild_id"] == str(GUILD) + + +async def test_the_months_say_which_calendar_they_were_cut_in( + aiohttp_client: AiohttpClientFactory, +) -> None: + # A meeting that opened at half past midnight in Berlin belongs to the + # month the people in it think it does, and a reader who is not told + # which calendar was used will assume theirs. + late = datetime(2026, 8, 31, 23, 30, tzinfo=UTC) + reports = FakeReports(recording(a_session(started_at=late, ended_at=None))) + client = await signed_in(aiohttp_client, build_test_api(reports=reports)) + + body = await (await client.get(report_url())).json() + + assert body["timezone"] == "Europe/Berlin" + assert [month["month"] for month in body["months"]] == ["2026-09"] + + +async def test_a_guild_that_has_never_recorded_gets_a_report_saying_so( + aiohttp_client: AiohttpClientFactory, +) -> None: + # Not a 404: "you administer this guild and it has recorded nothing" + # and "this is not your guild" are different answers. + empty = GuildRecording(sessions=(), distinct_participants=0, zone=UTC, zone_name="UTC") + client = await signed_in(aiohttp_client, build_test_api(reports=FakeReports(empty))) + + response = await client.get(report_url()) + + assert response.status == 200 + body = await response.json() + assert body["sessions"] == 0 + assert body["average_duration_seconds"] is None + + +async def test_nothing_in_between_may_cache_a_report( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in(aiohttp_client, build_test_api(reports=FakeReports(recording()))) + + response = await client.get(report_url()) + + assert response.headers["Cache-Control"] == "private, no-store" diff --git a/tests/console/test_reporting.py b/tests/console/test_reporting.py new file mode 100644 index 0000000..c94df4c --- /dev/null +++ b/tests/console/test_reporting.py @@ -0,0 +1,276 @@ +"""What a guild's recording adds up to, computed without a database. + +The shaping is pure, so these tests are about the decisions rather than +about SQL: null is not zero, an unfinished session has no length, an +average over nothing is not zero, and months are cut in the guild's own +calendar rather than in UTC. + +The last of those is the one worth stating plainly. A meeting that opened +at half past midnight in Berlin belongs to the month the people in it +think it does. Bucketing by UTC would file it under the previous one -- +and disagree with the timestamps printed in the protocol of that very +meeting, which the worker writes in the guild's timezone (Spec 11). +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from zoneinfo import ZoneInfo + +from sturnus.console.reporting import RecordedSession, guild_report, months + +BERLIN = ZoneInfo("Europe/Berlin") +GUILD = 4711 +T0 = datetime(2026, 8, 21, 12, 0, 0, tzinfo=UTC) + + +def a_session(**over: object) -> RecordedSession: + base: dict[str, object] = { + "id": 1, + "started_at": T0, + "ended_at": T0 + timedelta(hours=1), + "documented": True, + "participants": 3, + "tracks": 3, + "audio_seconds": 900.0, + "speech_seconds": 300.0, + "unmeasured_tracks": 0, + } + base.update(over) + return RecordedSession(**base) # type: ignore[arg-type] + + +def report( + *sessions: RecordedSession, + distinct_participants: int = 3, + zone: object = UTC, + zone_name: str = "UTC", +) -> dict[str, object]: + return dict( + guild_report( + sessions, + guild_id=GUILD, + distinct_participants=distinct_participants, + zone=zone, # type: ignore[arg-type] + zone_name=zone_name, + ) + ) + + +# --------------------------------------------------------------------------- +# The totals +# --------------------------------------------------------------------------- + + +def test_a_guild_that_has_never_recorded_reports_nothing_rather_than_zeroes() -> None: + """An average over nothing is not zero. + + Reporting `0` for the average length of a guild's meetings states that + its meetings are instantaneous, which is a claim about a guild that + has never had one. + """ + empty = report(distinct_participants=0) + + assert empty["sessions"] == 0 + assert empty["average_duration_seconds"] is None + assert empty["longest_duration_seconds"] is None + assert empty["average_participants"] is None + assert empty["largest_meeting"] is None + assert empty["first_session_at"] is None + assert empty["months"] == [] + + +def test_the_recorded_time_is_the_sum_of_what_actually_finished() -> None: + summed = report( + a_session(id=1, ended_at=T0 + timedelta(hours=1)), + a_session( + id=2, started_at=T0 + timedelta(days=1), ended_at=T0 + timedelta(days=1, hours=2) + ), + ) + + assert summed["recorded_seconds"] == 3 * 3600 + assert summed["sessions"] == 2 + + +def test_a_session_still_running_has_no_length_and_is_counted_separately() -> None: + """ "Now minus started_at" renders a meeting that grows on every refresh. + + Reported as its own number rather than dropped: a guild with one + session that has been "recording" for three days has a problem, and an + average length computed over the others would hide it. + """ + mixed = report( + a_session(id=1, ended_at=T0 + timedelta(hours=1)), + a_session(id=2, ended_at=None), + ) + + assert mixed["sessions"] == 2 + assert mixed["open_sessions"] == 1 + assert mixed["recorded_seconds"] == 3600 + assert mixed["average_duration_seconds"] == 3600 + + +def test_how_many_sessions_produced_a_protocol_is_reported_against_the_total() -> None: + # Against the total this is the pipeline's success rate as the guild + # experienced it, which is the question the page exists to answer. + rate = report(a_session(id=1, documented=True), a_session(id=2, documented=False)) + + assert (rate["documented"], rate["sessions"]) == (1, 2) + + +# --------------------------------------------------------------------------- +# Null is not zero +# --------------------------------------------------------------------------- + + +def test_a_track_nobody_measured_does_not_count_as_silence() -> None: + """The rule the whole codebase turns on for these three columns. + + Null means nobody ever measured; zero means somebody did and it was + nothing. A total that folds the first into the second understates + itself *and* destroys the distinction. + """ + unmeasured = report(a_session(speech_seconds=None, tracks=3, unmeasured_tracks=3)) + + assert unmeasured["speech_seconds"] == 0 + assert unmeasured["unmeasured_tracks"] == 3 + assert unmeasured["tracks"] == 3 + + +def test_the_hole_in_the_speech_total_travels_with_the_total() -> None: + # A total offered without it invites the reader to treat "we have no + # measurement" as "they said nothing". + partly = report( + a_session(id=1, speech_seconds=300.0, tracks=3, unmeasured_tracks=1), + a_session(id=2, speech_seconds=None, tracks=2, unmeasured_tracks=2), + ) + + assert partly["speech_seconds"] == 300.0 + assert partly["unmeasured_tracks"] == 3 + + +# --------------------------------------------------------------------------- +# About meetings, never about the people in them +# --------------------------------------------------------------------------- + + +def test_the_report_says_how_big_the_meetings_get_and_never_who_was_in_them() -> None: + """The boundary this module exists to hold. + + "How many people are usually in a meeting here" is a fact about a + guild's meetings. "Which of them was in the most" is a means of + monitoring performance and conduct, and it is not built here. + """ + sizes = report( + a_session(id=1, participants=2), + a_session(id=2, participants=6), + distinct_participants=7, + ) + + assert sizes["average_participants"] == 4 + assert sizes["largest_meeting"] == 6 + assert sizes["distinct_participants"] == 7 + assert not [key for key in sizes if "user" in key or "name" in key] + + +def test_the_guild_id_travels_as_a_string() -> None: + # A snowflake exceeds JavaScript's safe integer range, where a JSON + # number silently loses its last digits. + assert report()["guild_id"] == str(GUILD) + + +def test_when_a_guild_first_and_last_recorded_are_both_reported() -> None: + first = T0 - timedelta(days=400) + span = report( + a_session(id=1, started_at=first, ended_at=first + timedelta(hours=1)), + a_session(id=2, started_at=T0), + ) + + assert span["first_session_at"] == first.isoformat() + assert span["last_session_at"] == T0.isoformat() + + +# --------------------------------------------------------------------------- +# Months, in the guild's own calendar +# --------------------------------------------------------------------------- + + +def test_a_meeting_after_midnight_belongs_to_the_month_the_room_was_in() -> None: + """23:30 UTC on 31 August is 01:30 on 1 September in Berlin. + + Bucketing by UTC would file this under August and disagree with the + timestamps printed in the protocol of this very meeting. + """ + late = datetime(2026, 8, 31, 23, 30, tzinfo=UTC) + + assert [m["month"] for m in months([a_session(started_at=late)], BERLIN)] == ["2026-09"] + assert [m["month"] for m in months([a_session(started_at=late)], UTC)] == ["2026-08"] + + +def test_the_timezone_the_months_were_cut_in_is_named_in_the_payload() -> None: + # A month boundary is a choice, and a reader who is not told which + # calendar was used will assume theirs. + assert report(zone=BERLIN, zone_name="Europe/Berlin")["timezone"] == "Europe/Berlin" + + +def test_months_come_back_oldest_first() -> None: + # Lexicographic on `YYYY-MM` is chronological, which is the whole + # reason the key is written that way round. + spread = months( + [ + a_session(id=1, started_at=datetime(2026, 3, 4, tzinfo=UTC)), + a_session(id=2, started_at=datetime(2025, 11, 4, tzinfo=UTC)), + a_session(id=3, started_at=datetime(2026, 1, 4, tzinfo=UTC)), + ], + UTC, + ) + + assert [m["month"] for m in spread] == ["2025-11", "2026-01", "2026-03"] + + +def test_a_month_with_no_sessions_is_absent_rather_than_zero() -> None: + """A guild that met in March and again in November has eight empty months. + + A chart that draws them is a chart mostly of nothing, and a client that + wants a continuous axis can fill the gaps -- knowing, because they are + absent, which months were genuinely empty. + """ + apart = months( + [ + a_session(id=1, started_at=datetime(2026, 3, 4, tzinfo=UTC)), + a_session(id=2, started_at=datetime(2026, 11, 4, tzinfo=UTC)), + ], + UTC, + ) + + assert [m["month"] for m in apart] == ["2026-03", "2026-11"] + + +def test_a_month_carries_its_own_sessions_seconds_and_protocols() -> None: + march = datetime(2026, 3, 4, tzinfo=UTC) + counted = months( + [ + a_session(id=1, started_at=march, ended_at=march + timedelta(hours=1)), + a_session( + id=2, + started_at=march + timedelta(days=1), + ended_at=march + timedelta(days=1, minutes=30), + documented=False, + ), + ], + UTC, + ) + + assert counted[0]["sessions"] == 2 + assert counted[0]["recorded_seconds"] == 5400 + assert counted[0]["documented"] == 1 + + +def test_a_month_containing_an_unfinished_session_counts_it_without_any_seconds() -> None: + # The session is real and belongs in the count; it simply has no length + # yet, and inventing one would make the month grow on every refresh. + march = datetime(2026, 3, 4, tzinfo=UTC) + counted = months([a_session(started_at=march, ended_at=None)], UTC) + + assert counted[0]["sessions"] == 1 + assert counted[0]["recorded_seconds"] == 0 From 91de4bbfd21640dca135449591c021ef1726e397 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:25:01 +0200 Subject: [PATCH 10/14] feat(console): a Reporting page for how a server uses Sturnus The Admin View's fourth entry, and the interface half of the guild report. Headline figures, then a month-by-month breakdown, then the caveats the figures cannot be read correctly without. Two of those caveats are the reason the page is not just a row of numbers: - **An absent figure renders as absent, never as zero.** A guild with no finished session has no average meeting length, and a `0` there states that its meetings are instantaneous. The API sends `null` for exactly that reason and the page keeps the distinction. - **"Unmeasured tracks" is not silence.** `speech_seconds` is nullable, `SUM` skips nulls, and every recording predating the measurement columns has one -- so a large count of unmeasured tracks means the speaking figure describes part of what was recorded, not that the server was quiet. The page says so in words rather than leaving it to be inferred from two numbers side by side. Months are labelled in the guild's own timezone and the page names which one, because a meeting that opened at 00:30 Berlin time belongs to the month the people in it think it does. Months with no sessions are absent from the payload rather than zero-filled, and how the breakdown handles that gap is a decision made once, in `~/utils/reporting`, where it is tested. The page is framed as how a server uses Sturnus, and it names nobody. That framing matches what the endpoint sends: the report is about a guild, not about the people in it. --- console/app/pages/admin/reporting.vue | 475 ++++++++++++ console/app/utils/navigation.ts | 11 + console/app/utils/reporting.ts | 1002 +++++++++++++++++++++++++ console/test/navigation.spec.ts | 19 + console/test/reporting.spec.ts | 718 ++++++++++++++++++ 5 files changed, 2225 insertions(+) create mode 100644 console/app/pages/admin/reporting.vue create mode 100644 console/app/utils/reporting.ts create mode 100644 console/test/reporting.spec.ts diff --git a/console/app/pages/admin/reporting.vue b/console/app/pages/admin/reporting.vue new file mode 100644 index 0000000..f65ff5b --- /dev/null +++ b/console/app/pages/admin/reporting.vue @@ -0,0 +1,475 @@ + + + diff --git a/console/app/utils/navigation.ts b/console/app/utils/navigation.ts index 72f440a..a6e224f 100644 --- a/console/app/utils/navigation.ts +++ b/console/app/utils/navigation.ts @@ -96,6 +96,17 @@ export const ADMIN_VIEW: NavSection = { icon: 'M12 2 2 7l10 5 10-5-10-5Zm0 20 10-5-2.5-1.25L12 19.5 4.5 15.75 2 17l10 5Zm0-5.5 10-5-2.5-1.25L12 14 4.5 10.25 2 11.5l10 5Z', adminOnly: true, }, + { + to: '/admin/reporting', + label: 'Reporting', + // Three bars of different heights, which is what the page's + // by-month breakdown actually looks like. Deliberately not a pie: + // this report is a run of months, and a pie would promise a + // breakdown of a whole into named parts -- which is precisely the + // per-person readout the page refuses to be. + icon: 'M4 20V10h4v10H4Zm6 0V4h4v16h-4Zm6 0v-7h4v7h-4Z', + adminOnly: true, + }, ], } diff --git a/console/app/utils/reporting.ts b/console/app/utils/reporting.ts new file mode 100644 index 0000000..1782164 --- /dev/null +++ b/console/app/utils/reporting.ts @@ -0,0 +1,1002 @@ +/** + * How one Discord server uses Sturnus, said in figures that do not + * overstate themselves. + * + * A module rather than expressions in the page, for the same reason + * `~/utils/queue` is one: every function here is a *decision* -- what an + * absent average reads like, which month comes first, whether a gap in a + * bar row is drawn or skipped, what the speech total is actually a total + * of, whether an empty report is a fault -- and a decision embedded in a + * template can only be tested by rendering one. + * + * Five facts govern the wording below, and none of them are softened + * anywhere in this file. Each is a way this page could quietly mislead + * somebody who trusted it: + * + * - **This report is about a server, never about the people in it.** The + * payload carries no ids and no names, and nothing here invents a + * per-person figure or hints that one is coming. How long one named + * person sat in meetings, or spoke in them, is a measure of that + * person's conduct at work -- a matter for a works council, not a + * console feature. Counts of people are counts and stop there. + * - **Null is not zero.** `average_duration_seconds`, `longest_duration_ + * seconds`, `average_participants` and `largest_meeting` are null for a + * server whose meetings have not finished. Printing `0` would state that + * its meetings are instantaneous and attended by nobody, which is a + * claim rather than an absence, so an absence renders as one and carries + * the sentence that says why. + * - **`unmeasured_tracks` is the size of a hole in `speech_seconds`.** The + * per-track speech column is nullable: null means nobody ever measured, + * zero means somebody measured and it was silence. `SUM` skips nulls + * without comment, so a server with most of its tracks unmeasured gets a + * speech total that is short by an unknown amount. Every rendering of + * that total says so, in the words "not that this server was quiet" -- + * because a small number under a large one is read as quiet by anybody + * not told otherwise. + * - **`recorded_seconds` excludes the meetings still open.** A session + * with no `ended_at` has no length yet, and a session that has been + * "recording" for three days is a fault rather than a long meeting. So + * `open_sessions` is shown as a figure of its own instead of being + * allowed to vanish into a total it is not part of. + * - **The months were cut in the server's own calendar.** A meeting that + * starts at 00:30 belongs to the month the people in it think it does, + * which is why the API buckets by `timezone` rather than by UTC -- and + * why a reader who is not told the zone will assume it is theirs. The + * instants on this page are still written in UTC, since a server render + * has no idea what zone the reader is in, so the page says both. + * + * Nothing here decides what to do about a backlog. That question belongs + * to the Queue page, which reads the pipeline rather than the calendar; a + * second answer to "is something stuck" would be a second definition of + * stuck, and the two would drift. + */ +import { formatCount, formatDuration, formatMoment } from '~/utils/format' + +/* -------------------------------------------------------------------- */ +/* What the API describes */ +/* -------------------------------------------------------------------- */ + +/** + * One calendar month of recording, as the API bucketed it. + * + * Months with no sessions are **absent** from the payload rather than sent + * as zeros -- a server that met in March and again in November sends two + * entries, not thirteen. What this module does about that gap is decided + * in `reportMonthRows`, deliberately and in one place. + */ +export interface ReportMonth { + /** `YYYY-MM`, in the server's own zone. Malformed months never reach + * here; see `parseGuildReport`. */ + month: string + sessions: number + /** Null when the API had no total to give, which is not the same as a + * month in which nothing ran for any time at all. */ + recorded_seconds: number | null + documented: number +} + +/** + * One server's whole history with Sturnus. + * + * The four nullable figures are nullable on purpose and are kept nullable + * all the way to the screen. Collapsing them to zero at the edge of this + * module would put the lie somewhere no test could see it. + */ +export interface GuildReport { + /** Null only when the payload named no server, which the page uses to + * refuse to show one server's figures under another's heading. */ + guild_id: string | null + sessions: number + documented: number + /** Sessions with no `ended_at`. Counted in `sessions`, counted in + * nothing else. */ + open_sessions: number + recorded_seconds: number | null + /** The sum of a nullable column, so null means "nothing was ever + * measured" rather than "nothing was said". */ + speech_seconds: number | null + /** How many tracks the sum above had to skip. */ + unmeasured_tracks: number + tracks: number + /** How many different people this server has recorded. Not a list, and + * never on its way to becoming one. */ + distinct_participants: number + /** Null until a meeting has finished. */ + average_participants: number | null + /** The most people in any one meeting. Null until a meeting has + * finished. */ + largest_meeting: number | null + average_duration_seconds: number | null + longest_duration_seconds: number | null + first_session_at: string | null + last_session_at: string | null + /** The IANA zone the months were cut in. Empty when the API did not say, + * which the timezone note reports as the uncertainty it is. */ + timezone: string + months: ReportMonth[] +} + +/* -------------------------------------------------------------------- */ +/* Reading what the API sent */ +/* -------------------------------------------------------------------- */ + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** `null` stays `null`; anything else becomes the string it prints as. Ids + * are strings on the wire, and a number that arrived instead has already + * lost whatever precision it was going to lose. */ +function asText(value: unknown): string | null { + if (value === null || value === undefined) return null + const text = typeof value === 'string' ? value : String(value) + return text.trim() === '' ? null : text +} + +/** + * A count that can be printed. + * + * Anything absent, negative or not a number is a defect upstream, and + * rendering it as "-3 meetings" would put that defect in front of the + * reader as though it were a fact about their server. Zero is the honest + * floor: it says "none", which is as far as this console can vouch. + */ +function asCount(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return 0 + return Math.round(value) +} + +/** + * A count that is allowed to be missing. + * + * The distinction this keeps is the whole point of the field being + * nullable: `largest_meeting` is null for a server whose meetings have not + * finished, and rounding that to zero would claim it holds meetings nobody + * attends. Nonsense collapses to null rather than to zero for the same + * reason -- "we do not know" is true of a broken figure and "none" is not. + */ +function asOptionalCount(value: unknown): number | null { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return null + return Math.round(value) +} + +/** A quantity that is allowed to be missing and is not a whole number -- + * seconds, and an average of people. Kept unrounded; how many digits it + * is worth showing is a rendering decision and is made where it shows. */ +function asOptionalNumber(value: unknown): number | null { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return null + return value +} + +/** `YYYY-MM`, and nothing looser. A month this module cannot place on a + * calendar cannot be ordered, cannot be labelled and -- worst -- would + * anchor the gap filling in `reportMonthRows` at an arbitrary point in + * history, turning one bad string into a thousand rows of zeros. */ +const MONTH_KEY = /^\d{4}-(0[1-9]|1[0-2])$/ + +function asMonths(value: unknown): ReportMonth[] { + if (!Array.isArray(value)) return [] + return value.flatMap((entry) => { + if (!isRecord(entry)) return [] + const month = asText(entry.month) + if (!month || !MONTH_KEY.test(month)) return [] + return [ + { + month, + sessions: asCount(entry.sessions), + recorded_seconds: asOptionalNumber(entry.recorded_seconds), + documented: asCount(entry.documented), + }, + ] + }) +} + +/** + * The report in a payload. + * + * Always yields a well-formed value, never null. A page that has to + * distinguish "the API refused" from "the API answered something odd" + * already has the thrown `ApiError` for the first; a parser that returned + * null for the second would turn a strange payload into a blank page with + * no error anywhere, which is the failure mode hardest to report. + */ +export function parseGuildReport(payload: unknown): GuildReport { + const raw = isRecord(payload) ? payload : {} + return { + guild_id: asText(raw.guild_id), + sessions: asCount(raw.sessions), + documented: asCount(raw.documented), + open_sessions: asCount(raw.open_sessions), + recorded_seconds: asOptionalNumber(raw.recorded_seconds), + speech_seconds: asOptionalNumber(raw.speech_seconds), + unmeasured_tracks: asCount(raw.unmeasured_tracks), + tracks: asCount(raw.tracks), + distinct_participants: asCount(raw.distinct_participants), + average_participants: asOptionalNumber(raw.average_participants), + largest_meeting: asOptionalCount(raw.largest_meeting), + average_duration_seconds: asOptionalNumber(raw.average_duration_seconds), + longest_duration_seconds: asOptionalNumber(raw.longest_duration_seconds), + first_session_at: asText(raw.first_session_at), + last_session_at: asText(raw.last_session_at), + timezone: asText(raw.timezone) ?? '', + months: asMonths(raw.months), + } +} + +/** Where a server's report is read from. The id is escaped: it is a string + * from an API, and a string allowed to contain a slash is a string + * allowed to address a different endpoint. */ +export function reportPath(guildId: string): string { + return `/guilds/${encodeURIComponent(guildId)}/report` +} + +/* -------------------------------------------------------------------- */ +/* Writing a figure down */ +/* -------------------------------------------------------------------- */ + +/** The one thing that means "there is no figure here". Never "0", and the + * same glyph the dashboard uses, so an absence looks the same in both + * places a reader might meet one. */ +const NO_FIGURE = '—' + +/** + * Full month names, written out here rather than taken from `Intl`. + * + * `Intl.DateTimeFormat` formats for the runtime's locale, so the same + * month would render as "August" during the server render and "August" + * only by luck in a browser set to German -- which Vue reports as a + * hydration mismatch and the reader sees as a flicker across every row of + * the table. The console's own text is English; its months should be too. + */ +const MONTH_NAMES = [ + 'January', + 'February', + 'March', + 'April', + 'May', + 'June', + 'July', + 'August', + 'September', + 'October', + 'November', + 'December', +] + +/** `2026-08` → `August 2026`. A month key this function cannot read comes + * back unchanged rather than as a blank: a raw key is at least something + * the reader can match against the payload. */ +export function reportMonthLabel(month: string): string { + if (!MONTH_KEY.test(month)) return month + const year = Number(month.slice(0, 4)) + const index = Number(month.slice(5, 7)) - 1 + return `${MONTH_NAMES[index]} ${year}` +} + +/** An average of people, to one decimal, without the trailing `.0` that + * makes a whole number look like a measurement it is not. */ +function averageInWords(value: number): string { + const rounded = Math.round(value * 10) / 10 + return Number.isInteger(rounded) ? String(rounded) : rounded.toFixed(1) +} + +/** One or the other, chosen by the count. Written out at each call rather + * than derived by adding an `s`, because half the pairs this page needs + * are `is`/`are` and `was`/`were`. */ +function plural(count: number, one: string, many: string): string { + return count === 1 ? one : many +} + +function meetings(count: number): string { + return `${formatCount(count)} ${plural(count, 'meeting', 'meetings')}` +} + +/* -------------------------------------------------------------------- */ +/* The figures themselves */ +/* -------------------------------------------------------------------- */ + +/** + * Three tones, and they are about the reader rather than about the data. + * + * `plain` is a figure that says what it says. `absent` is the deliberate + * lack of one, which must not be coloured like a number that happens to be + * small. `watch` is a figure worth a second look -- today that is only the + * count of meetings still open, which is either a meeting happening right + * now or a session that never closed. + */ +export type ReportTone = 'plain' | 'absent' | 'watch' + +/** + * One figure on the page: a label, what it says, and the sentence under + * it. + * + * Deliberately not `Figure` from `~/utils/format`. That type is the + * dashboard's, where every figure is a number somebody has and none of + * them can be missing; widening it with a tone for this page's sake would + * change a shape three other things depend on in order to serve one that + * does not exist yet. + */ +export interface ReportFigure { + key: string + label: string + value: string + /** The line under the figure. Never null for a missing value: an em dash + * with nothing beside it reads as "still loading", which is the one + * thing this page is not doing. */ + note: string | null + tone: ReportTone +} + +/** + * What share of this server's meetings reached a protocol, as a percentage + * -- or null when it has held none. + * + * Rounding is clamped away from both ends on purpose. A server with 999 of + * 1000 meetings written up rounds to 100 %, and "100 %" beside a figure + * that is not all of them is the page telling somebody every meeting is + * covered when one is not. The same holds at the bottom: one success out + * of a thousand is not "0 %" of them. + */ +export function reportDocumentedShare(report: GuildReport): number | null { + if (report.sessions <= 0) return null + if (report.documented >= report.sessions) return 100 + const raw = Math.round((report.documented / report.sessions) * 100) + if (report.documented > 0 && raw <= 0) return 1 + return Math.min(99, Math.max(0, raw)) +} + +/** + * How the pipeline has done, as this server experienced it. + * + * Written as "n of m", not as a bare rate: a rate on its own is read as a + * property of the software, and this is a property of what happened here. + * Where meetings are still recording that is said too -- a meeting that + * has not ended cannot have been written up, and counting it as a failure + * of the pipeline would be blaming the pipeline for the clock. + */ +export function reportDocumentedLine(report: GuildReport): string { + if (report.sessions <= 0) { + return 'Nothing has been recorded in this server yet, so there is nothing to write up.' + } + const open + = report.open_sessions > 0 + ? ` ${meetings(report.open_sessions)} ${plural(report.open_sessions, 'is', 'are')} still ` + + 'recording and cannot have been written up yet; they are counted in the total all the ' + + 'same.' + : '' + if (report.documented >= report.sessions) { + return ( + `Every one of the ${meetings(report.sessions)} recorded in this server reached a protocol.` + + open + ) + } + const missing = report.sessions - report.documented + const share = reportDocumentedShare(report) + return ( + `${formatCount(report.documented)} of the ${meetings(report.sessions)} recorded in this server ` + + `reached a protocol — ${share} %. The other ${formatCount(missing)} ` + + `${plural(missing, 'was', 'were')} recorded and never written up.${open}` + ) +} + +/** + * What the recorded total covers, and what it leaves out. + * + * Said whether or not anything is open. A total whose exclusions are + * mentioned only when they bite is a total whose scope the reader has to + * infer from its own silence. + */ +export function reportRecordedLine(report: GuildReport): string { + if (report.open_sessions > 0) { + return ( + 'Adds up the length of every meeting in this server that has ended. The ' + + `${meetings(report.open_sessions)} still recording ${plural(report.open_sessions, 'is', 'are')} ` + + 'not in it — a meeting has no length until it stops.' + ) + } + return 'Adds up the length of every meeting this server has recorded, all of which have ended.' +} + +/** + * What is happening in this server at this moment, or the fact that + * nothing is. + * + * Shown whether or not it is zero. A figure that appears only when it is + * bad news is a figure whose absence has to be interpreted, and "nothing + * is stuck open" and "this page does not report sessions left open" look + * identical on screen. + * + * The sentence names the second reading on purpose. One meeting open for + * ten minutes is a meeting; one open since Tuesday is a session that never + * closed, and the number alone cannot tell them apart. + */ +export function reportOpenSessionsLine(report: GuildReport): string { + if (report.open_sessions <= 0) { + return ( + 'Nothing is being recorded in this server right now — every meeting it has recorded has ' + + 'ended and has a length.' + ) + } + const count = report.open_sessions + return ( + `${meetings(count)} in this server ${plural(count, 'has', 'have')} no end time yet. That is ` + + 'either a meeting happening at this moment or a session that never closed, and this figure ' + + 'cannot tell the two apart — a session open for days is the second. Neither its length nor ' + + 'its speech is counted anywhere else on this page.' + ) +} + +/** The reason a figure is absent, said as the absence it is rather than + * left as a dash the reader has to account for. */ +function noFinishedMeetings(what: string): string { + return ( + `No meeting in this server has finished, so there is no ${what} to give. This is the absence ` + + 'of a figure and not a figure of zero.' + ) +} + +/** + * The headline band: how much this server has used Sturnus at all. + * + * Meetings first, because it is the question the page is opened with, and + * every figure after it is context -- four hours of recording means one + * thing across three meetings and another across sixty. + */ +export function reportHeadlineFigures(report: GuildReport): ReportFigure[] { + return [ + { + key: 'sessions', + label: 'Meetings recorded', + value: formatCount(report.sessions), + note: reportSpanLine(report), + tone: 'plain', + }, + { + key: 'documented', + label: 'Meetings written up', + value: formatCount(report.documented), + note: reportDocumentedLine(report), + tone: 'plain', + }, + { + key: 'recorded', + label: 'Time recorded', + value: formatDuration(report.recorded_seconds), + note: reportRecordedLine(report), + tone: report.recorded_seconds === null ? 'absent' : 'plain', + }, + { + key: 'speech', + label: 'Time spoken', + value: formatDuration(report.speech_seconds), + note: reportSpeechCaveat(report), + tone: report.speech_seconds === null ? 'absent' : 'plain', + }, + ] +} + +/** + * What a meeting in this server looks like: how long, and how many people. + * + * Kept apart from the headline band because these are a different kind of + * number. Those describe how much has happened; these describe the shape + * of one meeting, and four of the six can legitimately be missing. + * + * Nothing in here names anybody. `distinct_participants` is a count of + * people and is the closest this page comes to the individuals in a + * server; it goes no further, and the note beside it says so rather than + * leaving the reader to wonder whether a list is one click away. + */ +export function reportShapeFigures(report: GuildReport): ReportFigure[] { + const average = report.average_duration_seconds + const longest = report.longest_duration_seconds + const perMeeting = report.average_participants + const largest = report.largest_meeting + + return [ + { + key: 'average-duration', + label: 'Typical meeting', + value: average === null ? NO_FIGURE : formatDuration(average), + note: + average === null + ? noFinishedMeetings('length to average') + : 'The mean length of the meetings that have ended here.', + tone: average === null ? 'absent' : 'plain', + }, + { + key: 'longest-duration', + label: 'Longest meeting', + value: longest === null ? NO_FIGURE : formatDuration(longest), + note: + longest === null + ? noFinishedMeetings('length to compare') + : 'The longest single meeting this server has recorded from start to finish.', + tone: longest === null ? 'absent' : 'plain', + }, + { + key: 'average-participants', + label: 'People per meeting', + value: perMeeting === null ? NO_FIGURE : averageInWords(perMeeting), + note: + perMeeting === null + ? noFinishedMeetings('attendance to average') + : 'The mean number of people recorded in a meeting that has ended here.', + tone: perMeeting === null ? 'absent' : 'plain', + }, + { + key: 'largest-meeting', + label: 'Largest meeting', + value: largest === null ? NO_FIGURE : formatCount(largest), + note: + largest === null + ? noFinishedMeetings('attendance to compare') + : 'The most people recorded in any one meeting in this server.', + tone: largest === null ? 'absent' : 'plain', + }, + { + key: 'participants', + label: 'People recorded', + value: formatCount(report.distinct_participants), + note: + 'How many different people this server has recorded at all. A count and nothing else — ' + + 'Sturnus does not send this page their names, and this page does not ask.', + tone: 'plain', + }, + { + key: 'open', + label: 'Still recording', + value: formatCount(report.open_sessions), + note: reportOpenSessionsLine(report), + tone: report.open_sessions > 0 ? 'watch' : 'plain', + }, + ] +} + +/* -------------------------------------------------------------------- */ +/* The caveats that travel with the figures */ +/* -------------------------------------------------------------------- */ + +/** + * What the speech total is a total *of*. + * + * The single most misreadable number on this page, and the reason this + * paragraph exists. Speaking time is measured per track and the column is + * nullable: null means nobody ever measured that track -- jobs that + * predate the measurement columns -- and zero means somebody measured and + * heard nothing. `SUM` skips the nulls without saying so, so the total is + * short by however much those tracks held. + * + * The failure this wording exists to prevent is specific: a server with + * most of its tracks unmeasured shows a small speech figure under a large + * recorded figure, and the obvious reading is "these meetings were quiet". + * The obvious reading is wrong, so the sentence rules it out in words + * rather than leaving it to be inferred from a ratio. + */ +export function reportSpeechCaveat(report: GuildReport): string { + const { tracks, unmeasured_tracks: unmeasured } = report + + if (tracks <= 0) { + return ( + 'No audio has been recorded in this server, so there is nothing to have measured. The ' + + 'speaking time above is missing rather than zero.' + ) + } + + if (unmeasured <= 0) { + return ( + `Every one of the ${formatCount(tracks)} recorded ${plural(tracks, 'track', 'tracks')} in ` + + 'this server carries a measured speaking time, so the figure above covers all of what was ' + + 'recorded.' + ) + } + + const measured = Math.max(0, tracks - unmeasured) + if (measured === 0) { + return ( + `None of the ${formatCount(tracks)} recorded ${plural(tracks, 'track', 'tracks')} in this ` + + 'server was ever measured for speaking time — they all predate the columns that hold it — ' + + 'so there is no speaking time here to report. Read the figure above as a measurement that ' + + 'was never taken, not as a server that was quiet.' + ) + } + + return ( + `${formatCount(unmeasured)} of the ${formatCount(tracks)} recorded ` + + `${plural(tracks, 'track', 'tracks')} in this server ${plural(unmeasured, 'was', 'were')} ` + + 'never measured for speaking time — they predate the columns that hold it — and a sum skips ' + + `them in silence. The speaking time above is therefore the total for the other ` + + `${formatCount(measured)} ${plural(measured, 'track', 'tracks')} only: it describes part of ` + + 'what was recorded, and the larger this number grows the further short the figure falls. It ' + + 'does not mean this server was quiet.' + ) +} + +/** + * Which calendar the months were cut in. + * + * A reader who is not told a zone assumes their own, and month boundaries + * are exactly where that assumption costs something: a meeting at 00:30 on + * the first falls into either month depending on who is asking. Sturnus + * buckets by the server's own zone on purpose -- a meeting belongs to the + * month the people in it think it does -- and that choice is only worth + * anything if the page names the zone it made. + * + * The second half is the seam this page carries and cannot remove: the + * instants are written in UTC, because a server-side render has no idea + * what zone the reader is in and a second rendering in the browser would + * disagree with the first. So two clocks appear on one page, and saying + * which is which is cheaper than a reader discovering it from a date that + * does not match a month. + */ +export function reportTimezoneNote(report: GuildReport): string { + if (!report.timezone) { + return ( + 'Sturnus did not say which calendar the months below were cut in, so do not assume it is ' + + 'yours: a meeting near midnight falls on either side of a month boundary depending on the ' + + 'zone. Every instant on this page is written in UTC, which may not be that calendar ' + + 'either.' + ) + } + return ( + `The months below are cut in ${report.timezone}, this server's own calendar — not in UTC and ` + + 'not in yours. A meeting that begins at 00:30 belongs to the month the people in it think it ' + + 'does, which is why Sturnus does not bucket by UTC. The instants elsewhere on this page are ' + + 'written in UTC all the same, because a page rendered on a server cannot know your zone, so ' + + 'a meeting near a month boundary can carry a UTC date that reads as the neighbouring month.' + ) +} + +export interface ReportCaveat { + key: string + label: string + text: string +} + +/** + * The two things a reader has to know before the figures above mean what + * they appear to mean. + * + * In a panel of their own rather than as footnotes. A footnote is read + * once, by the person who was already being careful; these two are the + * difference between a figure and a wrong figure. + */ +export function reportCaveats(report: GuildReport): ReportCaveat[] { + return [ + { + key: 'speech', + label: 'What the speaking time covers', + text: reportSpeechCaveat(report), + }, + { + key: 'timezone', + label: 'Which calendar the months use', + text: reportTimezoneNote(report), + }, + ] +} + +/* -------------------------------------------------------------------- */ +/* The span this report covers */ +/* -------------------------------------------------------------------- */ + +/** + * From when to when, in UTC and saying so. + * + * Both ends can be missing independently: a server whose only session is + * still open has a first session and, depending on how the API dates the + * last one, may have nothing to close the span with. Each case gets its + * own sentence rather than an em dash standing in for half a range. + */ +export function reportSpanLine(report: GuildReport): string { + const first = report.first_session_at + const last = report.last_session_at + + if (!first && !last) { + return 'No meeting has been recorded in this server yet, so this report covers no time at all.' + } + if (first && last) { + if (first === last) { + return `One meeting, recorded ${formatMoment(first)}.` + } + return `Everything recorded in this server between ${formatMoment(first)} and ${formatMoment(last)}.` + } + const known = first ?? last + return ( + `Everything recorded in this server. Only one end of the span is known: ${formatMoment(known)}.` + ) +} + +/* -------------------------------------------------------------------- */ +/* The months, and the gaps between them */ +/* -------------------------------------------------------------------- */ + +/** + * The shortest bar a month with any recording is allowed to draw. + * + * A busy server makes its quiet months round to nothing, and a month with + * one meeting rendered as an empty row is indistinguishable from a month + * with none -- which is the one distinction the filled gaps below exist to + * draw. Two per cent is enough to be a mark and little enough not to read + * as a quantity. + */ +export const REPORT_MIN_BAR_EXTENT = 0.02 + +/** + * How long a span this page will fill in with silent months. + * + * Ten years. Beyond that the filling stops being a service and becomes a + * wall: a single stray month in the payload would otherwise produce + * hundreds of rows of zeros and bury the months that carry something. The + * limit is stated on the page when it bites, because a list of months with + * gaps silently left out is exactly what the filling exists to prevent. + */ +export const REPORT_MONTH_FILL_LIMIT = 120 + +export interface ReportMonthRow { + /** `YYYY-MM`, unique across the rows, so it keys a `v-for` safely. */ + month: string + label: string + sessions: number + documented: number + recorded: string + /** True for a month this module added because the payload skipped it. */ + silent: boolean + /** 0 to 1, against the busiest month in the list. The bar's width, and + * nothing else -- it is deliberately not a percentage anybody reads. */ + extent: number + /** The row said as a sentence, for the reader who is listening to the + * page rather than looking at it. A bar with no text is a bar only its + * author can read. */ + detail: string +} + +/** A month key as a count of months since year zero, so two of them can be + * compared and stepped between without ever becoming a `Date` -- which + * would drag a timezone into a calculation that has no instant in it. */ +function monthIndex(month: string): number { + return Number(month.slice(0, 4)) * 12 + (Number(month.slice(5, 7)) - 1) +} + +function monthKey(index: number): string { + const year = Math.floor(index / 12) + const month = (index % 12) + 1 + return `${String(year).padStart(4, '0')}-${String(month).padStart(2, '0')}` +} + +/** + * The months, oldest first, with the empty ones put back. + * + * **The gaps are filled, deliberately.** The API sends only the months in + * which something happened, and a bar row that puts March next to November + * draws them as neighbours -- a server that went quiet for eight months + * would read as one that recorded steadily. The silence is the finding, so + * it gets a row: the same width, a zero, and a sentence saying nothing was + * recorded. Filling happens only *between* the first and last month that + * carry something; a server is not silent in the months before it existed, + * and inventing rows there would be inventing history rather than showing + * a gap in it. + * + * Oldest first because the row is read as a timeline, and a timeline that + * runs backwards has to be re-read before it can be understood. That is + * the opposite of the Queue page's newest-first list, and for the opposite + * reason: nobody scans this for one particular month, they look at its + * shape. + * + * Bars are scaled against the busiest month's meeting count rather than + * against its recorded time. Meetings are the figure the rest of the page + * leads with, and two rows scaled by different quantities cannot be + * compared to each other at a glance, which is all a bar is for. + */ +export function reportMonthRows(report: GuildReport): ReportMonthRow[] { + const present = [...report.months].sort((a, b) => monthIndex(a.month) - monthIndex(b.month)) + if (present.length === 0) return [] + + const byMonth = new Map() + // Last one wins for a duplicated month. A duplicate is a defect + // upstream; two rows with the same heading and different numbers would + // put that defect on screen as though the server had lived the month + // twice. + for (const entry of present) byMonth.set(entry.month, entry) + + const firstIndex = monthIndex(present[0]!.month) + const lastIndex = monthIndex(present[present.length - 1]!.month) + const span = lastIndex - firstIndex + 1 + + const keys + = span <= REPORT_MONTH_FILL_LIMIT + ? Array.from({ length: span }, (_, offset) => monthKey(firstIndex + offset)) + : [...byMonth.keys()].sort() + + const busiest = present.reduce((most, entry) => Math.max(most, entry.sessions), 0) + + return keys.map((key) => { + const entry = byMonth.get(key) + const sessions = entry?.sessions ?? 0 + const documented = entry?.documented ?? 0 + const recorded = entry ? formatDuration(entry.recorded_seconds) : NO_FIGURE + const silent = entry === undefined + const label = reportMonthLabel(key) + + // A month present in the payload with no sessions in it is not the + // same as one the payload skipped, and it does not get the silent + // row's wording -- the API said something about it, and this page + // should not overwrite that with an assumption. + const detail = silent + ? `${label}: nothing was recorded in this server.` + : `${label}: ${meetings(sessions)}, ${recorded} recorded, ` + + `${formatCount(documented)} written up.` + + const scaled = busiest > 0 ? sessions / busiest : 0 + return { + month: key, + label, + sessions, + documented, + recorded, + silent, + extent: sessions > 0 ? Math.max(scaled, REPORT_MIN_BAR_EXTENT) : 0, + detail, + } + }) +} + +/** + * What the month rows are, and what was done about the gaps. + * + * Said above the rows rather than left to be worked out. A reader who + * assumes the API sent every month will read a filled zero as a fact from + * the database, and a reader who assumes it sent only the busy ones will + * read an unfilled list as a steady run of months. Both are wrong in a way + * the page can settle in one sentence. + */ +export function reportMonthsNote(report: GuildReport): string { + const rows = reportMonthRows(report) + if (rows.length === 0) { + return 'No month in this server has any recording in it yet.' + } + // The two questions are asked in this order because a list too long to + // fill has no silent rows in it at all, and would otherwise answer the + // "nothing was skipped" branch by saying nothing was skipped -- which is + // the exact opposite of what happened. + const span = monthIndex(rows[rows.length - 1]!.month) - monthIndex(rows[0]!.month) + 1 + if (rows.length < span) { + return ( + 'Only the months in which something was recorded are listed, oldest first. This server\'s ' + + `history spans more than ${REPORT_MONTH_FILL_LIMIT / 12} years, which is too long to list ` + + 'month by month, so the quiet months between these are not shown as rows — two rows next ' + + 'to each other are not necessarily neighbouring months.' + ) + } + const silent = rows.filter((row) => row.silent).length + if (silent === 0) { + return ( + 'Every month from the first recording in this server to the most recent one, oldest first. ' + + 'Something was recorded in each of them.' + ) + } + return ( + 'Every month from the first recording in this server to the most recent one, oldest first. ' + + `The ${formatCount(silent)} ${plural(silent, 'month', 'months')} in which nothing was ` + + `recorded ${plural(silent, 'is', 'are')} listed with a zero rather than left out, so a quiet ` + + 'stretch reads as a gap instead of closing up.' + ) +} + +/* -------------------------------------------------------------------- */ +/* A server with nothing to report */ +/* -------------------------------------------------------------------- */ + +/** + * Whether this server has ever been recorded at all. + * + * Deliberately stricter than `sessions === 0`. A server with no sessions + * standing next to a hundred and sixty tracks is a defect upstream, and + * showing the figures makes it visible where an empty state would hide it + * behind an invitation to do something that has already been done. The + * same reasoning `hasNothingRecorded` uses on the dashboard, applied to a + * server rather than to a person. + */ +export function isReportEmpty(report: GuildReport): boolean { + return ( + report.sessions === 0 + && report.tracks === 0 + && report.distinct_participants === 0 + && report.months.length === 0 + && report.first_session_at === null + && report.last_session_at === null + ) +} + +/** The empty state, as one sentence and a second saying what would fill + * it. A page of dashes and zeros for a server that has recorded nothing + * reads as a page that failed to load, and is also a claim -- that this + * server holds meetings of no length attended by nobody. */ +export const REPORT_EMPTY_HEADING = 'Sturnus has not recorded anything in this server yet' + +export const REPORT_EMPTY_NOTE = + 'There are no meetings here to report on, so this page shows no figures rather than a grid of ' + + 'zeros — a zero would be a measurement, and nothing has been measured. Once a meeting happens ' + + 'in a channel Sturnus watches and the people in it have consented, this page fills in: how ' + + 'many meetings, how long they ran, how many of them were written up, and how that changed ' + + 'month by month.' + +/** + * What this report is about, and what it is deliberately not about. + * + * On the page rather than only in this file. The payload holds no names + * and no ids, and that is a decision rather than an oversight: a + * per-person readout of who attended which meetings and who spoke for how + * long is a means of monitoring conduct and performance at work, which is + * a matter for a works council and not a console feature. Saying so is + * also the honest answer to the reader who was about to ask where the + * breakdown is. + */ +export const REPORT_SCOPE_NOTE = + 'Every figure here is about the server as a whole and never about the people in it. Sturnus ' + + 'sends this page no names and no ids, and there is no per-person breakdown behind it — how ' + + 'long one named person sat in meetings, or spoke in them, is a measure of that person rather ' + + 'than of this server, and that is not something a console should hand out. Counts of people ' + + 'are counts, and stop there.' + +/* -------------------------------------------------------------------- */ +/* When the API says no */ +/* -------------------------------------------------------------------- */ + +/** `ApiError` names it `status`; a raw `$fetch` failure may name it + * `statusCode`; a request that never got a response has neither, and null + * says so rather than standing in a number that would read as an + * answer. */ +function statusOf(error: unknown): number | null { + if (!isRecord(error)) return null + for (const candidate of [error.status, error.statusCode]) { + if (typeof candidate === 'number' && Number.isFinite(candidate)) { + // `ApiError` uses 0 for "never reached the API", which is + // deliberately distinguishable from every real status. + return candidate === 0 ? null : candidate + } + } + return null +} + +/** + * A failed request, in a sentence somebody can act on. + * + * Built from the status alone. `useApi` throws `ApiError`, which carries + * no body by design -- the API's own `{"error": "no such guild"}` never + * reaches this console -- so every sentence below has to stand on its own + * without it. + * + * Named `describeReportError` rather than `describeError` for the same + * reason `describeQueueError` is: everything under `app/utils` is + * auto-imported into every component, and two exports sharing a name is a + * build warning and a coin toss over which one a page actually gets. + */ +export function describeReportError(error: unknown): string { + const status = statusOf(error) + switch (status) { + case 401: + return 'Your session has ended. Sign in again to see this server’s figures.' + case 403: + return ( + 'You do not administer this server. Administrators are the members holding the role named ' + + 'by that guild’s `admin_role_id`.' + ) + case 404: + // The API answers 404 both for a server that does not exist and for + // one the caller does not administer, on purpose: it will not + // confirm the existence of a server to somebody with no business + // there. So this sentence has to cover both without guessing which. + return ( + 'Sturnus does not know this server, or you no longer administer it — it answers the same ' + + 'way to both. Reload the page; the list of servers is rebuilt from Discord.' + ) + case null: + return ( + 'Could not reach the API. Nothing here is out of date on purpose; check the connection and ' + + 'retry.' + ) + default: + return `Sturnus answered ${status} and could not produce this server’s figures. Nothing is known about why.` + } +} diff --git a/console/test/navigation.spec.ts b/console/test/navigation.spec.ts index 9399cae..adff477 100644 --- a/console/test/navigation.spec.ts +++ b/console/test/navigation.spec.ts @@ -72,6 +72,21 @@ describe('who is offered the Admin View', () => { expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('Queue') }) + it('offers the reporting to an administrator', () => { + expect(visibleEntries(ADMIN).map((e) => e.label)).toContain('Reporting') + }) + + it('hides the reporting from somebody who administers nothing', () => { + // The report describes a whole server -- how much of it was recorded, + // how much of that was written up -- which is a thing an administrator + // is accountable for and a thing a participant has no standing to + // read. The endpoint answers 404 to them, the same answer it gives for + // a server that does not exist, so an entry left visible would offer a + // page that can only ever refuse them. + expect(visibleEntries(PARTICIPANT).map((e) => e.label)).not.toContain('Reporting') + expect(visibleEntries(null).map((e) => e.label)).not.toContain('Reporting') + }) + it('hides the queue from somebody who administers nothing', () => { // The queue endpoint answers 404 to a non-administrator -- the same // answer it gives for a guild that does not exist -- so an entry left @@ -86,10 +101,14 @@ describe('who is offered the Admin View', () => { // that one member, which is rarer and always deliberate. The queue is // rarer still and is never scanned -- it is opened because a protocol // did not appear, which is a question somebody arrives holding. + // Reporting is last because it is the only entry nothing is ever + // wrong on: it is read on a schedule, or when somebody asks how much + // this server actually uses Sturnus, and never in a hurry. expect(ADMIN_VIEW.entries.map((e) => e.label)).toEqual([ 'Bot Settings', 'User Settings', 'Queue', + 'Reporting', ]) }) diff --git a/console/test/reporting.spec.ts b/console/test/reporting.spec.ts new file mode 100644 index 0000000..ec2d214 --- /dev/null +++ b/console/test/reporting.spec.ts @@ -0,0 +1,718 @@ +/** + * What a server's report means, and what the page is allowed to claim + * about it. + * + * All of it lives in `~/utils/reporting` rather than in the page, because + * every one of these is a decision -- how an absent average renders, which + * month comes first, whether a gap in the bar row is drawn, what the + * speaking time is a total of, whether an empty report is a fault -- and a + * decision embedded in a template can only be tested by rendering one. + * + * The wording is asserted here on purpose, and heavily. Four of this + * page's figures are misreadable in a specific direction, and a test that + * only checked the numbers would let the sentence that prevents each + * misreading quietly drop out: + * + * - `speech_seconds` is a sum over a nullable column, so a server with + * many unmeasured tracks looks quiet when it was merely unmeasured. + * - the four nullable figures are null because nothing has finished, and a + * zero in their place would claim meetings of no length attended by + * nobody. + * - `recorded_seconds` excludes the sessions still open, which is where a + * session that never closed goes to hide. + * - `months` are cut in the server's zone and are absent rather than + * zero-filled, so a quiet stretch closes up unless something puts it + * back. + * + * And one thing is asserted by its absence: nothing in this module names + * or ranks a person, because this report is about a server and a + * per-person readout of attendance and speaking time is a works-council + * matter rather than a console feature. + */ +import { describe, expect, it } from 'vitest' + +import { + REPORT_EMPTY_NOTE, + REPORT_MIN_BAR_EXTENT, + REPORT_MONTH_FILL_LIMIT, + REPORT_SCOPE_NOTE, + describeReportError, + isReportEmpty, + parseGuildReport, + reportCaveats, + reportDocumentedLine, + reportDocumentedShare, + reportHeadlineFigures, + reportMonthLabel, + reportMonthRows, + reportMonthsNote, + reportOpenSessionsLine, + reportPath, + reportRecordedLine, + reportShapeFigures, + reportSpanLine, + reportSpeechCaveat, + reportTimezoneNote, + type GuildReport, + type ReportMonth, +} from '../app/utils/reporting' + +/** A server that has recorded a good deal and has nothing odd about it, so + * each test states only the one property it is actually about. */ +function report(overrides: Partial = {}): GuildReport { + return { + guild_id: '4711', + sessions: 42, + documented: 38, + open_sessions: 0, + recorded_seconds: 151200, + speech_seconds: 48000, + unmeasured_tracks: 0, + tracks: 160, + distinct_participants: 12, + average_participants: 3.8, + largest_meeting: 9, + average_duration_seconds: 3600, + longest_duration_seconds: 9000, + first_session_at: '2025-11-04T09:00:00+00:00', + last_session_at: '2026-08-21T12:00:00+00:00', + timezone: 'Europe/Berlin', + months: [], + ...overrides, + } +} + +/** A server that has recorded nothing at all. */ +function emptyReport(overrides: Partial = {}): GuildReport { + return report({ + sessions: 0, + documented: 0, + open_sessions: 0, + recorded_seconds: null, + speech_seconds: null, + unmeasured_tracks: 0, + tracks: 0, + distinct_participants: 0, + average_participants: null, + largest_meeting: null, + average_duration_seconds: null, + longest_duration_seconds: null, + first_session_at: null, + last_session_at: null, + months: [], + ...overrides, + }) +} + +function month(overrides: Partial & { month: string }): ReportMonth { + return { sessions: 1, recorded_seconds: 3600, documented: 1, ...overrides } +} + +/** What `ApiError` looks like to the function that reads a failure. */ +function failure(status: number) { + return { status, path: '/guilds/4711/report' } +} + +/** Every sentence this module can produce for one report, so a test can + * assert on what none of them says. */ +function everySentence(value: GuildReport): string { + return [ + ...reportHeadlineFigures(value).map((figure) => `${figure.label} ${figure.value} ${figure.note}`), + ...reportShapeFigures(value).map((figure) => `${figure.label} ${figure.value} ${figure.note}`), + ...reportCaveats(value).map((caveat) => `${caveat.label} ${caveat.text}`), + ...reportMonthRows(value).map((row) => row.detail), + reportMonthsNote(value), + reportSpanLine(value), + REPORT_SCOPE_NOTE, + REPORT_EMPTY_NOTE, + ].join(' ') +} + +describe('reading the report payload', () => { + it('reads the whole envelope the endpoint sends', () => { + const parsed = parseGuildReport({ + guild_id: '4711', + sessions: 42, + documented: 38, + open_sessions: 1, + recorded_seconds: 151200.0, + speech_seconds: 48000.0, + unmeasured_tracks: 7, + tracks: 160, + distinct_participants: 12, + average_participants: 3.8, + largest_meeting: 9, + average_duration_seconds: 3600.0, + longest_duration_seconds: 9000.0, + first_session_at: '2025-11-04T09:00:00+00:00', + last_session_at: '2026-08-21T12:00:00+00:00', + timezone: 'Europe/Berlin', + months: [{ month: '2026-08', sessions: 5, recorded_seconds: 18000.0, documented: 5 }], + }) + expect(parsed.guild_id).toBe('4711') + expect(parsed.sessions).toBe(42) + expect(parsed.open_sessions).toBe(1) + expect(parsed.average_participants).toBe(3.8) + expect(parsed.timezone).toBe('Europe/Berlin') + expect(parsed.months).toEqual([ + { month: '2026-08', sessions: 5, recorded_seconds: 18000, documented: 5 }, + ]) + }) + + it('keeps a null figure null rather than rounding it to zero', () => { + // The whole reason these four fields are nullable. A server whose + // meetings have not finished has no average length, and a zero here + // would claim its meetings are instantaneous. + const parsed = parseGuildReport({ + average_duration_seconds: null, + longest_duration_seconds: null, + average_participants: null, + largest_meeting: null, + }) + expect(parsed.average_duration_seconds).toBeNull() + expect(parsed.longest_duration_seconds).toBeNull() + expect(parsed.average_participants).toBeNull() + expect(parsed.largest_meeting).toBeNull() + }) + + it('turns a nonsensical optional figure into an absence, not a zero', () => { + // "We do not know" is true of a broken figure; "none" is not. + const parsed = parseGuildReport({ + average_duration_seconds: -5, + largest_meeting: 'nine', + speech_seconds: Number.NaN, + }) + expect(parsed.average_duration_seconds).toBeNull() + expect(parsed.largest_meeting).toBeNull() + expect(parsed.speech_seconds).toBeNull() + }) + + it('never reports a negative or nonsensical count', () => { + // A defect upstream must not render as "-3 meetings" beside a + // server's name, where it reads as a fact about that server. + const parsed = parseGuildReport({ + sessions: -3, + documented: 'many', + tracks: 2.4, + unmeasured_tracks: null, + }) + expect(parsed.sessions).toBe(0) + expect(parsed.documented).toBe(0) + expect(parsed.tracks).toBe(2) + expect(parsed.unmeasured_tracks).toBe(0) + }) + + it('drops a month it cannot place on a calendar', () => { + // A malformed month cannot be ordered or labelled, and would anchor + // the gap filling at an arbitrary point in history -- turning one bad + // string into a thousand rows of zeros. + const parsed = parseGuildReport({ + months: [ + { month: 'last winter', sessions: 3 }, + { month: '2026-13', sessions: 3 }, + { month: '2026-8', sessions: 3 }, + { month: '2026-08', sessions: 3 }, + ], + }) + expect(parsed.months.map((entry) => entry.month)).toEqual(['2026-08']) + }) + + it('yields a well-formed report for a payload it cannot make sense of', () => { + // Never null: a parser that gave up would turn a strange payload into + // a blank page with no error anywhere, which is the failure mode + // hardest to report. + const parsed = parseGuildReport('nonsense') + expect(parsed.guild_id).toBeNull() + expect(parsed.sessions).toBe(0) + expect(parsed.months).toEqual([]) + expect(parsed.timezone).toBe('') + }) + + it('escapes the guild id in the path it builds', () => { + // A string from an API allowed to contain a slash is a string allowed + // to address a different endpoint. + expect(reportPath('4711')).toBe('/guilds/4711/report') + expect(reportPath('../guilds/1')).toBe('/guilds/..%2Fguilds%2F1/report') + }) +}) + +describe('the headline figures', () => { + it('leads with the meetings and says what span they cover', () => { + const figures = reportHeadlineFigures(report()) + expect(figures.map((figure) => figure.label)).toEqual([ + 'Meetings recorded', + 'Meetings written up', + 'Time recorded', + 'Time spoken', + ]) + expect(figures[0]!.value).toBe('42') + expect(figures[0]!.note).toContain('4 Nov 2025, 09:00 UTC') + expect(figures[0]!.note).toContain('21 Aug 2026, 12:00 UTC') + }) + + it('renders a missing total as an absence rather than as no time at all', () => { + const figures = reportHeadlineFigures(report({ recorded_seconds: null, speech_seconds: null })) + const recorded = figures.find((figure) => figure.key === 'recorded')! + expect(recorded.value).toBe('—') + expect(recorded.tone).toBe('absent') + }) + + it('groups the shape of a meeting apart from how much has happened', () => { + expect(reportShapeFigures(report()).map((figure) => figure.label)).toEqual([ + 'Typical meeting', + 'Longest meeting', + 'People per meeting', + 'Largest meeting', + 'People recorded', + 'Still recording', + ]) + }) + + it('writes an average of people to one decimal, without a false one', () => { + const perMeeting = (value: number | null) => + reportShapeFigures(report({ average_participants: value })).find( + (figure) => figure.key === 'average-participants', + )!.value + expect(perMeeting(3.84)).toBe('3.8') + // A whole number is not a measurement to a tenth, and "4.0" claims it + // is. + expect(perMeeting(4)).toBe('4') + }) +}) + +describe('a figure that is missing rather than zero', () => { + it('renders every null as an absence and says why', () => { + // `null` is deliberately not `0`: a server with no closed sessions has + // no average length, and printing a zero would state that its meetings + // are instantaneous and attended by nobody. + const figures = reportShapeFigures( + report({ + average_duration_seconds: null, + longest_duration_seconds: null, + average_participants: null, + largest_meeting: null, + }), + ) + for (const key of ['average-duration', 'longest-duration', 'average-participants', 'largest-meeting']) { + const figure = figures.find((candidate) => candidate.key === key)! + expect(figure.value).toBe('—') + expect(figure.tone).toBe('absent') + expect(figure.note).toContain('No meeting in this server has finished') + expect(figure.note).toContain('not a figure of zero') + } + }) + + it('never leaves a missing figure without the sentence that explains it', () => { + // An em dash with nothing beside it reads as "still loading", which is + // the one thing this page is not doing. + for (const figure of [...reportHeadlineFigures(emptyReport()), ...reportShapeFigures(emptyReport())]) { + expect(figure.note?.trim()).not.toBe('') + expect(figure.note).not.toBeNull() + } + }) +}) + +describe('how the documented rate reads', () => { + it('says how many of this server’s meetings reached a protocol, and how many did not', () => { + // A rate on its own is read as a property of the software; this is a + // property of what happened here, so it is written as "n of m". + const line = reportDocumentedLine(report({ sessions: 42, documented: 38 })) + expect(line).toContain('38 of the 42 meetings recorded in this server reached a protocol') + expect(line).toContain('90 %') + expect(line).toContain('The other 4 were recorded and never written up') + }) + + it('says so plainly when every meeting was written up', () => { + expect(reportDocumentedLine(report({ sessions: 42, documented: 42 }))).toContain( + 'Every one of the 42 meetings recorded in this server reached a protocol', + ) + }) + + it('never rounds an incomplete rate up to all of them', () => { + // 999 of 1000 rounds to 100 %, and "100 %" beside a figure that is not + // all of them tells somebody every meeting is covered when one is not. + expect(reportDocumentedShare(report({ sessions: 1000, documented: 999 }))).toBe(99) + expect(reportDocumentedShare(report({ sessions: 1000, documented: 1000 }))).toBe(100) + }) + + it('never rounds a real success down to none of them', () => { + expect(reportDocumentedShare(report({ sessions: 1000, documented: 1 }))).toBe(1) + expect(reportDocumentedShare(report({ sessions: 1000, documented: 0 }))).toBe(0) + }) + + it('has no rate at all for a server that has recorded nothing', () => { + expect(reportDocumentedShare(emptyReport())).toBeNull() + expect(reportDocumentedLine(emptyReport())).toContain('nothing to write up') + }) + + it('does not count a meeting still recording as a failure to write one up', () => { + // A meeting that has not ended cannot have been written up, and + // blaming the pipeline for the clock is the wrong reading of the same + // two numbers. + const line = reportDocumentedLine(report({ sessions: 42, documented: 38, open_sessions: 1 })) + expect(line).toContain('1 meeting is still recording and cannot have been written up yet') + }) +}) + +describe('the meetings still open', () => { + it('says nothing is open rather than letting the figure vanish', () => { + // A figure that appears only when it is bad news is a figure whose + // absence has to be interpreted. + expect(reportOpenSessionsLine(report({ open_sessions: 0 }))).toContain( + 'Nothing is being recorded in this server right now', + ) + }) + + it('names the second reading of an open session', () => { + // One meeting open for ten minutes is a meeting; one open since + // Tuesday is a session that never closed, and the number alone cannot + // tell them apart. + const line = reportOpenSessionsLine(report({ open_sessions: 1 })) + expect(line).toContain('1 meeting in this server has no end time yet') + expect(line).toContain('a session that never closed') + expect(line).toContain('a session open for days is the second') + }) + + it('marks an open session as worth a second look', () => { + const open = (count: number) => + reportShapeFigures(report({ open_sessions: count })).find((figure) => figure.key === 'open')! + expect(open(0).tone).toBe('plain') + expect(open(3).tone).toBe('watch') + }) + + it('says the recorded total leaves the open meetings out', () => { + // `recorded_seconds` excludes them deliberately, and a total whose + // exclusions go unmentioned is a total whose scope the reader has to + // infer from its own silence. + expect(reportRecordedLine(report({ open_sessions: 2 }))).toContain( + 'The 2 meetings still recording are not in it', + ) + expect(reportRecordedLine(report({ open_sessions: 0 }))).toContain('all of which have ended') + }) +}) + +describe('what the speaking time is a total of', () => { + it('refuses to let a hole in the measurement read as a quiet server', () => { + // The single most misreadable number on this page. `speech_seconds` is + // a SUM over a nullable column and skips the nulls in silence, so a + // small figure under a large recorded total reads as "these meetings + // were quiet" to anybody not told otherwise. + const caveat = reportSpeechCaveat(report({ tracks: 160, unmeasured_tracks: 7 })) + expect(caveat).toContain('7 of the 160 recorded tracks in this server were never measured') + expect(caveat).toContain('a sum skips them in silence') + expect(caveat).toContain('the total for the other 153 tracks only') + expect(caveat).toContain('it describes part of what was recorded') + expect(caveat).toContain('It does not mean this server was quiet.') + }) + + it('says a total covers everything when it does', () => { + const caveat = reportSpeechCaveat(report({ tracks: 160, unmeasured_tracks: 0 })) + expect(caveat).toContain('Every one of the 160 recorded tracks') + expect(caveat).toContain('covers all of what was recorded') + }) + + it('says a measurement was never taken when none of it was', () => { + // Zero measured tracks is not a silent server; it is a server whose + // recordings predate the columns that hold the figure. + const caveat = reportSpeechCaveat(report({ tracks: 160, unmeasured_tracks: 160 })) + expect(caveat).toContain('None of the 160 recorded tracks') + expect(caveat).toContain('a measurement that was never taken, not as a server that was quiet') + }) + + it('says there is nothing to measure when nothing was recorded', () => { + const caveat = reportSpeechCaveat(emptyReport()) + expect(caveat).toContain('No audio has been recorded in this server') + expect(caveat).toContain('missing rather than zero') + }) + + it('carries the caveat with the figure, not only in a panel', () => { + // A footnote is read once, by the person who was already being + // careful. + const speech = reportHeadlineFigures(report({ tracks: 160, unmeasured_tracks: 7 })).find( + (figure) => figure.key === 'speech', + )! + expect(speech.note).toContain('It does not mean this server was quiet.') + }) +}) + +describe('which calendar the months were cut in', () => { + it('names the zone rather than letting the reader assume theirs', () => { + const note = reportTimezoneNote(report({ timezone: 'Europe/Berlin' })) + expect(note).toContain('cut in Europe/Berlin') + expect(note).toContain('not in UTC and not in yours') + }) + + it('says why the server does not bucket by UTC', () => { + // A meeting at 00:30 belongs to the month the people in it think it + // does. + expect(reportTimezoneNote(report())).toContain( + 'A meeting that begins at 00:30 belongs to the month the people in it think it does', + ) + }) + + it('warns that the instants on the page use a different clock again', () => { + // A page rendered on a server cannot know the reader's zone, so the + // timestamps stay in UTC while the months do not. Two clocks on one + // page is a seam worth naming. + expect(reportTimezoneNote(report())).toContain('written in UTC all the same') + }) + + it('reports the uncertainty when the API named no zone at all', () => { + const note = reportTimezoneNote(report({ timezone: '' })) + expect(note).toContain('did not say which calendar') + expect(note).toContain('do not assume it is yours') + }) + + it('puts both caveats where the figures are, in a fixed order', () => { + expect(reportCaveats(report()).map((caveat) => caveat.key)).toEqual(['speech', 'timezone']) + }) +}) + +describe('the span the report covers', () => { + it('writes both ends in UTC and says which zone that is', () => { + expect(reportSpanLine(report())).toBe( + 'Everything recorded in this server between 4 Nov 2025, 09:00 UTC and 21 Aug 2026, 12:00 UTC.', + ) + }) + + it('says there is no span rather than printing a dash for one', () => { + expect(reportSpanLine(emptyReport())).toContain('covers no time at all') + }) + + it('reads a single meeting as one meeting, not as a range of no length', () => { + const line = reportSpanLine( + report({ first_session_at: '2026-08-21T12:00:00+00:00', last_session_at: '2026-08-21T12:00:00+00:00' }), + ) + expect(line).toBe('One meeting, recorded 21 Aug 2026, 12:00 UTC.') + }) + + it('says so when only one end of the span is known', () => { + const line = reportSpanLine(report({ last_session_at: null })) + expect(line).toContain('Only one end of the span is known') + expect(line).toContain('4 Nov 2025, 09:00 UTC') + }) +}) + +describe('the months, and the gaps between them', () => { + it('lists them oldest first, so the row reads as a timeline', () => { + // The opposite of the Queue page's newest-first list, and for the + // opposite reason: nobody scans this for one particular month, they + // look at its shape. + const rows = reportMonthRows( + report({ months: [month({ month: '2026-03' }), month({ month: '2026-01' })] }), + ) + expect(rows.map((row) => row.month)).toEqual(['2026-01', '2026-02', '2026-03']) + }) + + it('puts a skipped month back as a row rather than letting the gap close up', () => { + // The API sends only the months in which something happened. A bar row + // that puts March next to November draws them as neighbours, and a + // server that went quiet for eight months reads as one that recorded + // steadily. + const rows = reportMonthRows( + report({ months: [month({ month: '2026-03' }), month({ month: '2026-11' })] }), + ) + expect(rows).toHaveLength(9) + expect(rows.filter((row) => row.silent)).toHaveLength(7) + expect(rows[1]!.silent).toBe(true) + expect(rows[1]!.sessions).toBe(0) + expect(rows[1]!.detail).toBe('April 2026: nothing was recorded in this server.') + }) + + it('invents no months before the first or after the last', () => { + // A server is not silent in the months before it existed, and rows + // there would be inventing history rather than showing a gap in it. + const rows = reportMonthRows(report({ months: [month({ month: '2026-08' })] })) + expect(rows.map((row) => row.month)).toEqual(['2026-08']) + }) + + it('has no rows at all for a server with no months', () => { + expect(reportMonthRows(emptyReport())).toEqual([]) + expect(reportMonthsNote(emptyReport())).toContain('No month in this server has any recording') + }) + + it('says what it did about the gaps, either way', () => { + const filled = report({ months: [month({ month: '2026-03' }), month({ month: '2026-06' })] }) + expect(reportMonthsNote(filled)).toContain( + 'The 2 months in which nothing was recorded are listed with a zero rather than left out', + ) + const solid = report({ months: [month({ month: '2026-03' }), month({ month: '2026-04' })] }) + expect(reportMonthsNote(solid)).toContain('Something was recorded in each of them') + }) + + it('stops filling a span too long to list, and says it stopped', () => { + // A single stray month would otherwise produce hundreds of rows of + // zeros and bury the months that carry something -- but a list with + // gaps silently left out is exactly what the filling exists to + // prevent, so the page has to admit it. + const far = report({ + months: [month({ month: '2010-01' }), month({ month: '2026-08' })], + }) + expect(reportMonthRows(far)).toHaveLength(2) + const note = reportMonthsNote(far) + expect(note).toContain('Only the months in which something was recorded are listed') + expect(note).toContain(`more than ${REPORT_MONTH_FILL_LIMIT / 12} years`) + expect(note).toContain('not necessarily neighbouring months') + }) + + it('scales the bars against the busiest month, and never to nothing', () => { + // A busy server makes its quiet months round to nothing, and a month + // with one meeting rendered as an empty row is indistinguishable from + // a month with none. + const rows = reportMonthRows( + report({ + months: [ + month({ month: '2026-01', sessions: 1 }), + month({ month: '2026-02', sessions: 400 }), + ], + }), + ) + expect(rows[1]!.extent).toBe(1) + expect(rows[0]!.extent).toBe(REPORT_MIN_BAR_EXTENT) + expect(rows[0]!.extent).toBeGreaterThan(0) + }) + + it('draws no bar for a month in which nothing happened', () => { + const rows = reportMonthRows( + report({ months: [month({ month: '2026-01' }), month({ month: '2026-03' })] }), + ) + expect(rows[1]!.extent).toBe(0) + }) + + it('gives every row a sentence for somebody who cannot see the bar', () => { + // A bar with no text is a bar only its author can read. + const rows = reportMonthRows( + report({ + months: [month({ month: '2026-08', sessions: 5, recorded_seconds: 18000, documented: 5 })], + }), + ) + expect(rows[0]!.detail).toBe('August 2026: 5 meetings, 5 h recorded, 5 written up.') + }) + + it('does not describe a month the API sent as empty the way it describes one it skipped', () => { + // The API said something about it, and this page should not overwrite + // that with an assumption. + const rows = reportMonthRows( + report({ + months: [ + month({ month: '2026-01', sessions: 0, recorded_seconds: 0, documented: 0 }), + month({ month: '2026-02' }), + ], + }), + ) + expect(rows[0]!.silent).toBe(false) + expect(rows[0]!.detail).toContain('0 meetings') + }) + + it('keys every row uniquely, so a duplicate month cannot render twice', () => { + const rows = reportMonthRows( + report({ months: [month({ month: '2026-01' }), month({ month: '2026-01', sessions: 9 })] }), + ) + expect(rows).toHaveLength(1) + expect(rows[0]!.sessions).toBe(9) + }) + + it('names a month in full rather than by its key', () => { + expect(reportMonthLabel('2026-08')).toBe('August 2026') + expect(reportMonthLabel('2025-11')).toBe('November 2025') + }) + + it('hands back a month key it cannot read rather than a blank', () => { + // A raw key is at least something the reader can match against the + // payload. + expect(reportMonthLabel('whenever')).toBe('whenever') + }) +}) + +describe('a server with nothing to report', () => { + it('recognises one that has recorded nothing at all', () => { + expect(isReportEmpty(emptyReport())).toBe(true) + }) + + it('is not empty once a single meeting exists', () => { + expect(isReportEmpty(emptyReport({ sessions: 1 }))).toBe(false) + expect(isReportEmpty(emptyReport({ first_session_at: '2026-08-21T12:00:00+00:00' }))).toBe(false) + expect(isReportEmpty(emptyReport({ months: [month({ month: '2026-08' })] }))).toBe(false) + }) + + it('shows the figures rather than the empty state for a report that contradicts itself', () => { + // No sessions standing next to a hundred and sixty tracks is a defect + // upstream, and showing the figures makes it visible where an empty + // state would hide it behind an invitation to do something that has + // already been done. + expect(isReportEmpty(emptyReport({ tracks: 160 }))).toBe(false) + expect(isReportEmpty(emptyReport({ distinct_participants: 4 }))).toBe(false) + }) + + it('says so in a sentence rather than in a wall of dashes', () => { + expect(REPORT_EMPTY_NOTE).toContain('shows no figures rather than a grid of zeros') + expect(REPORT_EMPTY_NOTE).toContain('a zero would be a measurement') + expect(REPORT_EMPTY_NOTE).toContain('Once a meeting happens') + }) +}) + +describe('what this report is not about', () => { + it('says outright that there is no per-person breakdown behind it', () => { + // A per-person readout of meeting attendance and speaking time is a + // means of monitoring conduct and performance at work, which is a + // works-council matter rather than a console feature. The payload + // carries no names and no ids; the framing has to match. + expect(REPORT_SCOPE_NOTE).toContain('about the server as a whole and never about the people in it') + expect(REPORT_SCOPE_NOTE).toContain('no per-person breakdown behind it') + expect(REPORT_SCOPE_NOTE).toContain('Counts of people are counts, and stop there.') + }) + + it('describes the participant count as a count and nothing more', () => { + const people = reportShapeFigures(report()).find((figure) => figure.key === 'participants')! + expect(people.value).toBe('12') + expect(people.note).toContain('A count and nothing else') + expect(people.note).toContain('does not send this page their names') + }) + + it('never hints that a per-person readout is on its way', () => { + // Every sentence this module can produce, checked at once: an + // interface that promises a breakdown is as much a promise as one that + // ships it. + const prose = everySentence( + report({ months: [month({ month: '2026-01' }), month({ month: '2026-03' })], open_sessions: 1 }), + ) + for (const forbidden of [' soon', 'coming', 'per person', 'per-person breakdown of', 'who spoke most', 'top speaker']) { + expect(prose.toLowerCase()).not.toContain(forbidden.toLowerCase()) + } + }) +}) + +describe('when the API says no', () => { + it('says a session has ended rather than that the server is gone', () => { + expect(describeReportError(failure(401))).toContain('Sign in again') + }) + + it('names what an administrator is when it refuses one who is not', () => { + expect(describeReportError(failure(403))).toContain('admin_role_id') + }) + + it('covers both readings of a 404 without guessing which', () => { + // The API answers 404 both for a server that does not exist and for + // one the caller does not administer, on purpose: it will not confirm + // the existence of a server to somebody with no business there. + const message = describeReportError(failure(404)) + expect(message).toContain('does not know this server, or you no longer administer it') + expect(message).toContain('answers the same way to both') + }) + + it('separates a refusal from never having reached the API at all', () => { + // `ApiError` uses 0 for "never got a response", which must not read as + // an answer from the API. + expect(describeReportError(failure(0))).toContain('Could not reach the API') + expect(describeReportError(null)).toContain('Could not reach the API') + }) + + it('names an unexpected status rather than inventing a reason for it', () => { + expect(describeReportError(failure(503))).toContain('Sturnus answered 503') + expect(describeReportError(failure(503))).toContain('Nothing is known about why') + }) + + it('reads a raw fetch failure’s statusCode as well as an ApiError’s status', () => { + expect(describeReportError({ statusCode: 403 })).toContain('admin_role_id') + }) +}) From 65c65881e323478db214d5e8816204f2eaaf93ae Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:26:40 +0200 Subject: [PATCH 11/14] docs(operations): how to read the guild report, and what it deliberately omits Two numbers on that page are easy to read as something they are not, and both are written down here: months are cut in the guild's own calendar rather than in UTC or the reader's -- deliberately a different choice from the personal calendar view, and the reason for the difference is stated -- and "unmeasured tracks" is the size of the hole in the speaking total rather than evidence that the guild was quiet. The section also records why the report stops where it does. It is about a guild and never about a named person, and that is a decision rather than an omission: a per-person readout of meeting attendance and speaking time is subject to co-determination, and the rows for it exist. Somebody reading the runbook should find that written down rather than conclude the feature was simply forgotten. --- docs/operations.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/docs/operations.md b/docs/operations.md index dc040c3..7b15cd5 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1152,6 +1152,47 @@ There is no re-queue control on this page. Each row links to of whether a re-queue is safe stays in one place (`plan_requeue`) rather than being made twice. +### 6.2.8 The report + +**Admin View → Reporting**, per guild: how often this guild meets, how +long its meetings run, how many of them produced a protocol, how big they +get, and the same broken down by month. + +It answers the question an administrator configuring Sturnus otherwise +cannot: is this working out. Every figure comes from rows the system +already writes. + +Two properties are worth knowing before reading a number off it. + +**Months are cut in the guild's `timezone`**, the same calendar the +protocols are written in (§Spec 11), and the payload names which zone was +used. A meeting that opened at 00:30 Berlin time belongs to the month the +people in it think it does; bucketing by UTC would file it under the +previous one and disagree with the timestamps printed in the protocol of +that very meeting. An unusable `timezone` value falls back to UTC — the +same fallback the worker applies — and the named zone in the report is how +you find out that happened. Note this is a *different* choice from the +per-person calendar view (§6.2.5), which groups by UTC day because one +person's sessions can span guilds and no guild's zone is right for them. + +**"Unmeasured tracks" is not zero speech.** `speech_seconds` is nullable +and null means nobody ever measured, while zero means somebody did and it +was nothing. Recordings from before the measurement columns existed have +null, and `SUM` skips them silently. The count of skipped tracks is +therefore printed beside the total: a large one means the speech figure +describes only part of what was recorded, not that the guild was quiet. + +**What the report is not.** It is about a guild and never about a named +person. It says how big meetings get and how many distinct people the +guild has recorded; it does not say who they were or rank them. + +That boundary is a decision rather than an omission. A per-person readout +of meeting attendance and speaking time is a means of monitoring +performance and conduct — in Germany and the EU a matter for a works +council (BetrVG §87(1)(6)) rather than something a console adds because +the columns happen to be there. The rows exist and the ranking is +buildable; building it is a separate, deliberate act. + ### 6.3 Listening to a recording by hand Every automated check this system has can describe a track — its level, From d80ee6f5680b4da8f942fa91ce4f3ce325377fea Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:25:21 +0200 Subject: [PATCH 12/14] feat(console): rank who took part in the most of a guild's meetings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `GET /api/guilds/{guild_id}/report/participation`. **This is the one thing Sturnus produces that names other people and ranks them, and whether to have it is not a technical decision.** Everything else in the console is about the person reading it or about a guild in aggregate; `sturnus.console.reporting` stops short of this on purpose and says so. This is where that line is crossed, deliberately, in a change that can be reverted on its own. An ordered list of colleagues by meeting attendance and speaking time is a `technische Einrichtung, die dazu bestimmt ist, das Verhalten oder die Leistung der Arbeitnehmer zu überwachen` -- BetrVG §87(1)(6) -- and is subject to co-determination in a German workplace with a works council, whether or not anybody intended it as a monitoring tool. The GDPR half is the same point from the other side: the recordings were collected so a protocol could be written, and an attendance ranking serves a further purpose from the same data. Neither makes it wrong to have. Both make it a decision for the people who run the guild rather than a field that appeared in a payload. So the change is shaped to keep that decision visible and reversible: - **Its own port, endpoint, adapter and module.** Not a method on `GuildReports`, which promises in its own docstring to name nobody and would have that made false. Not having this is one revert rather than an audit of a shared response shape, and `/report` keeps working without it. - **Every read emits an audit line**, which no other read in this console does. `console.participation_read` carries `guild_id`, `requested_by` and how many people were in the answer -- and never who they were, because the list is the thing under discussion and copying it into a retained searchable store would be making a second copy of it. "Who looked at the attendance ranking, and when" is the first question anybody reviewing this arrangement will ask, and there was no answer to it before this line. - **It reports attendance and speaking time and nothing further.** No words spoken, no punctuality, no share of talk, nothing per meeting. Each of those is another purpose and would need deciding again. - **It is ordered by meetings attended and never by speaking time.** "Was present most often" and "talked the most" are different statements about a colleague, and only one of them was asked for. The order is total -- meetings, then name, then id -- so a list of named people does not appear to reshuffle itself between two page loads, and somebody the system has no name for sorts after everyone it does. The session count travels with the ranking because a rank means nothing without it: "in eleven meetings" is one claim out of twelve and quite another out of four hundred. Null is not zero here either: somebody whose recordings predate the measurement columns has no speaking total rather than a zero, and the count of unmeasured tracks says how large the hole is. --- src/sturnus/console/adapters.py | 146 ++++++++ src/sturnus/console/app.py | 5 +- src/sturnus/console/participation.py | 138 +++++++ src/sturnus/console/ports.py | 38 ++ src/sturnus/console/routes_report.py | 119 ++++-- src/sturnus/entrypoints/api.py | 2 + src/sturnus/observability/events.py | 9 + tests/console/conftest.py | 22 ++ tests/console/test_participation.py | 539 +++++++++++++++++++++++++++ 9 files changed, 996 insertions(+), 22 deletions(-) create mode 100644 src/sturnus/console/participation.py create mode 100644 tests/console/test_participation.py diff --git a/src/sturnus/console/adapters.py b/src/sturnus/console/adapters.py index 749d7d7..e0f8e5a 100644 --- a/src/sturnus/console/adapters.py +++ b/src/sturnus/console/adapters.py @@ -29,9 +29,11 @@ from sturnus.application.linking import new_state from sturnus.application.publishing import DOCUMENTED_STATUS +from sturnus.console.participation import Attendance from sturnus.console.ports import ( AdminDirectory, ConsentHolder, + GuildParticipation, GuildQueue, GuildRecording, QueuedSession, @@ -797,3 +799,147 @@ def _zone(name: str) -> tuple[tzinfo, str]: return ZoneInfo(name), name except (ZoneInfoNotFoundError, ValueError): return UTC, "UTC" + + +class ConsoleParticipationReports: + """Who took part in a guild's meetings, counted per person. + + A class of its own rather than a method on `ConsoleGuildReports`, for + the reason `ParticipationReports` is a protocol of its own: that one + reads participant rows and never carries an identity out of the + statement, and this one exists to do exactly that. Keeping them apart + is what lets the aggregate report keep saying it names nobody, and + what makes not having this one a revert rather than an audit. + + Read `sturnus.console.participation` before extending it. This is the + only place in the console where a list of colleagues is ranked, and + the reasons that is a decision rather than a feature are written down + there. + + Two statements rather than a join, the same trade the rest of this + module makes: a join from participants to jobs multiplies rows, and a + person in twelve sessions with a track in each comes back a hundred + and forty-four times. + """ + + def __init__( + self, + session_factory: async_sessionmaker[AsyncSession], + admins: AdminDirectory, + ) -> None: + self._session_factory = session_factory + self._admins = admins + + async def attendance_in(self, guild_id: int, *, requested_by: int) -> GuildParticipation | None: + if not await self._admins.is_admin(guild_id, requested_by): + return None + + async with self._session_factory() as db: + sessions = int( + await db.scalar( + select(func.count()) + .select_from(SessionRow) + .where(SessionRow.guild_id == guild_id) + ) + or 0 + ) + in_guild = select(SessionRow.id).where(SessionRow.guild_id == guild_id) + attended = ( + await db.execute( + select( + SessionParticipant.discord_user_id, + # `DISTINCT` is redundant today and kept anyway: + # `uq_participant_per_session` is what actually + # guarantees one row per person per session, and a + # plain `COUNT(*)` would be a ranking of colleagues + # that silently becomes a ranking by how often + # their connection dropped the day that constraint + # is relaxed. Cheap insurance on a number nobody + # would re-derive by hand. + func.count(func.distinct(SessionParticipant.session_id)), + func.min(SessionRow.started_at), + func.max(SessionRow.started_at), + ) + .join(SessionRow, SessionRow.id == SessionParticipant.session_id) + .where(SessionRow.guild_id == guild_id) + .group_by(SessionParticipant.discord_user_id) + ) + ).all() + if not attended: + return GuildParticipation((), sessions) + + people = [row[0] for row in attended] + spoken = { + discord_user_id: (speech, int(unmeasured)) + for discord_user_id, speech, unmeasured in ( + await db.execute( + select( + TranscriptionJob.discord_user_id, + func.sum(TranscriptionJob.speech_seconds), + # Null is not zero: `SUM` skips nulls silently, + # so the rows it skipped are counted beside it. + # Without this a person whose recordings + # predate the measurement columns reads as + # having said nothing. + func.count().filter(TranscriptionJob.speech_seconds.is_(None)), + ) + .where( + TranscriptionJob.session_id.in_(in_guild), + TranscriptionJob.discord_user_id.in_(people), + ) + .group_by(TranscriptionJob.discord_user_id) + ) + ).all() + } + names = await self._latest_display_names(guild_id, people) + + return GuildParticipation( + people=tuple( + Attendance( + discord_user_id=discord_user_id, + display_name=names.get(discord_user_id), + sessions=int(count), + speech_seconds=spoken.get(discord_user_id, (None, 0))[0], + unmeasured_tracks=spoken.get(discord_user_id, (None, 0))[1], + first_seen_at=first_seen, + last_seen_at=last_seen, + ) + for discord_user_id, count, first_seen, last_seen in attended + ), + sessions=sessions, + ) + + async def _latest_display_names(self, guild_id: int, people: Sequence[int]) -> dict[int, str]: + """The name each person last appeared under in this guild. + + The same read `ConsoleConsentDirectory` makes and for the same + reason -- a page of eighteen-digit numbers is not one anybody can + act on -- and scoped to this guild for the same reason too: a + display name is per-guild, and borrowing one from elsewhere would + put a nickname from another server next to a statement about this + one. + """ + async with self._session_factory() as db: + rows = ( + await db.execute( + select( + SessionParticipant.discord_user_id, + SessionParticipant.discord_display_name, + SessionRow.started_at, + ) + .join(SessionRow, SessionRow.id == SessionParticipant.session_id) + .where( + SessionRow.guild_id == guild_id, + SessionParticipant.discord_user_id.in_(people), + ) + .order_by( + SessionParticipant.discord_user_id, + SessionRow.started_at.desc(), + ) + ) + ).all() + + names: dict[int, str] = {} + for discord_user_id, display_name, _started_at in rows: + names.setdefault(discord_user_id, display_name) + return names diff --git a/src/sturnus/console/app.py b/src/sturnus/console/app.py index c4b98db..35b4334 100644 --- a/src/sturnus/console/app.py +++ b/src/sturnus/console/app.py @@ -40,6 +40,7 @@ GuildReports, LinkDirectory, OAuthClient, + ParticipationReports, QueueControl, QueueOverview, SessionReads, @@ -52,7 +53,7 @@ from sturnus.console.routes_consent import register as register_consent from sturnus.console.routes_queue import QUEUE_CONTROL, QUEUE_OVERVIEW from sturnus.console.routes_queue import register as register_queue -from sturnus.console.routes_report import GUILD_REPORTS +from sturnus.console.routes_report import GUILD_REPORTS, PARTICIPATION_REPORTS from sturnus.console.routes_report import register as register_report from sturnus.console.session import ( ExpiredSession, @@ -278,6 +279,7 @@ def build_api( queues: QueueOverview, consents: ConsentDirectory, reports: GuildReports, + participation: ParticipationReports, ) -> web.Application: """Builds the application, with every collaborator injected. @@ -306,6 +308,7 @@ def build_api( app[QUEUE_OVERVIEW] = queues app[CONSENT_DIRECTORY] = consents app[GUILD_REPORTS] = reports + app[PARTICIPATION_REPORTS] = participation app.add_routes( [ web.get("/healthz", healthz), diff --git a/src/sturnus/console/participation.py b/src/sturnus/console/participation.py new file mode 100644 index 0000000..7a2b656 --- /dev/null +++ b/src/sturnus/console/participation.py @@ -0,0 +1,138 @@ +"""Who took part in the most of a guild's meetings. + +**Read this before extending anything in this module.** + +Everything else the console reports is about a guild or about the person +reading it. This is the one thing that is about *other people*, named, and +ranked. `sturnus.console.reporting` says so in as many words and stops +short of it deliberately; this module is where that line is crossed, on +purpose, in a change that can be reverted on its own. + +What it produces is a list of colleagues ordered by how many meetings they +attended and how long they spoke. In Germany and the EU that is a +`technische Einrichtung, die dazu bestimmt ist, das Verhalten oder die +Leistung der Arbeitnehmer zu überwachen` -- BetrVG §87(1)(6) -- and +introducing one is subject to co-determination whether or not anybody +intended it as a monitoring tool. The GDPR half is the same point from the +other side: the recordings were collected so a protocol could be written, +and an attendance ranking is a different purpose served from the same +data. + +None of that makes it wrong to have. It makes it a decision for the people +who run the guild rather than a field that appeared in a payload. So: + +- it is its own port, its own endpoint and its own module, so that not + having it is one revert rather than an audit of a shared response shape; +- reading it emits an audit line, because "who looked at the attendance + ranking, and when" is precisely the question anyone reviewing this + arrangement would ask first; +- the numbers it reports are the ones that answer the stated question and + no more. There is no words-spoken, no punctuality, no talk-ratio, and + nothing derived per meeting. Each of those would be a further purpose, + and each would need deciding again. + +The rules from `sturnus.console.statistics` hold here unchanged: + +- **Null is not zero.** A track nobody measured contributes nothing to a + speaking total and is counted separately, so a person whose recordings + predate the measurement columns does not read as silent. +- **Every id is a string.** A Discord snowflake exceeds JavaScript's safe + integer range. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from dataclasses import dataclass +from datetime import datetime +from typing import TypedDict + + +@dataclass(frozen=True) +class Attendance: + """One person's participation in one guild's meetings.""" + + discord_user_id: int + #: The name they last appeared under in this guild, from + #: `session_participant`. Per guild, because a display name is. + display_name: str | None + sessions: int + #: Summed over the tracks that carry a measurement. `None` when not one + #: of them does -- a different fact from zero, and reported as one. + speech_seconds: float | None + #: Tracks of theirs that nobody measured. The size of the hole in the + #: figure above, for this person. + unmeasured_tracks: int + first_seen_at: datetime + last_seen_at: datetime + + +class AttendanceJson(TypedDict): + discord_user_id: str + display_name: str | None + sessions: int + speech_seconds: float | None + unmeasured_tracks: int + first_seen_at: str + last_seen_at: str + + +class ParticipationJson(TypedDict): + guild_id: str + #: How many of the guild's sessions the ranking is computed over. Sent + #: because "eleven meetings" means one thing out of twelve and quite + #: another out of four hundred, and a bare rank invites the first + #: reading regardless. + sessions: int + people: list[AttendanceJson] + + +def participation( + attendance: Sequence[Attendance], *, guild_id: int, sessions: int +) -> ParticipationJson: + """The ranking, ordered by attendance and then made stable. + + Most meetings first, because that is the question. Ties break on the + display name and then on the id -- never left to the order rows came + back in, which changes between two page loads and would make a list of + named colleagues appear to reshuffle itself. + + Speaking time is deliberately *not* the sort key and is not offered as + one. Ordering people by how much they talked is a different claim from + ordering them by how often they were present, and it is the one that + reads as a judgement. + """ + ordered = sorted( + attendance, + key=lambda person: ( + -person.sessions, + # `None` sorts after every name rather than before, so somebody + # the system has no name for does not head a tie in a list + # people read top-down. + person.display_name is None, + (person.display_name or "").lower(), + person.discord_user_id, + ), + ) + return ParticipationJson( + guild_id=str(guild_id), + sessions=sessions, + people=[_person_json(person) for person in ordered], + ) + + +def _person_json(person: Attendance) -> AttendanceJson: + return AttendanceJson( + # A Discord snowflake exceeds JavaScript's safe integer range, + # where a JSON number silently loses its last digits and produces + # an id that looks right and names nobody. + discord_user_id=str(person.discord_user_id), + display_name=person.display_name, + sessions=person.sessions, + # Straight through, null included. Null means nobody ever + # measured; zero means somebody did and it was nothing. + speech_seconds=person.speech_seconds, + unmeasured_tracks=person.unmeasured_tracks, + first_seen_at=person.first_seen_at.isoformat(), + last_seen_at=person.last_seen_at.isoformat(), + ) diff --git a/src/sturnus/console/ports.py b/src/sturnus/console/ports.py index b5581c5..3f16c4f 100644 --- a/src/sturnus/console/ports.py +++ b/src/sturnus/console/ports.py @@ -20,6 +20,7 @@ from datetime import date, datetime, tzinfo from typing import Protocol +from sturnus.console.participation import Attendance from sturnus.console.reporting import RecordedSession from sturnus.console.statistics import AttendedSession from sturnus.infrastructure.documents.outline_oauth import ExternalIdentity @@ -472,3 +473,40 @@ class GuildReports(Protocol): """ async def recording_of(self, guild_id: int, *, requested_by: int) -> GuildRecording | None: ... + + +@dataclass(frozen=True) +class GuildParticipation: + """Who took part in a guild's meetings, and how many there were. + + The session count travels with the people because a rank means + nothing without it: "in eleven meetings" is one claim out of twelve + and quite another out of four hundred. + + Read `sturnus.console.participation` before touching anything that + reaches this. It is the one thing the console reports that is about + other people, named and ranked, and it is a separate port for a + separate endpoint precisely so that not having it stays one revert. + """ + + people: tuple[Attendance, ...] + sessions: int + + +class ParticipationReports(Protocol): + """A guild's attendance ranking, if the person asking administers it. + + Deliberately not a method on `GuildReports`. That protocol answers + about a guild in aggregate and says in its own docstring that it names + nobody; adding a ranking to it would make that sentence false and + would put the two behind one authorisation call, one endpoint and one + decision. They are different decisions. + + `requested_by` is not optional, for the reason no directory here takes + an optional one, and `None` covers "no such guild" and "you do not + administer it" alike. + """ + + async def attendance_in( + self, guild_id: int, *, requested_by: int + ) -> GuildParticipation | None: ... diff --git a/src/sturnus/console/routes_report.py b/src/sturnus/console/routes_report.py index 82c7e9b..f95fda6 100644 --- a/src/sturnus/console/routes_report.py +++ b/src/sturnus/console/routes_report.py @@ -1,6 +1,7 @@ -"""What a guild's recording adds up to. +"""What a guild's recording adds up to, and who took part in it. - `GET /api/guilds/{guild_id}/report` +- `GET /api/guilds/{guild_id}/report/participation` **Why this exists.** An administrator configuring Sturnus has no way to tell whether it is working out. How often does this guild actually meet, @@ -9,50 +10,70 @@ from rows the system already writes, and none of them is answerable from anywhere in the product today. -**What this endpoint is not, and why that is deliberate.** It reports on a -*guild*: how much was recorded, over which months, how big the meetings -were, how many distinct people the guild has recorded. It does not name a -person and it does not rank anybody. - -That is a boundary rather than a gap. A per-person readout of meeting -attendance and speaking time — "who was in the most sessions" — is a -different artifact from a usage report: in Germany and the EU it is a -means of monitoring performance and conduct, which is a matter for a works -council rather than something a console adds because the columns happen to -be there. The rows exist. Turning them into a ranking of colleagues is a -decision somebody has to take on purpose, and this endpoint is built so -that taking it is a visible act rather than a field that appeared in a -payload. +**Two endpoints, because they are two decisions.** + +The first reports on a *guild*: how much was recorded, over which months, +how big the meetings were, how many distinct people the guild has +recorded. It names nobody, and `sturnus.console.reporting` is built so +that it cannot: it is handed counts rather than people. + +The second is an attendance ranking — named individuals, ordered by how +many meetings they were in. That is a different artifact from a usage +report. In Germany and the EU a per-person readout of attendance and +speaking time is a means of monitoring performance and conduct, subject to +co-determination (BetrVG §87(1)(6)) whether or not anybody intended it as +one. So it is a separate path, a separate collaborator and a separate +module, and reading it emits an audit line — see `participation_view` and +`sturnus.console.participation`. A deployment that should not offer it +does not have to unpick the first one to stop. **404, never 403**, for a guild this person does not administer — the same answer as for a guild that does not exist. The report says when a guild meets and how often, which is a description of a team's working week. -**No decision is taken here.** The shaping is -`sturnus.console.reporting`, which is pure and tested without a database; -this module is the shape of one HTTP response and nothing else. +**No arithmetic is done here.** The shaping is +`sturnus.console.reporting` and `sturnus.console.participation`, both +pure and tested without a database; this module is the shape of two HTTP +responses and one log line. """ from __future__ import annotations +import logging + from aiohttp import web -from sturnus.console.ports import GuildReports +from sturnus.console.participation import participation +from sturnus.console.ports import GuildReports, ParticipationReports from sturnus.console.reporting import guild_report +from sturnus.observability.events import Event, log_event + +log = logging.getLogger(__name__) #: Where the collaborator is found. Its own key rather than a parameter to #: `register`, so `build_api` stays a one-line edit -- several agents are #: adding sections to that function and each extra line is a merge by hand. GUILD_REPORTS: web.AppKey[GuildReports] = web.AppKey("guild_reports") +#: The attendance ranking's collaborator. A second key rather than one +#: object answering both, because the two are different decisions -- see +#: `participation_view` below and `sturnus.console.participation`. +PARTICIPATION_REPORTS: web.AppKey[ParticipationReports] = web.AppKey("participation_reports") + _REPORT_PATH = "/api/guilds/{guild_id}/report" +_PARTICIPATION_PATH = "/api/guilds/{guild_id}/report/participation" def register(app: web.Application) -> None: - """Adds the report route to an application that already has its reports.""" + """Adds the report routes to an application that already has its reports.""" from sturnus.console.app import require_session - app.add_routes([web.get(_REPORT_PATH, require_session(guild_report_view))]) + app.add_routes( + [ + web.get(_REPORT_PATH, require_session(guild_report_view)), + web.get(_PARTICIPATION_PATH, require_session(participation_view)), + ] + ) async def guild_report_view(request: web.Request) -> web.Response: @@ -85,6 +106,62 @@ async def guild_report_view(request: web.Request) -> web.Response: ) +async def participation_view(request: web.Request) -> web.Response: + """Who took part in the most of this guild's meetings. + + **The one endpoint in this console that names other people and ranks + them**, and the reasons that is a decision rather than a feature are + in `sturnus.console.participation`. Two things follow from it here. + + It is a *separate route with a separate collaborator*, so a + deployment that should not offer it is one revert away rather than an + audit of a shared response shape. + + And reading it is logged, which no other read in this console is. That + asymmetry is deliberate: `console.track_served` records one person + playing another's voice back, and this records one person reading an + ordered list of their colleagues' attendance. Both are uses of other + people's data that leave no other trace, and "who looked, and when" is + the first question anybody reviewing the arrangement will ask. The + line names the guild and the reader and never who was in the list -- + the list is the point, and copying it into a retained log store would + be making a second copy of exactly the thing under discussion. + """ + from sturnus.console.app import current_user + + viewer = current_user(request).discord_user_id + try: + guild_id = int(request.match_info["guild_id"]) + except ValueError: + return _no_such_guild() + + found = await request.app[PARTICIPATION_REPORTS].attendance_in(guild_id, requested_by=viewer) + if found is None: + # No line here. Nothing was disclosed and nobody was authorised, + # and logging refusals by guild id would let anybody with a + # session fill the audit trail with guilds of their choosing. + return _no_such_guild() + + log_event( + log, + logging.INFO, + Event.CONSOLE_PARTICIPATION_READ, + "Somebody read a guild's meeting attendance ranking", + guild_id=guild_id, + requested_by=viewer, + # How many people were in the answer, which is the one thing about + # the list that is safe to retain and is enough to tell an empty + # guild from a whole team. + participants=len(found.people), + ) + return web.json_response( + participation(found.people, guild_id=guild_id, sessions=found.sessions), + # It is an ordered list of named colleagues. Nothing in between + # this and the browser has any business keeping a copy. + headers={"Cache-Control": "private, no-store"}, + ) + + def _no_such_guild() -> web.Response: """One refusal for every reason there is to refuse. diff --git a/src/sturnus/entrypoints/api.py b/src/sturnus/entrypoints/api.py index dcae2eb..4bc3c00 100644 --- a/src/sturnus/entrypoints/api.py +++ b/src/sturnus/entrypoints/api.py @@ -42,6 +42,7 @@ ConsoleConsentDirectory, ConsoleGuildReports, ConsoleLinkDirectory, + ConsoleParticipationReports, ConsoleQueueControl, ConsoleQueueOverview, ConsoleStateStore, @@ -218,6 +219,7 @@ def now() -> datetime: queues=ConsoleQueueOverview(session_factory, admins, now), consents=ConsoleConsentDirectory(session_factory, admins, config, now), reports=ConsoleGuildReports(session_factory, admins, config), + participation=ConsoleParticipationReports(session_factory, admins), ) runner = web.AppRunner(app) diff --git a/src/sturnus/observability/events.py b/src/sturnus/observability/events.py index a495fba..5795840 100644 --- a/src/sturnus/observability/events.py +++ b/src/sturnus/observability/events.py @@ -178,6 +178,15 @@ class Event(StrEnum): #: which of the two it was. CONSOLE_CONSENT_REVOKE_REFUSED = "console.consent_revoke_refused" + #: Somebody read a guild's attendance ranking -- the one thing the + #: console reports that names other people and orders them. Logged on + #: a *read*, which nothing else here is, because that is the point: a + #: ranking of colleagues by meeting attendance is subject to + #: co-determination (BetrVG §87(1)(6)), and "who looked at it, and + #: when" is the first question anybody reviewing the arrangement will + #: ask. `guild_id` and `requested_by`; never who was in the list. + CONSOLE_PARTICIPATION_READ = "console.participation_read" + # -- cross-cutting ------------------------------------------------------ PROCESS_STARTING = "process.starting" SHUTDOWN_BEGIN = "shutdown.begin" diff --git a/tests/console/conftest.py b/tests/console/conftest.py index 914811d..99c4e08 100644 --- a/tests/console/conftest.py +++ b/tests/console/conftest.py @@ -23,11 +23,13 @@ AdminDirectory, ConsentDirectory, ConsentHolder, + GuildParticipation, GuildQueue, GuildRecording, GuildReports, LinkDirectory, OAuthClient, + ParticipationReports, QueueControl, QueueOverview, QueueSnapshot, @@ -471,6 +473,24 @@ async def recording_of(self, guild_id: int, *, requested_by: int) -> GuildRecord return self.recording +class FakeParticipation: + """An attendance ranking nobody administers, until a test says otherwise. + + Its own double rather than a second method on `FakeReports`, mirroring + the production split: the aggregate report names nobody and this one is + the whole of what does, so a test that wires up one and not the other + is expressing something real. + """ + + def __init__(self, attendance: GuildParticipation | None = None) -> None: + self.attendance = attendance + self.asked: list[tuple[int, int]] = [] + + async def attendance_in(self, guild_id: int, *, requested_by: int) -> GuildParticipation | None: + self.asked.append((guild_id, requested_by)) + return self.attendance + + def build_test_api( *, oauth: OAuthClient | None = None, @@ -484,6 +504,7 @@ def build_test_api( queues: QueueOverview | None = None, consents: ConsentDirectory | None = None, reports: GuildReports | None = None, + participation: ParticipationReports | None = None, sessions: SessionCookie | None = None, now: Callable[[], datetime] | None = None, schema_ready: bool = True, @@ -521,6 +542,7 @@ def build_test_api( queues=queues or FakeQueueOverview(), consents=consents or FakeConsents(), reports=reports or FakeReports(), + participation=participation or FakeParticipation(), sessions=sessions or SessionCookie(SECRET, timedelta(hours=12)), now=now or now_at(), schema_ready=lambda: schema_ready, diff --git a/tests/console/test_participation.py b/tests/console/test_participation.py new file mode 100644 index 0000000..6109648 --- /dev/null +++ b/tests/console/test_participation.py @@ -0,0 +1,539 @@ +"""The attendance ranking: its order, its shape, and who may read it. + +Three files' worth of subject in one, because the feature is small and its +properties are not independent of each other. What is pinned: + +- the ordering is total and stable, so a list of named colleagues does not + appear to reshuffle itself between two page loads; +- speaking time is not the sort key, and never becomes one by accident; +- null is not zero, so somebody whose recordings predate the measurement + columns does not read as silent; +- reading it is logged, which is the one thing that makes this feature + reviewable after the fact. + +The last of those is the test most worth having. This is the only endpoint +in the console that names other people and orders them, and the log line +is the entire answer to "who looked at the attendance ranking, and when". +""" + +from __future__ import annotations + +import logging +from datetime import datetime, timedelta + +import pytest +from aiohttp import web +from aiohttp.test_utils import TestClient +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine + +from sturnus.console.adapters import ConsoleParticipationReports +from sturnus.console.app import SESSION_COOKIE +from sturnus.console.participation import Attendance, ParticipationJson, participation +from sturnus.console.ports import GuildParticipation +from sturnus.console.session import SessionCookie, SignedSession +from sturnus.infrastructure.db.models import ( + Base, + Session, + SessionParticipant, + TranscriptionJob, +) +from sturnus.observability.events import Event +from tests.console.conftest import ( + ANNA, + BEN, + GUILD, + SECRET, + T0, + AiohttpClientFactory, + FakeParticipation, + build_test_api, +) + +CARL = 300 +OTHER_GUILD = 9999 + + +def someone(**over: object) -> Attendance: + base: dict[str, object] = { + "discord_user_id": BEN, + "display_name": "ben", + "sessions": 3, + "speech_seconds": 600.0, + "unmeasured_tracks": 0, + "first_seen_at": T0 - timedelta(days=30), + "last_seen_at": T0, + } + base.update(over) + return Attendance(**base) # type: ignore[arg-type] + + +def ranked(*people: Attendance, sessions: int = 10) -> ParticipationJson: + """The shaped payload, typed as the contract rather than as a dict. + + Keeping `ParticipationJson` means these tests type-check against the + shape the API actually promises: a key renamed in the TypedDict and + not here is a red type check rather than a green test asserting on a + key that no longer exists. + """ + return participation(people, guild_id=GUILD, sessions=sessions) + + +# --------------------------------------------------------------------------- +# The order +# --------------------------------------------------------------------------- + + +def test_the_most_meetings_come_first() -> None: + order = ranked( + someone(discord_user_id=1, display_name="anna", sessions=2), + someone(discord_user_id=2, display_name="ben", sessions=9), + ) + + assert [person["display_name"] for person in order["people"]] == ["ben", "anna"] + + +def test_speaking_time_is_not_the_order_and_does_not_become_it() -> None: + """Ordering people by how much they talked is a different claim. + + "Was present most often" and "talked most" are not the same statement + about a colleague, and the second is the one that reads as a + judgement. It is reported and never sorted on. + """ + order = ranked( + someone(discord_user_id=1, display_name="anna", sessions=9, speech_seconds=10.0), + someone(discord_user_id=2, display_name="ben", sessions=2, speech_seconds=99_999.0), + ) + + assert [person["display_name"] for person in order["people"]] == ["anna", "ben"] + + +def test_a_tie_breaks_on_the_name_rather_than_on_the_row_order() -> None: + # Left to the order rows came back in, a list of named colleagues would + # appear to reshuffle itself between two page loads. + order = ranked( + someone(discord_user_id=3, display_name="carl", sessions=4), + someone(discord_user_id=1, display_name="anna", sessions=4), + someone(discord_user_id=2, display_name="ben", sessions=4), + ) + + assert [person["display_name"] for person in order["people"]] == ["anna", "ben", "carl"] + + +def test_somebody_the_system_has_no_name_for_sorts_after_everyone_named() -> None: + # A row of eighteen digits at the head of a tie is the least useful + # place for it in a list people read top-down. + order = ranked( + someone(discord_user_id=1, display_name=None, sessions=4), + someone(discord_user_id=2, display_name="ben", sessions=4), + ) + + assert [person["display_name"] for person in order["people"]] == ["ben", None] + + +def test_two_people_with_the_same_name_still_have_a_stable_order() -> None: + order = ranked( + someone(discord_user_id=22, display_name="ben", sessions=4), + someone(discord_user_id=11, display_name="ben", sessions=4), + ) + + assert [person["discord_user_id"] for person in order["people"]] == ["11", "22"] + + +# --------------------------------------------------------------------------- +# The shape +# --------------------------------------------------------------------------- + + +def test_every_discord_id_travels_as_a_string() -> None: + big = 308_000_000_000_000_001 + order = ranked(someone(discord_user_id=big)) + + assert order["guild_id"] == str(GUILD) + assert order["people"][0]["discord_user_id"] == str(big) + + +def test_the_number_of_sessions_the_ranking_is_over_travels_with_it() -> None: + """A rank means nothing without it. + + "In eleven meetings" is one claim out of twelve and quite another out + of four hundred, and a bare rank invites the first reading regardless. + """ + assert ranked(someone(sessions=11), sessions=400)["sessions"] == 400 + + +def test_a_track_nobody_measured_is_not_reported_as_silence() -> None: + # Null means nobody ever measured; zero means somebody did and it was + # nothing. A person whose recordings predate the measurement columns + # must not read as having said nothing. + order = ranked(someone(speech_seconds=None, unmeasured_tracks=4)) + + assert order["people"][0]["speech_seconds"] is None + assert order["people"][0]["unmeasured_tracks"] == 4 + + +def test_an_empty_guild_ranks_nobody() -> None: + assert ranked(sessions=0)["people"] == [] + + +# --------------------------------------------------------------------------- +# The endpoint, and its audit line +# --------------------------------------------------------------------------- + + +def token(discord_user_id: int = ANNA) -> str: + return SessionCookie(SECRET, timedelta(hours=12)).issue(SignedSession(discord_user_id), now=T0) + + +async def signed_in( + aiohttp_client: AiohttpClientFactory, app: web.Application, as_user: int = ANNA +) -> TestClient[web.Request, web.Application]: + client = await aiohttp_client(app) + client.session.cookie_jar.update_cookies({SESSION_COOKIE: token(as_user)}) + return client + + +def url(guild_id: int | str = GUILD) -> str: + return f"/api/guilds/{guild_id}/report/participation" + + +def _events(caplog: pytest.LogCaptureFixture, event: Event) -> list[logging.LogRecord]: + return [r for r in caplog.records if getattr(r, "sturnus_event", None) == str(event)] + + +async def test_an_administrator_sees_who_took_part_in_the_most_meetings( + aiohttp_client: AiohttpClientFactory, +) -> None: + people = FakeParticipation(GuildParticipation((someone(),), sessions=10)) + client = await signed_in(aiohttp_client, build_test_api(participation=people)) + + response = await client.get(url()) + + assert response.status == 200 + body = await response.json() + assert body["people"][0]["sessions"] == 3 + assert body["sessions"] == 10 + + +async def test_the_ranking_is_asked_for_on_behalf_of_the_signed_in_person( + aiohttp_client: AiohttpClientFactory, +) -> None: + people = FakeParticipation(GuildParticipation((), sessions=0)) + client = await signed_in(aiohttp_client, build_test_api(participation=people), as_user=BEN) + + await client.get(url()) + + assert people.asked == [(GUILD, BEN)] + + +async def test_a_guild_this_person_does_not_administer_does_not_exist( + aiohttp_client: AiohttpClientFactory, +) -> None: + client = await signed_in(aiohttp_client, build_test_api(participation=FakeParticipation())) + + response = await client.get(url()) + + assert response.status == 404 + + +async def test_reading_the_ranking_is_written_down( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + """The one read in this console that leaves a trace, on purpose. + + An ordered list of colleagues by meeting attendance is subject to + co-determination, and "who looked at it, and when" is the first + question anybody reviewing the arrangement will ask. Without this line + there is no answer to it at all. + """ + people = FakeParticipation(GuildParticipation((someone(), someone(discord_user_id=CARL)), 10)) + client = await signed_in(aiohttp_client, build_test_api(participation=people), as_user=ANNA) + + with caplog.at_level(logging.INFO): + await client.get(url()) + + lines = _events(caplog, Event.CONSOLE_PARTICIPATION_READ) + assert len(lines) == 1 + fields = lines[0].sturnus_fields # type: ignore[attr-defined] + assert fields["guild_id"] == GUILD + assert fields["requested_by"] == ANNA + assert fields["participants"] == 2 + + +async def test_the_audit_line_never_names_the_people_in_the_list( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + # The list is the thing under discussion; copying it into a retained, + # searchable log store would be making a second copy of it. + people = FakeParticipation(GuildParticipation((someone(),), 10)) + client = await signed_in(aiohttp_client, build_test_api(participation=people)) + + with caplog.at_level(logging.INFO): + await client.get(url()) + + fields = _events(caplog, Event.CONSOLE_PARTICIPATION_READ)[0].sturnus_fields # type: ignore[attr-defined] + assert "discord_user_id" not in fields + assert "display_name" not in fields + + +async def test_a_refusal_is_not_an_audit_line( + aiohttp_client: AiohttpClientFactory, + caplog: pytest.LogCaptureFixture, +) -> None: + # Nothing was disclosed and nobody was authorised. A line here would + # let anybody with a session fill the audit trail with guild ids of + # their choosing. + client = await signed_in(aiohttp_client, build_test_api(participation=FakeParticipation())) + + with caplog.at_level(logging.INFO): + await client.get(url()) + + assert not _events(caplog, Event.CONSOLE_PARTICIPATION_READ) + + +async def test_the_ranking_needs_a_session_like_every_other_endpoint( + aiohttp_client: AiohttpClientFactory, +) -> None: + people = FakeParticipation(GuildParticipation((someone(),), 10)) + client = await aiohttp_client(build_test_api(participation=people)) + + assert (await client.get(url())).status == 401 + + +async def test_nothing_in_between_may_cache_the_ranking( + aiohttp_client: AiohttpClientFactory, +) -> None: + people = FakeParticipation(GuildParticipation((someone(),), 10)) + client = await signed_in(aiohttp_client, build_test_api(participation=people)) + + response = await client.get(url()) + + assert response.headers["Cache-Control"] == "private, no-store" + + +# --------------------------------------------------------------------------- +# The reads, against the real database +# --------------------------------------------------------------------------- + + +@pytest.fixture +async def factory(clean_database: str) -> async_sessionmaker[AsyncSession]: + engine = create_async_engine(clean_database) + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + return async_sessionmaker(engine, expire_on_commit=False) + + +class Admins: + """The mirrored administrator membership, per guild.""" + + def __init__(self, by_guild: dict[int, set[int]] | None = None) -> None: + self.by_guild = by_guild if by_guild is not None else {GUILD: {ANNA}} + + async def is_admin_anywhere(self, discord_user_id: int) -> bool: + return any(discord_user_id in members for members in self.by_guild.values()) + + async def administered_guilds(self, discord_user_id: int) -> tuple[int, ...]: + return tuple( + sorted(g for g, members in self.by_guild.items() if discord_user_id in members) + ) + + async def is_admin(self, guild_id: int, discord_user_id: int) -> bool: + return discord_user_id in self.by_guild.get(guild_id, set()) + + +def reports( + factory: async_sessionmaker[AsyncSession], admins: Admins | None = None +) -> ConsoleParticipationReports: + return ConsoleParticipationReports(factory, admins or Admins()) + + +async def a_meeting( + factory: async_sessionmaker[AsyncSession], + *, + guild_id: int = GUILD, + started_at: datetime = T0, + people: dict[int, str] | None = None, + speech: dict[int, float | None] | None = None, +) -> int: + async with factory() as db: + session = Session( + guild_id=guild_id, + channel_id=555, + channel_name="meeting", + started_at=started_at, + ended_at=started_at + timedelta(hours=1), + status="documented", + ) + db.add(session) + await db.flush() + for discord_user_id, name in (people or {}).items(): + db.add( + SessionParticipant( + session_id=session.id, + discord_user_id=discord_user_id, + discord_display_name=name, + first_seen_at=started_at, + ) + ) + for discord_user_id, speech_seconds in (speech or {}).items(): + db.add( + TranscriptionJob( + session_id=session.id, + discord_user_id=discord_user_id, + s3_key=f"sessions/{session.id}/speakers/{discord_user_id}.enc", + encryption_key_id="k1", + wrapped_data_key=b"wrapped", + retention_until=started_at + timedelta(days=30), + status="done", + attempts=1, + speech_seconds=speech_seconds, + ) + ) + await db.commit() + return session.id + + +async def test_somebody_who_administers_nothing_gets_no_ranking( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, people={BEN: "ben"}) + + assert await reports(factory).attendance_in(GUILD, requested_by=BEN) is None + + +async def test_an_administrator_of_another_guild_is_nobody_here( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, people={BEN: "ben"}) + admins = Admins({OTHER_GUILD: {CARL}}) + + assert await reports(factory, admins).attendance_in(GUILD, requested_by=CARL) is None + + +async def test_attendance_is_counted_across_this_guild_s_meetings( + factory: async_sessionmaker[AsyncSession], +) -> None: + for day in range(3): + await a_meeting(factory, started_at=T0 - timedelta(days=day), people={BEN: "ben"}) + await a_meeting(factory, started_at=T0 - timedelta(days=9), people={CARL: "carl"}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert {person.discord_user_id: person.sessions for person in found.people} == { + BEN: 3, + CARL: 1, + } + assert found.sessions == 4 + + +async def test_one_person_cannot_be_counted_twice_for_one_meeting( + factory: async_sessionmaker[AsyncSession], +) -> None: + """`uq_participant_per_session` is what makes this true, not the query. + + Worth a test all the same: the number being defended is a ranking of + colleagues, and if that constraint were ever relaxed a plain count + would quietly turn it into a ranking by how often somebody's + connection dropped. The statement counts distinct sessions so the + query does not depend on the schema for a property this important. + """ + async with factory() as db: + db.add( + Session( + guild_id=GUILD, + channel_id=555, + started_at=T0, + ended_at=T0 + timedelta(hours=1), + status="documented", + ) + ) + await db.commit() + + with pytest.raises(IntegrityError): + async with factory() as db: + for _ in range(2): + db.add( + SessionParticipant( + session_id=1, + discord_user_id=BEN, + discord_display_name="ben", + first_seen_at=T0, + ) + ) + await db.commit() + + +async def test_meetings_in_another_guild_do_not_count_here( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, people={BEN: "ben"}) + await a_meeting(factory, guild_id=OTHER_GUILD, people={BEN: "ben"}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert found.people[0].sessions == 1 + assert found.sessions == 1 + + +async def test_speaking_time_sums_only_this_guild_s_recordings( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, people={BEN: "ben"}, speech={BEN: 120.0}) + await a_meeting(factory, guild_id=OTHER_GUILD, people={BEN: "ben"}, speech={BEN: 9_000.0}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert found.people[0].speech_seconds == 120.0 + + +async def test_a_person_whose_recordings_were_never_measured_has_no_speaking_total( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, people={BEN: "ben"}, speech={BEN: None}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert found.people[0].speech_seconds is None + assert found.people[0].unmeasured_tracks == 1 + + +async def test_a_person_gets_the_name_they_last_appeared_under( + factory: async_sessionmaker[AsyncSession], +) -> None: + await a_meeting(factory, started_at=T0 - timedelta(days=9), people={BEN: "old name"}) + await a_meeting(factory, started_at=T0 - timedelta(days=1), people={BEN: "ben"}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert found.people[0].display_name == "ben" + + +async def test_when_somebody_was_first_and_last_in_a_meeting_is_reported( + factory: async_sessionmaker[AsyncSession], +) -> None: + first = T0 - timedelta(days=90) + await a_meeting(factory, started_at=first, people={BEN: "ben"}) + await a_meeting(factory, started_at=T0, people={BEN: "ben"}) + + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert (found.people[0].first_seen_at, found.people[0].last_seen_at) == (first, T0) + + +async def test_a_guild_that_has_never_met_ranks_nobody( + factory: async_sessionmaker[AsyncSession], +) -> None: + found = await reports(factory).attendance_in(GUILD, requested_by=ANNA) + + assert found is not None + assert found.people == () + assert found.sessions == 0 From c93695d3a3f6ab07e66b1485a847befe2ba5716d Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:38:52 +0200 Subject: [PATCH 13/14] feat(console): show the attendance ranking, on request and on the record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The interface half of the participation endpoint, added to the Reporting page rather than given a page of its own -- it belongs next to the figures it is the exception to. **It loads only when the reader asks.** A separate request behind a control that says what pressing it will do. Somebody who opens Reporting to check whether transcription is keeping up must not silently generate an audit line saying they looked at a ranking of their colleagues, and the control's own note says as much. Three standing notes sit above that control rather than above the list, where they are read before the decision instead of after it. They say what the ranking is, that every read is written to the audit log -- which server, who looked, when, and never who was in the list -- that the recordings were made to write meetings up and counting attendance is a further purpose than that, and that such a facility is subject to co-determination under BetrVG §87(1)(6). And the one that matters most: being present in more meetings is not a measure of contribution, and speaking time even less so. A meeting held in a room, in a call elsewhere, in a channel Sturnus does not watch, or with somebody who has not consented, is invisible here -- so a low place is not evidence of anything and a high one is not either. The presentation carries the same restraint as the wording. Speaking time is a sentence under the attendance rather than a column, because a column of durations is ranked by the eye whether or not anybody sorted it. Rows are drawn identically: no bars, no medals, no emphasis on the first. Equal attendance shares a place, labelled as shared, because the tie-break is alphabetical and is about their names rather than about them. Every figure is "n of m" and never a percentage -- a percentage travels into a performance review without its denominator. `REPORT_SCOPE_NOTE` is amended here, not in the report change: it promised there was no per-person breakdown behind it, and that sentence stops being true at this commit. A scope note caught lying in one claim earns no belief in the others. --- console/app/pages/admin/reporting.vue | 292 ++++++++++- console/app/utils/participation.ts | 725 ++++++++++++++++++++++++++ console/app/utils/reporting.ts | 39 +- console/test/participation.spec.ts | 571 ++++++++++++++++++++ console/test/reporting.spec.ts | 36 +- 5 files changed, 1629 insertions(+), 34 deletions(-) create mode 100644 console/app/utils/participation.ts create mode 100644 console/test/participation.spec.ts diff --git a/console/app/pages/admin/reporting.vue b/console/app/pages/admin/reporting.vue index f65ff5b..ef4f74c 100644 --- a/console/app/pages/admin/reporting.vue +++ b/console/app/pages/admin/reporting.vue @@ -20,14 +20,17 @@ * * Three things this page refuses to do, all three on purpose: * - * - **It never names or ranks a person, and never hints that it might.** - * The payload carries no ids and no names, and that is a decision rather - * than an omission: a per-person readout of who attended which meetings - * and who spoke for how long is a means of monitoring conduct and - * performance at work, which is a works-council matter and not a console - * feature. `REPORT_SCOPE_NOTE` says so on the page, because the reader - * who was about to go looking for the breakdown deserves the answer - * rather than a dead end. + * - **None of the figures above the ranking is about a person.** The report + * payload carries no ids and no names, and that is a decision rather than + * an omission. The attendance ranking at the foot of the page is the one + * thing here that names people, and everything about how it is built + * keeps it apart from the rest: its own endpoint, its own module + * (`~/utils/participation`), its own standing notes, and a request that + * goes out only when somebody presses the control that says what it will + * do. Reading it is written to the audit log, which is exactly why it is + * not loaded alongside the figures — somebody checking whether + * transcription is keeping up has not asked to look at a ranking of their + * colleagues, and their name should not end up in a log saying they did. * - **It draws its own bars.** A chart library for one row of rectangles * would be a dependency for four glyphs, and the Content-Security-Policy * this console is served under would not load it anyway. The bars are @@ -38,6 +41,25 @@ * seconds would be a cost with no reader. The Refresh button is the * whole of the refresh policy. */ +import { + PARTICIPATION_EMPTY_HEADING, + PARTICIPATION_EMPTY_NOTE, + PARTICIPATION_HEADING, + PARTICIPATION_HIDE_LABEL, + PARTICIPATION_HIDE_NOTE, + PARTICIPATION_LOADING_NOTE, + PARTICIPATION_REVEAL_BUSY_LABEL, + PARTICIPATION_REVEAL_LABEL, + PARTICIPATION_REVEAL_NOTE, + type GuildParticipation, + describeParticipationError, + isParticipationEmpty, + parseGuildParticipation, + participationNotes, + participationPath, + participationRows, + participationScopeLine, +} from '~/utils/participation' import { REPORT_EMPTY_HEADING, REPORT_EMPTY_NOTE, @@ -145,6 +167,93 @@ async function refreshNow() { } } +/* -------------------------------------------------------------------- */ +/* The attendance ranking, which is not fetched with anything else */ +/* -------------------------------------------------------------------- */ + +/** + * Whether the reader has asked for the ranking. + * + * The whole reason this is a separate request rather than another field on + * the report: reading the ranking is written to the audit log, and somebody + * who opened this page to check whether transcription is keeping up has not + * asked to look at a ranking of the people they work with. Loading it + * alongside the figures would have put every one of them in that log, + * without ever having been asked. + * + * `immediate: false` and no `watch`, deliberately. A watched fetcher would + * re-issue the request -- and write another audit line -- every time + * somebody flipped the guild picker, which is the opposite of asking. + */ +const revealed = ref(false) + +/** Whether the request is out. Its own ref rather than `useAsyncData`'s + * status, for the same reason `refreshing` above is one: the status is + * about the data, and this is about the button the reader is looking at + * while they wait for an answer they deliberately asked for. */ +const asking = ref(false) + +const { + data: rankingData, + error: rankingError, + execute: loadRanking, + clear: clearRanking, +} = await useAsyncData( + 'guild-participation', + async () => { + const guildId = selected.value + if (!guildId) { + return { guildId: null as string | null, participation: null as GuildParticipation | null } + } + return { guildId, participation: parseGuildParticipation(await api(participationPath(guildId))) } + }, + { immediate: false, default: () => null }, +) + +// The list appears only once the answer is in hand. The control stays put +// and says it is reading, rather than being replaced by a spinner where it +// stood: the reader pressed a button that named what it would do, and the +// thing that reports back should be that button. +async function revealRanking() { + asking.value = true + try { + await loadRanking() + } + finally { + asking.value = false + revealed.value = true + } +} + +function hideRanking() { + revealed.value = false + clearRanking() +} + +// Switching servers puts the ranking away rather than carrying it across. +// A list of named people left standing under a different server's heading +// is the worst version of the mistake the guild picker exists to prevent -- +// and re-fetching it for the new server on the reader's behalf would be +// this page deciding to look at that server's people for them. +watch(selected, () => { + revealed.value = false + clearRanking() +}) + +/** Nothing is shown while the answer on hand belongs to another guild, for + * the same reason the report itself is guarded that way — with the + * difference that a stale row here carries somebody's name. */ +const ranking = computed(() => + rankingData.value && rankingData.value.guildId === selected.value + ? rankingData.value.participation + : null, +) + +const rankingNotes = participationNotes() +const rankingRows = computed(() => (ranking.value ? participationRows(ranking.value) : [])) +const rankingScope = computed(() => (ranking.value ? participationScopeLine(ranking.value) : '')) +const rankingEmpty = computed(() => Boolean(ranking.value && isParticipationEmpty(ranking.value))) + /** An absence is not a small number, and must not be coloured like one. A * muted em dash reads as "there is nothing here"; the same dash in the * figure colour reads as a value the reader failed to parse. */ @@ -467,6 +576,173 @@ const TONE_COLOUR: Record = { + + +
+

{{ PARTICIPATION_HEADING }}

+ + +
+
+
{{ note.label }}
+
+ {{ note.text }} +
+
+
+ + +
+

+ {{ asking ? PARTICIPATION_LOADING_NOTE : PARTICIPATION_REVEAL_NOTE }} +

+ +
+ + +
diff --git a/console/app/utils/participation.ts b/console/app/utils/participation.ts new file mode 100644 index 0000000..0b1686d --- /dev/null +++ b/console/app/utils/participation.ts @@ -0,0 +1,725 @@ +/** + * Who Sturnus has recorded in one server, named, and how many of its + * meetings each of them was in. + * + * This is the only thing in the console that names other people and puts + * them in an order, and everything in this file is shaped by that. It is + * not a report about a server the way `~/utils/reporting` is: it is a + * readout about individuals at work, assembled from recordings that were + * collected in order to write up meetings. Nothing here pretends otherwise. + * + * A module rather than expressions in the page, for the same reason + * `~/utils/reporting` is one: every function below is a *decision* -- what + * order the rows are in, what an unmeasured speaking time reads like, what + * somebody with no name is called, what the reader is told before they ask + * for any of it -- and a decision embedded in a template can only be tested + * by rendering one. The wording is the safeguard here, so the wording is + * what the tests assert on. + * + * Six facts govern everything below, and none of them are softened + * anywhere in this file: + * + * - **This ranks people, and it says so.** Not "engagement", not + * "activity", not a euphemism that lets somebody quote it without + * noticing what they are quoting. `PARTICIPATION_STANDING_NOTE` names it + * in the first sentence, and it stands above the list whether or not the + * list has been loaded. + * - **In a German workplace this is subject to co-determination** + * (BetrVG §87(1)(6): technical equipment suited to monitoring conduct or + * performance), and it serves a further purpose than the one the + * recordings were collected for. That is not a reason it cannot exist. It + * is the reason `PARTICIPATION_PURPOSE_NOTE` is on the page rather than + * in a design document nobody reading the ranking will ever open. + * - **Reading it is logged.** Which server, who looked, and when. Not who + * was in the list -- a log of who was ranked would be a second copy of + * the ranking, kept forever, in a place nobody would think to look for + * it. The reader is told this *before* the request goes out, because + * afterwards is too late to decline. + * - **Nothing is fetched until somebody asks.** The page loads its + * aggregate figures on arrival; this list does not come with them. + * Somebody opening Reporting to check whether transcription is keeping up + * must not silently generate an audit line saying they looked at a + * ranking of their colleagues. The reveal control is what turns looking + * into a deliberate act, and `PARTICIPATION_REVEAL_NOTE` says what + * pressing it does. + * - **Attendance is the figure; speaking time is a caveat underneath it.** + * Speaking time is a sentence in muted text, never a second column and + * never a second order. A "top talker" is a thing this console will not + * compute, and a column of durations lined up down a page is one whether + * or not anybody sorted it. + * - **Null is not zero, and an id is not a name.** `speech_seconds` is + * null when nobody ever measured, which is a measurement that was not + * taken rather than a person who said nothing; `unmeasured_tracks` is the + * size of the hole in a figure that is not null. `display_name` is null + * when this server has never had a name for them, and the row says + * plainly that what it shows instead is an id. + * + * The order is the server's and is never recomputed here. See + * `participationRows`. + */ +import { formatCount, formatDuration, formatMoment } from '~/utils/format' + +/* -------------------------------------------------------------------- */ +/* What the API describes */ +/* -------------------------------------------------------------------- */ + +/** + * One person, as the endpoint describes them. + * + * Every field is about a named individual, which is why this interface + * exists in a file of its own rather than as a member of `GuildReport`: a + * shape that can be passed around is a shape that gets passed around, and + * this one should never end up on a page that did not decide to carry it. + */ +export interface ParticipationPerson { + /** A string, always. A Discord snowflake exceeds JavaScript's safe + * integer range, where a JSON number silently drops its last digits and + * produces an id that looks right and names somebody else. In a list + * that ranks people, that is not a rounding error -- it is the wrong + * person's name against somebody else's figures. */ + discord_user_id: string + /** Null when this server has never had a name for them. Rendered as the + * id, said to be an id; see `participationIdentityNote`. */ + display_name: string | null + /** How many of this server's recorded meetings they were in. The figure + * the list is ordered by, and the only one it is ordered by. */ + sessions: number + /** Null means nobody ever measured, which is not zero. Zero means + * somebody measured and heard nothing. */ + speech_seconds: number | null + /** How many of this person's tracks carried no measurement. The size of + * the hole in `speech_seconds` when it is not null, and the reason it is + * null when it is. */ + unmeasured_tracks: number + first_seen_at: string | null + last_seen_at: string | null +} + +/** + * The ranking for one server, and the number of meetings it is out of. + * + * `sessions` is not decoration. A rank without it is a claim with no + * denominator: "in 11 meetings" is most of the year in a server that held + * twelve and a rounding error in one that held four hundred, and the two + * read identically on a row that omits the total. Every rendering below + * carries it. + */ +export interface GuildParticipation { + /** Null only when the payload named no server, which the page uses to + * refuse to show one server's people under another's heading. */ + guild_id: string | null + /** How many meetings the whole ranking is computed over. */ + sessions: number + /** In the order the API sent them, which is the order they are shown in. + * See `participationRows`. */ + people: ParticipationPerson[] +} + +/* -------------------------------------------------------------------- */ +/* Reading what the API sent */ +/* -------------------------------------------------------------------- */ + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** `null` stays `null`; anything else becomes the string it prints as. Ids + * are strings on the wire, and a number that arrived instead has already + * lost whatever precision it was going to lose. */ +function asText(value: unknown): string | null { + if (value === null || value === undefined) return null + const text = typeof value === 'string' ? value : String(value) + return text.trim() === '' ? null : text +} + +/** A count that can be printed. Anything absent, negative or not a number + * is a defect upstream, and rendering it as "-3 meetings" beside somebody's + * name would put that defect in front of the reader as though it were a + * fact about that person. Zero is the honest floor. */ +function asCount(value: unknown): number { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return 0 + return Math.round(value) +} + +/** A quantity that is allowed to be missing and is not a whole number -- + * seconds of speech. Nonsense collapses to null rather than to zero, + * because "we do not know" is true of a broken figure and "they said + * nothing" is not. */ +function asOptionalNumber(value: unknown): number | null { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) return null + return value +} + +/** + * The ranking in a payload. + * + * Always yields a well-formed value, never null, for the same reason + * `parseGuildReport` does: a page that has to distinguish "the API refused" + * from "the API answered something odd" already has the thrown `ApiError` + * for the first, and a parser returning null for the second would turn a + * strange payload into a blank section with no error anywhere. + * + * An entry naming no user is dropped rather than shown as an anonymous row. + * A row in a ranking of people that identifies nobody cannot be checked, + * cannot be corrected by the person it is about, and is the one kind of row + * this list must not contain. + */ +export function parseGuildParticipation(payload: unknown): GuildParticipation { + const raw = isRecord(payload) ? payload : {} + const people: ParticipationPerson[] = [] + + if (Array.isArray(raw.people)) { + for (const entry of raw.people) { + if (!isRecord(entry)) continue + const id = asText(entry.discord_user_id) ?? asText(entry.user_id) + if (!id) continue + people.push({ + discord_user_id: id, + display_name: asText(entry.display_name), + sessions: asCount(entry.sessions), + speech_seconds: asOptionalNumber(entry.speech_seconds), + unmeasured_tracks: asCount(entry.unmeasured_tracks), + first_seen_at: asText(entry.first_seen_at), + last_seen_at: asText(entry.last_seen_at), + }) + } + } + + return { + guild_id: asText(raw.guild_id), + sessions: asCount(raw.sessions), + people, + } +} + +/** Where a server's ranking is read from. The id is escaped: it is a string + * from an API, and a string allowed to contain a slash is a string allowed + * to address a different endpoint. */ +export function participationPath(guildId: string): string { + return `/guilds/${encodeURIComponent(guildId)}/report/participation` +} + +/* -------------------------------------------------------------------- */ +/* Saying it in words */ +/* -------------------------------------------------------------------- */ + +/** One or the other, chosen by the count. Written out at each call rather + * than derived by adding an `s`, because half the pairs this file needs + * are `was`/`were` and `has`/`have`. */ +function plural(count: number, one: string, many: string): string { + return count === 1 ? one : many +} + +function meetings(count: number): string { + return `${formatCount(count)} ${plural(count, 'meeting', 'meetings')}` +} + +/** + * What to call somebody on screen. + * + * The whole id when there is no name -- never a shortened one, since + * snowflakes minted in the same era share their leading digits and a + * truncated id identifies a group rather than a person. The same rule + * `personLabel` follows on the consent list, for the same reason: this is a + * list somebody may act on, and a label that fits several people is worse + * than a long one. + */ +export function participationPersonLabel(person: ParticipationPerson): string { + return person.display_name ?? `Discord user ${person.discord_user_id}` +} + +/** + * The line under a nameless row, or `null` when there is a name. + * + * A bare snowflake in a list of names reads as a fault in the console, and + * it is not one: Sturnus learns a display name from Discord and does not + * always have one for a server it recorded somebody in. Saying outright + * that the string is an id is also the honest thing to do to the reader who + * is about to write this row into something -- an id is not a person's + * name, and a ranking that quietly presents one as the other invites + * somebody to be identified by guesswork. + */ +export function participationIdentityNote(person: ParticipationPerson): string | null { + if (person.display_name) return null + return ( + 'This is a Discord user id, not a name. Sturnus has no display name on record for them in ' + + 'this server, so there is nothing else to call them here — and an id is a poor thing to put ' + + 'in a list about people. Work out who it is from Discord rather than from the digits.' + ) +} + +/** + * How much of this server's history one person was present for. + * + * Always "n of m", never a bare count and never a percentage. The bare + * count invites the reader to supply their own denominator, and a + * percentage is a figure that survives being quoted without one -- "she was + * in 26 % of meetings" travels into a performance review far more easily + * than "she was recorded in 11 of the 42 meetings this server held", which + * is the same fact carrying its own scope. + */ +export function participationAttendanceLine( + person: ParticipationPerson, + total: number, +): string { + if (total <= 0) { + return ( + `Recorded in ${meetings(person.sessions)}, out of a total this ranking does not know: ` + + 'Sturnus reported no meeting count for this server, so there is nothing to read the figure ' + + 'against.' + ) + } + return ( + `Recorded in ${meetings(person.sessions)} of the ${formatCount(total)} this server has ` + + 'recorded.' + ) +} + +/** + * What this person's speaking time is, and what it is not. + * + * Written as a sentence rather than offered as a number, and that is the + * decision, not the phrasing. A duration in a column beside a name is + * ranked by the eye whether or not anybody sorted it, and speaking time is + * the figure on this page most likely to be mistaken for a measure of + * somebody's worth. In prose it has to be read, and reading it means + * reading the caveat attached to it. + * + * Null is never "0 s". The column behind it is nullable: null means nobody + * ever measured that track -- jobs that predate the measurement columns -- + * and a zero in its place would say that this named person sat through + * eleven meetings without speaking, which is an accusation rather than a + * gap in the data. + */ +export function participationSpeechLine(person: ParticipationPerson): string { + const unmeasured = person.unmeasured_tracks + + if (person.speech_seconds === null) { + if (unmeasured > 0) { + return ( + `No speaking time was ever measured for them: all ${formatCount(unmeasured)} of their ` + + `${plural(unmeasured, 'recording', 'recordings')} here ${plural(unmeasured, 'predates', 'predate')} ` + + 'the columns that hold it. That is a measurement nobody took, not a person who said ' + + 'nothing.' + ) + } + return ( + 'Sturnus holds no measured speaking time for them, and does not say why. Read that as a ' + + 'missing measurement rather than as silence.' + ) + } + + const spoken = formatDuration(person.speech_seconds) + if (unmeasured > 0) { + return ( + `Their microphone carried ${spoken} of speech across the recordings that were measured. ` + + `${formatCount(unmeasured)} of their ${plural(unmeasured, 'recording', 'recordings')} here ` + + `${plural(unmeasured, 'was', 'were')} never measured at all, and a sum skips those in ` + + 'silence — so this figure covers part of what was recorded and falls short by an unknown ' + + 'amount.' + ) + } + return ( + `Their microphone carried ${spoken} of speech across those meetings. That is how long a ` + + 'microphone was open on speech, and it is not a measure of anything else.' + ) +} + +/** + * When this person was first and last recorded here. + * + * On the row because a rank read without it is read as a fact about the + * present. Somebody who joined the team in June is behind a colleague who + * has been in every meeting since November for a reason that has nothing to + * do with either of them, and this is the only line on the row that can say + * so. + */ +export function participationSeenLine(person: ParticipationPerson): string { + const first = person.first_seen_at + const last = person.last_seen_at + + if (!first && !last) { + return 'Sturnus did not say when they were first or last recorded here.' + } + if (first && last) { + if (first === last) { + return `Recorded here once, on ${formatMoment(first)}.` + } + return `First recorded ${formatMoment(first)}, most recently ${formatMoment(last)}.` + } + const known = first ?? last + return `Only one end of their span here is known: ${formatMoment(known)}.` +} + +/* -------------------------------------------------------------------- */ +/* The rows */ +/* -------------------------------------------------------------------- */ + +export interface ParticipationRow { + /** The person's id, unique across the rows, so it keys a `v-for` + * safely. */ + key: string + discord_user_id: string + /** Their display name, or their id said to be one; see + * `participationPersonLabel`. */ + name: string + /** The sentence explaining a row whose name is an id, or null. */ + identity: string | null + /** Their position in the list as the server ordered it, counting from + * one. Equal attendance shares a number; see below. */ + rank: number + /** True when at least one other row carries the same rank. */ + tied: boolean + sessions: number + /** "Recorded in 11 of the 42 this server has recorded." */ + attendance: string + /** Speaking time, as a sentence and never as a competing figure. */ + speech: string + /** True when there is no measured speaking time for them at all, so the + * page can render the sentence as the absence it is rather than as a + * number the reader failed to parse. */ + speechAbsent: boolean + seen: string + /** The whole row said in one go, for the reader who is listening to the + * page rather than looking at it. */ + detail: string +} + +/** + * The people, in the order the API sent them. + * + * **Nothing here sorts.** The server orders the list -- most meetings + * first, ties broken by name and then by id -- and it is left exactly as it + * arrived. Two reasons, and the second is the important one. A second + * ordering in the browser would be a second definition of the order, and + * the two would drift the first time either changed. And re-sorting is one + * line away from offering a sort control, which would turn a list somebody + * has to justify opening into a tool for finding whoever is at the bottom + * of whichever column you like. This console will not compute a top talker; + * it should not hand out the means to derive one either. + * + * Ranks are shared between equal attendance rather than being the row's + * index. Two people who were each in eleven meetings are not first and + * second -- printing them that way would invent a distinction out of the + * tie-break, which is alphabetical and about their names rather than about + * them. The rank after a shared one skips, the way places do. + */ +export function participationRows(participation: GuildParticipation): ParticipationRow[] { + const people = participation.people + const total = participation.sessions + const rows: ParticipationRow[] = [] + + // The place most recently handed out. A row whose attendance matches the + // row above it keeps this value instead of taking its own index, which is + // what makes 1, 1, 3 out of three rows rather than 1, 2, 3. + let place = 0 + + for (let index = 0; index < people.length; index += 1) { + const person = people[index]! + const previous = index > 0 ? people[index - 1] : undefined + const next = index + 1 < people.length ? people[index + 1] : undefined + + if (!previous || previous.sessions !== person.sessions) place = index + 1 + + const name = participationPersonLabel(person) + const attendance = participationAttendanceLine(person, total) + const speech = participationSpeechLine(person) + const seen = participationSeenLine(person) + + rows.push({ + key: person.discord_user_id, + discord_user_id: person.discord_user_id, + name, + identity: participationIdentityNote(person), + rank: place, + tied: + (previous !== undefined && previous.sessions === person.sessions) + || (next !== undefined && next.sessions === person.sessions), + sessions: person.sessions, + attendance, + speech, + speechAbsent: person.speech_seconds === null, + seen, + detail: `${name}. ${attendance} ${speech} ${seen}`, + }) + } + + return rows +} + +/* -------------------------------------------------------------------- */ +/* What the reader is told, and when */ +/* -------------------------------------------------------------------- */ + +export const PARTICIPATION_HEADING = 'Attendance ranking' + +/** + * What this is, said before anybody can ask for it. + * + * The first sentence names it as a ranking of people, on purpose and + * without a softer word for it. Somebody who quotes this list elsewhere + * should have had to read that sentence first; a heading like "engagement" + * would have let them quote it without ever noticing what they were + * quoting. + * + * The audit line is stated here rather than in a tooltip or a confirmation + * dialog, and it is stated *before* the request goes out, because + * afterwards is too late to decline. What the log holds is spelled out too: + * the reader is entitled to know that their own colleagues' names are not + * being copied into it every time somebody looks. + */ +export const PARTICIPATION_STANDING_NOTE = + 'This is a ranking of named people by how many of this server’s meetings each of them was ' + + 'recorded in, and by how long their microphone carried speech. It is a different kind of ' + + 'report from the figures above it: those describe a server, this describes the individuals in ' + + 'it. Every time somebody opens it, that is written to the audit log — which server, who ' + + 'looked, and when. The list itself is not: the log records that a ranking was read, never who ' + + 'was in it.' + +/** + * What the numbers do not mean. + * + * Second, and never folded into the note above. The first note tells the + * reader what they are looking at; this one tells them what they must not + * conclude from it, which is the sentence that has to survive being read by + * somebody in a hurry who has already decided what they think. + * + * The list of what is invisible to Sturnus is specific for the same reason. + * "Attendance" sounds like a complete record and is not one: the only + * meetings in here are the ones held in a voice channel Sturnus watches, + * with people who consented to being recorded in it. A person can do a + * year's work and appear near the bottom. + */ +export const PARTICIPATION_CONTRIBUTION_NOTE = + 'Being present in more meetings is not a measure of contribution, and speaking time even less ' + + 'so. These numbers describe attendance in the voice channels Sturnus records, and nothing ' + + 'else. A meeting held in a room, in a call elsewhere, in a channel Sturnus does not watch, or ' + + 'with somebody who has not consented to being recorded, is invisible here — so a low place on ' + + 'this list is not evidence of anything, and a high one is not either.' + +/** + * Why this is fenced off the way it is. + * + * The recordings behind these figures were collected in order to write + * meetings up. Counting how often each named person turned up, and how long + * each of them talked, is a further purpose, and in a German workplace a + * facility of this kind is subject to co-determination -- BetrVG §87(1)(6) + * covers technical equipment suited to monitoring the conduct or + * performance of employees, and this is squarely that. + * + * Said on the page rather than kept in a design document, because the + * person who most needs to know it is the administrator about to paste this + * list into a message, and they are not going to read the design document. + */ +export const PARTICIPATION_PURPOSE_NOTE = + 'These recordings were made in order to write meetings up. Counting how often each named person ' + + 'attended, and how long each of them spoke, is a further purpose than that one. Where Sturnus ' + + 'runs in a workplace, a facility that can be used to observe how individual employees behave ' + + 'or perform is subject to co-determination — in Germany, BetrVG §87(1)(6) — so this list is ' + + 'something to agree on with a works council before it is used, not something to quietly start ' + + 'quoting.' + +export interface ParticipationNote { + key: string + label: string + text: string +} + +/** + * The three things that stand above this list at all times. + * + * Above the reveal control as well as above the loaded rows, deliberately. + * A note that appears only once the ranking is on screen is a note that + * arrives after the decision it exists to inform; the reader is supposed to + * be able to decide *not* to press the button, and they cannot do that on + * information they have not been given yet. + */ +export function participationNotes(): ParticipationNote[] { + return [ + { key: 'what', label: 'What this is', text: PARTICIPATION_STANDING_NOTE }, + { key: 'meaning', label: 'What it does not measure', text: PARTICIPATION_CONTRIBUTION_NOTE }, + { key: 'purpose', label: 'What it is for, and what it is not', text: PARTICIPATION_PURPOSE_NOTE }, + ] +} + +/* -------------------------------------------------------------------- */ +/* Asking for it */ +/* -------------------------------------------------------------------- */ + +/** + * The control that loads the list, and it says what it will do. + * + * Not "Show more", not "Details", not an arrow on a collapsible section. + * The label names the thing on the other side of the click, because the + * click is the moment somebody becomes a person who looked at a ranking of + * their colleagues, and a control that hides that behind a generic word has + * arranged for them to do it by accident. + */ +export const PARTICIPATION_REVEAL_LABEL = 'Show the attendance ranking' + +/** While the request is out. Present tense and no cancel: the audit line is + * written by the API when it answers, and a button that looked like it + * could take that back would be lying. */ +export const PARTICIPATION_REVEAL_BUSY_LABEL = 'Reading the ranking…' + +/** Putting it away again. It does not unsay anything -- see + * `PARTICIPATION_HIDE_NOTE` -- but a list of named colleagues left on + * screen behind somebody who has finished with it is worth one click to + * clear. */ +export const PARTICIPATION_HIDE_LABEL = 'Hide the ranking' + +export const PARTICIPATION_HIDE_NOTE = + 'Hidden again, and the audit line stays: it records that the ranking was read, and hiding it ' + + 'afterwards does not change that it was.' + +/** + * What pressing the button does, said before it is pressed. + * + * The last sentence is the reason this section is loaded on demand at all, + * and it is on the page rather than only in the code comment that + * implements it. Somebody who opened Reporting to see whether transcription + * is keeping up has not asked to see a ranking of the people they work + * with, and their name should not appear in an audit log saying they did. + * Fetching this alongside the aggregate figures would have put it there for + * everybody who ever loaded the page. + */ +export const PARTICIPATION_REVEAL_NOTE = + 'Nothing has been loaded. Pressing this asks Sturnus for the list — the people it has recorded ' + + 'in this server, by name, ordered by how many meetings each of them was in — and writes a line ' + + 'in the audit log saying that you asked. The figures above were loaded without any of that, so ' + + 'opening this page to see whether transcription is keeping up does not record you as having ' + + 'looked at a ranking of your colleagues.' + +/** While the request is in flight, in the page's own voice. Named as a + * ranking here too: the word does not get to disappear once somebody has + * agreed to load it. */ +export const PARTICIPATION_LOADING_NOTE = 'Reading this server’s attendance ranking…' + +/* -------------------------------------------------------------------- */ +/* A server with nobody in it */ +/* -------------------------------------------------------------------- */ + +/** Whether there is anybody to rank at all. A list of nobody rendered as an + * empty table reads as a page that failed to load, which is the failure + * mode hardest to report -- and the one most likely to be retried, which + * in this section costs another audit line. */ +export function isParticipationEmpty(participation: GuildParticipation): boolean { + return participation.people.length === 0 +} + +export const PARTICIPATION_EMPTY_HEADING = 'Sturnus has recorded nobody in this server' + +/** + * The empty state, as a sentence and a second saying what would fill it. + * + * It also says the thing the reader is most likely to get wrong about an + * empty list here: nobody is missing from it. A person Sturnus has never + * recorded has no row rather than a row of zeros, because a zero would be a + * measurement -- it would say this named person attended nothing -- and + * nothing has been measured about them at all. + */ +export const PARTICIPATION_EMPTY_NOTE = + 'There is nobody to list: no meeting in a channel Sturnus watches has recorded anybody here who ' + + 'consented to it. Nobody is missing from this list — somebody Sturnus has never recorded has ' + + 'no row rather than a row of zeros, because a zero would say they attended nothing, and ' + + 'nothing about them has been measured at all.' + +/** + * What the list is ordered by, and out of how much. + * + * The total belongs above the rows as well as inside each of them. A reader + * scanning names down a column has stopped reading the sentences by the + * third row, and the denominator is the one thing that must not be lost on + * the way down -- eleven meetings is most of a year here or a fortnight of + * it, and only this line says which. + */ +export function participationScopeLine(participation: GuildParticipation): string { + const people = participation.people.length + const total = participation.sessions + + if (total <= 0) { + return ( + `${formatCount(people)} ${plural(people, 'person', 'people')}, ordered by how many meetings ` + + 'each was recorded in, most first. Sturnus reported no meeting count for this server, so ' + + 'there is no total to read these figures against — a place in this order means very little ' + + 'without one.' + ) + } + return ( + `${formatCount(people)} ${plural(people, 'person', 'people')}, ordered by how many of this ` + + `server’s ${meetings(total)} each was recorded in, most first. Equal attendance shares a ` + + 'place; the order within a tie is alphabetical and means nothing. Every figure below is out ' + + `of those ${formatCount(total)}.` + ) +} + +/* -------------------------------------------------------------------- */ +/* When the API says no */ +/* -------------------------------------------------------------------- */ + +/** `ApiError` names it `status`; a raw `$fetch` failure may name it + * `statusCode`; a request that never got a response has neither, and null + * says so rather than standing in a number that would read as an + * answer. */ +function statusOf(error: unknown): number | null { + if (!isRecord(error)) return null + for (const candidate of [error.status, error.statusCode]) { + if (typeof candidate === 'number' && Number.isFinite(candidate)) { + // `ApiError` uses 0 for "never reached the API", which is + // deliberately distinguishable from every real status. + return candidate === 0 ? null : candidate + } + } + return null +} + +/** + * A failed request, in a sentence somebody can act on. + * + * Built from the status alone. `useApi` throws `ApiError`, which carries no + * body by design -- the API's own `{"error": "no such guild"}` never + * reaches this console -- so every sentence below has to stand on its own + * without it. + * + * Named `describeParticipationError` rather than `describeError` for the + * same reason `describeReportError` is: everything under `app/utils` is + * auto-imported into every component, and two exports sharing a name is a + * build warning and a coin toss over which one a page actually gets. + * + * Every message here says the ranking was not shown, rather than that + * "nothing could be loaded". A reader who is unsure whether they saw a + * partial list is a reader who will press the button again, and pressing it + * again is another audit line. + */ +export function describeParticipationError(error: unknown): string { + const status = statusOf(error) + switch (status) { + case 401: + return 'Your session has ended, so the ranking was not loaded. Sign in again to ask for it.' + case 403: + return ( + 'You do not administer this server, so its ranking was not loaded. Administrators are the ' + + 'members holding the role named by that guild’s `admin_role_id`.' + ) + case 404: + // The API answers 404 both for a server that does not exist and for + // one the caller does not administer, on purpose: it will not + // confirm the existence of a server to somebody with no business + // there. So this sentence has to cover both without guessing which. + return ( + 'Sturnus does not know this server, or you no longer administer it — it answers the same ' + + 'way to both, and no ranking was loaded. Reload the page; the list of servers is rebuilt ' + + 'from Discord.' + ) + case null: + return ( + 'Could not reach the API, so no ranking was loaded. Check the connection and ask again if ' + + 'you still want it.' + ) + default: + return ( + `Sturnus answered ${status} and produced no ranking. Nothing is known about why, and ` + + 'nothing of the list was shown.' + ) + } +} diff --git a/console/app/utils/reporting.ts b/console/app/utils/reporting.ts index 1782164..fc9f46b 100644 --- a/console/app/utils/reporting.ts +++ b/console/app/utils/reporting.ts @@ -15,10 +15,15 @@ * * - **This report is about a server, never about the people in it.** The * payload carries no ids and no names, and nothing here invents a - * per-person figure or hints that one is coming. How long one named - * person sat in meetings, or spoke in them, is a measure of that - * person's conduct at work -- a matter for a works council, not a - * console feature. Counts of people are counts and stop there. + * per-person figure. How long one named person sat in meetings, or spoke + * in them, is a measure of that person's conduct at work -- a works- + * council matter -- and none of it is in this module. The Reporting page + * does carry an attendance ranking, and it is deliberately a separate + * module (`~/utils/participation`) with its own framing, its own + * endpoint, and a reveal the reader has to press: mixing it in here would + * have put every one of these figures behind the same audit line. + * `REPORT_SCOPE_NOTE` names that boundary on the page. Counts of people + * are counts and stop there. * - **Null is not zero.** `average_duration_seconds`, `longest_duration_ * seconds`, `average_participants` and `largest_meeting` are null for a * server whose meetings have not finished. Printing `0` would state that @@ -924,20 +929,24 @@ export const REPORT_EMPTY_NOTE = /** * What this report is about, and what it is deliberately not about. * - * On the page rather than only in this file. The payload holds no names - * and no ids, and that is a decision rather than an oversight: a - * per-person readout of who attended which meetings and who spoke for how - * long is a means of monitoring conduct and performance at work, which is - * a matter for a works council and not a console feature. Saying so is - * also the honest answer to the reader who was about to ask where the - * breakdown is. + * On the page rather than only in this file. Nothing this module produces + * carries a name or an id, and that is a decision rather than an oversight: + * how long one named person sat in meetings, or spoke in them, is a measure + * of that person rather than of this server. + * + * The last sentence exists because the page below now has an attendance + * ranking on it, and a scope note that claimed no per-person readout exists + * anywhere would be false the moment somebody scrolled. It is named here, + * and named as the separate thing it is -- loaded only when asked for, and + * logged when it is -- rather than left for the reader to discover under a + * note telling them it could not be there. */ export const REPORT_SCOPE_NOTE = 'Every figure here is about the server as a whole and never about the people in it. Sturnus ' - + 'sends this page no names and no ids, and there is no per-person breakdown behind it — how ' - + 'long one named person sat in meetings, or spoke in them, is a measure of that person rather ' - + 'than of this server, and that is not something a console should hand out. Counts of people ' - + 'are counts, and stop there.' + + 'sends this page no names and no ids for any of them, and none of these figures can be traced ' + + 'back to one person. Counts of people are counts, and stop there. The attendance ranking at ' + + 'the foot of this page is the one exception and is kept apart on purpose: it names people, it ' + + 'is fetched only when somebody asks for it, and asking for it is written to the audit log.' /* -------------------------------------------------------------------- */ /* When the API says no */ diff --git a/console/test/participation.spec.ts b/console/test/participation.spec.ts new file mode 100644 index 0000000..c0c4d6a --- /dev/null +++ b/console/test/participation.spec.ts @@ -0,0 +1,571 @@ +/** + * What the attendance ranking says about the people in it, and what it + * refuses to say. + * + * All of it lives in `~/utils/participation` rather than in the page, + * because every one of these is a decision -- what order the rows are in, + * what an unmeasured speaking time reads like, what somebody with no name + * is called, what the reader is told before the request goes out -- and a + * decision embedded in a template can only be tested by rendering one. + * + * The wording is asserted here far more heavily than in the other specs, + * and deliberately. This is the only thing in the console that names other + * people and puts them in an order; the sentences around it are the whole + * of what stops it being read as a scoreboard, and a test that checked only + * the numbers would let every one of them be deleted without a failure. + * + * Five things are asserted that are not facts about the data at all: + * + * - the standing note calls it a ranking of people, in those words; + * - it says that opening it is written to the audit log, and that the log + * does not hold who was in the list; + * - it says outright that attendance is not a measure of contribution; + * - the reveal control names what it will load, rather than saying "show + * more"; + * - and no sentence this module can produce anywhere contains the + * vocabulary of a leaderboard. + */ +import { describe, expect, it } from 'vitest' + +import { + PARTICIPATION_CONTRIBUTION_NOTE, + PARTICIPATION_EMPTY_HEADING, + PARTICIPATION_EMPTY_NOTE, + PARTICIPATION_HIDE_LABEL, + PARTICIPATION_HIDE_NOTE, + PARTICIPATION_LOADING_NOTE, + PARTICIPATION_PURPOSE_NOTE, + PARTICIPATION_REVEAL_BUSY_LABEL, + PARTICIPATION_REVEAL_LABEL, + PARTICIPATION_REVEAL_NOTE, + PARTICIPATION_STANDING_NOTE, + describeParticipationError, + isParticipationEmpty, + parseGuildParticipation, + participationAttendanceLine, + participationIdentityNote, + participationNotes, + participationPath, + participationPersonLabel, + participationRows, + participationScopeLine, + participationSeenLine, + participationSpeechLine, + type GuildParticipation, + type ParticipationPerson, +} from '../app/utils/participation' + +/** Somebody the system knows a name for, who was in a good share of the + * meetings and whose recordings were all measured -- so each test states + * only the one property it is actually about. */ +function person(overrides: Partial = {}): ParticipationPerson { + return { + discord_user_id: '200', + display_name: 'ben', + sessions: 11, + speech_seconds: 4200, + unmeasured_tracks: 0, + first_seen_at: '2025-11-04T09:00:00+00:00', + last_seen_at: '2026-08-21T12:00:00+00:00', + ...overrides, + } +} + +function ranking(overrides: Partial = {}): GuildParticipation { + return { + guild_id: '4711', + sessions: 42, + people: [person()], + ...overrides, + } +} + +/** What `ApiError` looks like to the function that reads a failure. */ +function failure(status: number) { + return { status, path: '/guilds/4711/report/participation' } +} + +/** Every sentence this module can produce for one ranking, so a test can + * assert on what none of them says. The standing notes are in here too: + * they are as much a part of what the page claims as any row is. */ +function everySentence(value: GuildParticipation): string { + return [ + ...participationRows(value).map( + (row) => `${row.name} ${row.attendance} ${row.speech} ${row.seen} ${row.identity ?? ''}`, + ), + ...participationNotes().map((note) => `${note.label} ${note.text}`), + participationScopeLine(value), + PARTICIPATION_REVEAL_LABEL, + PARTICIPATION_REVEAL_BUSY_LABEL, + PARTICIPATION_REVEAL_NOTE, + PARTICIPATION_HIDE_LABEL, + PARTICIPATION_HIDE_NOTE, + PARTICIPATION_LOADING_NOTE, + PARTICIPATION_EMPTY_HEADING, + PARTICIPATION_EMPTY_NOTE, + ].join(' ') +} + +describe('reading the ranking payload', () => { + it('reads the whole envelope the endpoint sends', () => { + const parsed = parseGuildParticipation({ + guild_id: '4711', + sessions: 42, + people: [ + { + discord_user_id: '200', + display_name: 'ben', + sessions: 11, + speech_seconds: 4200.0, + unmeasured_tracks: 0, + first_seen_at: '2025-11-04T09:00:00+00:00', + last_seen_at: '2026-08-21T12:00:00+00:00', + }, + ], + }) + + expect(parsed.guild_id).toBe('4711') + expect(parsed.sessions).toBe(42) + expect(parsed.people).toEqual([ + { + discord_user_id: '200', + display_name: 'ben', + sessions: 11, + speech_seconds: 4200, + unmeasured_tracks: 0, + first_seen_at: '2025-11-04T09:00:00+00:00', + last_seen_at: '2026-08-21T12:00:00+00:00', + }, + ]) + }) + + it('keeps an id as a string even when it arrived as a number', () => { + // A snowflake past 2^53 loses its last digits as a JSON number, and in + // a list that ranks people that is not a rounding error -- it is one + // person's figures under another person's id. + const parsed = parseGuildParticipation({ people: [{ discord_user_id: 200 }] }) + expect(parsed.people[0]!.discord_user_id).toBe('200') + }) + + it('drops an entry that names nobody', () => { + // A row in a ranking of people that identifies nobody cannot be + // checked and cannot be corrected by the person it is about. + const parsed = parseGuildParticipation({ + people: [{ display_name: 'ben', sessions: 11 }, { discord_user_id: '200' }], + }) + expect(parsed.people).toHaveLength(1) + expect(parsed.people[0]!.discord_user_id).toBe('200') + }) + + it('keeps an absent speaking time absent rather than turning it into zero', () => { + // The whole point of the column being nullable. Zero here would say + // this named person sat through eleven meetings without speaking. + const parsed = parseGuildParticipation({ + people: [{ discord_user_id: '200', speech_seconds: null, unmeasured_tracks: 3 }], + }) + expect(parsed.people[0]!.speech_seconds).toBeNull() + expect(parsed.people[0]!.unmeasured_tracks).toBe(3) + }) + + it('refuses a negative count and a nonsensical duration', () => { + const parsed = parseGuildParticipation({ + sessions: -4, + people: [{ discord_user_id: '200', sessions: -1, speech_seconds: Number.NaN, unmeasured_tracks: -2 }], + }) + expect(parsed.sessions).toBe(0) + expect(parsed.people[0]!.sessions).toBe(0) + // Nonsense collapses to null rather than to zero: "we do not know" is + // true of a broken figure and "they said nothing" is not. + expect(parsed.people[0]!.speech_seconds).toBeNull() + expect(parsed.people[0]!.unmeasured_tracks).toBe(0) + }) + + it('yields a well-formed ranking for a payload that is not one', () => { + const parsed = parseGuildParticipation('nonsense') + expect(parsed).toEqual({ guild_id: null, sessions: 0, people: [] }) + }) + + it('escapes the guild id in the path', () => { + expect(participationPath('4711')).toBe('/guilds/4711/report/participation') + expect(participationPath('a/b')).toBe('/guilds/a%2Fb/report/participation') + }) +}) + +describe('the order of the rows', () => { + it('leaves the order exactly as the API sent it', () => { + // The server orders this list -- most meetings first, ties broken by + // name and then by id -- and a second ordering in the browser would be + // a second definition of the order. A payload that arrives in an order + // this module would not have chosen is still shown in that order. + const rows = participationRows( + ranking({ + people: [ + person({ discord_user_id: '1', display_name: 'ana', sessions: 3 }), + person({ discord_user_id: '2', display_name: 'ben', sessions: 30 }), + person({ discord_user_id: '3', display_name: 'cai', sessions: 12 }), + ], + }), + ) + expect(rows.map((row) => row.name)).toEqual(['ana', 'ben', 'cai']) + }) + + it('shares a place between people with the same attendance', () => { + // Two people who were each in eleven meetings are not first and + // second. Printing them that way would invent a distinction out of the + // tie-break, which is alphabetical and about their names rather than + // about them. + const rows = participationRows( + ranking({ + people: [ + person({ discord_user_id: '1', display_name: 'ana', sessions: 20 }), + person({ discord_user_id: '2', display_name: 'ben', sessions: 11 }), + person({ discord_user_id: '3', display_name: 'cai', sessions: 11 }), + person({ discord_user_id: '4', display_name: 'dee', sessions: 2 }), + ], + }), + ) + expect(rows.map((row) => row.rank)).toEqual([1, 2, 2, 4]) + expect(rows.map((row) => row.tied)).toEqual([false, true, true, false]) + }) + + it('keys every row by the id, so no two rows can collide', () => { + const rows = participationRows( + ranking({ + people: [person({ discord_user_id: '1' }), person({ discord_user_id: '2' })], + }), + ) + expect(new Set(rows.map((row) => row.key)).size).toBe(2) + }) +}) + +describe('naming a person', () => { + it('uses the display name when there is one', () => { + expect(participationPersonLabel(person({ display_name: 'ben' }))).toBe('ben') + expect(participationIdentityNote(person({ display_name: 'ben' }))).toBeNull() + }) + + it('shows the whole id when there is no name, and says it is an id', () => { + // Never a shortened one: snowflakes minted in the same era share their + // leading digits, so a truncated id identifies a group. + const nameless = person({ display_name: null, discord_user_id: '987654321098765432' }) + expect(participationPersonLabel(nameless)).toBe('Discord user 987654321098765432') + + const note = participationIdentityNote(nameless)! + expect(note).toContain('This is a Discord user id, not a name') + expect(note).toContain('no display name on record for them in this server') + expect(note).toContain('an id is a poor thing to put in a list about people') + }) +}) + +describe('how much of the server one person was present for', () => { + it('says it out of the total, never as a bare count', () => { + // A percentage survives being quoted without its scope; "11 of the 42 + // this server has recorded" carries it. + const line = participationAttendanceLine(person({ sessions: 11 }), 42) + expect(line).toBe('Recorded in 11 meetings of the 42 this server has recorded.') + expect(line).not.toContain('%') + }) + + it('counts one meeting as a meeting', () => { + expect(participationAttendanceLine(person({ sessions: 1 }), 42)).toContain('1 meeting of the 42') + }) + + it('says so when there is no total to read the figure against', () => { + const line = participationAttendanceLine(person({ sessions: 11 }), 0) + expect(line).toContain('out of a total this ranking does not know') + expect(line).toContain('nothing to read the figure against') + }) + + it('puts the total above the rows as well as inside them', () => { + const line = participationScopeLine(ranking({ sessions: 42, people: [person(), person({ discord_user_id: '3' })] })) + expect(line).toContain('2 people') + expect(line).toContain('42 meetings') + expect(line).toContain('Equal attendance shares a place') + expect(line).toContain('the order within a tie is alphabetical and means nothing') + }) + + it('says a place means little when the server reported no meeting count', () => { + const line = participationScopeLine(ranking({ sessions: 0 })) + expect(line).toContain('no meeting count for this server') + expect(line).toContain('a place in this order means very little without one') + }) +}) + +describe('speaking time', () => { + it('reads an absent measurement as an absence and never as silence', () => { + const line = participationSpeechLine(person({ speech_seconds: null, unmeasured_tracks: 4 })) + expect(line).toContain('No speaking time was ever measured for them') + expect(line).toContain('all 4 of their recordings here predate the columns that hold it') + expect(line).toContain('a measurement nobody took, not a person who said nothing') + expect(line).not.toContain('0 s') + }) + + it('says it does not know why when there is no measurement and no explanation', () => { + const line = participationSpeechLine(person({ speech_seconds: null, unmeasured_tracks: 0 })) + expect(line).toContain('holds no measured speaking time for them, and does not say why') + expect(line).toContain('missing measurement rather than as silence') + }) + + it('says how far short the figure falls when some recordings were never measured', () => { + const line = participationSpeechLine(person({ speech_seconds: 4200, unmeasured_tracks: 3 })) + expect(line).toContain('1 h 10 min of speech') + expect(line).toContain('3 of their recordings here were never measured at all') + expect(line).toContain('falls short by an unknown amount') + }) + + it('says what a complete figure is and is not a measure of', () => { + const line = participationSpeechLine(person({ speech_seconds: 4200, unmeasured_tracks: 0 })) + expect(line).toContain('1 h 10 min of speech') + expect(line).toContain('how long a microphone was open on speech') + expect(line).toContain('not a measure of anything else') + }) + + it('marks a row with no measurement so the page can render it as an absence', () => { + const rows = participationRows( + ranking({ + people: [ + person({ discord_user_id: '1', speech_seconds: null }), + person({ discord_user_id: '2', speech_seconds: 0 }), + ], + }), + ) + // Nobody ever measured, against somebody measured it and heard + // nothing. Two different facts, and only the first is an absence. + expect(rows[0]!.speechAbsent).toBe(true) + expect(rows[1]!.speechAbsent).toBe(false) + }) +}) + +describe('when somebody was recorded here', () => { + it('gives both ends of the span', () => { + const line = participationSeenLine(person()) + expect(line).toContain('First recorded 4 Nov 2025, 09:00 UTC') + expect(line).toContain('most recently 21 Aug 2026, 12:00 UTC') + }) + + it('says once rather than printing the same instant twice', () => { + const at = '2026-08-21T12:00:00+00:00' + expect(participationSeenLine(person({ first_seen_at: at, last_seen_at: at }))).toBe( + 'Recorded here once, on 21 Aug 2026, 12:00 UTC.', + ) + }) + + it('says which end is missing rather than standing a dash in for half a range', () => { + const line = participationSeenLine(person({ last_seen_at: null })) + expect(line).toContain('Only one end of their span here is known') + expect(line).toContain('4 Nov 2025, 09:00 UTC') + }) + + it('says nothing is known when neither end is', () => { + const line = participationSeenLine(person({ first_seen_at: null, last_seen_at: null })) + expect(line).toContain('did not say when they were first or last recorded here') + }) + + it('is on every row, because a rank read without it is read as a fact about the present', () => { + // Somebody who joined in June is behind a colleague who has been here + // since November for a reason that is about neither of them. + const row = participationRows(ranking())[0]! + expect(row.detail).toContain(row.attendance) + expect(row.detail).toContain(row.speech) + expect(row.detail).toContain(row.seen) + expect(row.detail.startsWith('ben.')).toBe(true) + }) +}) + +describe('what the reader is told, and when', () => { + it('calls it a ranking of named people in the first sentence', () => { + // Not "engagement", not "activity". Somebody who quotes this list + // elsewhere should have had to read that sentence first. + expect(PARTICIPATION_STANDING_NOTE).toContain('a ranking of named people') + expect(PARTICIPATION_STANDING_NOTE).toContain('how long their microphone carried speech') + }) + + it('says it is a different kind of report from the figures above it', () => { + expect(PARTICIPATION_STANDING_NOTE).toContain('a different kind of report from the figures above it') + expect(PARTICIPATION_STANDING_NOTE).toContain('those describe a server, this describes the individuals in it') + }) + + it('says that opening it is written to the audit log, and what the log holds', () => { + // Before the request goes out, because afterwards is too late to + // decline -- and the log holds who looked, never who was looked at. + expect(PARTICIPATION_STANDING_NOTE).toContain('written to the audit log') + expect(PARTICIPATION_STANDING_NOTE).toContain('which server, who looked, and when') + expect(PARTICIPATION_STANDING_NOTE).toContain('never who was in it') + }) + + it('says outright that attendance is not a measure of contribution', () => { + expect(PARTICIPATION_CONTRIBUTION_NOTE).toContain( + 'Being present in more meetings is not a measure of contribution, and speaking time even less so.', + ) + }) + + it('names what is invisible to it, so attendance is not read as a complete record', () => { + expect(PARTICIPATION_CONTRIBUTION_NOTE).toContain('attendance in the voice channels Sturnus records') + expect(PARTICIPATION_CONTRIBUTION_NOTE).toContain('a channel Sturnus does not watch') + expect(PARTICIPATION_CONTRIBUTION_NOTE).toContain('has not consented to being recorded') + expect(PARTICIPATION_CONTRIBUTION_NOTE).toContain('a low place on this list is not evidence of anything') + }) + + it('says this is a further purpose than the recordings were made for', () => { + expect(PARTICIPATION_PURPOSE_NOTE).toContain('made in order to write meetings up') + expect(PARTICIPATION_PURPOSE_NOTE).toContain('a further purpose than that one') + }) + + it('names co-determination rather than leaving it to a design document', () => { + expect(PARTICIPATION_PURPOSE_NOTE).toContain('subject to co-determination') + expect(PARTICIPATION_PURPOSE_NOTE).toContain('BetrVG §87(1)(6)') + expect(PARTICIPATION_PURPOSE_NOTE).toContain('agree on with a works council') + }) + + it('stands all three notes above the list at all times', () => { + // Above the reveal control as well as above the loaded rows: a note + // that appears only once the ranking is on screen arrives after the + // decision it exists to inform. + expect(participationNotes().map((note) => note.key)).toEqual(['what', 'meaning', 'purpose']) + expect(participationNotes().map((note) => note.text)).toEqual([ + PARTICIPATION_STANDING_NOTE, + PARTICIPATION_CONTRIBUTION_NOTE, + PARTICIPATION_PURPOSE_NOTE, + ]) + for (const note of participationNotes()) expect(note.label.length).toBeGreaterThan(0) + }) +}) + +describe('asking for it', () => { + it('names what the control will load rather than saying "show more"', () => { + // The click is the moment somebody becomes a person who looked at a + // ranking of their colleagues; a generic label arranges for them to do + // it by accident. + expect(PARTICIPATION_REVEAL_LABEL).toBe('Show the attendance ranking') + expect(PARTICIPATION_REVEAL_BUSY_LABEL).toBe('Reading the ranking…') + expect(PARTICIPATION_LOADING_NOTE).toContain('attendance ranking') + }) + + it('says what pressing it does before it is pressed', () => { + expect(PARTICIPATION_REVEAL_NOTE).toContain('Nothing has been loaded') + expect(PARTICIPATION_REVEAL_NOTE).toContain('by name, ordered by how many meetings each of them was in') + expect(PARTICIPATION_REVEAL_NOTE).toContain('writes a line in the audit log saying that you asked') + }) + + it('says why it is not loaded with the figures above it', () => { + // The reason this section is on demand at all, on the page rather than + // only in the code comment that implements it. + expect(PARTICIPATION_REVEAL_NOTE).toContain('The figures above were loaded without any of that') + expect(PARTICIPATION_REVEAL_NOTE).toContain('whether transcription is keeping up') + expect(PARTICIPATION_REVEAL_NOTE).toContain('does not record you as having looked at a ranking of your colleagues') + }) + + it('does not pretend hiding it again unsays anything', () => { + expect(PARTICIPATION_HIDE_LABEL).toBe('Hide the ranking') + expect(PARTICIPATION_HIDE_NOTE).toContain('the audit line stays') + expect(PARTICIPATION_HIDE_NOTE).toContain('hiding it afterwards does not change that it was') + }) +}) + +describe('a server with nobody in it', () => { + it('knows an empty list from a list', () => { + expect(isParticipationEmpty(ranking({ people: [] }))).toBe(true) + expect(isParticipationEmpty(ranking())).toBe(false) + // Nobody to rank even though meetings were recorded is still empty: + // there is no row to draw for a person who does not exist in it. + expect(isParticipationEmpty(ranking({ sessions: 42, people: [] }))).toBe(true) + }) + + it('says nobody is missing from an empty list', () => { + expect(PARTICIPATION_EMPTY_HEADING).toBe('Sturnus has recorded nobody in this server') + expect(PARTICIPATION_EMPTY_NOTE).toContain('There is nobody to list') + expect(PARTICIPATION_EMPTY_NOTE).toContain('Nobody is missing from this list') + expect(PARTICIPATION_EMPTY_NOTE).toContain('no row rather than a row of zeros') + expect(PARTICIPATION_EMPTY_NOTE).toContain('a zero would say they attended nothing') + }) + + it('draws no rows for it', () => { + expect(participationRows(ranking({ people: [] }))).toEqual([]) + expect(participationScopeLine(ranking({ people: [] }))).toContain('0 people') + }) +}) + +describe('what this list is never allowed to become', () => { + it('uses none of the vocabulary of a leaderboard', () => { + // Every sentence this module can produce, checked at once. Speaking + // time is secondary to attendance and must never be presented as a + // competing order, and the words that would do that are the words that + // creep back in first. + const prose = everySentence( + ranking({ + people: [ + person({ discord_user_id: '1', display_name: 'ana', sessions: 20 }), + person({ discord_user_id: '2', display_name: null, sessions: 11, speech_seconds: null, unmeasured_tracks: 4 }), + person({ discord_user_id: '3', display_name: 'cai', sessions: 11, unmeasured_tracks: 2 }), + ], + }), + ).toLowerCase() + + for (const forbidden of [ + 'top talker', + 'top speaker', + 'leaderboard', + 'podium', + 'winner', + 'champion', + 'trophy', + 'medal', + 'most active', + 'talk share', + 'share of talk', + 'productivity', + 'engagement', + ]) { + expect(prose).not.toContain(forbidden) + } + }) + + it('states no figure about a person as a percentage', () => { + // A percentage is the form of this figure that travels furthest from + // its own scope, which is exactly what a page about named individuals + // must not hand out. + expect(everySentence(ranking({ people: [person(), person({ discord_user_id: '3', sessions: 40 })] }))).not.toContain('%') + }) +}) + +describe('when the API says no', () => { + it('says a session has ended rather than that the ranking is empty', () => { + const message = describeParticipationError(failure(401)) + expect(message).toContain('Your session has ended') + expect(message).toContain('the ranking was not loaded') + }) + + it('names what an administrator is when it refuses one who is not', () => { + expect(describeParticipationError(failure(403))).toContain('admin_role_id') + }) + + it('covers both readings of a 404 without guessing which', () => { + // The API answers 404 both for a server that does not exist and for + // one the caller does not administer, on purpose: it will not confirm + // the existence of a server to somebody with no business there. + const message = describeParticipationError(failure(404)) + expect(message).toContain('does not know this server, or you no longer administer it') + expect(message).toContain('answers the same way to both') + expect(message).toContain('no ranking was loaded') + }) + + it('says a request that never arrived did not arrive', () => { + const message = describeParticipationError(new Error('offline')) + expect(message).toContain('Could not reach the API') + expect(message).toContain('no ranking was loaded') + }) + + it('reports an unexpected status with the number and no invention', () => { + const message = describeParticipationError(failure(503)) + expect(message).toContain('503') + expect(message).toContain('Nothing is known about why') + }) + + it('never leaves the reader unsure whether they saw part of the list', () => { + // A reader who is unsure will press the button again, and pressing it + // again is another audit line. + for (const status of [401, 403, 404, 500]) { + expect(describeParticipationError(failure(status)).toLowerCase()).toMatch(/no ranking|not loaded|no ranking was/) + } + }) +}) diff --git a/console/test/reporting.spec.ts b/console/test/reporting.spec.ts index ec2d214..9c86b1c 100644 --- a/console/test/reporting.spec.ts +++ b/console/test/reporting.spec.ts @@ -25,9 +25,11 @@ * back. * * And one thing is asserted by its absence: nothing in this module names - * or ranks a person, because this report is about a server and a - * per-person readout of attendance and speaking time is a works-council - * matter rather than a console feature. + * or ranks a person, because this report is about a server. The Reporting + * page does carry an attendance ranking, in `~/utils/participation` and + * behind a reveal of its own -- the boundary between the two is the point, + * and `REPORT_SCOPE_NOTE` is tested here for naming it rather than for + * denying that anything of the sort exists. */ import { describe, expect, it } from 'vitest' @@ -652,16 +654,26 @@ describe('a server with nothing to report', () => { }) describe('what this report is not about', () => { - it('says outright that there is no per-person breakdown behind it', () => { + it('says outright that none of these figures is about a person', () => { // A per-person readout of meeting attendance and speaking time is a // means of monitoring conduct and performance at work, which is a - // works-council matter rather than a console feature. The payload - // carries no names and no ids; the framing has to match. + // works-council matter. This payload carries no names and no ids; the + // framing has to match. expect(REPORT_SCOPE_NOTE).toContain('about the server as a whole and never about the people in it') - expect(REPORT_SCOPE_NOTE).toContain('no per-person breakdown behind it') + expect(REPORT_SCOPE_NOTE).toContain('none of these figures can be traced back to one person') expect(REPORT_SCOPE_NOTE).toContain('Counts of people are counts, and stop there.') }) + it('names the one thing on the page that is about people, rather than denying it', () => { + // The page below carries an attendance ranking, in its own module and + // behind a reveal. A scope note claiming no per-person readout exists + // anywhere would be false the moment somebody scrolled, and a reader + // who caught it in one claim has no reason to believe the others. + expect(REPORT_SCOPE_NOTE).toContain('The attendance ranking at the foot of this page is the one exception') + expect(REPORT_SCOPE_NOTE).toContain('fetched only when somebody asks for it') + expect(REPORT_SCOPE_NOTE).toContain('written to the audit log') + }) + it('describes the participant count as a count and nothing more', () => { const people = reportShapeFigures(report()).find((figure) => figure.key === 'participants')! expect(people.value).toBe('12') @@ -669,10 +681,12 @@ describe('what this report is not about', () => { expect(people.note).toContain('does not send this page their names') }) - it('never hints that a per-person readout is on its way', () => { - // Every sentence this module can produce, checked at once: an - // interface that promises a breakdown is as much a promise as one that - // ships it. + it('never turns one of these figures into a readout about a person', () => { + // Every sentence this module can produce, checked at once. The page + // does carry an attendance ranking, and it is a separate module behind + // its own reveal; what must not happen is one of the figures *here* + // acquiring a per-person reading, which is how a server-level report + // becomes a per-person one without anybody deciding it should. const prose = everySentence( report({ months: [month({ month: '2026-01' }), month({ month: '2026-03' })], open_sessions: 1 }), ) From 2e514df7856c2de2682edbaa971dc6e7c6acf0df Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Sat, 22 Aug 2026 22:39:06 +0200 Subject: [PATCH 14/14] docs(operations): what the attendance ranking is, before switching it on MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The runbook section for the one thing Sturnus produces that names other people and ranks them. It states the co-determination question plainly -- BetrVG §87(1)(6), and the GDPR purpose-limitation half of the same point -- and says outright that neither makes the feature wrong to have, only that the decision belongs to the people who run the guild. The rest is what an operator needs in order to act on that: which LogQL query answers "who read the ranking, and when", why the log line carries a count rather than the list, what the ranking deliberately does not report, and that declining it is one revert which leaves the rest of the reporting page working. --- docs/operations.md | 53 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/docs/operations.md b/docs/operations.md index 7b15cd5..8a48ae5 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -1193,6 +1193,59 @@ council (BetrVG §87(1)(6)) rather than something a console adds because the columns happen to be there. The rows exist and the ranking is buildable; building it is a separate, deliberate act. +### 6.2.9 The attendance ranking, and why it is its own thing + +**Admin View → Reporting → attendance**, per guild: the people this guild +has recorded, ordered by how many of its meetings they were in, with how +long each of them spoke. + +**Read this before switching it on.** This is the only thing Sturnus +produces that names other people and ranks them. Everything else in the +console is either about the person reading it or about a guild in +aggregate. + +An ordered list of colleagues by meeting attendance and speaking time is +a `technische Einrichtung, die dazu bestimmt ist, das Verhalten oder die +Leistung der Arbeitnehmer zu überwachen` — BetrVG §87(1)(6) — and is +therefore subject to co-determination in a German workplace with a works +council, regardless of what anybody intended it for. The GDPR half is the +same point from the other side: the recordings were collected so a +protocol could be written, and an attendance ranking serves a further +purpose from the same data. Neither of those is a reason it cannot exist. +Both are reasons the decision belongs to the people who run the guild +rather than to whoever deploys the bot. + +Practically: + +- **It is a separate endpoint, port and module** (`/report/participation`, + `ParticipationReports`, `sturnus.console.participation`). Removing it is + a revert of one change, not an audit of a shared response shape. The + aggregate report at `/report` is untouched by that and continues to name + nobody. +- **Every read is logged**, at INFO, which no other read in the console + is: + + ```logql + {namespace="sturnus"} | json | sturnus_event="console.participation_read" + ``` + + The line carries `guild_id`, `requested_by` and how many people were in + the answer. It deliberately does *not* carry who they were — the list is + the thing under discussion, and copying it into a retained, searchable + log store would be making a second copy of it. +- **It reports attendance and speaking time and nothing further.** No + words spoken, no punctuality, no share-of-talk, nothing per meeting. + Each of those would be another purpose and would need deciding again. +- **It is ordered by meetings attended, never by speaking time.** "Was + present most often" and "talked the most" are different statements about + a colleague, and only one of them was asked for. +- **`speech_seconds` is null-aware.** A person whose recordings predate + the measurement columns has no speaking total, and the page says so + rather than showing a zero that reads as silence. + +If this is not wanted in a deployment, do not merge or do revert the +change that adds it; the rest of the reporting page works without it. + ### 6.3 Listening to a recording by hand Every automated check this system has can describe a track — its level,