Skip to content

Commit 2ee0708

Browse files
authored
feat(insights): add organization activity dashboards (#7884)
* feat(insights): add organization activity dashboards * fix(insights): address accessibility and review edge cases * fix(charts): preserve stable series identities * fix(insights): restore original header actions
1 parent 8d1f07b commit 2ee0708

60 files changed

Lines changed: 29456 additions & 1090 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/test-build.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -105,11 +105,11 @@ jobs:
105105
lib/auth/sso/application/admit-sso-user.postgres.test.ts
106106
lib/auth/sso/primary-provider.postgres.test.ts
107107
108-
- name: Verify cumulative billing timeout recovery in PostgreSQL
108+
- name: Verify billing and organization activity in PostgreSQL
109109
working-directory: apps/sim
110110
env:
111111
BILLING_USAGE_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
112-
run: bunx vitest run lib/billing/core/usage-log.postgres.test.ts
112+
run: bunx vitest run lib/billing/core/usage-log.postgres.test.ts lib/billing/core/organization-activity.postgres.test.ts
113113

114114
- name: Verify cumulative billing timeout recovery on PostgreSQL 16
115115
if: matrix.provision == 'push'

apps/docs/content/docs/platform/enterprise/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Sim Enterprise adds organization controls for access, provisioning, operations,
1414
| [Security](/platform/enterprise/security) | Manage session policies and view configured outbound IP addresses |
1515
| [Session policies](/platform/enterprise/session-policies) | Set session lifetimes and revoke member sessions |
1616
| [Audit logs](/platform/enterprise/audit-logs) | Investigate configuration and security events |
17-
| [Usage tracking](/platform/enterprise/usage-tracking) | Review usage by member, workspace, model, and source |
17+
| [Insights](/platform/enterprise/usage-tracking) | Explore credit usage, workflow performance, and recorded chat activity |
1818
| [Data retention](/platform/enterprise/data-retention) | Set retention windows and configure PII redaction |
1919
| [Data drains](/platform/enterprise/data-drains) | Export logs and Chat records to your own destination |
2020
| [Custom blocks](/platform/enterprise/custom-blocks) | Share a workflow as a block across the organization |

apps/docs/content/docs/platform/enterprise/self-hosted.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Three features do not need a flag at all: **custom branding**, **session policie
5151
| Directory provisioning (SCIM) | `SCIM_ENABLED` | `NEXT_PUBLIC_SCIM_ENABLED` |
5252
| Custom branding — on by default | `WHITELABELING_ENABLED` | `NEXT_PUBLIC_WHITELABELING_ENABLED` |
5353
| Audit logs | `AUDIT_LOGS_ENABLED` | `NEXT_PUBLIC_AUDIT_LOGS_ENABLED` |
54-
| Usage tracking | `USAGE_MONITORING_ENABLED` | `NEXT_PUBLIC_USAGE_MONITORING_ENABLED` |
54+
| Insights | `USAGE_MONITORING_ENABLED` | `NEXT_PUBLIC_USAGE_MONITORING_ENABLED` |
5555
| Custom blocks | `CUSTOM_BLOCKS_ENABLED` | `NEXT_PUBLIC_CUSTOM_BLOCKS_ENABLED` |
5656
| Session policies — on by default | `SESSION_POLICIES_ENABLED` | `NEXT_PUBLIC_SESSION_POLICIES_ENABLED` |
5757
| Data retention deletion | `DATA_RETENTION_ENABLED` | `NEXT_PUBLIC_DATA_RETENTION_ENABLED` |

apps/docs/content/docs/platform/enterprise/usage-tracking.mdx

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,21 @@
11
---
2-
title: Usage Tracking
3-
description: See where your organization's credits go, by member, workspace, and model
2+
title: Insights
3+
description: Explore organization credit usage, workflow performance, and recorded chat activity
44
---
55

66
import { Callout } from 'fumadocs-ui/components/callout'
77
import { FAQ } from '@/components/ui/faq'
88
import { Image } from '@/components/ui/image'
99

10-
Usage tracking shows how your organization consumes credits across every part of the platform — which members, which workspaces, which models, and which product features. Use it to monitor spend against your commitment, find what is driving it, and export the underlying events for chargeback.
10+
Insights shows how your organization consumes credits across every part of the platform — which members, which workspaces, which models, and which product features. Use it to monitor spend against your commitment, find what is driving it, and export the underlying events for chargeback.
1111

12-
All figures are in **credits** (1 credit = $0.005). See [cost calculation](/platform/costs) for how credits are derived.
12+
Spend figures are in **credits** (1 credit = $0.005). See [cost calculation](/platform/costs) for how credits are derived.
1313

1414
---
1515

1616
## Viewing usage
1717

18-
Go to **Settings → Organization → Usage tracking** in your workspace.
19-
20-
<Image src="/static/enterprise/usage-tracking-overview.png" alt="Usage tracking Overview tab showing the period selector, credits used against the organization limit, a daily usage chart, and a Sources section pairing a ranked list of sources with a radar chart of the same mix" width={900} height={781} className="mx-auto h-auto w-full max-w-xl" />
18+
Go to **Settings → Organization → Insights** in your workspace.
2119

2220
The period selector applies to every tab:
2321

@@ -36,7 +34,8 @@ The period selector applies to every tab:
3634

3735
| Tab | Answers |
3836
|-----|---------|
39-
| **Overview** | How much have we used, against what limit, and what kind of work was it |
37+
| **Overview** | Credit consumption, recorded activity, workflow failure rate, and usage by source |
38+
| **Activity** | Run trends and performance, grouped by workspace, workflow, chat member, or trigger |
4039
| **Members** | Which people are driving usage |
4140
| **Workspaces** | Which workspaces are driving usage — select one to drill in |
4241
| **Models** | Which models we are paying for |
@@ -49,6 +48,22 @@ Selecting a workspace opens its detail view, which splits that workspace's usage
4948

5049
---
5150

51+
## Activity and performance
52+
53+
The **Activity** tab shows workflow runs, recorded chat runs, members with chat activity, and workflow failure rate. Group the breakdown by **Workspaces**, **Workflows**, **Members**, or **Triggers**, and sort by run volume, failure count, or average workflow duration. Member breakdowns rank chat runs only. Results are paginated; totals and sorting include the whole selected period, not just the visible page. Select a workspace to narrow the activity view, then use **All workspaces** to clear that filter.
54+
55+
- **Workflow runs** count retained execution records whose start time falls in the selected period, including runs that are still running, paused, or cancelled.
56+
- **Recorded chat runs** count distinct recorded execution IDs across workspace and organization chats. Continuations count once, in the period containing their first retained start. Calls without a persisted chat run are not included.
57+
- **Members with chat activity** count distinct users attributed to those chat runs. This measures recorded chat activity, not logins, all active users, or people billed for automated workflows.
58+
- **Workflow failure rate** is failed runs divided by completed plus failed runs. Paused, cancelled, and unfinished runs do not enter the denominator. A dash means there are no completed or failed runs to measure.
59+
- **Average duration** uses the recorded duration of completed or failed workflows. Missing durations are excluded; a dash means no duration is available.
60+
61+
Activity follows retained history and current organization/workspace ownership. Deleting history or moving a workspace can change historical activity totals. Status and duration reflect the latest recorded outcome, even for a run that started in an earlier period. The activity view does not read chat transcripts or workflow trace payloads.
62+
63+
Credit totals continue to come from the billing ledger. Activity counts need not match billing-event counts, and the credit export does not export activity. Exact per-chat cost, connector/skill adoption, and generated-artifact attribution are not available in this view.
64+
65+
---
66+
5267
## What each source means
5368

5469
A **source** is the part of the platform that consumed the credits. Every charge belongs to exactly one source, so the Sources breakdown always adds up to your total.
@@ -102,17 +117,17 @@ When a workspace or organization supplies its own provider key, Sim does not cha
102117

103118
## Exporting
104119

105-
**Export** downloads the events behind the current period and filters as a CSV with columns `Date, Source, Description, Workflow, Credits`. Credits are exported as plain numbers so the column can be summed, and carry decimals — an individual event often costs a fraction of a credit.
120+
**Export credits** downloads the events behind the current period and filters as a CSV with columns `Date, Source, Description, Workflow, Credits`. Credits are exported as plain numbers so the column can be summed, and carry decimals — an individual event often costs a fraction of a credit.
106121

107-
**All events** opens the full ledger — every credit-consuming event, newest first, with its own filters and export.
122+
**Credit events** opens the full ledger — every credit-consuming event, newest first, with its own filters and export.
108123

109124
Very large exports are capped. When that happens the download still succeeds and Sim tells you it was truncated; narrow the date range to capture everything.
110125

111126
---
112127

113128
<FAQ items={[
114129
{
115-
question: "Who can view usage tracking?",
130+
question: "Who can view Insights?",
116131
answer: "Organization owners and admins. On Sim Cloud you must be on the Enterprise plan. Regular members cannot see organization usage, including their own."
117132
},
118133
{
@@ -158,4 +173,4 @@ USAGE_MONITORING_ENABLED=true
158173
NEXT_PUBLIC_USAGE_MONITORING_ENABLED=true
159174
```
160175

161-
Once enabled, usage tracking is viewable in **Settings → Organization → Usage tracking** for organization owners and admins.
176+
Once enabled, Insights is viewable in **Settings → Organization → Insights** for organization owners and admins.
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import { getOrganizationActivityBreakdownContract } from '@/lib/api/contracts/organization-activity'
2+
import {
3+
defineInternalJsonRoute,
4+
internalRateLimits,
5+
internalSessionAuth,
6+
} from '@/lib/api/server/routes'
7+
import { getOrganizationActivityBreakdown } from '@/lib/billing/application/organization-usage/get-organization-activity'
8+
import { organizationUsageOperations } from '@/lib/billing/application/organization-usage/operations'
9+
import { organizationUsageErrorPolicy } from '@/app/api/organizations/[id]/usage/error-policy'
10+
11+
export const dynamic = 'force-dynamic'
12+
13+
export const GET = defineInternalJsonRoute({
14+
contract: getOrganizationActivityBreakdownContract,
15+
auth: internalSessionAuth,
16+
operation: organizationUsageOperations.readActivityBreakdown,
17+
rateLimit: internalRateLimits.user({
18+
bucketName: 'organization-activity',
19+
}),
20+
errorPolicy: organizationUsageErrorPolicy,
21+
staticResponseHeaders: { 'Cache-Control': 'private, no-store' },
22+
mapInput: ({ params, query }) => ({
23+
...query,
24+
organizationId: params.id,
25+
startDate: query.startDate ? new Date(query.startDate) : undefined,
26+
endDate: query.endDate ? new Date(query.endDate) : undefined,
27+
}),
28+
useCase: getOrganizationActivityBreakdown,
29+
present: (result) => result,
30+
})
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
import { getOrganizationActivitySummaryContract } from '@/lib/api/contracts/organization-activity'
2+
import {
3+
defineInternalJsonRoute,
4+
internalRateLimits,
5+
internalSessionAuth,
6+
} from '@/lib/api/server/routes'
7+
import { getOrganizationActivitySummary } from '@/lib/billing/application/organization-usage/get-organization-activity'
8+
import { organizationUsageOperations } from '@/lib/billing/application/organization-usage/operations'
9+
import { organizationUsageErrorPolicy } from '@/app/api/organizations/[id]/usage/error-policy'
10+
11+
export const dynamic = 'force-dynamic'
12+
13+
export const GET = defineInternalJsonRoute({
14+
contract: getOrganizationActivitySummaryContract,
15+
auth: internalSessionAuth,
16+
operation: organizationUsageOperations.readActivitySummary,
17+
rateLimit: internalRateLimits.user({
18+
bucketName: 'organization-activity',
19+
}),
20+
errorPolicy: organizationUsageErrorPolicy,
21+
staticResponseHeaders: { 'Cache-Control': 'private, no-store' },
22+
mapInput: ({ params, query }) => ({
23+
...query,
24+
organizationId: params.id,
25+
startDate: query.startDate ? new Date(query.startDate) : undefined,
26+
endDate: query.endDate ? new Date(query.endDate) : undefined,
27+
}),
28+
useCase: getOrganizationActivitySummary,
29+
present: (result) => result,
30+
})

apps/sim/app/o/[organizationId]/settings/components/integrations/organization-source-stats.test.tsx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,8 @@ const mocks = vi.hoisted(() => ({
1414
vi.mock('@/hooks/queries/organization-search-stats', () => ({
1515
useOrganizationSearchStats: mocks.query,
1616
}))
17-
vi.mock('@/components/charts', () => ({
17+
vi.mock('@sim/emcn', async (importOriginal) => ({
18+
...(await importOriginal<typeof import('@sim/emcn')>()),
1819
BarChart: (props: unknown) => {
1920
mocks.chart(props)
2021
return <div>Daily chart</div>

apps/sim/app/o/[organizationId]/settings/components/integrations/organization-source-stats.tsx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
11
'use client'
22

33
import { type ReactNode, useMemo } from 'react'
4-
import { Chip, ChipSelect, Tooltip } from '@sim/emcn'
4+
import { BarChart, Chip, ChipSelect, Tooltip } from '@sim/emcn'
55
import { CircleInfo } from '@sim/emcn/icons'
66
import { useQueryStates } from 'nuqs'
7-
import { BarChart } from '@/components/charts'
87
import {
98
SEARCH_STATS_PEOPLE_LIMIT,
109
SEARCH_STATS_SURFACE_LABELS,

apps/sim/app/workspace/[workspaceId]/logs/components/dashboard/dashboard.tsx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,8 @@
11
'use client'
22

33
import { memo, useCallback, useMemo, useRef, useState } from 'react'
4-
import { Loader } from '@sim/emcn'
4+
import { LineChart, Loader } from '@sim/emcn'
55
import { useParams } from 'next/navigation'
6-
import { LineChart } from '@/components/charts'
76
import {
87
DashboardSegmentsContext,
98
type SegmentSelectionMode,

apps/sim/app/workspace/[workspaceId]/logs/utils.ts

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
import React from 'react'
2-
import { Badge } from '@sim/emcn'
2+
import { Badge, formatChartLatency } from '@sim/emcn'
33
import { formatRelativeTime } from '@sim/utils/formatting'
44
import { format } from 'date-fns'
5-
import { formatChartLatency } from '@/components/charts/chart-format'
65
import { getIntegrationMetadata } from '@/lib/logs/get-trigger-options'
76
import { getBlock } from '@/blocks/registry'
87
import { CORE_TRIGGER_TYPES } from '@/stores/logs/filters/types'

0 commit comments

Comments
 (0)