Extract the dashboard into the actionagent gem, with feature parity - #353
Merged
Conversation
…platform's The dashboard engine shipped model classes for agents, runs, versions, templates, sandboxes and recordings, but no migrations to back them — the platform's own copies had grown apart and were the only ones running. Replace the drifted copies with the platform's implementations, add the schema they need, and give the engine the seams a host app has to reach: - table_name_prefix, so the engine's active_agent_* tables and a host app that already owns them unprefixed are both supported without renaming production tables - Ownable, resolving the owner association from configuration rather than assuming an Account or a User exists - quota_checker / provider_credentials_resolver / trace_owner_resolver, so plan limits, per-tenant LLM credentials and agent attribution stay in the app that has that data - sandbox_backends, replacing the hardcoded Incus/Kubernetes/Cloud Run switch with a registry; the engine ships the in-memory backend and the operator registers the rest Conversation persistence (contexts, messages, generations), evaluations, scorecards and cost estimates come along, so a mounted engine has the same feature set the hosted product does. Queries that were PostgreSQL-only now have portable forms: COUNT FILTER becomes SUM CASE, DISTINCT ON becomes a max-id pass, jsonb traversal falls back to reading the column in Ruby, and jsonb columns are declared json on adapters without jsonb. The dashboard runs on SQLite and MySQL again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
Adds the JSON API and the React frontend to the engine, so mounting it gives you the surface the hosted product has rather than traces and metrics alone: agents and versions, runs, interactions, evaluations, templates, sandboxes, recordings, provider and API keys, and the agents-as-MCP-server endpoint. The controllers are the platform's, rewritten onto the engine's seams. Anything they reached for directly — the account, the signed-in user's agents, plan limits, usage counters, Stripe checkout — now goes through configuration, so the same controller serves a single-user install, a per-user install and a multi-tenant platform: - `owned(relation)` scopes by whatever the model declared with owned_by, rather than assuming who the owner is - quota_denial / record_usage / upgrade_url replace can_run_agent?, increment_agent_runs! and the checkout POST - execution_enabled? leaves a read-only observability dashboard when an operator would rather the mount not call providers at all The React app drops Inertia: initial state arrives as a JSON data attribute and the bundle ships prebuilt in the gem, so a host app mounts the dashboard without adopting a frontend framework or running a build. One fetch shim rewrites /api/... onto the mount path, which is what lets the engine live at any prefix. Action Cable is optional now — without a cable mount the views poll instead of failing. /api/traces keeps its meaning on both verbs: POST is still the SDK's token-authenticated ingest, GET is the dashboard's session-authenticated read (Api::TraceReportsController). Remaining PostgreSQL-only queries got portable forms (ILIKE, DISTINCT ON, jsonb traversal), and session recordings are scoped by a real owner column instead of a JSON metadata key. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The install generator now writes the dashboard's tables alongside the telemetry ones, so `rails generate active_agent:dashboard:install` gets an app the full surface rather than a dashboard whose models have nothing to read. --traces_only keeps the old behaviour for an app that only wants to be a trace sink, and the generator points at db:encryption:init because API keys and provider credentials are encrypted at rest. Traces gain agent_id, which is what attributes a reported execution to a dashboard agent — the platform had grown that column but the generator never shipped it, so AgentRegistrar had nothing to write to on a self-hosted install. Adds coverage for the surface that just moved: agent CRUD and versioning, case-insensitive search, the quota/usage/execution-toggle seams, the ownership declarations, the configurable table prefix, and traces and metrics reading what ingest writes. All of it runs on SQLite, which is the point — the last PostgreSQL-only query (date_trunc) now has portable forms for SQLite and MySQL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
…eparate homes The React app pushed absolute /dashboard/... URLs, which only worked for an engine mounted at exactly that path — mounted anywhere else, every in-app navigation left the dashboard. Client routes now resolve against the mount the server published, so /activeagents, /dashboard and a subdomain root all behave the same. That surfaced a collision: the React dashboard and the server-rendered traces pages both wanted <mount>/traces. The React app takes the mount and everything under it — a catch-all serves its client routes, and refuses /api paths so a mistyped endpoint still answers like an API. The server-rendered console moves to <mount>/console/traces, where it stays useful when the bundle can't run. Also fixes seams that only bite a host app with tenants: - current_user_method = :current_user recursed forever, because the engine's controllers are their own base class and the accessor called itself. Resolvers take a controller instead, and a named method is only called when the controller actually has it. - current_owner no longer falls back to the signed-in user in multi-tenant mode, which had been handing a user with no tenant a scope they were not part of. - Reachable agents go through agent_scope_resolver, since ownership and reachability differ when an account's key must reach its members' agents. - Telemetry resolves through tenant_for, because traces belong to accounts while agents may belong to users. - The executions query no longer reads reported traces in multi-tenant mode without a tenant to scope them to. - A quota denial may answer with a payload, so a host app can surface its own usage numbers with the 402. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The self-hosted guide sold interactions, evaluations, scorecards and cost estimates as reasons to use the hosted platform instead. They ship in the engine now, so the comparison tables say what actually differs: the platform adds accounts, plans, billing and managed sandboxes around the same feature set. Also documents what changed for anyone mounting it: the full schema the generator writes and its --traces_only escape hatch, the encryption keys credentials need, the prebuilt React bundle, the server-rendered console's move to <mount>/console/traces, running agents and registering a sandbox backend, and the retention job that replaced "write your own delete_all". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The class was MCPRecordingMiddleware in a file called mcp_recording_middleware.rb, which only resolves in an app that has registered an "mcp" acronym with its autoloader. In the platform it did, which is why this never surfaced there; anywhere else the engine failed to eager load. Adds the test that would have caught it: eager loading the engine, which autoloading a few constants per test never did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
Follows the extraction: the dashboard engine is most of this gem now, and the docs still described it as the traces-and-metrics half. - AGENTS.md gains a Dashboard Engine section — where the code lives, how the namespacing works, what the mount-relative paths are, and that the React bundle is committed so host apps never run a build. It is the largest subsystem in the repo and a contributor had no map to it. - The VitePress sidebar matches the pages' actual titles again. - telemetry.md's endpoint paths match the engine's routes. - v2-extraction-roadmap.md is re-audited against what shipped: the MCP server half is done, "hosted UI" leaves the platform layer, and AgentToolbox is marked as landed-but-in-the-wrong-layer, since generic execution code is now reachable only through the dashboard engine. - rails.md's generator output listing matches what the generator writes (test/agents/*_test.rb and test/docs/previews/*_preview.rb). The VitePress site builds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The dashboard had grown to 77% of the activeagent payload — 932 KB of
prebuilt assets, 641 KB of JSX sources and 384 KB of Ruby against 557 KB of
actual framework. Size was the visible problem; the dependency graph was the
real one.
`activeagent` deliberately depends on actionpack/actionview/activesupport/
activemodel/activejob and NOT on activerecord or rails. But the engine's
models inherit ActiveRecord::Base, lib/active_agent.rb required the engine
unconditionally in every Rails app, and eight app/* paths were appended to
every host's autoload and eager-load paths. An API-only app on
--skip-active-record would have crashed in production on a dependency the
gemspec never declared, and CI could not have caught it: the dummy app sets
eager_load = false and requires active_record/railtie unconditionally.
Worse, AgentExecutionService mixes in SolidAgent::HasContext, and solid_agent
depends on activeagent. The framework could never declare a dependency its
own dashboard code has — that is a cycle. There is no packaging of this code
inside activeagent where the dependency graph is true.
So the dashboard becomes `actionagent`, a sibling gem in this repo with its
own gemspec, declaring activerecord, railties and solid_agent honestly.
Rails names the data layer Active* and the request/response layer Action*,
and a mountable dashboard is squarely the latter — so the namespace follows
the gem: ActiveAgent::Dashboard::Agent is ActionAgent::Agent, and the engine
mounts as ActionAgent::Engine.
activeagent 2546.1 KB -> 134.5 KB (163 files, no dashboard)
actionagent 360.0 KB (85 files, no JSX sources)
The engine now sits at a real gem root, so the find_root override it needed
while buried under lib/ is gone, and Rails locates app/ and config/routes.rb
by itself.
Nothing breaks for existing installs: ActionAgent::Compatibility keeps
ActiveAgent::Dashboard, ActiveAgent::TelemetryTrace and
ActiveAgent::ProcessTelemetryTracesJob resolving through const_missing with
a deprecation. That last one matters beyond tidiness — Active Job serializes
the class name into the queue payload, so jobs enqueued before an upgrade are
dequeued after it.
Two defects found along the way and fixed here: activeagent never declared
railties despite requiring "rails", and the frontend JSX sources were
shipping in the gem for no reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
Renames the namespace throughout the guides and, where a find-and-replace could not help, rewrites the prose: the install story is two gems now, the dev console page no longer opens by claiming the framework ships a dashboard, and configuration.md documents ActionAgent.configure, which it had never mentioned because there was nothing separate to mention. v2-extraction-roadmap.md gains a fourth layer between solid_agent and the platform, and its AgentToolbox item is re-audited: the code left activeagent with the dashboard, so it is further from the framework than when the item was written, and an app now installs the whole engine to get a safe fetch_url. Corrects a claim I had made in three places, including the actionagent gemspec: railties is not a dashboard-only dependency. activeagent declares it too — its railtie requires "rails". Active Record is the dependency that actually distinguishes the two gems. CI fixes: - rubocop: the regenerated dummy schema violated the array-bracket rule. - Test API Gems: the dummy schema was regenerated locally against Rails 8.1 and pinned to ActiveRecord::Schema[8.1], which the older Rails in that matrix cannot load. Back to [8.0], the version the file already declared. - the dummy's dashboard migration still called ActiveAgent::Dashboard. The VitePress site builds and rubocop is clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The gemspec globbed both, but neither file existed under actionagent/, so the packaged gem carried no license text and its RubyGems page would have been blank — sloppy for a gem released on its own. The README documents the install, the generator flags, the configuration seams and the deprecated ActiveAgent::Dashboard constants. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
TonsOfFun
marked this pull request as ready for review
August 13, 2026 16:47
This was referenced Aug 13, 2026
A review of this stack turned up defects that CI could not see, because the
engine's own 77 tests were never in what CI runs and no host-shaped install
is exercised anywhere. Fixed here, on the branch that introduced them.
Authentication and tenant isolation
* Api::SandboxesController opted out of authentication entirely for an
anonymous free tier. That made #run and #compare an open proxy: any
caller could execute arbitrary prompts against the host app's provider
credentials, and #show read any session back by id. Opting out also
skipped the callback that refuses to serve an unauthenticated dashboard
outside development, so it defeated that safeguard too. The controller
now authenticates, honours the execution kill switch and the quota seam,
and scopes its lookups through the ownership seam.
* Session recordings carry a replayable timeline of a real browser session,
including values typed into forms. start_user_session, record_action,
complete_session and demo were all anonymous. GET .../demo fell back to
the oldest completed recording when no seeded demo row existed, so an
install with no demo published a real customer session to anyone who
found the mount. The demo endpoint is removed and the rest authenticate.
Reads and writes go through the controller's existing ownership check,
which set_recording never called — every recording was readable by id.
* Templates#show looked a template up by bare id while anonymous, serving
unpublished drafts and private prompt libraries. No exemption now.
* for_owner(nil) returned every row. "No owner model configured" and "an
owner model is configured but did not resolve" are now distinguished:
the first is the single-user install and sees everything, the second
sees nothing. Api::BaseController#owned likewise scopes to none rather
than to where(user_id: nil), which matched every unowned row.
* Owner resolution could not actually succeed. The documented resolver
shape re-enters the engine's own accessor, and recursed until
SystemStackError; a guard degrades that to nil instead. Without it,
failing closed would have emptied the dashboard rather than secured it.
* Provider credentials are filtered from the logs.
Installs that could not run
* Nothing required solid_agent, a transitive dependency that Bundler
installs but never loads, so every Run failed on an uninitialized
constant unless the host happened to list the gem itself. The gem now
requires it, and active_agent, which Compatibility.install! dereferences
at load time.
* has_context was called with an unknown keyword and left to infer bare
AgentContext/AgentMessage/AgentGeneration, which resolve against Object
and so only exist in an app with its own top-level models. Named
explicitly.
* jsonb_array_elements was called on a column the template creates as
json, so metrics and the agents list failed on every PostgreSQL install.
The query casts, which is what existing installs need, and the template
picks jsonb per adapter, which is what new ones need.
* has_one_attached lost its `if defined?(ActiveStorage)` guard in the
extraction, so --skip-active-storage hosts died during eager load.
* The prebuilt bundles were never registered for precompilation, so the
dashboard's only page raised on any sprockets-rails host.
* The install generator aborted on a duplicate migration name for exactly
the installs upgrading from the in-gem dashboard. It now detects what is
already there and emits a guarded top-up for the missing agent_id.
Release
* The gemspec globbed relative to the working directory: building from the
repository root produced an "actionagent" gem containing the framework's
lib and no app at all. It now globs relative to itself, and a rake task
builds it from the right place and asserts the entry point is present.
* Both gems go to 2.0.0. activeagent 1.1.0 is published and this stack
removes ActiveAgent::Dashboard from it, so a ~> 1.1 constraint must not
pick it up; actionagent's floor on activeagent rises to 2.0 so it cannot
resolve against the gem that still contains the old dashboard.
* CHANGELOG covers the split, the upgrade order and the behaviour changes.
Testing
* bin/test now loads the engine's suite as well as the framework's. CI
runs bin/test, so the engine's assertions had never gated a merge. It
also skips the dummy app's tmp artifacts, which the Rakefile already did.
* New coverage for the class-building path that every existing execution
test stopped short of, for an unresolved owner seeing nothing, and for
the endpoints that are no longer anonymous.
1315 runs, 0 failures on rails7 and rails8; rubocop clean.
Both the README and the generated initializer showed owner resolution being wired the one way that cannot work. `controller.send(:current_user)` inside current_user_resolver reaches the engine's own accessor, not the host app's — the dashboard's controllers descend from ActionController::Base, not from the app's ApplicationController — so it re-entered itself; `current_user_method = :current_user` is refused for the same reason and returns nil unconditionally. That was survivable while an unresolved owner saw everything. Now that it sees nothing, following the documented recipe would produce an empty dashboard, so the examples read the session directly and say what an empty dashboard means.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two things happened here, in order.
First, the dashboard engine that shipped in this gem had model classes for agents, runs, versions, templates, sandboxes and recordings — but no migrations to back them. They were dead code. The real implementations lived in the activeagents platform, had drifted from these copies, and were the only ones anyone ran. So the engine became the product: mount it and you get what activeagents.ai gives you.
Then it became clear that could not live inside
activeagent.The dashboard is now its own gem
The framework gem shrank 95%. Size was the visible problem; the dependency graph was the real one:
activeagentdeliberately declares actionpack/actionview/activesupport/activemodel/activejob and not activerecord. The engine's models inheritActiveRecord::Base.lib/active_agent.rbrequired the engine unconditionally in every Rails app, appending eightapp/*paths to every host's autoload and eager-load paths. An API-only app on--skip-active-recordwould have crashed in production on an undeclared dependency — and CI could not have caught it, because the dummy app setseager_load = falseand requiresactive_record/railtieunconditionally.AgentExecutionServicemixes inSolidAgent::HasContext, and solid_agent depends on activeagent. The framework could never declare a dependency its own dashboard code has. That's a cycle. There is no packaging of this code insideactiveagentwhere the graph is true.actionagentdeclares all three honestly. Rails puts data inActive*and the request/response layer inAction*, so the namespace follows the gem:ActiveAgent::Dashboard::Agent→ActionAgent::Agent, mounted asActionAgent::Engine.The engine now sits at a real gem root, so the
find_rootoverride it needed while buried underlib/is gone.What a mounted engine gets you
Agent builder, runs and versions. Conversations (contexts, messages, generations). Evaluations, scorecards and cost estimates. Templates, sandboxes, session recordings, provider and API keys. Your agents as an authenticated MCP server at
<mount>/mcp. Plus the traces and metrics that were already here.The React dashboard ships prebuilt in the gem — mounting it never asks the host to run a JavaScript build, or to have one. Inertia is gone; state arrives as a JSON data attribute.
Seams, not assumptions
Everything the platform's code reached for is configuration now, defaulting to what a single-user self-hosted install wants:
table_name_prefixactive_agent_*tables""— its tables predate the engineowned_by(per model)quota_checker/usage_recorderprovider_credentials_resolverconfig/active_agent.ymlagent_scope_resolvertenant_resolversandbox_backendsexecution_enabledcurrent_user_resolver/current_account_resolverreplace the*_methodsymbols, which could not work: the engine's controllers are their own base class, so a host app'scurrent_userhelper isn't on them. Configuringcurrent_user_method = :current_user— the obvious thing to write — recursed forever.Nothing breaks for existing installs
ActionAgent::CompatibilitykeepsActiveAgent::Dashboard,ActiveAgent::TelemetryTraceandActiveAgent::ProcessTelemetryTracesJobresolving throughconst_missingwith a deprecation. That last one matters beyond tidiness — Active Job serializes the class name into the queue payload, so jobs enqueued before an upgrade are dequeued after it.Runs on more than PostgreSQL
Every PostgreSQL-only query has a portable form and takes the fast path only when the adapter has it:
COUNT(*) FILTER→SUM(CASE …),DISTINCT ON→ a max-id pass,ILIKE→LOWER(…) LIKE,date_trunc→ per-adapter bucketing, jsonb traversal → reading the column in Ruby. JSON columns arejsonbon PostgreSQL andjsonelsewhere. The whole suite runs against the SQLite dummy app.Testing
OPENAI_API_KEYones, same count asmainnode_modulesleaked/api/*404sMCPRecordingMiddlewareinmcp_recording_middleware.rb— resolvable only in an app that had registered anmcpacronymThe server-rendered traces console moves from
<mount>/tracesto<mount>/console/traces.<mount>/tracesis now the React traces view — same data, richer — so old links land somewhere sensible, but the ERB page is no longer there.Review notes
actionagent/app/assets/builds/*is build output, committed on purpose — that's what lets a host app skip the build. Sources are inactionagent/frontend/;node_modulesis excluded from git and the gemspec.Schema[8.1], which the older Rails in the API-gems matrix cannot load (back to[8.0]), and rubocop's array-bracket rule on the same file.