Skip to content

Extract the dashboard into the actionagent gem, with feature parity - #353

Merged
TonsOfFun merged 12 commits into
mainfrom
claude/activeagents-local-engine-ky54n7
Aug 13, 2026
Merged

Extract the dashboard into the actionagent gem, with feature parity#353
TonsOfFun merged 12 commits into
mainfrom
claude/activeagents-local-engine-ky54n7

Conversation

@TonsOfFun

@TonsOfFun TonsOfFun commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Why

Two things happened here, in order.

First, the dashboard engine that shipped in this gem had model classes for agents, runs, versions, templates, sandboxes and recordings — but no migrations to back them. They were dead code. The real implementations lived in the activeagents platform, had drifted from these copies, and were the only ones anyone ran. So the engine became the product: mount it and you get what activeagents.ai gives you.

Then it became clear that could not live inside activeagent.

The dashboard is now its own gem

activeagent   2546.1 KB → 134.5 KB    163 files, zero dashboard
actionagent                 360.0 KB    85 files, zero JSX sources

The framework gem shrank 95%. Size was the visible problem; the dependency graph was the real one:

  • activeagent deliberately declares actionpack/actionview/activesupport/activemodel/activejob and not activerecord. The engine's models inherit ActiveRecord::Base.
  • lib/active_agent.rb required the engine unconditionally in every Rails app, appending eight app/* paths to every host's autoload and eager-load paths. An API-only app on --skip-active-record would have crashed in production on an undeclared dependency — and CI could not have caught it, because the dummy app sets eager_load = false and requires active_record/railtie unconditionally.
  • AgentExecutionService mixes in SolidAgent::HasContext, and solid_agent depends on activeagent. The framework could never declare a dependency its own dashboard code has. That's a cycle. There is no packaging of this code inside activeagent where the graph is true.

actionagent declares all three honestly. Rails puts data in Active* and the request/response layer in Action*, so the namespace follows the gem: ActiveAgent::Dashboard::AgentActionAgent::Agent, mounted as ActionAgent::Engine.

The engine now sits at a real gem root, so the find_root override it needed while buried under lib/ is gone.

What a mounted engine gets you

Agent builder, runs and versions. Conversations (contexts, messages, generations). Evaluations, scorecards and cost estimates. Templates, sandboxes, session recordings, provider and API keys. Your agents as an authenticated MCP server at <mount>/mcp. Plus the traces and metrics that were already here.

The React dashboard ships prebuilt in the gem — mounting it never asks the host to run a JavaScript build, or to have one. Inertia is gone; state arrives as a JSON data attribute.

Seams, not assumptions

Everything the platform's code reached for is configuration now, defaulting to what a single-user self-hosted install wants:

Seam Unset means The platform passes
table_name_prefix active_agent_* tables "" — its tables predate the engine
owned_by (per model) nothing is owned agents per user, keys per account
quota_checker / usage_recorder unmetered plan limits and run counters
provider_credentials_resolver config/active_agent.yml the account's own provider keys
agent_scope_resolver the agents you own every agent the account's members own
tenant_resolver the owner itself a user's account, since traces hang off accounts
sandbox_backends in-memory only Incus, Kubernetes, Cloud Run
execution_enabled agents can run

current_user_resolver / current_account_resolver replace the *_method symbols, which could not work: the engine's controllers are their own base class, so a host app's current_user helper isn't on them. Configuring current_user_method = :current_user — the obvious thing to write — recursed forever.

Nothing breaks for existing installs

ActionAgent::Compatibility keeps ActiveAgent::Dashboard, ActiveAgent::TelemetryTrace and ActiveAgent::ProcessTelemetryTracesJob resolving through const_missing with a deprecation. That last one matters beyond tidiness — Active Job serializes the class name into the queue payload, so jobs enqueued before an upgrade are dequeued after it.

Runs on more than PostgreSQL

Every PostgreSQL-only query has a portable form and takes the fast path only when the adapter has it: COUNT(*) FILTERSUM(CASE …), DISTINCT ON → a max-id pass, ILIKELOWER(…) LIKE, date_trunc → per-adapter bucketing, jsonb traversal → reading the column in Ruby. JSON columns are jsonb on PostgreSQL and json elsewhere. The whole suite runs against the SQLite dummy app.

Testing

  • Engine suite: 42 runs, 0 failures, on SQLite
  • Framework suite: 1264 runs, 0 failures — the 253 errors are the pre-existing missing-OPENAI_API_KEY ones, same count as main
  • Both gems build; contents verified — no JSX sources or node_modules leaked
  • Smoke-tested mounted in the platform against PostgreSQL: sign-in → dashboard renders with fingerprinted engine assets, props carry the mount path and agents, API/metrics/console/client routes respond, unknown /api/* 404s
  • Added an eager-load test, which immediately caught MCPRecordingMiddleware in mcp_recording_middleware.rb — resolvable only in an app that had registered an mcp acronym

⚠️ Breaking

The server-rendered traces console moves from <mount>/traces to <mount>/console/traces. <mount>/traces is now the React traces view — same data, richer — so old links land somewhere sensible, but the ERB page is no longer there.

Review notes

  • The engine's models are the platform's, so the diff reads as new files rather than edits to the drifted copies they replace.
  • actionagent/app/assets/builds/* is build output, committed on purpose — that's what lets a host app skip the build. Sources are in actionagent/frontend/; node_modules is excluded from git and the gemspec.
  • Benchmarks (the ragents harness) deliberately did not come along.
  • Two CI fixes ride along: the dummy schema was regenerated against Rails 8.1 and pinned to Schema[8.1], which the older Rails in the API-gems matrix cannot load (back to [8.0]), and rubocop's array-bracket rule on the same file.

claude added 6 commits August 12, 2026 23:25
…platform's

The dashboard engine shipped model classes for agents, runs, versions,
templates, sandboxes and recordings, but no migrations to back them — the
platform's own copies had grown apart and were the only ones running.

Replace the drifted copies with the platform's implementations, add the
schema they need, and give the engine the seams a host app has to reach:

- table_name_prefix, so the engine's active_agent_* tables and a host app
  that already owns them unprefixed are both supported without renaming
  production tables
- Ownable, resolving the owner association from configuration rather than
  assuming an Account or a User exists
- quota_checker / provider_credentials_resolver / trace_owner_resolver, so
  plan limits, per-tenant LLM credentials and agent attribution stay in the
  app that has that data
- sandbox_backends, replacing the hardcoded Incus/Kubernetes/Cloud Run
  switch with a registry; the engine ships the in-memory backend and the
  operator registers the rest

Conversation persistence (contexts, messages, generations), evaluations,
scorecards and cost estimates come along, so a mounted engine has the same
feature set the hosted product does.

Queries that were PostgreSQL-only now have portable forms: COUNT FILTER
becomes SUM CASE, DISTINCT ON becomes a max-id pass, jsonb traversal falls
back to reading the column in Ruby, and jsonb columns are declared json on
adapters without jsonb. The dashboard runs on SQLite and MySQL again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
Adds the JSON API and the React frontend to the engine, so mounting it
gives you the surface the hosted product has rather than traces and
metrics alone: agents and versions, runs, interactions, evaluations,
templates, sandboxes, recordings, provider and API keys, and the
agents-as-MCP-server endpoint.

The controllers are the platform's, rewritten onto the engine's seams.
Anything they reached for directly — the account, the signed-in user's
agents, plan limits, usage counters, Stripe checkout — now goes through
configuration, so the same controller serves a single-user install, a
per-user install and a multi-tenant platform:

- `owned(relation)` scopes by whatever the model declared with owned_by,
  rather than assuming who the owner is
- quota_denial / record_usage / upgrade_url replace can_run_agent?,
  increment_agent_runs! and the checkout POST
- execution_enabled? leaves a read-only observability dashboard when an
  operator would rather the mount not call providers at all

The React app drops Inertia: initial state arrives as a JSON data
attribute and the bundle ships prebuilt in the gem, so a host app mounts
the dashboard without adopting a frontend framework or running a build.
One fetch shim rewrites /api/... onto the mount path, which is what lets
the engine live at any prefix. Action Cable is optional now — without a
cable mount the views poll instead of failing.

/api/traces keeps its meaning on both verbs: POST is still the SDK's
token-authenticated ingest, GET is the dashboard's session-authenticated
read (Api::TraceReportsController).

Remaining PostgreSQL-only queries got portable forms (ILIKE, DISTINCT ON,
jsonb traversal), and session recordings are scoped by a real owner column
instead of a JSON metadata key.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The install generator now writes the dashboard's tables alongside the
telemetry ones, so `rails generate active_agent:dashboard:install` gets an
app the full surface rather than a dashboard whose models have nothing to
read. --traces_only keeps the old behaviour for an app that only wants to
be a trace sink, and the generator points at db:encryption:init because
API keys and provider credentials are encrypted at rest.

Traces gain agent_id, which is what attributes a reported execution to a
dashboard agent — the platform had grown that column but the generator
never shipped it, so AgentRegistrar had nothing to write to on a
self-hosted install.

Adds coverage for the surface that just moved: agent CRUD and versioning,
case-insensitive search, the quota/usage/execution-toggle seams, the
ownership declarations, the configurable table prefix, and traces and
metrics reading what ingest writes. All of it runs on SQLite, which is the
point — the last PostgreSQL-only query (date_trunc) now has portable forms
for SQLite and MySQL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
…eparate homes

The React app pushed absolute /dashboard/... URLs, which only worked for an
engine mounted at exactly that path — mounted anywhere else, every in-app
navigation left the dashboard. Client routes now resolve against the mount
the server published, so /activeagents, /dashboard and a subdomain root all
behave the same.

That surfaced a collision: the React dashboard and the server-rendered
traces pages both wanted <mount>/traces. The React app takes the mount and
everything under it — a catch-all serves its client routes, and refuses
/api paths so a mistyped endpoint still answers like an API. The
server-rendered console moves to <mount>/console/traces, where it stays
useful when the bundle can't run.

Also fixes seams that only bite a host app with tenants:

- current_user_method = :current_user recursed forever, because the engine's
  controllers are their own base class and the accessor called itself.
  Resolvers take a controller instead, and a named method is only called
  when the controller actually has it.
- current_owner no longer falls back to the signed-in user in multi-tenant
  mode, which had been handing a user with no tenant a scope they were not
  part of.
- Reachable agents go through agent_scope_resolver, since ownership and
  reachability differ when an account's key must reach its members' agents.
- Telemetry resolves through tenant_for, because traces belong to accounts
  while agents may belong to users.
- The executions query no longer reads reported traces in multi-tenant mode
  without a tenant to scope them to.
- A quota denial may answer with a payload, so a host app can surface its
  own usage numbers with the 402.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The self-hosted guide sold interactions, evaluations, scorecards and cost
estimates as reasons to use the hosted platform instead. They ship in the
engine now, so the comparison tables say what actually differs: the
platform adds accounts, plans, billing and managed sandboxes around the
same feature set.

Also documents what changed for anyone mounting it: the full schema the
generator writes and its --traces_only escape hatch, the encryption keys
credentials need, the prebuilt React bundle, the server-rendered console's
move to <mount>/console/traces, running agents and registering a sandbox
backend, and the retention job that replaced "write your own delete_all".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The class was MCPRecordingMiddleware in a file called
mcp_recording_middleware.rb, which only resolves in an app that has
registered an "mcp" acronym with its autoloader. In the platform it did,
which is why this never surfaced there; anywhere else the engine failed to
eager load.

Adds the test that would have caught it: eager loading the engine, which
autoloading a few constants per test never did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
claude added 3 commits August 13, 2026 02:07
Follows the extraction: the dashboard engine is most of this gem now, and
the docs still described it as the traces-and-metrics half.

- AGENTS.md gains a Dashboard Engine section — where the code lives, how the
  namespacing works, what the mount-relative paths are, and that the React
  bundle is committed so host apps never run a build. It is the largest
  subsystem in the repo and a contributor had no map to it.
- The VitePress sidebar matches the pages' actual titles again.
- telemetry.md's endpoint paths match the engine's routes.
- v2-extraction-roadmap.md is re-audited against what shipped: the MCP
  server half is done, "hosted UI" leaves the platform layer, and
  AgentToolbox is marked as landed-but-in-the-wrong-layer, since generic
  execution code is now reachable only through the dashboard engine.
- rails.md's generator output listing matches what the generator writes
  (test/agents/*_test.rb and test/docs/previews/*_preview.rb).

The VitePress site builds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
The dashboard had grown to 77% of the activeagent payload — 932 KB of
prebuilt assets, 641 KB of JSX sources and 384 KB of Ruby against 557 KB of
actual framework. Size was the visible problem; the dependency graph was the
real one.

`activeagent` deliberately depends on actionpack/actionview/activesupport/
activemodel/activejob and NOT on activerecord or rails. But the engine's
models inherit ActiveRecord::Base, lib/active_agent.rb required the engine
unconditionally in every Rails app, and eight app/* paths were appended to
every host's autoload and eager-load paths. An API-only app on
--skip-active-record would have crashed in production on a dependency the
gemspec never declared, and CI could not have caught it: the dummy app sets
eager_load = false and requires active_record/railtie unconditionally.

Worse, AgentExecutionService mixes in SolidAgent::HasContext, and solid_agent
depends on activeagent. The framework could never declare a dependency its
own dashboard code has — that is a cycle. There is no packaging of this code
inside activeagent where the dependency graph is true.

So the dashboard becomes `actionagent`, a sibling gem in this repo with its
own gemspec, declaring activerecord, railties and solid_agent honestly.
Rails names the data layer Active* and the request/response layer Action*,
and a mountable dashboard is squarely the latter — so the namespace follows
the gem: ActiveAgent::Dashboard::Agent is ActionAgent::Agent, and the engine
mounts as ActionAgent::Engine.

    activeagent  2546.1 KB -> 134.5 KB   (163 files, no dashboard)
    actionagent                360.0 KB  (85 files, no JSX sources)

The engine now sits at a real gem root, so the find_root override it needed
while buried under lib/ is gone, and Rails locates app/ and config/routes.rb
by itself.

Nothing breaks for existing installs: ActionAgent::Compatibility keeps
ActiveAgent::Dashboard, ActiveAgent::TelemetryTrace and
ActiveAgent::ProcessTelemetryTracesJob resolving through const_missing with
a deprecation. That last one matters beyond tidiness — Active Job serializes
the class name into the queue payload, so jobs enqueued before an upgrade are
dequeued after it.

Two defects found along the way and fixed here: activeagent never declared
railties despite requiring "rails", and the frontend JSX sources were
shipping in the gem for no reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
Renames the namespace throughout the guides and, where a find-and-replace
could not help, rewrites the prose: the install story is two gems now, the
dev console page no longer opens by claiming the framework ships a
dashboard, and configuration.md documents ActionAgent.configure, which it
had never mentioned because there was nothing separate to mention.

v2-extraction-roadmap.md gains a fourth layer between solid_agent and the
platform, and its AgentToolbox item is re-audited: the code left activeagent
with the dashboard, so it is further from the framework than when the item
was written, and an app now installs the whole engine to get a safe
fetch_url.

Corrects a claim I had made in three places, including the actionagent
gemspec: railties is not a dashboard-only dependency. activeagent declares
it too — its railtie requires "rails". Active Record is the dependency that
actually distinguishes the two gems.

CI fixes:
- rubocop: the regenerated dummy schema violated the array-bracket rule.
- Test API Gems: the dummy schema was regenerated locally against Rails 8.1
  and pinned to ActiveRecord::Schema[8.1], which the older Rails in that
  matrix cannot load. Back to [8.0], the version the file already declared.
- the dummy's dashboard migration still called ActiveAgent::Dashboard.

The VitePress site builds and rubocop is clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
@TonsOfFun TonsOfFun changed the title Make the dashboard engine the whole dashboard, mountable in any Rails app Extract the dashboard into the actionagent gem, with feature parity Aug 13, 2026
The gemspec globbed both, but neither file existed under actionagent/, so
the packaged gem carried no license text and its RubyGems page would have
been blank — sloppy for a gem released on its own.

The README documents the install, the generator flags, the configuration
seams and the deprecated ActiveAgent::Dashboard constants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fsu5fid9JgX97zFNkkQVNz
@TonsOfFun
TonsOfFun marked this pull request as ready for review August 13, 2026 16:47
@TonsOfFun
TonsOfFun requested a lite review from Copilot August 13, 2026 16:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

claude added 2 commits August 13, 2026 22:27
A review of this stack turned up defects that CI could not see, because the
engine's own 77 tests were never in what CI runs and no host-shaped install
is exercised anywhere. Fixed here, on the branch that introduced them.

Authentication and tenant isolation

  * Api::SandboxesController opted out of authentication entirely for an
    anonymous free tier. That made #run and #compare an open proxy: any
    caller could execute arbitrary prompts against the host app's provider
    credentials, and #show read any session back by id. Opting out also
    skipped the callback that refuses to serve an unauthenticated dashboard
    outside development, so it defeated that safeguard too. The controller
    now authenticates, honours the execution kill switch and the quota seam,
    and scopes its lookups through the ownership seam.
  * Session recordings carry a replayable timeline of a real browser session,
    including values typed into forms. start_user_session, record_action,
    complete_session and demo were all anonymous. GET .../demo fell back to
    the oldest completed recording when no seeded demo row existed, so an
    install with no demo published a real customer session to anyone who
    found the mount. The demo endpoint is removed and the rest authenticate.
    Reads and writes go through the controller's existing ownership check,
    which set_recording never called — every recording was readable by id.
  * Templates#show looked a template up by bare id while anonymous, serving
    unpublished drafts and private prompt libraries. No exemption now.
  * for_owner(nil) returned every row. "No owner model configured" and "an
    owner model is configured but did not resolve" are now distinguished:
    the first is the single-user install and sees everything, the second
    sees nothing. Api::BaseController#owned likewise scopes to none rather
    than to where(user_id: nil), which matched every unowned row.
  * Owner resolution could not actually succeed. The documented resolver
    shape re-enters the engine's own accessor, and recursed until
    SystemStackError; a guard degrades that to nil instead. Without it,
    failing closed would have emptied the dashboard rather than secured it.
  * Provider credentials are filtered from the logs.

Installs that could not run

  * Nothing required solid_agent, a transitive dependency that Bundler
    installs but never loads, so every Run failed on an uninitialized
    constant unless the host happened to list the gem itself. The gem now
    requires it, and active_agent, which Compatibility.install! dereferences
    at load time.
  * has_context was called with an unknown keyword and left to infer bare
    AgentContext/AgentMessage/AgentGeneration, which resolve against Object
    and so only exist in an app with its own top-level models. Named
    explicitly.
  * jsonb_array_elements was called on a column the template creates as
    json, so metrics and the agents list failed on every PostgreSQL install.
    The query casts, which is what existing installs need, and the template
    picks jsonb per adapter, which is what new ones need.
  * has_one_attached lost its `if defined?(ActiveStorage)` guard in the
    extraction, so --skip-active-storage hosts died during eager load.
  * The prebuilt bundles were never registered for precompilation, so the
    dashboard's only page raised on any sprockets-rails host.
  * The install generator aborted on a duplicate migration name for exactly
    the installs upgrading from the in-gem dashboard. It now detects what is
    already there and emits a guarded top-up for the missing agent_id.

Release

  * The gemspec globbed relative to the working directory: building from the
    repository root produced an "actionagent" gem containing the framework's
    lib and no app at all. It now globs relative to itself, and a rake task
    builds it from the right place and asserts the entry point is present.
  * Both gems go to 2.0.0. activeagent 1.1.0 is published and this stack
    removes ActiveAgent::Dashboard from it, so a ~> 1.1 constraint must not
    pick it up; actionagent's floor on activeagent rises to 2.0 so it cannot
    resolve against the gem that still contains the old dashboard.
  * CHANGELOG covers the split, the upgrade order and the behaviour changes.

Testing

  * bin/test now loads the engine's suite as well as the framework's. CI
    runs bin/test, so the engine's assertions had never gated a merge. It
    also skips the dummy app's tmp artifacts, which the Rakefile already did.
  * New coverage for the class-building path that every existing execution
    test stopped short of, for an unresolved owner seeing nothing, and for
    the endpoints that are no longer anonymous.

1315 runs, 0 failures on rails7 and rails8; rubocop clean.
Both the README and the generated initializer showed owner resolution being
wired the one way that cannot work. `controller.send(:current_user)` inside
current_user_resolver reaches the engine's own accessor, not the host app's —
the dashboard's controllers descend from ActionController::Base, not from the
app's ApplicationController — so it re-entered itself; `current_user_method =
:current_user` is refused for the same reason and returns nil unconditionally.

That was survivable while an unresolved owner saw everything. Now that it
sees nothing, following the documented recipe would produce an empty
dashboard, so the examples read the session directly and say what an empty
dashboard means.
@TonsOfFun
TonsOfFun merged commit 8e808ee into main Aug 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants