Repository navigation
test(playground): add e2e suite for execution identity - #630
Draft
MarioCadenas wants to merge 1 commit into
Draft
MarioCadenas wants to merge 1 commit into
MarioCadenas wants to merge 1 commit into
Conversation
Adds an e2e (TesterArmy) suite that checks which principal AppKit runs each call as: the app service principal by default, the signed-in user on the OBO path. A minimal test app is started in production mode and requests carry the Databricks Apps proxy headers. All assertions are deterministic. Covers checklist rows B1, B2, B3, B4, B5, B7 plus the fail-closed and no-deprecation guardrails. Without an SP secret the suite runs in local mode and skips the SP-vs-user comparisons with an explicit reason. Requires the execution-identity stack (#592, #594, #595, #597, #601). Also bumps @playwright/test to 1.63.0 (peer range of @e2edev/web). Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Contributor
🤖 AppKit PR bot🔬 Run evalsStart an eval for this PR from the evals-monitor app: Go to Evals Monitor → 📦 Try this PR's app templateScaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh run download 37440274781 -R databricks/appkit -n appkit-template-0.82.0-pr.cb63e48-tests-e2e-execution-identity-630 -D appkit-pr-630 \
&& unzip -o "appkit-pr-630/appkit-template-0.82.0-pr.cb63e48-tests-e2e-execution-identity-630.zip" -d "appkit-pr-630" \
&& databricks apps init --template "appkit-pr-630"The template pins |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an end-to-end suite (the
e2epackage from TesterArmy) that checks which principal AppKit runs each call as: the app service principal by default, the signed-in user on the on-behalf-of path.Depends on the execution-identity stack (#592, #594, #595, #597, #601). Without it,
appkit.asUser(req)doesn't exist, so this stays a draft until the stack lands.How it works
apps/dev-playground/e2e/identity/app/) runs in production mode. In development mode a missing user token silently runs as the SP, which would hide the fail-closed checks.x-forwarded-access-token,x-forwarded-user,x-forwarded-email).current_user()values, error codes, and tool outputs read from the agent SSE stream. Nothing depends on the model's text.Coverage
B1, B2, B3 (block and one-call forms), B4, B5, B7, plus the fail-closed and no-deprecation guardrails from the execution-identity test checklist. The mapping table is in
e2e/identity/README.md.Results (run against the stack tip
7b451e32merged with main in a scratch worktree, against dogfood)Local mode (your own token, no SP secret): 5 passed, 7 skipped, 0 failed. The 7 skipped tests compare SP vs user identity, which needs a distinct SP, so they skip with an explicit reason instead of passing for nothing. They have not run yet.
Run
The other env vars are listed in the README.
Also in this PR
@playwright/testbumped from 1.61.0 to 1.63.0, plusplaywright1.63.0, to satisfy the peer range of@e2edev/web. dev-playground's existing Playwright integration tests still pass (29/29).e2eand@e2edev/webpinned exact to the matching canary build. npm says@e2edev/webmoved to@e2e-dev/web, but the Databricks npm proxy doesn't serve the new name yet.Not covered
genieorserving, which is where the original regression was.agent.actUI steps are not wired yet (no model provider chosen).This pull request and its description were written by Isaac.