Skip to content

test(e2e): port provider tests from e2e/rust to testkit - #5657

Draft
matthewgrossman wants to merge 8 commits into
mainfrom
test/5656-port-provider-tests/matthewgrossman
Draft

matthewgrossman wants to merge 8 commits into
mainfrom
test/5656-port-provider-tests/matthewgrossman

Conversation

@matthewgrossman

@matthewgrossman matthewgrossman commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Port the provider tests from e2e/rust to testkit and delete the legacy files. The provider files and auto-create tests join the conformance suite. The service bearer passthrough test moves to a new features/service-exposure suite.

Related Issue

Closes #5656
Part of #5281

Changes

  • testkit: create_provider and install_provider return a ProviderBuilder. Chain .config(key, value) to pass --config; existing calls are unchanged.
  • testkit: cx.service_get(url, headers) requests an exposed sandbox service through the gateway endpoint, with the service URL's authority as Host. It uses the gateway's mTLS bundle when the gateway has one, so tmachine environments need no extra routing.
  • testkit: testkit::Profile and testkit::Credential build provider profiles in Rust instead of format! YAML. Setters are named after the profile YAML fields, endpoints use policy::Endpoint (which gains enforce() and path()), and profile.to_yaml() renders through openshell-providers, the CLI's export path. Unit tests compare built profiles with the YAML they name. providers.rs and features/keycloak-provider-refresh use it; credential_gating.rs is converted in test(e2e): port network, L7 and policy-update tests from e2e/rust to conformance #5665.
  • conformance: providers.rs ports provider_file_open_update_and_detach and auto_created_provider_credential_available_in_sandbox.
  • features: new service-exposure package with bearer_passthrough_preserves_authorization_header, plus rows in .github/workflows/e2e-suites.yml.
  • Deleted provider_files.rs, provider_auto_create.rs, service_bearer_passthrough.rs, their [[test]] entries, the provider_auto_create entry in tests/tmachine/artifacts.nix, and the now-unused rustls and tokio-rustls e2e dependencies.

Differences from the legacy tests

  • Wider coverage. The provider tests run on every conformance row. Legacy provider_files ran on Docker only, and tmachine lanes skipped provider_auto_create. Service exposure runs on deb docker, rpm podman (rootful and rootless), k3s, and k3s-ha. Legacy ran it on Docker only.
  • Auto-create uses a run-unique profile. The legacy test relied on the harness importing every providers/*.yaml globally and used claude-code. The port imports its own run-unique profile and passes the credential through the CLI environment. That removes the global import that deferred this move in test(conformance): migrate workspace, upload, and template e2e tests #4246.
  • No provider-delete cleanup poll. The issue planned one. testkit already deletes sandboxes before providers, so the test doesn't need it.
  • The provider files test keeps the legacy path, /run/openshell/providers/<provider>/client.toml.

Testing

  • cargo fmt --check and cargo clippy -D warnings for testkit, conformance, features, and e2e/rust
  • testkit unit tests, including new service tests
  • E2E tests added/updated
  • E2E: every conformance row, all five features/service-exposure rows, all three features/keycloak-provider-refresh rows, and the legacy docker and podman lanes pass in Branch E2E Checks.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

create_provider and install_provider return a builder that creates the
provider when awaited, so tests can chain .config(key, value) for
provider create --config. Existing callers are unchanged.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Move provider_files and provider_auto_create from e2e/rust into a
providers conformance suite. The auto-create test imports a run-unique
profile instead of relying on a globally imported claude-code profile,
so it no longer changes shared gateway state.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Requests go to the gateway's endpoint with the service URL's host as the
Host header, which is how the gateway routes service traffic. Over TLS
the client trusts the gateway CA and presents the registered gateway's
client certificate, through openshell-crypto's rustls configuration.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
… suite

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the state:accepted A maintainer decided OpenShell should pursue this issue label Nov 17, 2026
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test b70615f

@matthewgrossman matthewgrossman added the test:e2e Requires end-to-end coverage label Nov 17, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for b70615f. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…mage

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test c7af9e2

Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
@matthewgrossman

Copy link
Copy Markdown
Member Author

/ok to test b9d5b3e

matthewgrossman added a commit that referenced this pull request Nov 18, 2026
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>

#5657 makes the same install_provider change in that file along with its typed profile, so this branch leaves it unchanged to avoid a conflict.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:accepted A maintainer decided OpenShell should pursue this issue test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(e2e): port provider tests from e2e/rust to testkit

1 participant