Repository navigation
test(e2e): port network, L7 and policy-update tests from e2e/rust to conformance - #5665
Draft
matthewgrossman wants to merge 19 commits into
Conversation
…olicy set helpers Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…vers to host-server Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…ewrite tests Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
🌿 Preview your docs: https://nvidia-preview-pr-5665.docs.buildwithfern.com/openshell |
Member
Author
|
/ok to test 858dbc6 |
|
Label |
This was referenced Nov 18, 2026
… tests These tests now run in the conformance suite and the routable-host-ip feature package. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…suites Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
…er limit The gateway rejects sandbox names longer than 19 characters, which leaves 5 for a suffix after test-<run-id>-. The static credential binding test also needs the Python workload image, which has the sandbox user that the base policy runs as. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Member
Author
|
/ok to test 45fd589 |
matthewgrossman
force-pushed
the
test/5655-port-network-l7-policy/matthewgrossman
branch
from
November 18, 2026 05:10
858dbc6 to
45fd589
Compare
This was referenced Nov 18, 2026
…matthewgrossman' into test/5655-port-network-l7-policy/matthewgrossman Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> # Conflicts: # tests/testkit/README.md # tests/testkit/testkit/src/policy.rs
matthewgrossman
changed the base branch from
test/5426-port-cli-attach-exec/matthewgrossman
to
test/5689-share-sandbox-helpers/matthewgrossman
November 18, 2026 06:23
Drops the include of a conformance client script from another workspace. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Replace conformance's support module and generic client scripts with testkit's run_python, last_json, policy readers, providers and binaries_yaml, and the connect, http and http_raw probes. Protocol and specialized clients move to the sandbox-clients scripts module. The pipelined request tests now count every response in the reply; the 403 follows the first response's body with no line break, so the old line-based count missed it. The policy update tests use the Python image, which has the sandbox user the base policy runs as. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Member
Author
|
/ok to test 251835b |
4 of 6 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port the network, L7, policy-update, and port-forward tests from
e2e/rustto testkit, and delete the legacy files. Most tests move into the conformance suite. The IP-literal egress test moves to a newroutable-host-ipfeature package.Stack:
main← #5442 ← #5691 ← this PR ← #5676. The base istest/5689-share-sandbox-helpers/matthewgrossman(#5691), merged in with a normal merge. I'll retarget as the PRs below this one merge.Related Issue
Closes #5655
Part of #5281
GitHub links #5655 only after this PR is retargeted to
main, because closing keywords apply only to PRs into the default branch.Changes
Context::compute_driver,SandboxBuilder::driver_config,Sandbox::logs/logs_until,policy::set,policy::from_yaml, andpolicy::base_with.TcpServer,free_port, and a loopbackexchangeclient.Sandbox::run_python,ExecOutput::last_json,Sandbox::policy_revisions/active_policy,SandboxBuilder::providersandpolicy::binaries_yaml, plushost_server_support::TEST_HOST. Conformance no longer has asupportmodule.sandbox_clients_support::{connect, http, http_raw}calls. Credential tests read the provider placeholder withprintenvand send it withhttp.tests/support/sandbox-clients/scripts: the GraphQL, JSON-RPC, JSON-RPC audit, MCP, localhost, listener lifetime, preconnect owner, socket inheritance, reload keep-alive, echo server, WebSocket text and binary, and bound credential (bash) clients, exposed throughsandbox_clients_support::scripts.routable-host-ipno longer includes a conformance script from another workspace.network: bypass denial, reserved destinations, localhost without proxy env, listener lifetime.network_l7+network_l7/mcp: REST, GraphQL, JSON-RPC, and MCP.network_egress: reload, ambiguous update,tls: skip, middleware, pipelining.policy_updateandport_forward.credential_gatingadditions: header and body rewrite, WebSocket text rewrite, and static credentials bound to profile endpoints.tests/suites/features/routable-host-ip: the IP-literal endpoint test (explicitallowed_ipsand implicit). CI rows: Docker rootful (deb), Podman rootful (rpm),k3s, andk3s-ha.live_policy_updateis removed frome2e:rust:push. That lane already runsopenshell-conformanceagainst a push-mode gateway, so it runs thepolicy_updatetests instead. The tmachineconfig-pushenvironment is left to test(conformance): run the conformance suite against a gateway that pushes configuration #5622.static_provider_credentials_are_bound_to_profile_endpointsand its helpers fromhost_gateway_alias.rs. The other two tests in that file stay until PR5.[[test]]entries,PODMAN_CI_TESTSentries, andartifacts.nixfollow-up binary.e2e:websocket-conformancetask and the now-unusedSandboxGuard::spawn_forward.TESTING.md(routable-host-ip), and a note onContext::namethat sandbox suffixes can have at most 5 characters (test(testkit): share sandbox client helpers across testkit suites #5691'sMAX_SANDBOX_NAME_LENcheck enforces it).Defaults applied
host.openshell.internal. The legacy private-rangeallowed_ipsentries are gone.Sandbox::execinsleep infinitysandboxes, using pipes. The legacy tests were also non-TTY, so PTY/pipe parity is unchanged.exec_untilhelper;Sandbox::poll_execalready covers it.policy::base(),python_image(), and the shared Python binaries list.live_policy_updateround-trip and from-empty tests are merged intopolicy_set_versions_and_ignores_repeats.Deviations from the plan
port_forwardis its own module, not part ofcli.rs, to avoid conflicts with test(e2e): port CLI exec, attach, and SSH host identity tests to testkit #5442. It startsforward start -dwithcli_spawnand waits for the exit code, because the backgroundedsshkeeps the CLI's pipes open. A cleanup hook runsforward stop.credential_gating.rs. The unbound endpoint is a second port on the test host instead ofhost.docker.internal, so the 403 holds on every driver. The legacy Podmanconnect-deniedspecial case is gone.Rows that kept a container fixture
None. The IP-literal test needs a host address that the supervisor allows as an IP literal. Rootless Podman's address is link-local, which is always blocked. So instead of keeping a container fixture, that test is a feature package that does not run on rootless Podman.
Driver coverage changes
bypass_detection,forward_proxy_*,host_gateway_alias,live_policy_update, andno_proxy. It now runs all of the ported conformance tests, on rootful and rootless.host_gateway_aliasfrom this set. It now runs all of them throughopenshell-conformanceine2e-vm.sh. That is new coverage. If a newly covered row fails, I'll raise it here before skipping anything.Testing
cargo fmt --checkandcargo clippy --locked --all-targets -D warningsfortests/testkit,tests/support,tests/suites/conformance,tests/suites/featurescargo testfortests/testkitandtests/support(new host-server unit tests included)tasks/scripts/check-conformance-program.sh(71 tests)cargo check --testsfore2e/rustwith docker, podman, kubernetes, vm, and config-push features.cargo clippyonhost_gateway_aliasis clean.Checklist