Skip to content

test(e2e): port network, L7 and policy-update tests from e2e/rust to conformance #5655

Open

Description

@matthewgrossman

User Story

As a maintainer retiring e2e/rust, I want the network enforcement, L7 and live policy-update tests on testkit, so every driver proves the same egress contract and the legacy files can be deleted.

Description

All destinations are under tests/suites/conformance/src/. Merges keep every assertion of the tests they combine.

Legacy test Action Destination Notes
proxy_egress_pipeline::transparent_destination_denials_fail_connect_with_eacces port network.rs Absorbs bypass_detection (next row).
bypass_detection::bypass_attempt_is_rejected_fast merge network.rs (direct_connect_denials) One sandbox: unlisted target, metadata, control plane, outside allowed_ips; assert EACCES/EPERM within 8 s each.
listener_lifetime::closing_or_exiting_listener_refuses_connections_and_releases_port port network.rs
listener_lifetime::unrelated_exec_does_not_inherit_broker_preconnect_socket port network.rs Main command via SandboxBuilder::command.
no_proxy::sandbox_reaches_localhost_without_proxy_environment port network.rs Next to the listener tests.
forward_proxy_l7_bypass::forward_proxy_{allows,denies}_l7_*_request (2 tests) merge network_l7.rs (rest_l7_allows_get_denies_post) One fixture, one sandbox.
forward_proxy_l7_bypass::rest_upgrade_is_refused_in_enforce_and_audit_modes port network_l7.rs One sandbox, two endpoints (enforce and audit).
forward_proxy_graphql_l7::graphql_l7_enforces_high_level_and_raw_transparent_paths port network_l7.rs Drop "forward" from the name.
forward_proxy_jsonrpc_l7::jsonrpc_l7_enforces_high_level_and_raw_transparent_paths port network_l7.rs Also hosts the audit case below.
forward_proxy_jsonrpc_l7::jsonrpc_forward_proxy_hard_denies_response_frames_in_default_audit_mode merge network_l7.rs Second endpoint in the JSON-RPC test's sandbox.
mcp_sessionless::legacy_and_multi_version_profiles_authorize_tools_through_sandbox split network_l7/mcp.rs Three tests, one per protocol version set (each needs its own policy).
mcp_sessionless::sessionless_discovery_tools_and_subscription_use_request_metadata merge network_l7/mcp.rs Its SSE subscription step joins the newest-version test.
proxy_egress_pipeline::policy_reload_updates_transparent_requests_and_closes_existing_http_stream port network_egress.rs
proxy_egress_pipeline::ambiguous_policy_update_is_rejected_without_replacing_active_policy port network_egress.rs Compare revisions with policy list -o json, not text.
proxy_egress_pipeline::explicit_allowed_ips_and_implicit_ip_literals_succeed_transparently port network_egress.rs Test-host IP on two ports instead of two containers.
proxy_egress_pipeline:: TLS skip relay, middleware redaction, fail-closed and fail-open middleware, first-request overflow, chunked pipeline (6 tests) port network_egress.rs Port as-is.
proxy_egress_pipeline::http_credentials_are_rewritten_in_transparent_headers_and_bodies port credential_gating.rs Joins the credential_gating module from #5318.
websocket_conformance::websocket_text_placeholder_is_rewritten_transparently port credential_gating.rs Reuses the WebSocket probe from #5318.
host_gateway_alias::static_provider_credentials_are_bound_to_profile_endpoints port credential_binding.rs Unique provider and profile names per run; replace host.docker.internal with an unbound host so the 403 check holds on every driver.
port_forward::port_forward_echo port cli.rs forward start -d plus a cleanup hook that runs forward stop.
live_policy_update::l7_append_target_scope_round_trip port policy_update.rs Already uses JSON output.
live_policy_update::live_policy_update_round_trip + live_policy_update_from_empty_network_policies merge policy_update.rs Start from empty policies, then A, then B; read versions from policy get/list -o json; assert no revision stays Pending.
live_policy_update::initial_sparse_policy_is_acknowledged_as_loaded port policy_update.rs Poll policy list -o json statuses; use a unique sandbox name.

Helpers this PR adds

  • testkit: compute_driver() (refactor of expect_compute_driver), Sandbox::logs / logs_until, exec_until, sandbox_driver_config.
  • tests/support/host-server: request handler with Request and chunked bodies, TcpServer, a WebSocket text helper, free_port.
  • testkit policy: policy::set, policy::from_yaml.

Files deleted

bypass_detection.rs, forward_proxy_graphql_l7.rs, forward_proxy_jsonrpc_l7.rs, forward_proxy_l7_bypass.rs, listener_lifetime.rs, live_policy_update.rs, mcp_sessionless.rs, no_proxy.rs, port_forward.rs, proxy_egress_pipeline.rs, websocket_conformance.rs.

host_gateway_alias.rs stays until PR5 ports its other two tests.

Defaults (change in review if needed)

  • L7 fixtures become in-process Rust servers reached at host.openshell.internal, and the legacy private allowed_ips ranges go away. If a row can't reach the test host, that test keeps a container fixture on that row.
  • The policy-reload keep-alive client runs with nohup … & through Sandbox::exec; if it doesn't survive, it runs as the sandbox's main command. No new testkit API.

Known risks

  • On Podman the host address may be link-local and blocked before allowed_ips applies. If so, the IP-literal test keeps a container fixture.
  • Moving the L7 bypass tests to host.openshell.internal assumes every conformance row can reach the test host. Not yet verified on k3s.

Context

  • Wave 1, after test(conformance): port the credential gating e2e tests #5318 merges (it adds the policy helpers, install_provider, HttpRecorder and the credential_gating module this PR builds on).
  • PR1 also edits testkit's sandbox context; whichever lands second rebases.
  • PR5, PR6 and PR9 use the helpers this PR adds.

Definition of Done

  • Every test in the table is ported or merged as listed.
  • The listed files are deleted, and every reference to them in wrapper scripts, tasks/*.toml (including e2e:websocket-conformance) and nextest config is removed.
  • Each ported test runs on at least the drivers its legacy lane covered.
  • The PR body states which defaults above were applied and any row that kept a container fixture.

Part of #5281. Planned in #5297 (PR2).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

topic:testingUnit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions