Skip to content

test(e2e): port Docker and Podman driver feature tests from e2e/rust #5676

Open

Description

@matthewgrossman

User Story

As a maintainer retiring e2e/rust, I want the tests that need a container engine on testkit, running on both the Docker and Podman rows, so container-driver behavior stays covered and the legacy files can be deleted.

Description

Most of these tests were gated to one engine in e2e/rust, but the behavior they check is shared by Docker and Podman. Each new feature package runs on the Docker and Podman rows unless noted.

Legacy test Action Destination Notes
ocsf_console::network_events_are_json_on_supervisor_stderr_only port tests/suites/features/ocsf Supervisor stderr isn't reachable through the public API, so not conformance. Also validate stderr records against the OCSF schema.
ocsf_jsonl_conformance::supervisor_jsonl_conforms_to_the_native_schema merge tests/suites/features/ocsf Collect OCSF-JSON records from supervisor stderr through engine logs. Keep the class, attribution, schema, version, opt-in and clean-stdout checks. Use an in-process host server instead of a container fixture.
↳ its /var/log file-sink check drop — Same formatter as stderr; the file sink is best-effort per the docs. Needs approval in #5297.
driver_config_volume::sandbox_mounts_existing_driver_config_volume port tests/suites/features/driver-config Create the volume with the engine CLI, labeled for resource admission, so the test works with admission enabled.
driver_config_volume::oci_workspace_preparation_skips_nested_volume_ownership port tests/suites/features/driver-config Build from testkit::base_image() without apt, so no Internet. Runs on Podman too.
driver_config_volume::sandbox_mounts_enabled_driver_config_bind port tests/suites/features/driver-config
local_driver_token_restart::local_driver_restart_or_stop_respects_main_process_lifecycle split tests/suites/drivers/{docker,podman} Container part only, no gateway restart. The VM part joins PR6's gateway-restart test.
podman_resource_limits::sandbox_resource_limits_are_enforced_via_cgroups port tests/suites/features/resource-limits Not Podman-specific: Docker, Podman and Kubernetes advertise and enforce CPU and memory limits. First assert the gateway reports limit support, then read the cgroup values.
host_gateway_alias::sandbox_receives_eof_after_closing_http_response port tests/suites/features/host-gateway (response_eof.rs) Three cases in one sandbox.
host_gateway_alias::sandbox_reaches_host_openshell_internal_via_host_gateway_alias drop — features/host-gateway checks reachability more strictly on 5 rows. Needs approval in #5297.
podman_host_gateway::podman_sandbox_reaches_host_openshell_internal split tests/suites/features/host-gateway (host_server_reachable_after_stop_start) Keeps the stop/start re-check on every host-gateway row. Drops the reachability half and the hard-coded nameserver 127.0.0.53 check (the policy-DNS relay is the same on every driver; the transparent-TCP tests prove it works). Needs approval in #5297.
transparent_tcp::local_container_native_tcp_uses_policy_dns_and_fails_closed port tests/suites/features/transparent-tcp Transparent TCP isn't offered on Kubernetes or VM, so not conformance.
transparent_tcp::rootless_podman_private_dns_pins_do_not_cross_binary_rules port tests/suites/features/transparent-tcp On Docker the fixture uses host.openshell.internal, since the supervisor can't resolve network aliases there.
transparent_tcp::rootless_podman_musl_getaddrinfo_uses_udp_policy_dns drop — Runs in no CI lane and needs a musl probe nobody builds (#3009 closed undone). Needs approval in #5297. If kept, rename it musl_getaddrinfo_uses_udp_policy_dns, build the probe as a musl binary in tests/support, and file a successor to #3009.

This PR also adds the "no driver resources left" half of the ephemeral cleanup test that PR1 split off: one test per driver under tests/suites/drivers/{docker,podman}. Kubernetes and VM get theirs in PR8 and PR9.

Helpers this PR adds

  • tests/support/container-engine: running, wait_running, split stdout/stderr logs, copy_from, build_image_from_dir. Uses the existing OPENSHELL_TEST_CONTAINER_ENGINE.
  • A driver resource probe ("nothing left for this sandbox").
  • testkit: a per-test workspace, gateway info, OCSF record parsing.
  • Environment: a driver-config-enabled gateway variant, bind-mount source paths, and the container network name.

Files deleted

driver_config_volume.rs, host_gateway_alias.rs, local_driver_token_restart.rs, ocsf_console.rs, ocsf_jsonl_conformance.rs, podman_host_gateway.rs, podman_resource_limits.rs, transparent_tcp.rs.

Defaults (change in review if needed)

  • features/resource-limits adds the k3s row only after a check that k3s enforces CPU and memory limits. File an issue that the VM driver ignores limits.
  • ocsf_console and ocsf_jsonl share one features/ocsf package.

Context

Definition of Done

  • Every test in the table is ported, merged, split or dropped as listed, with the drops approved in test(e2e): decide which e2e/rust tests to port and in what order #5297.
  • The listed files are deleted, and every reference to them (PODMAN_CI_TESTS, OPENSHELL_E2E_DOCKER_TEST, podmanE2eFollowUpBinaries, wrapper scripts, nextest groups) is removed.
  • If this PR merges after PR6, it deletes the host Podman lane as described in PR6's Definition of Done (ci(e2e): delete the host Podman lane after #5297 PR6 #5423).
  • Each new feature package has Docker and Podman rows in .github/workflows/e2e-suites.yml; tests that ran only on one engine now run on both.
  • The PR body lists coverage that is new on a driver, so a red first run isn't read as a regression.

Part of #5281. Planned in #5297 (PR5). Stacked on #5665 (PR2).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

topic:testingUnit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions