ci: prepare Python Flight dependencies in existing host tests - #457
LeftHandCold merged 1 commit into
Conversation
LeftHandCold
left a comment
There was a problem hiding this comment.
Reviewed 6f7d4ea2fd90e51e63e21fac9b1afc0c73ed95be against main at c76c48c66de9e4684696a02510835a21d4a0aad9. No blocking findings; approve.
Traced all three changed files through the Linux/sharded UT, currently disabled macOS UT, and coverage consumers. The requirements-file guard preserves older checkouts. The action installs the checked-out requirements into a unique runner-owned venv, checks Flight before exporting PATH, and fails the job on setup errors. Both make ut and the coverage subprocess preserve PATH for the Go tests' exec.LookPath("python3") and worker children.
Validation:
- Reused the successful workflow validation check for this head; independently validated the new composite-action metadata schema and shell syntax.
- Executed the action's unchanged shell on Linux amd64 with CPython 3.12.14 and the
pyarrow==24.0.0requirements from matrixorigin/matrixone#29152 at36e831b66dfb658145070de55971c2e12f2a4c08. Installation/import checks passed; bothpythonandpython3resolved to the venv after applying its exported PATH. - The companion PR's real worker started and listened within its Go fixture's five-second readiness budget using that interpreter; the probe reaped its process afterward.
- Checked unavailable-dependency and import-failure controls: both returned nonzero without publishing PATH.
- Ran
python -m unittest discover -s scripts -p 'test_run_coverage_ut.py' -vin that environment: 6 passed.
Scope: this validates the CI dependency setup and its consumers, not the entire Python UDF feature. The local probe did not run actions/setup-python on the actual GitHub runners or the full MO UT/coverage suites. Callers use CI@main, so the companion jobs receive this setup after this PR merges; their next run remains the hosted integration check.
Problem and change
The existing host UT and coverage jobs used by matrixorigin/matrixone#29152 run real Python Flight tests, but do not install the checked-out worker requirements. The Python child exits before listening because pyarrow is unavailable.
Add a shared setup action to the existing UT paths and coverage job. It selects Python 3.12, creates a job-owned virtual environment, installs the checked-out MatrixOne requirements, verifies pyarrow.flight, and exports that interpreter on PATH for Go test subprocesses. Revisions without worker requirements retain their existing path. No job is added, and missing dependencies remain a hard failure.
Validation
This is a delivery dependency of matrixorigin/matrixone#29152. Its reusable workflows reference matrixorigin/CI@main, so the MO host jobs receive this setup only after this change is merged upstream. Hosted CI has not been claimed green.