Skip to content

ci: prepare Python Flight dependencies in existing host tests - #457

Merged
LeftHandCold merged 1 commit into
matrixorigin:mainfrom
iamlinjunhong:fix/python-udf-test-prerequisites
Sep 23, 2026
Merged

LeftHandCold merged 1 commit into
matrixorigin:mainfrom
iamlinjunhong:fix/python-udf-test-prerequisites

Conversation

@iamlinjunhong

Copy link
Copy Markdown
Contributor

Problem and change

The existing host UT and coverage jobs used by matrixorigin/matrixone#29152 run real Python Flight tests, but do not install the checked-out worker requirements. The Python child exits before listening because pyarrow is unavailable.

Add a shared setup action to the existing UT paths and coverage job. It selects Python 3.12, creates a job-owned virtual environment, installs the checked-out MatrixOne requirements, verifies pyarrow.flight, and exports that interpreter on PATH for Go test subprocesses. Revisions without worker requirements retain their existing path. No job is added, and missing dependencies remain a hard failure.

Validation

  • actionlint passed for the action and workflow changes.
  • Executed the setup shell using Python 3.12 and the checked-out PyArrow 24.0.0 requirements; the interpreter/import checks passed.
  • Existing coverage helper tests: 6 passed under that Python environment.
  • MatrixOne real-worker Go integration and race tests passed with the same interpreter selection on macOS arm64. Linux validation is recorded in the linked MO PR.

This is a delivery dependency of matrixorigin/matrixone#29152. Its reusable workflows reference matrixorigin/CI@main, so the MO host jobs receive this setup only after this change is merged upstream. Hosted CI has not been claimed green.

@LeftHandCold LeftHandCold left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 6f7d4ea2fd90e51e63e21fac9b1afc0c73ed95be against main at c76c48c66de9e4684696a02510835a21d4a0aad9. No blocking findings; approve.

Traced all three changed files through the Linux/sharded UT, currently disabled macOS UT, and coverage consumers. The requirements-file guard preserves older checkouts. The action installs the checked-out requirements into a unique runner-owned venv, checks Flight before exporting PATH, and fails the job on setup errors. Both make ut and the coverage subprocess preserve PATH for the Go tests' exec.LookPath("python3") and worker children.

Validation:

  • Reused the successful workflow validation check for this head; independently validated the new composite-action metadata schema and shell syntax.
  • Executed the action's unchanged shell on Linux amd64 with CPython 3.12.14 and the pyarrow==24.0.0 requirements from matrixorigin/matrixone#29152 at 36e831b66dfb658145070de55971c2e12f2a4c08. Installation/import checks passed; both python and python3 resolved to the venv after applying its exported PATH.
  • The companion PR's real worker started and listened within its Go fixture's five-second readiness budget using that interpreter; the probe reaped its process afterward.
  • Checked unavailable-dependency and import-failure controls: both returned nonzero without publishing PATH.
  • Ran python -m unittest discover -s scripts -p 'test_run_coverage_ut.py' -v in that environment: 6 passed.

Scope: this validates the CI dependency setup and its consumers, not the entire Python UDF feature. The local probe did not run actions/setup-python on the actual GitHub runners or the full MO UT/coverage suites. Callers use CI@main, so the companion jobs receive this setup after this PR merges; their next run remains the hosted integration check.

@LeftHandCold
LeftHandCold merged commit c1a30c4 into matrixorigin:main Sep 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants