Skip to content

test(workflow-operator): share the sklearn spec template - #9202

Open
aglinxinyuan wants to merge 2 commits into
apache:mainfrom
aglinxinyuan:test/sklearn-spec-behaviors
Open

aglinxinyuan wants to merge 2 commits into
apache:mainfrom
aglinxinyuan:test/sklearn-spec-behaviors

Conversation

@aglinxinyuan

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The 51 per-model sklearn descriptor specs (25 classifiers, 26 training-only descriptors) each repeated the same five tests. A new test trait, SklearnModelOpDescBehaviors, registers them. Each spec now makes one sklearnClassifier(...) or sklearnTrainer(...) call with its own model name and sklearn import written out as literals.

Surface Pinned
operatorInfo name, description, group, input port names, one blocking output
config defaults countVectorizer, tfidfTransformer, target, text
getOutputSchemas model_name STRING and model BINARY on the declared output port
generatePythonCode the estimator import, make_pipeline, the model name
JSON the operatorType discriminator and a round-trip through LogicalOp

Test names are unchanged (<Descriptor>[.<member>] should ...). The seven specs with extra tests (the naive Bayes family and the decision tree) keep them. Expected values stay literal in each spec, so a renamed model or a changed import still fails that model's own spec. sklearnClassifier only accepts a SklearnClassifierOpDesc and sklearnTrainer only a SklearnTrainingOpDesc, so a descriptor cannot be checked against the wrong family's ports and group.

Any related issues, documentation, discussions?

Closes #9201.

How was this PR tested?

  • sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.sklearn.*": 58 suites, 431 tests pass. These are the same 431 test names as on main, compared from the JUnit XML reports before and after.
  • Mutation check: a wrong model name in SklearnBaggingOpDescSpec fails its operatorInfo and generatePythonCode tests, and a wrong import in SklearnTrainingKNNOpDescSpec fails its generatePythonCode test; the other tests in both specs pass.
  • sbt "WorkflowOperator/Test/scalafmtCheck" "WorkflowOperator/Test/scalafix --check"

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5.5)

The 51 per-model sklearn descriptor specs each repeated the same five
tests. SklearnModelOpDescBehaviors now registers them, and each spec
passes its own literal model name and sklearn import. Test names and
assertions are unchanged; the specs with extra tests keep them.
Copilot AI balanced review requested due to automatic review settings October 28, 2026 08:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @kz930
    You can notify them by mentioning @kz930 in a comment.

@aglinxinyuan
aglinxinyuan requested a review from mengw15 October 28, 2026 08:05
@codecov-commenter

codecov-commenter commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.98%. Comparing base (92e86d4) to head (49580b5).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #9202   +/-   ##
=========================================
  Coverage     91.98%   91.98%           
  Complexity     5083     5083           
=========================================
  Files          1285     1285           
  Lines         56094    56094           
  Branches       7137     7137           
=========================================
  Hits          51599    51599           
  Misses         2742     2742           
  Partials       1753     1753           
Flag Coverage Δ *Carryforward flag
access-control-service 77.44% <ø> (ø)
agent-service 99.16% <ø> (ø) Carriedforward from 92e86d4
amber 86.43% <ø> (ø)
computing-unit-managing-service 64.02% <ø> (ø)
config-service 87.61% <ø> (ø)
file-service 82.29% <ø> (ø)
frontend 96.47% <ø> (ø) Carriedforward from 92e86d4
notebook-migration-service 83.77% <ø> (ø)
pyamber 98.54% <ø> (ø) Carriedforward from 92e86d4
workflow-compiling-service 74.25% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

✅ No material benchmark regressions detected

🟢 4 better · 🔴 0 worse · ⚪ 11 noise (<±5%) · 0 without baseline

Compared against main 92e86d4 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🟢 bs=10 sw=10 sl=64 441 0.269 21,589/30,356/30,356 us 🟢 -11.9% / 🔴 +120.3%
🟢 bs=100 sw=10 sl=64 955 0.583 101,355/151,457/151,457 us 🟢 -9.1% / 🔴 +62.6%
⚪ bs=1000 sw=10 sl=64 1,157 0.706 862,756/917,989/917,989 us ⚪ within ±5% / 🔴 -11.3%
Baseline details

Latest main 92e86d4 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 441 tuples/sec 446 tuples/sec 941.58 tuples/sec -1.1% -53.2%
bs=10 sw=10 sl=64 MB/s 0.269 MB/s 0.273 MB/s 0.575 MB/s -1.5% -53.2%
bs=10 sw=10 sl=64 p50 21,589 us 22,457 us 10,991 us -3.9% +96.4%
bs=10 sw=10 sl=64 p95 30,356 us 34,467 us 13,776 us -11.9% +120.3%
bs=10 sw=10 sl=64 p99 30,356 us 34,467 us 16,872 us -11.9% +79.9%
bs=100 sw=10 sl=64 throughput 955 tuples/sec 941 tuples/sec 1,264 tuples/sec +1.5% -24.5%
bs=100 sw=10 sl=64 MB/s 0.583 MB/s 0.574 MB/s 0.772 MB/s +1.6% -24.4%
bs=100 sw=10 sl=64 p50 101,355 us 96,544 us 86,073 us +5.0% +17.8%
bs=100 sw=10 sl=64 p95 151,457 us 166,612 us 93,135 us -9.1% +62.6%
bs=100 sw=10 sl=64 p99 151,457 us 166,612 us 111,124 us -9.1% +36.3%
bs=1000 sw=10 sl=64 throughput 1,157 tuples/sec 1,177 tuples/sec 1,304 tuples/sec -1.7% -11.3%
bs=1000 sw=10 sl=64 MB/s 0.706 MB/s 0.718 MB/s 0.796 MB/s -1.7% -11.3%
bs=1000 sw=10 sl=64 p50 862,756 us 849,212 us 841,986 us +1.6% +2.5%
bs=1000 sw=10 sl=64 p95 917,989 us 895,649 us 881,107 us +2.5% +4.2%
bs=1000 sw=10 sl=64 p99 917,989 us 895,649 us 918,208 us +2.5% -0.0%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,453.96,200,128000,441,0.269,21589.13,30355.77,30355.77
1,100,10,64,20,2094.13,2000,1280000,955,0.583,101355.31,151456.96,151456.96
2,1000,10,64,20,17280.97,20000,12800000,1157,0.706,862756.23,917989.17,917989.17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Share the per-model sklearn operator spec template through one test trait

3 participants