Wait for validation readiness without consuming rendered frames

[bkaradzic-microsoft]

Summary

• Wait for scene readiness, GUI image readiness, and clean material defines with ready effects in the active camera's render pass. Include utility scenes using the main scene's camera.
• Restore the prior render pass on every exit, including errors.
• Refresh render IDs without rendering extra animation/particle frames; preserve renderCount, screenshot indices, and RenderDoc capture/finalization budgets.
• Allow up to 240 non-rendering wait ticks; on the next readiness check, render if ready or fail if still unready (tick 241 for a continuously unready scene). Preserve once-only cleanup and suite continuation; reject stale callbacks after readiness timeout or failure.
• Restore the original seeded Math.random function as well as its seed between scenes.
• Add a standalone Node test job that executes the entire runner with simulated host services, without exporting or extracting private helpers.

Independence

Based on upstream master. Uses existing published Babylon.js 9.21.2 APIs. No JavaScript engine changes, dependency updates, protocol changes, engine-option changes, fixture/reference changes, capture-count changes, or tolerance changes.

Validation

• Full-script regressions: 42 pass. The six added failure-path regressions reproduce the render-pass leak and stale screenshot evaluation before d03cbc56; a delayed successful screenshot remains supported.
• Coverage includes GUI/material/effect waits, utility-scene association, render-pass restoration and exceptions, the exact 240-tick waiting bound, continuation/once-only cleanup, RNG function replacement, capture-frame budgets, and late callbacks.
• Stock Windows x64 / D3D11 / Chakra / RelWithDebInfo Playground: Native Canvas, setParent, and GUI Gradient Linear with transparency all pass with unchanged fixtures and gates.

Linux CI follow-up

6e48a96b fixes the GUI Near Menu timeout from Linux QuickJS CI. A local Linux/llvmpipe reproduction exhausted 240 convergence checks in 114 ms with five utility-scene loads still pending and zero rendered frames. The initial executeWhenReady wait covered only the main scene.

The existing initial readiness wait and 10-minute timeout now cover associated utility scenes too. Resource loads therefore complete before bounded convergence polling begins. The 240-check limit, canonical Near Menu render count (60), references, and pixel tolerances are unchanged.

Three new regressions cover both main/utility callback orders and utility timeout with late callbacks; they fail before and pass after. Near Menu, Native Canvas, setParent, and the transparent GUI gradient all pass together on Linux QuickJS/OpenGL/llvmpipe and Windows Chakra/D3D11. Fresh CI is pending.

Terminal shader error follow-up

Added Native-only fail-fast handling for terminal engine effect-error notifications, preserving the newer utility-scene ownership/readiness fixes already on this branch.

• Fail pending scene creation or scene processing outside the compiler's notification callback.
• Ignore remaining fallbacks, retained ready pipelines, and disposed effects.
• Remove observers/timers and invalidate pending readiness/render/screenshot callbacks; late-loaded scenes cannot overwrite the next test.
• Keep the existing readiness budgets, render counts, references, tolerances, dependency pins, and Babylon.js 9.21.2 unchanged.

All 42 full-script Node cases pass. Running these cases against the previous PR head exposes eight failures fixed here. An actual invalid GLSL ShaderMaterial followed by a black-frame control on stock Windows x64 / D3D11 / Chakra / Release produces ran=2 passed=1 failed=1 missingRef=0 in 2.121 seconds; the expected process exit is -1.

This does not claim unsupported WGSL preprocessing now reports an error on stock Babylon.js. Early WGSL rejection and asynchronous shader-preparation fixes remain separate Babylon.js changes. This PR consumes terminal errors that the released engine already emits.

[Copilot]

🟡 Changes recommended

Unresolved moderate issues remain in render-pass handling and asynchronous framebuffer evaluation.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds readiness polling for native validation without consuming rendered frames, preserving deterministic state and adding standalone regression coverage.

Changes:

• Waits for scene, GUI, material, effect, and utility-scene readiness.
• Preserves render budgets, render-pass state, cleanup, and seeded randomness.
• Adds documentation, Node tests, and CI execution.

File summaries

File	Summary
Documentation/AddingNewValidationTests.md	Documents readiness and deterministic capture behavior.
Apps/Playground/Tests/validation_native.test.cjs	Adds standalone host-independent runner regression tests.
Apps/Playground/Scripts/validation_native.js	Implements readiness polling and lifecycle handling; findings remain regarding pass ordering/restoration and stale asynchronous evaluation.
.github/workflows/ci.yml	Runs the standalone validation tests in CI.

Review details

Suppressed comments (1)

Apps/Playground/Scripts/validation_native.js:318

• scene.isReady() runs before currentRenderPassId is switched to the active camera pass. Babylon's readiness path calls mesh.isReady(true), which prepares/checks pass-specific draw wrappers; the later materialDefines/effect reads can therefore see an uninitialized camera-pass wrapper and treat the missing values as ready, allowing the first captured frame before that pass's effect is compiled. Select the camera pass before the entire readiness check and keep it inside the existing try/finally.

        if (!scene.isReady()) {

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[Copilot]

🟡 Changes recommended

Initial readiness can hang indefinitely when a pending associated utility scene is disposed or detached.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Utility layers created before the main camera can bypass both readiness phases.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The asynchronous lifecycle handling and private utility-layer instrumentation warrant final human validation across supported native engines.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

Copilot review overview

🔵 Needs a closer look

Native asynchronous lifecycle handling and private utility-layer instrumentation warrant final human validation.

Review effort: Balanced
Findings: None