feat: add Slurm command client and renderer

[nabinchha]

📋 Summary

Adds the first independently reviewable #868 slice: a structured Slurm command boundary and deterministic thin batch renderer. This gives M0 a plan-to-script proof and supplies the scheduler primitives needed by #867 without pulling allocation runtime or persistence policy into this PR.

🔗 Related Issue

Part of #868

🔄 Changes

• Add typed sbatch, squeue, sacct, scancel, and bounded sinfo operations with exact argv construction, minimal environments, timeouts, normalized errors, and returned-selector correlation.
• Add strict parsers for array submissions, active and terminal scheduler states, exit codes, accounting absence, and configured GPU GRES values.
• Add a deterministic generation-script renderer with typed directives, shell-safe values, pinned host-tool lookup, checksum verification, shard/attempt paths, and no user-provided executable shell text.
• Extend deterministic Slurm fakes and sanitized golden scripts to mirror the production command formats.
• Verify launcher behavior through focused boundary tests and isolated built-wheel imports.

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

• client.py — managed-selector correlation and the --export=NIL submission boundary.
• renderer.py — directive/resource semantics and checksum-before-source ordering.

🧪 Testing

• make test passes — 4,564 passed, 1 skipped
• Unit tests added/updated — 129 focused launcher/fake tests; 99% launcher coverage with all public paths covered
• E2E tests added/updated — N/A; this local/fake slice extends deterministic scheduler and golden-script coverage
• make check-all passes
• make test-slurm-wheel-install passes
• Five consecutive ten-pass review cycles completed with no additional findings

✅ Checklist

• Follows commit message conventions
• Commits are signed off (DCO)
• Architecture docs updated — N/A; this implements the existing reviewed plan and contracts

[greptile-apps]

Greptile Summary

The PR adds a structured Slurm command boundary and deterministic batch-script renderer for the first scheduler execution slice.

• Adds typed clients and parsers for submission, queue, accounting, cancellation, and GPU inventory commands.
• Adds deterministic, shell-safe batch rendering with checksum verification and attempt-specific runtime paths.
• Adds deterministic Slurm fakes, golden scripts, focused launcher tests, and built-wheel import coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
packages/data-designer-slurm/src/data_designer/slurm/launcher/client.py	Adds typed, argument-vector operations for Slurm submission, observation, inventory, and cancellation with normalized command failures.
packages/data-designer-slurm/src/data_designer/slurm/launcher/parsing.py	Adds strict parsing and normalization for scheduler identifiers, states, exit codes, submissions, and GPU GRES output.
packages/data-designer-slurm/src/data_designer/slurm/launcher/renderer.py	Adds deterministic batch rendering with validated directives, shell-safe values, integrity checks, and attempt-specific paths.
packages/data-designer-slurm/src/data_designer/slurm/launcher/runner.py	Adds bounded subprocess execution with a minimal locale-stable environment; the two previously reported default-PATH failures are fixed at current HEAD.
packages/data-designer-slurm/tests/slurm_test_fakes/slurm.py	Extends deterministic scheduler fakes to mirror the new production command formats.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Plan[Resolved Slurm run plan] --> Renderer[Batch script renderer]
    Renderer --> Script[Deterministic sbatch script]
    Client[Slurm command client] --> Runner[Bounded subprocess runner]
    Runner --> Slurm[Slurm CLI tools]
    Slurm --> Parsers[Strict output parsers]
    Parsers --> Records[Typed scheduler records]
    Client --> Script

Loading

_{Reviews (3): Last reviewed commit: "fix: fall back from empty command path" | Re-trigger Greptile}

[greptile-apps]

Default environment loses Slurm PATH

If a cluster installs Slurm outside the platform fallback path, the default runner replaces the configured environment with only LC_ALL=C while using bare executable names, causing every default client operation to fail with SlurmCommandError even though the tools are available through the caller's PATH.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/data-designer-slurm/src/data_designer/slurm/launcher/runner.py
Line: 43

Comment:
**Default environment loses Slurm PATH**

If a cluster installs Slurm outside the platform fallback path, the default runner replaces the configured environment with only `LC_ALL=C` while using bare executable names, causing every default client operation to fail with `SlurmCommandError` even though the tools are available through the caller's `PATH`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[nabinchha]

Fixed in bb5bc8e. The default runner now forwards only the caller's PATH plus LC_ALL=C; an explicitly supplied environment remains an exact replacement. This preserves cluster command discovery without unrestricted ambient-environment forwarding. The environment regression coverage and the full Slurm suite pass (490 tests after the follow-up fix).

[greptile-apps]

Empty PATH bypasses fallback resolution

If the parent environment contains PATH="", os.environ.get preserves the empty value instead of using os.defpath. The default client's bare Slurm executable names then cannot resolve, causing every operation to fail with SlurmCommandError even when the tools are installed in a platform fallback directory.

Suggested change

	dict(environment) if environment is not None else {"PATH": os.environ.get("PATH", os.defpath)}
	dict(environment) if environment is not None else {"PATH": os.environ.get("PATH") or os.defpath}

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/data-designer-slurm/src/data_designer/slurm/launcher/runner.py
Line: 39

Comment:
**Empty PATH bypasses fallback resolution**

If the parent environment contains `PATH=""`, `os.environ.get` preserves the empty value instead of using `os.defpath`. The default client's bare Slurm executable names then cannot resolve, causing every operation to fail with `SlurmCommandError` even when the tools are installed in a platform fallback directory.

```suggestion
            dict(environment) if environment is not None else {"PATH": os.environ.get("PATH") or os.defpath}
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[nabinchha]

Fixed in ca308e2 by falling back to os.defpath when the ambient PATH is absent or empty. Added a regression test for an empty PATH; the full Slurm suite passes (490 tests), and Slurm lint and formatting checks pass.