Add a DeepSpeed full-training backend

[kailash109]

Summary

• add a clean CommandBackend implementation for Hugging Face models trained with DeepSpeed ZeRO-2
• add Qwen3-8B and Qwen3.5-9B-Base 8×H100 definitions for distributed full-parameter training
• support Qwen3.5's multimodal model loader and nested text token configuration
• move the full-model delta publisher out of Megatron runtime and feed it through backend-specific HF weight exporters
• leave engine/ and control_plane/ unchanged; provider integration is limited to the image, definitions, and registry

Validation

• uv run pytest -q — 270 passed, 1 skipped
• changed-file Ruff and git diff --check
• Qwen3-8B DeepSpeed checkpoint E2E — fresh-client restore error 0.0, next-update error 0.0, identical loss and grad norm
• Qwen3-8B rollout-to-RL E2E — delta publication, SGLang sampling, importance-sampling backward, and optimizer update succeeded
• Qwen3.5-9B-Base math RL — completed 20 steps at 32 trajectories/step and 1e-6 learning rate; five-step mean exact-answer rate rose from 61.25% to 96.88%, with 100% on the final step

Qwen3.5-9B-Base convergence

Checkpoint report: scripts/results/deepspeed_qwen3_8b_checkpoint_e2e.20260904203306.json

RL report: scripts/results/deepspeed_qwen3_8b_e2e.20260904203909.json

Current scope

• full-parameter, single-model engines
• DeepSpeed ZeRO stages 1 and 2; the sample definitions use ZeRO-2 data parallelism
• cross_entropy and importance_sampling losses
• checkpoint capture writes an immutable local DeepSpeed checkpoint before asynchronous Volume persistence
• ZeRO-3, TP/PP, multi-model adapters, and throughput optimization are intentionally deferred

Boundary result

• no DeepSpeed imports or conditionals in engine or control-plane code
• DistributedExecutor, EngineServer, control-plane placement, checkpoint orchestration, and sampling orchestration are reused unchanged

Made with Cursor