simpo

This repo learns a llm in 3 step to predict stock movement based on social news and stock + dollar prices for Tehran stock market

Full post-training pipeline for Qwen2.5-1.5B — SFT → SimPO → GRPO on free T4/P100 GPUs. GSM8K accuracy jumps from 23% (base) to 61% (GRPO) using Unsloth 4-bit LoRA, TRL, and HuggingFace Hub checkpointing.

Self-Align to Explain: Comparing Post-Training Methods for Counterfactual Generation

Official code and reproducibility artifacts for Reasoning on a Spectrum (COLM 2026)

Headless LLM fine-tuning in 3 lines — smart defaults, VRAM-aware batch sizing, multi-run SLAO, GGUF export for Ollama.

LocalSight · 198M-A64M 思考型 MoE LLM，2×RTX4090 从零训练（pretrain→SFT→SimPO→RLAIF→Agent RL），GGUF/Ollama 可运行

Domain-specific benchmark for B2B sales agents — 250 tasks, SimPO judge model, published on HuggingFace.

End-to-end RLHF pipeline with reward debiasing, DPO vs SimPO comparison, and statistical significance testing on Anthropic hh-rlhf dataset.

CPU-only post-training study: SFT vs DPO vs ORPO vs SimPO on a verifiable digit-addition task. Multi-seed; shows DPO/SimPO win-rate climbing to ~0.99 while real generation accuracy collapses (likelihood displacement).

Trustworthy QA alignment experiments with citation, abstention, and from-scratch DPO, SimPO, and ORPO losses.

RLHF experiments with Unsloth: DPO, ORPO, SimPO, KTO. Training scripts, notebooks, and quick evaluation utilities.