Introducing CFG (classifier free guidance) - #6

Open

vklimkov-nvidia wants to merge 9 commits into

vklimkov/modeldeffrom

Commits on Jul 4, 2025

[DRAFT] Initial implementation of CFG (classifier free guidance) for LLMs
vklimkov-nvidia
committed
7b94adf
cpp/tensorrt_llm/batch_manager/handleContextLogits.cpp: apply CFG during context phase too (first frame of logits)
vklimkov-nvidia
committed
feba85f
CFG in T5TTS: introduce acoustic token which expands to zeros for unconditional generation
vklimkov-nvidia
committed
3848630
runtimeBuffers: remove print, replace hardcode with vocab size from config
vklimkov-nvidia
committed
d9dc161
switch to using cublas to compute logits after cfg
vklimkov-nvidia
committed
0ff0aad
Fixes to freeing cache and slots for case when llm request contains multiple sequences
vklimkov-nvidia
committed
3829d85
Adjust gptManagerBenchmark to work with decoder models with cross attention, where encoder features are provided explicitely
vklimkov-nvidia
committed
3100c28
gptManagerBenchmark.cpp: pre-copy encoder features to GPU in benchmark
vklimkov-nvidia
committed
b626eb7

Commits on Jul 8, 2025

Enable latest grpo checkpoint, apply attention prior in leader thread
vklimkov-nvidia
committed
43daa79