LLM 训练内存估算器
基于 UltraScale Playbook 公式，可预测不同配置下的训练内存消耗。调整下方参数实时查看内存分布变化。
Memory usage breakdown
Total: 0 GB | Activation: 0 GB | Params/Grads/Opt: 0 GB
Model Configuration
Presets
ⓘ
Custom
Llama 3 Tiny
Llama 3.2 1B
Llama 3.2 3B
Llama 3 8B
Llama 3 70B
Attention Heads (a)
ⓘ
Micro Batch (b)
ⓘ
Grad Accum Steps
ⓘ
Hidden (h)
ⓘ
FF Dim (h_ff)
ⓘ
Layers (L)
ⓘ
Seq Length (s)
ⓘ
Tensor Parallel (t)
ⓘ
Data Parallel (d)
ⓘ
Mixed Precision
ⓘ
Sequence Parallel
ⓘ
Recomputation
ⓘ
None
Selective
Full
ZeRO
ⓘ
Vocab Size (v)
ⓘ
Opt Params (k)
ⓘ