cmedina/agenx-lora-training
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add CPU offload for optimizer states

Browse Source
This commit is contained in:
Christian Medina committed 2026-07-02 17:53:56 -04:00
1 parent a51a744d7c
commit 5a84b0eee5
2 files changed
+2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

training/configs/ornith-35b-lora.yaml
+1
View File
@@ -43,6 +43,7 @@ train_params:
save_total_limit: 3
output_dir: ../../output/ornith-35b-lora
optim: adamw_bnb_8bit # 8-bit optimizer to save VRAM
optim_args: "offload_optimizer_device=cpu" # Offload optimizer to CPU RAM
# Precision
mixed_precision: bf16