cmedina/agenx-lora-training
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: single-process training, remove FSDP (model pre-distributed via device_map)

Browse Source
This commit is contained in:
Christian Medina
2026-07-02 15:53:51 -04:00
parent b8214f64cb
commit 9e1c5120fb
2 changed files with 53 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
train-on-this-server.sh
View File

@@ -18,9 +18,9 @@ pip install --upgrade pip
echo "Installing training dependencies..."
pip install transformers datasets trl peft accelerate bitsandbytes deepspeed
# Run training with accelerate data parallelism
echo "Starting training with accelerate..."
# Run training with single process (model already distributed via device_map)
echo "Starting training (single process, model pre-distributed)..."
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
accelerate launch --multi_gpu --num_processes 2 --mixed_precision bf16 train.py --config training/configs/ornith-35b-lora.yaml
python train.py --config training/configs/ornith-35b-lora.yaml
echo "Training completed!"