cmedina

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 12:17:45 -04:00

7f498134f2 fix: FSDP loads model to GPU first, then shards across GPUs

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 12:10:02 -04:00

f9c748706f fix: use already-quantized 4-bit model for training (no BnB needed)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 12:09:51 -04:00

3e26d7d37e fix: use already-quantized 4-bit model for test

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 12:04:56 -04:00

e3ea60e6c6 fix: clarify distribution pattern detection in test script

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 12:04:06 -04:00

3049aa9b0a feat: add test script to verify model loading and GPU distribution

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:55:23 -04:00

14ef1a07c4 fix: remove sync_module_states=True (model is on CPU)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:48:06 -04:00

5a13ca1d1c feat: FSDP training failures now fallback to next strategy

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:38:56 -04:00

ac1417567c fix: fix another indentation error

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:38:39 -04:00

243223d899 fix: fix indentation error in strategy 3

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:35:10 -04:00

1b3f678b50 feat: add Strategy 1 - 4-bit QLoRA with device_map=auto (distributed across GPUs)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:30:37 -04:00

93ac7391dc fix: clarify strategy naming + init distributed process group for FSDP

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:12:33 -04:00

8cabc0e986 fix: correct transformer_auto_wrap_policy API signature

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 11:03:42 -04:00

651965e844 feat: manually wrap model with FSDP on CPU before trainer

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 10:54:19 -04:00

b056ec0306 fix: force FSDP1 with string value to avoid FSDP2 gather spike

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:38:52 -04:00

f1e016c8af fix: set gradient_checkpointing=False (FSDP handles it)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:38:11 -04:00

361a58addc fix: simplify FSDP config, comment out mixed_precision

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:37:39 -04:00

ab1988705b fix: add use_orig_params=True for LoRA/PEFT compatibility

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:36:10 -04:00

b7d680966e feat: restore FSDP with SHARD_GRAD_OP + sync_module_states

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:26:39 -04:00

7b39fc3a1b fix: add --mixed_precision bf16 to accelerate launch

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:22:30 -04:00

fe7df7c92f fix: disable FSDP, use standard accelerate data parallelism