cmedina

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:14:20 -04:00

42d61c15b0 fix: disable sync_module_states + add mixed precision for FSDP

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 09:05:03 -04:00

4eb06a625e fix: add bnb_4bit_quant_storage for FSDP sharding of 4-bit weights

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 08:54:44 -04:00

4c9522072e fix: add BitsAndBytesConfig import + remove bf16 GPU strategy

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 08:47:58 -04:00

70a2bf0dd6 fix: use FSDP1 with sync_module_states to avoid GPU gather spike

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 08:26:25 -04:00

c607888d8f fix: load QLoRA model to CPU first, FSDP shards later

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 00:58:43 -04:00

dbc97138e9 fix: use SHARD_GRAD_OP instead of FULL_SHARD for 2-GPU setup

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 00:41:59 -04:00

e3ea45fe96 fix: remove gradient_checkpointing (FSDP activation_checkpointing handles it)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 00:16:35 -04:00

dfe85c3c82 fix: fsdp=True instead of list + add activation_checkpointing

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 00:11:14 -04:00

8747111671 fix: update config comment to reflect Ornith-35B

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 00:09:05 -04:00

11c635b0e0 fix: use 4-bit model AS-IS + fix FSDP config

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:46:35 -04:00

d5ccd28e39 chore: remove temporary debugging code

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:45:06 -04:00

8a5edb25e0 fix: specify correct transformer layer class for FSDP auto-wrap

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:27:26 -04:00

05cbd7b6b2 show layer class names

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:24:13 -04:00

5d2be60311 show layer class names

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:21:38 -04:00

ce575ee71b show layer class names

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:17:39 -04:00

ee446a2b85 show layer class names

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 23:12:32 -04:00

dc494dd91c show layer class names

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 22:33:08 -04:00

e64515e3ca refactor: switch from DeepSpeed ZeRO-3 to FSDP for QLoRA compatibility

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 22:08:40 -04:00

a655040db2 feat: use DeepSpeed ZeRO-3 for proper model sharding across GPUs

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-01 21:38:43 -04:00

9d8c2b6cf0 refactor: cleaner loading strategies with error tracking