cmedina

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:39:28 -04:00

d769d39865 feat: add single shard test script

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:36:05 -04:00

0f624e1e75 fix: use bitsandbytes.functional for expert layer quantization

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:33:37 -04:00

410718bc73 feat: add CPU offloading for 70GB → 17GB conversion

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:32:40 -04:00

bee6683396 feat: simple NF4 quantization with device_map=auto (proven method)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:06:53 -04:00

bfa569f4e9 fix: remove streaming from output path name

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:06:01 -04:00

959c43d44c feat: true streaming NF4 quantization with safetensors

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 22:00:16 -04:00

52b93837f9 fix: use NF4 naming convention for quantized model

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:57:05 -04:00

2f997377f5 feat: add simple BnB 4-bit quantization script with device_map=auto

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:44:29 -04:00

d41648d4fc feat: load shards directly to GPU, no CPU bottleneck

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:41:46 -04:00

82677070ff fix: weights_only=False for safetensors loading

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:30:37 -04:00

aec7f405fe feat: use both GPUs in parallel for streaming quantization

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:28:06 -04:00

0ea51488a7 feat: add streaming quantization script (process one shard at a time)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:14:36 -04:00

170243de25 fix: remove GPU test, just check quantization

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:11:03 -04:00

43f5426583 feat: test BnB 4-bit loading and GPU transfer

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 21:09:56 -04:00

04a95f362f fix: load bf16 model directly with BnB 4-bit (single load)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 20:58:24 -04:00

1f804c95cd feat: load bf16 to CPU, quantize with BnB, move to GPU

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 20:47:13 -04:00

a9b066246d feat: add model size check script

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 20:42:06 -04:00

fa2f21562f fix: load BnB 4-bit model to single GPU (no distribution)

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 20:27:08 -04:00

a23ecc49f0 feat: use transformers built-in BnB loading with device_map=cpu

cmedina pushed to main at cmedina/agenx-lora-training

2026-07-02 20:25:27 -04:00

11f9c3a56c fix: force quantization with dummy forward pass