Christian Medina
|
aec7f405fe
|
feat: use both GPUs in parallel for streaming quantization
|
2026-07-02 21:30:35 -04:00 |
|
Christian Medina
|
0ea51488a7
|
feat: add streaming quantization script (process one shard at a time)
|
2026-07-02 21:28:05 -04:00 |
|
Christian Medina
|
170243de25
|
fix: remove GPU test, just check quantization
|
2026-07-02 21:14:34 -04:00 |
|
Christian Medina
|
43f5426583
|
feat: test BnB 4-bit loading and GPU transfer
|
2026-07-02 21:10:57 -04:00 |
|
Christian Medina
|
04a95f362f
|
fix: load bf16 model directly with BnB 4-bit (single load)
|
2026-07-02 21:09:54 -04:00 |
|
Christian Medina
|
1f804c95cd
|
feat: load bf16 to CPU, quantize with BnB, move to GPU
|
2026-07-02 20:58:22 -04:00 |
|
Christian Medina
|
a9b066246d
|
feat: add model size check script
|
2026-07-02 20:47:11 -04:00 |
|
Christian Medina
|
fa2f21562f
|
fix: load BnB 4-bit model to single GPU (no distribution)
|
2026-07-02 20:42:05 -04:00 |
|
Christian Medina
|
a23ecc49f0
|
feat: use transformers built-in BnB loading with device_map=cpu
|
2026-07-02 20:27:06 -04:00 |
|
Christian Medina
|
11f9c3a56c
|
fix: force quantization with dummy forward pass
|
2026-07-02 20:25:26 -04:00 |
|
Christian Medina
|
b624e90b02
|
fix: use BnB quantize_batch to actually quantize weights
|
2026-07-02 20:22:39 -04:00 |
|
Christian Medina
|
ec2b1be864
|
chore: clean up temp files
|
2026-07-02 20:18:13 -04:00 |
|
Christian Medina
|
3716051abc
|
fix: remove PEFT prepare (causes OOM)
|
2026-07-02 20:17:23 -04:00 |
|
Christian Medina
|
ac891fe25b
|
fix: replace with proper BnB 4-bit quantization script
|
2026-07-02 20:13:19 -04:00 |
|
Christian Medina
|
7984ae93e7
|
feat: proper BnB 4-bit quantization with PEFT prepare
|
2026-07-02 20:11:38 -04:00 |
|
Christian Medina
|
57b169f202
|
feat: use BnB 4-bit model for QLoRA training
|
2026-07-02 20:04:55 -04:00 |
|
Christian Medina
|
76ff09b140
|
feat: add BnB 4-bit quantization script
|
2026-07-02 20:04:44 -04:00 |
|
Christian Medina
|
3c31e4dfb0
|
feat: load bf16 model, quantize with BnB on CPU, move to GPU
|
2026-07-02 19:55:01 -04:00 |
|
Christian Medina
|
a0f4f644b0
|
fix: load 4-bit model with device_map=auto (let transformers distribute)
|
2026-07-02 19:33:37 -04:00 |
|
Christian Medina
|
20626e7a78
|
feat: manual 4-bit quantization with PEFT prepare + Linear4bit
|
2026-07-02 18:46:05 -04:00 |
|
Christian Medina
|
42c25962b2
|
feat: load model to CPU with BnB 4-bit, then move to GPU
|
2026-07-02 18:23:39 -04:00 |
|
Christian Medina
|
1c7e73e927
|
fix: load already-quantized 4-bit model directly (no BnB)
|
2026-07-02 18:20:43 -04:00 |
|
Christian Medina
|
199176c8d0
|
feat: load model with BnB 4-bit quantization (fit in single GPU)
|
2026-07-02 18:05:27 -04:00 |
|
Christian Medina
|
5a84b0eee5
|
feat: add CPU offload for optimizer states
|
2026-07-02 17:53:56 -04:00 |
|
Christian Medina
|
a51a744d7c
|
fix: reduce seq_length to 512 and increase gradient_accumulation
|
2026-07-02 17:53:11 -04:00 |
|
Christian Medina
|
691e5d3611
|
feat: use 8-bit AdamW optimizer to save VRAM
|
2026-07-02 17:53:05 -04:00 |
|
Christian Medina
|
92999b7783
|
fix: load model to single GPU (MoE requires all layers on same device)
|
2026-07-02 17:50:01 -04:00 |
|
Christian Medina
|
0456d55c8a
|
fix: disable gradient checkpointing (device mismatch with distributed MoE)
|
2026-07-02 17:29:19 -04:00 |
|
Christian Medina
|
d9c3781242
|
fix: quote 'no' in eval_strategy (YAML boolean issue)
|
2026-07-02 17:09:40 -04:00 |
|
Christian Medina
|
565ddd416e
|
fix: disable eval (no test split in dataset)
|
2026-07-02 16:49:34 -04:00 |
|
Christian Medina
|
dd86a4df37
|
fix: rename dataset column to 'text' for SFTTrainer compatibility
|
2026-07-02 16:39:21 -04:00 |
|
Christian Medina
|
36fe2c8d45
|
fix: pass dataset_text_field to SFTTrainer from config
|
2026-07-02 16:24:34 -04:00 |
|
Christian Medina
|
728073ef6a
|
fix: use 'output' column (not 'text')
|
2026-07-02 16:14:53 -04:00 |
|
Christian Medina
|
0752c03b83
|
fix: use absolute path for dataset (no ~ expansion)
|
2026-07-02 16:04:21 -04:00 |
|
Christian Medina
|
9e1c5120fb
|
fix: single-process training, remove FSDP (model pre-distributed via device_map)
|
2026-07-02 15:53:51 -04:00 |
|
Christian Medina
|
b8214f64cb
|
fix: use 4-bit model, remove DeepSpeed, fix dataset path
|
2026-07-02 15:23:26 -04:00 |
|
Christian Medina
|
4348323116
|
feat: use accelerate device_map for DISTRIBUTED model loading (Test 7 method)
|
2026-07-02 15:19:04 -04:00 |
|
Christian Medina
|
48e44cdf1c
|
feat: add Test 11 - PEFT prepare + manual 4-bit quantization
|
2026-07-02 14:30:19 -04:00 |
|
Christian Medina
|
da8658ae73
|
fix: switch to CompressedTensors 4-bit checkpoint (BnB on CPU not working)
|
2026-07-02 14:17:52 -04:00 |
|
Christian Medina
|
81dea9b9c8
|
docs: add model class verification to test output
|
2026-07-02 13:59:25 -04:00 |
|
Christian Medina
|
f608c7656f
|
feat: dynamically detect layer names from model config
|
2026-07-02 13:56:46 -04:00 |
|
Christian Medina
|
a55c39c818
|
fix: use device_map=cpu with BnB config for quantization
|
2026-07-02 13:54:19 -04:00 |
|
Christian Medina
|
3480dd9fbd
|
docs: add model inspection script and comment failing tests
|
2026-07-02 13:53:45 -04:00 |
|
Christian Medina
|
048e68f91a
|
refactor: comment out failing tests, add Test 7-10 variations
|
2026-07-02 13:45:17 -04:00 |
|
Christian Medina
|
e2b351e04c
|
fix: actually quantize model with quantize_4bit()
|
2026-07-02 13:34:33 -04:00 |
|
Christian Medina
|
aff91c990b
|
feat: add Test 6 - bf16 to CPU → BnB 4-bit quantize → GPU
|
2026-07-02 13:26:50 -04:00 |
|
Christian Medina
|
ffd1d29c35
|
fix: use bf16 model with BnB 4-bit (ON-THE-FLY quantization)
|
2026-07-02 13:20:26 -04:00 |
|
Christian Medina
|
48a2518b4c
|
feat: comprehensive test of 5 loading strategies
|
2026-07-02 12:32:59 -04:00 |
|
Christian Medina
|
3892ea7fec
|
feat: test with BnB 4-bit quantization to force proper loading
|
2026-07-02 12:23:34 -04:00 |
|
Christian Medina
|
c5a3b87eed
|
feat: test both strategies - device_map=auto and FSDP
|
2026-07-02 12:19:05 -04:00 |
|