 Christian Medina
|
a549084013
|
feat: add test for quantized shard loading
|
2026-07-03 01:52:46 -04:00 |
|
|
Christian Medina
|
4c7ca01d7d
|
feat: add single shard test script
|
2026-07-03 01:52:07 -04:00 |
|
|
Christian Medina
|
d8393969fc
|
fix: save quantization states with torch.save
|
2026-07-03 01:51:23 -04:00 |
|
|
Christian Medina
|
aa7961605f
|
fix: add index file creation for sharded models
|
2026-07-03 01:25:18 -04:00 |
|
|
Christian Medina
|
5d0ff6c8af
|
simplify: only train, no quantization logic
|
2026-07-03 01:25:11 -04:00 |
|
|
Christian Medina
|
6f45390be8
|
fix: remove quantization_config from config.json to prevent re-quantization
|
2026-07-03 01:24:25 -04:00 |
|
|
Christian Medina
|
1c3da52028
|
feat: create proper index file from safetensors metadata
|
2026-07-03 01:22:54 -04:00 |
|
|
Christian Medina
|
b15bc5c975
|
feat: detect pre-quantized model and skip re-quantization
|
2026-07-03 01:19:55 -04:00 |
|
|
Christian Medina
|
4308fd3391
|
feat: continuous processing - add to GPU with fewer tasks (max 4 per GPU)
|
2026-07-03 00:26:38 -04:00 |
|
|
Christian Medina
|
8a30ec92c6
|
fix: reduce to 3 shards per GPU (6 total) for memory
|
2026-07-03 00:23:21 -04:00 |
|
|
Christian Medina
|
23dcdd2c6f
|
feat: rename 0-indexed saved files to 1-indexed
|
2026-07-03 00:16:15 -04:00 |
|
|
Christian Medina
|
b4538c013b
|
fix: use 1-indexing for shard names (model-00001-of-00016)
|
2026-07-03 00:15:39 -04:00 |
|
|
Christian Medina
|
50623cdcd0
|
fix: use 0-indexing consistently (shard 0-15)
|
2026-07-03 00:14:57 -04:00 |
|
|
Christian Medina
|
262b1181b7
|
fix: skip to first unsaved shard, fix GPU assignment
|
2026-07-02 23:39:24 -04:00 |
|
|
Christian Medina
|
b742b5549e
|
fix: alternate GPUs for each shard (0,1,0,1)
|
2026-07-02 23:32:38 -04:00 |
|
|
Christian Medina
|
78817532ad
|
fix: import concurrent.futures at top level
|
2026-07-02 23:27:14 -04:00 |
|
|
Christian Medina
|
bc3bc6d6eb
|
feat: process 8 shards at once (4 per GPU) with better logging
|
2026-07-02 23:24:48 -04:00 |
|
|
Christian Medina
|
9e2c7bec8d
|
fix: remove duplicate sequential code
|
2026-07-02 23:12:03 -04:00 |
|
|
Christian Medina
|
7040c84618
|
feat: process 2 shards at a time (one per GPU)
|
2026-07-02 23:05:57 -04:00 |
|
|
Christian Medina
|
d698a32f0f
|
feat: skip already processed shards (resume support)
|
2026-07-02 23:04:52 -04:00 |
|
|
Christian Medina
|
d1f0c81ac4
|
fix: remove quant_states reference
|
2026-07-02 22:52:49 -04:00 |
|
|
Christian Medina
|
18ef0f73ef
|
fix: don't save quant_state objects (BnB rebuilds on load)
|
2026-07-02 22:48:09 -04:00 |
|
|
Christian Medina
|
8c13be4d1f
|
fix: free GPU memory after each layer quantization
|
2026-07-02 22:43:27 -04:00 |
|
|
Christian Medina
|
b9f6ddaf50
|
fix: free GPU memory after each quantization
|
2026-07-02 22:41:52 -04:00 |
|
|
Christian Medina
|
1751d33f69
|
fix: load shard to CPU, quantize on GPU
|
2026-07-02 22:40:31 -04:00 |
|
|
Christian Medina
|
d769d39865
|
feat: add single shard test script
|
2026-07-02 22:39:25 -04:00 |
|
|
Christian Medina
|
0f624e1e75
|
fix: use bitsandbytes.functional for expert layer quantization
|
2026-07-02 22:36:02 -04:00 |
|
|
Christian Medina
|
410718bc73
|
feat: add CPU offloading for 70GB → 17GB conversion
|
2026-07-02 22:33:36 -04:00 |
|
|
Christian Medina
|
bee6683396
|
feat: simple NF4 quantization with device_map=auto (proven method)
|
2026-07-02 22:32:38 -04:00 |
|
|
Christian Medina
|
bfa569f4e9
|
fix: remove streaming from output path name
|
2026-07-02 22:06:50 -04:00 |
|
|
Christian Medina
|
959c43d44c
|
feat: true streaming NF4 quantization with safetensors
|
2026-07-02 22:05:59 -04:00 |
|
|
Christian Medina
|
52b93837f9
|
fix: use NF4 naming convention for quantized model
|
2026-07-02 22:00:15 -04:00 |
|
|
Christian Medina
|
2f997377f5
|
feat: add simple BnB 4-bit quantization script with device_map=auto
|
2026-07-02 21:57:02 -04:00 |
|
|
Christian Medina
|
d41648d4fc
|
feat: load shards directly to GPU, no CPU bottleneck
|
2026-07-02 21:44:28 -04:00 |
|
|
Christian Medina
|
82677070ff
|
fix: weights_only=False for safetensors loading
|
2026-07-02 21:41:44 -04:00 |
|
|
Christian Medina
|
aec7f405fe
|
feat: use both GPUs in parallel for streaming quantization
|
2026-07-02 21:30:35 -04:00 |
|
|
Christian Medina
|
0ea51488a7
|
feat: add streaming quantization script (process one shard at a time)
|
2026-07-02 21:28:05 -04:00 |
|
|
Christian Medina
|
170243de25
|
fix: remove GPU test, just check quantization
|
2026-07-02 21:14:34 -04:00 |
|
|
Christian Medina
|
43f5426583
|
feat: test BnB 4-bit loading and GPU transfer
|
2026-07-02 21:10:57 -04:00 |
|
|
Christian Medina
|
04a95f362f
|
fix: load bf16 model directly with BnB 4-bit (single load)
|
2026-07-02 21:09:54 -04:00 |
|
|
Christian Medina
|
1f804c95cd
|
feat: load bf16 to CPU, quantize with BnB, move to GPU
|
2026-07-02 20:58:22 -04:00 |
|
|
Christian Medina
|
a9b066246d
|
feat: add model size check script
|
2026-07-02 20:47:11 -04:00 |
|
|
Christian Medina
|
fa2f21562f
|
fix: load BnB 4-bit model to single GPU (no distribution)
|
2026-07-02 20:42:05 -04:00 |
|
|
Christian Medina
|
a23ecc49f0
|
feat: use transformers built-in BnB loading with device_map=cpu
|
2026-07-02 20:27:06 -04:00 |
|
|
Christian Medina
|
11f9c3a56c
|
fix: force quantization with dummy forward pass
|
2026-07-02 20:25:26 -04:00 |
|
|
Christian Medina
|
b624e90b02
|
fix: use BnB quantize_batch to actually quantize weights
|
2026-07-02 20:22:39 -04:00 |
|
|
Christian Medina
|
ec2b1be864
|
chore: clean up temp files
|
2026-07-02 20:18:13 -04:00 |
|
|
Christian Medina
|
3716051abc
|
fix: remove PEFT prepare (causes OOM)
|
2026-07-02 20:17:23 -04:00 |
|
|
Christian Medina
|
ac891fe25b
|
fix: replace with proper BnB 4-bit quantization script
|
2026-07-02 20:13:19 -04:00 |
|
|
Christian Medina
|
7984ae93e7
|
feat: proper BnB 4-bit quantization with PEFT prepare
|
2026-07-02 20:11:38 -04:00 |
|
|
Christian Medina
|
57b169f202
|
feat: use BnB 4-bit model for QLoRA training
|
2026-07-02 20:04:55 -04:00 |
|
|
Christian Medina
|
76ff09b140
|
feat: add BnB 4-bit quantization script
|
2026-07-02 20:04:44 -04:00 |
|
|
Christian Medina
|
3c31e4dfb0
|
feat: load bf16 model, quantize with BnB on CPU, move to GPU
|
2026-07-02 19:55:01 -04:00 |
|
|
Christian Medina
|
a0f4f644b0
|
fix: load 4-bit model with device_map=auto (let transformers distribute)
|
2026-07-02 19:33:37 -04:00 |
|
|
Christian Medina
|
20626e7a78
|
feat: manual 4-bit quantization with PEFT prepare + Linear4bit
|
2026-07-02 18:46:05 -04:00 |
|
|
Christian Medina
|
42c25962b2
|
feat: load model to CPU with BnB 4-bit, then move to GPU
|
2026-07-02 18:23:39 -04:00 |
|
|
Christian Medina
|
1c7e73e927
|
fix: load already-quantized 4-bit model directly (no BnB)
|
2026-07-02 18:20:43 -04:00 |
|
|
Christian Medina
|
199176c8d0
|
feat: load model with BnB 4-bit quantization (fit in single GPU)
|
2026-07-02 18:05:27 -04:00 |
|
|
Christian Medina
|
5a84b0eee5
|
feat: add CPU offload for optimizer states
|
2026-07-02 17:53:56 -04:00 |
|
|
Christian Medina
|
a51a744d7c
|
fix: reduce seq_length to 512 and increase gradient_accumulation
|
2026-07-02 17:53:11 -04:00 |
|
|
Christian Medina
|
691e5d3611
|
feat: use 8-bit AdamW optimizer to save VRAM
|
2026-07-02 17:53:05 -04:00 |
|
|
Christian Medina
|
92999b7783
|
fix: load model to single GPU (MoE requires all layers on same device)
|
2026-07-02 17:50:01 -04:00 |
|
|
Christian Medina
|
0456d55c8a
|
fix: disable gradient checkpointing (device mismatch with distributed MoE)
|
2026-07-02 17:29:19 -04:00 |
|
|
Christian Medina
|
d9c3781242
|
fix: quote 'no' in eval_strategy (YAML boolean issue)
|
2026-07-02 17:09:40 -04:00 |
|
|
Christian Medina
|
565ddd416e
|
fix: disable eval (no test split in dataset)
|
2026-07-02 16:49:34 -04:00 |
|
|
Christian Medina
|
dd86a4df37
|
fix: rename dataset column to 'text' for SFTTrainer compatibility
|
2026-07-02 16:39:21 -04:00 |
|
|
Christian Medina
|
36fe2c8d45
|
fix: pass dataset_text_field to SFTTrainer from config
|
2026-07-02 16:24:34 -04:00 |
|
|
Christian Medina
|
728073ef6a
|
fix: use 'output' column (not 'text')
|
2026-07-02 16:14:53 -04:00 |
|
|
Christian Medina
|
0752c03b83
|
fix: use absolute path for dataset (no ~ expansion)
|
2026-07-02 16:04:21 -04:00 |
|
|
Christian Medina
|
9e1c5120fb
|
fix: single-process training, remove FSDP (model pre-distributed via device_map)
|
2026-07-02 15:53:51 -04:00 |
|
|
Christian Medina
|
b8214f64cb
|
fix: use 4-bit model, remove DeepSpeed, fix dataset path
|
2026-07-02 15:23:26 -04:00 |
|
|
Christian Medina
|
4348323116
|
feat: use accelerate device_map for DISTRIBUTED model loading (Test 7 method)
|
2026-07-02 15:19:04 -04:00 |
|
|
Christian Medina
|
48e44cdf1c
|
feat: add Test 11 - PEFT prepare + manual 4-bit quantization
|
2026-07-02 14:30:19 -04:00 |
|
|
Christian Medina
|
da8658ae73
|
fix: switch to CompressedTensors 4-bit checkpoint (BnB on CPU not working)
|
2026-07-02 14:17:52 -04:00 |
|
|
Christian Medina
|
81dea9b9c8
|
docs: add model class verification to test output
|
2026-07-02 13:59:25 -04:00 |
|
|
Christian Medina
|
f608c7656f
|
feat: dynamically detect layer names from model config
|
2026-07-02 13:56:46 -04:00 |
|
|
Christian Medina
|
a55c39c818
|
fix: use device_map=cpu with BnB config for quantization
|
2026-07-02 13:54:19 -04:00 |
|
|
Christian Medina
|
3480dd9fbd
|
docs: add model inspection script and comment failing tests
|
2026-07-02 13:53:45 -04:00 |
|
|
Christian Medina
|
048e68f91a
|
refactor: comment out failing tests, add Test 7-10 variations
|
2026-07-02 13:45:17 -04:00 |
|
|
Christian Medina
|
e2b351e04c
|
fix: actually quantize model with quantize_4bit()
|
2026-07-02 13:34:33 -04:00 |
|
|
Christian Medina
|
aff91c990b
|
feat: add Test 6 - bf16 to CPU → BnB 4-bit quantize → GPU
|
2026-07-02 13:26:50 -04:00 |
|
|
Christian Medina
|
ffd1d29c35
|
fix: use bf16 model with BnB 4-bit (ON-THE-FLY quantization)
|
2026-07-02 13:20:26 -04:00 |
|
|
Christian Medina
|
48a2518b4c
|
feat: comprehensive test of 5 loading strategies
|
2026-07-02 12:32:59 -04:00 |
|
|
Christian Medina
|
3892ea7fec
|
feat: test with BnB 4-bit quantization to force proper loading
|
2026-07-02 12:23:34 -04:00 |
|
|
Christian Medina
|
c5a3b87eed
|
feat: test both strategies - device_map=auto and FSDP
|
2026-07-02 12:19:05 -04:00 |
|
|
Christian Medina
|
7f498134f2
|
fix: FSDP loads model to GPU first, then shards across GPUs
|
2026-07-02 12:17:44 -04:00 |
|
|
Christian Medina
|
f9c748706f
|
fix: use already-quantized 4-bit model for training (no BnB needed)
|
2026-07-02 12:09:59 -04:00 |
|
|
Christian Medina
|
3e26d7d37e
|
fix: use already-quantized 4-bit model for test
|
2026-07-02 12:09:49 -04:00 |
|
|
Christian Medina
|
e3ea60e6c6
|
fix: clarify distribution pattern detection in test script
|
2026-07-02 12:04:54 -04:00 |
|
|
Christian Medina
|
3049aa9b0a
|
feat: add test script to verify model loading and GPU distribution
|
2026-07-02 12:04:04 -04:00 |
|
|
Christian Medina
|
14ef1a07c4
|
fix: remove sync_module_states=True (model is on CPU)
|
2026-07-02 11:55:21 -04:00 |
|
|
Christian Medina
|
5a13ca1d1c
|
feat: FSDP training failures now fallback to next strategy
|
2026-07-02 11:48:03 -04:00 |
|
|
Christian Medina
|
ac1417567c
|
fix: fix another indentation error
|
2026-07-02 11:38:54 -04:00 |
|
|
Christian Medina
|
243223d899
|
fix: fix indentation error in strategy 3
|
2026-07-02 11:38:37 -04:00 |
|
|
Christian Medina
|
1b3f678b50
|
feat: add Strategy 1 - 4-bit QLoRA with device_map=auto (distributed across GPUs)
|
2026-07-02 11:35:09 -04:00 |
|
|
Christian Medina
|
93ac7391dc
|
fix: clarify strategy naming + init distributed process group for FSDP
|
2026-07-02 11:30:34 -04:00 |
|
|
Christian Medina
|
8cabc0e986
|
fix: correct transformer_auto_wrap_policy API signature
|
2026-07-02 11:12:31 -04:00 |
|
|
Christian Medina
|
651965e844
|
feat: manually wrap model with FSDP on CPU before trainer
|
2026-07-02 11:03:41 -04:00 |
|
|
Christian Medina
|
b056ec0306
|
fix: force FSDP1 with string value to avoid FSDP2 gather spike
|
2026-07-02 10:54:17 -04:00 |
|
|
Christian Medina
|
f1e016c8af
|
fix: set gradient_checkpointing=False (FSDP handles it)
|
2026-07-02 09:38:51 -04:00 |
|