Commit Graph

Select branches

Hide Pull Requests

main

Mono Color

• e25f146fb8 feat: add proper BnB quantization script for 2x RTX 5090 main Christian Medina 2026-07-03 04:19:44 -04:00
• d2dde1079f fix: Path conversion for shard_path Christian Medina 2026-07-03 04:10:54 -04:00
• 3498a40390 fix: add argparser and fix Path issue Christian Medina 2026-07-03 04:09:23 -04:00
• c84a3b6252 feat: add safetensors conversion script Christian Medina 2026-07-03 04:05:14 -04:00
• 1ebad2365c fix: use torch.load for quantized shards index creation Christian Medina 2026-07-03 03:58:24 -04:00
• 5078d7c8dc feat: test all quantized shards loading Christian Medina 2026-07-03 03:58:16 -04:00
• 1c576aac5d fix: weights_only=False for QuantState Christian Medina 2026-07-03 02:07:09 -04:00
• 5b2d629d11 feat: test quantized tensor shapes directly Christian Medina 2026-07-03 02:04:50 -04:00
• 1c792d23e4 fix: save as model.safetensors for from_pretrained Christian Medina 2026-07-03 02:00:29 -04:00
• 7115d56723 fix: copy config.json for load test Christian Medina 2026-07-03 01:57:29 -04:00
• a549084013 feat: add test for quantized shard loading Christian Medina 2026-07-03 01:52:46 -04:00
• 4c7ca01d7d feat: add single shard test script Christian Medina 2026-07-03 01:52:07 -04:00
• d8393969fc fix: save quantization states with torch.save Christian Medina 2026-07-03 01:51:23 -04:00
• aa7961605f fix: add index file creation for sharded models Christian Medina 2026-07-03 01:25:18 -04:00
• 5d0ff6c8af simplify: only train, no quantization logic Christian Medina 2026-07-03 01:25:11 -04:00
• 6f45390be8 fix: remove quantization_config from config.json to prevent re-quantization Christian Medina 2026-07-03 01:24:25 -04:00
• 1c3da52028 feat: create proper index file from safetensors metadata Christian Medina 2026-07-03 01:22:54 -04:00
• b15bc5c975 feat: detect pre-quantized model and skip re-quantization Christian Medina 2026-07-03 01:19:55 -04:00
• 4308fd3391 feat: continuous processing - add to GPU with fewer tasks (max 4 per GPU) Christian Medina 2026-07-03 00:26:38 -04:00
• 8a30ec92c6 fix: reduce to 3 shards per GPU (6 total) for memory Christian Medina 2026-07-03 00:23:21 -04:00
• 23dcdd2c6f feat: rename 0-indexed saved files to 1-indexed Christian Medina 2026-07-03 00:16:15 -04:00
• b4538c013b fix: use 1-indexing for shard names (model-00001-of-00016) Christian Medina 2026-07-03 00:15:39 -04:00
• 50623cdcd0 fix: use 0-indexing consistently (shard 0-15) Christian Medina 2026-07-03 00:14:57 -04:00
• 262b1181b7 fix: skip to first unsaved shard, fix GPU assignment Christian Medina 2026-07-02 23:39:24 -04:00
• b742b5549e fix: alternate GPUs for each shard (0,1,0,1) Christian Medina 2026-07-02 23:32:38 -04:00
• 78817532ad fix: import concurrent.futures at top level Christian Medina 2026-07-02 23:27:14 -04:00
• bc3bc6d6eb feat: process 8 shards at once (4 per GPU) with better logging Christian Medina 2026-07-02 23:24:48 -04:00
• 9e2c7bec8d fix: remove duplicate sequential code Christian Medina 2026-07-02 23:12:03 -04:00
• 7040c84618 feat: process 2 shards at a time (one per GPU) Christian Medina 2026-07-02 23:05:57 -04:00
• d698a32f0f feat: skip already processed shards (resume support) Christian Medina 2026-07-02 23:04:52 -04:00
• d1f0c81ac4 fix: remove quant_states reference Christian Medina 2026-07-02 22:52:49 -04:00
• 18ef0f73ef fix: don't save quant_state objects (BnB rebuilds on load) Christian Medina 2026-07-02 22:48:09 -04:00
• 8c13be4d1f fix: free GPU memory after each layer quantization Christian Medina 2026-07-02 22:43:27 -04:00
• b9f6ddaf50 fix: free GPU memory after each quantization Christian Medina 2026-07-02 22:41:52 -04:00
• 1751d33f69 fix: load shard to CPU, quantize on GPU Christian Medina 2026-07-02 22:40:31 -04:00
• d769d39865 feat: add single shard test script Christian Medina 2026-07-02 22:39:25 -04:00
• 0f624e1e75 fix: use bitsandbytes.functional for expert layer quantization Christian Medina 2026-07-02 22:36:02 -04:00
• 410718bc73 feat: add CPU offloading for 70GB → 17GB conversion Christian Medina 2026-07-02 22:33:36 -04:00
• bee6683396 feat: simple NF4 quantization with device_map=auto (proven method) Christian Medina 2026-07-02 22:32:38 -04:00
• bfa569f4e9 fix: remove streaming from output path name Christian Medina 2026-07-02 22:06:50 -04:00
• 959c43d44c feat: true streaming NF4 quantization with safetensors Christian Medina 2026-07-02 22:05:59 -04:00
• 52b93837f9 fix: use NF4 naming convention for quantized model Christian Medina 2026-07-02 22:00:15 -04:00
• 2f997377f5 feat: add simple BnB 4-bit quantization script with device_map=auto Christian Medina 2026-07-02 21:57:02 -04:00
• d41648d4fc feat: load shards directly to GPU, no CPU bottleneck Christian Medina 2026-07-02 21:44:28 -04:00
• 82677070ff fix: weights_only=False for safetensors loading Christian Medina 2026-07-02 21:41:44 -04:00
• aec7f405fe feat: use both GPUs in parallel for streaming quantization Christian Medina 2026-07-02 21:30:35 -04:00
• 0ea51488a7 feat: add streaming quantization script (process one shard at a time) Christian Medina 2026-07-02 21:28:05 -04:00
• 170243de25 fix: remove GPU test, just check quantization Christian Medina 2026-07-02 21:14:34 -04:00
• 43f5426583 feat: test BnB 4-bit loading and GPU transfer Christian Medina 2026-07-02 21:10:57 -04:00
• 04a95f362f fix: load bf16 model directly with BnB 4-bit (single load) Christian Medina 2026-07-02 21:09:54 -04:00
• 1f804c95cd feat: load bf16 to CPU, quantize with BnB, move to GPU Christian Medina 2026-07-02 20:58:22 -04:00
• a9b066246d feat: add model size check script Christian Medina 2026-07-02 20:47:11 -04:00
• fa2f21562f fix: load BnB 4-bit model to single GPU (no distribution) Christian Medina 2026-07-02 20:42:05 -04:00
• a23ecc49f0 feat: use transformers built-in BnB loading with device_map=cpu Christian Medina 2026-07-02 20:27:06 -04:00
• 11f9c3a56c fix: force quantization with dummy forward pass Christian Medina 2026-07-02 20:25:26 -04:00
• b624e90b02 fix: use BnB quantize_batch to actually quantize weights Christian Medina 2026-07-02 20:22:39 -04:00
• ec2b1be864 chore: clean up temp files Christian Medina 2026-07-02 20:18:13 -04:00
• 3716051abc fix: remove PEFT prepare (causes OOM) Christian Medina 2026-07-02 20:17:23 -04:00
• ac891fe25b fix: replace with proper BnB 4-bit quantization script Christian Medina 2026-07-02 20:13:19 -04:00
• 7984ae93e7 feat: proper BnB 4-bit quantization with PEFT prepare Christian Medina 2026-07-02 20:11:38 -04:00
• 57b169f202 feat: use BnB 4-bit model for QLoRA training Christian Medina 2026-07-02 20:04:55 -04:00
• 76ff09b140 feat: add BnB 4-bit quantization script Christian Medina 2026-07-02 20:04:44 -04:00
• 3c31e4dfb0 feat: load bf16 model, quantize with BnB on CPU, move to GPU Christian Medina 2026-07-02 19:55:01 -04:00
• a0f4f644b0 fix: load 4-bit model with device_map=auto (let transformers distribute) Christian Medina 2026-07-02 19:33:37 -04:00
• 20626e7a78 feat: manual 4-bit quantization with PEFT prepare + Linear4bit Christian Medina 2026-07-02 18:46:05 -04:00
• 42c25962b2 feat: load model to CPU with BnB 4-bit, then move to GPU Christian Medina 2026-07-02 18:23:39 -04:00
• 1c7e73e927 fix: load already-quantized 4-bit model directly (no BnB) Christian Medina 2026-07-02 18:20:43 -04:00
• 199176c8d0 feat: load model with BnB 4-bit quantization (fit in single GPU) Christian Medina 2026-07-02 18:05:27 -04:00
• 5a84b0eee5 feat: add CPU offload for optimizer states Christian Medina 2026-07-02 17:53:56 -04:00
• a51a744d7c fix: reduce seq_length to 512 and increase gradient_accumulation Christian Medina 2026-07-02 17:53:11 -04:00
• 691e5d3611 feat: use 8-bit AdamW optimizer to save VRAM Christian Medina 2026-07-02 17:53:05 -04:00
• 92999b7783 fix: load model to single GPU (MoE requires all layers on same device) Christian Medina 2026-07-02 17:50:01 -04:00
• 0456d55c8a fix: disable gradient checkpointing (device mismatch with distributed MoE) Christian Medina 2026-07-02 17:29:19 -04:00
• d9c3781242 fix: quote 'no' in eval_strategy (YAML boolean issue) Christian Medina 2026-07-02 17:09:40 -04:00
• 565ddd416e fix: disable eval (no test split in dataset) Christian Medina 2026-07-02 16:49:34 -04:00
• dd86a4df37 fix: rename dataset column to 'text' for SFTTrainer compatibility Christian Medina 2026-07-02 16:39:21 -04:00
• 36fe2c8d45 fix: pass dataset_text_field to SFTTrainer from config Christian Medina 2026-07-02 16:24:34 -04:00
• 728073ef6a fix: use 'output' column (not 'text') Christian Medina 2026-07-02 16:14:53 -04:00
• 0752c03b83 fix: use absolute path for dataset (no ~ expansion) Christian Medina 2026-07-02 16:04:21 -04:00
• 9e1c5120fb fix: single-process training, remove FSDP (model pre-distributed via device_map) Christian Medina 2026-07-02 15:53:51 -04:00
• b8214f64cb fix: use 4-bit model, remove DeepSpeed, fix dataset path Christian Medina 2026-07-02 15:23:26 -04:00
• 4348323116 feat: use accelerate device_map for DISTRIBUTED model loading (Test 7 method) Christian Medina 2026-07-02 15:19:04 -04:00
• 48e44cdf1c feat: add Test 11 - PEFT prepare + manual 4-bit quantization Christian Medina 2026-07-02 14:30:19 -04:00
• da8658ae73 fix: switch to CompressedTensors 4-bit checkpoint (BnB on CPU not working) Christian Medina 2026-07-02 14:17:52 -04:00
• 81dea9b9c8 docs: add model class verification to test output Christian Medina 2026-07-02 13:59:25 -04:00
• f608c7656f feat: dynamically detect layer names from model config Christian Medina 2026-07-02 13:56:46 -04:00
• a55c39c818 fix: use device_map=cpu with BnB config for quantization Christian Medina 2026-07-02 13:54:19 -04:00
• 3480dd9fbd docs: add model inspection script and comment failing tests Christian Medina 2026-07-02 13:53:45 -04:00
• 048e68f91a refactor: comment out failing tests, add Test 7-10 variations Christian Medina 2026-07-02 13:45:17 -04:00
• e2b351e04c fix: actually quantize model with quantize_4bit() Christian Medina 2026-07-02 13:34:33 -04:00
• aff91c990b feat: add Test 6 - bf16 to CPU → BnB 4-bit quantize → GPU Christian Medina 2026-07-02 13:26:50 -04:00
• ffd1d29c35 fix: use bf16 model with BnB 4-bit (ON-THE-FLY quantization) Christian Medina 2026-07-02 13:20:26 -04:00
• 48a2518b4c feat: comprehensive test of 5 loading strategies Christian Medina 2026-07-02 12:32:59 -04:00
• 3892ea7fec feat: test with BnB 4-bit quantization to force proper loading Christian Medina 2026-07-02 12:23:34 -04:00
• c5a3b87eed feat: test both strategies - device_map=auto and FSDP Christian Medina 2026-07-02 12:19:05 -04:00
• 7f498134f2 fix: FSDP loads model to GPU first, then shards across GPUs Christian Medina 2026-07-02 12:17:44 -04:00
• f9c748706f fix: use already-quantized 4-bit model for training (no BnB needed) Christian Medina 2026-07-02 12:09:59 -04:00
• 3e26d7d37e fix: use already-quantized 4-bit model for test Christian Medina 2026-07-02 12:09:49 -04:00
• e3ea60e6c6 fix: clarify distribution pattern detection in test script Christian Medina 2026-07-02 12:04:54 -04:00
• 3049aa9b0a feat: add test script to verify model loading and GPU distribution Christian Medina 2026-07-02 12:04:04 -04:00