Christian Medina e25f146fb8 feat: add proper BnB quantization script for 2x RTX 5090

2026-07-03 04:19:44 -04:00

training

fix: use NF4 naming convention for quantized model

2026-07-02 22:00:15 -04:00

.gitignore

fix: remove large dataset from git tracking, add to .gitignore

2026-06-30 15:06:13 -04:00

check_model_size.py

fix: remove GPU test, just check quantization

2026-07-02 21:14:34 -04:00

convert_to_safetensors.py

fix: Path conversion for shard_path

2026-07-03 04:10:54 -04:00

deploy-agenx-lora.sh

fix: use $HOME/loras by default

2026-06-30 15:51:28 -04:00

deploy-and-train.sh

refactor: simplify deploy script, add train-on-this-server.sh

2026-06-30 15:22:54 -04:00

inference.py

refactor: restructure project - scripts at root level

2026-07-01 16:39:54 -04:00

inspect_model.py

docs: add model inspection script and comment failing tests

2026-07-02 13:53:45 -04:00

prepare_dataset.py

refactor: restructure project - scripts at root level

2026-07-01 16:39:54 -04:00

quantize_model.py

feat: add CPU offloading for 70GB → 17GB conversion

2026-07-02 22:33:36 -04:00

quantize_proper_bnb.py

feat: add proper BnB quantization script for 2x RTX 5090

2026-07-03 04:19:44 -04:00

quantize_streaming.py

fix: save quantization states with torch.save

2026-07-03 01:51:23 -04:00

quantize_to_bnb.py

feat: use transformers built-in BnB loading with device_map=cpu

2026-07-02 20:27:06 -04:00

test_full_quantized.py

feat: test all quantized shards loading

2026-07-03 03:58:16 -04:00

test_load_quantized.py

fix: copy config.json for load test

2026-07-03 01:57:29 -04:00

test_model_loading.py

feat: add Test 11 - PEFT prepare + manual 4-bit quantization

2026-07-02 14:30:19 -04:00

test_quantize_one_shard.py

fix: save as model.safetensors for from_pretrained

2026-07-03 02:00:29 -04:00

test_quantize_shapes.py

fix: weights_only=False for QuantState

2026-07-03 02:07:09 -04:00

test_quantize_single_shard.py

fix: free GPU memory after each quantization

2026-07-02 22:41:52 -04:00

train-on-this-server.sh

fix: single-process training, remove FSDP (model pre-distributed via device_map)

2026-07-02 15:53:51 -04:00

train.py

fix: use torch.load for quantized shards index creation

2026-07-03 03:58:24 -04:00

Description

LoRA training infrastructure for Cyron summary generation

2.6 MiB

Languages

Python 94.9%

Shell 5.1%