init: add LoRA training infrastructure and 20k dataset

2026-06-30 14:49:44 -04:00
commit 418a4cc76d
7 changed files with 20536 additions and 0 deletions
--- a/training/README.md
+++ b/training/README.md
@@ -0,0 +1,129 @@
+# Cyron Summary LoRA Training
+
+This directory contains scripts and configurations for training a LoRA adapter on the Cyron summary dataset.
+
+## Directory Structure
+
+```
+training/
+├── configs/          # Training configurations
+│   └── llama2-7b-lora.yaml
+├── data/            # Prepared datasets (train/test splits)
+├── output/          # Trained model outputs
+├── scripts/         # Training and inference scripts
+│   ├── prepare_dataset.py
+│   ├── train.py
+│   └── inference.py
+└── notebooks/       # Jupyter notebooks for analysis
+```
+
+## Prerequisites
+
+- Python 3.10+
+- GPU with 40GB+ VRAM (A100 recommended)
+- 64GB+ system RAM
+- 100GB+ free disk space
+
+## Installation
+
+```bash
+cd scripts/lora_training/training
+
+# Create virtual environment
+python -m venv venv
+source venv/bin/activate
+
+# Install dependencies
+pip install transformers datasets trl peft accelerate bitsandbytes
+
+# Or use conda
+conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
+pip install transformers datasets trl peft accelerate bitsandbytes
+```
+
+## Usage
+
+### 1. Prepare Dataset
+
+```bash
+python scripts/prepare_dataset.py --input ../combined_20k.jsonl --output data
+```
+
+This splits the 20k dataset into train/test sets (95/5).
+
+### 2. Train Model
+
+```bash
+python scripts/train.py --config configs/llama2-7b-lora.yaml
+```
+
+Or with custom parameters:
+
+```bash
+python scripts/train.py --config configs/llama2-7b-lora.yaml --epochs 5 --batch-size 8
+```
+
+Training takes approximately 6-24 hours depending on GPU.
+
+### 3. Generate Summaries
+
+```bash
+python scripts/inference.py \
+    --model output/llama2-7b-lora \
+    --task "Fix parser crash on malformed JSON" \
+    --files src/parser.cpp \
+    --tests-run --test-count 294 \
+    --commit --push
+```
+
+## Configuration
+
+Edit `configs/llama2-7b-lora.yaml` to adjust:
+
+- **Base model**: Change `base_model` to use different foundation models
+- **LoRA rank**: Adjust `lora_r` (higher = more capacity, slower)
+- **Learning rate**: Tune `learning_rate`
+- **Epochs**: Change `num_train_epochs`
+- **Batch size**: Adjust `per_device_train_batch_size`
+
+## Dataset Format
+
+The dataset (`combined_20k.jsonl`) contains 20,000 examples with:
+
+- `task`: Task description
+- `files_changed`: List of changed files
+- `tests_run`: Boolean
+- `commit`: Boolean
+- `push`: Boolean
+- `errors_seen`: Boolean
+- `test_count`: Number of tests
+- `output`: Expected Cyron summary
+
+## Output
+
+Trained model is saved to `output/llama2-7b-lora/`:
+
+- `adapter_model.safetensors` - LoRA weights
+- `config.json` - Model configuration
+- `tokenizer.json` - Tokenizer
+- `peft_config.json` - LoRA configuration
+
+## Troubleshooting
+
+**Out of memory:**
+- Reduce `per_device_train_batch_size`
+- Enable `gradient_checkpointing: true`
+- Use `load_in_4bit: true` (already enabled)
+
+**Training loss not decreasing:**
+- Lower learning rate (try 1e-4)
+- Increase warmup ratio
+- Check dataset quality
+
+**Generation too long/short:**
+- Adjust `max_new_tokens` in inference script
+- Tune `temperature` and `top_p`
+
+## License
+
+This training infrastructure is part of the AgenX project.