130 lines
3.1 KiB
Markdown
130 lines
3.1 KiB
Markdown
# Cyron Summary LoRA Training
|
|
|
|
This directory contains scripts and configurations for training a LoRA adapter on the Cyron summary dataset.
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
training/
|
|
├── configs/ # Training configurations
|
|
│ └── llama2-7b-lora.yaml
|
|
├── data/ # Prepared datasets (train/test splits)
|
|
├── output/ # Trained model outputs
|
|
├── scripts/ # Training and inference scripts
|
|
│ ├── prepare_dataset.py
|
|
│ ├── train.py
|
|
│ └── inference.py
|
|
└── notebooks/ # Jupyter notebooks for analysis
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.10+
|
|
- GPU with 40GB+ VRAM (A100 recommended)
|
|
- 64GB+ system RAM
|
|
- 100GB+ free disk space
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
cd scripts/lora_training/training
|
|
|
|
# Create virtual environment
|
|
python -m venv venv
|
|
source venv/bin/activate
|
|
|
|
# Install dependencies
|
|
pip install transformers datasets trl peft accelerate bitsandbytes
|
|
|
|
# Or use conda
|
|
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
|
|
pip install transformers datasets trl peft accelerate bitsandbytes
|
|
```
|
|
|
|
## Usage
|
|
|
|
### 1. Prepare Dataset
|
|
|
|
```bash
|
|
python scripts/prepare_dataset.py --input ../combined_20k.jsonl --output data
|
|
```
|
|
|
|
This splits the 20k dataset into train/test sets (95/5).
|
|
|
|
### 2. Train Model
|
|
|
|
```bash
|
|
python scripts/train.py --config configs/llama2-7b-lora.yaml
|
|
```
|
|
|
|
Or with custom parameters:
|
|
|
|
```bash
|
|
python scripts/train.py --config configs/llama2-7b-lora.yaml --epochs 5 --batch-size 8
|
|
```
|
|
|
|
Training takes approximately 6-24 hours depending on GPU.
|
|
|
|
### 3. Generate Summaries
|
|
|
|
```bash
|
|
python scripts/inference.py \
|
|
--model output/llama2-7b-lora \
|
|
--task "Fix parser crash on malformed JSON" \
|
|
--files src/parser.cpp \
|
|
--tests-run --test-count 294 \
|
|
--commit --push
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Edit `configs/llama2-7b-lora.yaml` to adjust:
|
|
|
|
- **Base model**: Change `base_model` to use different foundation models
|
|
- **LoRA rank**: Adjust `lora_r` (higher = more capacity, slower)
|
|
- **Learning rate**: Tune `learning_rate`
|
|
- **Epochs**: Change `num_train_epochs`
|
|
- **Batch size**: Adjust `per_device_train_batch_size`
|
|
|
|
## Dataset Format
|
|
|
|
The dataset (`combined_20k.jsonl`) contains 20,000 examples with:
|
|
|
|
- `task`: Task description
|
|
- `files_changed`: List of changed files
|
|
- `tests_run`: Boolean
|
|
- `commit`: Boolean
|
|
- `push`: Boolean
|
|
- `errors_seen`: Boolean
|
|
- `test_count`: Number of tests
|
|
- `output`: Expected Cyron summary
|
|
|
|
## Output
|
|
|
|
Trained model is saved to `output/llama2-7b-lora/`:
|
|
|
|
- `adapter_model.safetensors` - LoRA weights
|
|
- `config.json` - Model configuration
|
|
- `tokenizer.json` - Tokenizer
|
|
- `peft_config.json` - LoRA configuration
|
|
|
|
## Troubleshooting
|
|
|
|
**Out of memory:**
|
|
- Reduce `per_device_train_batch_size`
|
|
- Enable `gradient_checkpointing: true`
|
|
- Use `load_in_4bit: true` (already enabled)
|
|
|
|
**Training loss not decreasing:**
|
|
- Lower learning rate (try 1e-4)
|
|
- Increase warmup ratio
|
|
- Check dataset quality
|
|
|
|
**Generation too long/short:**
|
|
- Adjust `max_new_tokens` in inference script
|
|
- Tune `temperature` and `top_p`
|
|
|
|
## License
|
|
|
|
This training infrastructure is part of the AgenX project.
|