Train Your Own Custom Model
For developers who want to fine-tune Llama on their own data. Requires Google Colab (free tier works).
โ ๏ธ Important Notes
- GPU Required: Fine-tuning needs a GPU. Use Google Colab (free T4 GPU) or your own NVIDIA GPU with 8GB+ VRAM.
- Time Required: Expect 2-4 hours for training on a typical dataset (1-5K samples).
- Not beginner-friendly: This section assumes you're comfortable with Python, Jupyter notebooks, and command line.
Prepare Your Training Data
Create a JSON file with your training examples in this format:
[
{
"instruction": "Summarize the following document",
"input": "Your document text here...",
"output": "The expected summary..."
},
{
"instruction": "Answer this question about the document",
"input": "What is the main topic?",
"output": "The main topic is..."
}
]Need 500-5000 examples for good results. Quality matters more than quantity.
Open in Google Colab
We provide a ready-to-use notebook. Click the badge to open:
Make sure to select Runtime โ Change runtime type โ T4 GPU (free tier).
Configure Training Parameters
Key settings in the notebook (already optimized for T4 GPU):
# LoRA Configuration
lora_r = 8 # Rank (higher = more capacity, more VRAM)
lora_alpha = 16 # Scaling factor
lora_dropout = 0.1 # Regularization
# Quantization (saves VRAM)
load_in_4bit = True # 4-bit NF4 quantization
bnb_4bit_compute_dtype = "float16"
# Training
num_epochs = 3
batch_size = 2 # Increase if you have more VRAM
gradient_accumulation = 8 # Effective batch = 2 ร 8 = 16
learning_rate = 2e-4Run Training
Execute all cells in the notebook. Training will:
- Load Llama 3.2 1B with 4-bit quantization
- Apply LoRA adapters to attention layers
- Train on your data with checkpointing
- Save the adapter to Google Drive
Training ~1K samples takes about 2 hours on T4. Watch the loss curveโit should decrease.
Export and Use Locally
After training, download the adapter and merge with the base model:
# In the notebook, after training completes:
# The adapter is saved to Google Drive automatically
# On your local machine:
# 1. Download the adapter folder from Drive
# 2. Merge with Ollama:
ollama create my-custom-model -f Modelfile๐ก Pro Tips
Data Quality > Quantity
500 high-quality examples beats 5000 noisy ones. Clean your data carefully.
Monitor the Loss
Training loss should decrease. If it plateaus, try lower learning rate.
Save Checkpoints
Colab sessions disconnect. Enable Drive auto-save in the notebook.
Test Iteratively
Train for 1 epoch, test, then continue. Don't train blindly for 10 epochs.