Training, Fine-Tuning, LoRA & Quantization

Pre-training trains a base model on multi-terabyte web text datasets to learn language patterns, costing millions of dollars in GPU compute. Inference is the runtime execution of a pre-trained model to generate predictions. Fine-Tuning further trains a base model on domain-specific datasets, while Instruction Tuning teaches models to follow human prompt directives.

Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA (Low-Rank Adaptation) freeze base model weights and train small adapter matrices (updating <1% of parameters). QLoRA combines LoRA with 4-bit quantization, enabling fine-tuning of 70B parameter models on a single consumer GPU.

Quantization reduces numerical precision (e.g. from FP16 to INT8 or INT4), drastically lowering memory footprint and inference cost with minimal quality loss. Distillation trains a smaller 'student' model to emulate a larger 'teacher' model. Mixture of Experts (MoE) architectures route each request to a subset of specialized expert subnetworks.

Before

Full Parameter Fine-Tuning (Requires 8x A100 GPUs)

1// ❌ Updates 70 billion parameters directly - expensive & high VRAM2model.train(); // All weights unfrozen

After

LoRA PEFT Configuration (Fits on Single GPU)

1// ✅ Freezes base model, trains low-rank rank-8 adapter matrices2from peft import LoraConfig, get_peft_model3 4peft_config = LoraConfig(5    r=8, lora_alpha=16, target_modules=["q_proj", "v_proj"],6    lora_dropout=0.05, bias="none", task_type="CAUSAL_LM"7)8model = get_peft_model(base_model, peft_config)

Exercise

Configure a LoRA adapter with rank r=8 and alpha=16, and calculate VRAM savings when quantizing an FP16 model to INT4.

Check your understanding

How does LoRA reduce fine-tuning costs compared to full parameter tuning?Show answerHide answer
Answer
LoRA freezes the original model weights and injects small trainable rank-decomposition matrices, updating under 1% of total parameters.
What is model Quantization?Show answerHide answer
Answer
Converting model weights from higher precision (16-bit float) to lower precision (8-bit or 4-bit integer) to reduce VRAM requirements and accelerate inference.