LLM fine-tuning glossary
18 terms buyers meet when evaluating fine-tuning vendors.
- Fine-tuning
- Additional training of a pretrained model on task-specific data, which changes the model’s weights.
- SFT (supervised fine-tuning)
- Training on example prompts paired with the ideal responses.
- LoRA (low-rank adaptation)
- A method that trains a small set of added adapter weights while the base model’s weights stay frozen.
- QLoRA
- LoRA training on a base model loaded in a quantized (lower-precision) format to reduce GPU memory use.
- Parameter-efficient fine-tuning (PEFT)
- Any method that updates only a small subset of parameters, such as LoRA adapters.
- Full fine-tuning
- Training that updates all of a model’s weights. Also called full-parameter training.
- DPO (direct preference optimization)
- Preference tuning that trains directly on pairs of preferred and rejected responses, without a separate reward model.
- ORPO
- A preference-tuning method that combines supervised and preference objectives in one training stage.
- RFT (reinforcement fine-tuning)
- Training where a grader or reward function scores model outputs and the model learns to raise the score.
- Reward function (grader)
- Code or a model that scores an output so reinforcement training can learn from it.
- Distillation
- Using a larger teacher model’s outputs to train a smaller, cheaper student model.
- Training tokens
- The billing unit most managed APIs use: tokens in the training dataset multiplied by the number of epochs.
- Epoch
- One complete pass through the training dataset.
- Checkpoint
- A saved copy of model weights at a point during training, which can be evaluated, resumed, or deployed.
- Merged weights
- A LoRA adapter folded into the base model to produce one self-contained set of weights.
- Dedicated endpoint
- Inference capacity reserved for your model, usually billed per hour or minute whether or not it receives traffic.
- SOC 2 Type II
- An independent auditor’s report on how a company’s security controls operated over a period.
- BAA (business associate agreement)
- The contract a US vendor signs to handle protected health information under HIPAA.