Fine-Tuning Starts With a Data Contract

Fine-tuning is often presented as a switch: bring data, train a model, get a specialist. In practice, the highest-leverage work happens before training. A data contract should state the intended task, eligible source material, annotation rules, exclusions, ownership, retention, evaluation set, and the behavior the system must not learn.

A model can only be adapted safely when the desired behavior is observable. For a support assistant, that may mean correct routing, grounded answers, and appropriate escalation—not merely a more confident tone. The evaluation set should include difficult examples, out-of-scope requests, and examples where the model is expected to abstain.

Parameter-efficient methods make iteration cheaper, but they do not repair a vague target. LoRA freezes the pre-trained weights and learns low-rank updates; the method reduces the training footprint while preserving a base model (Hu et al.). The data contract is what decides whether those updates encode useful specialization or noise.

I would treat data quality as an engineering deliverable: version it, sample it manually, document transformation steps, and protect the held-out test set from prompt and training leakage. The model choice comes after that foundation.

The contract should also describe representation. Which languages, institutions, customer segments, or document formats are present—and which are missing? A training corpus can be legally collected and still be unfit for a broad deployment if it systematically underrepresents the situations the product will face.

Before spending on training, run a baseline evaluation with the unadapted model and a retrieval-assisted alternative. This prevents fine-tuning from becoming a default answer to a knowledge problem. If the baseline already meets the quality floor, the simpler system may be easier to update, explain, and retire.