Fine-tuning

The problem fine-tuning solves is very direct: powerful pre-trained models already exist, so how do you make one specialize in the exact thing you want to do?

First, look at the path without fine-tuning. Training a large model from scratch requires massive data, thousands of GPUs, and months of time—most people and teams simply cannot afford it. Fine-tuning takes a different approach: take someone else’s pre-trained model as the starting point, use your own small batch of data, let it continue training for a little while, and the weights are only lightly adjusted. After this step, a generalist becomes a specialist.

More precisely, fine-tuning has three inputs: a pre-trained model, samples collected for the target task, and a training configuration (learning rate, number of training epochs, etc.). The output is adapted weights, or an adapter that can be attached to the original model. The training process itself is not mysterious: it still computes a task loss and still updates parameters with gradients. The only differences from training from scratch are two: the starting point is not random weights, but pre-trained weights; and the training data no longer covers everything, but is concentrated on your task. Because the starting point is already close to a solution region, only a small step is needed.

How should the results be interpreted? The most important judgment criterion is the held-out task metric: if the fine-tuned model’s performance on the target task improves, it means the model is better suited to the current task. At the same time, note the other side: if performance on general capabilities or other language slices declines, it means this specialization is accompanied by regression—the model is transferring capability from elsewhere to your task. An ideal fine-tuning is one where the target metric improves while losses on unrelated capabilities are as small as possible.

The applicable boundary of this path is equally clear. When there is too little data, the task distribution differs too much from what the pre-trained model has seen, or the base model simply does not know the task, a “light nudge” may not be enough. Fine-tuning can efficiently channel existing capabilities toward new uses, but it cannot teach a model something it has never learned out of thin air.

So in one sentence: fine-tuning is not “building a new model,” but “pushing an existing good model one small step in your direction.”

The most critical question about fine-tuning is: why is it enough to “continue training briefly” on a small batch of data, without having to start learning from scratch?

The answer is that the pre-trained model has already learned a large amount of general, reusable things. During pre-training, language models master syntactic structure, common sense, and basic reasoning; vision models master low-level features such as edges, textures, and shapes. These capabilities are useful for almost all downstream tasks and serve as the foundation for any specific application. The specialized thing you want to master often only requires a small adjustment on top of these general capabilities—nudging the existing representations toward your target direction. For example, a model that has read vast amounts of text already knows the word order rules of “subject–predicate–object” and the meanings of a large number of words. When you teach it sentiment classification, the gradients no longer need to rediscover these rules; they only need to slightly shift the existing word-meaning representations toward the “positive/negative sentiment” direction.

This idea of “moving capabilities learned in one domain to another task” is called transfer learning. Its input is the parameter representation learned in the source task (pre-training), plus a small number of target-task samples; its output is a decision boundary or generation behavior more appropriate for the target task. At the parameter level: during fine-tuning, gradients preferentially adjust the representations and output directions most relevant to the current sample, rather than rediscovering all language rules from scratch. This is the entire source of its savings in data and compute—the most expensive step, pre-training, has already been done for you by others. Figure 1 expresses exactly this relationship: transfer learning is doing small-scale fine-tuning on top of massive pre-trained capability.

The effectiveness of transfer has clear dependency conditions. The closer the target data is to the coverage of the pre-training corpus or images, the more reusable the base representations, and transfer typically requires fewer samples. Conversely, if the domain, modality, or meaning of labels differs greatly, negative transfer can occur—transfer not only fails to help, but the final performance may even be worse than a simpler specialized solution. By analogy, a knowledgeable person changing careers does not need to go back to elementary school; they only need a period of specialized training for the new position to get started. Fine-tuning relative to a pre-trained model is exactly this specialized training. But the premise of specialized training is that the old knowledge and the new position are truly connected; if the new position requires completely unfamiliar skills, this training has no foundation.

This is the most practical—and most easily misused—section of the page: which problems should use fine-tuning, and which should not?

One-sentence watershed: fine-tuning is good at changing “behavior”—format, tone, style, way of doing things; it is not good at reliably stuffing in “lots of new facts”—that is better suited to external retrieval (RAG). Along this watershed, common needs can be placed one by one.

If you want it to consistently output only a certain format, such as JSON, you should fine-tune. Format is a behavioral habit; with enough demonstrations it becomes fixed in the weights. If you want it to answer with a certain fixed tone or style, you should also fine-tune: style is trainable behavior. If you want it to master the way of expressing a particular professional domain, that also goes to fine-tuning—here what is adjusted is “how to say it,” not “which facts to remember.” The common point of these three types of needs is: they change the way the model does things, not what the model knows.

Conversely, if you want it to know your company's latest documents or data, you should use RAG. Facts need to be updateable and traceable; retrieval can swap knowledge sources at any time and provide sources, while cramming facts into weights both makes them easy to forget and may crowd out other capabilities (Section 6's “forgetting” expands on this). If you only want the model to do as required this once, the lightest choice is prompts and in-context learning: no training, fast iteration, but each call still has token and evaluation costs, and it cannot be said to be solidified.

The most common misuse scenario is “I want the model to know our company's knowledge”—many people's first reaction is fine-tuning, but most of the time they should use RAG: company knowledge keeps changing, retrieval can update at any time and provide sources. Pouring a pile of facts into weights doesn't learn them solidly and they become outdated. So you can remember a rule of thumb: facts that can't be remembered go to retrieval; habits that can't be changed go to fine-tuning.

When making a choice, the inputs should be the type of need, knowledge update frequency, traceability requirements, call volume and latency budget, and the output is a solution chosen from prompts, RAG, fine-tuning, or a combination. When interpreting results, also keep the evidence boundary: an improved format pass rate only proves more stable behavior, not that factual coverage is more complete. Behavior and knowledge often appear together, so a real system can completely combine the two—use fine-tuning to fix the output format, then use RAG to provide updateable evidence.

The previous section said fine-tuning is good at changing “behavior.” One of the most important behavior changes is something you use every day—this is how conversational assistants come about.

A base model that has just finished pre-training can only “continue writing.” If you ask it a question, it may continue your sentence with more questions instead of answering. This is a direct product of the pre-training objective: the task it learned on massive text is to predict the next word, not to respond to questions. Instruction fine-tuning (SFT) is a supervised fine-tuning step specifically targeting this behavior: it uses a large amount of “instruction → ideal answer” demonstration data to train it, teaching it the behavior of “answer when asked.” After this step, the model changes from a “base model that can continue writing” into an “assistant that can respond.”

From a mechanistic perspective, instruction fine-tuning is standard supervised learning: here the “instruction” is the input, the “ideal answer” is the label, and during training cross-entropy is used to gradually increase the probability of each token in the demonstration answer (this set of mechanisms is detailed in the “Supervised Learning” deep-dive page). Therefore, strictly speaking: instruction fine-tuning = supervised learning performed on a pre-trained model. It does not introduce any new algorithm; it only takes the pre-trained weights as a starting point and applies the supervised learning objective.

After instruction fine-tuning, there is often another step of preference alignment (RLHF or DPO): using human preferences for “which answer is better” to make the model more helpful and safer. These three steps—“pre-training → instruction fine-tuning → preference alignment”—are the origin of today’s conversational assistants (see Section 8 of the “Large Language Model” deep-dive page for details).

When interpreting the results of instruction fine-tuning, you must maintain a boundary: an increase in the instruction-following rate means the model more often reproduces the norms in the training demonstrations; it does not mean the facts in its answers are necessarily correct—errors in the demonstration data will be learned as-is. Situations not covered by the demonstrations, conflicting demonstrations, and incorrectly written refusal criteria in the demonstrations will all be inherited by the model as defects. Preference alignment is likewise only the next stage in the pipeline and cannot replace independent safety evaluation and factuality evaluation.

A practical question: if a model has several hundred billion parameters, does fine-tuning it once require changing all parameters, and is it still expensive? The good news is no. Since fine-tuning was originally only meant to “push it a small step,” there is no need to touch all weights.

The idea of parameter-efficient fine-tuning (PEFT) is to freeze the vast majority of the original parameters and train only a small number of newly added parameters. The most popular approach is LoRA—attach two low-rank small matrices alongside the original weights, train only them, and use them to represent “the small change to be added to the original weights.” “Rank” can first be understood as the number of mutually independent directions of variation in a matrix; the low-rank hypothesis assumes that task adaptation requires only a few such directions, so there is no need to arbitrarily change every degree of freedom of the original weights. Figure 2 contrasts the two approaches: full fine-tuning updates the entire weight block, which is expensive and requires storing a whole model for each task; LoRA freezes the original weights and trains only the two low-rank small matrices beside them. Trainable parameters can often drop to less than 1%, GPU memory is greatly reduced, and one base model can have multiple “adapters” attached, switched on demand.

5.1 Worked Example: How a rank-1 LoRA Changes the Output

Use a 2×2 toy weight to walk through a low-rank update. Let the base matrix W = I (the identity matrix), and let the LoRA adapter be formed by multiplying a column vector B = [0.2, 0.1]ᵀ and a row vector A = [1, −1]. The weight increment ΔW = B × A is a 2×2 matrix of rank at most 1: W is the frozen base weight, ΔW is the increment learned by the adapter; A compresses the input into one low-dimensional direction, and B expands that direction back out to the output space. Here A has only one row and B has only one column, so the product can provide at most one independent direction of change, written as rank(ΔW) ≤ 1.

Now concretely compute the result for input x = [3, 1]. The base output is Wx = [3, 1], because W is the identity matrix. Adapter increment ΔWx: first A compresses x into a scalar A·x = 1×3 + (−1)×1 = 2, then multiplying by B gives [0.2×2, 0.1×2] = [0.4, 0.2]. Combined output (W + ΔW)x = [3.4, 1.2]. You can see that the adapter applies only a limited correction along the single direction of “increase the first dimension, decrease the second,” without having to change W itself.

During training, only A and B are updated; W always remains frozen. At deployment time, you can dynamically attach the adapter, or you can merge ΔW back into W and freeze it together. Real LoRA also has a scaling factor, target layer selection, and dropout, which all change the strength of the effective update, but the structure of the low-rank bypass remains unchanged.

This also explains why such methods are the default choice in practice: they save GPU memory, train quickly, and enable “one base model + multiple pluggable adapters”—to switch tasks, just swap in a small matrix, without having to store a full-size large model for each one. For most teams, parameter-efficient fine-tuning is the starting point; full fine-tuning is used only when necessary.

Fine-tuning looks simple, but the failures are concentrated in a few specific places.

The first pitfall is overfitting. Fine-tuning data is often scarce, and the model can easily memorize those few hundred samples, losing generalization ability—it answers beautifully on training samples but falls apart on a similar unseen question. The countermeasures are lowering the learning rate, early stopping, adding regularization, and not training for too many epochs (see the deep-dive pages on “Overfitting” and “Regularization” for mechanisms).

The second pitfall is catastrophic forgetting. By focusing single-mindedly on learning a new task, the model forgets the general capabilities learned during pre-training: it becomes able only to output JSON, and everything else degrades. The countermeasures are switching to parameter-efficient fine-tuning (freezing most of the original parameters so that a smaller change surface means less forgetting), further lowering the learning rate, and mixing some general-purpose samples into the training data to remind the model that old skills still need to be preserved.

The third pitfall is data quality, and it matters more than quantity. The effectiveness of fine-tuning depends heavily on the quality of the demonstration data: a few hundred high-quality, consistently formatted samples often beat a large pile of noise. Dirty data will be faithfully learned by the model—incorrect formats and wrong refusal criteria in the examples will be transferred into the model's behavior unchanged.

Here is a counterintuitive point: fine-tuning is not “the more you feed it, the better.” Dirty data, large volume, and long training are often worse than small and refined, because you are teaching it to “follow these (bad) examples.” The correct order is to first polish a few hundred samples until they are clean, and only then talk about scale.

The inputs to the risk check are training curves, an independent task set, a general capability baseline, and a stratified data audit; the output is the localization of overfitting, forgetting, or data defects. Several signals map directly: training loss keeps falling while validation set worsens—that is overfitting; target task improves while general set declines—that is forgetting; errors concentrate on paraphrasing the demonstration wording—then fix the data first. Low learning rate, early stopping, mixing in general-purpose samples, and parameter-efficient fine-tuning can all reduce risk, but they cannot replace an independent held-out set—no matter how you guard against it, the final conclusion can only come from data the model has not seen.

When facing a real requirement, what should the first step be? The answer is usually not 'fine-tune right away.'

View the three methods as a ladder from light to heavy, starting with the most economical and moving up only when it's insufficient. The first level, lightest: prompt and in-context learning. If adjusting wording and providing a few examples is enough, stop there—no training needed, and iteration is fastest (for the mechanism, see the 'In-context Learning' deep-dive page). The second level, in the middle: RAG retrieval. If what's missing is facts or up-to-date knowledge, use external retrieval and don't stuff those into the weights. The third level, heaviest: fine-tuning. Only when you need stable behavior, format, or style, and prompts can't stabilize it and call volume has risen, does fine-tuning become relevant.

The practical order is therefore: first use prompt and in-context learning to quickly verify 'whether the model can do this at all'; add RAG if facts are missing; only when you need to solidify a certain behavior, and you need long-term stability while saving per-request cost, does fine-tuning truly become worthwhile. In one sentence: fine-tuning is the endpoint, not the starting point.

The input to this ladder is the same set of representative requirements, quality thresholds, and total cost constraints; the output is the lightest solution that meets the threshold. Comparison must be done on the same test set: quality, latency, per-request tokens, training and maintenance costs must all be considered together. The heavier the solution, the greater the burden of version management and regression testing, so 'stop when it's good enough' itself saves money. Of course, this is the default trial order rather than an absolute rule: in scenarios of strict offline use, extremely low latency, or high call volume, fine-tuning may become economical earlier.

Here is how the entire causal chain from beginning to end is connected.

Training from scratch is too expensive, so we take a pre-trained model as the starting point and continue training for a short while with a small amount of data—this is fine-tuning. It is cheap and effective because pre-training has already learned general capabilities, and fine-tuning only needs a small adjustment; this is transfer learning at work. Precisely because of this, fine-tuning is good at changing 'behavior/format/style', while cramming 'a large amount of new facts' should be left to Retrieval-Augmented Generation (RAG): behaviors can be solidified through demonstration, but facts must be updatable and traceable.

The most important behavior modification on this chain is instruction fine-tuning (SFT): using 'instruction → ideal answer' to teach the base model to become an assistant that follows instructions, essentially a supervised learning exercise, and then through preference alignment it takes today's conversational assistant form. Since we only push a small step, we do not need to modify all parameters—LoRA freezes the original weights and trains only the small matrices attached on the side, saving memory and being pluggable, making 'pushing a small step' truly feasible in engineering. But fine-tuning is not without risks: overfitting, catastrophic forgetting, and poor data quality—these three pitfalls all arise from this very characteristic of 'data being scarce and narrow, behavior being specialized', and require a low learning rate, early stopping, mixing in general samples, and a held-out set as safeguards.

Therefore, when facing a new requirement, the endpoint of the causal chain is a sequence of choices from light to heavy: prompt first, then RAG, and only finally fine-tuning—first verify feasibility, then supplement facts, and finally pay the training cost only for behaviors that truly need to be solidified.

The passing standard is thus also clear: being able to clearly explain 'why fine-tuning is cheap (transfer learning)' and accurately state 'what should be fine-tuned and what should use RAG' captures its most practical core.

Before studying this page, you need to have these concepts: pre-training (where the starting point of fine-tuning comes from), supervised learning (the essence of instruction fine-tuning), large language models (the most frequently appearing application object on this page), and neural networks and weights (what training actually changes).

The core concepts of this page are: transfer learning (why fine-tuning is cheap and effective), instruction fine-tuning SFT (the step that tunes the base model into an assistant), parameter-efficient fine-tuning and LoRA (the engineering implementation that does not require changing all parameters), catastrophic forgetting (one of the most typical risks of fine-tuning), and the division-of-labor boundaries among fine-tuning, RAG, and prompting.

The natural next steps extending from these core concepts are overfitting and regularization (means of controlling fine-tuning risk), in-context learning (the lightest rung on the ladder), retrieval-augmented generation RAG (a fact-supply solution complementary to fine-tuning), and alignment and RLHF (the step after instruction fine-tuning).

Further learning directions include: quantization and deployment (how the trained model actually goes into production), distillation (squeezing large-model capabilities into a small model), evaluation (how to objectively confirm that fine-tuning is really effective), and model selection (questions that should be thought through before fine-tuning).