Article 6 of 6 · September 2026
What fine-tuning changes, and what QLoRA is
Article 6 of 6 · the stage you can afford to run yourself · after Andrej Karpathy, and after Dettmers, Pagnoni, Holtzman and Zettlemoyer
The first four articles were Andrej Karpathy's. This one starts from him and then walks one step further than his lectures go. His Deep Dive into LLMs like ChatGPT explains the three training stages and what the middle one, fine-tuning, actually does. It does not cover the technique called QLoRA. For that we go to the people who invented it: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman and Luke Zettlemoyer, whose 2023 paper is the source for everything in sections 4 and 5, and to Edward Hu and colleagues, whose 2021 LoRA paper it builds on. Where this article states a number, it is theirs.
1. Where fine-tuning sits
Recall the school picture from article 2. Reading the exposition is pretraining: months, thousands of computers, a copy of the internet. Studying worked examples is the second stage: the model reads a pile of question-and-answer conversations and learns the shape of a good answer. Practice problems are the third.
Fine-tuning is the second stage, done by you. Karpathy is explicit about what it is: the same guessing game, the same knobs, a different pile of text. Nothing else changes. That is why it is cheap compared with pretraining. He gives the rough scale: months for the first stage, hours for the second.
It helps to say what fine-tuning changes and what it does not, using article 1's two memories. Fine-tuning edits the month-old book: the knobs, what the model carries with it into every conversation. It does not touch the desk. So it is the right tool for things that should be the same in every conversation: the shape of the answers, the tone, the habits, the vocabulary of a field, the format you always want. It is the wrong tool for facts you could simply paste in. Karpathy's own comparison holds: what sits in the knobs is a vague recollection, and what sits on the desk is exact.
2. Why bother, when the desk exists
A fair question. If pasting works so well, why train anything?
Three reasons, all in his lectures. The desk is finite. He calls the context window a precious resource, and every instruction you repeat at the start of every conversation is space and time spent again and again. Fine-tuning bakes the habits in, so the desk is free for the actual work.
The second reason is that habits taught by a pasted instruction are followed only as well as the model reads them each time. Habits learned through training are the model's own. The morning wipe from article 1 does not touch them.
The third is where computing is going. Karpathy describes the current arrangement as the 1960s: the big models live in the cloud and we all share them over the network, because they are too expensive to own. He also notes that small machines are starting to run smaller models locally, and that anyone can download a model and run it on a laptop today. A small model that has learned your habits, running on your own machine, is one of the first things that becomes possible when that arrives.
3. The cost problem: the knobs are heavy
Here is the wall you hit the moment you try.
A knob is a number, and the knobs are stored as numbers in a computer's memory. Stored the usual way for training, each takes two bytes. A model with 65 billion knobs, the largest in the QLoRA paper, needs about 130 gigabytes just to hold its knobs, before any training happens. Training needs several more copies for the bookkeeping, some of them twice that size: the nudges to apply, and the optimizer's running notes about each knob. The QLoRA paper's own figure for ordinary 16-bit fine-tuning of that model is more than 780 gigabytes. The graphics cards that do this work have far less memory than that. The QLoRA paper's own headline card has 48 gigabytes. You cannot even load the knobs onto it, never mind train them.
So the problem is not cleverness. It is memory. Two ideas, one from 2021 and one from 2023, attack that problem from two sides.
4. LoRA: do not move all the knobs, train a thin correction
The first idea, from Edward Hu and colleagues at Microsoft, is LoRA, for low-rank adaptation.
Do not retune the whole desk. Freeze the big model exactly as it is. Beside each bank of knobs, bolt on a small correction panel, and train only the panel. Mathematically, the correction is a pair of small matrices whose product adjusts the big one, and "low-rank" is the name for how thin that pair is. The number of knobs you actually train drops by a very large factor; the paper's claim for GPT-3 is a reduction of ten thousand times in trainable parameters and about three times in the memory needed to train.
Two things follow. The original model is untouched, so you can keep several corrections and swap them in and out for different jobs. And when you are done, the correction can be folded into the original, so using the finished model costs nothing extra.
5. The Q: keep the big model compressed while the thin correction learns
LoRA shrinks what you train. It does not shrink what you have to hold: the frozen model is still there, in full, at two bytes a knob. QLoRA's move is to compress it.
Dettmers and his co-authors store the frozen model's knobs in four bits instead of sixteen, a quarter of the memory. They designed a four-bit number format for it, called NF4, which is shaped to fit the way the knobs' values are actually distributed, most near zero and few far out. They then compress the compression itself, the small constants that the four-bit format needs, which they call double quantization. And they handle the memory spikes that appear during training with paged optimizers, which move the optimizer's notes out of the way when the card is full.
The training signal flows backwards through the compressed, frozen model into the LoRA correction panels, which are the only thing that learns, and which learn at 16-bit precision, the same precision ordinary fine-tuning uses. The abstract's result: a 65-billion-knob model fine-tuned on a single 48-gigabyte card, while, in their words, preserving full 16-bit finetuning task performance. Their demonstration model family, Guanaco, reached 99.3 percent of ChatGPT's level on one benchmark, the Vicuna benchmark, after 24 hours of fine-tuning on a single GPU.
Where this picture muddies. That 99.3 percent is one benchmark, judged one way, in 2023. It says the technique does not throw performance away; it does not say a home fine-tune equals ChatGPT. And the word "quantized" describes how the frozen knobs are stored. It does not describe the correction, which is what you are training.
6. The family keeps growing
The LoRA idea has cousins. One worth knowing is DoRA, from Shih-Yang Liu and colleagues at NVIDIA in 2024. It splits each bank of knobs into a size and a direction, applies the low-rank correction to the direction only, and claims to close some of the remaining gap to full fine-tuning without costing anything extra when the model is used.
7. What you would feed it
Now the part that has nothing to do with hardware and everything to do with the first four articles.
The pile of text you fine-tune on is worked examples: questions, and the answers you want, in the shape you want. Karpathy's account of how the labs built the assistant stage is the recipe. People wrote ideal answers following written instructions; the model learned their shape. When you fine-tune, you are those people, and the writing instructions are yours.
Three rules from the earlier articles apply directly. From article 3: show the work before the answer, in small steps, or you will teach the model to guess in one word and rationalise after. From article 1: the model will imitate the confident tone of whatever you write, so if you want it to say "I don't know", you must write examples that say it. From article 5: the dataset is the document in the middle. Every example can be read, checked and fixed by a person before a single knob moves, which is exactly the leash he describes for a course, applied to training.
How many? Karpathy's figures for the labs run from about a hundred thousand conversations to millions, largely machine-assisted today. For a habit, not a body of knowledge, the smaller end is where people start. Quality decides more than quantity, because every example is a worked solution the model will imitate, mistakes included.
What to do tomorrow
- Try the desk first. A clear standing instruction plus pasted documents solves most of what people imagine they need training for, and it costs nothing.
- If a habit is stable and you repeat it every day, start collecting. Real questions, the answers you wish you had received, work shown before conclusions.
- Expect a fine-tune to change habits, not knowledge. Facts stay on the desk. The Swiss cheese stays.
- Keep the human check. For style and judgment there is no answer key, so the check is a person reading. That is Karpathy's verifiable-versus-unverifiable line, and it lands on you.
We are about to run this stage on our own record: a small model and a few hundred examples of how we work. We will write down what happens, including the parts that fail.