You probably don't need to fine-tune

A post crossed my feed a fortnight ago: “30 fine-tuning interview questions, with answers.” The framing was sharp. “Should we fine-tune?” is a trap question in 2026 interviews, the author argued, because the expected answer is usually “not yet”, and how you get there is the whole test. Fair call. Two lines from it are worth keeping: every fine-tune has to beat the best prompt on the base model, not the lazy one, and you fine-tune the interface while you retrieve the content, because knowledge changes daily and behaviour changes quarterly.

I agree with all of it, and none of it told me what happens the day after a team says yes and someone opens the Bedrock console. The questions underneath were model-agnostic: LoRA, QLoRA, DPO, the method zoo. That’s the gap I want to close. Not another “should you fine-tune” explainer. What the ladder looks like before you touch weights, what “fine-tune” actually decomposes into once you’re inside Bedrock, and what each path costs to run, priced today, not at launch.

The ladder, before you touch weights

Prompt, then RAG, then caching, then fine-tune, in that order, and only past whichever step actually stops solving your problem. Most teams that reach for fine-tuning first are trying to fix one of three things: the model doesn’t know something (that’s retrieval), the model is slow or expensive per call (that’s caching, or a smaller model), or the model’s tone drifts (that’s a better system prompt, most of the time). None of those need a training job.

The three reasons that genuinely do:

1. Structured output you can’t prompt your way to. A rigid schema, a house format, a tool-call convention the base model won’t hold consistently across thousands of calls even with a strong system prompt.
2. Cost or latency distillation. You’ve proven a big model does the job well and you want a small, cheap, fast model to do the same job for a fraction of the price.
3. Narrow domain vocabulary or tone the prompt can’t carry. Not “be more formal”, but a genuine specialist register: clinical shorthand, legal drafting conventions, a product’s own internal taxonomy, the kind of thing a few-shot prompt approximates and a fine-tune nails.

The industry has mostly converged on that ladder. What’s missing from every version of it I’ve read is the other side of the “yes”: once you actually commit, the decision forks twice, once into different job types and again into different ways to serve the result, each with its own bill, and almost everything written about it treats “fine-tuning” as one thing.

”Fine-tune” is four different jobs on Bedrock

Bedrock’s CreateModelCustomizationJob API takes a customizationType parameter, and the values are not flavours of the same thing:

• FINE_TUNING: supervised fine-tuning on your own labelled examples. The classic case: input/output pairs, a training job, a customised model at the end.
• CONTINUED_PRE_TRAINING: unlabelled domain data, for building broader domain adaptation rather than a specific task.
• DISTILLATION: you don’t write training pairs at all. You point Bedrock at a stronger “teacher” model and a weaker “student” model, hand it your prompts or invocation logs, and Bedrock generates the synthetic training set and runs the job for you. General availability landed in May 2025, and AWS’s own published figures put distilled models at up to 500% faster and 75% cheaper than the teacher, with under 2% accuracy loss on retrieval-style tasks. If your actual reason to fine-tune is reason 2 above, cost and latency distillation, this is very often the more direct route to it than hand-rolled supervised fine-tuning, because you skip building the labelled dataset entirely.
• REINFORCEMENT_FINE_TUNING: the newest of the four, Nova-only today. You supply a reward function, either a Bedrock model acting as judge or your own Lambda grading logic, and the job trains against that signal instead of a fixed answer key. It’s the most direct fit for reason 1 above, structured output and tool-calling reliability, because you can grade “did the tool call parse” programmatically rather than needing labelled pairs.

Serving the result forks again

Bedrock’s own documentation still says, plainly, “if you customized a model, you must purchase Provisioned Throughput to be able to use it.” Read literally, and read against an older doc page in isolation, that sounds like the whole story. It isn’t, and the gap between the two is exactly the kind of thing that’s easy to get wrong reading one page instead of the console.

Since 16 July 2025, a second path exists: CreateCustomModelDeployment, which AWS explicitly describes as complementing Provisioned Throughput, not replacing it. Deploy a natively customised model this way and you pay the base model’s own on-demand token rate, no reservation, no hourly floor. The catch is eligibility, and it’s narrow: only Amazon Nova Lite, Nova 2 Lite, Nova Micro, Nova Pro (all us-east-1) and Meta Llama 3.3 70B Instruct (us-west-2), and only if the underlying model was customised on or after 16 July 2025 for Nova, or 15 September 2025 for Llama. Everything else, every Titan customisation, every model customised before those dates, every region outside those two, still has exactly one way to serve it: Provisioned Throughput.

So the real shape is three serving paths, not two:

1. Provisioned Throughput: universal. Works for any customised model, any region Bedrock customisation supports. Reserved capacity, billed hourly, whether a request arrives or not.
2. Custom Model Deployment: on-demand, same per-token price as the base model, but only for the specific Nova and Llama models above, customised after their respective GA dates.
3. Custom Model Import: for models you trained entirely outside Bedrock (SageMaker, your own PEFT or LoRA run) in a supported open-weight architecture, imported and served on demand, billed per Custom Model Unit-minute, scaling to zero when idle. More on this one below.

The Provisioned Throughput numbers, priced today

If your model or region falls outside Custom Model Deployment’s narrow eligibility, this table is what you’re actually pricing. Pulled from the AWS Price List API for us-east-1, effective 2026-08-01, in US dollars per model unit per hour:

Model	No commitment	1-month commitment	6-month commitment
Titan Text Lite	$7.10	$6.40	$5.10
Titan Text Express	$20.50	$18.40	$14.80
Titan Text Premier (custom)	$32.25	$27.95	$16.38
Nova Micro	$60.50	$55.00	$30.25
Nova Lite	$60.50	$55.00	$30.25
Nova Pro	$60.50	$55.00	$30.25
Nova Canvas	$60.50	$55.00	$30.25

Read that Nova block again. On demand, Nova Micro costs a small fraction of Nova Pro per token, that’s the entire point of the model family. Provisioned, they are the same price. If you’re on Provisioned Throughput at all, whether because your model doesn’t qualify for Custom Model Deployment or because you deliberately want reserved, latency-consistent capacity, the tier you picked to save money on inference is invisible once you reserve capacity for it: you’re renting a model unit, not a token budget, and AWS prices the unit the same across the family regardless of which model is sitting on it. This is exactly why Custom Model Deployment matters when it’s available: on Nova Lite, Micro or Pro, customised after 16 July 2025, in us-east-1, you skip this table entirely and pay Nova’s own on-demand token price instead.

There’s a third path that doesn’t use CreateModelCustomizationJob at all: Custom Model Import. Train an open-weight model yourself, anywhere (SageMaker, your own PEFT or LoRA run, wherever), then import the resulting weights into Bedrock for a supported architecture (Llama and Mistral families among them). Bedrock serves it on demand: billed in five-minute windows by Custom Model Unit, currently $0.05718 per CMU-minute in us-east-1, auto-scaling to zero when nothing has invoked it for five minutes. The trade-off is cold start: AWS and early adopters report anywhere from several seconds to a couple of minutes on the first request after the model has scaled to zero, depending on model size, so this isn’t a free lunch against Provisioned Throughput for a genuinely latency-sensitive workload, it’s a different point on the cost-versus-latency curve.

What that actually costs, worked through

Illustrative maths, not a measured production bill, built from the table above and the live Custom Model Import rate:

Steady, 24/7 traffic, one model unit, one month (730 hours):

• Titan Text Lite, native fine-tune, no commitment: 730 × $7.10 = $5,183/month. Six-month commitment: 730 × $5.10 = $3,723/month, but that’s a six-month lock-in, $22,338 total, whether traffic shows up or not.
• Custom Model Import, a small Llama-class model needing 2 CMUs, run flat out: 730 × 60 × 2 × $0.05718 = $5,009/month. Roughly the same as the Titan no-commitment price, with zero commitment, plus whatever cold starts cost you in latency along the way.

Bursty traffic, same imported model, genuinely invoked 4 hours a day:

• Custom Model Import: 4 × 30 × 60 × 2 × $0.05718 = $823/month.
• Provisioned Throughput can’t do this at all. You are renting capacity, not metering usage, so the bill is identical whether the model answers one request that day or ten thousand. Custom Model Deployment, where it’s eligible, can: it’s metered like the base model, not reserved like Provisioned Throughput.

That’s the actual decision hiding inside “should we fine-tune”: it is two decisions stacked as one. Do you need a customised model, and can you commit to buying dedicated, always-on capacity for it, or does your model and region qualify for one of the on-demand routes instead. Most of what’s written addresses only the first.

How you’d know it worked

Whichever job type and serving path you take, the trigger post’s baseline rule is the right one, and I’d go further: don’t trust your own eyeball on “it feels better,” measure it. A fine-tune, a distillation, or an RFT run has to beat the best prompt on the base model on a real evaluation, the same way, every time you change it. That’s not a throwaway line. I spent the last four posts on exactly how that measurement goes wrong: an LLM-as-judge score isn’t a fixed property of the output you’re grading, it shifts depending on what sits next to it in the batch, and a continuous quality signal doesn’t come with a threshold until you derive one from your own traffic. If you’re leaning on an LLM judge to decide whether the customised model earned whatever it costs to serve, the same failure modes apply to that decision as to any other regression gate.

Before you commit to serving anything, you should be able to answer three questions with numbers, not confidence: what does the best prompt on the base model score, on the same eval, today; what does the customised model score; and what is the smallest sample size at which that gap is actually distinguishable from noise. If you can’t answer the third one, you don’t know if the first two differ at all.

If you’ve actually shipped a fine-tune on Bedrock, which serving path did you end up on, and did Provisioned Throughput’s bill, or the narrower eligibility for the on-demand routes, change the decision after the fact rather than before it?