Every serious project I ship pins its dependencies. Lockfiles, exact versions, no floating ranges that let a transitive package upgrade itself out from under me overnight. Then, for the first year or so of running AI in production, I did the one thing I would never do with a library: I pointed my agents at a floating model alias and let it resolve to whatever "latest" happened to mean that week. The model is a dependency. It's, in fact, the least stable dependency in the entire stack, and it deserves the same version discipline you already give everything else.
The failure mode is quiet. A model gets upgraded behind an alias, and nothing errors. Prompts that worked start drifting. Output formats shift by a hair. An agent that made reliable decisions for months starts making slightly different ones, and there's no changelog in your repo to blame, because nothing in your repo changed.
Pin the explicit ID, not a floating alias
Most providers give you two ways to name a model. There's a convenience alias that always points at the newest snapshot in a family, and there's a fully qualified version ID that names one specific frozen build. The alias is fine for a throwaway script or a chat window. It's the wrong choice for anything durable.
If you build an agent fleet, a data pipeline, or a product feature on top of a model, name the exact version ID. Put it in config, not scattered through code. In my setup the model ID lives in the same layer as every other environment value, alongside things like \${DATABASE_URL}, so there's exactly one place that decides which build of the model my agents run against. When I want to move, I change one line, on purpose, and I can see it in the diff.
The distinction matters because an alias makes a silent decision for you on the provider's schedule. A pinned ID makes the decision yours, on your schedule. That's the whole game with reproducibility: you want the inputs to a run to be things you chose and can point to, not things that resolved to a moving target at request time.
Record which model produced which output
Pinning stops the model from changing without your knowledge. It doesn't, by itself, let you trace a regression after the fact. For that you need provenance: every meaningful output should carry a record of which model version produced it.
This is cheap to add and painful to retrofit. When one of my agents writes a note, files a record, or makes a classification, the run gets logged with the model ID that did the work. So when I notice that, say, my CRM email sync started categorizing a certain kind of message differently two weeks ago, the first question I can actually answer is "did the model change on that date." If provenance says the version ID is the same across the good runs and the bad runs, the model is exonerated and I go look at my prompt, my data, or my tools. If it changed, I have my culprit in one query instead of a day of guessing.
Without that record you're debugging a regression with one of your prime suspects invisible. You will stare at your own code, which didn't change, and find nothing, because the thing that changed never left a fingerprint. Log the version. Treat it like the commit SHA of the model.
Test a new model before you roll it forward
Pinning isn't an argument for staying frozen forever. New models are genuinely better, and I want them. The point is to adopt them the way you adopt a major dependency upgrade: deliberately, against your real workload, before it touches production.
The mistake is treating a model swap as a config tweak that "should just work." A newer, more capable model isn't a drop-in the way a patch release is. It can be better on average and still break a specific prompt you depend on, because your prompt was tuned, implicitly, to the behavior of the model you wrote it against. Formatting conventions shift. A model that used to reliably return bare JSON starts wrapping it in prose. An extraction that was exact gets slightly more liberal. On average it improved; on your one load-bearing case it regressed.
So run the candidate through your actual prompts and your evals before you move the pin. If you've built a verification loop where AI reviews AI, point it at the new model's output and compare. Watch the cost too, because a more capable default is often more expensive per call, and at fleet scale that shows up fast; that's its own reason to make token spend visible before and after a swap. This is also where matching the model to the task pays off, because you're usually not upgrading one model but reevaluating which build handles planning, which handles bulk generation, and which handles review. Test each role against its real work, not against a demo prompt.
Version drift and non-determinism are two different problems
There's a trap in the word "reproducibility" here, so it's worth being precise. Two separate things make an AI workflow non-reproducible, and pinning only fixes one of them.
Version drift is the model itself changing underneath you. You send the same prompt to what you think is the same model and get different behavior because the build actually changed. Pinning the explicit ID fixes this entirely. Same pinned version, same underlying model.
Non-determinism is variation between calls to the same fixed model. Sampling settings like temperature mean that even a perfectly pinned model can return different outputs for identical inputs. A pinned version doesn't give you byte-identical results run to run; it gives you a stable behavioral distribution. If you need tighter reproducibility for a given task, that's a separate set of levers: lower the temperature, constrain the output format, add a validation and retry step so a stray sampling result gets caught rather than committed.
You need both in view. I've watched people crank temperature to zero and declare their pipeline reproducible while still pointing at a floating alias, which means the ground can still move under them on the provider's next release. And I've watched the reverse: a carefully pinned model feeding a workflow that assumed identical output every time and broke the first time sampling handed it a differently-worded but equally valid answer. Pin the version to kill drift. Manage sampling and validation to handle non-determinism. They're different knobs for different problems.
Keep a deliberate upgrade path
The goal of all this isn't to freeze on one model forever. Frozen is its own kind of debt; you fall behind on capability and cost, and eventually the version you pinned gets deprecated and forces a scramble. The goal is that every model change in your system is a decision with a reason attached.
In practice that means a small, boring routine. New model ships. I read what changed. I run it against my real prompts and evals in a non-production configuration. I look at behavior and at cost. If it's better for a given role, I move that pin, note the date, and watch the provenance logs for anything that shifted. If it isn't, I stay put and revisit next release. The pin isn't a wall; it's a gate that only opens when I open it.
That's the throughline. The model is the least stable dependency in your stack: it's the only one that can be swapped without your consent, on a schedule you don't control, with no changelog in your repo and no stack trace when it drifts. So give it the discipline you already give the boring dependencies. Pin the exact version. Record what produced each output. Test before you roll forward. Upgrade on purpose. None of it's exotic; it's the same version hygiene you've practiced for years, finally applied to the one dependency that needs it most.