There's a size past which a model stops doing good work, and it's smaller than you think. Hand it a task that spans twelve files, four constraints, and three loosely related goals, and the output doesn't fail loudly. It fails quietly: a constraint you stated up top gets dropped, the code contradicts a decision made a page earlier, or the model gets eighty percent of the way through and loses the thread on the last, most important part. The problem is almost never capability. It's that the task was too big to hold clearly at once, and attention spread thin across a bloated window.
The symptoms of an over-scoped task
You can diagnose this without any instrumentation. The failure modes are distinctive, and once you've seen them a few times you recognize them on sight.
- It forgets a constraint you stated earlier. You said "do not add dependencies" in the first paragraph, and forty lines of context later the model reaches for a new library as if you never spoke. The instruction is still technically in the window, but it has been crowded out of effective attention by everything piled on top of it.
- It contradicts itself. Early in a long response the model commits to one approach, then partway through switches to an incompatible one, and never reconciles the two. In a tight task it would have held both halves in view. In a sprawling one, the beginning has effectively scrolled off.
- It does eighty percent, then loses the plot. The first several files come out clean. The last one, the one the whole task was actually for, comes out rushed, generic, or wrong. The model spent its working memory getting through the volume and had nothing left for the part that mattered.
None of these are prompt-wording problems, and rephrasing won't fix them. They're load problems. The task exceeded what the model could hold with enough headroom left over to reason well, and quality is the first thing that gets sacrificed under that load.
Cut work into single-concern units
The fix is decomposition: break the large task into smaller units, each with one clear concern, each small enough to fit comfortably in the window with room to spare for actual reasoning. Not room to barely fit. Room to spare. The reasoning is the point, and reasoning needs slack.
The test I apply to a unit is whether I can state its job in one sentence without using "and." "Add the dedup check to the email-sync parser" is a unit. "Add the dedup check, refactor the retry logic, and update the index format" is three units wearing a trench coat. Each "and" is a seam where the work naturally splits. When you find yourself listing concerns, you've found your unit boundaries.
A well-scoped unit has a few properties. It touches a small, nameable set of files, ideally one or two. It has a single clear definition of done, so you can tell at a glance whether it worked. It carries only the constraints that actually bear on it, not the entire project's rulebook. And it produces something concrete you can inspect before moving on, rather than a partial change tangled up with three others you can't evaluate independently.
Sequence so each output feeds the next
Decomposition isn't just chopping a task into arbitrary pieces. The units have an order, and the order matters, because a good sequence means each unit's output becomes clean input for the next.
Design the interface before the implementation, so the implementation unit starts from a settled contract instead of inventing one on the fly. Get the data model right before the code that reads it. Write the function before the tests that exercise it, or the other way around if you're working test-first, but pick the order deliberately so each step stands on a finished foundation rather than shifting sand. When one unit's output is the next unit's starting context, you never ask the model to hold the whole chain in its head at once. It holds one link, finishes it, and hands a solid result forward.
This is also where the model choice comes in. I tend to plan the decomposition and sequence with a high-capability model, Opus or Fable, then let a fast model like Sonnet execute the individual well-scoped units, because once a unit is small and clearly specified the bulk work doesn't need the heavier reasoning. The expensive thinking goes into cutting the task correctly. The execution of each clean piece is cheap.
Keep only the relevant context in view
A scoped task and a scoped context window go together. There's no point decomposing the work if you then drag the entire codebase along for a two-file change. The eight irrelevant files you pasted "just so it has the full picture" don't help the model; they compete with the two that matter and pull attention sideways. This is the same discipline as giving the AI the context it actually needs, applied at the level of a single unit: include what bears on this unit, leave out everything else.
For a change to the email-sync dedup logic, the relevant context is the parser file, the message-ID schema, and the one constraint about how duplicates are keyed. It's not the LinkedIn inbox processor, the planner agent, or the CMS routing layer, however much those all live in the same repo. Bringing them into the window doesn't make the model more informed. It makes it more distracted, and it costs you tokens for the privilege. Scope the context to the unit, and the unit's signal stays sharp.
When to start fresh versus continue
Long sessions accumulate silt. Even when each individual unit is well scoped, a chat that has run through eight of them in sequence is carrying the residue of all eight, and by the tenth the early ones are dead weight that dilutes attention and can pull the model back toward a decision that no longer applies.
The rule I use is about relatedness, not length. If the next unit genuinely builds on what just happened, and the accumulated context is load-bearing for it, continue in the same window. If the next unit is a fresh concern that doesn't need the prior history, start a new context and bring in only the specific artifacts it requires. Don't carry a completed, unrelated task forward out of momentum. A finished unit's context is baggage for the next unrelated one, and clearing it is the cheapest quality improvement available. Starting fresh isn't a reset of progress. It's dropping ballast you no longer need.
Planning first is where decomposition comes from
The good news is you don't have to force decomposition as a separate discipline. It falls out of planning. When you write the plan before the code, the plan is the decomposition: a sequence of steps, each with a stated concern and a clear output, is exactly a set of scoped units in the right order. A spec that reads "first settle the schema, then the parser, then the dedup check, then the index update" has already cut the work into windows-sized pieces and sequenced them so each feeds the next. The plan isn't paperwork you produce and set aside. It's the map you decompose against.
There's a cost argument here too, and it's not small. A tight, well-scoped task is cheaper to run, because you're not paying to shovel a whole codebase through the model on every request. It's also more cacheable: a stable, focused context, the same reference files and constraints for a unit, is exactly what prompt caching rewards, and a bloated window that changes every message is exactly what defeats it. Scoping the task well isn't only better output. It's a smaller, faster, cheaper request, and those tend to travel together.
The instinct with a capable model is to hand it the whole mountain and expect it to move. It moves rock better one clean load at a time. Cut the work down to what it can hold with room to think, feed each piece the next one's foundation, and the eighty-percent-then-lost failure mode simply stops happening.