You can’t subtract your way to quality

An attractive but ultimately flawed mental model for the effective use of LLMs for programming is that of a potter, who throws clay on the wheel, and cuts material away until only the necessary remains. The “Potter” mentality is held, if not in name, by those who argue that subpar generated code along the journey is simply a necessary step in the process, and is immaterial, as it will be removed later. It takes the form of -

Being a conscientious engineer,

Having the scars of reviewing and introducing much thoughtless LLM code, adopting a heavy-handed and intentional approach to crafting LLM plans,

Executing the plan, incorporating small course corrections, often by hand, and producing x amount of code,

Thoughtfully evaluating the generated feature, through functional and non-functional lenses, always considering how stakeholders will receive the work,

Iterating by hand and with LLMs to resolve any gaps. The amount of code is now approaching 2x, but the feature is robust and can be confidently communicated with product stakeholders, and with some timidity, other engineers,

With quality in mind and product needs satisfied, the engineer now surveys and refactors their code with the Potter mentality. Much is removed and improved, but some hardened pathways generated by the LLM are challenging to unravel without greater investment, and are left “for the time being”.

This surely isn’t the worst of all possible approaches to LLM use. But unlike the Potter, who knows a priori that they will be crafting a bowl, or a plate, or a vase, as programmers we work in a place of vastly higher uncertainty, and must anticipate our need to add as well as subtract in the pursuit of quality. And if we must add, then we must build in a way that is conducive to adding with quality. This is the space where the Potter, who builds only to subtract, cannot surpass the engineer who builds with both addition and subtraction always front of mind.