Skip to content

The agent can order the tomatoes. It just can’t know why you won’t eat them in December.

First published on LinkedIn on 24 March 2026.

A note on method: This article was developed using LLM-augmented argumentation. Multiple LLM conversations were used to build, stress-test, and fact-check the structural analysis. The conclusions are the author’s own; the LLMs were used as analytical mirrors, not as sources.

The embodiment gap: why AI agents can buy, but not need

Imagine an AI agent that manages your grocery shopping. It knows your budget, your usual brands, your delivery preferences. One day it orders tomatoes.

Reasonable enough. You buy tomatoes. But here’s what the agent doesn’t know: you only buy them in summer, because that’s when they’re actually worth eating. You buy them from a specific organic shop, not the supermarket. And you have an allergy that rules out certain varieties entirely.

The agent watched your behaviour for months. It saw irregular tomato purchases. It couldn’t see the four-variable logic operating underneath — seasonality, ethics, store preference, health constraint. That logic is invisible to observation alone. You never stated it, because it felt too obvious to explain.

This is the embodiment gap. And it is one of the most underexplored limits in agentic commerce. The Lasagna Dilemma takes the same limit from the other end: what is lost when intent is compressed into a single request.

I. The illusion of delegation

The promise of agentic commerce is easy to understand. An AI agent will know your preferences, compare your options, negotiate on your behalf, and complete the transaction. It will remove friction from ordinary life.

That promise is plausible. But it contains a weakness that becomes clearest the moment we move from digital tasks to physical products.

The agent can read signals around a choice. It cannot inhabit the life behind it.

Go back to the tomatoes. The agent has your purchase history, your preferred retailers, your delivery patterns. It sees you buy tomatoes occasionally from an independent shop. That’s enough to place an order.

But the order it places is not the order you would have placed. It arrives in December. Wrong variety. Wrong shop. Not because the agent made a technical error — it matched your profile perfectly. It just had no access to the reasoning structure underneath: the seasonality rule, the ethical commitment to organic sourcing, the allergy operating silently beneath every decision.

The system works with behavioural legibility. The human lives with embodied consequence.

That distinction matters. The agent may reproduce our choices without reaching our needs.

II. Three layers of preference — and what sits beneath them

The limit becomes easier to see if we split preference into three layers.

Declared preference is what you can say. I buy tomatoes. Organic when possible. Nothing too expensive.

Inferred preference is what the system can guess. This user purchases tomatoes irregularly, favours organic options, tends to buy from independent retailers.

Experienced preference is what only appears in use — or in this case, in the moment the wrong tomatoes arrive. December. Wrong variety. Wrong shop. That’s when the agent’s model of you is revealed as incomplete.

Current AI agents can work reasonably well with the first two layers. The third is where the real limit appears.

Beneath the layers: constraints nobody declares

But the tomato example points to something that sits beneath all three: constraints and principles that the user never experiences as preferences at all. The allergy isn’t a taste. The organic sourcing is an ethical commitment. The seasonality rule is a contextual protocol that activates only in certain months. These don’t register as information worth communicating — they feel like background, like obvious context. So they’re never declared. And because they don’t produce consistent behavioural patterns, they can’t be reliably inferred either.

The agent’s inference problem here isn’t solved by more time or more data. The behaviour it observes — irregular tomato purchasing — is consistent with dozens of different underlying logics. Only disclosure closes this particular gap. And disclosure requires the user to realise the agent doesn’t already know — which they typically assume it does.

The biology makes this concrete. Tomato allergenicity varies by as much as 700% between cultivars, driven by proteins that differ significantly across varieties. Some allergens are destroyed by cooking; others, concentrated in the skin and seeds, survive it. An agent that observes five tomato purchases has no access to the fact that four were cooked and one was a specific low-allergen variety. Its confident inference — user likes tomatoes — is not just incomplete. In the wrong combination of variety, preparation, and season, it is potentially dangerous.

III. How the agent might learn

If an agent is to represent a user more faithfully, it cannot stop at the decision. It has to learn something about the consequence — and about the reasoning structure underneath the choice.

This raises a practical question: how?

One option is the diary model. The user records what happened after the purchase, what worked, what failed. High-quality information, but too much burden to be a default.

Another is passive inference — the system tries to read satisfaction from later behaviour: avoidance, replacement, repeat searching. Less burden, but the signal is deeply ambiguous. A slower purchasing pattern tells you nothing about why.

A more plausible model sits between these: agent-led micro-discovery. Small questions, asked at the right moments. What mattered most this time — comfort, ethics, price? A week after delivery, is this being used as expected? Not a diary. Not surveillance. A light research loop built into the relationship.

A fourth option, emerging from the merchant side, is product-context enrichment — brands feeding agents structured post-purchase signals, durability data, use-pattern telemetry. This narrows the gap from the supply side without burdening the user. But it captures product-level patterns, not the individual’s embodied constraints. It would never surface the allergy. It would never know about the seasonal rule.

IV. The cost of being known

None of these models is neutral.

The diary model asks too much. The passive model risks too much intimacy and interpretive overreach — a changed purchasing pattern may mean dissatisfaction, or it may mean a different season, a different budget, a different week. The micro-discovery model is more humane, but introduces its own problem: interruption. Ask too often and the agent becomes friction. Ask badly and it breaks trust.

There is also a subtler trust failure specific to this domain. Users don’t just lose faith when an agent does something wrong. They lose faith when an agent reveals, through a bad purchase, that it never understood a rule the user assumed was obvious. The tomato arrives in December, wrong variety, from the wrong shop. The agent didn’t fail to predict a preference. It failed to discover a constraint that had never been named — because it had never needed to be.

The design challenge, then, is not simply how to ask more. It’s how to ask well enough, at the right moment, to surface what the user doesn’t know to volunteer.

V. Choosing well is not the same as knowing well

There is a distinction worth holding onto as agentic systems become more capable.

We are rightly focused on making agents transparent — on being able to ask why did the agent choose this? That matters. But even a fully transparent decision leaves a second question unanswered: how would the agent know whether the choice actually worked?

The first question concerns interpretation before action. The second concerns consequence after action.

Together, they define the real boundary of delegation. Not whether the agent can choose on our behalf — in many cases, it can. But whether it can learn from what happens once the choice enters the world.

What comes next

AI agents may become very good at buying like us before they become good at needing like us.

That gap is not a minor technical limitation. It defines the boundary of delegation in domains where need is bodily, contextual, and structured by rules the user has never thought to state.

If the agent depends on a diary, the burden is too high. If it depends on silent inference, the intimacy is too great. If it learns through small, well-timed questions — and through disclosure that both parties understand is necessary — it begins to look less like a shopping assistant and more like a continuous researcher of the user.

That may be the real shape of human-centred agents: not systems that know us in advance, but systems that learn carefully, progressively, and with restraint.

Until then, the agent remains strongest at navigating markets, not lives.

Previous
Next