For two years the agentic commerce conversation has run on surveys and projections. On November 5, 2026, it finally got a scoreboard. Visa published the Visa Agentic Commerce Benchmark, a first-of-its-kind test that put 22 AI models from 12 labs through 15 realistic shopping challenges, and the headline number is blunt: the models failed 42.8 percent of their test runs. The best performers passed 82.2 percent of attempts. The worst passed 28.9 percent.

But the failure rate is not the most important finding in the study. Buried in the results section is a statistic that should reorganize how the industry thinks about agent trust: in 54 percent of failed runs, the AI still made a purchase. Either it recommended a product that did not meet the customer’s identified requirements, or it bought before it had uncovered all of them. When modern AI shoppers fail, they fail toward spending money, not away from it.

That asymmetry is an evidence problem, not a payments problem. And it lands in the same week that the Federal Trade Commission unredacted details of how Amazon’s ad auction was secretly manipulated, which means the shortcuts failing agents fall back on are exactly the signals the government alleges were rigged. The rails being built for agentic commerce verify the agent’s identity, its budget, and its authorization to pay. Nothing in the stack verifies the product.

What Visa actually tested

Most “AI shopping” evaluations to date have measured retrieval: can the model find a product, recall a spec, or summarize reviews. Visa’s benchmark is stricter, and the design matters for anyone building in this space. The scenarios begin the way real purchases begin, with a customer who knows roughly what they want but not precisely. You know you want a TV. You do not know the refresh rate you need, or that your family videos require HDR10+ support, or that your console habits make gaming features non-negotiable.

To pass, a model had to do two things:

  • Uncover every task-defined customer requirement through conversation, including requirements the customer never volunteered. Visa explicitly designed against lucky guesses: the AI must dig out enough information to justify its decision.
  • Make the correct buy-or-decline decision. If nothing on the market fits the customer’s constraints, the correct answer is to buy nothing. A confident purchase of the wrong product is a failure, not a near-miss.

Visa calls it the first agentic commerce benchmark requiring both full requirement discovery and a correct purchase or no-purchase decision for a run to pass. The first round covered buying a TV, a laptop, and a personal care appliance. Checkout, payment, and order placement were not tested yet; Visa says those phases come later.

The numbers, in full

The full result set deserves attention, because each figure carves out a different failure mode:

  • Overall pass rate: 57.2 percent. Across 22 models from 12 labs, nearly half of realistic shopping runs ended in failure.
  • Spread between best and worst: 82.2 percent vs 28.9 percent. The gap is enormous. Per Fortune’s reporting, agents from Google and Meta shared the top score, a Claude agent came third at 76 percent, and an OpenAI agent ranked sixth at 64 percent. A consumer’s experience of “AI shopping” depends almost entirely on which assistant they happen to use.
  • Discovery discipline predicts success. The five models that uncovered the most hidden buying factors through conversation averaged a 70.2 percent pass rate. The five that uncovered the least passed 44.4 percent of the time. Asking good questions is the whole game.
  • Expensive models win, but do not guarantee anything. The five models with the highest average cost per run passed 67.1 percent of runs, versus 50.2 percent for the five cheapest. Money buys improvement, not reliability.
  • Consistency is the deepest problem. Every model got three attempts at each scenario. All three attempts passed only 34.2 percent of the time. Visa notes that models which passed a scenario often failed it on repeat, “showing how difficult it is to deliver a consistent consumer experience.”
  • Failures lean toward purchase. 54 percent of failed runs still ended in the AI buying something.

That last number is the one the industry should underline twice. A good human shopper who cannot verify that a product fits walks away. An agent that cannot verify still clicks buy.

The statistic that should scare merchants: 54 percent of failures still bought

Think through what a “failed run that ends in a purchase” means in production. A customer delegates a TV purchase. The agent never discovers the budget ceiling, or misses the HDR10+ requirement, or ignores the membership constraint. It then completes a transaction on a product that violates the customer’s needs. The failure is not an error message the customer sees. It is a package that arrives.

This is worse than a checkout failure in one specific way: it is silent. PYMNTS, covering the benchmark, noted that nearly 132 million US adults have already bought a product with AI’s help. AI has won a real role in product discovery. What it has not won is the discipline of declining, and the buy-or-decline decision is precisely where delegation breaks.

Rubail Birwadker, Global Head of Growth at Visa, put it plainly in the release: “AI agents have the potential to power the next era of commerce, but trust is key to making it work.” He was more direct with Fortune: “We’re not there yet from a trust or technology perspective. But we’ll soon get into a world where consumers can trust and delegate.”

Visa’s own Trust Index, released in September 2026, quantifies the gap: 72 percent of US consumers have used an AI assistant, but only 23 percent trust AI to handle payment transactions on their behalf. The benchmark explains the intuition behind that reluctance better than any survey. People are not worried that agents cannot shop. They are worried agents will shop wrong, confidently, with their card.

The field report agrees: the week the bubble quote landed

The benchmark landed 48 hours after the agentic shopping internet produced its own reality check. On November 3, a San Francisco tech worker posted on X: “I have NEVER in my life heard so much hype and discussion about a tech/product experience that basically does not work at all for me. I feel like I’m using theranos.” The post collected more than 3,500 likes and sparked a broad debate about where agents actually break, as i10X documented.

The failure modes in the thread were specific. A Sephora bot check stopped one agent outright. An AutoZone order was cancelled after roughly 17 hours of fraud review because it combined guest checkout, a one-time virtual card, and mismatched names. Meanwhile, a The Paypers report with Riskified from November 3 found that up to 34 percent of the fraud signals merchants normally rely on were missing in ChatGPT Instant Checkout orders.

Put the two documents side by side and a pattern emerges. Visa’s lab data says agents fail at the decision layer and buy anyway. The field data says agents fail at the execution layer and get blocked anyway. Different layers, same root cause: the systems assume a human on one end and have no verified evidence layer in between.

The trust stack being built verifies the wrong thing

Here is the uncomfortable synthesis. The industry’s answer to agent shopping is a set of payment and identity rails: Visa’s Trusted Agent Protocol, Mastercard’s Verifiable Intent, Google’s Agent Payments Protocol. These are genuinely important. They tell a merchant which agent is buying, on whose behalf, with what spending authority. They will fix the AutoZone-style fraud cancellations.

But look at what they verify: the agent’s identity, the customer’s intent, the transaction’s authorization. Nowhere in that stack does anything verify the product. Whether the TV actually supports HDR10+, whether the 4.8-star rating is organic or purchased, whether the “best seller” badge reflects quality or ad spend: all of that is outside the trust perimeter. Visa’s roadmap for the benchmark extends testing to checkout, payment, and order placement next, which is the right sequencing for a payments network. But it confirms the priorities. The entire emerging standard verifies the transaction, and the transaction is the last three seconds of a decision that takes three minutes.

An agent equipped with perfect payment credentials and a verified budget, pointed at a polluted product landscape, is not a solution. It is a faster way to buy the wrong thing.

The proxies failing agents use are the ones under investigation

There is a brutal coincidence of timing here. On November 5, the same day Visa released the benchmark, the FTC and 22 states filed an updated complaint against Amazon with unredacted details of an alleged ad auction manipulation scheme. The newly public allegations include:

  • Amazon secretly raised advertising prices by as much as 50 percent above auction results on ordinary days, and as much as 250 percent on Prime Day and Black Friday.
  • During the 2021 holiday season, advertisers’ average cost-per-click rose 20 percent, with some seeing increases over 200 percent. In 2022, the alleged surcharges generated an extra $87 million in worldwide revenue.
  • Internally, Amazon acknowledged the price increases were “destroying campaign performance” for at least one major advertiser, while account managers falsely told advertisers the increases came from competition, not manipulation.

The connection to the Visa benchmark is direct. When an agent fails to uncover a requirement and reaches for a shortcut, what does it reach for? Best-seller rank. Star rating. Position on the page. Those are the exact surfaces that sponsored placement and review manipulation reshape. A 42.8 percent failure rate means that in nearly half of runs, the model is choosing under uncertainty, and under uncertainty models regress to the most legible signal. If the most legible signal is for sale, the agent’s ceiling is set by the seller with the biggest ad budget, not the best product.

This is why product trust cannot remain outside the agentic stack. You cannot fix a decision-quality problem with transaction rails. The agent needs a signal it can verify before the buy-or-decline fork, not after it.

What a pass condition looks like: evidence before the transaction

The fix follows directly from the benchmark’s own findings. The five models that uncovered the most hidden requirements passed 70.2 percent of runs. Discovery is the mechanism. The question is what the agent discovers: marketing copy and placement, or verified product evidence.

That is the layer GoBuy builds. Before a buy decision, an agent consulting GoBuy gets:

  • A Smart Score from 0 to 100 computed from review quality, not review quantity, so a thousand purchased five-star reviews score lower than two hundred verified, substantive ones.
  • Fake review filtering, suppressing incentivized, coordinated, and generated reviews before they reach the score, so the legible shortcut an agent grabs is one that survives manipulation.
  • A shortlist of the top 7 products per category instead of thousands of listings, which maps cleanly onto what the benchmark rewards: fewer, better-justified decisions rather than confident guesses across a polluted shelf.
  • The GoBuy Verified badge, reserved for products holding a Smart Score above 80 across 90 days, giving an agent a durable pass/fail input rather than a snapshot.

All of it is exposed over MCP at gobuy.ai/api/mcp, so any agent that can call a tool can consult the trust layer before it commits the customer’s money. That is the buy-or-decline discipline Visa measured, expressed as infrastructure. Visa benchmarked whether models ask enough questions before buying. GoBuy makes sure the answers they get are real.

The scoreboard will keep running

Visa says it will add scenarios, extend testing to checkout, payment, and order placement, and has shared the benchmark with leading model providers. Anyone can attempt it on Visa Payments Frontier. Expect pass rates to rise fast; labs optimize against published benchmarks, and the discovery skills the test rewards are trainable.

But two things will not change as the scores climb. Consistency at 34.2 percent triple-pass will take real architectural work, not prompt tuning. And the 54 percent still-bought failure mode will persist as long as agents make buy decisions on unverified signals, because the incentive to close is structural: an agent that buys looks successful, and an agent that declines looks broken, even when declining was correct.

The benchmark’s quiet lesson is that the buy-or-decline decision is the product. The industry is spending billions making sure the agent is allowed to pay. Somebody has to make sure the agent is right. That is product trust, it belongs in the loop before authorization, and it is exactly what we build.

If you are shipping a shopping agent or an agentic checkout flow, wire trust in before the transaction: read the agent docs, connect to the MCP server at gobuy.ai/api/mcp, and give your agent a reason to say no.