Why Vertical AI Beats Generalist LLMs in Commerce

One of the most consequential things taking place in commerce today is the revolution in how discovery happens. This is taking three forms: First, where that discovery happens (within retailers’ owned properties; via Answer Engines; or via personal assistant agents like Instinct, Muse, and GrokBot). Second, how the discovery happens (traditional forms of discovery like search; onsite agents; or something else). And third, what underlying engine drives the discovery (traditional keyword matching and older forms of Machine Learning; generalist models; or specialist LLMs built specifically for commerce).

All of these questions are interesting, but the last may be the most interesting because the answer to it really underpins the answer to the others, and because there is a lot of evidence that it has a right answer and we already know it.

First, some background. Keyword matching is how discovery was traditionally done for years and is the technology older engines like Solr and Elasticsearch were built on. The idea was to match keywords between a query (i.e. “chips”) and a set of products (i.e. “Lays Potato Chips”). Heuristics would then be used to rank the results (i.e. relevance algorithms like TF-IDF and BM25). It’s now been a few years since pretty much everyone has agreed that AI is a better way to solve the problem, and it’s hard to find anyone still claiming keyword matching algorithms are better, even if many search and discovery engines were originally built on these and still have a deal of legacy code that relies on them.

The question of whether to choose generalist models (LLMs made by companies like OpenAI, Anthropic, and Google) or specialist models specifically built for commerce is the more interesting one, and what I’d largely like to focus on because so much about the answer to that question will determine the course of commerce.

Generalist models are clearly capable of amazing things, as anyone not living under a rock for the last six years will tell you. If you ask an LLM to do a task for you, it becomes more likely every year that that LLM will do an excellent job at completing the task. There is good reason to believe generalist LLMs will become better and better at commerce discovery tasks as well. The problem is that they suffer from a number of core issues that leave them with structural inferiority to commerce-specific models for this specific task: data, inference speed, inference cost, and commerce-specific tooling.

First, data: generalist LLMs lack the years of ecommerce clickstream data that is the best predictor of what people will buy and the best thing for the LLM to learn from in order to connect the right shopper to the right product at the right time and in the right context. This is the single biggest gap, and if you’ve ever tried to start your shopping journey via a general-purpose LLM or agent, you’ll know what I mean. These systems connect shoppers with products that look somewhat relevant, but they have trouble connecting shoppers with the products they’re most likely to actually buy. Try one and you’ll see.

There’s a very good reason for this. The data needed to get good at surfacing products a shopper will buy is largely clickstream data (what shoppers look for, which products they give positive feedback to via clicks, add-to-carts, and purchases; which products they give negative feedback to through scroll-pasts, cart-abandons, etc.). It’s accessible to retailers, brands, and some of their technology partners, but it is proprietary and not something generalist LLMs have access to. It winds up being a chicken-and-egg problem because that data is what LLMs need to connect shoppers to products they will actually buy, but it’s also data that those LLMs need to convince shoppers to use them instead of ecommerce sites in the first place. 

This is also data that retailers and brands aren’t eager to hand to LLMs because doing so could mean turning over the shopper relationship to an LLM— an LLM that might encourage shoppers to buy from a competitor, or charge a tax for bringing shoppers back to the retailer. Retailers and brands would rather own the shopper relationship and have shoppers discover products in their own digital properties.

The next problem after data is inference speed and cost. Generalist LLMs have hundreds of billions of parameters (sometimes even more) and need them because they need to know everything about the world. They try to know everything about everything. This both makes them slow to respond to a shopper query and increases their costs. 

The most interesting thing about this dynamic is that the vast majority of their knowledge is not helpful to a shopper looking for something to buy. An LLM made to help a shopper discover products on a retail site does not need to know what year Albert Einstein was born, or how tall the Statue of Liberty is. It just needs to know how to connect the right shopper to the right product at the right time and in the right context. Building a much bigger model that knows far more information that isn’t useful for ecommerce discovery actually makes it more expensive and slower at the job we actually care about.

In short, generalist LLMs miss the data you really need to make ecommerce discovery really effective while simultaneously being burdened with orders of magnitude more data that is completely unnecessary for ecommerce discovery. A vertical model built specifically for commerce solves both problems because it is trained on the data actually needed to connect shoppers with products they want to buy while avoiding overwhelming it with a bunch of data that it does not need.

On top of that, a number of additional issues exist that make generalist LLMs the wrong tools for retailers and brands:

• These systems lack explainability and are built as black boxes because they aren’t made for business users on the other side trying to understand the responses.
• These systems lack the fine-grained merchandising tools needed to strategically modify their results where a business priority dictates it (like boosting up an exciting new product during its launch period, or trying to sell out of a clearance item before you run out of space for it in the warehouse).
• These systems lack ecommerce-specific analytics dashboards that surface what business users need to know about the performance of discovery in their business.
• Potentially most critically, these systems aren’t constantly refined and tested to drive conversion and revenue. This is the most important thing for a retailer to get out of their product discovery system, but for generalist LLMs, it’s not even an afterthought. This means that the gap between them and specialist commerce LLMs only grows over time because one tries to constantly find ways to increase revenue for retailers while the other doesn’t even think about it.

These problems don’t make generalist LLMs any less impressive. They’re still incredible AI. But these are structural problems for generalist LLMs that vertical models do not suffer from, and this is why commerce is one of the industries where vertical models are winning and will continue to win.

Further, commerce-specific vertical models win not just because they lack the structural disadvantages of generalist LLMs, but also because they have their own structural advantages, the biggest of these being focus.

While a generalist LLM has a broad variety of potential success metrics, a vertical AI only wins if it focuses on the metrics most important to the industry it is made for. For commerce, for example, this tends to be business metrics like increasing conversion rates (CVR) and revenue per visitor (RPV). For the best vertical AIs in the space, they have entire teams devoted to increasing these metrics– creating new ranking algorithms with the goal of increasing CVR and RPV; AB testing new personalization algorithms for the sake of increasing CVR and RPV; working on recall improvements to get increases in CVR and RPV. Not surprisingly, when the teams behind vertical AIs devote themselves to increasing a specific set of metrics, they build systems that wind up being very good at increasing those metrics.

This all started as theory, but over the last four years, Constructor’s Commerce Reasoning Engine (our vertical AI for commerce) has been A/B tested against technology built on generalist LLMs or sold by the companies who made those generalist LLMs 13 times and has not lost a single test. We know some of the world’s smartest engineers work on these generalist LLMs. It’s not a question of intelligence or capability. The reason Constructor has won over and over again, and why at this point we’re convinced the debate is over, is that vertical AI simply has structural advantages that, for the specific challenge of product discovery in shopping, make it the better choice.