If there’s one thing ecommerce search specialists need to know, it’s this:
Optimizing search for “relevance” isn’t the best idea.
We’re fortunate to partner and work with some of the world’s largest retailers, and in addition to being leading brands in their respective verticals, they all agree that optimizing search for “relevancy” can actually hurt your bottom line.
Confused?
We don’t blame you. This is a fairly new concept, and one that we’ve written about in other articles. But let’s go through a short run down of the concept to ensure you’re up to speed:
Why Relevance Isn’t Enough (And What Should Take Its Place)
Traditionally, when we think about relevant search results, we’re talking about search results that in some way relate to the search term used to find them.
The challenge with optimizing your search for relevance is that it’s subjective depending on each person, not an objective goal the company can easily identify and optimize for. The products that appear for a query optimized only for “relevance” will almost certainly not be the products both you and your users will benefit from seeing.
Here’s an example of this concept:
Style Gallery is a leading apparel company with a decent size product catalog. 100 customers come onto their site and search for “t-shirt”. The company has over 500 different types of t-shirts that it sells. If you were working at Style Gallery, how would you determine which t-shirts show up at the very top of search? How do you determine if the t-shirts you choose to appear at the top are “relevant” to each customer that searches for a t-shirt?
A more extreme example is a search for “chips” on a grocery website:
If a user searches for chips on a grocery website, should the grocer show Cheetos? It’s not strictly relevant to the search because Cheetos are not chips, but what if users who search for chips are highly likely to buy Cheetos? Should the retailer pedantically disagree and only show potato chips, or should it show the user Cheetos, even if they aren’t strictly relevant?
So, that leads us to the big question:
What do you optimize search for if it isn’t relevancy?
We answered this question in one of our first articles on this idea:
“The answer is to focus on the business metrics the company cares about: revenue, purchases, profit, or whatever else most needs to be increased. Then both evaluate the search, and drive up the probability that each customer has a successful buying journey using the clickstream data that happens after the search. In short, show each customer products for each search in the order that will most likely lead to an increase in the business metric.
To achieve this, retailers should use three key attributes balanced together: relevance, attractiveness, and personalized attractiveness. Relevance should prevent something like shoes showing up for that search for laptop. Attractiveness should ensure that the most attractive laptops (the ones most likely to be purchased from that search) should show up at the top, and personalized attractiveness should use what we know about each user, their individual clickstream data, to show the particular results most attractive to them.”
Leading search companies like Google and Amazon optimize their search results to be attractive to each user and tie it to a particular business KPI to make it objective. This KPI can be any number of things (ex. conversions, margins, click to conversions, add-to-carts, etc.). If we take our t-shirt example from before and optimize it for conversions, the answer to the question becomes simple: we want to show each user t-shirts that they are most likely to convert on.
By optimizing your search for a particular KPI, you are able to generate search results that benefit both you and your users.
How to determine your search KPI
Like we mentioned before, one of the easiest ways to determine what KPI you should optimize for is to look at the business metrics you care about most:
- Revenue?
- Purchases?
- Profit?
- Or something else?
At Constructor, we also use this short questionnaire to help our clients determine their most important KPIs:
Question 1: How do you determine which items are the most relevant and should be at the top of your search results?
If your merchandising team (or anyone else working on search) has an active part in determining what items are ranked for a search query, those decisions must be based on something — most likely a metric trying to be driven. What is that metric?
As a side note, although many companies determine which results are “relevant” by eyeballing results and deciding what makes the most sense to rank, we believe this is not the ideal way to rank products — and in many cases, this method can actively harm revenue targets and customer loyalty.
Going back to our “Cheetos and chips” example, merchandisers may look at a product like “Cheetos” and determine that it is not relevant to the “chips” keyword, thus missing out on all the conversions from users who do, in fact, purchase Cheetos when searching for chips.
Product ranking and KPI optimization processes can (and should) be automated with AI and machine learning. Not only does this save time, but it also ensures the best results for any query are shown to users across your whole site.
Question 2: Have you or your executive team shared particular growth goals around ecommerce this year?
If yes, what are they? What is the metric you look at most to achieve that goal?
If no, how do you measure if search is successful today? Are you using any metrics to measure search? Are there any that your team members feel are more important than others, and if so, why?
Question 3: Does the company sell its own products or resell products made by other companies?
If the company sells its own products, do you get better margins from selling those items?
If the company sells its own products and resells other companies products, is there a difference in conversion rates? How much?
Determining your KPI by business type
Different types of businesses should optimize for different types of metrics. So, another useful way to determine what KPI you should optimize your search for is to look at the type of business you have.
We discussed this topic in our webinar, “Search Relevance Is Dead.”
Takeaways
If there are two things you should take away from this article, they’re these:
- When evaluating search engines, you should focus on business metrics, not relevance.
- Retailers should use clickstream data as the main driver to power search and evaluate it. Doing anything else does a disservice to both retailers’ customers and their bottom line.
Want to learn about how to measure and optimize search as a Search Product Manager?
On March 19th, we’re hosting our webinar, “The Search Product Manager Crash Course: Everything You Need To Know – Part 1.” We’ll equip you with all the knowledge you need — from learning the differences between AI, ML, and NLP, all the way to communicating the value of search to stakeholders — to create a winning site search road map.