Skip to content
Logo-black
  • Native Commerce Core™
    • IDC-marketscape-q4-2024-resources-hub-cover-562x562@2x

      Constructor is a leader in the IDC marketscape

      Constructor is a Leader in the IDC MarketScape on Worldwide Knowledge Discovery Software for External-Facing Use Cases, 2024.

        • Search & Autosuggest

          Hyper-personalized ecommerce search driven by AI and real-time shopper behavior.

        • Browse

          Personalize your ecommerce category pages for maximum conversions.

        • Recommendations

          Drive upsells and increase RPV with hyper-personalized product recommendations.

        • Collections

          Dynamic, personalized ecommerce collection pages that learn from every search, click, and purchase.

        • Quizzes

          Create smart product recommendation quizzes that drive shopper engagement.

        • Merchant Controls & Intelligence

          Get the ecommerce merchandising controls and insights you need to make high-impact changes that drive revenue.

        • Attribute Enrichment

          Automatically enrich product attributes and categories to give shoppers the best possible experience.

        • AI Shopping Assistant

          Help your online customers discover great products faster.

        • Retail Media

          Constructor Sponsored Listings complement (instead of compete with) your organic results.

        • Cross-Channel & Offsite Discovery

          Create and launch personalized, data-driven marketing campaigns across digital channels.

        • B2B Use-Cases

          Product discovery technology that’s purpose-built to handle the complex needs of B2B ecommerce.

  • Customers
    • A_B Testing Navigation Image

      The Ultimate Guide to A/B Testing for Search & Discovery

      Discover what your ecommerce team needs to know about fixing broken testing strategies, avoiding wasted effort, and unlocking the full value of A/B testing.

        • Upcoming Events

          Visit us in person at ecommerce industry events.

        • Reports & Guides

          New research, guidance, and insights for the future of retail.

        • Webinars

          Educational webinars, conversations with Constructor partners, and more.

        • Blog

          The latest ideas and news from the frontier of commerce search and discovery.

        • Split/Test Digest

          The most interesting learnings from A/B testing at Constructor.

        • News

          Read Constructor's latest news coverage and press announcements.

        • Documentation

          Learn about the components of Constructor's advanced, AI-powered product discovery suite.

        • About Us

          With AI in our DNA and by keeping ecommerce as our core focus, we’ve built the best product search and discovery solution specifically for the unique needs of retailers.

        • Partners

          Constructor partners with the world’s top retail consultants, systems integrators, and technology platforms.

        • Careers

          Constructor is building the next generation of search solutions.

Login
Book a Demo
  • Native Commerce Core™
    • IDC-marketscape-q4-2024-resources-hub-cover-562x562@2x

      Constructor is a leader in the IDC marketscape

      Constructor is a Leader in the IDC MarketScape on Worldwide Knowledge Discovery Software for External-Facing Use Cases, 2024.

        • Search & Autosuggest

          Hyper-personalized ecommerce search driven by AI and real-time shopper behavior.

        • Browse

          Personalize your ecommerce category pages for maximum conversions.

        • Recommendations

          Drive upsells and increase RPV with hyper-personalized product recommendations.

        • Collections

          Dynamic, personalized ecommerce collection pages that learn from every search, click, and purchase.

        • Quizzes

          Create smart product recommendation quizzes that drive shopper engagement.

        • Merchant Controls & Intelligence

          Get the ecommerce merchandising controls and insights you need to make high-impact changes that drive revenue.

        • Attribute Enrichment

          Automatically enrich product attributes and categories to give shoppers the best possible experience.

        • AI Shopping Assistant

          Help your online customers discover great products faster.

        • Retail Media

          Constructor Sponsored Listings complement (instead of compete with) your organic results.

        • Cross-Channel & Offsite Discovery

          Create and launch personalized, data-driven marketing campaigns across digital channels.

        • B2B Use-Cases

          Product discovery technology that’s purpose-built to handle the complex needs of B2B ecommerce.

  • Customers
    • A_B Testing Navigation Image

      The Ultimate Guide to A/B Testing for Search & Discovery

      Discover what your ecommerce team needs to know about fixing broken testing strategies, avoiding wasted effort, and unlocking the full value of A/B testing.

        • Upcoming Events

          Visit us in person at ecommerce industry events.

        • Reports & Guides

          New research, guidance, and insights for the future of retail.

        • Webinars

          Educational webinars, conversations with Constructor partners, and more.

        • Blog

          The latest ideas and news from the frontier of commerce search and discovery.

        • Split/Test Digest

          The most interesting learnings from A/B testing at Constructor.

        • News

          Read Constructor's latest news coverage and press announcements.

        • Documentation

          Learn about the components of Constructor's advanced, AI-powered product discovery suite.

        • About Us

          With AI in our DNA and by keeping ecommerce as our core focus, we’ve built the best product search and discovery solution specifically for the unique needs of retailers.

        • Partners

          Constructor partners with the world’s top retail consultants, systems integrators, and technology platforms.

        • Careers

          Constructor is building the next generation of search solutions.

Login
Book a Demo
Back to Blog

Adapting Ranking Models to Short-Term Sales Behavior

Machine Learning Customer Experience Product Discovery Ecommerce Technology
Posted on:
April 28, 2025
Author:
Nate Roy
Adapting Ranking Models to Sales Behavior: Experimenting with CTR-Based Signals
Table of Contents:
Adapting Ranking Models to Sales Behavior: Experimenting with CTR-Based Signals

This post was written in collaboration with Valery Bezrukova, VP Product at Constructor.

When sales season hits, it’s a win for shoppers — but it can throw a wrench into even the best-performing product ranking algorithms. In this experiment, we set out to improve our models’ ability to adapt to the unique, fast-shifting dynamics of sale periods across several customers and verticals.

The Problem: Short-Term Sales Behavior Is More Challenging to Predict

A few of our customers flagged a trend they noticed: when they ran short-term sales, it would sometimes impact the quality of product results. That was surprising. Our ML models are typically quite strong at learning from historical behavioral patterns. So, why the change?

The answer lies in how differently shoppers behave during sales:

  • Pricing changes rapidly and dramatically. Algorithms trained on regular pricing have a harder time adjusting to short-term, steep discounts, especially when they are unprecedented (in other words, no historic examples to model them after)

  • Query distribution changes. Sales-specific terms and behaviors spike (think: things like “clearance,” “last chance,” or “under $20”)

  • Item popularity shifts quickly. Products that previously received little attention can become highly attractive overnight, then go back to normal once the sale is over

Even a human would struggle to predict behavior based on out-of-distribution pricing. For instance, let’s say a car Model A ($30k) historically outsells Model B ($50k) 2:1. If the prices of both cars suddenly drop by 50%, would you expect Model A to continue to outsell Model B by 2:1? That’s uncharted territory and would be difficult for anyone to guess. 

The same is true for the ML. The model will interpolate based on past data, but it would have a harder time extrapolating when prices for products suddenly move below anything it's seen historically.

The Hypothesis: Can We Use Weighted CTR to Adapt Faster? 

We hypothesized that we could mitigate this problem by using Clickthrough Rate (CTR) instead of raw action counters (e.g., clicks, purchases) as a primary signal. Why?

  • CTR is a normalized metric. It adjusts for the number of impressions, which helps surface items that are performing well relative to how often they’re seen

  • CTR is more stable for new or rarely-seen items. A previously unpopular product might suddenly be shown more during a sale. It might still have fewer total clicks than a perennial bestseller, but a higher CTR

To improve signal quality, we experimented with a weighted CTR:

Weighted CTR = Weighted actions / Unweighted impressions

This approach blends recency sensitivity (giving more weight to recent user actions) with volume smoothing (by using stable impression counts), enabling the model to adapt to trends without overfitting to momentary noise.

The Test: Running Weighted CTR Across Vertical and Experience 

We ran the experiment across several interested customers and categories. Tests spanned both Search and Browse, over a minimum three-day period (sometimes longer depending on the customer’s specific circumstances). Each test introduced the weighted CTR-based signal into the ranking logic and compared it to the original counter-based model.

Here’s a sample of the results:

Browse Tests

  • Jewelry:
    +1.31% Add-to-Cart (ATC)
    +1.23% Checkout Revenue
    +2.49% Purchased Items

  • Furniture & Home:
    +1.53% ATC
    +2.93% Purchased Items

  • Toys:
    +7.93% Checkout Revenue
    +0.69% Purchased Items

Search Tests

  • Apparel:
    +2.05% ATC

  • General Merchandise:
    +2.1% ATC
    +0.05% Checkout Revenue
    +0.56% Purchased Items

  • Pets:
    +0.01% Checkout Revenue
    +0.89% Purchased Items

Note: Each test above is a standalone example and should not be seen as representative of the entire vertical. Customer behavior can vary widely within the same industry depending on the specific audience, test duration, and business strategies.

The Outcome: Early Signals on Faster Adaption 

While the results varied across verticals, a couple of clear themes emerged:

  • Browse experiences generally benefited more from weighted CTR. This makes sense. Browse is often where shoppers encounter new items, and CTR helps highlight what's trending in near-real time.

  • Weighted CTR helped the model respond faster to shifts in behavior, especially where previously unpopular or newer items began trending.

For retailers considering introducing this type of variable, it is worth noting that acting quickly on trends might not always be what matters most to shoppers (especially if they value your brand for its bestsellers, for example). 

While promoting new trends might improve short-term sales metrics, for some types of businesses, it may ultimately hurt overall performance by making it harder for shoppers to find the products they truly want.

Retailers should consider what their brand is known for and whether this type of strategy makes sense in the context of their business before testing it.

What's Next: Continuous Optimization 

This test confirmed that faster model adaptation to user behavior shifts is both possible and valuable in certain instances. Weighted CTR isn’t a silver bullet, but it’s a powerful signal that helps us enhance real-time trend responsiveness.

And it's just one step. We’re continuing to explore:

  • Further development of features that improve trend detection (e.g., slope features, new iterations with real-time signals)

  • Smarter training window selection to match periods of user behavior, and enhancing the training dataset to better account for period differences

  • Manual override rules as fallback mechanisms for extreme scenarios

All of this aligns with our broader goal: to create adaptive ranking systems that don’t just react to change, but anticipate it.

Subscribe to our Experiments Blog

Get conversion-driving insights delivered straight to your inbox.

What's next?
Eyeglasses icon

Uncover lost revenue opportunities with a complimentary Search Experience Audit.

According to the 2024 State of Ecommerce Search & Product Discovery Survey, nearly 70% of shoppers think the search function on retail websites needs an upgrade. Our team has run over 1000 A/B tests to identify easy-to-implement algorithmic and UX improvements that get results. Use their research to your advantage with a complimentary Search Experience Audit — no strings attached.

Request an Audit
  • Analyze your search results quality
  • Identify "no results" pages
  • Pinpoint irrelevant results for long-form queries
  • Uncover UX opportunities for Search, Autocomplete, and PLPs

According to the 2024 State of Ecommerce Search & Product Discovery Survey, nearly 70% of shoppers think the search function on retail websites needs an upgrade. Our team has run over 1000 A/B tests to identify easy-to-implement algorithmic and UX improvements that get results. Use their research to your advantage with a complimentary Search Experience Audit — no strings attached.

Request an Audit
  • MACH Certified
  • 21972-312_SOC_NonCPA-150x150
  • Cartification-mark_04-09-300x253-1-150x150
Constructor.io Logo
  • Technology
    • Native Commerce Core™
    • Request a Demo
    • The Proof Schedule®
  • Solutions
    • Search & Autosuggest
    • Browse
    • Recommendations
    • Collections
    • Quizzes
    • Merchant Controls & Intelligence
    • Attribute Enrichment
    • AI Shopping Assistant
    • B2B Solutions
  • Resources Hub
    • Events
    • Reports, Guides & Webinars
    • Blog
    • Experiments
    • News
    • Constructor vs Alternatives
    • Documentation
  • About
    • About Us
    • Partners
    • Security & Compliance
    • Careers
  • Technology
    • Native Commerce Core™
    • Request a Demo
    • The Proof Schedule®
  • Solutions
    • Search & Autosuggest
    • Browse
    • Recommendations
    • Collections
    • Quizzes
    • Merchant Controls & Intelligence
    • Attribute Enrichment
    • AI Shopping Assistant
    • B2B Solutions
  • Resources Hub
    • Events
    • Reports, Guides & Webinars
    • Blog
    • Experiments
    • News
    • Constructor vs Alternatives
    • Documentation
  • About
    • About Us
    • Partners
    • Security & Compliance
    • Careers
© 2025 Constructor All rights reserved
  • Terms of Service
  • Privacy Policy