This article was written in collaboration with Shweta Kumar, Full Stack Engineer, Constructor
Many ecommerce companies use product recommendation engines on their websites to help shoppers discover new products they’ll love, with the goal of driving upsells and increasing RPV.
Recommendations can appear in a variety of places on retail sites, such as product detail pages, shopping carts, listing pages, and checkout pages, making it easier for shoppers to discover products they’re otherwise likely to buy based on in-session or past behavior. They reduce friction in the product discovery process and give retailers the opportunity to show shoppers that they’re responding to their needs and interests in real time.
While recommendations have become a normal part of modern ecommerce experiences, especially onsite, there is little data publicly available that quantifies how offsite recommendations (such as in emails, SMS, social, and other channels) can further multiply their impact.
Constructor recently tested email recommendations, an Offsite Product Discovery offering, with one of our largest beauty customers. Read on to discover what we learned about email recommendations and how they influence shopper behavior.
What are Email Recommendations?
Email recommendations are personalized product suggestions that appear in marketing or transactional emails. They are part of Constructor’s Offsite Product Discovery offering.
Email recommendations are dynamically generated based on user behavior and individually tailored to shoppers’ needs and preferences.
While the concept is simple in theory, recommendations can be deployed in emails with a number of different strategies. These include:
- Trigger-Based Emails: These emails are sent on a 1:1 basis in response to certain events (e.g. abandoned cart flows, post-order follow-ups, etc). They are also sometimes referred to as transactional emails.
- Broadcast Emails: These are campaign emails sent to a large audience simultaneously (e.g. nurture campaigns, promotions, etc)
- And more
Retailers should consider broadcast vs. trigger strategies.
What we do differently (and why we predicted results would be positive)
In this particular test, the retailer we worked with had previously implemented a dedicated email recommendations platform and had used it for several years. They implemented Constructor initially for onsite recommendations – and after seeing positive results, decided to test offsite product discovery as well.
There are a few ways we approached the problem differently that we believed would lead to differentiated results:
We optimized our ML-based recommendations engine to the retailer’s chosen KPI. This meant that the ML would do the heavy lifting of solving for that KPI and drastically reduce the number of rules merchandisers were required to write. This allowed their merchandisers to focus on strategy, such as what types of emails to send and where to place the pods, leaving the personalization to AI. It also meant they could deploy more quickly without a ton of manual configuration while still allowing the merchants to manage by exception where desired.
The ML recommendations engine leveraged learnings across all the retailer’s onsite discovery experiences. Since this retailer used Constructor email recommendations alongside other elements of the discovery suite, such as search, browse, and onsite recommendations, email recommendations could leverage all of those learnings to better personalize the experience for each shopper. All of these elements cooperated to drive a better experience instead of competing for shopper attention.
Our Hypothesis and Test Parameters
Since the retailer already had a legacy solution in place, the goal wasn’t to test the viability of email recommendations in general but rather to test if Constructor’s recommendations engine could outperform the incumbent.
In other words, how much did the “cooperate not compete” concept matter, and how effectively did the holistic suite of learnings drive personalization?
The test parameters were as follows:
- Results Type: Email Recommendations
- Duration: > 20 email campaigns
- KPIs Monitored: Sales, orders, click %, CVR, Visits, AOV
The Results
As noted above, this retailer ran over 20 email campaigns A/B testing Constructor with their previous solution. They began with broadcast emails and eventually included trigger-based emails as well. The audience for most email sends included hundreds of thousands of recipients, with the largest being around 23 million.
In one of the earliest campaign broadcasts, they saw a 150% increase in sales with a traffic split of 30% Constructor / 70% legacy solution. This encouraged them to slowly transition the balance of traffic toward Constructor.
When they eventually reached a 50/50 traffic split on a key campaign, they saw the following results:
- Sales: +322%
- Orders: +200%
- Click %: +33%
- CVR: +23%
- Visits: +144%
- AOV: +41%
With consistently exceptional results, they eventually transitioned 100% of their traffic away from the legacy solution and to Constructor.
Note: To eliminate any potential bias, these results were calculated and validated by the retailer’s team with their own internal systems rather than the Constructor team.
What to Conclude
So, what did we ultimately learn?
The metrics show that Constructor’s ML recommendation engine made a significant difference in the effectiveness of this retailer’s Offsite Product Discovery efforts.
This supports our idea that discovery experiences should complement, not compete, with each other and that ML is extremely well-suited for KPI-optimization tasks.
As ecommerce continues to evolve, integrating discovery experiences can help retailers deliver holistic, personalized shopping journeys for their customers. The ability to leverage onsite learnings to drive offsite multi-channel product discovery experiences may seem small, but clearly in this case had an outsized impact.
