Whenever you visit an e-commerce website these days, 95% of the time you’ll get hit with a pop-up of some sort that offers you an “incentive.”
“Join our email list and get 10% off.”
“$25 discount on your first purchase!”
“Mystery discount!”
It seems that almost every brand these days is offering a 10% email opt-in discount.
Since when did this become the norm? Honestly, I have no idea…
If you’re reading this and you run a Shopify brand, should you just follow the crowd and do the same?
It’s tempting, for sure. However, I’d pump the brakes a little bit.
The problem is, the way most marketers “validate” pop-up offers these days is based on how much traffic it converts to the email list.
Unless you’re a subscription-first brand or you have an amazing assortment strategy with a strong retention marketing program, you’re putting a lot of blind trust in your back-end system to recover those margins.
Email List Size Is A Vanity Metric
An email list of 1,000 buyers is infinitely more valuable than a list of 100,000 promo hunters.
If your current welcome offer drives massive opt-ins, but they never end up converting, you’re just paying for these users to sit on your email list.
In other words, you’re actually losing money.
That’s why Klaviyo’s pop-up split test results are often fool’s gold. It determines the “winner” based on which variation converts more traffic onto your list.
But you don’t actually know how many of these subscribers bought, nor what is their long-term LTV.
The test I’m about to share with you helps you validate your pop-up offer the right way.
How to Know Your Welcome Discount Is Actually Working
Here’s an overview of what you need to do:
- Split test pop-up offers in Klaviyo where the only variable is the discount
- Create a customer profile property based on which pop-up they used to sign up
- Use the variable to create a segment in Klaviyo and analyze results with Lifetimely
Let’s break it down, step-by-step.
Step 1: Create a custom profile property called optin_method in Klaviyo. This helps you understand which pop-up form converted the user into a subscriber.
Step 2: Duplicate your current discount pop-up. Add a hidden field to the pop-up that sets optin_method to “CONTROL”.
Step 3: Create a copy of the last pop-up you just created. Change the hidden field to set the optin_method to TEST. Tweak the copy to remove (!) the discount incentive.
Step 4: Duplicate your welcome flow. Your CONTROL group will receive the same welcome series as before. Tweak the TEST group welcome sequence to remove the incentive.
Step 5: Create two segments based on which pop-up form they converted from (TEST vs CONTROL)
Run the test for as long as you have enough data to reach statistical significance. The more traffic you drive onto your store, the faster it’ll be.
Analyzing Cohorts In Lifetimely
Once you’re certain you have enough data, the next step is to upload both segments into Lifetimely.
From there, navigate to your cohort report. You want to see a monthly view of lifetime net sales data.
Compare the TEST vs CONTROL segments by adding a filter to your report. Download the .csv data
Now, the big question remains:
Which pop-up offer generated more total net sales?
What you may find is that while the discount pop-up drove more email subscribers, it led to less total net sales than the no discount variant.
In other words, the 10% margin hit you took wasn’t worth it.
On the other hand, it’s entirely possible the discount prompted customers to spend more. In that case, offering a discount is thoroughly justified.
The bottom line is this …
Don’t do things just because everyone else is doing it.
A 10% welcome discount for a brand with a 35% retention rate isn’t the same as a 10% discount for a brand that can only retain 15% of customers.
There are no best practices in e-commerce.
Until you look at the data, you’ll never know if it’s the right move for your brand.
What’s Next
Use this test to find your winning pop-up offer, whatever it might be:
10% vs free shipping
$25 discount vs 20% discount.
10% discount vs no discount
If all this seems a bit too much, and you’d rather have someone do it for you, reply to this email and let’s talk.
Siim “don’t follow da crowd” Pettai
