Mattresses and More has an onboarding workflow for new users of their website. It uses modal pop-ups to welcome users and show them important features of the site like:
- Welcome to Mattresses and More!
- Browse our bedding selection
- Select items to add to your cart
- View your cart by clicking on the icon
- Press ‘Buy Now!’ when you’re ready to checkout
The Product team at Mattresses and More has created a new design for the pop-ups that they believe will lead more users to complete the workflow.
They’ve set up an A/B test where:
- 50% of users view the original
control
version of the pop-ups - 50% of users view the new
variant
version of the pop-ups
Eventually, we’ll want to answer the question:
How is the funnel different between the two groups?
We will be using a table called onboarding_modals
with the following columns:
user_id
- the user identifiermodal_text
- the modal stepuser_action
- the user response (Close Modal or Continue)ab_group
- the version (control or variant)
Instructions
Start by getting a feel for the onboarding_modals
table.
Select all columns for the first 10 records from onboarding_modals
.
Delete your previous code.
Using GROUP BY
, count the number of distinct user_id
‘s for each value of modal_text
. This will tell us the number of users completing each step of the funnel.
This time, sort modal_text
so that your funnel is in order.
Delete your previous code.
The previous query combined both the control and variant groups.
We can use a CASE
statement within our COUNT()
aggregate so that we only count user_id
s whose ab_group
is equal to ‘control’:
SELECT modal_text, COUNT(DISTINCT CASE WHEN ab_group = 'control' THEN user_id END) AS 'control_clicks' FROM onboarding_modals GROUP BY 1 ORDER BY 1;
Paste this code into the code editor and see what happens.
Add an additional column to your previous query that counts the number of clicks from the variant group and alias it as ‘variant_clicks’.