User Guide

Using an A/B Test Within A Rule

14 views 0

The primary function of A/B testing within Zephr is within a Feature or Request Rule. A/B testing is used to help gather information on which transformation or outcome is more favorable to your end user and which transformation takes you closer to achieving your strategic goals. For example, you may wish to test a ‘lite’ registration form, asking for only and email address and password, against a more thorough registration form, and compare which converts anonymous users to registered users more efficiently.

In this guide, we will add a test to a Feature Rule to trial two different registration forms.

Firstly, make sure you have a test group set up under Settings > A/B Tests. Visit the Setting up an A/B Test page for more details.

Once you have your test group set up, navigate to the Feature Rules section. Here, create a rule, or start a new version of the rule you want the test to run on. For this example, we’ve created a new version of an Article rule, which performs transformations of article content. This rule checks if a user is logged in, and if logged in displays full access to the content.

Selecting a test segment in the rules builder

At the point where you want your test to run, locate the A/B Test decision point in the Rule Builder. You’ll find this on the left hand side of your canvas, under Test Segments.

Drag this onto your canvas, and choose your test group from the drop down list.

Screen for selecting an A/B test within a rule

Connect the A/B Test decision point to the rest of your rule. Following this, drag the different transformations your are testing onto your canvas and connecting them to Group A and Group B accordingly.

Complete Feature Rule with A/B Test

In the example above, you’ll see that if a user is not logged in, there is a check to see which test group they fall into. Group A users will see the Registration Form, whilst Group B users will see the Lite Registration Form. Users are randomly assigned into these groups, based upon the percentage split you have given the test sample.

Once you’ve finished, save your rule. Remember you’ll need to Publish your rule before you see it in live, or, if you want to test your rule, add a tag to the version, and view it in Preview Mode. Take a look at our Using Preview Mode guide for more information on testing your rules, or go to our next guide Reporting on A/B Tests for information on how to measure the impact of your A/B Tests using out Web Analytics function.

Reporting on A/B Tests

Reporting on the performance of your A/B Tests is simple with the Zephr Web Analytics feature. Simply navigate to Settings > Web Analytics and click Add Field.

From here, choose Test Group as your Selector Type and pick the relevant Test Group in the drop down – e.g. Registration Test.

The Selector Type dropdown list in the Web Analytics section

Set your Label and Key, and click Add. Remember to save your Web Analytics configuration once you’re done. This will then write data to a data layer for your analytics platform to ingest.

With this enabled, you’ll be able to track the results of your test via your Web Analytics platform.