A/B/n Testing module

License MLA

Issues

ABTEST

Maven site

A/B/n Testing

Latest

1.0.1

Note that A/B/n Testing comes with a special license and runs as a connected service. Please contact your Magnolia Customer Success Manager or Support to find out more.

The A/B/n Testing module installs the A/B/n Testing app and allows you to perform A/B/n testing via the Magnolia UI.

Test results from the A/B/n Testing module rely on the Analytics dashboard from the Analytics module.

Module structure

artifactID Description

magnolia-abtesting

Provides a default implementation to connect to the A/B/n Testing app.

magnolia-abtesting-ui

Provides the UI configuration for the A/B/n Testing app.

Installing with Maven

Maven is the easiest way to install the module. Add the following to your bundle:

<dependency>
  <groupId>info.magnolia.abtesting</groupId>
  <artifactId>magnolia-abtesting</artifactId>
  <version>1.0.1</version>
</dependency>
<dependency>
  <groupId>info.magnolia.abtesting</groupId>
  <artifactId>magnolia-abtesting-ui</artifactId>
  <version>1.0.1</version>
</dependency>

Configuring

To configure the following properties, go to ab-testing/config.yaml.

If you do not see a config.yaml file, create one.

In the module configuration, you can:

  • Define the minimum unique visitors required to produce test results.

  • Adjust the minimum sample size required.

  • Set a limit to the maximum number of remaining days.

You must also make sure your site allows CORS for the POST method.
ab-testing/config.yaml
abTestingConfig:
  apiGatewayURL: your-api-gateway-url
  authorEventsExcluded: true
  resultsConfig:
    minUniqueVisitors: 10
    minSampleSize: 1
  statsEngineConfig:
    maxRemainingDays: 10
Property Description

apiGatewayURL

required

The API gateway URL provided to you by Magnolia.

authorEventsExcluded

Default is true.

Excludes author instance events from the test.

minUniqueVisitors

required

The number of minimum unique visitors that must visit the site before results are produced.

minSampleSize

required

The minimum overall sample size that must visit the site before results are produced.

maxRemainingDays

required

The maximum number of days that can remain in an A/B/n testing campaign.

Enabling CORS for the POST method

You must add the POST method to the list of allowed HTTP methods in your CORS configuration (Cross-origin Resource Sharing).

User events for the test variants are submitted to the Magnolia REST endpoint using the POST method.

Typically, only the GET method is enabled by default in Magnolia.

To enable the POST method:

  1. In the Site app, go to <your-site>/cors/<your-site>/allowedMethods.

    Typically, a property named get with the value GET is configured by default. This means the site allows CORS only for the GET method.

  2. Update the get property name to all and the value to *.

    This effectively adds the POST method required by A/B/n Testing.

  3. Publish the change so that your public instance configuration is updated.

Travel site example

CORS configuration

Configuring permissions

Magnolia provides user role configuration to facilitate permissions for A/B/n testing.

Existing users with editor or publisher roles are be granted read only permissions to A/B/n Testing so that they can see the status of tests in the Pages app.

An ab-tester role is configured by default and grants minimal permissions to users to access the A/B/n Testing app tile in AdminCentral and create and start A/B/n tests.

You must assign the ab-tester role to your new and existing users who will use A/B/n Testing.

You can view, assign and edit roles in the Security app.

Usage

When running an A/B/n test using the A/B/n Testing module, a copy of the original page in the website workspace is made when the first variant is created.

The original page in the website workspace is not locked. Only the copy is locked and is used as a base for any variants created.

When a page is being used in a test, the status of the test in displayed in the Pages app. Variants of the original page are not displayed in the Pages app, only the original. If you want to see the variants, you can find them in the JCR Browser.

image

Editors can continue working on the original page while the test is running. When the test completes and users must decide which variant to use, they are warned that the original has changed since the test began.

Segments are not locked nor copied when used in a test. The effect of deleting or modifying a segment that is actively being used in a test is not defined. We recommend you create dedicated segments for each test.

For more information, see the A/B/n Testing app page.

A/B/n Testing for multilanguage websites

In the context of a multilanguage website, the A/B/n Testing feature behaves differently depending on the multilanguage structure configured:

  • When using a single tree structure, the multilanguage content is stored in a single JCR content node. This means that you must create one A/B/n Testing test that covers all the languages.

    For example, if your website is in German and French, one test covers both languages.

  • When using a multiple tree structure, the multilanguage structure consists of separate website trees. In this case, you must create an A/B/n Testing test for each language that you want to test.

    For example, if you want to run the same test for German and French, you must create two A/B/n Testing tests: one for each language.

The A/B/n Testing feature does not work with the URL Translation incubator module.

Uninstalling

To uninstall the A/B/n Testing module:

  1. Go to the A/B/n Testing app, abort any running tests and delete all tests.

  2. Go to the Site app and revert any changes you made when Enabling CORS for the POST method.

  3. Go to the Configuration app and delete the following nodes:

    • /modules/cache/config/contentCaching/defaultPageCache/cachePolicy/shouldBypassVoters/deny/abTestingPage

    • /server/filters/cms/testingResolver

    • /server/filters/sitemesh/decoratedFilter/contentProcessor/tagRuleBundles/magnolia/rules/DisableLinksInABVariantEditor

  4. Shut down Magnolia, remove the A/B/n Testing module JAR files (magnolia-abtesting.jar, magnolia-abtesting-ui.jar) from WEB-INF/lib and start up Magnolia again.

If you do not use the Analytics module outside of A/B/n Testing purposes, delete the magnolia-analytics-x JAR files too.

Release history

A/B/n Testing module 1.0.1

Released on July 26, 2021.

Bug fixing and improvement release.

A/B/n Testing module 1.0

Released on June 17, 2021.

General availability release of the A/B/n Testing Special Feature, which provides native A/B/n Testing to help you make decisions about your content based on real data from your existing traffic.

See A/B/n Testing for an overview of the functionality provided.

Note that A/B/n Testing is compatible with Magnolia CMS 6.2.4 onwards.

A/B/n Testing module compatibility

A/B/n Testing module version Magnolia version

1.0+

6.2.4+

Feedback