A/B/n Testing module

Analytics and optimization Unbundled: Extension

License MLA

Issues

ABTEST

Maven site

A/B/n Testing

Latest

2.0.0

Available on SPA/Headless

No

Note that A/B/n Testing comes with a special license and runs as a connected service. Contact your Magnolia Customer Success Manager or Support to find out more.

The A/B/n Testing module installs the A/B/n Testing app and allows you to perform A/B/n testing via the Magnolia UI.

Test results from the A/B/n Testing module rely on the Analytics dashboard from the Analytics module.

Module structure

artifactID Description

magnolia-abtesting

Provides a default implementation to connect to the A/B/n Testing app.

magnolia-abtesting-ui

Provides the UI configuration for the A/B/n Testing app.

Installing with Maven

Maven is the easiest way to install the module. Add the following to your bundle:

<dependency>
  <groupId>info.magnolia.abtesting</groupId>
  <artifactId>magnolia-abtesting</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>.
<dependency>
  <groupId>info.magnolia.abtesting</groupId>
  <artifactId>magnolia-abtesting-ui</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>.

Configuring

To configure the following properties, go to ab-testing/config.yaml.

If you don’t see a config.yaml file, create one.

In the module configuration, you can:

  • Define the minimum unique visitors required to produce test results.

  • Adjust the minimum sample size required.

  • Set a limit to the maximum number of remaining days.

You must also make sure your site allows CORS for the POST method.
ab-testing/config.yaml
abTestingConfig:
  apiGatewayURL: your-api-gateway-url
  authorEventsExcluded: true
  resultsConfig:
    minUniqueVisitors: 10
    minSampleSize: 1
  statsEngineConfig:
    maxRemainingDays: 10
Property Description

apiGatewayURL

required

The API gateway URL provided to you by Magnolia.

authorEventsExcluded

Default is true.

Excludes author instance events from the test.

minUniqueVisitors

required

The number of minimum unique visitors that must visit the site before results are produced.

minSampleSize

required

The minimum overall sample size that must visit the site before results are produced.

maxRemainingDays

required

The maximum number of days that can remain in an A/B/n testing campaign.

Enabling CORS for the POST method

You must add the POST method to the list of allowed HTTP methods in your CORS configuration (Cross-origin Resource Sharing).

User events for the test variants are submitted to the Magnolia REST endpoint using the POST method.

Typically, only the GET method is enabled by default in Magnolia.

To enable the POST method:

  1. Locate the allowedMethods node in your site definition.

    Typically, a property named get with the value GET is configured by default. This means the site allows CORS only for the GET method.

  2. Update the get property name to all and the value to *.

    This effectively adds the POST method required by A/B/n Testing.

  3. Publish the change so that your public instance configuration is updated.

Travel site example

CORS configuration

Configuring permissions

Magnolia provides user role configuration to facilitate permissions for A/B/n testing.

Existing users with editor or publisher roles are be granted read only permissions to A/B/n Testing so that they can see the status of tests in the Pages app.

An ab-tester role is configured by default and grants minimal permissions to users to access the A/B/n Testing app tile in AdminCentral and create and start A/B/n tests.

You must assign the ab-tester role to your new and existing users who will use A/B/n Testing.

You can view, assign and edit roles in the Security app.

Usage

When running an A/B/n test using the A/B/n Testing module, a copy of the original page in the website workspace is made when the first variant is created.

The original page in the website workspace isn’t locked. Only the copy is locked and is used as a base for any variants created.

When a page is being used in a test, the status of the test in displayed in the Pages app. Variants of the original page are not displayed in the Pages app, only the original. If you want to see the variants, you can find them in the JCR Browser.

JCR browser

Editors can continue working on the original page while the test is running. When the test completes and users must decide which variant to use, they’re warned that the original has changed since the test began.

Segments are not locked nor copied when used in a test. The effect of deleting or modifying a segment that is actively being used in a test is not defined. We recommend you create dedicated segments for each test.

For more information, see the A/B/n Testing app page.

A/B/n Testing for multilanguage websites

In the context of a multilanguage website, the A/B/n Testing feature behaves differently depending on the multilanguage structure configured:

  • When using a single tree structure, the multilanguage content is stored in a single JCR content node. This means that you must create one A/B/n Testing test that covers all the languages.

    For example, if your website is in German and French, one test covers both languages.

  • When using a multiple tree structure, the multilanguage structure consists of separate website trees. In this case, you must create an A/B/n Testing test for each language that you want to test.

    For example, if you want to run the same test for German and French, you must create two A/B/n Testing tests: one for each language.

The A/B/n Testing feature doesn’t work with the URL Translation incubator module.

Uninstalling

To uninstall the A/B/n Testing module:

  1. Go to the A/B/n Testing app, abort any running tests and delete all tests.

  2. In your site definition, revert any changes you made when Enabling CORS for the POST method.

  3. Go to the Configuration app and delete the following nodes:

    • /modules/cache/config/contentCaching/defaultPageCache/cachePolicy/shouldBypassVoters/deny/abTestingPage

    • /server/filters/cms/testingResolver

    • /server/filters/sitemesh/decoratedFilter/contentProcessor/tagRuleBundles/magnolia/rules/DisableLinksInABVariantEditor

  4. Shut down Magnolia, remove the A/B/n Testing module JAR files (magnolia-abtesting.jar, magnolia-abtesting-ui.jar) from WEB-INF/lib and start up Magnolia again.

If you don’t use the Analytics module outside of A/B/n Testing purposes, delete the magnolia-analytics-x JAR files too.

Changelog

Magnolia A/B/n Testing 2.0.0 is in beta with Magnolia 6.3.

We are updating docs based on development and feedback. Consider the beta docs currently in a state of progress and not final.

The changelog gives you an overview of the changes made to the A/B/n Testing module at each release.

This changelog covers the 2.x branch of the A/B/n Testing module. For the 1.x changelog, see here.

2.0.0-beta2

Not yet publicly released

This release is compatible with Magnolia 6.3.

Compatibility

A/B/n Testing module version Magnolia version

2.0+

6.3+

1.0+

6.2.4+

Feedback

DX Core

×

Location

This widget lets you know where you are on the docs site.

You are currently perusing through the DX Core docs.

Main doc sections

DX Core Headless PaaS Legacy Cloud Incubator modules
6.3 beta
X

Magnolia 6.3 beta

Magnolia 6.3 is in beta. We are updating docs based on development and feedback. Consider the 6.3 docs currently in a state of progress and not final.

We are working on some 6.3-beta known issues during this phase.