A/B/n Testing module
| License | MLA |
|---|---|
Issues |
|
Maven site |
|
Latest |
1.0.1 |
No |
| Note that A/B/n Testing comes with a special license and runs as a connected service. Please contact your Magnolia Customer Success Manager or Support to find out more. |
The A/B/n Testing module installs the A/B/n Testing app and allows you to perform A/B/n testing via the Magnolia UI.
| Test results from the A/B/n Testing module rely on the Analytics dashboard from the Analytics module. |
Module structure
| artifactID | Description |
|---|---|
|
Provides a default implementation to connect to the A/B/n Testing app. |
|
Provides the UI configuration for the A/B/n Testing app. |
Installing with Maven
Bundled modules are automatically installed for you.
If the module is unbundled, add the following to your bundle including your project’s <dependencyManagement> section and your webapp’s <dependencies> section.
If the module is unbundled but the parent POM manages the version, add the following to your webapp’s <dependencies> section.
<dependency>
<groupId>info.magnolia.abtesting</groupId>
<artifactId>magnolia-abtesting</artifactId>
<version>1.0.1</version> (1)
</dependency>| 1 | Should you need to specify the module version, do it using <version>. |
<dependency>
<groupId>info.magnolia.abtesting</groupId>
<artifactId>magnolia-abtesting-ui</artifactId>
<version>1.0.1</version> (1)
</dependency>| 1 | Should you need to specify the module version, do it using <version>. |
Configuring
To configure the following properties, go to
ab-testing/config.yaml.
If you do not see a config.yaml file, create one.
In the module configuration, you can:
-
Define the minimum unique visitors required to produce test results.
-
Adjust the minimum sample size required.
-
Set a limit to the maximum number of remaining days.
| You must also make sure your site allows CORS for the POST method. |
abTestingConfig:
apiGatewayURL: your-api-gateway-url
authorEventsExcluded: true
resultsConfig:
minUniqueVisitors: 10
minSampleSize: 1
statsEngineConfig:
maxRemainingDays: 10| Property | Description |
|---|---|
|
required The API gateway URL provided to you by Magnolia. |
|
Default is Excludes author instance events from the test. |
|
required The number of minimum unique visitors that must visit the site before results are produced. |
|
required The minimum overall sample size that must visit the site before results are produced. |
|
required The maximum number of days that can remain in an A/B/n testing campaign. |
Enabling CORS for the POST method
You must add the POST method to the list of allowed HTTP methods in your CORS configuration (Cross-origin Resource Sharing).
User events for the test variants are submitted to the Magnolia REST endpoint using the POST method.
Typically, only the GET method is enabled by default in Magnolia.
To enable the POST method:
-
In the Site app, go to
<your-site>/cors/<your-site>/allowedMethods.Typically, a property named
getwith the valueGETis configured by default. This means the site allows CORS only for the GET method. -
Update the
getproperty name toalland the value to*.This effectively adds the POST method required by A/B/n Testing.
-
Publish the change so that your public instance configuration is updated.
Configuring permissions
Magnolia provides user role configuration to facilitate permissions for A/B/n testing.
Existing users with editor or publisher roles are be granted read only permissions to A/B/n Testing so that they can see the status of tests in the Pages app.
An ab-tester role is configured by default and grants minimal permissions to users to access the A/B/n Testing app tile in AdminCentral and create and start A/B/n tests.
You must assign the ab-tester role to your new and existing users who will use A/B/n Testing.
You can view, assign and edit roles in the Security app.
Usage
When running an A/B/n test using the A/B/n Testing module, a copy of the original page in the website workspace is made when the first variant is created.
The original page in the website workspace is not locked. Only the copy is locked and is used as a base for any variants created.
When a page is being used in a test, the status of the test in displayed in the Pages app. Variants of the original page are not displayed in the Pages app, only the original. If you want to see the variants, you can find them in the JCR Browser.
Editors can continue working on the original page while the test is running. When the test completes and users must decide which variant to use, they are warned that the original has changed since the test began.
| Segments are not locked nor copied when used in a test. The effect of deleting or modifying a segment that is actively being used in a test is not defined. We recommend you create dedicated segments for each test. |
For more information, see the A/B/n Testing app page.
A/B/n Testing for multilanguage websites
In the context of a multilanguage website, the A/B/n Testing feature behaves differently depending on the multilanguage structure configured:
-
When using a single tree structure, the multilanguage content is stored in a single JCR content node. This means that you must create one A/B/n Testing test that covers all the languages.
For example, if your website is in German and French, one test covers both languages.
-
When using a multiple tree structure, the multilanguage structure consists of separate website trees. In this case, you must create an A/B/n Testing test for each language that you want to test.
For example, if you want to run the same test for German and French, you must create two A/B/n Testing tests: one for each language.
| The A/B/n Testing feature does not work with the URL Translation incubator module. |
Uninstalling
To uninstall the A/B/n Testing module:
-
Go to the A/B/n Testing app, abort any running tests and delete all tests.
-
Go to the Site app and revert any changes you made when Enabling CORS for the POST method.
-
Go to the Configuration app and delete the following nodes:
-
/modules/cache/config/contentCaching/defaultPageCache/cachePolicy/shouldBypassVoters/deny/abTestingPage -
/server/filters/cms/testingResolver -
/server/filters/sitemesh/decoratedFilter/contentProcessor/tagRuleBundles/magnolia/rules/DisableLinksInABVariantEditor
-
-
Shut down Magnolia, remove the A/B/n Testing module JAR files (
magnolia-abtesting.jar,magnolia-abtesting-ui.jar) fromWEB-INF/liband start up Magnolia again.
If you do not use the Analytics module outside of A/B/n Testing purposes, delete the magnolia-analytics-x JAR files too.
|
Release history
A/B/n Testing module 1.0
Released on June 17, 2021.
General availability release of the A/B/n Testing Special Feature, which provides native A/B/n Testing to help you make decisions about your content based on real data from your existing traffic.
See A/B/n Testing for an overview of the functionality provided.
Note that A/B/n Testing is compatible with Magnolia CMS 6.2.4 onwards.