Swift publication module

Edition

DX Core

License

MLA

Issues

SWIFTPUB

Maven site

Swift publication

Latest

1.0.0

This module has yet to be released. If you want to learn more, contact us at swift.publishing@magnolia-cms.com for information on availability and technology previews.

Swift publication is a standalone enterprise module that can be used as a drop-in replacement for DX Core publishing. It stores versions in an external version store, such as S3 or Azure Blob Storage. When an author publishes changes, the changes are sent to and stored in the external version store. Public instances then receive notifications to fetch the corresponding version from the version store.

It’s recommended that a message broker like RabbitMQ be used to communicate between author and public instances to coordinate the information needed for publishing. This allows instances to communicate asynchronously even if an instance is down or not reachable directly. For development and testing environments, you can directly coordinate via REST, bypassing the need for a message broker. Both the message broker coordination and external version store require additional infrastructure.

Module structure

The module consists of several submodules.

ArtifactID Description

magnolia-swift-versioning-api

Provides an API to create and restore versions in an external version store.

magnolia-swift-versioning-ui

Includes the integration of the user interface to manage versions in a module. Provides a means to manage versions in a module, including implementations for version checks and restoration.

magnolia-swift-versioning-s3

Provides functionality to implement S3 version storage.

magnolia-swift-versioning-azure

Provides functionality to implement Azure version storage.

magnolia-swift-publication-ui

Provides decorations for the Pages app, publishing dialogs, and i18n labels for the new UI buttons.

magnolia-swift-publication-api

Provides an API for external publishing, including services to process publication requests sent by authors, to represent problems that occur when a parent node is not yet published, and the ordering strategy during publication.

magnolia-swift-publication-rabbitmq-connector

Provides the same functionality as REST publication but with RabbitMQ as a central coordinator.

It uses the functionality in magnolia-swift-publication-rabbitmq-infra to connect to and set up the RabbitMQ topology.

magnolia-swift-publication-rest

Provides a REST publication client with the following.

  • A REST publication endpoint

  • Publishing and unpublishing operations

  • Exception handling adhering to the problem details for HTTP APIs specification as defined by RFC 7807

  • Ensuring proper serialization and deserialization

  • Dealing with invalid API tokens

magnolia-swift-publication-personalization

Extends PublicationCommand and handles deletion for personalization when page variants are deleted.

magnolia-swift-publication-rabbitmq-infra

Provides functionality to set up the RabbitMQ topology (queues and exchanges). The magnolia-swift-publication-rabbitmq-connector module uses it for on-prem and local development. In PaaS, an operator uses this module.

Installing with Maven

Bundled modules are automatically installed for you.

If the module is unbundled, add the following to your bundle including your project’s <dependencyManagement> section and your webapp’s <dependencies> section. If the module is unbundled but the parent POM manages the version, add the following to your webapp’s <dependencies> section.

Some steps below are required, and some parts are optional.

  1. Add the following mandatory dependencies.

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-api</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-ui</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-api</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-ui</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

  2. Choose dependencies below depending on the storage service and communication between the author and public instances you wish to install.

    • Amazon S3

    • Microsoft Azure Blob

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-s3</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-azure</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

    • RabbitMQ using MQTT

    • REST publication

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-rabbitmq-connector</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-rest</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

  3. If required, add a dependency for the Personalization module.

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-personalization</artifactId>
  <version>1.0.0</version> (1)
</dependency>
    1 It’s good practice to use the version variable from the parent POM (see in the next step), hence {mgnlSwiftVersion} would be here instead of 1.0.0.

  4. Add the module version to the properties of the parent POM.

    Parent POM
    <properties>
  …​
  <mgnlSwiftVersion>1.0.0</mgnlSwiftVersion>
  …​
</properties>

    Update dependency management in your project.

Configuration

Receivers

First, make sure you have receivers configured directly in JCR at the location below or modify them via decoration under /modules/publishing-core/config/receivers.

Table 1. /modules/publishing-core/config/receivers
Node name Value

⸬ receivers

     ⸬ magnoliaPublic8080

         ⬩ url

http://localhost:8080/magnoliaPublic

         ⬩ enabled

true

This node name is used for the queue naming setup.

Alternatively, specify them in a properties file.

swift.publication.receivers[0].name=public1
swift.publication.receivers[0].url=http://public1:8080/magnoliaPublic
swift.publication.receivers[0].enabled=true
swift.publication.receivers[1].name=public2
swift.publication.receivers[1].url=http://public2:8080/magnoliaPublic
swift.publication.receivers[1].enabled=true

Similar to other modules, defining your public instances for the workflow is mandatory. You can change the public instance name for the hostname in development environments. RabbitMQ generates a queue setup for each entry based on the name of the public instance if you decide to provision it.

Instance communication

Depending on whether you want to use a message broker or REST communication, you need to add property files for module customization or JVM arguments/magnolia properties for sensitive data as credentials.

Do not store your credentials in your versioning system.

The REST publication requires an API key to control access. APIs allow two software modules to exchange data and are used here for publication between author and public instances.

  • RabbitMQ using MQTT

  • REST publication

You can store properties in all property sources, such as system properties or the magnolia.properties file example below.
/apache-tomcat/webapps/magnoliaAuthor/WEB-INF/config/default/magnolia.properties
config:
swift.publication.rabbitmq.journalThreshold=900000
swift.publication.rabbitmq.retryIntervalInSeconds=60
swift.publication.rabbitmq.publicationRecords.minutesToExpire=60
swift.publication.rabbitmq.publicationRecords.maxSize=10000
swift.publication.rabbitmq.auditQueue.hoursToExpire=24
swift.publication.rabbitmq.auditQueue.maxMessagesToHold=10000
swift.publication.rabbitmq.auditQueue.maxSizeInBytes=100000000
Example configuration for the JVM arguments
-Dswift.publication.rabbitmq.host=<YOUR_HOST> (1)
-Dswift.publication.rabbitmq.port=optional (2)
-Dswift.publication.rabbitmq.user=<YOUR_USER> (1)
-Dswift.publication.rabbitmq.password=<YOUR_PASSWORD> (1)
1 Adjust to the host and credentials for your instance.
2 The port is optional, as it uses 5672 by default.
Example configuration for the JVM argument
-Dswift.publication.rest.api.token=<API_KEY> (1)
1 Put the mandatory API key value here; any password manager can create it. Remember that this value should be the same for the author and the public instances.

Versioning store

You need to add several properties in this section to configure instance communication between author and public instances. Start by adding the general versioning pool size property below.

/apache-tomcat/webapps/magnoliaAuthor/WEB-INF/config/default/magnolia.properties
swift.versioning.poolSize=9

Depending on the external store you have set up, copy and adapt one of the following configurations. Additionally, you must add credentials for Microsoft Azure Blob and Amazon S3. For Microsoft Azure Blob, see the example JVM arguments below. For Amazon S3, find details in the AWS connection section below.

  • Amazon S3

  • Microsoft Azure Blob

You can store properties in all property sources, such as system properties or the magnolia.properties file example below.
/apache-tomcat/webapps/magnoliaAuthor/WEB-INF/config/default/magnolia.properties
swift.versioning.s3.bucket=<YOUR_S3_BUCKET> (1)
swift.versioning.s3.region=<AWS_REGION> (2)
swift.versioning.s3.rootFolder=<YOUR_FOLDER> (3)
1 Specify your S3 bucket name.
2 Specify the AWS Region you’re using.
3 If you want to reuse the same bucket across different environments, like test or production, you can save your publications in a different space per environment. You can do this by modifying the root folder as a base folder for publication storage.
AWS S3 bucket names are globally unique, meaning no two buckets can have the same name. If you try to create a bucket and receive an error stating the bucket already exists, it means that the bucket name is already in use, either by you or someone else.
You can store properties in all property sources, such as system properties or the magnolia.properties file example below.
/apache-tomcat/webapps/magnoliaAuthor/WEB-INF/config/default/magnolia.properties
swift.versioning.azure.account=devstoreaccount1
swift.versioning.azure.container=aContainer
swift.versioning.azure.subscription=subscription
swift.versioning.azure.resourceGroup=resourceGroup
swift.versioning.azure.rootFolder=<YOUR_FOLDER> (1)
1 If you want to reuse the same blobs across different environments, like test or production, you can save your publications in a different space per environment. You can do this by modifying the root folder as a base folder for publication storage.
Example configuration for the JVM arguments
-Dswift.versioning.azure.clientId=<YOUR_CLIENT_ID>
-Dswift.versioning.azure.clientSecret=<YOUR_CLIENT_SECRET>
-Dswift.versioning.azure.tenantId=<YOUR_TENANT_ID>

AWS connection

The magnolia-aws-foundation module handles all Amazon connections from Magnolia. It’s installed automatically by Maven when you install any AWS-dependent module.

To use AWS in Magnolia, you must have a working AWS account.

You need AWS credentials to connect AWS to Magnolia. Credentials consist of:

  • AWS access key ID

  • AWS secret access key

  • Optionally, a session token (when using the AWS default credential provider chain)

Generate the key in the security credentials section of the Amazon IAM Management Console. In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials.

Using the AWS default credential provider chain

The AWS SDK uses a chain of sources to look for credentials in a specific order. For more information, see Default credentials provider chain.

  1. Set your AWS credentials by following the instructions in the AWS documentation: Provide temporary credentials to the SDK.

    For a more secure implementation using the default credential provider chain, we recommend using a session token, which expires, rather than a permanent user token.

  2. Disable Magnolia’s internal credential handling by doing one of the following:

    1. Adding the following configuration properties to your WEB-INF/config/default/magnolia.properties file:

      magnolia.aws.validateCredentials=false
magnolia.aws.useCredentials=false

    2. Using JVM arguments as shown in the next step.

  3. Set your AWS session or user token. AWS credentials can be injected using environment variables or JVM system properties. For more details, see Default credentials provider chain and Configure access to temporary credentials.

    Example configuration with a session token and JVM arguments
    -Dmagnolia.aws.validateCredentials=false(1)
-Dmagnolia.aws.useCredentials=false(1)
-Daws.accessKeyId=$AWS_ACCESS_KEY_ID(2)
-Daws.secretAccessKey=$AWS_SECRET_ACCESS_KEY(2)
-Daws.sessionToken=$AWS_SESSION_TOKEN(2)(3)
    1 Disables Magnolia’s internal credential handling using JVM properties.
    2 JVM properties to inject environment variables containing the AWS credentials. Ensure that your environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN are set.
    3 AWS_SESSION_TOKEN is optional.
    Example configuration with a permanent user token
    -Dmagnolia.aws.validateCredentials=false
-Dmagnolia.aws.useCredentials=false
-Daws.accessKeyId=<your-access-key-id>
-Daws.secretAccessKey=<your-secret-access-key>

Starting RabbitMQ and Magnolia

  1. Ensure you have added the RabbitMQ dependency, magnolia-swift-publication-rabbitmq-connector, to your custom DX Core webapp.

    Don’t include the dependency for REST publication.

  2. Ensure you have added the necessary RabbitMQ decoration.

  3. Start your RabbitMQ instance. For details on local setups, see Local development for testing.

  4. Start your Magnolia instance.

    With the RabbitMQ instance running and the configuration steps above complete, starting your Magnolia instance configures the additional RabbitMQ infrastructure for the author and public instances to communicate over the message broker. In particular, it configures the exchanges and queues in RabbitMQ and uses your receiver name for the queue naming setup.

  5. Log into your RabbitMQ dashboard and go to the Queues and Streams tab. The infrastructure should already be set up as shown in the example below.

    Queues and streams tab

  6. The infrastructure provided manages messages for asynchronous Magnolia publication. You can find the content of the messages in the Payload field. For example, the example message below is in the Get messages section of the audit-queue.

    Message payload field

Local development for testing

For local development, you can start a container for RabbitMQ by following the steps in this section.

The receiver’s value in your local author instance could also be your computer’s local hostname or local network name, such as My-Computer.local on a Mac.

  1. Install Docker Desktop if you don’t have it already on your system.

  2. Create a docker-compose.yaml file in a separate folder from your Magnolia installation with the following configuration.

    /docker-compose.yaml
    version: '3'

services:

  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    hostname: rabbitmq
    ports:
      - "5672:5672"
      - "15672:15672"
    volumes:
      - ./rabbitmq-data:/var/lib/rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: magnolia
      RABBITMQ_DEFAULT_PASS: <YOUR_PASSWORD> (1)
    1 Set a secure password for your instance.

  3. Open a terminal and build the docker container for the RabbitMQ instance.

    Starting the RabbitMQ instance
    docker-compose up -d --build

  4. You can log into RabbitMQ using the port and credentials defined in the docker-compose.yaml file.

    Logging into RabbitMQ

Related topics
Feedback

DX Core

×

Location

This widget lets you know where you are on the docs site.

You are currently perusing through the DX Core docs.

Main doc sections

DX Core Headless PaaS Legacy Cloud Incubator modules