Swift publication module

Collaboration and workflow Unbundled: Extension

Edition

DX Core

License

MLA

Issues

SWIFTPUB

Maven site

Swift publication

Latest

2.0.0

This module has yet to be released. If you want to learn more, contact us at swift.publishing@magnolia-cms.com for information on availability and technology previews.

Swift publication is a standalone enterprise module that can be used as a drop-in replacement for DX Core publishing. It stores versions in an external version store, such as S3 or Azure Blob Storage. When an author publishes changes, the changes are sent to and stored in the external version store. Public instances then receive notifications to fetch the corresponding version from the version store.

It’s recommended that a message broker like RabbitMQ be used to communicate between author and public instances to coordinate the information needed for publishing. This allows instances to communicate asynchronously even if an instance is down or not reachable directly. For development and testing environments, you can directly coordinate via REST, bypassing the need for a message broker. Both the message broker coordination and external version store require additional infrastructure.

Module structure

The module consists of several submodules.

ArtifactID Description

magnolia-swift-versioning-api

Provides an API to create and restore versions in external version store.

magnolia-swift-versioning-ui

Includes the integration of the user interface to manage versions in a module. Provides a means to manage versions in a module, including implementations for version checks and restoration.

magnolia-swift-versioning-s3

Provides functionality to implement S3 version storage.

magnolia-swift-versioning-azure

Provides functionality to implement Azure version storage.

magnolia-swift-publication-ui

Provides decorations for the Pages app, publishing dialogs, and i18n labels for the new UI buttons.

magnolia-swift-publication-api

Provides an API for external publishing, including services to process publication requests sent by authors, to represent problems that occur when a parent node is not yet published, and the ordering strategy during publication.

magnolia-swift-publication-rabbitmq-connector

Provides the same functionality as REST publication but with RabbitMQ as a central coordinator.

It uses the functionality in magnolia-swift-publication-rabbitmq-infra to connect to and set up the RabbitMQ topology.

magnolia-swift-publication-rest

Provides a REST publication client with the following.

  • A REST publication endpoint

  • Publishing and unpublishing operations

  • Exception handling adhering to the problem details for HTTP APIs specification as defined by RFC 7807

  • Ensuring proper serialization and deserialization

  • Dealing with invalid API tokens

magnolia-swift-publication-personalization

Extends PublicationCommand and handles deletion for personalization when page variants are deleted.

magnolia-swift-publication-rabbitmq-infra

Provides functionality to set up the RabbitMQ topology (queues and exchanges). The magnolia-swift-publication-rabbitmq-connector module uses it for on-prem and local development. In PaaS, an operator uses this module.

Installing with Maven

Maven is the easiest way to install the module. Add the following to your bundle:

Some steps below are required, and some parts are optional.

  1. Add the following mandatory dependencies.

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-api</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-ui</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-api</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-ui</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

  2. Choose dependencies below depending on the storage service and communication between the author and public instances you wish to install.

    • Amazon S3

    • Microsoft Azure Blob

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-s3</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-versioning-azure</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

    • RabbitMQ using MQTT

    • REST publication

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-rabbitmq-connector</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>. 
    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-rest</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 Should you need to specify the module version, do it using <version>.

  3. If required, add a dependency for the Personalization module.

    <dependency>
  <groupId>info.magnolia.swift.publication</groupId>
  <artifactId>magnolia-swift-publication-personalization</artifactId>
  <version>2.0.0</version> (1)
</dependency>
    1 It’s good practice to use the version variable from the parent POM (see in the next step), hence {mgnlSwiftVersion} would be here instead of 2.0.0.

  4. Add the module version to the properties of the parent POM.

    Parent POM
    <properties>
  …​
  <mgnlSwiftVersion>2.0.0</mgnlSwiftVersion>
  …​
</properties>

    Update dependency management in your project. Use the 6.3 version of the dx-core-bundle-parent. Swift publication modules are dependent on core version 6.3.

MicroProfile Config

First, make sure you have receivers configured in the MicroProfile Config. If you’re using local development, you must add a default receiver. Then, add the Swift publication configuration section to your MicroProfile Config, as shown in the examples below.

The REST publication requires an API key to control access. APIs allow two software modules to exchange data, and are used here for publication between author and public instances.

  • RabbitMQ using MQTT

  • REST publication

info:
  magnolia:
    develop: true
    superuser:
      enabled: true
      password: superuser!
    publishing:
      core:
        receiver: &receivers (1)
          - name: magnoliaPublic-0 (2)
            url: http://dev-magnolia-helm-public-0-svc:8080
            enabled: true
          - name: magnoliaPublic-1
            url: http://dev-magnolia-helm-public-1-svc:8080
            enabled: true
swift: (3)
  publication:
    rabbitmq:
      host: localhost
      user: <YOUR_USER>
      password: <YOUR_PASSWORD>
      vhost: / (4)
      journalThreshold: 900000
      retryIntervalInSeconds: 60
      publicationRecords:
        minutesToExpire: 60
        maxSize: 10000
      auditQueue:
        hoursToExpire: 24
        maxMessagesToHold: 10000
        maxSizeInBytes: 50000000
    receivers: *receivers
1 Configure your receivers.
2 Similar to other modules, defining your public instances for the workflow is mandatory. You can change the public instance name for the hostname in development environments. RabbitMQ generates a queue setup for each entry based on the name of the public instance if you decide to provision it.
3 Add the Swift publication configuration section.
4 RabbitMQ allows you to create virtual hosts to group exchanges, connections, queues, bindings, user permissions, and other system resources. If you’re using vhosts in RabbitMQ, you must provide the correct vhost property to reach the correct message exchanges. For more, see RabbitMQ Virtual Hosts. 
info:
  magnolia:
    develop: true
    superuser:
      enabled: true
      password: superuser!
    publishing:
      core:
        receiver: &receivers (1)
          - name: magnoliaPublic-0 (2)
            url: http://dev-magnolia-helm-public-0-svc:8080
            enabled: true
          - name: magnoliaPublic-1
            url: http://dev-magnolia-helm-public-1-svc:8080
            enabled: true
swift: (3)
  publication:
    receivers: *receivers
    rest:
      api:
        token: <API_KEY> (4)
1 Configure your receivers.
2 Similar to other modules, defining your public instances for the workflow is mandatory. You can change the public instance name for the hostname in development environments.
3 Add the Swift publication configuration section.
4 Put the mandatory API key value here; any password manager can create it. Remember that this value should be the same for the author and the public instances.

Version store configuration

You must extend your MicroProfile Config from the last section further. Depending on the external store you have set up, copy and adapt the following versioning section into your MicroProfile Config.

  • Amazon S3

  • Microsoft Azure Blob

swift:
  publication:
    …​
  versioning:
    s3:
      bucket: <YOUR_S3_BUCKET> (1)
      region: <AWS_REGION> (2)
      rootFolder: <YOUR_FOLDER> (3)
      connection-acquire-timeout: <YOUR_CONNECTION_ACQUIRE_TIMEOUT> (4)
      max-connections: <YOUR_MAX_CONNECTIONS> (5)
      max-connection-acquires: <YOUR_MAX_CONNECTION_ACQUIRES> (6)
      rate-limit-period: <YOUR_RATE_LIMIT_PERIOD> (7)
      rate-limit: <YOUR_RATE_LIMIT> (7)
      rate-limit-timeout: <YOUR_RATE_LIMIT_TIMEOUT> (7)
1 Specify your S3 bucket name.
2 Specify the AWS Region you’re using.
3 If you want to reuse the same bucket across different environments, like test or production, you can save your publications in a different space per environment. You can do this by modifying the root folder as a base folder for publication storage.
4 Timeout in seconds for acquiring an S3 connection from the pool. The default value is 10. See AWS SDK for Java API Reference for more details.
5 Maximum number of S3 connections in the pool. The default value is 50. See AWS SDK for Java API Reference for more details.
6 Maximum number of pending requests when the pool is empty. Additional requests are queued pending a free connection in the pool if all connections are in use. The default value is 10000. See AWS SDK for Java API Reference for more details.
7 Configuring the request rate limit.

rate-limit-period: Time period in milliseconds during which only a certain amount of calls, defined by the rate-limit, are permitted. The default value is 100.

rate-limit: Limit for the maximum number of calls permitted during a given time (the rate-limit-period). The default value is 5.

rate-limit-timeout: Duration in milliseconds, after which a call times out if the current rate limit doesn’t allow its execution. The default value is 60000.

AWS S3 bucket names are globally unique, meaning no two buckets can have the same name. If you try to create a bucket and receive an error stating the bucket already exists, it means that the bucket name is already in use, either by you or someone else. 
swift:
  publication:
    …​
  versioning:
    azure:
      blobConnectionString: <YOUR_CONNECTION_STRING> (1)
      account: devstoreaccount1
      container: aContainer
      clientId: clientId (2)
      clientSecret: clientSecret (2)
      tenantId: tenantId (2)
      subscription: subscription
      resourceGroup: resourceGroup
      rootFolder: <YOUR_FOLDER> (3)
      rate-limit-period: <YOUR_RATE_LIMIT_PERIOD> (4)
      rate-limit: <YOUR_RATE_LIMIT> (4)
      rate-limit-timeout: <YOUR_RATE_LIMIT_TIMEOUT> (4)
1 If you use a connection string for Azure Storage, it contains all the necessary information for connecting to your Azure Storage account.
2 A BlobServiceClientBuilder is used in conjunction with ClientSecretCredential to build a connection using these configuration properties. If a connection string is also specified, it takes precedence over these properties.
3 If you want to reuse the same blobs across different environments, like test or production, you can save your publications in a different space per environment. You can do this by modifying the root folder as a base folder for publication storage.
4 Configuring the request rate limit.

rate-limit-period: Time period in milliseconds during which only a certain amount of calls, defined by the rate-limit, are permitted. The default value is 100.

rate-limit: Limit for the maximum number of calls permitted during a given time (the rate-limit-period). The default value is 5.

rate-limit-timeout: Duration in milliseconds, after which a call times out if the current rate limit doesn’t allow its execution. The default value is 60000.

AWS connection

The magnolia-aws-foundation module handles all Amazon connections from Magnolia. It’s installed automatically by Maven when you install any AWS-dependent module.

To use AWS in Magnolia, you must have a working AWS account.

You need AWS credentials to connect AWS to Magnolia. Credentials consist of:

  • AWS access key ID

  • AWS secret access key

  • Optionally, a session token (when using the AWS default credential provider chain)

Generate the key in the security credentials section of the Amazon IAM Management Console. In the navigation bar on the upper right, choose your user name, and then choose My Security Credentials.

Using the AWS default credential provider chain

The AWS SDK uses a chain of sources to look for credentials in a specific order. For more information, see Default credentials provider chain.

  1. Set your AWS credentials by following the instructions in the AWS documentation: Provide temporary credentials to the SDK.

    For a more secure implementation using the default credential provider chain, we recommend using a session token, which expires, rather than a permanent user token.

  2. Disable Magnolia’s internal credential handling by doing one of the following:

    1. Adding the following configuration properties to your WEB-INF/config/default/magnolia.properties file:

      magnolia.aws.validateCredentials=false
magnolia.aws.useCredentials=false

    2. Using JVM arguments as shown in the next step.

  3. Set your AWS session or user token. AWS credentials can be injected using environment variables or JVM system properties. For more details, see Default credentials provider chain and Configure access to temporary credentials.

    Example configuration with a session token and JVM arguments
    -Dmagnolia.aws.validateCredentials=false(1)
-Dmagnolia.aws.useCredentials=false(1)
-Daws.accessKeyId=$AWS_ACCESS_KEY_ID(2)
-Daws.secretAccessKey=$AWS_SECRET_ACCESS_KEY(2)
-Daws.sessionToken=$AWS_SESSION_TOKEN(2)(3)
    1 Disables Magnolia’s internal credential handling using JVM properties.
    2 JVM properties to inject environment variables containing the AWS credentials. Ensure that your environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN are set.
    3 AWS_SESSION_TOKEN is optional.
    Example configuration with a permanent user token
    -Dmagnolia.aws.validateCredentials=false
-Dmagnolia.aws.useCredentials=false
-Daws.accessKeyId=<your-access-key-id>
-Daws.secretAccessKey=<your-secret-access-key>

RabbitMQ configuration

  1. Ensure you have added the RabbitMQ dependency, magnolia-swift-publication-rabbitmq-connector, to your custom DX Core webapp.

    Don’t include the dependency for REST publication.

  2. Ensure you have configured the rabbitmq section of the MicroProfile Config.

  3. Start your RabbitMQ instance. For details on local setups, see Local development for testing.

  4. Start your Magnolia instance.

    With the RabbitMQ instance running and the configuration steps above complete, starting your Magnolia instance configures the additional RabbitMQ infrastructure for the author and public instances to communicate over the message broker. In particular, it configures the exchanges and queues in RabbitMQ and uses your receiver name for the queue naming setup.

  5. Log into your RabbitMQ dashboard and go to the Queues and Streams tab. The infrastructure should already be set up as shown in the example below.

    Queues and streams tab

  6. The infrastructure provided manages messages for asynchronous Magnolia publication. You can find the content of the messages in the Payload field. For example, the example message below is in the Get messages section of the audit-queue.

    Message payload field

Local development for testing

For local development, you can start a container for RabbitMQ by following the steps in this section.

Below is an example MicroProfile Config for local development.

magnolia:
  develop: true
  superuser:
    enabled: true
    password: superuser
  publishing:
    receivers: &receivers (1)
      - name: magnoliaPublic
        url: http://localhost:8090/
        enabled: true
swift:
  publication:
    rabbitmq:
      host: localhost
      user: <YOUR_USER>
      password: <YOUR_PASSWORD>
      journalThreshold: 900000
      retryIntervalInSeconds: 60
      publicationRecords:
        minutesToExpire: 60
        maxSize: 10000
      auditQueue:
        hoursToExpire: 24
        maxMessagesToHold: 10000
        maxSizeInBytes: 50000000
    receivers: *receivers
1 The receiver’s value in your local author instance could also be your computer’s local hostname or local network name, such as My-Computer.local on a Mac.

  1. Install Docker Desktop if you don’t have it already on your system.

  2. Create a docker-compose.yaml file in a separate folder from your Magnolia installation with the following configuration.

    /docker-compose.yaml
    version: '3'

services:
  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    hostname: rabbitmq
    ports:
      - "5672:5672"
      - "15672:15672"
    volumes:
      - ./rabbitmq-data:/var/lib/rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: magnolia
      RABBITMQ_DEFAULT_PASS: magnolia (1)
  minio:
    image: quay.io/minio/minio:RELEASE.2024-04-18T19-09-19Z
    hostname: minio
    container_name: minio
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - ./minio-data:/data
    environment:
      MINIO_ROOT_USER: magnolia
      MINIO_ROOT_PASSWORD: magnolia
    command: server --address 0.0.0.0:9000 --console-address :9001 /data
  minio-init: (2)
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc alias set myminio http://minio:9000 magnolia magnolia;
      /usr/bin/mc mb myminio/minio-swift-bucket;
      /usr/bin/mc version enable myminio/minio-swift-bucket;
      exit 0;
      "
    1 Set a secure password for your instance.
    2 The minio-init automatically creates the minio-swift-bucket bucket with the version enabled. An example of the S3 versioning for local development is given below.

    • S3 versioning example

    • AWS credentials Example JVM arguments

    swift:
  versioning:
    s3:
      bucket: minio-swift-bucket
      region: ap-southeast-1
      rootFolder: /minio-swift
      endpoint: http://localhost:9000
      pathstyle: true
    -Dmagnolia.aws.validateCredentials=false
-Dmagnolia.aws.useCredentials=false
-Daws.accessKeyId=magnolia
-Daws.secretAccessKey=magnolia

  3. Open a terminal and build the docker container for the RabbitMQ instance.

    Starting the RabbitMQ instance
    docker-compose up -d --build

  4. You can log into RabbitMQ using the port and credentials defined in the docker-compose.yaml file.

    Logging into RabbitMQ

Related topics
Feedback

DX Core

×

Location

This widget lets you know where you are on the docs site.

You are currently perusing through the DX Core docs.

Main doc sections

DX Core Headless PaaS Legacy Cloud Incubator modules