External DAM module

Digital asset management Unbundled: Extension

Magnolia External DAM module 2.0.0 is in beta with Magnolia 6.3.

We are updating docs based on development and feedback. Consider the beta docs currently in a state of progress and not final.

License

MLA

Issues

EXTDAM

Maven site

external-dam

Latest

2.0.0

Module structure

artifactID Description

magnolia-external-dam-parent

Parent reactor.

     magnolia-external-dam

Provides the foundation for integrating with S3 and Bynder and a sample component for using assets in the Pages app.

     magnolia-external-dam-bynder

Provides a default implementation to connect to Bynder.

     magnolia-external-dam-s3

Provides a default implementation to connect to Amazon S3.

Installing with Maven

Maven is the easiest way to install the module. Add the following to your bundle:

<dependency>
  <groupId>info.magnolia.external.dam</groupId>
  <artifactId>magnolia-external-dam</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>. 
<dependency>
  <groupId>info.magnolia.external.dam</groupId>
  <artifactId>magnolia-external-dam-bynder</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>. 
<dependency>
  <groupId>info.magnolia.external.dam</groupId>
  <artifactId>magnolia-external-dam-s3</artifactId>
  <version>2.0.0</version> (1)
</dependency>
1 Should you need to specify the module version, do it using <version>.

Connection configuration

This section explains how to configure the connection between the External DAM module and your external asset management solution.

Essentially, you:

  • Provide connection credentials for your external solution.

  • Enable the subapp to display in the Magnolia Assets app.

  • Configure the cache if necessary.

You can create or edit the configuration in the JCR or the File System (YAML) under <module-name>/external-dams/<definition-name>.

Before you configure the connection, make sure you know:

  • Which solution you want to connect to. Default implementations are provided for:

    • AWS S3

    • Bynder

  • Connection credentials for your solution.

If you want to connect to another third-party tool, extend the Magnolia DAM API and create your own asset provider
There are several additional DAM connector modules. For more details, see Digital asset management modules.

AWS S3 connection configuration

To use AWS S3 assets in Magnolia, you must have a working AWS account.

You need AWS S3 credentials to connect your S3 buckets and display their content in Magnolia. Credentials consist of two parts:

  • AWS access key ID

  • AWS secret access key

You can store your AWS credentials using:

  • The Magnolia Passwords app

  • The AWS default credential provider chain

AWS Foundation password manager

The magnolia-aws-foundation-password-manager dependency is necessary if you want to use the Password Manager module to store your AWS credentials.

To do this:

  1. Go to your webapp.

  2. Add the following to your webapp’s pom.xml file as a dependency:

    <dependency>
    <groupId>info.magnolia.aws.foundation</groupId>
    <artifactId>magnolia-aws-foundation-password-manager</artifactId>
    <version>${awsFoundationVersion}</version> (1)
</dependency>
    1 Where this is the latest compatible version needed for your webapp. As of 2024-06-26, the latest available version is 1.0.4.

Using the Magnolia Passwords app

Add the credentials to your Magnolia instance in the Password manager app.

For example:

aws-credentials

aws_access_key_id

aws_secret_access_key

Using the AWS default credential provider chain

  1. Add the following configuration properties to your WEB-INF/config/default/magnolia.properties file:

    magnolia.aws.validateCredentials=false
magnolia.aws.useCredentials=false

  2. Set your AWS credentials by following the instructions in the AWS documentation: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html

The magnolia-aws-foundation module handles all Amazon connections from Magnolia. This module is installed automatically by Maven when you install the External DAM module.

Configuring an AWS S3 bucket whitelist

By default, all the buckets in the S3 account you connect to are displayed. If you do not want to display all your buckets, you can configure a whitelist in the Resource files app using regex.

For example:

dam-s3/config.yaml
whitelistedBuckets:
- "^a.*"
- a-particular-bucket

Adding a filter capability to S3 buckets

If you want to restrict access to a single bucket either because it’s used in different use cases or for some confidentiality-related reasons, you can add a bucket prefix that grants access only to that specific bucket without revealing its actual path which might contain unwanted access information. This way the user will see the bucket and folders will be seen as the only folders in the Magnolia instance.

To make use of the filter, simply add the following bucket prefix to your config.yaml file:

dam-s3/config.yaml
whitelistedBuckets:
	-  BUCKET_NAME
bucketPrefixes:
	BUCKET_NAME: “FOLDER_NAME/SUBFOLDER_NAME”

Bynder connection configuration

To use Bynder assets in Magnolia, you must have a working Bynder account.

API token

You need a permanent OAuth 2.0 API token from Bynder to access the assets.

To generate a token, see this Bynder documentation: https://help.bynder.com/system/oauth2-permanent-tokens.htm.

You can use the token value directly in the yaml configuration or add the permanent token to your Magnolia instance in the Password manager app and reference the path in the configuration.

For example:

bynder-credentials

Padlock bynder_permanent_token

Connecting

Add your connection configuration under /dam-bynder/config.yaml.

bynderPortalURL: https://magnoliaintegration.getbynder.com
permanentToken: <client_secret_or_path_to_password_manager>
Connection properties
Property Description

bynderPortalURL

required

URL to your Bynder service.

permanentToken

required

A permanent OAuth 2.0 API token from Bynder or the path to the Passwords app where the token is stored.

Enabling your subapp

By default, the AWS S3 and Bynder subapps are disabled.

To enable them, decorate the Magnolia Assets app under <dam-connector>/decorations/dam and set the enabled property to true.

Bynder enabled configuration example:

dam-bynder/decorations/dam/dam.config.yaml
providers:
  bynderProvider:
    identifier: bynder
    enabled: true
    class: info.magnolia.external.dam.bynder.BynderAssetProvider
    renderers:
      bynder:
        class: info.magnolia.external.dam.bynder.BynderAssetRenderer
        proxyMode: true
        mediaTypes:
          audio: 'audio/*'
          application: 'application/*'
          video: 'video/*'
          image: 'image/*'

S3 enabled configuration example:

dam-s3/decorations/dam/dam.config.yaml
providers:
  s3Provider:
    identifier: s3
    enabled: true
    class: info.magnolia.external.dam.s3.datasource.S3AssetProvider

Configuring a single asset provider for link fields

You can configure a link field to limit your users to selecting assets from only one asset provider.

This is possible from External DAM 1.0.7 (DAM module 3.0.8). Previously, you had to use the aggregated damLinkField and users had to choose from all the enabled asset providers.

This example shows a dialog definition with three link fields, each limited to different asset providers:

external-dam/dialogs/components/textExternalImage.yaml
form:
  properties:
    bynder:
      $type: damLinkField
      chooserId: dam-bynder:chooser
      datasource:
        class: info.magnolia.dam.app.data.AssetDatasourceDefinition
        name: bynder
    s3:
      $type: damLinkField
      chooserId: dam-s3:chooser
      datasource:
        class: info.magnolia.dam.app.data.AssetDatasourceDefinition
        name: s3
    jcr:
      $type: linkField
      converterClass: info.magnolia.ui.editor.converter.JcrAssetConverter
      datasource:
        $type: jcrDatasource
        workspace: website

Configuring the cache

This integration framework uses Caffeine, a high-performance Java cache library, to manage caching for external assets via a Magnolia helper module called magnolia-addon-commons-cache.

For Bynder and S3 caching, you can specify the behavior of the caches. Essentially, you can configure all the parameters available in CaffeineSpec.

For example:

  • Set a maximum cache size.

  • Define how often the cache expires.

Additionally, you can:

  • Enable or disable the cache.

You configure caching through decoration under /<dam-connector>/decorations/addon-commons-cache/config.yaml.

Bynder default configuration

/dam-bynder/decorations/addon-commons-cache/config.yaml
cacheConfigurations:
  bynder:
    caches:
      bynder-media: maximumSize=500, expireAfterAccess=10m
      bynder-all: expireAfterAccess=60m
      bynder-size: expireAfterAccess=60m
    enabled: true

S3 default configuration

/dam-s3/decorations/addon-commons-cache/config.yaml

cacheConfigurations:
  s3:
    caches:
      s3-objects: maximumSize=500, expireAfterAccess=10m
      s3-buckets: expireAfterAccess=60m
    s3-count: maximumSize=500, expireAfterAccess=10m
    s3-pages: maximumSize=500, expireAfterAccess=10m

Cache configuration properties

Property Description

caches

required

Use CaffeineSpec properties to specify the cache behavior such as the maximum size of the cache and when cache entries expire for each cache.

For example:

<cacheName-1>: maximumSize=500, expireAfterAccess=10m

<cacheName-2>: expireAfterAccess=60m

enabled

required, true by default

Set to false to disable caching.

Bynder loading times

Note that the Bynder API may be very slow in some regions. For example, one image upload may take up to 20 seconds in EU regions. This may cause some actions to fail in the Magnolia UI and the cache to be invalidated.

For more information, see https://github.com/Bynder/bynder-js-sdk/issues/43.

Manually flushing the cache

If you want to flush the cache, you can either set the cache expiry to a very short time (see above) or configure a script in the Groovy app to flush the cache manually. For example:

import info.magnolia.objectfactory.Components

cacheManager = Components.getComponent(info.magnolia.addon.commons.cache.CacheManager)
cacheManager.evictCache("s3-objects")

Configuring pagination to improve response times

Magnolia retrieves pages of objects from Amazon S3 and Bynder and then obtains the assets you require from those pages. The pages are cached.

You can configure the size of a page of objects using the property pageSize.

Bynder has an additional property you can use if you set the page size to more than 1000 objects.

dam-<s3/Bynder>/config.yaml
pageSize: 100
Property Description

pageSize

optional

The maximum number of objects per request retrieved from your external system.

  • S3 default: 1000

  • Bynder default: 100

For S3, the minimum value possible is 100. The maximum is 1000.

bynderMaxNumberPerRequest

optional, default is 1000 (Bynder only)

Maximum number of objects that Bynder returns if you specify a pageSize greater than 1000.

See the Bynder documentation: https://bynder.docs.apiary.io/#reference/oauth-1.0a/retrieve-assets, limit parameter.

Bynder-specific configuration

This section provides instructions specific to configuring the Bynder connector.

(See below for S3.)

Configuring the proxy service to protect private asset URLS

To protect the URLs of private Bynder assets rendered in Magnolia, you must use the proxy service provided.

The proxy service is also required if you want to use Bynder image variations.

The proxy service enforces security by generating a Magnolia URL that hides the private Bynder URL.

If you do not use the proxy service, private Bynder URLS are exposed in published Magnolia pages for example.

When using the proxy, if you want to make a Bynder asset public, for example an image asset used in the Pages app, you must publish the image as well as the page in Magnolia.

You can enable or disable the proxy service using the proxyMode parameter. The functionality is provided via the BynderAssetRenderer.

The proxy service is only available with Magnolia 6.2.6+. If you use an earlier version of Magnolia Core, you must disable the proxy service.
dam-bynder/decorations/dam/dam.config.yaml
providers:
  bynderProvider:
    identifier: bynder
    enabled: false
    class: info.magnolia.external.dam.bynder.BynderAssetProvider
    renderers:
      bynder:
        class: info.magnolia.external.dam.bynder.BynderAssetRenderer
        proxyMode: true
        mediaTypes:
          audio: 'audio/*'
          application: 'application/*'
          video: 'video/*'
          image: 'image/*'
Property Description

proxyMode

optional, default is true

Enables or disables the proxy service.

  • When set to true, the proxy mode is enabled. The Bynder asset generates a Magnolia URL to hide the Bynder URL and the asset is served through the Magnolia instance.

  • When set to false, the proxy mode is disabled. The Bynder asset generates a Bynder URL and the asset is served directly by Bynder. Private Bynder URLs may be exposed.

Using Bynder image derivatives

You must enable proxy mode to use Bynder derivatives in Magnolia.

Bynder provides image derivatives (referred to as variations or renditions in Magnolia) by default, such as:

  • small

  • medium

  • large

  • original

  • thumbnail

These derivatives can be used as the rendition names in the Magnolia DAM templating function (damfn).

For example, to retrieve the small Bynder derivative:

/templates/pages/sample.ftl
<!DOCTYPE html>
<html xml:lang="${cmsfn.language()}" lang="${cmsfn.language()}">
  <head>
    [@cms.page /]
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <title>${content.windowTitle!content.title!}</title>
  </head>
  <body class="sample-dam ${cmsfn.language()}">

    <div class="container">
      <h3>Bynder ID: ${content.assetId}</h3>

      <h4>small</h4>
      <img src="${damfn.getAssetLink(content.assetId, "small")}"/>

    </div>
  </body>
</html>

Only image type Bynder asset derivatives can be used in Magnolia.

To see which derivatives are available for a given Bynder image, look in the Size variations field in the asset details view:

Bynder derivatives information

Magnolia falls back to the original derivative if the requested derivative is unavailable.

Retrieving Bynder derivatives differs from using typical Magnolia image variations:

  • You do not have to define image variations in a theme when using Bynder derivatives.

  • In your template script, instead of the Magnolia asset itemKey or renditionName, you use the Bynder derivative name (small, medium, large, original and so on).

For more information about Bynder derivatives, we recommend the following pages in the Bynder documentation:

AWS S3-specific configuration

This section provides instructions specific to configuring the S3 connector.

Configuring the maximum search time for S3

You can configure the maximum time Magnolia Periscope spends searching for S3 assets to be displayed in the Magnolia interface when a user searches for an asset in the Find Bar.

dam-s3/config.yaml
maxSearchTimeInMilis: 10000
Property Description

maxSearchTimeInMilis

optional

The maximum time Magnolia Periscope spends searching for S3 assets. By default, the maximum search time is 10 seconds.

For example: maxSearchTimeInMilis: 10000

Configuring presigned URLs to enable previewing for S3 assets

When you upload assets via Magnolia to S3, they are set to private by default. If you want to keep the preview functionality available, you must use presigned urls.

dam-s3/config.yaml
presignedSignatureDurationInMinutes: 60
Property Description

presignedSignatureDurationInMinutes

optional

Determines how long a presigned URL is valid.

We recommend that the duration is at least equal to the expiration of the caches set for s3-objects`, s3-pages and s3-count. Otherwise, the presigned URL may be not valid while it is still cached.

For example: presignedSignatureDurationInMinutes: 60

Configuring the AWS regions list

When creating an S3 bucket in the Assets app, only AWS-enabled regions are displayed to users.

AWS S3 bucket names are globally unique, meaning no two buckets can have the same name. If you try to create a bucket and receive an error stating the bucket already exists, it means that the bucket name is already in use, either by you or someone else.

If your users want to use a disabled region, you must enable it first in AWS as described in Managing AWS Regions in the AWS documentation.

Then you must remove the newly enabled region from the list of disabled regions in the Magnolia configuration so that it appears in the chooser.

/dam-s3/config.yaml
disabledRegions:
  - "ap-east-1"
  - "af-south-1"
  - "eu-south-1"
  - "me-south-1"
  - "cn-north-1"
  - "cn-northwest-1"
  - "us-gov-east-1"
  - "us-gov-west-1"
  - "us-iso-east-1"
  - "us-isob-east-1"

Usage

The magnolia-external-dam submodule provides a sample Text and external image page component so that editors and marketers can select and use external assets in the Pages app.

See DAM integration framework for an example.

If an asset is uploaded into the DAM app that has been compromised, for example, an image file containing malicious code, an error message is displayed.

Release history

External DAM module changelog

Magnolia External DAM module 2.0.0 is in beta with Magnolia 6.3.

We are updating docs based on development and feedback. Consider the beta docs currently in a state of progress and not final.

The changelog gives you an overview of the changes made to the External DAM module at each release.

This changelog covers the 2.0.x branch of the External DAM module. For the 1.1.x changelog, see here.

2.0.0-beta2

Not yet publicly released

Ticket Type Description

EXTDAM-453

S3: Bring back the bucket icon.

EXTDAM-363

Build on JDK 11.

EXTDAM-270

S3: Remove bucket name from app folder links and item description.

EXTDAM-238

S3: getItem returns size 0 for assets.

EXTDAM-237

S3:Option to store width and height in the asset metadata.

EXTDAM-218

S3: Add filtering on the Name column.

DAM integration framework compatibility

Module version Magnolia CMS version

1.1.0+

6.2.13+

1.0.3+

6.2.4

1.0.2

6.2.4

1.0.1

6.2.1

1.0

6.2

A note about connector compatibilty

The third-party versions listed below are the versions Magnolia has developed and tested against. If you don’t see a particular version listed, it simply means that we do not routinely test on it. If you are using another version of the third-party tool and run into compatibility issues, please contact our Support.
Module version Bynder SDK

1.0.4

Tested against 2.2.1

1.0

Tested against 2.2.0
Module version AWS Java SDK

1.0.1

2.13.25

1.0

Tested against 2.9.7
Feedback

DX Core

×

Location

This widget lets you know where you are on the docs site.

You are currently perusing through the DX Core docs.

Main doc sections

DX Core Headless PaaS Legacy Cloud Incubator modules
6.3 beta
X

Magnolia 6.3 beta

Magnolia 6.3 is in beta. We are updating docs based on development and feedback. Consider the 6.3 docs currently in a state of progress and not final.

We are working on some 6.3-beta known issues during this phase.