AWS S3 extension configuration
This page explains how to configure the connection between the External DAM module and your AWS S3 external asset management solution.
Essentially, you:
-
Provide connection credentials for S3.
-
Enable the subapp to display in the Magnolia Assets app.
-
Configure the cache if necessary.
You can create or edit the configuration in the JCR or the File System (YAML) under magnolia-external-dam-s3/external-dams/<definition-name>
.
S3 connection configuration
To use AWS S3 assets in Magnolia, you must have a working AWS account.
You need AWS S3 credentials to connect your S3 buckets and display their content in Magnolia. Credentials consist of two parts:
-
AWS access key ID
-
AWS secret access key
You can store your AWS credentials using:
-
The Magnolia Passwords app
-
The AWS default credential provider chain
AWS Foundation password manager
The magnolia-aws-foundation-password-manager
dependency is necessary if you want to use the Password Manager module to store your AWS credentials.
To do this:
-
Go to your webapp.
-
Add the following to your webapp’s
pom.xml
file as a dependency:<dependency> <groupId>info.magnolia.aws.foundation</groupId> <artifactId>magnolia-aws-foundation-password-manager</artifactId> <version>${awsFoundationVersion}</version> (1) </dependency>
1 Where this is the latest compatible version needed for your webapp. As of 2024-07-01, the latest available version is 1.0.4
.
Using the Magnolia Passwords app
Add the credentials to your Magnolia instance in the Password manager app.
For example:
aws-credentials |
|
---|---|
|
|
|
Using the AWS default credential provider chain
-
Add the following configuration properties to your
WEB-INF/config/default/magnolia.properties
file:magnolia.aws.validateCredentials=false magnolia.aws.useCredentials=false
-
Set your AWS credentials by following the instructions in the AWS documentation: https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html
The magnolia-aws-foundation module handles all Amazon connections from Magnolia. This module is installed automatically by Maven when you install the External DAM module.
|
Configuring an AWS S3 bucket whitelist
By default, all the buckets in the S3 account you connect to are displayed. If you don’t want to display all your buckets, you can configure a whitelist in the Resource files app using regex.
For example:
whitelistedBuckets:
- "^a.*"
- a-particular-bucket
Adding a filter capability to S3 buckets
If you want to restrict access to a single bucket, either because it’s used in different use cases or for some confidentiality-related reasons, you can add a bucket prefix. Using a bucket prefix grants access only to that specific bucket without revealing its actual path which might contain unwanted access information. This way the user sees the bucket and folders as the only folders in the Magnolia instance.
To use the filter, add the following bucket prefix to your config.yaml
file:
whitelistedBuckets:
- BUCKET_NAME
bucketPrefixes:
BUCKET_NAME: “FOLDER_NAME/SUBFOLDER_NAME”
Enabling your subapp
By default, the AWS S3 subapp is disabled.
To enable it, decorate the Magnolia Assets app under <dam-connector>/decorations/dam
and set the enabled
property to true
.
providers:
s3Provider:
identifier: s3
enabled: true
class: info.magnolia.external.dam.s3.datasource.S3AssetProvider
Configuring a single asset provider for link fields
You can configure a link field to limit your users to selecting assets from only one asset provider.
This is possible from External DAM 1.0.7 (DAM module 3.0.8). Previously, you had to use the aggregated damLinkField and users had to choose from all the enabled asset providers.
|
This example shows a dialog definition with two link fields, each limited to different asset providers:
form:
properties:
s3:
$type: damLinkField
chooserId: dam-s3:chooser
datasource:
class: info.magnolia.dam.app.data.AssetDatasourceDefinition
name: s3
jcr:
$type: linkField
converterClass: info.magnolia.ui.editor.converter.JcrAssetConverter
datasource:
$type: jcrDatasource
workspace: website
Configuring the cache
This integration framework uses Caffeine, a high-performance Java cache library, to manage caching for external assets via a Magnolia helper module called magnolia-addon-commons-cache
.
For S3 caching, you can specify the behavior of the caches. Essentially, you can configure all the parameters available in CaffeineSpec.
For example:
-
Set a maximum cache size.
-
Define how often the cache expires.
Additionally, you can:
-
Enable or disable the cache.
You configure caching through decoration under /<dam-connector>/decorations/addon-commons-cache/config.yaml
.
cacheConfigurations:
s3:
caches:
s3-objects: maximumSize=500, expireAfterAccess=10m
s3-buckets: expireAfterAccess=60m
s3-count: maximumSize=500, expireAfterAccess=10m
s3-pages: maximumSize=500, expireAfterAccess=10m
Cache configuration properties
Property | Description |
---|---|
|
required Use CaffeineSpec properties to specify the cache behavior such as the maximum size of the cache and when cache entries expire for each cache.
For example:
|
|
required, Set to |
Manually flushing the cache
If you want to flush the cache, you can either set the cache expiry to a very short time (see above) or configure a script in the Groovy app to flush the cache manually. For example:
import info.magnolia.objectfactory.Components
cacheManager = Components.getComponent(info.magnolia.addon.commons.cache.CacheManager)
cacheManager.evictCache("s3-objects")
Configuring pagination to improve response times
Magnolia retrieves pages of objects from Amazon S3 and then obtains the assets you require from those pages. The pages are cached.
You can configure the size of a page of objects using the property pageSize
.
pageSize: 100
Property | Description |
---|---|
|
optional For S3, the minimum number of objects per request retrieved from S3 possible is 100. The maximum is 1000. |
Configuring the maximum search time for S3
You can configure the maximum time Magnolia Periscope spends searching for S3 assets to be displayed in the Magnolia interface when a user searches for an asset in the Find Bar.
maxSearchTimeInMilis: 10000
Property | Description |
---|---|
|
optional The maximum time Magnolia Periscope spends searching for S3 assets. By default, the maximum search time is 10 seconds. For example: maxSearchTimeInMilis: 10000 |
Configuring presigned URLs to enable previewing for S3 assets
When you upload assets via Magnolia to S3, they’re set to private by default. If you want to keep the preview functionality available, you must use presigned URLs.
presignedSignatureDurationInMinutes: 60
Property | Description |
---|---|
|
optional Determines how long a presigned URL is valid. We recommend that the duration is at least equal to the expiration of the caches set for s3-objects`, For example: |
Configuring the AWS regions list
When creating an S3 bucket in the Assets app, only AWS-enabled regions are displayed to users.
AWS S3 bucket names are globally unique, meaning no two buckets can have the same name. If you try to create a bucket and receive an error stating the bucket already exists, it means that the bucket name is already in use, either by you or someone else. |
If your users want to use a disabled region, you must enable it first in AWS as described in Managing AWS Regions in the AWS documentation.
Then you must remove the newly enabled region from the list of disabled regions in the Magnolia configuration so that it appears in the chooser.
disabledRegions:
- "ap-east-1"
- "af-south-1"
- "eu-south-1"
- "me-south-1"
- "cn-north-1"
- "cn-northwest-1"
- "us-gov-east-1"
- "us-gov-west-1"
- "us-iso-east-1"
- "us-isob-east-1"