Magnolia JCR failure with false status
When Magnolia fails to connect to the database, an exception is thrown much like the following:
info.magnolia.repository.RepositoryNotInitializedException: org.apache.jackrabbit.core.data.DataStoreException: Can not init data store, driver=org.postgresql.Driver url=null user=null schemaObjectPrefix=ds_ tableSQL=datastore createTableSQL=CREATE TABLE ds_datastore(ID VARCHAR(255) PRIMARY KEY, LENGTH BIGINT, LAST_MODIFIED BIGINT, DATA BYTEA)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (The connection attempt failed.)
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
Caused by: java.net.SocketTimeoutException: connect timed out
When Tomcat shuts down Magnolia, you see:
SEVERE org.apache.catalina.core.StandardContext startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
Requests sent to Tomcat receive a
404 since Magnolia is no longer running.
Requests sent to the Tomcat Magnolia REST service may work if they do not use JCR.
The REST status endpoint,
If the REST status endpoint is used as the health path (e.g.,
The REST status endpoint is the default value in the Helm chart.
The primary issue is that the pod may appear to be running normally despite Magnolia shutting down due to the exception and hence, no errors are reported in the container logs.
We are working on fixing this as it is considered a bug. As of 2024-02-24, it has not been fixed, so you should consider the workaround here.
Delete the affected Magnolia instance. You can do this via Rancher or using
StatefulSetrecreates the Magnolia pod.
Check the Magnolia logs to ensure the problem is not reoccurring.
You can do this in Magnolia using the Log Tools app.
If the Magnolia instance is serving life traffic, the
404 response may be cached.
If you’re using Fastly, flush the CDN cache.
If you’re using a different CDN, you’ll need to flush the cache as per your CDN instructions.