Secure ingresses
To secure unwanted access (for example, on /magnolia/.admincentral
or similar) and to avoid crawlers accessing unpublished content, it’s usually a good idea to work with IP Whitelisting on Ingress-Nginx and Robots-Tags (such as X-Robots-Tag:noindex, nofollow
).
Instructions
-
First ensure you are connected to your DX Cloud Kubernetes Cluster using the correct
kubeconfig
. -
Create an
HTPASSWD
file.# Create HTPASSWD File $ htpasswd -c auth myadmin New password: Re-type new password: Adding password for user myadmin
-
Convert to a Kubernetes secret.
$ kubectl create -n dev secret generic basic-auth --from-file=auth secret/basic-auth created
-
Annotate on the ingress and add whitelisting.
$ kubectl edit -n prod ingress myingress
-
Create the annotation template to update the ingress.
$ k annotate -n dev ingress --overwrite=true dev-magnolia-helm \ nginx.ingress.kubernetes.io/auth-realm="PLEASE LOGIN:" \ nginx.ingress.kubernetes.io/auth-secret=dev/basic-auth \ nginx.ingress.kubernetes.io/auth-type=basic \ nginx.ingress.kubernetes.io/configuration-snippet='more_set_headers "X-Robots-Tag: noindex, nofollow";' \ (1) nginx.ingress.kubernetes.io/whitelist-source-range="37.120.189.19/32,2a03:4000:6:b665::46/128,23.235.32.0/20,43.249.72.0/22,103.244.50.0/24,103.245.222.0/23,103.245.224.0/24,104.156.80.0/20,140.248.64.0/18,140.248.128.0/17,146.75.0.0/17,151.101.0.0/16,157.52.64.0/18,167.82.0.0/17,167.82.128.0/20,167.82.160.0/20,167.82.224.0/20,172.111.64.0/18,185.31.16.0/22,199.27.72.0/21,199.232.0.0/16" \ (2)
1 Set the robot tags for crawler information. Generally, you shouldn’t use configuration-snippet(s). Use the configuration-snippet exactly as instructed for this example. Otherwise, you may cause issues with your PaaS deployment. 2 Set the IP whitelisting for IPv4/v6 IPs and Subnets. To test before deployment, you can leave this out during a dry run ( Subnets—dry-run=client
).The following IPs must be whitelisted for Fastly to work properly. The example above includes these IPs, so feel free to copy those.
23.235.32.0/20,43.249.72.0/22,103.244.50.0/24,103.245.222.0/23,103.245.224.0/24,104.156.80.0/20,140.248.64.0/18,140.248.128.0/17,146.75.0.0/17,151.101.0.0/16,157.52.64.0/18,167.82.0.0/17,167.82.128.0/20,167.82.160.0/20,167.82.224.0/20,172.111.64.0/18,185.31.16.0/22,199.27.72.0/21,199.232.0.0/16
-
Run a remote
--dry-run=client
if fine and apply it.$ k get ingress -n dev dev-magnolia-helm -o yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: cert-manager.io/cluster-issuer: letsencrypt-prod field.cattle.io/publicEndpoints: '[{"addresses":["20.71.0.152"],"port":443,"protocol":"HTTPS","serviceName":"dev:dev-magnolia-helm-public-svc","ingressName":"dev:dev-magnolia-helm","hostname":"dev.azurepaas.magnolia-platform.com","path":"/","allNodes":false},{"addresses":["20.71.0.152"],"port":443,"protocol":"HTTPS","serviceName":"dev:dev-magnolia-helm-author-svc","ingressName":"dev:dev-magnolia-helm","hostname":"dev.azurepaas.magnolia-platform.com","path":"/author","allNodes":false}]' kubernetes.io/ingress.class: nginx meta.helm.sh/release-name: dev meta.helm.sh/release-namespace: dev nginx.ingress.kubernetes.io/auth-realm: 'PLEASE LOGIN:' nginx.ingress.kubernetes.io/auth-secret: basic-auth nginx.ingress.kubernetes.io/auth-type: basic nginx.ingress.kubernetes.io/configuration-snippet: 'more_set_headers "X-Robots-Tag: noindex, nofollow";' (1) nginx.ingress.kubernetes.io/proxy-body-size: 512m nginx.ingress.kubernetes.io/whitelist-source-range="37.120.189.19/32,2a03:4000:6:b665::46/128,23.235.32.0/20,43.249.72.0/22,103.244.50.0/24,103.245.222.0/23,103.245.224.0/24,104.156.80.0/20,140.248.64.0/18,140.248.128.0/17,146.75.0.0/17,151.101.0.0/16,157.52.64.0/18,167.82.0.0/17,167.82.128.0/20,167.82.160.0/20,167.82.224.0/20,172.111.64.0/18,185.31.16.0/22,199.27.72.0/21,199.232.0.0/16" ...
1 Generally, you shouldn’t use configuration-snippet(s). Use the configuration-snippet exactly as instructed for this example. Otherwise, you may cause issues with your PaaS deployment.
Test Basic Auth and Whitelisting
-
Run the following command:
$ curl ifconfig.me 87.123.205.103% $ curl https://dev.azurepaas.magnolia-platform.com <html> <head><title>403 Forbidden</title></head> <body> <center><h1>403 Forbidden</h1></center> <hr><center>nginx</center> </body> </html>
Require authentication from permitted IP range
-
Run the following command:
$ curl ifconfig.me 37.120.189.19% $ curl https://dev.azurepaas.magnolia-platform.com <html> <head><title>401 Authorization Required</title></head> <body> <center><h1>401 Authorization Required</h1></center> <hr><center>nginx</center> </body> </html>
-
Authenticate via Basic Auth using your stored credentials:
$ curl https://dev.azurepaas.magnolia-platform.com -u myadmin:myadmin
Verbose output
$ curl -v https://dev.azurepaas.magnolia-platform.com -H 'Host: dev.azurepaas.magnolia-platform.com' -u myadmin:myadmin * Trying 20.71.0.152:443... * Connected to dev.azurepaas.magnolia-platform.com (20.71.0.152) port 443 (#0) * ALPN: offers h2 * ALPN: offers http/1.1 * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (OUT), TLS handshake, Client hello (1): * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384 * ALPN: server accepted h2 * Server certificate: * subject: CN=dev.azurepaas.magnolia-platform.com * start date: Oct 26 12:57:00 2022 GMT * expire date: Jan 24 12:56:59 2023 GMT * subjectAltName: host "dev.azurepaas.magnolia-platform.com" matched cert's "dev.azurepaas.magnolia-platform.com" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Server auth using Basic with user 'myadmin' * h2h3 [:method: GET] * h2h3 [:path: /] * h2h3 [:scheme: https] * h2h3 [:authority: dev.azurepaas.magnolia-platform.com] * h2h3 [authorization: Basic bXlhZG1pbjpteWFkbWlu] * h2h3 [user-agent: curl/7.84.0] * h2h3 [accept: */*] * Using Stream ID: 1 (easy handle 0x156811400) > GET / HTTP/2 > Host: dev.azurepaas.magnolia-platform.com > authorization: Basic bXlhZG1pbjpteWFkbWlu # Basic Auth active! > user-agent: curl/7.84.0 > accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS == 128)! < HTTP/2 302 < date: Thu, 01 Dec 2022 08:47:20 GMT < content-length: 0 < set-cookie: JSESSIONID=E70616BACA0156F20E6A54A445181E59; Path=/; Secure; HttpOnly < set-cookie: csrf=55JCyz9SNh5mdE7Ke7k1cgkBX9e6huRpCUyj47GPjzM < x-magnolia-registration: Registered < location: /travel.html;jsessionid=E70616BACA0156F20E6A54A445181E59 < strict-transport-security: max-age=15724800; includeSubDomains < x-robots-tag: noindex, nofollow (1) < * Connection #0 to host dev.azurepaas.magnolia-platform.com left intact
1 x-robots-tags are set and delivered. Verify the Output for
x-robots-tag: noindex, nofollow
.