add X-Robots-Tag noindex, nofollow server-snippet to all ingresses

[tobybellwood]

This PR adds a default annotation to all ingresses to prevent robots from crawling development sites.

    {{- if eq .Values.environmentType "development"}}
    nginx.ingress.kubernetes.io/server-snippet: |-
       add_header X-Robots-Tag "noindex, nofollow";
    {{- end }}

A couple of important notes:

• This only applies to development sites
• This effectively duplicates the ingress annotation already provided in the nginx images - which may eventually be deprecated. (AFAIK, duplicate X-Robots-Tags are allowed, as long as they don't contradict)
• This extends the no-robots directive to all ingresses (not just nginx-backed ones)
• Documentation has been added to warn of setting additional server-snippet annotations in .lagoon.yml, as they will take precedence over the ingress-defined ones, which may inadvertently remove the protection.

We should probably flesh out the annotations tests a bit more to cover this (and other eventualities)

Closes #2865

[twardnw]

This really also needs to address autogenerated routes on production environments too, a customer on Lagoon highlighted to me that content on autogenerated routes was being crawled/indexed because of the issue with the current lua script in the nginx images

[tobybellwood]

ah - dang, that's an easy fix then!

I'll leave the development wrapper around the custom-ingress, and remove it for all the templated ones.