Secrets breaking auth, Status code 307

[onttm]

UPDATE: My problems revolve around secrets. When I place all of my credentials in .env everything works properly. But if anyone can help me implement secrets properly I'd really appreciate it.

I'm not getting dynamically generated uri's. I am getting the following error.
msg="Remote error http://oauth:4181. StatusCode: 307" middlewareType=ForwardedAuthType middlewareName=middlewares-oauth@file

Anyone know what I'm doing wrong?

Here is my compose.

version: "3.7"

########################### NETWORKS
networks:
  t2_proxy:
    external:
      name: t2_proxy
  default:
    driver: bridge
  socket_proxy:
    external:
      name: socket_proxy

########################### SECRETS
secrets:
  cloudflare_email:
    file: $SECRETSDIR/cloudflare_email
  cloudflare_api_key:
    file: $SECRETSDIR/cloudflare_api_key
  cloudflare_api_token:
    file: $SECRETSDIR/cloudflare_api_token
  authelia_jwt_secret:
    file: $SECRETSDIR/authelia_jwt_secret
  authelia_session_secret:
    file: $SECRETSDIR/authelia_session_secret
  authelia_storage_mysql_password:
    file: $SECRETSDIR/authelia_storage_mysql_password
  authelia_notifier_smtp_password:
    file: $SECRETSDIR/authelia_notifier_smtp_password
  authelia_duo_api_secret_key:
    file: $SECRETSDIR/authelia_duo_api_secret_key
  oauth_secret:
    file: $SECRETSDIR/oauth_secret
  google_client_secret:
    file: $SECRETSDIR/google_client_secret
  google_client_id:
    file: $SECRETSDIR/google_client_id
  my_email:
    file: $SECRETSDIR/my_email
  mysql_root_password:
    file: $SECRETSDIR/mysql_root_password
  plex_claim:
    file: $SECRETSDIR/plex_claim
  traefik-forward-auth:
    file: $SECRETSDIR/traefik-forward-auth
########################### SERVICES
services:
  ############################# FRONTENDS

  # Traefik 2 - Reverse Proxy
  traefik:
    container_name: traefik
    image: traefik:2.4.0 
    restart: unless-stopped
    environment:
      - CF_API_EMAIL_FILE=/run/secrets/cloudflare_email
      - CF_API_KEY_FILE=/run/secrets/cloudflare_api_key
    secrets:
      - cloudflare_email
      - cloudflare_api_key
    command: # CLI arguments
      - --global.checkNewVersion=true
      - --global.sendAnonymousUsage=true
      - --entryPoints.http.address=:80
      - --entryPoints.https.address=:443
      # Allow these IPs to set the X-Forwarded-* headers - Cloudflare IPs: https://www.cloudflare.com/ips/
      - --entrypoints.https.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22
      - --entryPoints.traefik.address=:8080
      # - --entryPoints.ping.address=:8081
      - --api=true
      # - --api.insecure=true
      - --api.dashboard=true
      #- --ping=true
      - --pilot.token=xxxx 
      #- --serversTransport.insecureSkipVerify=true
      - --log=true
      - --log.level=DEBUG # (Default: error) DEBUG, INFO, WARN, ERROR, FATAL, PANIC
      - --accessLog=true
      - --accessLog.filePath=/traefik.log
      - --accessLog.bufferingSize=100 # Configuring a buffer of 100 lines
      - --accessLog.filters.statusCodes=400-499
      - --providers.docker=true
      # - --providers.docker.endpoint=unix:///var/run/docker.sock # Use Docker Socket Proxy instead for improved security
      - --providers.docker.endpoint=tcp://socket-proxy:2375
      # Automatically set Host rule for services
      - --providers.docker.defaultrule=Host(`{{ index .Labels "com.docker.compose.service" }}.$DOMAINNAME`)
      - --providers.docker.exposedByDefault=false
      #- --entrypoints.https.http.middlewares=chain-basic-auth@file #Basic authentication
      - --entrypoints.https.http.middlewares=chain-oauth@file #Google OAuth 2.0authentication
      - --entrypoints.https.http.tls.options=tls-opts@file
      # Add dns-cloudflare as default certresolver for all services. Also enables TLS and no need to specify on individual services
      - --entrypoints.https.http.tls.certresolver=dns-cloudflare
      - --entrypoints.https.http.tls.domains[0].main=$DOMAINNAME
      - --entrypoints.https.http.tls.domains[0].sans=*.$DOMAINNAME
      # - --entrypoints.https.http.tls.domains[1].main=$DOMAIN # Pulls main cert for second domain
      # - --entrypoints.https.http.tls.domains[1].sans=*.$DOMAIN # Pulls wildcard cert for second domain
      - --providers.docker.network=t2_proxy
      - --providers.docker.swarmMode=false
      - --providers.file.directory=/rules # Load dynamic configuration from one or more .toml or .yml files in a directory
      # - --providers.file.filename=/path/to/file # Load dynamic configuration from a file
      - --providers.file.watch=true # Only works on top level files in the rules folder
      #- --certificatesResolvers.dns-cloudflare.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory # LetsEncrypt Staging Server - uncomment when testing
      - --certificatesResolvers.dns-cloudflare.acme.email=$CLOUDFLARE_EMAIL #CF_API_EMAIL
      - --certificatesResolvers.dns-cloudflare.acme.storage=/acme.json
      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.provider=cloudflare
      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53
      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.delayBeforeCheck=30 # To delay DNS check and reduce LE hitrate
    networks:
      t2_proxy:
        ipv4_address: 192.168.90.254 # You can specify a static IP
      # networks:
      #   - t2_proxy
      socket_proxy:
    #depends_on:
    #  - socket-proxy
    security_opt:
      - no-new-privileges:true
    #healthcheck:
    #  test: ["CMD", "traefik", "healthcheck", "--ping"]
    #  interval: 5s
    #  retries: 3
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host

    volumes:
      - $DOCKERDIR/traefik2/rules:/rules # file provider directory
      - /var/run/docker.sock:/var/run/docker.sock:ro # Use Docker Socket Proxy instead for improved security
      - $DOCKERDIR/traefik2/acme/acme.json:/acme.json # cert location - you must touch this file and change permissions to 600
      - $DOCKERDIR/traefik2/traefik.log:/traefik.log # for fail2ban - make sure to touch file before starting container
      - $DOCKERDIR/shared:/shared

    labels:
      #- "autoheal=true"
      - "traefik.enable=true"
      # HTTP-to-HTTPS Redirect
      - "traefik.http.routers.http-catchall.entrypoints=http"
      - "traefik.http.routers.http-catchall.rule=HostRegexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      # HTTP Routers
      - "traefik.http.routers.traefik-rtr.entrypoints=https"
      - "traefik.http.routers.traefik-rtr.rule=Host(`traefik.$DOMAINNAME`)"
      ## Services - API
      - "traefik.http.routers.traefik-rtr.service=api@internal"
      ## Healthcheck/ping
      #- "traefik.http.routers.ping.rule=Host(`traefik.$DOMAINNAME`) && Path(`/ping`)"
      #- "traefik.http.routers.ping.tls=true"
      #- "traefik.http.routers.ping.service=ping@internal"
      ## Middlewares
      - "traefik.http.routers.traefik-rtr.middlewares=chain-oauth@file"

  # Docker Socket Proxy - Security Enchanced Proxy for Docker Socket
  socket-proxy:
    container_name: socket-proxy
    image: fluencelabs/docker-socket-proxy
    restart: always
    networks:
      # t2_proxy:
      socket_proxy:
        ipv4_address: 192.168.91.254 # You can specify a static IP
    privileged: true
    ports:
      - "127.0.0.1:2375:2375" # Port 2375 should only ever get exposed to the internal network.
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    environment:
      - LOG_LEVEL=info # debug,info,notice,warning,err,crit,alert,emerg
      ## Variables match the URL prefix (i.e. AUTH blocks access to /auth/* parts of the API, etc.).
      # 0 to revoke access.
      # 1 to grant access.
      ## Granted by Default
      - EVENTS=1
      - PING=1
      - VERSION=1
      ## Revoked by Default
      # Security critical
      - AUTH=0
      - SECRETS=0
      - POST=1 # Watchtower
      - DELETE=1 # Watchtower
        # GET Optons
      - BUILD=0
      - COMMIT=0
      - CONFIGS=0
      - CONTAINERS=1 # Traefik, portainer, etc.
      - DISTRIBUTION=0
      - EXEC=0
      - IMAGES=1 # Portainer, Watchtower
      - INFO=1 # Portainer
      - NETWORKS=1 # Portainer, Watchtower
      - NODES=0
      - PLUGINS=0
      - SERVICES=1 # Portainer
      - SESSION=0
      - SWARM=0
      - SYSTEM=0
      - TASKS=1 # Portaienr
      - VOLUMES=1 # Portainer
      # POST Options
      - CONTAINERS_CREATE=1 # WatchTower
      - CONTAINERS_START=1 # WatchTower
      - CONTAINERS_UPDATE=1 # WatchTower
      # DELETE Options
      - CONTAINERS_DELETE=1 # WatchTower
      - IMAGES_DELETE=1 # WatchTower
      
  # Google OAuth - Single Sign On using OAuth 2.0
  # https://hub.docker.com/r/thomseddon/traefik-forward-auth
  # https://www.smarthomebeginner.com/google-oauth-with-traefik-docker/
  oauth:
    container_name: oauth
    image: thomseddon/traefik-forward-auth:latest
    # image: thomseddon/traefik-forward-auth:2.1-arm # Use this image with Raspberry Pi
    restart: unless-stopped
    networks:
      - t2_proxy
    security_opt:
      - no-new-privileges:true
    # Allow apps to bypass OAuth. Radarr example below will bypass OAuth if API key is present in the request (eg. from NZB360 mobile app).
    # While this is one way, the recommended way is to bypass authentication using Traefik labels shown in some of the apps later.
    # command: --rule.radarr.action=allow --rule.radarr.rule="Headers(`X-Api-Key`, `$RADARR_API_KEY`)"
    # command: --rule.sabnzbd.action=allow --rule.sabnzbd.rule="HeadersRegexp(`X-Forwarded-Uri`, `$SABNZBD_API_KEY`)"
    secrets: #secrets workaround https://github.com/htpcBeginner/docker-traefik/issues/91#issuecomment-715479143
      - source: traefik-forward-auth
        target: /config
    environment:
      - CONFIG=/config  #contains providers.google.client-id, providers.google.client-secret, secret and whitelist
      - log-level=warn
      - log-format=text
      - auth-host=oauth.$DOMAINNAME:8085
      - url-path=/_oauth
      - default-action=auth
      - default-provider=google
      - lifetime=86400
        
    labels:
      - "traefik.enable=true"
      ## HTTP Routers
      - "traefik.http.routers.oauth-rtr.tls=true"
      - "traefik.http.routers.oauth-rtr.entrypoints=https"
      - "traefik.http.routers.oauth-rtr.rule=Host(`oauth.$DOMAINNAME`)"
      ## Middlewares
     # - "traefik.http.middlewares.traefik-forward-auth.forwardauth.address=http://traefik-forward-auth:4181" #this is a  line from the example but I am still trying to figure out how to tailor it to the turorial from htpcbeginner
      - "traefik.http.routers.oauth-rtr.middlewares=chain-oauth@file"
      ## HTTP Services
      - "traefik.http.routers.oauth-rtr.service=oauth-svc"
      - "traefik.http.services.oauth-svc.loadbalancer.server.port=4181"

# Portainer - WebUI for Containers
  portainer:
    container_name: portainer
    image: portainer/portainer-ce:latest
    restart: unless-stopped
    # command: -H unix:///var/run/docker.sock # # Use Docker Socket Proxy instead for improved security
    command: -H tcp://socket-proxy:2375
    networks:
      - socket_proxy
    ports:
      - "$PORTAINER_PORT:9000"
    security_opt:
      - no-new-privileges:true
    volumes:
      # - /var/run/docker.sock:/var/run/docker.sock:ro # # Use Docker Socket Proxy instead for improved security
      - $DOCKERDIR/portainer/data:/data # Change to local directory if you want to save/transfer config locally
    environment:
      - TZ=$TZ
    labels:
      - "traefik.enable=true"
      ## HTTP Routers
      - "traefik.http.routers.portainer-rtr.entrypoints=https"
      - "traefik.http.routers.portainer-rtr.rule=Host(`portainer.$DOMAINNAME`)"
      - "traefik.http.routers.portainer-rtr.tls=true"
      ## Middlewares
#      - "traefik.http.routers.portainer-rtr.middlewares=chain-no-auth@file" # No Authentication
#      - "traefik.http.routers.portainer-rtr.middlewares=chain-basic-auth@file" # Basic Authentication
      - "traefik.http.routers.portainer-rtr.middlewares=chain-oauth@file" # Google OAuth 2.0
      ## HTTP Services
      - "traefik.http.routers.portainer-rtr.service=portainer-svc"
      - "traefik.http.services.portainer-svc.loadbalancer.server.port=9000"      
      
  # Cloudflare-Companion - Automatic CNAME DNS Creation
  cf-companion:
    container_name: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    security_opt:
      - no-new-privileges:true
    restart: unless-stopped
    networks:
      - socket_proxy
    #depends_on:
    #  - socket-proxy
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      #- CF_EMAIL=$CLOUDFLARE_EMAIL
      - CF_TOKEN_FILE=/run/secrets/cloudflare_api_token
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=xxxx # Copy from Cloudflare Overview page
      - DOMAIN1_PROXIED=TRUE
      - DOCKER_HOST=tcp://socket-proxy:2375
    secrets: # not working
      - cloudflare_api_token
#    labels:
      # Add hosts specified in rules here to force cf-companion to create the CNAMEs
      # Since cf-companion creates CNAMEs based on host rules, this a workaround for non-docker/external apps
#      - "traefik.http.routers.cf-companion-rtr.rule=Host(`portainer.$DOMAINNAME`)# || `synglances.$DOMAINNAME`) || `pihole.$DOMAINNAME`) || Host(`shell.$DOMAINNAME`) || Host(`syno.$DOMAINNAME`) || Host(`webmin.$DOMAINNAME`) || Host(`synplex.$DOMAINNAME`) || Host(`pwt.$DOMAINNAME`)"

Rules from middlewares-chain.toml

  [http.middlewares.chain-oauth]
    [http.middlewares.chain-oauth.chain]
      middlewares = [ "middlewares-rate-limit", "middlewares-secure-headers", "middlewares-oauth"]

and from middlewares.yml

    middlewares-oauth:
      forwardAuth:
        address: "http://oauth:4181" # Make sure you have the OAuth service in docker-compose.yml
        trustForwardHeader: true
        authResponseHeaders:
          - "X-Forwarded-User"

[thomseddon]

Hi - glad that you've narrowed it down to the secrets config, could you post an example of your config for the traefik-forward-auth container that works, and also how you want it to be (i.e. using a secrets file) but it isn't working? I should be able to work it out from that

[Kitem19]

Hi,

I think I'm having the same issue.
Can you please advise on how did you fix it?

Thanks

[jaytrickett1]

I'm having the same issue and clearly followed the same template as the OP. Any ideas out there?

[vojtechvelkjop]

Same issue as well :(