Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workload Identity: Support Envoy SDS API #43958

Merged
merged 20 commits into from
Aug 14, 2024

Conversation

strideynet
Copy link
Contributor

@strideynet strideynet commented Jul 9, 2024

Closes #38666

Introduces support for the Envoy SDS API directly into the existing spiffe-workload-api service.

I manually tested this with a simple envoy config:

  • HTTP requests forwarded to httpbin running on port 7331.
  • Curl using a spiffe-svid output to obtain credentials
  • Envoy using SDS to receive a certificate to use as a host cert, and using SDS to receive the SPIFFE CA for validation purposes. Envoy enforces that requests must come from a client authenticating with a valid SPIFFE client cert for this trust domain.
  • Envoy forwards the identity of the client to the backend service with the X-Forwarded-Client-Cert header.

There's probably a few rough edges with this that we'll discover as we try this out with tools like Istio - but I figure this PR is large enough as is. The key limitation I've noticed is that the SPIFFE validation configuration we return requires the client to authenticate, we may need to provide an option to return the more generic TLS validation configuration for use-cases which are not mTLS.

Example envoy config, tested using httpbin running on port 7331:

# Config based on the example provided in the SPIRE documentation - this provides confidence that this could effectively be "drop-in" replacement for SPIRE agent here.
node:
  id: "my-envoy-proxy"
  cluster: "my-cluster"
static_resources:
  listeners:
    - name: test_listener
      enable_reuse_port: false
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 9001
      access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: "/Users/noah/code/gravitational/teleport-scratch/envoy/inbound-proxy-listener.log"
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                common_http_protocol_options:
                  idle_timeout: 1s
                forward_client_cert_details: sanitize_set
                set_current_client_cert_details:
                  uri: true
                codec_type: auto
                access_log:
                  - name: envoy.access_loggers.file
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      path: "/Users/noah/code/gravitational/teleport-scratch/envoy/inbound-proxy.log"
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: test_service
                      domains: ["*"]
                      routes:
                        - match:
                            prefix: "/"
                          route:
                            cluster: test_service
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          transport_socket:
            name: envoy.transport_sockets.tls
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
              common_tls_context:
                tls_certificate_sds_secret_configs:
                  - name: "default" # "spiffe://leaf.tele.ottr.sh/server"
                    sds_config:
                      resource_api_version: V3
                      api_config_source:
                        api_type: GRPC
                        transport_api_version: V3
                        grpc_services:
                          envoy_grpc:
                            cluster_name: tbot_agent
                # combined validation context "melds" two validation contexts
                # together. This is handy for extending the validation context
                # from the SDS source.
                combined_validation_context:
                  default_validation_context:
                    match_typed_subject_alt_names: []
                  # obtain the trust bundle from SDS
                  validation_context_sds_secret_config:
                    name: "ROOTCA" # "spiffe://leaf.tele.ottr.sh"
                    sds_config:
                      resource_api_version: V3
                      api_config_source:
                        api_type: GRPC
                        transport_api_version: V3
                        grpc_services:
                          envoy_grpc:
                            cluster_name: tbot_agent
  clusters:
    - name: tbot_agent
      connect_timeout: 0.25s
      http2_protocol_options: {}
      load_assignment:
        cluster_name: tbot_agent
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    pipe:
                      path: /Users/noah/code/gravitational/teleport-scratch/envoy/out/workload.sock
    - name: test_service
      connect_timeout: 1s
      type: strict_dns
      load_assignment:
        cluster_name: local_service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 7331

And tbot config:

version: v2
auth_server: redacted
certificate_ttl: 24h
onboarding:
  join_method: token
  token: redacted
storage:
  type: directory
  path: /Users/noah/code/gravitational/teleport-scratch/envoy/data
# outputs will be filled in during the completion of an access guide.
outputs:
  - type: spiffe-svid
    destination:
      type: directory
      path: /Users/noah/code/gravitational/teleport-scratch/envoy/svid-output
    svid:
      path: /client
services:
  - type: spiffe-workload-api
    listen: unix:///Users/noah/code/gravitational/teleport-scratch/envoy/out/workload.sock
    svids:
      - path: /server
        sans:
          dns:
            - localhost
curl --cacert svid_bundle.pem --cert svid.pem --key svid_key.pem https://localhost:9001/headers
{
  "headers": {
    "Accept": "*/*", 
    "Host": "localhost:9001", 
    "User-Agent": "curl/8.6.0", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000", 
    "X-Forwarded-Client-Cert": "By=spiffe://leaf.tele.ottr.sh/server;Hash=87f9d40d1a30f48cf33ae9203241770d00aa40ec63cbd9084a57d7cc87cbc7fe;URI=spiffe://leaf.tele.ottr.sh/client"
  }
}

changelog: Introduce support for Envoy SDS into the Machine ID spiffe-workload-api service.

@strideynet strideynet marked this pull request as ready for review July 11, 2024 11:53
@strideynet
Copy link
Contributor Author

Need to rebase this !

@strideynet strideynet enabled auto-merge August 14, 2024 10:44
@strideynet strideynet added this pull request to the merge queue Aug 14, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 14, 2024
@strideynet strideynet enabled auto-merge August 14, 2024 11:22
@strideynet strideynet added this pull request to the merge queue Aug 14, 2024
Merged via the queue into master with commit 4eefb33 Aug 14, 2024
38 checks passed
@strideynet strideynet deleted the strideynet/tbot-workload-api-sds branch August 14, 2024 12:02
@strideynet
Copy link
Contributor Author

Forgot to add backport labels :(

strideynet added a commit that referenced this pull request Aug 14, 2024
* Start hacking on Workload Identity SDS API support

* Actually generate from SVIDs/Trustbundle

* Support for envoy "special meaning" resource names

* Add minimum version enforcement

* Simplify handling of clustername

* Start working on StreamSecrets RPC

* Extract response generation helper

* Log request

* Implemnt SDS versioning/nonce checking

* Handle renewal of SDS certs

* Refactor for ease of testing

* Start writing tests for SDS functionality

* Add E2E tests

* Fix test

* go mod tidy

* Appease linter

* Update go.mod

* Fix test with characters that windows dislikes
strideynet added a commit that referenced this pull request Aug 14, 2024
* Start hacking on Workload Identity SDS API support

* Actually generate from SVIDs/Trustbundle

* Support for envoy "special meaning" resource names

* Add minimum version enforcement

* Simplify handling of clustername

* Start working on StreamSecrets RPC

* Extract response generation helper

* Log request

* Implemnt SDS versioning/nonce checking

* Handle renewal of SDS certs

* Refactor for ease of testing

* Start writing tests for SDS functionality

* Add E2E tests

* Fix test

* go mod tidy

* Appease linter

* Update go.mod

* Fix test with characters that windows dislikes
github-merge-queue bot pushed a commit that referenced this pull request Aug 14, 2024
* Start hacking on Workload Identity SDS API support

* Actually generate from SVIDs/Trustbundle

* Support for envoy "special meaning" resource names

* Add minimum version enforcement

* Simplify handling of clustername

* Start working on StreamSecrets RPC

* Extract response generation helper

* Log request

* Implemnt SDS versioning/nonce checking

* Handle renewal of SDS certs

* Refactor for ease of testing

* Start writing tests for SDS functionality

* Add E2E tests

* Fix test

* go mod tidy

* Appease linter

* Update go.mod

* Fix test with characters that windows dislikes
vapopov pushed a commit that referenced this pull request Aug 15, 2024
* Start hacking on Workload Identity SDS API support

* Actually generate from SVIDs/Trustbundle

* Support for envoy "special meaning" resource names

* Add minimum version enforcement

* Simplify handling of clustername

* Start working on StreamSecrets RPC

* Extract response generation helper

* Log request

* Implemnt SDS versioning/nonce checking

* Handle renewal of SDS certs

* Refactor for ease of testing

* Start writing tests for SDS functionality

* Add E2E tests

* Fix test

* go mod tidy

* Appease linter

* Update go.mod

* Fix test with characters that windows dislikes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Machine ID: Support for Envoy SDS for Workload Identity
3 participants