Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Original Dst apparently not working for me #36804

Closed
AdrianSchlegel opened this issue Oct 24, 2024 · 3 comments
Closed

Original Dst apparently not working for me #36804

AdrianSchlegel opened this issue Oct 24, 2024 · 3 comments
Labels
question Questions that are neither investigations, bugs, nor enhancements stale stalebot believes this issue/PR has not been touched recently

Comments

@AdrianSchlegel
Copy link

AdrianSchlegel commented Oct 24, 2024 •
edited
Loading

Hello Dear Envoy Team,

I am trying to create a proxy which takes any traffic (on loopback addresses and http1.1) and converts it to http 2. So basically 127.0.0.10:7777 gets transferred to 127.0.0.10:7777 after a http2 conversion and so on.

I have solved the part of http2 converson with envoy proxy however I am having problems with the sending to the appropriate destination. I was using the original_dst cluster policy / listening filter with none working.

At some point I tried out the example configuration of this code from the envoy github repository found here: https://github.com/envoyproxy/envoy/blob/main/configs/original-dst-cluster/proxy_config.yaml

However this code doesnt work either. This code doesnt have the http2 conversion yet and is only supposed to dynamically find the destination. To test this I hosted a python http server with:
python3 -m http.server 80.

Then I tried reaching it (first without proxy):

adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$ curl localhost 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Directory listing for /</title>
</head>
<body>
<h1>Directory listing for /</h1>
<hr>
<ul>
</ul>
<hr>
</body>
</html>

As you see its working fine.

Now I use the code from the before mentioned link as config for my proxy (https://github.com/envoyproxy/envoy/blob/main/configs/original-dst-cluster/proxy_config.yaml) (please note i changed the listener address to 127.0.0.1:8082):

adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$ curl localhost -x 127.0.0.1:8082 -v

*   Trying 127.0.0.1:8082...
* Connected to (nil) (127.0.0.1) port 8082 (#0)
> GET http://localhost/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.81.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< content-length: 148
< content-type: text/plain
< date: Thu, 24 Oct 2024 11:19:28 GMT
< server: envoy
< x-envoy-upstream-service-time: 538
< 
* Connection #0 to host (nil) left intact
upstream connect error or disconnect/reset before headers. reset reason: local connection failure, transport failure reason: socket creation failure

With the envoy proxy showing this in stdout:

[2024-10-24 13:19:24.945][66651][info][main] [source/server/server.cc:990] starting main dispatch loop
[2024-10-24 13:19:29.328][66659][error][envoy_bug] [source/common/network/socket_interface_impl.cc:98] envoy bug failure: false. Details: socket(2) failed, got error: Too many open files
[2024-10-24 13:19:29.329][66659][error][envoy_bug] [./source/common/common/assert.h:38] stacktrace for envoy bug
[symbolize_elf.inc : 1072] RAW: /proc/self/task/66651/maps: errno=24
[2024-10-24 13:19:29.331][66659][error][envoy_bug] [./source/common/common/assert.h:45] #0 UNKNOWN [0x64f153577083]
[2024-10-24 13:19:29.331][66659][error][envoy_bug] [./source/common/common/assert.h:45] #1 UNKNOWN [0x64f1534577b5]
[2024-10-24 13:19:29.332][66659][error][envoy_bug] [./source/common/common/assert.h:45] #2 UNKNOWN [0x64f153452535]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #3 UNKNOWN [0x64f153446c8f]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #4 UNKNOWN [0x64f15343caa6]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #5 UNKNOWN [0x64f1530378ca]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #6 UNKNOWN [0x64f153037260]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #7 UNKNOWN [0x64f15301388b]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #8 UNKNOWN [0x64f1530136f4]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #9 UNKNOWN [0x64f1530149aa]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #10 UNKNOWN [0x64f153014d63]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #11 UNKNOWN [0x64f15302b412]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #12 UNKNOWN [0x64f15302e9f9]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #13 UNKNOWN [0x64f15301bd22]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #14 UNKNOWN [0x64f153273125]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #15 UNKNOWN [0x64f153295ea4]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [source/common/network/connection_impl.cc:89] envoy bug failure: false. Details: Client socket failure
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:38] stacktrace for envoy bug
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #0 UNKNOWN [0x64f1534526a5]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #1 UNKNOWN [0x64f15345255c]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #2 UNKNOWN [0x64f153446c8f]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #3 UNKNOWN [0x64f15343caa6]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #4 UNKNOWN [0x64f1530378ca]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #5 UNKNOWN [0x64f153037260]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #6 UNKNOWN [0x64f15301388b]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #7 UNKNOWN [0x64f1530136f4]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #8 UNKNOWN [0x64f1530149aa]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #9 UNKNOWN [0x64f153014d63]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #10 UNKNOWN [0x64f15302b412]
[2024-10-24 13:19:29.333][66659][error][envoy_bug] [./source/common/common/assert.h:45] #11 UNKNOWN [0x64f15302e9f9]
[2024-10-24 13:19:29.334][66659][error][envoy_bug] [./source/common/common/assert.h:45] #12 UNKNOWN [0x64f15301bd22]
[2024-10-24 13:19:29.334][66659][error][envoy_bug] [./source/common/common/assert.h:45] #13 UNKNOWN [0x64f153273125]
[2024-10-24 13:19:29.334][66659][error][envoy_bug] [./source/common/common/assert.h:45] #14 UNKNOWN [0x64f153295ea4]
[2024-10-24 13:19:29.334][66659][error][envoy_bug] [./source/common/common/assert.h:45] #15 UNKNOWN [0x64f15327c7bc]

I also know that there is no sort of issue with my envoy because when i am running my other envoy configuration for the http2 conversion (config code following). I get these results:

###endpoint that only can get http2 packets. With proof using --http2-prior-knowledge and withotu using it###
adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$ curl 'http://127.0.0.10:7777/nnrf-disc/v1/nf-instances?target-nf-type=AMF&requester-nf-type=SMF' --http2-prior-knowledge
{"validityPeriod":30,"nfInstances":[{"nfInstanceId":"452b4c46-9076-41ef-a30b-053a47f5ba84","nfType":"AMF","nfStatus":"REGISTERED","heartBeatTimer":10,"plmnList":[{"mcc":"999","mnc":"70"}],"ipv4Addresses":["127.0.0.5"],"allowedNfTypes":["SCP","SMF","AMF"],"priority":0,"capacity":100,"load":0,"amfInfo":{"amfSetId":"001","amfRegionId":"02","guamiList":[{"plmnId":{"mcc":"999","mnc":"70"},"amfId":"020040"}],"taiList":[{"plmnId":{"mcc":"999","mnc":"70"},"tac":"000001"}]},"nfServices":[{"serviceInstanceId":"452d0450-9076-41ef-a30b-053a47f5ba84","serviceName":"namf-comm","versions":[{"apiVersionInUri":"v1","apiFullVersion":"1.0.0"}],"scheme":"http","nfServiceStatus":"REGISTERED","ipEndPoints":[{"ipv4Address":"127.0.0.5","port":7777}],"allowedNfTypes":["SMF","AMF"],"priority":0,"capacity":100,"load":0}],"nfProfileChangesSupportInd":true}]}

adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$ curl 'http://127.0.0.10:7777/nnrf-disc/v1/nf-instances?target-nf-type=AMF&requester-nf-type=SMF'
curl: (1) Received HTTP/0.9 when not allowed

###Request works using my proxy config###
adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$ curl 'http://127.0.0.10:7777/nnrf-disc/v1/nf-instances?target-nf-type=AMF&requester-nf-type=SMF' -x http://127.0.0.1:8082
{"validityPeriod":30,"nfInstances":[{"nfInstanceId":"452b4c46-9076-41ef-a30b-053a47f5ba84","nfType":"AMF","nfStatus":"REGISTERED","heartBeatTimer":10,"plmnList":[{"mcc":"999","mnc":"70"}],"ipv4Addresses":["127.0.0.5"],"allowedNfTypes":["SCP","SMF","AMF"],"priority":0,"capacity":100,"load":0,"amfInfo":{"amfSetId":"001","amfRegionId":"02","guamiList":[{"plmnId":{"mcc":"999","mnc":"70"},"amfId":"020040"}],"taiList":[{"plmnId":{"mcc":"999","mnc":"70"},"tac":"000001"}]},"nfServices":[{"serviceInstanceId":"452d0450-9076-41ef-a30b-053a47f5ba84","serviceName":"namf-comm","versions":[{"apiVersionInUri":"v1","apiFullVersion":"1.0.0"}],"scheme":"http","nfServiceStatus":"REGISTERED","ipEndPoints":[{"ipv4Address":"127.0.0.5","port":7777}],"allowedNfTypes":["SMF","AMF"],"priority":0,"capacity":100,"load":0}],"nfProfileChangesSupportInd":true}]}adrian@adrian-virtual-machine:~/Documents/5g-pentesting-project$

MY PROXY CONFIG:

admin:
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 15000

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        address: 127.0.0.1
        port_value: 8082
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: service_http2

  clusters:
  - name: service_http2
    connect_timeout: 0.25s
    type: strict_dns
    load_assignment:
      cluster_name: service_http2
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: 127.0.0.10  # Upstream service IP
                port_value: 7777        # Upstream service port
    http2_protocol_options: {}  # Enable HTTP/2 for upstream

I would very much appreciate it if someone could tell me what I am doing wrong in the first part of the issue. I am literally using the example in the github repo and it is not working for me with the error codes which I have shown. I wish to have a proxy which just forwards the requests further to their destination.

I am also open to other solutions to this without the original_dst if they work too.

Thank you very much for the help :)
@AdrianSchlegel AdrianSchlegel added the triage Issue requires triage label Oct 24, 2024
@wbpcode
Copy link
Member

wbpcode commented Oct 24, 2024

original dst cluster will try to get SO_ORIGINAL_DST to get the original destination. Or you can also let the cluster to get original destination from the header. See https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/original_dst

@wbpcode wbpcode added question Questions that are neither investigations, bugs, nor enhancements and removed triage Issue requires triage labels Oct 24, 2024
@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label Nov 23, 2024
@github-actions GitHub Actions
Copy link

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Questions that are neither investigations, bugs, nor enhancements stale stalebot believes this issue/PR has not been touched recently
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wbpcode @AdrianSchlegel