LDS stuck after envoy reload

[Lookyan]

Envoy doesn't open new LDS stream after its reload

Description:
I use envoy with contour (https://github.com/heptio/contour) as management server.
When they initially start simultaneously everything works fine. But after I reload envoy server LDS and RDS resources don't receive any updates. And it looks like envoy doesn't query for LDS. Contour doesn't receive new LDS stream. And after contour reload I receive new LDS stream.
It reproduces only with 100+ streams for CDS/EDS opened.
What's interesting - there is no problem with CDS/EDS.
Envoy and contour use grpc.

I collected envoy logs with -l trace but there is no information about problems with LDS, listeneres and so on.

Are there any limitations for grpc streams or maybe smth else?

Info for contour connection from /stats:

contour::outlier::success_rate_average::-1
contour::outlier::success_rate_ejection_threshold::-1
contour::default_priority::max_connections::100000
contour::default_priority::max_pending_requests::100000
contour::default_priority::max_requests::10000000
contour::default_priority::max_retries::3
contour::high_priority::max_connections::100000
contour::high_priority::max_pending_requests::100000
contour::high_priority::max_requests::10000000
contour::high_priority::max_retries::3
contour::added_via_api::false
contour::192.168.64.1:8002::cx_active::1
contour::192.168.64.1:8002::cx_connect_fail::0
contour::192.168.64.1:8002::cx_total::1
contour::192.168.64.1:8002::rq_active::121
contour::192.168.64.1:8002::rq_error::0
contour::192.168.64.1:8002::rq_success::0
contour::192.168.64.1:8002::rq_timeout::0
contour::192.168.64.1:8002::rq_total::121
contour::192.168.64.1:8002::health_flags::healthy
contour::192.168.64.1:8002::weight::1
contour::192.168.64.1:8002::region::
contour::192.168.64.1:8002::zone::
contour::192.168.64.1:8002::sub_zone::
contour::192.168.64.1:8002::canary::false
contour::192.168.64.1:8002::success_rate::-1

Envoy config:

dynamic_resources:
  lds_config:
    api_config_source:
      api_type: GRPC
      cluster_names: [contour]
      grpc_services:
      - envoy_grpc:
          cluster_name: contour
  cds_config:
    api_config_source:
      api_type: GRPC
      cluster_names: [contour]
      grpc_services:
      - envoy_grpc:
          cluster_name: contour

static_resources:
  clusters:
  - name: contour
    connect_timeout: { seconds: 5 }
    type: STRICT_DNS
    hosts:
    - socket_address:
        address: 192.168.64.1
        port_value: 8002
    lb_policy: ROUND_ROBIN
    http2_protocol_options: { hpack_table_size: 4294967, max_concurrent_streams: 2147483647 }
    per_connection_buffer_limit_bytes: 100000000
    max_requests_per_connection: 10000
    circuit_breakers:
      thresholds:
        - priority: high
          max_connections: 100000
          max_pending_requests: 100000
          max_requests: 10000000
        - priority: default
          max_connections: 100000
          max_pending_requests: 100000
          max_requests: 10000000
    outlier_detection:
      max_ejection_percent: 0
admin:
  access_log_path: /dev/null
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 9001

And server_info

localhost:9001/server_info
envoy 4dd49d8809f7aaa580538b3c228dd99a2fae92a4/1.6.0/Clean/RELEASE live 384 384 0

[kyessenov]

cc @davecheney for attention

[Lookyan]

I found out that it is getting stuck at this stage:

[2018-03-30 08:38:16.838][872][info][upstream] source/common/upstream/cluster_manager_impl.cc:108] cm init: initializing secondary clusters

[Lookyan]

Solved by this PR: projectcontour/contour#308