Could Not Start Kong

[timothyley]

Hi, Kong was working fine previously. However about 2 weeks ago, it started to give me error when I start Kong. I am not sure what the problem is. It gave me this error.

dex@kongcassandra2:~$ sudo kong start
[INFO] Kong 0.6.1
[INFO] Using configuration: /etc/kong/kong.yml
[INFO] database...........cassandra replication_factor=1 ssl=verify=false enabled=false data_centers=        timeout=5000 replication_strategy=SimpleStrategy contact_points=127.0.0.1:9042 keyspace=kong
[INFO] dnsmasq............address=127.0.0.1:8053 dnsmasq=true port=8053
[WARN] ulimit is currently set to "1024". For better performance set it to at least "4096" using "ulimit -n"
[INFO] nginx .............admin_api_listen=0.0.0.0:8001 proxy_listen=0.0.0.0:80     proxy_listen_ssl=0.0.0.0:443
[INFO] Leaving cluster..
[ERR] Could not start serf: Failed to start the Serf agent: Error creating Serf: Failed to create     memberlist: No private IP address found, and explicit IP not provided
[ERR] Could not start Kong
dex@kongcassandra2:~$ 

Looking for some help please.

[subnetmarco]

@timothyley I think your issue is the same as #1037. What distribution of Kong are you using?

[timothyley]

Hi Macro,

I'm sorry as I am new to Kong. The setup was done by my ex colleague. I am
currently running on Kong 0.6.1. Also, how do I set an IP address using the
cluster.advertise property

Regard,
Tim
ᐧ

On Wed, Mar 30, 2016 at 1:11 AM, Marco Palladino notifications@github.com
wrote:

@timothyley https://github.com/timothyley I think your issue is the
same as #1037 #1037. What
distribution of Kong are you using?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1107 (comment)

[subnetmarco]

@timothyley you will have to both explicitly set the cluster_listen and cluster.advertise properties in the configuration file with an IP address that belongs to an interface that you can use. To better understand why the IP address can't be automatically found, I would need to know the distribution and environment of your setup (OS, cloud provider, etc).

For more info on the doc for 0.6.x, you can check: https://getkong.org/docs/0.6.x/configuration

[timothyley]

@thefosk This is the environment we are in. Linux kongcassandra2 3.19.0-49-generic #55~14.04.1-Ubuntu SMP Fri Jan 22 11:24:31 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux. Also our Kong is on Microsoft Azure.

[subnetmarco]

@timothyley can you please post the output of:

$ ifconfig

To figure out why serf cannot automatically find an IP address.

[timothyley]

eth0      Link encap:Ethernet  HWaddr 00:0d:3a:a0:e7:06  
          inet addr:100.75.34.83  Bcast:100.75.35.255  Mask:255.255.254.0
          inet6 addr: fe80::20d:3aff:fea0:e706/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1177618 errors:0 dropped:1 overruns:0 frame:0
          TX packets:1466399 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:406062947 (406.0 MB)  TX bytes:263152078 (263.1 MB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:4342788 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4342788 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:224875180 (224.8 MB)  TX bytes:224875180 (224.8 MB)

[subnetmarco]

@timothyley on the same machine, what happens if you execute:

$ serf agent

?

[timothyley]

@thefosk

$ serf agent
==> Starting Serf agent...
==> Failed to start the Serf agent: Error creating Serf: Failed to create memberlist: No private IP     address found, and explicit IP not provided
dex@kongcassandra2:/etc/kong$ 

[subnetmarco]

@timothyley thank you - I am trying to investigate this problem with the Serf team. Can you please keep the machine active in the meanwhile?

Also, can you try to start Kong by setting the following properties in kong.yml to these values:

cluster_listen: "100.75.34.83:7946"
cluster:
  advertise: "100.75.34.83:7946"

Make sure to uncomment any existing declaration of the same properties, and try running kong start again.

[timothyley]

@thefosk I am able to Start Kong. However when i tried call the API to list the node it provided me with an error.

$ curl -i -X GET http://ok.dex.sg/ 
HTTP/1.1 404 Not Found
Date: Wed, 30 Mar 2016 03:50:11 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: kong/0.6.1

{"request_path":"\/","message":"API not found with these values","request_host":["ok.dex.sg"]}

[subnetmarco]

@timothyley how did you start Kong? What was the problem with Serf?

[timothyley]

@thefosk I have added the properties that you have suggested

[subnetmarco]

@timothyley thanks, I will try to sort that issue out with the Serf team. Basically by specifying the cluster_listen property we forced Serf to listen on that IP (that for some reason it wasn't able to bind automatically).

Regarding the other problem, I would recommend taking a look at the Kong documentation. Kong runs on two ports:

• Port 8000, the proxy port that developers will use when consuming your APIs.
• Port 8001, the Kong admin API that you can use to operate Kong (and that should be only accessible to you with the proper firewall configuration).

The error you received is because Kong couldn't resolve the API. I would recommend reading https://getkong.org/docs/0.6.x/proxy/ to be sure that your APIs are properly configured on Kong.

[subnetmarco]

Related to hashicorp/serf#385.

This is not really a bug, and since the IP doesn't look like a private IP address it needs to be manually specified using cluster_listen.

[DonMartin76]

I just ran into this, as I had a network clash inside a docker network built with docker-compose with a network we were actually using inside our company (172.18.x.x). The super strange part with this is that serf works with 172.18.x.x, which is also not a private network range actually, but it does not work if I change the default network to:

networks:
  wicked:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 172.200.0.0/16
        gateway: 172.200.0.1

This is quite strange, and I have no clue who is the culprit here. The only way I could make Kong accept an IP address in the 172.200.x.x range was to explicitly assign the address to the container, plus add KONG_CLUSTER_ADVERTISE=172.200.0.3:7946 to my docker-compose.yml. That way it works, but it would break as soon as I tried to scale the Kong instances up, as each Kong instance would need its own IP address obviously.

[DonMartin76]

Hey okay, wait, I am mistaken. Of course 172.18.x is a private network, but 172.200.x isn't, and that's the difference.

[DonMartin76]

And it works with e.g. 172.31.255.0/24. So no need to reopen, I was just a little thick.

[tarunsaxenattn]

The below YAML worked for me. My DNS RESOLVER Is :- 172.0.0.10
I have assigned a advertise IP to Kong :- 172.244.3.10:7946 . Before that I was facing the same problem.

apiVersion: extensions/v1beta1

kind: Deployment
metadata:
name: kong
namespace: qa
spec:
replicas: 1
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
labels:
run: kong
spec:
containers:
- name: kong
image: kong
imagePullPolicy: Always
ports:
- containerPort: 8000
protocol: TCP
- containerPort: 8443
protocol: TCP
- containerPort: 8001
protocol: TCP
- containerPort: 7946
protocol: UDP
- containerPort: 7946
protocol: TCP
env:
- name: KONG_DATABASE
value: postgres
- name: KONG_PG_HOST
value: xxxxxx
- name: KONG_PG_USER
value: kong
- name: KONG_PG_PASSWORD
value: password
- name: KONG_PG_DATABASE
value: kong
- name: KONG_DNS_RESOLVER
value: 172.0.0.10
- name: KONG_DNSMASQ
value: "off"
- name: KONG_CLUSTER_ADVERTISE
value: 172.244.3.10:7946
resources:
limits:
memory: 1200Mi
requests:
memory: 1100Mi

apiVersion: v1
kind: Service
metadata:
name: svc-kong
namespace: qa
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:xxxxxxxx:certificate/xxxxxxxxxxxxxxxxxxx
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
ports:
- name: "www"
port: 8001
targetPort: 8001
- name: "www1"
port: 80
targetPort: 8000
- name: "https"
port: 443
targetPort: 8443

- name: "www2"

port: 7946

targetPort: 7946

selector:
run: kong
type: LoadBalancer