Deploying services on multiple instances using docker swarm

[odidev]

I’m trying to deploy the ftgo-application services on multiple instances using docker swarm on X86_64 and Arm64 platforms, but the containers are not starting and remain in pending state on both platforms.

I have created 5 instances, installed docker and docker-compose in all the instances and followed the below steps to deploy the services:

• Created the docker swarm cluster. Chose any of the instance as manager/Master node and ran below mentioned command:

root@ip-master:/home/ubuntu# docker swarm init --advertise-addr <Master node IP> 
Swarm initialized: current node (lv7ewslg2g7ip1m7kb4xn5s06) is now a manager. 

To add a worker to this swarm, run the following command: 

    docker swarm join --token <Token> <Master node IP>:2377 

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. 

• Added the worker nodes to the cluster by running the following command on all the worker nodes with the token generated above.

root@ip-worker:/home/ubuntu# docker swarm join --token <Token> <Master node IP>:2377 
This node joined a swarm as a worker. 

• Created an overlay network which is used to establish communication between multiple Docker daemon hosts. Run this following command to create a overlay network:

root@ip-master:/home/ubuntu# docker network create -d overlay testservice 
n7rag68jwexptd33fhm95p370 

• Updated the docker-compose.yml to deploy services on multiple instances.

• Deployed the container over the docker swarm cluster using the following command:

root@ip-master:/home/ubuntu/ftgo-application# docker stack deploy -c docker-compose.yml ftgo 
Ignoring unsupported options: restart 

Creating service ftgo_mysql 
Creating service ftgo_ftgo-accounting-service 
Creating service ftgo_kafka 
Creating service ftgo_cdc-service 
Creating service ftgo_dynamodblocal 
Creating service ftgo_kafka-gui 
Creating service ftgo_dynamodblocal-init 
Creating service ftgo_ftgo-order-service 
Creating service ftgo_ftgo-order-history-service 
Creating service ftgo_ftgo-kitchen-service 
Creating service ftgo_ftgo-consumer-service 
Creating service ftgo_zookeeper 
Creating service ftgo_ftgo-api-gateway 
Creating service ftgo_zipkin 
Creating service ftgo_ftgo-restaurant-service 
Creating service ftgo_ftgo-delivery-service 

• Checked the containers by running the following command:

root@ip-master:/home/ubuntu/ftgo-application# docker stack ps ftgo --no-trunc 
ID                          NAME                                    IMAGE                                                                                                                                     NODE               DESIRED STATE   CURRENT STATE                     ERROR                       PORTS 
16aav55hfalrh1uh4dx1esd5n   ftgo_cdc-service.1                      eventuateio/eventuate-cdc-service:0.13.0.RELEASE@sha256:e645251d6a96b4161580b5b1ef534cd8f035f1e83173d73568a409f05690a556                  ip-worker3    Ready           Ready 2 seconds ago 
11fbukbw2sc3wnfysx8qpkpyg    \_ ftgo_cdc-service.1                  eventuateio/eventuate-cdc-service:0.13.0.RELEASE@sha256:e645251d6a96b4161580b5b1ef534cd8f035f1e83173d73568a409f05690a556                  ip-worker4    Shutdown        Failed 3 seconds ago              "task: non-zero exit (1)" 
nx395plvbk5motv8z3edugynk    \_ ftgo_cdc-service.1                  eventuateio/eventuate-cdc-service:0.13.0.RELEASE@sha256:e645251d6a96b4161580b5b1ef534cd8f035f1e83173d73568a409f05690a556                  ip-worker4    Shutdown        Failed 13 seconds ago             "task: non-zero exit (1)" 
hybheldpqf9n5avxn4aen0k6w    \_ ftgo_cdc-service.1                  eventuateio/eventuate-cdc-service:0.13.0.RELEASE@sha256:e645251d6a96b4161580b5b1ef534cd8f035f1e83173d73568a409f05690a556                  ip-worker3    Shutdown        Failed 23 seconds ago             "task: non-zero exit (1)" 
l97276vogprf773nfx03g13qe    \_ ftgo_cdc-service.1                  eventuateio/eventuate-cdc-service:0.13.0.RELEASE@sha256:e645251d6a96b4161580b5b1ef534cd8f035f1e83173d73568a409f05690a556                  ip-worker4    Shutdown        Failed 34 seconds ago             "task: non-zero exit (1)" 
rg2phi27rgep2v7gork1zu0l8   ftgo_dynamodblocal-init.1               odidev/ftgo-application_dynamodblocal-init:latest@sha256:af9e15848ab9c26ccf3437c2ece47390bc2fad0a25396d56a83a27857f7aa623           ip-worker1   Running         Running about a minute ago 
5rtowbb007opd9a8ygfsy6ek5   ftgo_dynamodblocal.1                    odidev/ftgo-application_dynamodblocal:latest@sha256:0bef4df905d558b2a6d666531acef136de00c238a3184111c2d1a91464c1b54a                ip-worker4    Running         Running about a minute ago 
50inoyuchp1hpkhud0kn3p0qv   ftgo_ftgo-accounting-service.1          odidev/ftgo-application_ftgo-accounting-service:latest@sha256:72ef0f8b1893a888cb35be318d5b15309418f9d6871038b14d3843e08e0bb91f      ip-worker4    Running         Starting 10 seconds ago 
oanx952to6a2fcvqosccpxkic    \_ ftgo_ftgo-accounting-service.1      odidev/ftgo-application_ftgo-accounting-service:latest@sha256:72ef0f8b1893a888cb35be318d5b15309418f9d6871038b14d3843e08e0bb91f      ip-worker4    Shutdown        Failed 15 seconds ago             "task: non-zero exit (1)" 
xd6jiy9okagm0zb8fvs00lwj1    \_ ftgo_ftgo-accounting-service.1      odidev/ftgo-application_ftgo-accounting-service:latest@sha256:72ef0f8b1893a888cb35be318d5b15309418f9d6871038b14d3843e08e0bb91f      ip-worker4    Shutdown        Failed 30 seconds ago             "task: non-zero exit (1)" 
c77if220ehga9nfw5yg8ssboo    \_ ftgo_ftgo-accounting-service.1      odidev/ftgo-application_ftgo-accounting-service:latest@sha256:72ef0f8b1893a888cb35be318d5b15309418f9d6871038b14d3843e08e0bb91f      ip-worker2   Shutdown        Failed 46 seconds ago             "task: non-zero exit (1)" 
ipmzfmeaatdg7ry26nks4k1n4    \_ ftgo_ftgo-accounting-service.1      odidev/ftgo-application_ftgo-accounting-service:latest@sha256:72ef0f8b1893a888cb35be318d5b15309418f9d6871038b14d3843e08e0bb91f      ip-worker2   Shutdown        Failed about a minute ago         "task: non-zero exit (1)" 
cvlx2u9krnxez3312ked9x2wt   ftgo_ftgo-api-gateway.1                 odidev/ftgo-application_ftgo-api-gateway:latest@sha256:3db875c28d4f87686b3c2dc95ae0cbb27267fd4a720f1365de13d55b71cd5af6             ip-worker1   Running         Running about a minute ago 
arg4n5l2z34mde0k7i287clj7   ftgo_ftgo-consumer-service.1            odidev/ftgo-application_ftgo-consumer-service:latest@sha256:9b7e2ec7df02fa9682412dcb6434ea8779c720f73ff7987ec22d6711adaad88f        ip-worker1   Running         Starting less than a second ago 
dculobj7jrmfoo0fg580g0oar    \_ ftgo_ftgo-consumer-service.1        odidev/ftgo-application_ftgo-consumer-service:latest@sha256:9b7e2ec7df02fa9682412dcb6434ea8779c720f73ff7987ec22d6711adaad88f        ip-worker3    Shutdown        Failed 5 seconds ago              "task: non-zero exit (1)" 
xvl3mmna9putzhjtl1sy0asw4    \_ ftgo_ftgo-consumer-service.1        odidev/ftgo-application_ftgo-consumer-service:latest@sha256:9b7e2ec7df02fa9682412dcb6434ea8779c720f73ff7987ec22d6711adaad88f        ip-worker3    Shutdown        Failed 24 seconds ago             "task: non-zero exit (1)" 
it2x4jt961mmpx3t0cjn83qdf    \_ ftgo_ftgo-consumer-service.1        odidev/ftgo-application_ftgo-consumer-service:latest@sha256:9b7e2ec7df02fa9682412dcb6434ea8779c720f73ff7987ec22d6711adaad88f        ip-worker3    Shutdown        Failed 43 seconds ago             "task: non-zero exit (1)" 
38k8hxgy7a5av2at0g95vlmwa    \_ ftgo_ftgo-consumer-service.1        odidev/ftgo-application_ftgo-consumer-service:latest@sha256:9b7e2ec7df02fa9682412dcb6434ea8779c720f73ff7987ec22d6711adaad88f        ip-worker3    Shutdown        Failed about a minute ago         "task: non-zero exit (1)" 
bnkxvakps2nkrpsg90p99mon3   ftgo_ftgo-delivery-service.1            odidev/ftgo-application_ftgo-delivery-service:latest@sha256:f2e0f77ed75c1d5132069b6774b015b930ac476befdab1d2bf17dc3109cfc46f        ip-worker4    Ready           Ready 3 seconds ago 
86ulf7u5e0jova9w6fb2f242l    \_ ftgo_ftgo-delivery-service.1        odidev/ftgo-application_ftgo-delivery-service:latest@sha256:f2e0f77ed75c1d5132069b6774b015b930ac476befdab1d2bf17dc3109cfc46f        ip-worker3    Shutdown        Failed 5 seconds ago              "task: non-zero exit (1)" 
nso5y7bo3et7hny8fu3o4w7nl    \_ ftgo_ftgo-delivery-service.1        odidev/ftgo-application_ftgo-delivery-service:latest@sha256:f2e0f77ed75c1d5132069b6774b015b930ac476befdab1d2bf17dc3109cfc46f        ip-worker3    Shutdown        Failed 25 seconds ago             "task: non-zero exit (1)" 
bge1ag7ysi5no6ig3h4jvxb8t    \_ ftgo_ftgo-delivery-service.1        odidev/ftgo-application_ftgo-delivery-service:latest@sha256:f2e0f77ed75c1d5132069b6774b015b930ac476befdab1d2bf17dc3109cfc46f        ip-worker3    Shutdown        Failed 44 seconds ago             "task: non-zero exit (1)" 
yrco4dup1q14kx464z7c0gq3s    \_ ftgo_ftgo-delivery-service.1        odidev/ftgo-application_ftgo-delivery-service:latest@sha256:f2e0f77ed75c1d5132069b6774b015b930ac476befdab1d2bf17dc3109cfc46f        ip-worker3    Shutdown        Failed about a minute ago         "task: non-zero exit (1)" 
392nbtw7emlxn4numaa2ctuuk   ftgo_ftgo-kitchen-service.1             odidev/ftgo-application_ftgo-kitchen-service:latest@sha256:4390ab368c71f659ce7004befe9e548b03a40b65128bb645db5f1e18e80a607b         ip-worker2   Running         Starting less than a second ago 
uq04cpbhzsbrip9aus9s4hars    \_ ftgo_ftgo-kitchen-service.1         odidev/ftgo-application_ftgo-kitchen-service:latest@sha256:4390ab368c71f659ce7004befe9e548b03a40b65128bb645db5f1e18e80a607b         ip-worker2   Shutdown        Failed 5 seconds ago              "task: non-zero exit (1)" 
ld9j016wu0llz2r967kwg4ei7    \_ ftgo_ftgo-kitchen-service.1         odidev/ftgo-application_ftgo-kitchen-service:latest@sha256:4390ab368c71f659ce7004befe9e548b03a40b65128bb645db5f1e18e80a607b         ip-worker2   Shutdown        Failed 19 seconds ago             "task: non-zero exit (1)" 
ednp075q771sf1i8v5lhc3oro    \_ ftgo_ftgo-kitchen-service.1         odidev/ftgo-application_ftgo-kitchen-service:latest@sha256:4390ab368c71f659ce7004befe9e548b03a40b65128bb645db5f1e18e80a607b         ip-worker2   Shutdown        Failed 33 seconds ago             "task: non-zero exit (1)" 
dimdaa0z6lshxuuprkjmkyd9x    \_ ftgo_ftgo-kitchen-service.1         odidev/ftgo-application_ftgo-kitchen-service:latest@sha256:4390ab368c71f659ce7004befe9e548b03a40b65128bb645db5f1e18e80a607b         ip-worker3    Shutdown        Failed 46 seconds ago             "task: non-zero exit (1)" 
3cf2cw1r3uotqu2f0np2zi2mf   ftgo_ftgo-order-history-service.1       odidev/ftgo-application_ftgo-order-history-service:latest@sha256:afeec2e6eb81053fc543e6597dedc61dfaa2141cbcaab271e4968adac7c56040   ip-worker3    Running         Starting 2 seconds ago 
wpplnq6zdd1mwtuw2dg3vvhk9    \_ ftgo_ftgo-order-history-service.1   odidev/ftgo-application_ftgo-order-history-service:latest@sha256:afeec2e6eb81053fc543e6597dedc61dfaa2141cbcaab271e4968adac7c56040   ip-worker3    Shutdown        Failed 7 seconds ago              "task: non-zero exit (1)" 
8fgu1o8cnped8raki8nw9ado2    \_ ftgo_ftgo-order-history-service.1   odidev/ftgo-application_ftgo-order-history-service:latest@sha256:afeec2e6eb81053fc543e6597dedc61dfaa2141cbcaab271e4968adac7c56040   ip-worker4    Shutdown        Failed 23 seconds ago             "task: non-zero exit (1)" 
hrvuwtt3jdqejaq72ouoqezuu    \_ ftgo_ftgo-order-history-service.1   odidev/ftgo-application_ftgo-order-history-service:latest@sha256:afeec2e6eb81053fc543e6597dedc61dfaa2141cbcaab271e4968adac7c56040   ip-worker3    Shutdown        Failed 34 seconds ago             "task: non-zero exit (1)" 
rx2g51hc5f88yzuy88aurjika    \_ ftgo_ftgo-order-history-service.1   odidev/ftgo-application_ftgo-order-history-service:latest@sha256:afeec2e6eb81053fc543e6597dedc61dfaa2141cbcaab271e4968adac7c56040   ip-worker4    Shutdown        Failed 46 seconds ago             "task: non-zero exit (1)" 
tcutfgojr9uuymvd7du053807   ftgo_ftgo-order-service.1               odidev/ftgo-application_ftgo-order-service:latest@sha256:446ecbbe3540f40a643e9355487d8751f3f9fba2029012771d82d0fbf0ffec6e           ip-worker1   Running         Starting 1 second ago 
j2ekzodbuzpbme8y0xhqlyjs4    \_ ftgo_ftgo-order-service.1           odidev/ftgo-application_ftgo-order-service:latest@sha256:446ecbbe3540f40a643e9355487d8751f3f9fba2029012771d82d0fbf0ffec6e           ip-worker1   Shutdown        Failed 6 seconds ago              "task: non-zero exit (1)" 
qfb4trgud6u47vk5ab9t6lqzd    \_ ftgo_ftgo-order-service.1           odidev/ftgo-application_ftgo-order-service:latest@sha256:446ecbbe3540f40a643e9355487d8751f3f9fba2029012771d82d0fbf0ffec6e           ip-worker1   Shutdown        Failed 20 seconds ago             "task: non-zero exit (1)" 
yxqktrok6tipnyesticgng2iv    \_ ftgo_ftgo-order-service.1           odidev/ftgo-application_ftgo-order-service:latest@sha256:446ecbbe3540f40a643e9355487d8751f3f9fba2029012771d82d0fbf0ffec6e           ip-worker1   Shutdown        Failed 35 seconds ago             "task: non-zero exit (1)" 
v9a20bt7opx6rl3oxewd78f5h    \_ ftgo_ftgo-order-service.1           odidev/ftgo-application_ftgo-order-service:latest@sha256:446ecbbe3540f40a643e9355487d8751f3f9fba2029012771d82d0fbf0ffec6e           ip-worker4    Shutdown        Failed 49 seconds ago             "task: non-zero exit (1)" 
x0ql89dlwm40h3e6kd6hkkia5   ftgo_ftgo-restaurant-service.1          odidev/ftgo-application_ftgo-restaurant-service:latest@sha256:39b1d672d0f9600cf4cf204a7d7910fcd22893837c7b95199dab0a0e9c170ca3      ip-worker1   Running         Running 39 seconds ago 
wd9x5ysca3ncsz6tb71t6wiuc    \_ ftgo_ftgo-restaurant-service.1      odidev/ftgo-application_ftgo-restaurant-service:latest@sha256:39b1d672d0f9600cf4cf204a7d7910fcd22893837c7b95199dab0a0e9c170ca3      ip-worker1   Shutdown        Failed 55 seconds ago             "task: non-zero exit (1)" 
elu047qcv2wbc177hftep648a    \_ ftgo_ftgo-restaurant-service.1      odidev/ftgo-application_ftgo-restaurant-service:latest@sha256:39b1d672d0f9600cf4cf204a7d7910fcd22893837c7b95199dab0a0e9c170ca3      ip-worker1   Shutdown        Failed about a minute ago         "task: non-zero exit (1)" 
7cqokqbs6b949fvja6np5ybl7    \_ ftgo_ftgo-restaurant-service.1      odidev/ftgo-application_ftgo-restaurant-service:latest@sha256:39b1d672d0f9600cf4cf204a7d7910fcd22893837c7b95199dab0a0e9c170ca3      ip-worker1   Shutdown        Failed about a minute ago         "task: non-zero exit (1)" 
n5w5grlxc1mqxwpbq3ae8vefk   ftgo_kafka-gui.1                        quay.io/cloudhut/kowl:master-1d07695@sha256:c22fd2e3bb59f15c954b3b3d347ca7fcc804e4f12132536d903f35e67b0fedb4                              ip-worker3    Running         Running 15 seconds ago 
ss41w9iw5vg3sucazjob4mrpb    \_ ftgo_kafka-gui.1                    quay.io/cloudhut/kowl:master-1d07695@sha256:c22fd2e3bb59f15c954b3b3d347ca7fcc804e4f12132536d903f35e67b0fedb4                              ip-worker2   Shutdown        Failed 22 seconds ago             "task: non-zero exit (1)" 
lb4mhv2x7vlwxfm0dlxkulruv    \_ ftgo_kafka-gui.1                    quay.io/cloudhut/kowl:master-1d07695@sha256:c22fd2e3bb59f15c954b3b3d347ca7fcc804e4f12132536d903f35e67b0fedb4                              ip-worker2   Shutdown        Failed 59 seconds ago             "task: non-zero exit (1)" 
z6wfr0llqgywniuxvspirjcar   ftgo_kafka.1                            eventuateio/eventuate-kafka:0.15.0.RELEASE@sha256:dd37156b454cbf042a926fd72306c811593e512a84d0d9346a3de990823254f9                        ip-worker2   Ready           Ready 2 seconds ago 
67f31xqn71kcr3ytbbs5wi90i    \_ ftgo_kafka.1                        eventuateio/eventuate-kafka:0.15.0.RELEASE@sha256:dd37156b454cbf042a926fd72306c811593e512a84d0d9346a3de990823254f9                        ip-worker2   Shutdown        Failed 2 seconds ago              "task: non-zero exit (1)" 
mb4mixtg035l71xyox19gjzqh    \_ ftgo_kafka.1                        eventuateio/eventuate-kafka:0.15.0.RELEASE@sha256:dd37156b454cbf042a926fd72306c811593e512a84d0d9346a3de990823254f9                        ip-worker2   Shutdown        Failed 12 seconds ago             "task: non-zero exit (1)" 
9txdco4ke83hnsuegkhxd9smt    \_ ftgo_kafka.1                        eventuateio/eventuate-kafka:0.15.0.RELEASE@sha256:dd37156b454cbf042a926fd72306c811593e512a84d0d9346a3de990823254f9                        ip-worker1   Shutdown        Failed 22 seconds ago             "task: non-zero exit (1)" 
m9znlhum5asttmdxxakfoh1t0    \_ ftgo_kafka.1                        eventuateio/eventuate-kafka:0.15.0.RELEASE@sha256:dd37156b454cbf042a926fd72306c811593e512a84d0d9346a3de990823254f9                        ip-worker3    Shutdown        Failed 31 seconds ago             "task: non-zero exit (1)" 
nvhxwf8pr7qc8z78dpua7g527   ftgo_mysql.1                            odidev/ftgo-application_mysql:latest@sha256:67a997f7a18e0c1aff3a55c354bdfd98d30500f6d871121138ec0b1f2294a0a0                        ip-worker2   Running         Running about a minute ago 
qk2pdi7pt30fa766ejmhe6mpi   ftgo_zipkin.1                           openzipkin/zipkin:2@sha256:2d797c2da4aa98aee166eef82852cfb3cf09c1c3c3e48905be3564ce7e6acfc4                                               ip-worker4    Running         Running about a minute ago 
uvx8uzjikhsq8flj4ujk4r5oy   ftgo_zookeeper.1                        eventuateio/eventuate-zookeeper:0.15.0.RELEASE@sha256:714e2d3518ae1e7c91764a8fcbb4b3ef419128a066af053ba7b8390766d40c54                    ip-worker2   Running         Running about a minute ago 

Could you please have a look at this and share your suggestions on the same?