Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Figure out why Docker compose is failing in CI #1722

Closed
alecthomas opened this issue Jun 10, 2024 · 5 comments · Fixed by #1746
Closed

Figure out why Docker compose is failing in CI #1722

alecthomas opened this issue Jun 10, 2024 · 5 comments · Fixed by #1746
Assignees
@gak
Labels
bug Something isn't working

Comments

@alecthomas
Copy link
Collaborator

With so many integration tests, we're hitting this failure very very frequently now:

Error response from daemon: driver failed programming external connectivity on endpoint ftl-db-1 (77de40e71a5a9b79309209ca12abdb5f67ca1faefcab7fce30ceaa2891f560e0): Error starting userland proxy: listen tcp4 0.0.0.0:54320: bind: address already in use

It doesn't seem to be an issue with FTL itself, but it would be great to fix it because it's really annoying.
@github-actions github-actions bot added the triage Issue needs triaging label Jun 10, 2024
@ftl-robot ftl-robot mentioned this issue Jun 10, 2024
Open
@alecthomas alecthomas added bug Something isn't working next Work that will be be picked up next and removed triage Issue needs triaging labels Jun 10, 2024
@gak
Copy link
Contributor

gak commented Jun 11, 2024 •
edited
Loading

Examples

https://github.com/TBD54566975/ftl/actions/runs/9460605277/job/26059731611#step:5:564
https://github.com/TBD54566975/ftl/actions/runs/9457427212/job/26051227590#step:5:585
https://github.com/TBD54566975/ftl/actions/runs/9457388106/job/26051102482#step:5:597
https://github.com/TBD54566975/ftl/actions/runs/9444996323/job/26011687680#step:5:573
https://github.com/TBD54566975/ftl/actions/runs/9475147527/job/26105983192?pr=1743#step:7:577

Seen on "Integration Test" matrix and "Test Go"

@github-actions github-actions bot removed the next Work that will be be picked up next label Jun 11, 2024
@alecthomas
Copy link
Collaborator Author

@gak volunteered as tribute

@gak
Copy link
Contributor

gak commented Jun 12, 2024

  • The postgres port is always triggering the error.
  • The port is not used immediately beforehand.

@gak
Copy link
Contributor

gak commented Jun 12, 2024

tcp6 0 0 10.1.0.110:54320 13.107.42.16:443 ESTABLISHED 592/provisioner

@gak
Copy link
Contributor

gak commented Jun 12, 2024

Processes are randomly choosing high ports as the source port, randomly hitting 54320. Just need to choose a lower port under that range.

@gak gak mentioned this issue Jun 12, 2024
Merged
alecthomas pushed a commit that referenced this issue Jun 12, 2024
@gak
fix: port already in use investigation (#1746)
b7f7554 
Fixes: #1722

Port was in the random local range of ports which was sometimes being
used when trying to bind.

The solution is to use a port outside the random range (<32767), so I
picked 15432 instead of 54320.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gak gak

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: port already in use investigation TBD54566975/ftl
2 participants
@gak @alecthomas