Socket error when starting Synapse - Cannot bind - No such file or directory #1054

MadLittleMods · 2021-05-28T06:45:50Z

When trying to run Sytest against Synapse, I am running into the following error socket bind error, Error starting server-0: Cannot bind - No such file or directory @details=bind IO::Socket::UNIX=GLOB(0x7feab5d1df20) #sytest-JXOAUKDFIBVQMRPNAJEO.sock No such file or directory at tests/05homeserver.pl line 128..

Which probably corresponds to lib/SyTest/Homeserver/ProcessManager.pm#L355-L361

$ ./run-tests.pl tests/31sync/01filter.pl
Running tests/31sync/01filter.pl...
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Started test HTTPS Server at localhost:56633
 # Started test SMTP Server at localhost:56634
 # Starting Homeserver using SyTest::HomeserverFactory::Synapse=HASH(0x7fdf0680a990)
 # Starting server-0
 # Using DB config from /Users/eric/Documents/github/element/sytest/server-0/database.yaml
 # Clearing SQLite database at :memory:
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Generating config for port 8800
 # Creating config for server 0 with command python -m synapse.app.homeserver --config-path /Users/eric/Documents/github/element/sytest/server-0/config.yaml --generate-config --report-stats=no --server-name localhost:8800
 # [python]: Config file '/Users/eric/Documents/github/element/sytest/server-0/config.yaml' already exists. Generating any missing config files.
 # Starting server 0 for port 8800
  Testing if: Can create filter...
Tests: 0 / 2 | OK
 ** Error starting server-0: Cannot bind - No such file or directory @details=bind IO::Socket::UNIX=GLOB(0x7feab5d1df20) #sytest-JXOAUKDFIBVQMRPNAJEO.sock No such file or directory at tests/05homeserver.pl line 128.
  Testing if: Can create filter... FAIL:
  Testing if: Can download filter... SKIP due to lack of can_create_filter
 # Killing process server
 ** server process failed - code=2
 
1 tests FAILED

I think I have run into this problem in the past because I have debug logging around the sockets code in this PR, but I can't remember how I solved it. I know for sure I've run local Sytest against local Dendrite before.

Full run with my random debugs

 ./run-tests.pl tests/31sync/01filter.pl
Running tests/31sync/01filter.pl...
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Started test SMTP Server at localhost:61574
 # Started test HTTPS Server at localhost:61575
 # Starting Homeserver using SyTest::HomeserverFactory::Synapse=HASH(0x7feab480af90)
 # Starting server-0
 # Using DB config from /Users/eric/Documents/github/element/sytest/server-0/database.yaml
 # Clearing SQLite database at :memory:
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Generating config for port 8800
 # Creating config for server 0 with command python -m synapse.app.homeserver --config-path /Users/eric/Documents/github/element/sytest/server-0/config.yaml --generate-config --report-stats=no --server-name localhost:8800
 # [python]: Config file '/Users/eric/Documents/github/element/sytest/server-0/config.yaml' already exists. Generating any missing config files.
 # Starting server 0 for port 8800
  Testing if: Can create filter...
Tests: 0 / 2 | OKStarting synapse with command: python -m synapse.app.homeserver --config-path /Users/eric/Documents/github/element/sytest/server-0/config.yaml
New unix socket sytest-JXOAUKDFIBVQMRPNAJEO.sock
feawfefe
rdtnb
kuykuyyyukyukg
Binding unix socket sytest-JXOAUKDFIBVQMRPNAJEO.sock
jytjuytyjyjjy
Binding unix socket sytest-JXOAUKDFIBVQMRPNAJEO.sock
 ** Error starting server-0: Cannot bind - No such file or directory @details=bind IO::Socket::UNIX=GLOB(0x7feab5d1df20) #sytest-JXOAUKDFIBVQMRPNAJEO.sock No such file or directory at tests/05homeserver.pl line 128.
  Testing if: Can create filter... FAIL:
 | asdf at tests/05homeserver.pl line 130.
 | 	main::__ANON__("Cannot bind - No such file or directory", "bind", IO::Socket::UNIX=GLOB(0x7feab5d1df20), "#\x{1}\x{0}sytest-JXOAUKDFIBVQMRPNAJEO.sock", "No such file or directory") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 480
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab6ff0c18), "failed") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 687
 | 	Future::fail(IO::Async::Future=HASH(0x7feab6ff0c18), "Cannot bind - No such file or directory", "bind", IO::Socket::UNIX=GLOB(0x7feab5d1df20), "#\x{1}\x{0}sytest-JXOAUKDFIBVQMRPNAJEO.sock", "No such file or directory") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 853
 | 	Future::on_ready(IO::Async::Future=HASH(0x7feab6ff06c0), IO::Async::Future=HASH(0x7feab6ff0c18)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 471
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab6ff06c0), "wait_any") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 1905
 | 	Future::__ANON__(IO::Async::Future=HASH(0x7feab5e5fd38)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 480
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab5e5fd38), "failed") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 687
 | 	Future::fail(IO::Async::Future=HASH(0x7feab5e5fd38), "Cannot bind - No such file or directory", "bind", IO::Socket::UNIX=GLOB(0x7feab5d1df20), "#\x{1}\x{0}sytest-JXOAUKDFIBVQMRPNAJEO.sock", "No such file or directory") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 853
 | 	Future::on_ready(IO::Async::Future=HASH(0x7feab5d5e7a0), IO::Async::Future=HASH(0x7feab5e5fd38)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 471
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab5e08db0), "done") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 625
 | 	Future::done(IO::Async::Future=HASH(0x7feab5e08db0)) called at lib/SyTest/Homeserver.pm line 577
 | 	SyTest::Homeserver::__ANON__(IO::Async::Process=HASH(0x7feab5e32558), 0) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Notifier.pm line 837
 | 	IO::Async::Notifier::invoke_event(IO::Async::Process=HASH(0x7feab5e32558), "on_finish", 0) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Process.pm line 564
 | 	IO::Async::Process::__ANON__(undef) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 480
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab5e60578), "needs_all") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 2003
 | 	Future::__ANON__(IO::Async::Future=HASH(0x7feab5e32210)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 480
 | 	Future::_mark_ready(IO::Async::Future=HASH(0x7feab5e32210), "done") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 625
 | 	Future::done(IO::Async::Future=HASH(0x7feab5e32210)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Process.pm line 537
 | 	IO::Async::Process::__ANON__(undef, undef, undef, IO::Async::Process=HASH(0x7feab5e32558), 23116, 0, "", "") called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Internals/ChildManager.pm line 290
 | 	IO::Async::Internals::ChildManager::__ANON__(23116, 0) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop.pm line 2896
 | 	IO::Async::Loop::_reap_children(HASH(0x7feab4a05ea8)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop.pm line 2961
 | 	IO::Async::Loop::__ANON__() called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop.pm line 818
 | 	IO::Async::Loop::__ANON__() called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/OS.pm line 571
 | 	IO::Async::OS::_Base::__ANON__(IO::Async::Handle=HASH(0x7feab53ed8e0)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop/Poll.pm line 172
 | 	IO::Async::Loop::Poll::post_poll(IO::Async::Loop::Poll=HASH(0x7feab4a239c0)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop/Poll.pm line 292
 | 	IO::Async::Loop::Poll::loop_once(IO::Async::Loop::Poll=HASH(0x7feab4a239c0)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Loop.pm line 654
 | 	IO::Async::Loop::await(IO::Async::Loop::Poll=HASH(0x7feab4a239c0), IO::Async::Future=HASH(0x7feab5d1d9b0)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/IO/Async/Future.pm line 101
 | 	IO::Async::Future::await(IO::Async::Future=HASH(0x7feab5d1d9b0)) called at /Users/eric/perl5/perlbrew/perls/perl-5.32.0/lib/site_perl/5.32.0/Future.pm line 929
 | 	Future::get(IO::Async::Future=HASH(0x7feab5d1d9b0)) called at ./run-tests.pl line 763
 | 	eval {...} called at ./run-tests.pl line 710
 | 	main::_run_test0(SyTest::Output::Term::Test=HASH(0x7feab69f1d20), main::Test=ARRAY(0x7feab6e6b888), ARRAY(0x7feab70dfb40)) called at ./run-tests.pl line 674
 | 	main::_run_test(SyTest::Output::Term::Test=HASH(0x7feab69f1d20), main::Test=ARRAY(0x7feab6e6b888)) called at ./run-tests.pl line 897
 +----------------------
  Testing if: Can download filter... SKIP due to lack of can_create_filter
 # Killing process server
 ** server process failed - code=2

1 tests FAILED

Just creating this issue so I can track the solution and reference it in the future if I figure it out.

Environment:

macOS 11.1
Running inside my Synapse virtualenv
Python 3.8.6
perl 5, version 32

The text was updated successfully, but these errors were encountered:

richvdh · 2021-06-02T15:39:55Z

this is probably something to do with unix sockets not working on mac os?

richvdh · 2021-06-02T15:53:43Z

or, specifically, the "abstract" namespace for unix sockets. Try changing my $path = "\0sytest-$random_id.sock"; in ProcessManager.pm to omit the \0.

MadLittleMods · 2021-06-03T06:49:15Z

@richvdh Woot! That's it! Thank you 🐳

And now looking back at my history, I can see that I searched for all of this sort of stuff back on 2020-12-30:

Cannot bind - No such file or directory at unix sock macos
abstract socket cannot bind perl
perl socket connect bind
perldoc "IO::Async::Socket" bind
perl socket bind abstract namespace
perl socket bind no such file or directory
perl socket bind null byte "\0"
perl socket bind null byte
perl socket bind null terminator
perl abstract socket bind no such file or directory
"IO::Socket" "Cannot bind"
"IO::Socket::UNIX" "Cannot bind"
What does the @ symbol denote in the beginning of a unix domain socket path in Linux?, https://unix.stackexchange.com/a/206395/52320
mac os x - UNIX domain sockets in abstract namespace, https://apple.stackexchange.com/a/157509/164173
https://stackoverflow.com/q/16183794/796832

Now running successfully:

$ ./run-tests.pl tests/31sync/01filter.pl
Running tests/31sync/01filter.pl...
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Started test HTTPS Server at localhost:52469
 # Started test SMTP Server at localhost:52474
 # Starting Homeserver using SyTest::HomeserverFactory::Synapse=HASH(0x7fb34580af90)
 # Starting server-0
 # Using DB config from /Users/eric/Documents/github/element/sytest/server-0/database.yaml
 # Clearing SQLite database at :memory:
  Testing if: Can create filter...
Tests: 0 / 2 | OKSignature ok
subject=/CN=localhost
 # Generating config for port 8800
 # Creating config for server 0 with command python -m synapse.app.homeserver --config-path /Users/eric/Documents/github/element/sytest/server-0/config.yaml --generate-config --report-stats=no --server-name localhost:8800
 # [python]: Config file '/Users/eric/Documents/github/element/sytest/server-0/config.yaml' already exists. Generating any missing config files.
 # Starting server 0 for port 8800
 # Started synapse 0
 # Started server-0
  Testing if: Can create filter... PASS
  Testing if: Can download filter... PASS
 # Killing process server

All tests PASSED (with 2 passes)

MadLittleMods · 2021-06-03T07:08:46Z

In order to make the codebase compatible with macOS out of the box, could we drop the abstract namespace?

Otherwise some platform specific code to fallback ($^O)?

richvdh · 2021-06-03T12:57:15Z

happy to drop the abstract namespace if you can find a good place for the socket file to live.

Fix #1054 Abstract namespaced sockets are only supported on Linux so moving the socket to a real temporary file. - https://apple.stackexchange.com/a/157509/164173 - "abstract socket name, supported only on Linux", https://docs.racket-lang.org/unix-socket/index.html

This was referenced May 28, 2021

Add parse_strings_from_args to get prev_events array matrix-org/synapse#10048

Merged

Add depth parameter and pipe through the event creation methods matrix-org/synapse#10049

Closed

MadLittleMods added the T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. label Jun 3, 2021

MadLittleMods mentioned this issue Jun 18, 2021

Fix socket binding on macOS #1058

Merged

MadLittleMods self-assigned this Jun 18, 2021

richvdh closed this as completed in #1058 Jun 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Socket error when starting Synapse - Cannot bind - No such file or directory #1054

Socket error when starting Synapse - Cannot bind - No such file or directory #1054

MadLittleMods commented May 28, 2021 •

edited

Loading

richvdh commented Jun 2, 2021

richvdh commented Jun 2, 2021

MadLittleMods commented Jun 3, 2021 •

edited

Loading

MadLittleMods commented Jun 3, 2021

richvdh commented Jun 3, 2021

Socket error when starting Synapse - Cannot bind - No such file or directory #1054

Socket error when starting Synapse - Cannot bind - No such file or directory #1054

Comments

MadLittleMods commented May 28, 2021 • edited Loading

richvdh commented Jun 2, 2021

richvdh commented Jun 2, 2021

MadLittleMods commented Jun 3, 2021 • edited Loading

MadLittleMods commented Jun 3, 2021

richvdh commented Jun 3, 2021

MadLittleMods commented May 28, 2021 •

edited

Loading

MadLittleMods commented Jun 3, 2021 •

edited

Loading