Add support for listening on Unix socket #594

messense · 2018-03-28T05:42:06Z

TODO:

Safely delete existing socket on startup
Emit error when both address and port are specified(But what if I want to listen on both tcp and unix sockets?)
TLS over unix socket

Closes #545

messense · 2018-03-28T05:46:06Z

I am using the hello_world example to test it, added a Rocket.toml file with address set to unix:///tmp/hello.sock in development section, run cargo run to start the server, then use curl to send a simple request to it:

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `/Users/messense/Projects/Rocket/target/debug/hello_world`
🔧  Configured for development.
    => address: unix:///tmp/hello.sock
    => port: 8000
    => log: normal
    => workers: 1
    => secret key: generated
    => limits: forms = 32KiB, json* = 1MiB, msgpack* = 1MiB
    => keep-alive: 5s
    => tls: disabled
    => [extra] hi: "Hello!"
    => [extra] is_extra: true
🛰  Mounting '/':
    => GET / (hello)
🚀  Rocket has launched from http://unix:///tmp/hello.sock:0

$ curl --unix-socket /tmp/hello.sock http://localhost// -v
*   Trying /tmp/hello.sock...
* Connected to localhost (/tmp/hello.sock) port 80 (#0)
> GET // HTTP/1.1
> Host: localhost
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Server: Rocket
< Content-Length: 13
< Date: Wed, 28 Mar 2018 05:40:28 GMT
<
* Connection #0 to host localhost left intact
Hello, world!

messense · 2018-03-28T05:48:09Z

lib/src/data/net_stream.rs

@@ -16,6 +17,8 @@ pub enum NetStream {
    Http(HttpStream),
    #[cfg(feature = "tls")]
    Https(HttpsStream),
+    #[cfg(unix)]
+    UnixHttp(UnixSocketStream),


I don't like this, but it is needed to make concrete_stream work.

Yes, it's all very frustrating.

SergioBenitez

How does this deal with sockets that don't already exist? What about those that do? I'd like for configuration to fail if a socket already exists and for Rocket to create the socket if it doesn't.

lib/src/rocket.rs

SergioBenitez · 2018-03-28T12:33:45Z

lib/src/data/net_stream.rs

@@ -16,6 +17,8 @@ pub enum NetStream {
    Http(HttpStream),
    #[cfg(feature = "tls")]
    Https(HttpsStream),
+    #[cfg(unix)]
+    UnixHttp(UnixSocketStream),


Yes, it's all very frustrating.

lib/src/config/config.rs

lib/src/data/data.rs

lib/src/rocket.rs

lib/src/unix_socket.rs

SergioBenitez · 2018-03-28T12:43:24Z

Awesome, @messense! This is really great. Excited to get this in. :)

messense · 2018-03-28T13:08:27Z

How does this deal with sockets that don't already exist?

It will create a new one.

if the socket file exists, it crashes with Address already in use error.

     Running `/Users/messense/Projects/Rocket/target/debug/hello_world`
🔧  Configured for development.
    => address: unix:///tmp/hello.sock
    => port: 8000
    => log: normal
    => workers: 1
    => secret key: generated
    => limits: forms = 32KiB, json* = 1MiB, msgpack* = 1MiB
    => keep-alive: 5s
    => tls: disabled
    => [extra] hi: "Hello!"
    => [extra] is_extra: true
🛰  Mounting '/':
    => GET / (hello)
Error: Rocket failed to launch due to an I/O error.
thread 'main' panicked at 'Address already in use (os error 48)', lib/src/error.rs:207:17
note: Run with `RUST_BACKTRACE=1` for a backtrace.

I think it's the exact behavior of std::unix::net::UnixListener::bind method.

messense · 2018-03-28T13:18:32Z

Thanks for the detailed review, I hope to address them soon.

lib/src/config/config.rs

lib/src/config/error.rs

lib/src/config/builder.rs

lib/src/lib.rs

SergioBenitez

I'd like more code that depends on whether we're on Unix or not to be contained in the unix module. The cfgs sprinkled around make it harder to follow what the intended flow is.

lib/src/config/address.rs

lib/src/config/builder.rs

lib/src/config/address.rs

lib/src/config/config.rs

lib/src/config/error.rs

lib/src/lib.rs

lib/src/rocket.rs

lib/src/config/config.rs

lib/src/rocket.rs

messense · 2019-07-08T15:40:35Z

Unix domain socket support for Windows is added in messense@47c9dea

Tested on Windows 10 version 1903

core/http/src/lib.rs

SergioBenitez · 2019-07-08T20:46:04Z

What happens if you try to use UDS on a unsupported version of Windows? Does Rocket still compile on unsupported Windows versions? Is it simply a runtime error? Which error message do we print if so?

messense · 2019-07-09T00:25:20Z

Windows Unix domain socket is created with WSASocketW API using AF_UNIX address family, WSASocketW is an existing Windows API and AF_UNIX is just a int in winapi crate. Rocket should compile fine on unsupported Windows versions but return WSAEAFNOSUPPORT meaning that The specified address family is not supported.

Which error message do we print if so?

~~I don't have an older Windows system so I can't easily verify that. Help wanted.~~ See #594 (comment)

messense · 2019-07-09T03:53:34Z

Side note: curl on Windows seems not supporting unix socket yet, but you can use curl in WSL to access a Windows Unix socket for example: curl --unix-socket /mnt/c/ProgramData/hello.sock http://localhost// -v. Really cool.

messense · 2019-07-13T06:30:58Z

I've created a Windows Server 2016 on Azure which does not support UDS, it failed to launch with error message:

Error: Rocket failed to bind network socket to given address/port.
thread 'main' panicked at 'An address incompatible with the requested protocol was used. (os error 10047)', core\lib\src\error.rs:197:17

What do you think? @SergioBenitez @jebrosen

jebrosen · 2019-07-14T00:50:44Z

That looks reasonable. We could detect that error and print a custom message but it might not be worth the effort. We can note in the documentation that this only works on Windows 1809 or later.

…ows platforms

messense · 2019-07-14T02:39:48Z

We could detect that error and print a custom message but it might not be worth the effort.

Added the detection code here: 3c9e579

cyberb · 2019-09-26T20:27:55Z

Do you plan to merge this?

jebrosen · 2019-09-27T03:30:20Z

I need to review this again, but I'm generally in favor of this. If it's backwards-compatible we could do it in 0.4, but I suspect the changes to Address aren't.

For 0.5 the socket and connection code will need to be rewritten, but it can probably be done somewhat more simply. Furthermore, I would like to make the connection handling extensible such that one could implement unix socket handling, FastCGI, and other I/O all without modifying rocket.

luciusmagn · 2020-02-16T16:28:09Z

Hello, what's the status on this? I'd love to use Rocket over unix socket.

Thank you

kotovalexarian · 2020-10-17T07:27:44Z

Furthermore, I would like to make the connection handling extensible such that one could implement unix socket handling, FastCGI, and other I/O all without modifying rocket.

In Ruby ecosystem this is done by separating common HTTP server interface (Rack), HTTP server implementations (Puma, Unicorn, etc) and HTTP applications (which may use Ruby on Rails, Sinatra, etc, or no frameworks at all). In other words, every HTTP application is a Rack application, every HTTP server is a Rack server, every HTTP middleware is a Rack middleware. Everything is built around Rack, which is a very thin and common interface.

vanackere · 2020-12-13T18:20:49Z

@jebrosen , @SergioBenitez : what is the current status and how would you prefer to have unix-socket support implemented ?
I also have the need for unix socket support for Rocket and wouldn't mind helping if possible...

jebrosen · 2020-12-17T04:55:09Z

#594 (comment) is still accurate. Unix socket support requires roughly these steps:

The configuration system needs to support specifying a unix socket address. In master, the configuration system has been revamped so this will need some tweaks.
The I/O layer needs to support it. In 0.5, this should now only require an impl Listener for tokio::net::UnixListener similar to impl Listener for tokio::net::TcpListener (https://github.com/SergioBenitez/Rocket/blob/master/core/http/src/listener.rs#L164-L180).
When the configured address is a unix socket, rocket must set it up (e.g. deleting a stale socket file) and start listening. This can work largely the same as in this PR, but some of the files have been renamed or moved.

At this point I don't believe there are any more significant changes planned for those areas, and I'd be happy to work on and/or review this soon.

@SergioBenitez, any preferences for timing this breaking change to configuration or handling it in an non-breaking way? As far as I can see, the other two changes could be done separately and even in a minor release.

cyberb · 2021-03-04T21:42:24Z

I know open-source people hate this, but can I offer $50 (I do not have a pound sterling symbol on my keyboard) for this?

SergioBenitez · 2021-06-30T19:51:42Z

This is now wholly outdated. On the one hand, prioritizing this earlier would have prevented staleness. On the other, there are still unresolved questions, and the implementation now would look nothing like the implementation then.

Nevertheless, I've implemented support for a Unix domain socket listener in a local domain branch. The holdback is configuration: how do we configure which listener to use?

There's #1070, which if implemented, would let us say "well, do so programatically." But this isn't particularly satisfying. Starting a Rocket application that listens on a UDS should be as simple as:

ROCKET_ADDRESS="unix:/path/to/socket" cargo run

This is nice and tenable, but the problem is that it ignores the existence of the port configuration parameter. What happens to it if the address is a UDS? The approaches to consider are:

Ignore it. Allow setting a port, but if the address type doesn't use a port, simply don't use it.
Error. If a non-default port is provided for an address type that doesn't make use of it, emit an error. This seems difficult to implement correctly.
Remove port entirely and make it a part of address. So what is today ROCKET_PORT=9000 cargo run would become ROCKET_ADDRESS=127.0.0.1:9000 cargo run. This feels like the "most correct" solution but 1) will almost certainly break almost every deployed Rocket application, and 2) makes changing just the port impossible.

I'm inclined to suggest we follow 1.

In any case, I'm closing out this particular PR as the implementation here has no path forward given the divergence upstream.

messense commented Mar 28, 2018

View reviewed changes

messense mentioned this pull request Mar 28, 2018

Support for listening on Unix sockets #545

Closed

messense force-pushed the unix-socket branch from bfb13ec to 9c26fd6 Compare March 28, 2018 08:58

SergioBenitez requested changes Mar 28, 2018

View reviewed changes

messense commented Mar 31, 2018

View reviewed changes

lib/src/config/config.rs Outdated Show resolved Hide resolved

messense force-pushed the unix-socket branch 6 times, most recently from cf53a31 to 4acd52a Compare April 3, 2018 05:54

messense commented Apr 3, 2018

View reviewed changes

lib/src/config/error.rs Outdated Show resolved Hide resolved

messense commented Apr 3, 2018

View reviewed changes

lib/src/config/builder.rs Outdated Show resolved Hide resolved

messense commented Apr 3, 2018

View reviewed changes

lib/src/lib.rs Outdated Show resolved Hide resolved

SergioBenitez requested changes Apr 3, 2018

View reviewed changes

messense force-pushed the unix-socket branch from 4acd52a to 09a96e8 Compare April 4, 2018 06:35

messense commented Apr 5, 2018

View reviewed changes

lib/src/config/config.rs Outdated Show resolved Hide resolved

messense commented Apr 5, 2018

View reviewed changes

lib/src/config/config.rs Outdated Show resolved Hide resolved

messense commented Apr 5, 2018

View reviewed changes

lib/src/config/config.rs Outdated Show resolved Hide resolved

messense commented Apr 5, 2018

View reviewed changes

lib/src/rocket.rs Outdated Show resolved Hide resolved

messense force-pushed the unix-socket branch 6 times, most recently from e62be17 to 466350e Compare April 6, 2018 07:51

messense force-pushed the unix-socket branch from 60783ac to 47c9dea Compare July 8, 2019 15:38

messense mentioned this pull request Jul 8, 2019

Add socket timeout options haraldh/rust_uds_windows#1

Merged

SergioBenitez reviewed Jul 8, 2019

View reviewed changes

core/http/src/lib.rs Show resolved Hide resolved

messense force-pushed the unix-socket branch from 47c9dea to d935c79 Compare July 9, 2019 02:32

messense force-pushed the unix-socket branch from d935c79 to 8f80cbe Compare July 9, 2019 05:03

messense added 2 commits July 14, 2019 10:11

Add support for listening on Unix socket.

4e905e8

Add Unix domain socket support for Windows

d6fa54a

messense force-pushed the unix-socket branch from 8f80cbe to d6fa54a Compare July 14, 2019 02:15

Add a friendly note when using Unix domain socket on unsupported Wind…

3c9e579

…ows platforms

jebrosen self-requested a review July 15, 2019 02:29

jebrosen mentioned this pull request Aug 2, 2019

Provide a lower-level connection interface #1070

Closed

cyberb mentioned this pull request Sep 18, 2019

API and web integration pi-hole/pi-hole#2692

Closed

13 tasks

SergioBenitez force-pushed the master branch 2 times, most recently from 74113c3 to 95c981d Compare February 15, 2020 12:02

SergioBenitez closed this Jun 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for listening on Unix socket #594

Add support for listening on Unix socket #594

messense commented Mar 28, 2018 •

edited

Loading

messense commented Mar 28, 2018

messense Mar 28, 2018

SergioBenitez Mar 28, 2018

SergioBenitez left a comment

SergioBenitez Mar 28, 2018

SergioBenitez commented Mar 28, 2018

messense commented Mar 28, 2018 •

edited

Loading

messense commented Mar 28, 2018 •

edited

Loading

SergioBenitez left a comment

messense commented Jul 8, 2019

SergioBenitez commented Jul 8, 2019 •

edited

Loading

messense commented Jul 9, 2019 •

edited

Loading

messense commented Jul 9, 2019

messense commented Jul 13, 2019

jebrosen commented Jul 14, 2019

messense commented Jul 14, 2019

cyberb commented Sep 26, 2019

jebrosen commented Sep 27, 2019

luciusmagn commented Feb 16, 2020

kotovalexarian commented Oct 17, 2020 •

edited

Loading

vanackere commented Dec 13, 2020

jebrosen commented Dec 17, 2020

cyberb commented Mar 4, 2021

SergioBenitez commented Jun 30, 2021 •

edited

Loading

Add support for listening on Unix socket #594

Add support for listening on Unix socket #594

Conversation

messense commented Mar 28, 2018 • edited Loading

messense commented Mar 28, 2018

messense Mar 28, 2018

Choose a reason for hiding this comment

SergioBenitez Mar 28, 2018

Choose a reason for hiding this comment

SergioBenitez left a comment

Choose a reason for hiding this comment

SergioBenitez Mar 28, 2018

Choose a reason for hiding this comment

SergioBenitez commented Mar 28, 2018

messense commented Mar 28, 2018 • edited Loading

messense commented Mar 28, 2018 • edited Loading

SergioBenitez left a comment

Choose a reason for hiding this comment

messense commented Jul 8, 2019

SergioBenitez commented Jul 8, 2019 • edited Loading

messense commented Jul 9, 2019 • edited Loading

messense commented Jul 9, 2019

messense commented Jul 13, 2019

jebrosen commented Jul 14, 2019

messense commented Jul 14, 2019

cyberb commented Sep 26, 2019

jebrosen commented Sep 27, 2019

luciusmagn commented Feb 16, 2020

kotovalexarian commented Oct 17, 2020 • edited Loading

vanackere commented Dec 13, 2020

jebrosen commented Dec 17, 2020

cyberb commented Mar 4, 2021

SergioBenitez commented Jun 30, 2021 • edited Loading

messense commented Mar 28, 2018 •

edited

Loading

messense commented Mar 28, 2018 •

edited

Loading

messense commented Mar 28, 2018 •

edited

Loading

SergioBenitez commented Jul 8, 2019 •

edited

Loading

messense commented Jul 9, 2019 •

edited

Loading

kotovalexarian commented Oct 17, 2020 •

edited

Loading

SergioBenitez commented Jun 30, 2021 •

edited

Loading