Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close connections when vhost is unavailable (supervision tree is down) #1293

Merged
merged 8 commits into from
Jul 25, 2017

Conversation

hairyhum
Copy link
Contributor

Follow up to #1158

When vhost is down, we close all the connections to the vhost on the node.
When a client tries to connect to the vhost on the node - connection is refused with internal_error.

There are changes in supervision tree, which makes sure that vhost restarts are performed in order without concurrent message store processes running on the same vhosts.

Recovery is performed in the vhost_process process, which is former vhost_sup_watcher.
On termination the vhost process will notify that vhost is down, so the vhost connections can be closed.

A vhost "aliveness" status can be checked by checking aliveness of the vhost process.

@michaelklishin michaelklishin changed the title Close connections when vhost is down Close connections when vhost is unavailable (supervision tree is down) Jul 18, 2017
@hairyhum hairyhum force-pushed the rabbitmq-server-close-connection-on-vhost-down branch from 23366e8 to 9138a48 Compare July 19, 2017 09:07
Daniil Fedotov added 3 commits July 19, 2017 10:07
In order to be aware if vhost is alive or not, introduce
a rabbit_vhost_process gen_server, which manages recovery and
teardown of a vhost.
Also aliveness of the process can be used to determine a vhost state.
Vhost process termination emits an event to close all the vhost connections.
Addresses [#145106713]
If a vhost supervision tree is not active, which can be
a result of an error in message store, refuse connections
to this vhost on the node.
This is a follow-up to [#140841611] [#1158]

[#145106713]
Move non-limit tests from vhost limits suite to vhost suite.
Add tests for connection closure and refuse when vhost
supervisor is down.

[#145106713]
@hairyhum hairyhum force-pushed the rabbitmq-server-close-connection-on-vhost-down branch from 9138a48 to 049c902 Compare July 19, 2017 09:07
When a node is starting or restarting it can fail to recover a vhost.
The `continue` vhost recovery strategy means that vhosts can ba down if cannot
be recovered. This behaviour should also work for node startup and it should
skip failing vhosts.

%% The process will also check periodically if the vhost still
%% present in mnesia DB and stop the vhost supervision tree when it
%% desapears.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spelling

@kjnilsson kjnilsson merged commit 6115475 into master Jul 25, 2017
@dumbbell dumbbell deleted the rabbitmq-server-close-connection-on-vhost-down branch January 2, 2018 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants