Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File descriptors to log buffers are not released after publications are shutdown. #339

Closed
bedding opened this issue Apr 12, 2017 · 6 comments

Comments

@bedding
Copy link

bedding commented Apr 12, 2017

I ran into the same issue on this ticket: #329 a few days ago. I updated to the 1.2.4, and now the space are freed but the file descriptors are still held by the aeron client process.

In my case, I need to add and remove publications/subscriptions occasionally and I would like the code to run almost 24/7 without restarting. Linux has a limitation on the number of files that a process can open, which is 65535 on my machine. Since the aeron client code never releases the file descriptor even the log buffer itself is deleted, the process will eventually run out of fds, which will further lead to the pretty bad situation where no more files can be opened, no publication can be added and you can even open a socket.

@mjpt777
Copy link
Contributor

mjpt777 commented Apr 12, 2017

Can you check with Aeron 1.2.5 which has just been released?

@tmontgomery
Copy link
Contributor

tmontgomery commented Apr 12, 2017

I'm not able to see this with either a single publication or a single subscription. lsof reports no FDs or mappings in use after either a Publication.close or an UnavailableImage.

What is the pattern of usage for publications and subscriptions you have?

Or do you have something simple that reproduces this?

Update: is this C++ or Java?

@bedding
Copy link
Author

bedding commented Apr 12, 2017

The sub/pub pattern is like,

class A
{
    std::shared_ptr<aeron::Publication> pub;
    std::shared_ptr<aeron::Subscription> sub1; // subscribe to B::pub1
    std::shared_ptr<aeron::Subscription> sub2; // subscribe to B::pub2
};

class B
{
    std::shared_ptr<aeron::Subscription> sub;
    std::shared_ptr<aeron::Publication> pub1;
    std::shared_ptr<aeron::Publication> pub2;
};

Instances of class A and B are created/deleted occasionally, but they are always deleted as a pair

@tmontgomery
Copy link
Contributor

mapExisting for C++ did not have a close of the FD. Give the latest a shot (not 1.2.5 unfortunately as this change happened after the release) and see if the FDs are around or not.

@bedding
Copy link
Author

bedding commented Apr 17, 2017

The above commit did fix it for me. Thanks a lot!

@mjpt777 mjpt777 closed this as completed Apr 17, 2017
@sthornington
Copy link
Contributor

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants