-
Notifications
You must be signed in to change notification settings - Fork 305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Low socket limit #104
Comments
Can you be more specific? Which version of Ruby are you talking about? For 1.8.x we used to increase FD_SIZE to 256. Ruby 1.9 does not suffer from that. https://github.com/oneclick/rubyinstaller/blob/master/config/ruby_installer.rb#L18-31 If you found that Ruby 1.9.2 or 1.9.3 suffered from that, perhaps your report is better suited to Ruby itself: Please confirm. |
Ruby 1.9.3 suffers from it, only for asynchronous IO and in any libraries that use functions Ruby core 1.9 now avoids such as select(2). So the FD_SIZE should still be set and much higher than 256 for any serious async application. |
Basically libraries such as eventmachine require it to be set otherwise they are limited to 64 sockets which is pretty meaningless and considering I've been doing production deployments using EM and thin on windows it is pretty scary. |
for 1.8.x it's set to 256 isn't it? |
A bit off-topic, but I'm curious on if/how the following relate to what you're doing, and what you think of Greg's patch from 9 months ago? |
Yes, however 256 is way too low. I've been hacking at Eventmachine for the last 2 days as I have application that is using ~150 sockets now and is intended to scale well beyond 1000 before the end of the year. Right now it is 3am here and I'm configuring a Linux VM to NAT with the windows host so I don't hit the 64 limit. I would prefer not to be doing this again in a couple of months after I've reached the 256 limit... There are 16383 ephemeral ports on windows (ie the probable max an application will ever use) can we use that? |
@jonforums The 64 is the number of sockets, not 64bits.. I wish it was bits. |
Of course I'm not entirely sure of ruby internals, if there is a windows file descriptor wrapper for every socket (not just a ruby facade) then the limiting number of sockets would be Windows limit of 2048 file descriptors: http://msdn.microsoft.com/en-us/library/6e3b887c(v=vs.71).aspx At least that limit is somewhat configurable at run time. |
@stakach I can't find the issue reported by @rdp on Ruby-Core about I see the value of the request, however, please see this on MSDN: http://msdn.microsoft.com/en-us/library/windows/desktop/ms739169(v=vs.85).aspx
But, further reading:
We can up that numbre to 32767, which seems to be the maximum supported by If you agree, I'll change 1.9 recipe so next build will handle that. |
@stakach hehe..yeh, I know the 64 number is the number of sockets. I asked about 64bits because some say they need 64bit (more than 4GB, etc) for "larger scale apps" as 32bit won't do. Seems like 32bit is fine for what you're doing, but I was interested in more details. |
@jonforums the larger scale applications we build are building automation systems: https://github.com/stakach/em_control We also make a digital signage system that is entirely HTML5 (management + playback). Local deployments of that usually run on windows as we have to do powerpoint conversions and offloading to windows is more infrastructure + cost people don't want to see. Some of those deployments will be nearing 64 displays very soon. |
@luislavena 32767 sounds great! |
@stakach how very cool...cloning your repo...OT, could https://github.com/adoxa/ansicon be helpful for your colorization needs? |
@jonforums you looking at my telnet hack? (I stopped using that) Otherwise everyone will have to clone the repo :) |
yes
just scanned your repo, but would a single distributable .exe containing your Ruby stuff be a deployment option for you? FWIW, Luis has adopted a clone of exerb project https://github.com/luislavena/exerb-mingw and he and another guy have made some updates for 1.9.3. I haven't had time to play with the mods yet, but exerb/1.8.7 has worked well for https://github.com/vertiginous/pik |
It could be and probably something we'd consider when we get this running in the cloud, for peoples local deployments. |
Good luck...no more from me as it's now OT to your issue and Luis has accepted the mod for the next release :) Please feel free to drop an ANN if/when it makes sense to http://groups.google.com/group/rubyinstaller since you're using RubyInstaller. And I'd also drop an ANN on http://www.ruby-forum.com/forum/ruby since many Rubyist's don't realize all the cool things being done with Ruby on Windows! |
Thanks mate, will do! |
isn't the max number limited by msvcrt.dll? if so what's the max actually usable maybe we should use that number (of course, that number might vary based on whether you're on a server or standard windows box...) |
That's just the c run time. Technically socket limits are purely memory based, except for this particular case where we want to use this particular function for async io. Windows imposes limits on IIS connections and network shares however they are all higher level. |
http://www.squid-cache.org/mail-archive/squid-users/200601/0665.html is where I "guess" I got my info. |
That limit does exist. I mention it in my 6th comment. |
Interesting. (oh BTW ruby core in 1.9 uses select(2) if you call IO.select doesn't it?) Anyway, with Ruby sockets per se (like require 'socket'; TCPSocket.new) I think it creates "a windows file descriptor wrapper for every socket". I don't know if EM does or not. It'd be interesting to see what the theoretical limit "actually is" with EM or the like, given a high FD_SETSIZE. The only drawback to a large FD_SETSIZE is that the whole set is put on the stack, but since 1.9 has individual threads anyway now, it probably wouldn't matter if it were large I guess.... |
Yeah it would be good to know, if I start getting near the 2048 boundary I'd be playing it cautious. |
@stakach I've committed a fix, and will be available in the next patchlevel release of Ruby 1.9.3. In the meantime, if you want to test, you can build locally by cloning RubyInstaller repository and doing Really cool the em_control project. Will be interesting when you're done if you can do a formal announcement in RubyInstaller mailing list: http://groups.google.com/group/rubyinstaller/ Cheers. |
libffi didn't compile properly. Giving it another go. |
It was a path with a space. All good now |
Ok the FD_SETSIZE size we set clashed with event machine (which defines a smaller number) causing a seg fault. So the next release of eventmachine and ruby installer must always match! Previously eventmachine was higher than ruby and it created so much weirdness! I'm creating an issue to let the eventmachine guys know. |
Also, thanks everyone for your help! Couldn't have got this working without you! |
@stakach please link the issue of EventMachine with this using Cheers. |
Ok, the eventmachine one is: eventmachine/eventmachine#303 |
Hi @luislavena looks like Windows will never support more than 2048 (technically 2045 thanks to STD streams) and the limits on file handles. So we might as well drop the FD_SETSIZE before your next release to 2048 to save memory - the change won't effect eventmachine |
Where is the root of this assumption? Can you provide a link about this? Windows is not limited to 2k sockets, so would love to know where this Sorry for top posting. Sent from mobile.
|
It is a Microsoft limit on the standard c library: http://msdn.microsoft.com/en-us/library/6e3b887c%28vs.71%29.aspx It's easy to test: Server require 'rubygems'
require 'eventmachine'
module EchoServer
@@connected_clients = 0
def post_init
@@connected_clients += 1
p "there are now #{@@connected_clients} connected"
end
def receive_data data
send_data ">>>you sent: #{data}"
close_connection if data =~ /quit/i
end
end
EventMachine::run {
EventMachine::start_server "127.0.0.1", 8081, EchoServer
} Client require 'rubygems'
require 'eventmachine'
class Echo < EventMachine::Connection
def post_init
@timer = EventMachine::PeriodicTimer.new(5) do
send_data 'Hello'
end
send_data 'Hello'
end
def receive_data(data)
p 'recieved:' + data
end
def unbind
p ' connection totally closed'
@timer.cancel
end
end
EventMachine.run {
(1..3000).each do |h|
EventMachine.connect '127.0.0.1', 8081, Echo
end
} |
@luislavena ping |
Sorry, I'm with limited Internet access right now. Will check tomorrow morning on my computer. Sorry for top posting. Sent from mobile.
|
@stakach I haven't tested this example yet, but I don't think Ruby is using open methods on the sockets mapped to fd's, or it does? |
I have a feeling it is the file descriptor limit being hit. |
Running the example I get this error on
Here is the It looks like |
Ruby associates windows socket to a C run-time file descriptor by _open_osfhandle(). https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L3486 So that socket size seems limited to file descriptor max (2048). |
Thank you @shirosaki So we conclude that having Sometimes I wish Ruby did IO/socket implementation differently. Anyhow, going to change the limit, again. Thanks @stakach |
Hi Everyone, and sorry for the headache here, but I'm facing almost the same problem is my production server, I would really appreciate any help regarding this issue, I'm running my system with Ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux] , at OS Ubuntu 12.04.1 LTS and em-http-request (1.0.3). I have a ruby processes that consume msgs from a RabbitMQ queue using amqp gem, something like this :
Now the URL is slow, and the queue has so much messages ( > 30K ), I got this error in the log: Any help would be highly appreciated since I've been trying all my time figuring out how to solve it but with no results at all. Thanks in advance |
@cubiic have you tried to increase the open file descriptor maximum? such as: |
@shirosaki I really appreciate your welling to help, but guide me more here since I'm not an expert with this, please check out what I've been trying to do : https://gist.github.com/cubiic/f7c8b1ca3f56ac9fd73b As you can see I've been trying to do that unsuccessfully, Thanks in advance |
@cubiic while interesting, this linux issue has nothing to do with rubyinstaller or ruby on windows. Worse, the documentation for any fix will be buried in this thread and very difficult to for a linux user to find. That's a shame, and a waste of your and Shirosaki-san's efforts. The EM list, or the ruby-talk list (or even stackoverflow) are much better places to work/document the problem. |
Sorry man , but I though the problem is relevant based on the return error i'm getting, if you don't feel so I can just close it, but I found "unable to create new socket: Too many open files" error relevant by googling it |
It's related, but irrelevant on this project's issue list. But don't miss my main point: I don't want the doco for a fix getting "buried" here. Switch the conversation (with an fyi link back here if you like) to a more linux visible place, like ruby-talk so we linux users can better help/benefit. |
@cubiic If more guide is needed, please switch the place as Jon stated. |
Thanks a lot for the help here, |
Windows defines a default of 64 asynchronous sockets per application as defined by FD_SETSIZE
See: http://support.microsoft.com/kb/111855
This limits windows as a platform for running large scale applications such as eventmachine.
Could we please set FD_SETSIZE to a considerably large limit in the makefile, say 99999, to allow for larger scale applications. Currently it is quite useless.
The text was updated successfully, but these errors were encountered: