socket limitations question
in [TCL]
|
Prev: Announce: Runtime Library 3.0 Gestalt Items 1.1 and Hugelist 1.4,Tcl-Mode for XCode
Next: http::geturl intermittent hang - looks for ideas...
From: keithv on 13 Oct 2009 19:50 This is probably more an OS question than a tcl one, but does anybody know the how many sockets one application can listen on? The design I inherited (but is not yet finalized) is to monitor a large number of idle devices, all with a 2-way connection via sockets. I was hoping to use tcl but I need to know how large it can scale. Are we talking hundreds, thousands, or what? The OS in question is some kind of Linux. Thanks, Keith
From: tom.rmadilo on 13 Oct 2009 22:07 On Oct 13, 4:50 pm, keithv <kvet...(a)gmail.com> wrote: > This is probably more an OS question than > a tcl one, but does anybody know the how > many sockets one application can listen > on? > > The design I inherited (but is not yet > finalized) is to monitor a large number > of idle devices, all with a 2-way connection > via sockets. I was hoping to use tcl but > I need to know how large it can scale. Are > we talking hundreds, thousands, or what? > > The OS in question is some kind of Linux. It is common practice to choose a one port (HTTP usually uses 80, 8000, etc). You can also have more than one interface on a physical server, so the special ip address 0.0.0.0 is shorthand for all available addresses on the machine. A server listens on a fixed number of well known ports, but when a client connections is accepted, a new port is used to handle the connection. The maximum number of these client connections is adjustable, but there are only like 64k ports, and those below 1024 are reserved. Tcl itself doesn't impose any additional limit. Usually applications are ignorant of these details.
From: Alexandre Ferrieux on 14 Oct 2009 03:41 On Oct 14, 1:50 am, keithv <kvet...(a)gmail.com> wrote: > This is probably more an OS question than > a tcl one, but does anybody know the how > many sockets one application can listen > on? > > The design I inherited (but is not yet > finalized) is to monitor a large number > of idle devices, all with a 2-way connection > via sockets. I was hoping to use tcl but > I need to know how large it can scale. Are > we talking hundreds, thousands, or what? > > The OS in question is some kind of Linux. The obvious bottleneck is the max number of allowed file descriptors per process, which is typically 1024 but can be pushed up by sysctl. However, as a recent discussion on tclcore showed, there is another one that kicks in, and that is not configurable at runtime: FD_SETSIZE, which is the static size of the bit arrays passed to select (). IIRC it is 1024 for glibc. You can look up this discussion on the tclcore archive, you'll learn about the plans to get over this limitation. In short, the select() API is inherently inefficient for larger numbers, that's why you won't find a libc with a much bigger FD_SETSIZE anyway. Hence epoll() seems the way to go when available. The only problem is that pending universal availability of epoll, the Tcl core will need to maintain an ifdef'ed pair of implementations. Eeek. -Alex
From: Donal K. Fellows on 14 Oct 2009 09:30 On 14 Oct, 01:41, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> wrote: > However, as a recent discussion on tclcore showed, there is another > one that kicks in, and that is not configurable at runtime: > FD_SETSIZE, which is the static size of the bit arrays passed to select. > IIRC it is 1024 for glibc. On at least some OSes, you can set it at build-time. Donal.
From: Alexandre Ferrieux on 14 Oct 2009 09:48
On Oct 14, 3:30 pm, "Donal K. Fellows" <donal.k.fell...(a)manchester.ac.uk> wrote: > On 14 Oct, 01:41, Alexandre Ferrieux <alexandre.ferri...(a)gmail.com> > wrote: > > > However, as a recent discussion on tclcore showed, there is another > > one that kicks in, and that is not configurable at runtime: > > FD_SETSIZE, which is the static size of the bit arrays passed to select.. > > IIRC it is 1024 for glibc. > > On at least some OSes, you can set it at build-time. Yes. Recompile your libc, reboot, and pray. -Alex |