Question

Does the erlang TCP/IP library have some limitations? I ve done some searching but can t find any definitive answers.

I have set the ERL_MAX_PORTS environment variable to 12000 and configured Yaws to use unlimited connections.

I ve written a simple client application that connects to an appmod I ve written for Yaws and am testing the number of simultaneous connections by launch X number of clients all at the same time.

I find that when I get to about 100 clients, the Yaws server stops accepting more TCP connections and the client errors out with

Error in process  with exit value: {{badmatch,{error,socket_closed_remotely}}

I know there must be a limit to the number of open simultaneous connections, but 100 seems really low. I ve looked through all the yaws documentation and have removed any limit on connections.

This is on a 2.16Ghz Intel Core 2 Duo iMac running Snow Leopard.

A quick test on a Vista Machine shows that I get the same problems at about 300 connections.

Is my test unreasonable? I.e. is it silly to open 100+ connections simultaneously to test Yaws concurrency?

Thanks.

Answer 1

After trying out everybody s suggestion and scouring the Erlang docs, I ve come to the conclusion that my problem is with Yaws not being able to keep up with the load.

On the same machine, an Apache Http Components web server (non-blocking I/O) does not have the same problems handling connections at the same thresholds.

Thanks for all your help. I m going to move on to other erlang based web servers, like Mochiweb.

Answer 2

It seems you hit a system limitation, try to increase the max number of open files using

$ ulimit -n 500

Python on Snow Leopard, how to open >255 sockets?

Erlang itself has a limit of 1024:

From http://www.erlang.org/doc/man/erlang.html

The maximum number of ports that can be open at the same time is 1024 by default, but can be configured by the environment variable ERL_MAX_PORTS.

EDIT:

The system call listen() has a parameter backlog which determines how many requests can be queued, please check whether a delay between requests to establish connections helps. This could be your problem.

Answer 3

All Erlang system limits are reported in the Erlang Efficiency Guide:

http://erlang.org/doc/efficiency_guide/advanced.html#id2265856

Reading from the open ports section:

The maximum number of simultaneously open Erlang ports is by default 1024. This limit can be raised up to at most 268435456 at startup (see environment variable ERL_MAX_PORTS in erlang(3)) The maximum limit of 268435456 open ports will at least on a 32-bit architecture be impossible to reach due to memory shortage.

友情链接