Question

Today I wrote a performance testing program to insert data and add attachments to couchdb. The server software is couchbase (which is a wrapper of couchdb) and the operating system is Windows 2003 server. The program is developed in C# and the couchdb driver is LoveSeat(which encapsulates some http methods to deal with couchdb). The data is quite simple, but the attachments are not small, about 70kb each. There were about 200 attachments and I was attaching them repeatedly. I started 5 threads in the client program.

Everything looked all right until couchdb server terminated unexpectedly. Actually it was not the first time it crashed. At first I thought it was because of the client program. But at last I found couchdb crashed. I don t think it was because of the data or attachment. because I was inserting the same data and adding the same attachment repeatedly. The program run for about an hour without problem until it crashed.

[Tue, 05 Jul 2011 11:00:19 GMT] [info] [<0.142.0>] 192.168.1.135 - -  GET  /test/67366 200

[Tue, 05 Jul 2011 11:00:19 GMT] [info] [<0.108.0>] 192.168.1.135 - -  GET  /test/7136 200

[Tue, 05 Jul 2011 11:00:19 GMT] [info] [<0.108.0>] 192.168.1.135 - -  GET  /test/47306 200

[Tue, 05 Jul 2011 11:00:19 GMT] [info] [<0.108.0>] 192.168.1.135 - -  GET  /test/27257 200

[Tue, 05 Jul 2011 11:00:19 GMT] [info] [<0.108.0>] 192.168.1.135 - -  PUT  /test/7136/f?rev=1-334efd144dcdc52fd3a3a981dce4472f 201

[Tue, 05 Jul 2011 11:00:25 GMT] [error] [<0.145.0>] ** Generic server <0.145.0> terminating 
** Last message in was {pread_iolist,4294342003}
** When Server state == {file,{file_descriptor,prim_file,{#Port<0.3143>,1464}},
                              0,4295164786}
** Reason for termination == 
** {{badmatch,{ok,<<183,92,29,219,169,127,153,2,50,217,252,186,178,175,202,
                    144,215,209,191,69,109,230,227,154,114,174,173,157,231,
                    153,246,124,105,239,174,51,143,24,108,175,101,215,175,
                    221,35,99,53,124,108,109,249,112,202,29,85,87,81,176,94,
                    219,11,103,129,231,25,111,242,108,246,207,107,72,173,172,
                    57,246,195,16,236,79,243,134,211,93,131,218,180,93,240,
                    173,213,199,226,175,176,217,250,154,89,39,237,157,250,77,
                    173,151,156,139,248,106,85,21,134,253,85,234,108,85,208,
                    67,177,130,124,247,161,98,77,173,126,170,111,80,84,45,
                    212,201,72,149,90,138,252,89,23,85,165,252,105,187,191,
                    41,86,125,148,106,149,175,252,78,185,198,154,207,172,142,
                    148,101,83,140,99,222,102,26,41,131,206,132,221,31,74,3,
                    172,176,158,236,136,71,120,169,63,35,161,251,208,86,202,
                    1,95,208,25,51,76,250,100,182,177,122,31,91,230,249,214,
                    245,229,250,212,118,86,167,120,116,6,173,78,113,18,171,
                    143,215,191,38,207,51,92,150,10,10,83,164,98,154,181,157,
                    ......... a loooooot of numbers.

Answer 1

The final conclustion:

couchbase 1.02 does NOT support data file larger than 4GB on a windows 2003 32 bit machine.

couchbase 2.0 developer prereview version DOES support data file larger than 4GB on a windows 2003 32 bit machine. But as far as I can tell, version 2.0 is at least 5 times slower than version 1.02.

couchdb 1.1 from this link DOES support file larger than 4GB on a windows 2003 32 bit machine. But it is as slow as couchbase 2.0.

Couchdb is tooooo slow on windows(at least in my user case it is slow). At last, I tried using mysql to store the files. Mysql turned out to be 8 times faster! inserting an attachment into couchdb takes 650ms while the number for mysql is only 80ms.

Answer 2

I m sorry you had this error. May I suggest to post this question on the Couchbase forums? Our support crew is monitoring those more closely than SO: http://www.couchbase.org/forums/

Answer 3

I think I ve found the cause. The file size reached 4GB. But according to this wiki page, Erlang/OTP release R14B01 no longer has this bug. So I think couchbase uses an embedded erlang/otp version which is previous to R14B01. (I also installed a standalone Erlang/OTP R14B03 on the machine, but seemed that it was not utilized)

友情链接