A high concurrent IPV4/V6 threading server
Thread.js
=========
Thread.js is a very high concurrent ipv4/v6 server of a node.js addon with a subset of common.js.
Please see homepage for detail.
#####Main hightlights of thread.js are:
+ Multithreading by pthread with a subset of commonjs implements.
+ Non-blocking I/O base on epoll.
+ Implements a binary event emitter in v8.
+ Very high concurent performance and throughout.
+ It's stable and it's scalable.
###Usage
Thread.js provides three ways for applications.
1. Run with node.js - act as a replacement IP layer of node.js. Please see examples/net.js for example.
2. Run on the thread - support a subset of commonjs on the thread, currently implements a subset of common.js. Please see examples/net_regfile.js and examples/net_regfile_cc.js for example.
3. Implement a service handler - support writting a service handler to implement a protocol, like http, websocket etc. Please see examples/http.js and examples/http_regfile.js for example.
Additionally, thread.js provides a continuous write mode. Applications can send data to client at any time and doesn't need to wait for ondrain event .
**Note: Continuous write mode will try to send data all at once. Application should take care of the length received.
####Examples
+ examples/net.js - use thread.js as a network layer.
+ examples/net_regscript.js - register a function to thread.js, support doing pure javascript process. (simple thread.js mode)
+ examples/net_regfile.js, examples/client.js - register a javascript file to thread.js and runs thread.js api in the file.
+ examples/net_regfile_cc.js, examples/client_cc.js - demostrate using thread.js api of net.createConnection.
Todo: improve performance by applying a command/connection queue manager.
+ examples/http.js - use thread.js as a network layer and implements a http service handler.
+ examples/http_regfile.js, examples/http_client.js - register a javascript file to thread.js, runs thread.js api in the file and implements a http service handler.
Hope you like it.
Installation
--------------
npm install thread.js
Benchmark
---------
Benchmark is done by using weighttp on an i7 2.67Ghz CPU (4cores) on a Ubuntu 12.04 LTS 64-bit server.
######install weighttp
1 Install libev by apt-get install libev-dev
2 Follow the instruction at weighttp wiki to install weighttp.
######Tuning Linux
To achieve high concurrent performance, Linux must be tuned. The configuration is in deps/10-web.conf, you can apply it by using sysctl -p [filename]. Or you can put the file to /etc/sysctl.d for next reboot. Additionally, you can put deps/nofile.limit to /etc/security/limit.d to modify the limit of open file number. Though thread.js can modify that by options too.
User should at least set the TCP\_TW\_REUSE, or there is very likely system will fail to connect with 'Cannot assign requested address (99)', see description at weighttp wiki.
Here is the list of sysctl settings of deps/10-web.conf:
net.ipv4.tcp_fin_timeout = 15
fs.epoll.max_user_watches = 1000000
fs.file-max = 5000000
net.core.netdev_max_backlog = 100000
net.core.optmem_max = 10000000
net.core.rmem_default = 10000000
net.core.rmem_max = 10000000
net.core.somaxconn = 1000000
net.core.wmem_default = 10000000
net.core.wmem_max = 10000000
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 12000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_wmem = 30000000 30000000 30000000
net.ipv4.tcp_orphan_retries = 1
net.ipv4.tcp_max_orphans = 8192
#net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 1
Note 1: All detail outputs and test scripts can be found at directory /benchmark.
Note 2: There is a reuse mechanism in thread.js, you can get higher scores by running the benchmark manually at the second time.
Note 3: You may need to copy examples/*.js to the directory where you run the benchmark.
#####NET Performance:
A 'hello world' server using net module.
+ node with cluster module - examples/node_cluster.js
+ node with thread.js - examples/net.js
+ thread.js - examples/net_regfile.js
All tests run as 8 cpu units.
net connections/sec
node_cluster node with tj threadjs
200 48352 81343 83121
500 47978 78007 81971
750 40199 77905 76840
1000 47170 73458 78905
1250 41250 72063 76738
1500 44624 73598 74492
1750 40386 72416 75769
2000 35174 73918 75174
!net_conn
net keepalive requests/sec
node_cluster node with tj threadjs
200 173069 297282 383607
500 174376 311152 376256
750 177343 315716 374010
1000 178202 296393 365689
1250 174102 283776 345388
1500 174480 276573 336369
1750 171244 266155 319608
2000 170675 267328 313555
!net_keep
#####HTTP Performance:
A 'hello world' server using simple_http service handler.
+ node http with cluster module - examples/node_cluster_http.js
+ node with thread.js with a service handler - examples/http.js
+ thread.js with a service handler - examples/http_regfile.js, examples/http_client.js
All tests run as 8 cpu units.
http connections/sec
node_cluster node with tj threadjs
200 31552 82054 81518
500 29468 78882 80104
750 30008 75741 75938
1000 29278 71373 77577
1250 14400 72631 75945
1500 13859 76791 75165
1750 12417 72147 74491
2000 12667 73394 74089
!http_conn
http keepalive requests/sec
node_cluster node with tj threadjs
200 70151 213382 329009
500 70231 230153 328236
750 71910 223123 329234
1000 58294 216859 321123
1250 69058 212391 307097
1500 69118 210384 298151
1750 66007 205793 287315
2000 69120 203913 279819
!http_keep
#####Throughput:
+ use node cluster and node buffer - test/http_simple_cluster.js
+ use threadjs and a 50k ArrayBuffer - test/net_regfile-lw-50k-byte.js
+ use threadjs and a 100k ArrayBuffer - test/net_regfile-lw-100k-byte.js
All tests run as 8 cpu units.
50k response server
cluster threadjs cluster-keep threadjs-keep
200 948283 1553675 1396788 1954481
500 962058 1455664 1464215 1821893
750 938583 1338557 1399691 1815001
1000 592578 1425235 1432528 1791102
1250 702026 1326193 964106 1743758
1500 597193 1300322 1348425 1704217
1750 793223 1216900 1337272 1713497
2000 548075 1203525 1358335 1642143
!lw50k
100k response server
cluster threadjs cluster-keep threadjs-keep
200 1618670 2021279 2425676 2261467
500 1622245 1843869 2428176 2199467
750 1615689 1819297 2236474 2134023
1000 1593252 1817217 2337331 2053169
1250 1541604 1670520 2321876 2067337
1500 1570330 1708348 1893426 2071124
1750 656290 1750326 2123765 1965395
2000 1119305 1669303 2180329 1920265
!lw100k
#####Stability:
Thread.js reuses and caches most of the javascript objects.
You can see a stable memory allocator by running node with --trace_gc --trace_gc_verbose v8 flags.
API
----------------
please see at http://rob333.github.com/thread.js/
Compile flag
--------------
You can define compile flag to set different behavior.
+ USE_NONBLOCKLISTEN : set listen fd to use epoll non-blocking mechanism.
+ PRINT_THREADID : print which thread is working.
+ WRITE_EMIT_ONDRAIN : enable emitting a ondrain event after write operation.
Todo
--------------
+ Add metrics operation to typedarray, doing collision detection etc..
+ Add more common api in thread.js.
Contributors
--------------
Pulling and contribution is very welcome. Thanks.
Changelog
------------------
+ 09-09-2012 v0.0.2: implements service handler and fix bugs.
+ 08-23-2012 v0.0.1: First announce, implements main-loop, regScript, regFile and continuous write mode.
Thanks
-----------------
Thanks to all platform I have been developed with. Thanks very much.