I’ve been keeping an eye on the JRuby scene for a while now and I recently went about benchmarking again. JRuby is interesting because it has the potential to bring robust Java deployment and tools to the Ruby world.
I made a simple widgets application as outlined here and installed JRuby/Glassfish as outlined here (both great articles at AD TechFL) and then proceeded to benchmark:
Glassfish/Jruby on httperf:
Elena:~/Applications/jruby/lib vishnu$ time httperf --client=0/1 --server=localhost --port=8080 --uri=/jruby-demo/widgets/list --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --rate=60 httperf --client=0/1 --server=localhost --port=8080 --uri=/jruby-demo/widgets/list --rate=60 --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --num-calls=1 httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE Maximum connect burst length: 2 Total: connections 480 requests 480 replies 480 test-duration 7.999 s Connection rate: 60.0 conn/s (16.7 ms/conn, <=7 concurrent connections) Connection time [ms]: min 1.4 avg 24.6 max 251.3 median 15.5 stddev 35.7 Connection time [ms]: connect 0.1 Connection length [replies/conn]: 1.000 Request rate: 60.0 req/s (16.7 ms/req) Request size [B]: 83.0 Reply rate [replies/s]: min 60.0 avg 60.0 max 60.0 stddev 0.0 (1 samples) Reply time [ms]: response 24.5 transfer 0.0 Reply size [B]: header 360.0 content 1077.0 footer 0.0 (total 1437.0) Reply status: 1xx=0 2xx=438 3xx=0 4xx=0 5xx=42 CPU time [s]: user 1.42 system 5.79 (user 17.7% system 72.4% total 90.1%) Net I/O: 89.1 KB/s (0.7*10^6 bps) Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 real 0m8.008s user 0m1.418s sys 0m5.799s
Glassfish/Jruby on ab:
Elena:~/Applications/jruby/lib vishnu$ ab -n400 http://localhost:8080/jruby-demo/widgets/show/1 This is ApacheBench, Version 1.3d apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Finished 400 requests Server Software: Sun Server Hostname: localhost Server Port: 8080 Document Path: /jruby-demo/widgets/show/1 Document Length: 611 bytes Concurrency Level: 1 Time taken for tests: 8.551 seconds Complete requests: 400 Failed requests: 0 Broken pipe errors: 0 Total transferred: 397600 bytes HTML transferred: 244400 bytes Requests per second: 46.78 [#/sec] (mean) Time per request: 21.38 [ms] (mean) Time per request: 21.38 [ms] (mean, across all concurrent requests) Transfer rate: 46.50 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 13 21 52.6 16 1019 Waiting: 13 21 52.6 16 1019 Total: 13 21 52.6 16 1019 Percentage of the requests served within a certain time (ms) 50% 16 66% 17 75% 17 80% 17 90% 18 95% 20 98% 37 99% 116 100% 1019 (last request)
Let’s compare. Here’s how a single Mongrel/C does with httperf:
Elena:~/Applications/jruby/lib vishnu$ time httperf --client=0/1 --server=localhost --port=3000 --uri=/widgets/list --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --rate=30 httperf --client=0/1 --server=localhost --port=3000 --uri=/widgets/list --rate=30 --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --num-calls=1 httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE Maximum connect burst length: 3 Total: connections 480 requests 480 replies 480 test-duration 16.009 s Connection rate: 30.0 conn/s (33.4 ms/conn, <=18 concurrent connections) Connection time [ms]: min 9.0 avg 56.4 max 607.6 median 9.5 stddev 99.4 Connection time [ms]: connect 0.2 Connection length [replies/conn]: 1.000 Request rate: 30.0 req/s (33.4 ms/req) Request size [B]: 72.0 Reply rate [replies/s]: min 29.4 avg 29.8 max 30.0 stddev 0.3 (3 samples) Reply time [ms]: response 55.7 transfer 0.5 Reply size [B]: header 278.0 content 1011.0 footer 0.0 (total 1289.0) Reply status: 1xx=0 2xx=480 3xx=0 4xx=0 5xx=0 CPU time [s]: user 3.91 system 11.35 (user 24.4% system 70.9% total 95.3%) Net I/O: 39.9 KB/s (0.3*10^6 bps) Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 real 0m16.017s user 0m3.912s sys 0m11.354s
And Mongrel/C with ab:
Elena:~/Applications/jruby/lib vishnu$ ab -n400 http://localhost:3000/widgets/list This is ApacheBench, Version 1.3d apache-1.3 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Finished 400 requests Server Software: Mongrel Server Hostname: localhost Server Port: 3000 Document Path: /widgets/list Document Length: 1011 bytes Concurrency Level: 1 Time taken for tests: 7.974 seconds Complete requests: 400 Failed requests: 0 Broken pipe errors: 0 Total transferred: 515600 bytes HTML transferred: 404400 bytes Requests per second: 50.16 [#/sec] (mean) Time per request: 19.93 [ms] (mean) Time per request: 19.93 [ms] (mean, across all concurrent requests) Transfer rate: 64.66 [Kbytes/sec] received Connnection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 9 19 63.4 11 1167 Waiting: 9 19 63.4 11 1167 Total: 9 19 63.4 11 1167 Percentage of the requests served within a certain time (ms) 50% 11 66% 12 75% 12 80% 12 90% 13 95% 15 98% 113 99% 149 100% 1167 (last request)
Observations
- Advantage: For the same number of connections, Glassfish/Jruby versus a single instance Mongrel/C has twice the reply rate in half the benchmark time.
- Disadvantage: Request latency is more for Glassfish/Jruby. (around 5ms av. more)
- Disadvantage: JRuby Ruby/Rails support is not perfect but very good.
- Advantage: Much simpler to set up. Copy the war file to the server and deployment is done, autodeployment is also possible w/o restarting Glassfish.
- Advantage: Much simpler to install and get going on the server, just requires a JDK, glassfish installed anywhere, etc.
- Advantage: Runs on Java, so Java deployment experience will do.
- Disadvantage: Does not support C library extensions for Ruby, question: how many of them are there & useful (memcached?) ?
- Advantage: Supports Java libraries kind of easily, again question: how many of them are there, useful and relevant?
- Question: This benchmark is versus a single mongrel, how does cluster vs. cluster compare?
- Question: Is glassfish clustering easy? Observation: mongrel clustering is cumbersome at present. Maybe swiftiply will help there with dynamic addition of mongrels?
Leave a Reply