I’ve been keeping an eye on the JRuby scene for a while now and I recently went about benchmarking again. JRuby is interesting because it has the potential to bring robust Java deployment and tools to the Ruby world.

I made a simple widgets application as outlined here and installed JRuby/Glassfish as outlined here (both great articles at AD TechFL) and then proceeded to benchmark:

Glassfish/Jruby on httperf:


<pre>
Elena:~/Applications/jruby/lib vishnu$ time httperf --client=0/1 --server=localhost --port=8080 --uri=/jruby-demo/widgets/list --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --rate=60
httperf --client=0/1 --server=localhost --port=8080 --uri=/jruby-demo/widgets/list --rate=60 --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 2

real 0m8.008s
user 0m1.418s
sys 0m5.799s
</pre>


Glassfish/Jruby on ab:


<pre>
Elena:~/Applications/jruby/lib vishnu$ ab -n400 http://localhost:8080/jruby-demo/widgets/show/1
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Percentage of the requests served within a certain time (ms)
50% 16
66% 17
75% 17
80% 17
90% 18
95% 20
98% 37
99% 116
100% 1019 (last request)
</pre>


Let’s compare. Here’s how a single Mongrel/C does with httperf:


<pre>
Elena:~/Applications/jruby/lib vishnu$ time httperf --client=0/1 --server=localhost --port=3000 --uri=/widgets/list --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --rate=30
httperf --client=0/1 --server=localhost --port=3000 --uri=/widgets/list --rate=30 --send-buffer=4096 --recv-buffer=16384 --num-conns=480 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 3

real 0m16.017s
user 0m3.912s
sys 0m11.354s
</pre>


And Mongrel/C with ab:


<pre>
Elena:~/Applications/jruby/lib vishnu$ ab -n400 http://localhost:3000/widgets/list
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Percentage of the requests served within a certain time (ms)
50% 11
66% 12
75% 12
80% 12
90% 13
95% 15
98% 113
99% 149
100% 1167 (last request)
</pre>


Observations

• Advantage: For the same number of connections, Glassfish/Jruby versus a single instance Mongrel/C has twice the reply rate in half the benchmark time.
• Disadvantage: Request latency is more for Glassfish/Jruby. (around 5ms av. more)
• Disadvantage: JRuby Ruby/Rails support is not perfect but very good.
• Advantage: Much simpler to set up. Copy the war file to the server and deployment is done, autodeployment is also possible w/o restarting Glassfish.
• Advantage: Much simpler to install and get going on the server, just requires a JDK, glassfish installed anywhere, etc.
• Advantage: Runs on Java, so Java deployment experience will do.
• Disadvantage: Does not support C library extensions for Ruby, question: how many of them are there & useful (memcached?) ?
• Advantage: Supports Java libraries kind of easily, again question: how many of them are there, useful and relevant?
• Question: This benchmark is versus a single mongrel, how does cluster vs. cluster compare?
• Question: Is glassfish clustering easy? Observation: mongrel clustering is cumbersome at present. Maybe swiftiply will help there with dynamic addition of mongrels?