vtd-xml

Interesting results. I’m absolutely amazed and impressed at how far JRuby has come. Keep it up:).

Script:
1..10_000_000.times {|i| print ‘.’ if i % 100_000 == 0}

JRuby RC3: 5.5s
JRuby RC3 looping five times: 18.8s
Ruby 1.8: 5s
Ruby 1.8 looping five times: 25.2s

It’s also important to run on the most current JVM possible, since that has a tremendous impact on performance.

The only thing inside my loop is:
if(i % 100000 == 0) System.out.print(’.');

Java prints this out as expected. Interestingly, Ruby 1.9.0 (which is basically 2.0 RC1) does not. It pauses for a while (10 seconds), then spits out all the dots at once. I’m guessing this is a bug in its bytecode compiler, which is new to 1.9.

As I understand it though, Java has been a match for C/C++ in many areas since it got a good JIT compiler. Frankly I hated Java before then; its performance was terrible.

I’d imagine Java is doing some optimizations (at runtime, I’m told javac’s -O option is now useless), but its not smart enough to sidestep the useless loop and just print some dots. Increasing the iterations to 100 million and the mod value to 1 million (i.e., keeping the number of dots printed the same, but increasing the loop length by 10) produces a delay of .828 seconds.

Still, looks like you are right, Grant. Scripting languages don’t seem to be as fast (for a 10m loop at least). I think the general consensus in the comments is that they’re fast enough though.

How crazy is it that we’re comparing language performance with JAVA as a baseline? A few years back (and even currently, though much less so) it was “Java isn’t as fast as C++”. It’s amazing that Java has come so far and gotten so “fast”.

And even better, the scripting languages, especially Groovy, will keep getting faster too, which is great news for Java.

If you benchmark other currently-popular dynamic languages like Ruby and JRuby, I think you’ll find their execution times are around the same. JRuby actually seems to be slower on pure loops than normal Ruby.

No matter what dynamic “scripting” language you use, I think its probably best not to use it to write loops with 10 million iterations.

My results for a 1..10m loop were:
Java: .042s
Ruby 1.8.6: 4.407s
Groovy: 6.407s
JRuby: 11.391s
JRuby 1.1 RC2: 11.407
Ruby 1.9.0: 10.344 (with some weird behavior)

Being able to fall back to Java is what really interested me in Groovy in the first place. I’ve always wanted to try Ruby and Rails, but I could never make the leap – there was just too much I didn’t know. Groovy makes things nice and comfortable – familiar syntax and being able to reuse all of my Java knowledge where big factors in deciding to learn Groovy.

That is exactly why I really got excited about groovy! The language is really nice and I can fall back to Java when need be. I would use it for my own scripts, plug-ins, etc… but I don’t think I would want to roll out a new app with it yet.

Jonchase,
About the wrapping of the ints, I assumed this was the reason too. My code really was just a for loop, exactly the same code in Java and groovy. I tried to used “Integers” in java too, and nothing changed, although I’d chalk that up to clever Java optimizations.

Speed comes with maturity. Java was dog slow in the early days, but the productivity gains it gave me vs C++ were worth it. If I have any XML to process at all, picking Groovy over Java is a no brainer. If the resulting code turns out to be too slow, I can always drop back to a Java implementation later.

I’m sure that each new version of Groovy will by much faster than the last one. In the meantime, writing Groovy is just so natural and easy that coding is fun again.

[...] on from the previous post (and here’s the last post), here’s a very quick and dirty, completely unscientific, prone to error, your mileage may [...]

I wonder if the difference you saw related to how Groovy uses numbers (primitives) – since it wraps all primitives in their equivalent Java wrapper classes (int -> Integer), I’d imagine that would slow things down quite a bit. I’d be interested in seeing the performance of Java code that loops 10,000,000 using the wrapper classes instead of ints. Also, did you try running the Groovy test more than once in the same run? I wonder if the JVM would cache those generated Integers from the first run and use them the second time (isn’t this called interning with Strings?).

There’s talk that there are going to be some optimizations around the way Groovy internally handles numbers – that could speed things up quite a bit.

Groovy (like most scripting languages) is slower than many other “heavyweight” languages (like Java) – but one of the great things about Groovy is how easy it is to fall back on Java if your really need to write some performant code.

Right now I couldn’t see myself writing an entire application in Groovy – I’m sure there would be parts where Java would be the better fit. Thus the wonder of Groovy:)

What was the code you used for your test?

I don’t think a loop to ten million is common, but if I want to parse 100,000 entries in an XML file, how can it possibly keep up?

http://www.jroller.com/ff/entry/groovy_good_bad_and_the

The most important comparison I would like to see is JRuby vs. Groovy. Unfortunately the raytracer benchmark didn’t include JRuby, but I’d be very interested to see how it fairs. Jython too!

I think we need a benchmark suite for JVM languages!

Clinton