I’m in the process of executing the Java Matrix Benchmark on a Google Cloud Platform Compute Engine (n1-highmem-4 (4 vCPUs, 26 GB memory) with Intel Skylake processors). Actually I’m doing 3 separate benchmark executions using 3 different JVM:s on otherwise identical machines:
- Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
- Java(TM) SE Runtime Environment (build 9.0.4+11)
- Zing Runtime Environment for Java Applications (build 1.8.0-zing_18.04.0.0-b2)
When all is done it’ll be very interesting to compare the results between the JVM:s.
The results for all pure Java libraries running on Oracle 8 are already complete. Here’s the summary performance chart for that:
It’ll be another 2 weeks or so before everything is done. By then there will be results for these library and JVM combinations:
|Oracle 8||Oracle 9||Zing 8|
|Apache Commons Math||3.6.1||X||X||X|
|jblas (using native code)||1.2.4||X|
|MTJ (pure Java)||1.0.7||X||X||X|
|MTJ-N (using native code)||1.0.7||X||X|
|UJMP (pure Java)||0.3.0||X||X||X|
|UJMP-N (using native code)||0.3.0||X||X|
The jblas library (using native code) crashed the benchmark twice running on different JVM:s. As part of the plan for how to recover and resume the executions it was decided to not re-run the libraries using native code on the all the instances – to save time. The native code libraries are not the most interesting when comparing JVM:s.
Further, and more detailed, results will be published here, at the ojAlgo wiki and/or at the Java Matrix Benchmark web site:
Where are the results?
It took more than 2 months to finish that benchmark. When it was done the motivation to analyse the results had “declined”, but there are published results:
The full results (incl. the libraries using native code) for Oracle Java HotSpot(TM) 64-Bit Server 1.8.0_161 are published at the Java-Matrix-Benmchmark web site.
Partial results (no libraries using native code) for all 3 JVM:s were published at the ojAlgo wiki over at GitHub. Each of the (12) benchmarked operations got its own wiki page. On each of those pages there are 3 charts showing the relative performance of the libraries for the 3 JVM:s.
There are no published results explicitly comparing the performance of the JVM:s based on this benchmark execution. At this site, however, there are several other posts doing precisely that (based on data from other benchmark executions).