I’m in the process of executing the Java Matrix Benchmark on a Google Cloud Platform Compute Engine (n1-highmem-4 (4 vCPUs, 26 GB memory) with Intel Skylake processors). Actually I’m doing 3 separate benchmark executions using 3 different JVM:s on otherwise identical machines:
- Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
- Java(TM) SE Runtime Environment (build 9.0.4+11)
- Zing Runtime Environment for Java Applications (build 1.8.0-zing_18.04.0.0-b2)
When all is done it’ll be very interesting to compare the results between the JVM:s.
The results for all pure Java libraries running on Oracle 8 are already complete. Here’s the summary performance chart for that:
It’ll be another 2 weeks or so before everything is done. By then there will be results for these library and JVM combinations:
| Oracle 8 | Oracle 9 | Zing 8 | ||
|---|---|---|---|---|
| Colt | 1.2.0 | X | X | X |
| Apache Commons Math | 3.6.1 | X | X | X |
| EJML | 0.33 | X | X | X |
| JAMA | 1.0.3 | X | X | X |
| jblas (using native code) | 1.2.4 | X | ||
| la4j | 0.6.0 | X | X | X |
| MTJ (pure Java) | 1.0.7 | X | X | X |
| MTJ-N (using native code) | 1.0.7 | X | X | |
| ojAlgo | 45.0.0 | X | X | X |
| Parallel Colt | 0.9.4 | X | X | X |
| UJMP (pure Java) | 0.3.0 | X | X | X |
| UJMP-N (using native code) | 0.3.0 | X | X |
The jblas library (using native code) crashed the benchmark twice running on different JVM:s. As part of the plan for how to recover and resume the executions it was decided to not re-run the libraries using native code on the all the instances – to save time. The native code libraries are not the most interesting when comparing JVM:s.
Further, and more detailed, results will be published here, at the ojAlgo wiki and/or at the Java Matrix Benchmark web site:
Update: 2019-11-30
Where are the results?
It took more than 2 months to finish that benchmark. When it was done the motivation to analyse the results had “declined”, but there are published results:
The full results (incl. the libraries using native code) for Oracle Java HotSpot(TM) 64-Bit Server 1.8.0_161 are published at the Java-Matrix-Benmchmark web site.
Partial results (no libraries using native code) for all 3 JVM:s were published at the ojAlgo wiki over at GitHub. Each of the (12) benchmarked operations got its own wiki page. On each of those pages there are 3 charts showing the relative performance of the libraries for the 3 JVM:s.
More Advanced Operations
Solving Linear Systems
SVD & EvD
There are no published results explicitly comparing the performance of the JVM:s based on this benchmark execution. At this site, however, there are several other posts doing precisely that (based on data from other benchmark executions).