New Java Matrix Benchmark Results Coming

I’m in the process of executing the Java Matrix Benchmark on a Google Cloud Platform Compute Engine (n1-highmem-4 (4 vCPUs, 26 GB memory) with Intel Skylake processors). Actually I’m doing 3 separate benchmark executions using 3 different JVM:s on otherwise identical machines: 

  1. Java(TM) SE Runtime Environment (build 1.8.0_161-b12) 
  2. Java(TM) SE Runtime Environment (build 9.0.4+11) 
  3. Zing Runtime Environment for Java Applications (build 1.8.0-zing_18.04.0.0-b2) 

When all is done it’ll be very interesting to compare the results between the JVM:s.

The results for all pure Java libraries running on Oracle 8 are already complete. Here’s the summary performance chart for that:

It’ll be another 2 weeks or so before everything is done. By then there will be results for these library and JVM combinations:

Oracle 8Oracle 9Zing 8
Colt1.2.0XXX
Apache Commons Math3.6.1XXX
EJML0.33XXX
JAMA1.0.3XXX
jblas (using native code)1.2.4
X
la4j0.6.0XXX
MTJ (pure Java)1.0.7XXX
MTJ-N (using native code)1.0.7XX
ojAlgo45.0.0XXX
Parallel Colt0.9.4XXX
UJMP (pure Java)0.3.0XXX
UJMP-N (using native code)0.3.0XX

The jblas library (using native code) crashed the benchmark twice running on different JVM:s. As part of the plan for how to recover and resume the executions it was decided to not re-run the libraries using native code on the all the instances – to save time. The native code libraries are not the most interesting when comparing JVM:s.

Further, and more detailed, results will be published here, at the ojAlgo wiki and/or at the Java Matrix Benchmark web site:

Update: 2019-11-30

Where are the results?

It took more than 2 months to finish that benchmark. When it was done the motivation to analyse the results had “declined”, but there are published results:

The full results (incl. the libraries using native code) for Oracle Java HotSpot(TM) 64-Bit Server 1.8.0_161 are published at the Java-Matrix-Benmchmark web site.

Partial results (no libraries using native code) for all 3 JVM:s were published at the ojAlgo wiki over at GitHub. Each of the (12) benchmarked operations got its own wiki page. On each of those pages there are 3 charts showing the relative performance of the libraries for the 3 JVM:s.

Solving Linear Systems

SVD & EvD

There are no published results explicitly comparing the performance of the JVM:s based on this benchmark execution. At this site, however, there are several other posts doing precisely that (based on data from other benchmark executions).