Native math backends for faster indexing
pixolution flow ships with a pure Java implementation of linear algebra operations as default. This guaranties operability in any Java environment, but is much slower than native implementations like jblas .
pixolution flow has an auto discovery mechanism for the Java BLAS (Basic Linear Algebra Subprograms) backends jblas and netlib-java. It allows to use native implementation like cblas (using jblas) or Intel MKL, ATLAS or OpenBLAS (using netlib-java). This section focuses on using jblas on a Ubuntu Server OS. It is easy to use, runs on many different architectures and is up to 10 times faster compared to pure java implementation (tested on Intel CPUs).
In order to use jblas as math backend in pixolution flow your OS needs to provide
libgfortran3 and you have to add the
jblas-1.2.4.jar to the java classpath.
In Ubuntu you need to install the package
libgfortran3, other linux systems provide packages with similar names.
To ensure the backend is operational and can be used by pixolution flow execute the
BLAS_CHECK routine as shown below.
Please note that pixolution flow as well as
jblas-1.2.4.jar needs to be part of the classpath (
Your specific filenames in the classpath may vary.
Adjust them accordingly if necessary.
Running built-in BLAS discovery
BLAS_CHECK with jblas−1.2.4.jar in the classpath:
user@server ~$ java -cp "pixolution_flow3.4.1_solrX.X.X.jar:jblas-1.2.4.jar" BLAS_CHECK [ ... ] Backends found that do work: - JBlas: fast linear algebra library for Java from http://jblas.org/ (auto discovery would use this backend) - Pixolution: Pure Java implementation of BLAS from http://pixolution.org/ Created 10 random images in 138ms loaded descriptor generator in 275ms Run benchmark with backend JBLAS calculated 10 image descriptors in 4s 16ms Run benchmark with backend PIXOLUTION_BLAS calculated 10 image descriptors in 13s 282ms
To use jblas with pixolution flow in Solr place the
jblas-1.2.4.jar file beside the
pixolution_flow3.4.1_solrX.X.X.jar file (e.g.
/opt/solr/server/solr-webapp/webapp/WEB-INF/lib/ see Place the JAR File for details).
After restarting your Solr instance pixolution flow automatically uses jblas when new images are indexed or
rank.by URL is used.