How to Benchmark Barcode SDK Performance – ZXing vs ZBar

I saw many posts arguing the performance winner between open-source barcode SDKs – ZXing and ZBar. As an engineer, who is developing commercial barcode reader software for Dynamsoft, I am curious about which open source project is better, ZXing or ZBar? Considering ZXing is implemented in Java, whereas ZBar is implemented in C/C++. To fairly compare their performance, I decided to use JNI to wrap ZBar C/C++ source code and benchmark them in a Java program.

zxing zbar

Prerequisites of Barcode SDK

ZXing Source code

https://github.com/zxing/zxing

ZBar Source Code

https://github.com/ZBar/ZBar

ZBar Windows Installer

http://sourceforge.net/projects/zbar/files/zbar/

How to Decode TIFF in Java

I need to use a dataset that includes many TIFF files for testing barcode reading performance. The Java Class ImageIO could read many image formats such as JPEG, PNG, BMP, but not TIFF. I searched Oracle’s Website and found Java Advanced Imaging (JAI) API, which provides methods for decoding TIFF files.

Where to download JAI jar packages?

I was surprised that the download links of Oracle official Website pointed to 404 error page. If you Google Java JAI, you may find there is no valid link on the first page of searching results. I patiently looked for download links page by page, and even changed the search engine. Luckily there is a working page existed: http://www.java2s.com/Code/Jar/j/Downloadjaicore113jar.htm. To make JAI work, you need to download jai_codec-1.1.3.jar and jai_core-1.1.3.jar. Here is the source code demonstrating how to read TIFF file to int[]:

File file = new File(fileName);
RenderedImage tiff = JAI.create("tiffload", fileName);
BufferedImage image = PlanarImage.wrapRenderedImage(tiff).getAsBufferedImage();
int[] pixels = image.getRGB(0, 0, image.getWidth(), image.getHeight(), null, 0, image.getWidth());

How to Read Multiple Barcodes of an Image with ZXing

Previously, I shared a post – How to Write and Read QR Code with ZXing in Java, which demonstrates how to read barcodes with MultiFormatReader. But MultiFormatReader only returns one result. What if we want to read multiple barcodes from an image, such as a document with code39, code93 and QR code? In this case, we need to use another reader class GenericMultipleBarcodeReader.

About GenericMultipleBarcodeReader:

Attempts to locate multiple barcodes in an image by repeatedly decoding portion of the image. After one barcode is found, the areas left, above, right and below the barcode’s ResultPoints are scanned, recursively.

To read multiple barcode results, you can write Java code as follows:

         RGBLuminanceSource source = new RGBLuminanceSource(image.getWidth(),
				image.getHeight(), pixels);
		bitmap = new BinaryBitmap(new HybridBinarizer(source));

		Map<DecodeHintType, Object> hints = new HashMap<DecodeHintType, Object>();
		hints.put(DecodeHintType.TRY_HARDER, null);
		Collection<BarcodeFormat> formats = new ArrayList<>();
		formats.add(BarcodeFormat.QR_CODE);
		formats.add(BarcodeFormat.CODABAR);
		formats.add(BarcodeFormat.CODE_39);
		formats.add(BarcodeFormat.CODE_93);
		formats.add(BarcodeFormat.CODE_128);
		formats.add(BarcodeFormat.EAN_8);
		formats.add(BarcodeFormat.EAN_13);
		formats.add(BarcodeFormat.ITF);
		formats.add(BarcodeFormat.UPC_A);
		formats.add(BarcodeFormat.UPC_E);
		formats.add(BarcodeFormat.UPC_EAN_EXTENSION);

		hints.put(DecodeHintType.POSSIBLE_FORMATS, formats);
		MultiFormatReader reader = new MultiFormatReader(); 

		// read multi barcodes
		GenericMultipleBarcodeReader multiReader = new GenericMultipleBarcodeReader(
				reader);
		try {
			Result[] results = multiReader.decodeMultiple(bitmap, hints);
			System.out.println(ZXING + TIME_COST
					+ ((System.nanoTime() - start) / 1000000) + MS);
			if (results != null) {
				for (Result result : results) {
					System.out.println(ZXING + TYPE + result.getBarcodeFormat() + VALUE + result.getText());
				}
			}
		} catch (NotFoundException e) {
			e.printStackTrace();
			return;
		}

Creating Java Native Interface (JNI) for ZBar

How to use ZBar to decode barcodes in C/C++? If no idea, you can refer to the sample scan_image.cpp provided by ZBar in source code or installer. With a few changes, the JNI sample may be as follows:

#include <iostream>
#include <Magick++.h>
#include <zbar.h>
#include <jni.h>
#define STR(s) #s

using namespace std;
using namespace zbar;

#ifndef DEBUG
#define DEBUG(...) printf(__VA_ARGS__)
#endif

extern "C" {
	JNIEXPORT jobjectArray JNICALL Java_com_dynamsoft_zbar_ZBarReader_decode(JNIEnv *env, jobject obj, jstring fileName);
}

JNIEXPORT jobjectArray JNICALL Java_com_dynamsoft_zbar_ZBarReader_decode(JNIEnv *env, jobject obj, jstring fileName)
{
	const char *pszFileName = env->GetStringUTFChars(fileName, 0);

#ifdef MAGICK_HOME
	// http://www.imagemagick.org/Magick++/
	//    under Windows it is necessary to initialize the ImageMagick
	//    library prior to using the Magick++ library
	Magick::InitializeMagick(MAGICK_HOME);
#endif

	// create a reader
	ImageScanner scanner;

	// configure the reader
	scanner.set_config(ZBAR_NONE, ZBAR_CFG_ENABLE, 1);

	// obtain image data
	Magick::Image magick(pszFileName);  // read an image file
	int width = magick.columns();   // extract dimensions
	int height = magick.rows();
	Magick::Blob blob;              // extract the raw data
	magick.modifyImage();
	magick.write(&blob, "GRAY", 8);
	const void *raw = blob.data();

	// wrap image data
	Image image(width, height, "Y800", raw, width * height);

	// scan the image for barcodes
	int n = scanner.scan(image);

	// find java class
	jclass clsZBarResult = env->FindClass("com/dynamsoft/zbar/ZBarResult");
	// create java array
	int data_length = 0;
	for (Image::SymbolIterator symbol = image.symbol_begin();
		symbol != image.symbol_end();
		++symbol) {
		++data_length;
	}

	jobjectArray clsZBarResultArray = env->NewObjectArray(data_length, clsZBarResult, 0);
	int iIndex = 0;

	// extract results
	for (Image::SymbolIterator symbol = image.symbol_begin();
		symbol != image.symbol_end();
		++symbol) {
		// do something useful with results
		//cout << "ZBR Type: " << symbol->get_type_name()
		//	<< ", Value \"" << symbol->get_data() << '"' << endl;

		// save result to java array
		jmethodID init = env->GetMethodID(clsZBarResult, "<init>", "()V");
		jobject clsZBarResultObj = env->NewObject(clsZBarResult, init);
		jfieldID jType = env->GetFieldID(clsZBarResult, "mType", "Ljava/lang/String;");
		jfieldID jValue = env->GetFieldID(clsZBarResult, "mValue", "Ljava/lang/String;");
		env->SetObjectField(clsZBarResultObj, jType, env->NewStringUTF(symbol->get_type_name().c_str()));
		env->SetObjectField(clsZBarResultObj, jValue, env->NewStringUTF(symbol->get_data().c_str()));
		env->SetObjectArrayElement(clsZBarResultArray, iIndex, clsZBarResultObj);
		++iIndex;		
	}

	// clean up
	image.set_data(NULL, 0);

	// release string
	env->ReleaseStringUTFChars(fileName, pszFileName);

	return clsZBarResultArray;
}

I changed the main function to JNI method and packaged barcode results to Java objects. See the corresponding Java classes.

ZBarReader.java:

package com.dynamsoft.zbar;

import com.dynamsoft.utils.BaseReader;

public class ZBarReader extends BaseReader {

	static {
		System.loadLibrary("zbarjni");
	}

	public void testZBar(String fileName) {
		long start = System.nanoTime();
		ZBarReader reader =  new ZBarReader();
		ZBarResult[] results = (ZBarResult[])reader.decode(fileName);
		System.out.println(ZBAR + TIME_COST
				+ ((System.nanoTime() - start) / 1000000) + MS);

		if (results != null && results.length > 0) {
			mCount += 1;
			for (ZBarResult result : results) {
				System.out.println(ZBAR + TYPE + result.mType + VALUE + result.mValue);
			}
		}
	}

	@Override
	public int getCount() {
		// TODO Auto-generated method stub
		return super.getCount();
	}

	public native Object[] decode(String fileName);
}

ZBarResult.java:

package com.dynamsoft.zbar;

public class ZBarResult {
	public String mType;
	public String mValue;
}

Benchmark Test between ZXing and ZBar

Dynamsoft and ZXing provide some testing image sets, you can get them from https://github.com/Dynamsoft/Dynamsoft-Barcode-Reader/tree/master/Images and https://github.com/zxing/zxing/tree/master/core/src/test/resources. To make results convincing, I used the testing images provided by ZXing: https://github.com/zxing/zxing/tree/master/core/src/test/resources/blackbox/qrcode-1 Here is the report:

F:\resources\blackbox\qrcode-1\1.png

ZXI Time cost: 122ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 33ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\10.png

ZXI Time cost: 66ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\11.png

ZXI Time cost: 53ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 29ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\12.png

ZXI Time cost: 71ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 29ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\13.png

ZXI Time cost: 84ms

ZXI Type: QR_CODE, value: http://google.com/gwt/n?u=bluenile.com

ZBA Time cost: 27ms

ZBA Type: QR-Code, value: http://google.com/gwt/n?u=bluenile.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\14.png

com.google.zxing.NotFoundException

ZBA Time cost: 32ms

ZBA Type: QR-Code, value: http://google.com/gwt/n?u=bluenile.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\15.png

ZXI Time cost: 90ms

ZXI Type: QR_CODE, value: http://google.com/gwt/n?u=bluenile.com

ZBA Time cost: 34ms

ZBA Type: QR-Code, value: http://google.com/gwt/n?u=bluenile.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\16.png

ZXI Time cost: 77ms

ZXI Type: QR_CODE, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

ZBA Time cost: 35ms

ZBA Type: QR-Code, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\17.png

ZXI Time cost: 82ms

ZXI Type: QR_CODE, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

ZBA Time cost: 32ms

ZBA Type: QR-Code, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\18.png

ZXI Time cost: 80ms

ZXI Type: QR_CODE, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

ZBA Time cost: 35ms

ZBA Type: QR-Code, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\19.png

ZXI Time cost: 64ms

ZXI Type: QR_CODE, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\2.png

ZXI Time cost: 61ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\20.png

ZXI Time cost: 55ms

ZXI Type: QR_CODE, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

ZBA Time cost: 30ms

ZBA Type: QR-Code, value: Sean Owen

srowen@google.com

917-364-2918

http://awesome-thoughts.com

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\3.png

ZXI Time cost: 56ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 29ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\4.png

ZXI Time cost: 70ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\5.png

ZXI Time cost: 64ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\6.png

ZXI Time cost: 56ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 28ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\7.png

ZXI Time cost: 73ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 30ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\8.png

ZXI Time cost: 55ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 27ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

       F:\resources\blackbox\qrcode-1\9.png

ZXI Time cost: 55ms

ZXI Type: QR_CODE, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

ZBA Time cost: 27ms

ZBA Type: QR-Code, value: MEBKM:URL:http\://en.wikipedia.org/wiki/Main_Page;;

-------------------------------------------------------------------------------------

ZXI passed: 19

ZBA passed: 20

According to the results, ZBar seems to be much better than ZXing when reading QR code. Because I only tested QRcode, the conclusion may not be accurate. If you are interested, you can try other datasets yourself.

Source Code

https://github.com/dynamsoftsamples/java-barcode-reader

  • http://www.dynamsoft.com Desmond Shaw

    Could you leave your email address? I’ll send possible solutions to you privately.

  • Rmstar Francis

    We are Edu group, not commercial identity. However, we can use what ever we want as long as the cost is low. What do you suggest that can meet our work flow as describe above?

  • http://www.dynamsoft.com Desmond Shaw

    QR is better than PDF417 and DataMatrix. The page you shared mentioned the issue is fixed in zbar’s source control repository. You can try to build it yourself. Dynamsoft Barcode SDK will support PDF file in next version. Are you interested in using commercial SDK?

  • Rmstar Francis

    Great post. Big fan of CLI. I have question and I hope you can help me out. Scope: We generate exams (4 pages each exam booklet) with QR code (which store only student number) so we can email the exam individually to each student. Process: After all the exams marked and scan (@300dpi) save as one big PDF file. We then use pdftk to separate the PDF file back to individual exam (1.pdf, 2.pdf, n.pdf). Next, use Zbar to scan QR code in each pdf files, out put the result with student number from QR code and n.pdf file name pair in a CSV text file. Next use mailx parse the CSV file and our student list which contain student number and email. Once the 2 CSV match, email sent.
    Issue: I read old post that pdf is not easy for Zbar to read and jpeg has issue. http://www.rubydoc.info/gems/zbar/0.2.0/ZBar/JPEG
    What is your suggestion for using Zbar to out put CSV file from individual PDF? Is PDF417 or DataMatrix better for our workflow?
    Thanks
    Francis