How to Make HTML5 Barcode Reader with Desktop and Mobile Cameras

Recently, I noticed someone asked whether Dynamsoft Barcode Reader SDK can work for mobile devices. Because Dynamsoft Barcode Reader SDK does not support ARM yet, we can’t use it to develop native mobile apps. However, don’t be frustrated. Inspired by the article – Face detection using HTML5, javascript, webrtc, websockets, Jetty and OpenCV, I figured out the alternative solution. You can transmit the real-time camera stream to a remote server using HTML5 in Web browsers and decode the barcode image with Dynamsoft Barcode Reader SDK on the server that running on Windows, Linux or Mac. In this article, I will illustrate how to implement an HTML5 Barcode Reader with Cameras for PC and mobile devices step by step.

Prerequisites

  • Server-side Barcode SDK: Dynamsoft Barcode SDK 4.0 – available for Windows, Linux and Mac.
  • WebSocket Server: Jetty – a Java HTTP (Web) Server and Java Servlet container.
  • Web browsers: Chrome and Firefox for Android or Windows. Not supported for iOS.

How to Open Webcam or Mobile Cameras in Chrome and Firefox with HTML5

In HTML5, Navigator.getUserMedia() prompts the user for camera permission. The method behaves differently in Chrome and Firefox, here is how we can correctly use it:

// https://developer.mozilla.org/en-US/docs/Web/API/Navigator/getUserMedia
navigator.getUserMedia = navigator.getUserMedia ||
                         navigator.webkitGetUserMedia ||
                         navigator.mozGetUserMedia;

if (navigator.getUserMedia) {
   navigator.getUserMedia({ audio: true, video: { width: 1280, height: 720 } },
      function(stream) {
         var video = document.querySelector('video');
         video.src = window.URL.createObjectURL(stream);
         video.onloadedmetadata = function(e) {
           video.play();
         };
      },
      function(err) {
         console.log("The following error occured: " + err.name);
      }
   );
} else {
   console.log("getUserMedia not supported");
}

Camera Preview Data for Transmission: Base64 or Binary?

When using Webcam or cameras in Web browsers, we can capture preview image as base64 String and then convert it to binary:

Convert canvas to base64 String:

data = canvas.toDataURL('image/png', 1.0);

Convert base64 to binary:

// stackoverflow: http://stackoverflow.com/questions/4998908/convert-data-uri-to-file-then-append-to-formdata/5100158
function dataURItoBlob(dataURI) {
    // convert base64/URLEncoded data component to raw binary data held in a string
    var byteString;
    if (dataURI.split(',')[0].indexOf('base64') >= 0)
        byteString = atob(dataURI.split(',')[1]);
    else
        byteString = unescape(dataURI.split(',')[1]);

    // separate out the mime component
    var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];

    // write the bytes of the string to a typed array
    var ia = new Uint8Array(byteString.length);
    for (var i = 0; i < byteString.length; i++) {
        ia[i] = byteString.charCodeAt(i);
    }

    return new Blob([ia], {type:mimeString});
}

// convert base64 to binary
newblob = dataURItoBlob(data);
ws.send(newblob);

We can wrap received binary data as BufferedImage on server-side:

        ByteArrayInputStream in = new ByteArrayInputStream(data);
    	BufferedImage bi;
		try {
			bi = ImageIO.read(in);
			File file = new File(mFile);
			// save client data to local image file
			ImageIO.write(bi, "PNG", file);

		} catch (IOException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}

However, the conversion of base64 to binary may affect the performance on mobile devices. If so, we can send base64 String to Jetty WebSocket server and convert base64 String to binary in Java on server-side.

// StackOverflow: http://stackoverflow.com/questions/23979842/convert-base64-string-to-image
String base64Image = message.split(",")[1];
byte[] imageBytes = javax.xml.bind.DatatypeConverter.parseBase64Binary(base64Image);

Besides data type, we also need to consider the resolution of the preview image. Since PC is more powerful than the smartphone, we can set resolution 640×480 for desktop and 240×320 (holding your smartphone in portrait orientation) for mobile.

How to Use WebSocket to Send and Receive Data

WebSocket Client in JavaScript

Create WebSocket:

// create websocket
var ws = new WebSocket("ws://192.168.8.84:88");

ws.onopen = function() {

};

ws.onmessage = function (evt) {

};

ws.onclose = function() {

};

ws.onerror = function(err) {

};

Set an interval time for sending camera stream:

// scan barcode
function scanBarcode() {
  intervalId = window.setInterval(function() {
    if (!isConnected || isPaused) {
        return;
    }

    var data = null, newblob = null;

    if (isPC) {
      ws.send("is pc");
      ctx.drawImage(videoElement, 0, 0, videoWidth, videoHeight);
      // convert canvas to base64
      data = canvas.toDataURL('image/png', 1.0);
      // convert base64 to binary
      newblob = dataURItoBlob(data);
      ws.send(newblob);
    }
    else {
      ws.send("is phone");
      mobileCtx.drawImage(videoElement, 0, 0, mobileVideoWidth, mobileVideoHeight);
      // convert canvas to base64
      data = mobileCanvas.toDataURL('image/png', 1.0);
      ws.send(data);
    }

  }, 200);
  console.log("create id: " + intervalId);
}

Do not forget to empty interval events if you stop scanning barcode:

window.clearInterval(intervalId);

To detect PC or mobile devices, you can use following code:

// check devices
function browserRedirect() {
   var deviceType;
   var sUserAgent = navigator.userAgent.toLowerCase();
   var bIsIpad = sUserAgent.match(/ipad/i) == "ipad";
   var bIsIphoneOs = sUserAgent.match(/iphone os/i) == "iphone os";
   var bIsMidp = sUserAgent.match(/midp/i) == "midp";
   var bIsUc7 = sUserAgent.match(/rv:1.2.3.4/i) == "rv:1.2.3.4";
   var bIsUc = sUserAgent.match(/ucweb/i) == "ucweb";
   var bIsAndroid = sUserAgent.match(/android/i) == "android";
   var bIsCE = sUserAgent.match(/windows ce/i) == "windows ce";
   var bIsWM = sUserAgent.match(/windows mobile/i) == "windows mobile";
   if (bIsIpad || bIsIphoneOs || bIsMidp || bIsUc7 || bIsUc || bIsAndroid || bIsCE || bIsWM) {
    deviceType = 'phone';
   } else {
    deviceType = 'pc';
   }
   return deviceType;
}

WebSocket Server in Java

Create server:

public class WebSocketServer {

    public static void main(String[] args) throws Exception {
        Server server = new Server(88);
        server.setHandler(new WSHandler());
        server.setStopTimeout(0);
        server.start();
        server.join();
    }
}

Configure policies:

@WebSocket
public class WSHandler extends WebSocketHandler {

	@Override
	public void configure(WebSocketServletFactory factory) {
		// TODO Auto-generated method stub
		factory.setCreator(new BarcodeCreator());

		// configuration
		factory.getPolicy().setMaxBinaryMessageBufferSize(1024 * 1024);
		factory.getPolicy().setMaxTextMessageBufferSize(1024 * 1024);
		factory.getPolicy().setMaxTextMessageSize(1024 * 1024);
		factory.getPolicy().setMaxBinaryMessageSize(1024 * 1024);
	}
}

Create WebSocket:

@WebSocket
public class BarcodeSocket {

    @OnWebSocketClose
    public void onClose(int statusCode, String reason) {

    }

    @OnWebSocketError
    public void onError(Throwable t) {

    }

    @OnWebSocketConnect
    public void onConnect(Session session) {

    }

    @OnWebSocketMessage
    public void onMessage(byte[] data, int off, int len) {

    }

    @OnWebSocketMessage
    public void onMessage(String message) {

    }

}

public class BarcodeCreator implements WebSocketCreator {

	@Override
	public Object createWebSocket(ServletUpgradeRequest arg0, ServletUpgradeResponse arg1) {
		// TODO Auto-generated method stub
		return new BarcodeSocket();
	}

}

How to Read Barcode in Java with Dynamsoft Barcode Reader SDK

Dynamsoft Barcode Reader does not provide any Java APIs. If you want to use the SDK in Java, you need to create a JNI (Java Native Interface) wrapper DynamsoftBarcodeJNIx64.dll. Please read the article How to Make Java Barcode Reader with Dynamsoft Barcode SDK.

Create a new project in Eclipse. Copy DynamsoftBarcodeJNIx64.dll and DynamsoftBarcodeReaderx64.dll to the project root directory.

jetty project

Reading barcode with a few lines of code:

      JBarcode.DBR_InitLicense(DBRLicense);

      private String readBarcode(String fileName) {
		// Result array
		tagBarcodeResultArray paryResults = new tagBarcodeResultArray();
		// Read barcode
		int iret = JBarcode.DBR_DecodeFile(fileName, 100,
				EnumBarCode.OneD | EnumBarCode.QR_CODE
				| EnumBarCode.PDF417 | EnumBarCode.DATAMATRIX, paryResults);

		System.out.println("DBR_DecodeFile return value: " + iret);
		String r = "";
		for (int iIndex = 0; iIndex < paryResults.iBarcodeCount; iIndex++) {
			tagBarcodeResult result = paryResults.ppBarcodes[iIndex];
			int barcodeDataLen = result.iBarcodeDataLength;

			byte[] pszTemp1 = new byte[barcodeDataLen];
			for (int x = 0; x < barcodeDataLen; x++) {
				pszTemp1[x] = result.pBarcodeData[x];
			}

			r += "Value: " + new String(pszTemp1);
		}

		return r;
      }

Testing HTML5 Barcode Reader in Chrome for Android and Windows

Chrome for Android

mobile android camera

mobile barcode reader

Chrome for Windows

desktop barcode reader

Source Code

https://github.com/dynamsoftsamples/HTML5-Webcam-Barcode-Reader