Enhance Camera Scanning with the New Dynamsoft Camera Enhancer

Dynamsoft Camera Enhancer (DCE) provides high-level APIs for developers to quickly create a camera application, such as a barcode scanner and a text scanner.

The API surface works across most Android and iOS devices. While it leverages the capabilities of Android’s Camera2 and iOS’s AVCaptureSession, it is much easier to use and has more features.

Primary Benefits

Dynamsoft Camera Enhancer has the following benefits:

  • It has a set of easy-to-use high-level camera APIs. The number of lines of code for implementing an Android barcode scanner is less than 120.
  • It is a library with cross-platform support for iOS and Android. The APIs of the two platforms are consistent.
  • It can enhance the efficiency and accuracy of scanning with features like frame filtering and frame cropping.

Build an Android Barcode Scanner with Dynamsoft Camera Enhancer

Let’s create a barcode scanner using Dynamsoft Barcode Reader (DBR) and Dynamsoft Camera Enhancer.

  1. Create a new project in Android Studio and configure DBR and DCE.

    There are two ways to include DBR and DCE in your project.

    • Use maven

      Add the following lines to the project’s build.gradle:

        allprojects {
      repositories {
          maven {
              url "https://download2.dynamsoft.com/maven/dce/aar"
          }
          maven{
              url "http://download2.dynamsoft.com/maven/dbr/aar"
          }
      }
  }

      Add the following lines to the module’s build.gradle:

        dependencies {
      implementation 'com.dynamsoft:dynamsoftbarcodereader:8.4.0@aar'
      implementation 'com.dynamsoft:dynamsoftcameraenhancer:1.0.0@aar'
  }

    • Directly use AAR files

      1. Download DBR and DCE. Put the aar files in app\libs.
      2. Add the following lines to the modules’s build.gradle:
        android {
      repositories {
          flatDir {
              dirs 'libs'
          }
      }
  }
  dependencies {
      implementation(name: 'DynamsoftBarcodeReaderAndroid', ext: 'aar')
      implementation(name: 'DynamsoftCameraEnhancerAndroid', ext: 'aar') 
  }

  2. Configure the layout. Add DCE’s CameraView for camera preview and a TextView to display results.

     <?xml version="1.0" encoding="utf-8"?>
 <androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
     xmlns:app="http://schemas.android.com/apk/res-auto"
     xmlns:tools="http://schemas.android.com/tools"
     android:layout_width="match_parent"
     android:layout_height="match_parent"
     tools:context=".MainActivity">

     <com.dynamsoft.dce.CameraView
         android:id="@+id/cameraView"
         android:layout_width="match_parent"
         android:layout_height="match_parent"
         app:layout_constraintBottom_toBottomOf="parent"
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintStart_toStartOf="parent"
         app:layout_constraintTop_toTopOf="parent">

     </com.dynamsoft.dce.CameraView>

     <TextView
         android:id="@+id/resultView"
         android:layout_width="match_parent"
         android:layout_height="150dp"
         android:background="#80000000"
         android:text="Result:"
         android:textAlignment="center"
         android:textColor="@color/white"
         app:layout_constraintBottom_toBottomOf="parent"
         app:layout_constraintEnd_toEndOf="parent"
         app:layout_constraintStart_toStartOf="parent" />

 </androidx.constraintlayout.widget.ConstraintLayout>

  3. Initialize DBR and DCE.

     private void initDBR(){
     try {
         reader = new BarcodeReader();
         com.dynamsoft.dbr.DMLTSConnectionParameters parameters = new com.dynamsoft.dbr.DMLTSConnectionParameters();
         parameters.organizationID = "200001";
         reader.initLicenseFromLTS(parameters, new DBRLTSLicenseVerificationListener() {
             @Override
             public void LTSLicenseVerificationCallback(boolean b, Exception e) {
                 if (!b) {
                     e.printStackTrace();
                 }
             }
         });
     } catch (BarcodeReaderException e) {
         e.printStackTrace();
     }
 }

 private void initDCE(){
     mCameraEnhancer = new CameraEnhancer(MainActivity.this);
     mCameraEnhancer.addCameraView(cameraView);
     com.dynamsoft.dce.DMLTSConnectionParameters info = new com.dynamsoft.dce.DMLTSConnectionParameters();
     info.organizationID = "200001";
     mCameraEnhancer.initLicenseFromLTS(info, new CameraLTSLicenseVerificationListener() {
         @Override
         public void LTSLicenseVerificationCallback(boolean isSuccess, Exception e) {
             if (!isSuccess) {
                 e.printStackTrace();
             }
         }
     });
 }

  4. Set callbacks of DCE. The decoding is made in the onPreviewOriginalFrame callback.

    There are three frame callbacks. You can get the original frame in onPreviewOriginalFrame, filtered frames in onPreviewFilterFrame and cropped frames as well as the original frame in onPreviewFastFrame.

     private void setDCECallback(){
     mCameraEnhancer.addCameraListener(new CameraListener() {
         @Override
         public void onPreviewOriginalFrame(Frame frame) {
             try {
                 TextResult[] results = reader.decodeBuffer(frame.getData(),frame.getWidth(),frame.getHeight(),frame.getStrides()[0],frame.getFormat(),"");
                 showResult(results);
             } catch (BarcodeReaderException e) {
                 e.printStackTrace();
             }
         }
         @Override
         public void onPreviewFilterFrame(Frame frame) {}

         @Override
         public void onPreviewFastFrame(Frame frame) {}
     });
 }
    
 private void showResult(TextResult[] results) {
     if (results != null && results.length > 0) {
         String strRes = "";
         for (int i = 0; i < results.length; i++)
             strRes += results[i].barcodeText + "\n\n";
         resultView.setText(strRes);
     }else{
         resultView.setText("No barcodes found.");
     }
 }

  5. Initialize and start scanning on creation. Stop scanning if the activity is in the background.

     @Override
 protected void onCreate(Bundle savedInstanceState) {
     super.onCreate(savedInstanceState);
     setContentView(R.layout.activity_main);
     cameraView = findViewById(R.id.cameraView);
     resultView = findViewById(R.id.resultView);
     initDBR();
     initDCE();
     setDCECallback();
     mCameraEnhancer.startScanning();
 }
    
 @Override
 public void onResume() {
     mCameraEnhancer.startScanning();
     super.onResume();
 }

 @Override
 public void onPause() {
     mCameraEnhancer.stopScanning();
     super.onPause();
 }

The demo is completed and the number of lines of code is 120.

Android demo

More Camera Scanning Settings

Let’s explore DCE a bit further.

Add Overlay

Enable overlay:

cameraView.addOverlay();

Set result points of detected regions of interest (here we use barcode localization results as an example):

if (results.length==0){
    mCameraEnhancer.setResultPoints(new ArrayList<>());
} else{           
    ArrayList<android.graphics.Point> pointsList = new ArrayList<>();
    //Get result points from DBR results
    for (TextResult result:results){
        for (com.dynamsoft.dbr.Point point:result.localizationResult.resultPoints){
            android.graphics.Point newPoint = new android.graphics.Point();
            newPoint.x = point.x;
            newPoint.y = point.y;
            pointsList.add(newPoint);
        }
    }
    mCameraEnhancer.setResultPoints(pointsList);
}

The detected barcodes will have overlays.

Android demo

Control Focus

DCE has a series of focus control functions(doc).

For example, we can set a point for the camera to focus at with the setManualFocusPosition method.

We can create a TextView with 100% parent width and 100% parent height in the front of the cameraview, set up an OnTouchListener to detect whether the user has touched the screen and the location and then call the method.

The XML:

<TextView
    android:id="@+id/touchView"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toTopOf="parent" />

The Java code:

@SuppressLint("ClickableViewAccessibility")
private void setupTouchEvent(){
    touchView = findViewById(R.id.touchView);
    touchView.setLongClickable(true);
    touchView.setOnTouchListener(new View.OnTouchListener() {
        @SuppressLint("ClickableViewAccessibility")
        @Override
        public boolean onTouch(View v, MotionEvent event) {
            int x = -1, y = -1;
            if (event.getAction() == MotionEvent.ACTION_UP) {
                x = (int) event.getX();
                y = (int) event.getY();
            }
            if (x != -1 && y != -1) {
                mCameraEnhancer.setManualFocusPosition(x, y);
                Log.d("DBR","Manual Focus");
            }
            return true;
        }
    });
}

Frame Filter

Enable Frame Filter to filter out blurry frames.

mCameraEnhancer.enableFrameFilter(true);

Blurry frame filtered:

Blurry frame

Fast Mode

Fast Mode crops frames to improve decoding speed.

The original frame will be cropped to images half in height, half in both height and width.

Original:

Fast mode 1

Half in height:

Fast mode 2

Half in both height and width:

Fast mode 3

The localization results of barcodes in cropped frames have to be updated if we want to set result points.

private void UpdateLocationIfFastFrame(TextResult[] results, Frame frame){
    if (frame.isFastFrame()) {
        Rect rect = frame.getCropRect();
        if (rect != null){
            for (int i = 0; i < results.length; i++) {
                for (int j = 0; j < 4; j++) {
                    ((results[i]).localizationResult.resultPoints[j]).x += rect.left;
                    ((results[i]).localizationResult.resultPoints[j]).y += rect.top;
                }
            }
        }
    }
}

Auto Zoom

If the target is far away, we can zoom in for a close-up.

Auto zoom

DCE has an AutoZoom feature. We can enable it and use setZoomRegion to zoom in.

mCamera.enableAutoZoom(true);

Let’s use barcode scanning as an example. If DBR has detected the barcode region but cannot decode it, we can obtain the localization results from DBR’s intermediate results.

First, we have to make the following runtime settings:

private void updateRuntimeSettings(){
    try {
        PublicRuntimeSettings rs = reader.getRuntimeSettings();
        rs.intermediateResultTypes = EnumIntermediateResultType.IRT_TYPED_BARCODE_ZONE;
        rs.intermediateResultSavingMode = EnumIntermediateResultSavingMode.IRSM_MEMORY;
        rs.resultCoordinateType = EnumResultCoordinateType.RCT_PIXEL;
        reader.updateRuntimeSettings(rs);
    } catch (BarcodeReaderException e) {
        e.printStackTrace();
    }
}

Then, set region to zoom in:

Point[] resultPoints = getResultsPointsWithHighestConfidence(reader.getIntermediateResults());
mCameraEnhancer.setZoomRegion(GetRect(resultPoints,frame),frame.getOrientation()); //auto zoom

Only the result points of the detected barcode which has the highest confidence are used.

private Point[] getResultsPointsWithHighestConfidence(IntermediateResult[] intermediateResults){
    for (IntermediateResult ir:intermediateResults){
        if (ir.resultType == EnumIntermediateResultType.IRT_TYPED_BARCODE_ZONE){
            int maxConfidence = 0;
            for (Object result:ir.results)
            {
                LocalizationResult lr = (LocalizationResult) result;
                maxConfidence = Math.max(lr.confidence,maxConfidence);
                Log.d("DBR", "confidence: "+lr.confidence);
            }
            Log.d("DBR", "max confidence: "+maxConfidence);
            for (Object result:ir.results)
            {
                LocalizationResult lr = (LocalizationResult) result;
                if (lr.confidence == maxConfidence && maxConfidence>80){
                    return lr.resultPoints;
                }
            }
        }
    }
    return null;
}

A GetRect method is used to convert points to Rect. If the frame is cropped (Fast Frame), then the coordinates will be recalculated.

private Rect GetRect(Point[] points, Frame frame) {
    int leftX = (points[0]).x, rightX = leftX;
    int leftY = (points[0]).y, rightY = leftY;
    for (Point pt : points) {
        if (pt.x < leftX)
            leftX = pt.x;
        if (pt.y < leftY)
            leftY = pt.y;
        if (pt.x > rightX)
            rightX = pt.x;
        if (pt.y > rightY)
            rightY = pt.y;
    }
    if (frame.isFastFrame()) {
        int original_w = frame.getOriW();
        int original_h = frame.getOriH();
        if (frame.getFastFrameId() % 4 == 1) {
            if (frame.getOrientation() == 1) {
                leftX += original_w / 4;
                rightX += original_w / 4;
            } else if (frame.getOrientation() == 2) {
                leftY += original_h / 4;
                rightY += original_h / 4;
            }
        } else if (frame.getFastFrameId() % 4 != 0) {
            leftX += original_w / 4;
            rightX += original_w / 4;
            leftY += original_h / 4;
            rightY += original_h / 4;
        }
    }
    Rect frameRegion = new Rect(leftX, leftY, rightX, rightY);
    return frameRegion;
}

Use Frame Queue

There is an inner frame queue which temporarily stores the filtered and cropped video frames. Using a frame queue can improve the decoding speed as shown in this video.

We can get the latest frame from the queue with the following line of code:

mCameraEnhancer.AcquireListFrame(true);

This makes it possible to create a background decoding task.

private Timer timer = new Timer();
@Override
protected void onCreate(Bundle savedInstanceState) {
    //......
    timer.scheduleAtFixedRate(task, 1000, 100);
}

TimerTask task = new TimerTask() {
    @Override
    public void run() {
        Frame frame = mCameraEnhancer.AcquireListFrame(true);
        if (frame != null){
            if (frame.getFrameId() != lastFrameID){
                lastFrameID = frame.getFrameId();
                decode(frame);
            }
        }
    }
};

Use DBR to Control DCE

DBR has the ability to interact with DCE. We can pass DCE to DBR using DBR’s SetCameraEnhancerParam method. It will handle the decoding itself. We don’t need to write code related to result points, auto zoom and frame queue.

//Get the text result from Dynamsoft Barcode Reader
TextResultCallback mTextResultCallback = new TextResultCallback() {
    @Override
    public void textResultCallback(int i, TextResult[] textResults, Object o) {
        showResult(textResults);
    }
};                
//Set DCE setting parameters in Dynamsoft Barcode Reader
DCESettingParameters dceSettingParameters = new DCESettingParameters();
dceSettingParameters._cameraInstance = mCameraEnhancer;
dceSettingParameters._textResultCallback = mTextResultCallback;
//Instantiate DCE, send result and immediate result call back to Dynamsoft Barcode Reader
reader.SetCameraEnhancerParam(dceSettingParameters);

Download the camera enhancer to have a try on your own!

Source Code

https://github.com/xulihang/dynamsoft-samples/tree/main/suite/Android/DCESimple