OpenCV in Android – An Introduction (Part 1/2)

Hello world! I am very excited to write this particular blog on the setup of OpenCV in Android Studio. There are many solutions there online which include setting up OpenCV using Eclipse, Android NDK etc but I didn’t find a single reliable source for doing the same setup using Android Studio. So, we (Me and V.Avinash) finally came up with a feasible solution with which you can setup Native Development setup in Android environment for designing Computer Vision applications using OpenCV and C++!!!

A quick intro about me, I am a Computer Vision enthusiast with nearly 4 years of theoretical and practical experience in the field. That said, I am quite good at implementing CV algorithms on Matlab and Python. But with years, the same field has been developing rapidly from the mere academic interest to industrial interest. But most of the standard algorithms in this field are not really optimized to run in real-time (60 FPS) or not designed specifically for the mobile platform. This has caught my interest and I have been working on this since the Summer 2016. I think about various techniques and hacks for optimizing the existing algorithms for mobile platform and how to acquire (and play with) 3D data from the 2D camera during my free time from being a research assistant.

Before starting this project, I am assuming that you already have basic setup of Android Studio up and running on your machines and you have decent experience working on it.

  • If you don’t already have Android Studio, you can download and install it from the following link.
  • Once you have the Android Studio up and running, you can download OpenCV for Android from the following link. After downloading, extract the contents from the zip file and move it to a specific location. Let it be ‘/Users/user-name/OpenCV-android-sdk’. I am currently using Android Studio v2.2.3 and OpenCV v3.2
  • Now start the Android Studio and click on ‘Start a new Android Studio project’. This will open a new window. Specify your ‘Application Name’, ‘Company Domain’ and ‘Project Location’. Make sure you select the checkbox ‘Include C++ Support‘. Now click Next!
  • In the ‘Targeted Android Devices’ window, select ‘Phone and Tablet’ with Minimum SDK: ‘API 21: Android 5.0 (Lollipop)’. Click Next.screen-shot-2017-02-27-at-6-07-56-pm
  • In the Activity selection window select ‘Empty Activity’ and click Next.screen-shot-2017-02-27-at-6-34-08-pm
  • In the Activity customization window leave everything as it is without any edits and click Next.screen-shot-2017-02-27-at-6-37-17-pm
  • In the Customize C++ Support, select C++ Standard: Toolchain Default and leave all the other checkboxes unchecked (for now, but you are free to experiment) and click Finish!
  • The Android Studio will take some time to load the project with necessary settings. Since you are developing an app that depends on Camera of your mobile, you can’t test these apps on an emulator. You need to connect your Android Phone (with developer options enabled) to computer and select the device when you pressed the debug option. After running the application, you should see the following on your mobile if everything works fine!screenshot_20170227-184805
  • At this point of the project you have your basic native-development (C++ support) enabled in your app. Now let us start integrating OpenCV into your application.
  • Click on File -> New -> Import Module. In the pop-up window, give path to your ‘OpenCV-android-sdk/sdk/java’ directory and click on OK. You can find the module name as ‘openCVLibrary320’ and click Next, Finish to complete the importing.
  • Now, go to “openCVLibrary320/build.gradle” and change the following variables to those in the “app/build.gradle”: compileSdkVersion, buildToolsVersion, minSdkVersion, and targetSdkVersion. Sync the project after editing the gradle files. My “openCVLibrary320/build.gradle” file looks like this!
apply plugin: 'com.android.library'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.2"

    defaultConfig {
        minSdkVersion 21
        targetSdkVersion 25
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}
  • Add a new folder named ‘jniLibs’ to “app/src/main/” by right click -> New -> Directory. Copy and paste the directories in the ‘OpenCV-android-sdk/sdk/native/libs/’ to jniLibs folder in your app. After the import, remove all *.a files from your imported directories. At the end, you should have 7 directories with libopencv_java3.so files in them.Screen Shot 2017-02-28 at 8.08.39 AM.png
  • Now go to ‘app/CMakeLists.txt’ and link the OpenCV by doing the following (Refer to those lines following the [EDIT] block for quick changes):
# library. You should either keep the default value or only pass a
# value of 3.4.0 or lower.

cmake_minimum_required(VERSION 3.4.1)

# [EDIT] Set Path to OpenCV and include the directories
# pathToOpenCV is just an example to how to write in Mac.
# General format: /Users/user-name/OpenCV-android-sdk/sdk/native
set(pathToOpenCV /Users/sriraghu95/OpenCV-android-sdk/sdk/native)
include_directories(${pathToOpenCV}/jni/include)

# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds it for you.
# Gradle automatically packages shared libraries with your APK.

add_library( # Sets the name of the library.
             native-lib

             # Sets the library as a shared library.
             SHARED

             # Provides a relative path to your source file(s).
             # Associated headers in the same location as their source
             # file are automatically included.
             src/main/cpp/native-lib.cpp )

# [EDIT] Similar to above lines, add the OpenCV library
add_library( lib_opencv SHARED IMPORTED )
set_target_properties(lib_opencv PROPERTIES IMPORTED_LOCATION /Users/sriraghu95/Documents/Projects/ComputerVision/OpenCVAndroid-AnIntroduction/app/src/main/jniLibs/${ANDROID_ABI}/libopencv_java3.so)

# Searches for a specified prebuilt library and stores the path as a
# variable. Because system libraries are included in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.

find_library( # Sets the name of the path variable.
              log-lib

              # Specifies the name of the NDK library that
              # you want CMake to locate.
              log )

# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in the
# build script, prebuilt third-party libraries, or system libraries.

target_link_libraries( # Specifies the target library.
                       native-lib

                       # Links the target library to the log library
                       # included in the NDK.
                       ${log-lib} lib_opencv) #EDIT

  • Edit the ‘app/build.gradle’ set the cppFlags and refer to jniLibs source directories and some other minor changes, you can refer to the code below and replicate the same for your project. All new changes made on the pre-existing code are followed by comments “//EDIT”.
apply plugin: 'com.android.application'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.2"
    defaultConfig {
        applicationId "com.example.sriraghu95.opencvandroid_anintroduction"
        minSdkVersion 21
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags "-std=c++11 -frtti -fexceptions" //EDIT
                abiFilters 'x86', 'x86_64', 'armeabi', 'armeabi-v7a', 'arm64-v8a', 'mips', 'mips64' //EDIT
            }
        }
    }

    //EDIT
    sourceSets {
        main {
            jniLibs.srcDirs = ['/Users/sriraghu95/Documents/Projects/ComputerVision/OpenCVAndroid-AnIntroduction/app/src/main/jniLibs'] //EDIT: Use your custom location to jniLibs. Path given is only for example purposes.
        }
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }
}

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.1.1'
    testCompile 'junit:junit:4.12'
    compile project(':openCVLibrary320') //EDIT
}
  • Once you are done with all the above steps, do sync the gradle and go to src/main/cpp/native-lib.cpp . To make sure that the project setup is done properly, start including OpenCV files in native-lib.cpp and it should not raise any errors.
#include <jni.h>
#include <string>
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/features2d/features2d.hpp>

extern "C"
jstring
Java_com_example_sriraghu95_opencvandroid_1anintroduction_MainActivity_stringFromJNI(
        JNIEnv *env,
        jobject /* this */) {
    std::string hello = "Hello from C++";
    return env->NewStringUTF(hello.c_str());
}
  • Now make sure all your gradle files are in perfect sync and Rebuild the project once to check there are no errors in your setup.

By the end of this blog, we finished setting up OpenCV in your android project. This is a pre-requisite for any type of android application you want to build using OpenCV. Considering that there will be two types of possibilities using OpenCV in your application: a) Doing processing on images from your own personal library on mobiles and b) Doing real-time processing on live-feed from camera, I think this is best place to stop this part of the blog.

In my next post, I will be focusing on how to use camera in your application and do some simple processing on the data that you acquire from it.

Next: OpenCV in Android – An Introduction (Part 2/2)

Source Code: Link

[New] Android Application: Link

Advertisements

15 thoughts on “OpenCV in Android – An Introduction (Part 1/2)

      1. If you use relative paths in build.gradle and CMakeLists.txt to jniLibs and/or OpenCV libraries, then it will raise build errors. Check that your paths are absolute and are relevant to your OS as the paths that I mentioned in the blog above are for MacOS. 🙂

        Like

  1. Error:error: ‘/src/main/jniLibs/arm64-v8a/libopencv_java3.so’, needed by ‘../../../../build/intermediates/cmake/debug/obj/arm64-v8a/libnative-lib.so’, missing and no known rule to make it
    Have you never seen this error while you did opencv setup ?

    Like

    1. That is mainly due to the paths you gave in the CMakeLists.txt file. They should not be relative. The paths I gave are mainly for the MacOS, you might need to use different paths for Windows/Linux.

      Like

    2. I also get the same error (except for the additional ../../../../ for libopencv_java3.so. I’ve check the CMakeLists.txt and paths are absolute. I’m using Ubuntu 16.04

      Execution failed for task ‘:app:externalNativeBuildDebug’.
      > Build command failed.
      Error while executing process /home/epatto/Android/Sdk/cmake/3.6.4111459/bin/cmake with arguments {–build /home/epatto/AndroidStudioProjects/ImageViewExampleMkyong/ImageViewFromScratch2/app/.externalNativeBuild/cmake/debug/arm64-v8a –target native-lib}
      ninja: error: ‘../../../../jniLibs/src/main/jniLibs/arm64-v8a/libopencv_java3.so’, needed by ‘../../../../build/intermediates/cmake/debug/obj/arm64-v8a/libnative-lib.so’, missing and no known rule to make it

      Like

  2. I am getting a dlopen failed error on libopencv_java3.so. I tried installing the Opencv manager
    adb install OpenCV_3.2.0_Manager_3.20_arm64-v8a.apk
    but that didn’t work. 😦
    The opencv manager says the .so is installed in /data/app.org.opencv.engine-1/lib/arm64. Could it be that the .so needs to be in /system/lib64?

    FATAL EXCEPTION: main
    Process: com.example.epatto.imageviewfromscratch, PID: 16093
    java.lang.UnsatisfiedLinkError: dlopen failed: library “libopencv_java3.so” not found
    at java.lang.Runtime.loadLibrary0(Runtime.java:977)
    at java.lang.System.loadLibrary(System.java:1530)

    Like

    1. Hi, Thanks for following our blog. From reading your series of comments what I can understand is that your system might not support arm64 build. Try removing the directory related to arm64 under jniLibs and try building again.

      Like

      1. Thanks for the reply. I thought that may be the issue too. I am using a Snapdragon 820 EVM. From the documentation I think its a 64 bit. I also added code
        String abiName = Build.SUPPORTED_ABIS[0];
        Log.i(TAG, “Device ABI = ” + abiName);
        That outputs
        Device ABI = arm64-v8a
        So I’m pretty sure its an arm64.

        Adding opencv is not working yet, but with your post I’m further than I’ve gotten before.
        I also tried adding static libraries, but now I get an error

        add_library( lib_opencv_core STATIC IMPORTED )
        set_target_properties(lib_opencv_core PROPERTIES IMPORTED_LOCATION /home/epatto/AndroidStudioProjects/ImageViewExampleMkyong/ImageViewFromScratch2/app/jniLibs/${ANDROID_ABI}/libopencv_core.a)

        and the same “add_library” for libopencv_imgproc.a and libopencv_features2d.a

        BUT, at this code

        mRgba = new Mat(height, width, CvType.CV_8UC4);

        I get a run time error on

        E/art: No implementation found for long org.opencv.core.Mat.n_Mat(int, int, int) (tried Java_org_opencv_core_Mat_n_1Mat and Java_org_opencv_core_Mat_n_1Mat__III)

        It seems java can’t see the static libraries either, or maybe I need to add the full suite of .a libs in the app/jniLibs/arm64-v8a. It seems strange that “libopencv_core.a” wouldn’t include “Mat”. I’ll try that next.

        Like

      2. I didn’t face this issue before and I will get back to you if I find a fix. I am placing it open on the blog to see if other viewers have anything to comment about this. 🙂

        Like

      3. I found a way to make it work. I backed out of using a static library and tried the dynamic library again. Since it appeared my app couldn’t find the opencv lib (see “dlopen failed” issue above), I copied the libopencv_java3.so from the opencv 3.2 install to /system/lib64. (it was a last ditch attempt to get this to work)……

        $adb remount
        adb push ./OpenCV-android-sdk-3-2-0/sdk/native/libs/arm64-v8a/libopencv_java3.so /system/lib64

        A better solution would be to somehow use the Android Studio gradles to tell runtime to look for the libopencv_java3.so in the location where the OpenCV Manager has it stored:

        /data/app/org.opencv.engine-1/lib/arm64

        It could be that the Intrinsycs Snapdragon 820 that I have doesn’t have all the bells and whistles installed that’s on a mature tablet or phone.

        Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s