Google’s ML kit is the new firebase SDK that provides machine learning power to Android and iOS apps. The tool set includes: label image, text recognition, face detection, landmark recognition, barcode scanning and smart reply (coming soon at the time of writing).

In this tutorial, I will show you how to set up the image labelling function on Android using Kotlin.

Step 1: Groundworks

First of all, you will need to sign up a firebase account and have your project registered with it (link). After that, you should have the google-service.json file, save it in your project, and next thing is to build the Gradle file.

In the project level gradle, add following:

allprojects {
    repositories {
        maven { url "https://maven.google.com" }      
    }
}

In the module level gradle, add following:

dependencies { implementation 'com.google.firebase:firebase-core:15.0.0'
    implementation 'com.google.firebase:firebase-ml-vision:15.0.0'
    implementation 'com.google.firebase:firebase-ml-vision-image-   label-model:15.0.0'
}

//add following line at the end
apply plugin: 'com.google.gms.google-services'

There are two types of image-label API: on-device and on-cloud. Google suggested that on-cloud returns a better result, but to use it, you will need Blaze-level(pay-as-you-go) firebase price plan, and enable your project with Cloud Vision API. In this tutorial, I will show you the code for both.

In order to provide a better user experience for on-device mode, I would suggest to add following code in Manifest, it enables the app to automatically download the ML model after installation.

<application<meta-data
       android:name="com.google.android.gms.vision.DEPENDENCIES"
       android:value="label" />
   <!--or sandroid:value="label,face"/> for multi models purpose-->
</application>

Step 2: Application class

Add following code in your Application class:

val firebaseOptions = FirebaseOptions.Builder()
        .setApiKey(API_KEY_FIREBASE)
        .setApplicationId(APP_ID).build()
FirebaseApp.initializeApp(this, firebaseOptions, APP_NAME)

You should find the API_KEY and APP_ID in your google-service.json file.

Step 3: Creating Detector Objecthttps://spltech.co.uk/media/cb231dc9dcf6e37197456c9b7db2669d

Step 4: Detecting the imagehttps://spltech.co.uk/media/3512f73fce3d9c0e20b3f9b8bc00c5f3

Full code can be found here including layout.xml and UI settings.

I have run the code, and result comes back not 100% satisfy, but I am 80% happy with it. This api is probably more efficient and quicker to set up comparing to the Tensorflow Lite or Tensorflow Android SDK. Here are some of my running result:

Thanks for reading this article, if you are interested in how Tensorflow model works and how to train your own model, please see my other article: TensorFlow Image Recognition: train your own model