Introduction

If you want to run inference from Android native code instead of the Java or Kotlin API, the practical answer today is usually TensorFlow Lite through its C or C++ API, not the full TensorFlow runtime. The Android NDK can call that native API directly, but you still need to package the model, native library, and build configuration correctly.

Use TensorFlow Lite, Not Full TensorFlow

For Android on-device inference, the full TensorFlow runtime is usually too large and awkward compared with TensorFlow Lite. The native path is therefore typically:

• package a .tflite model
• link the TensorFlow Lite C or C++ library in your NDK build
• run inference from C++
• expose results to the rest of the app through JNI if needed

That is the normal architecture for native Android ML inference today.

A Minimal C++ Inference Example

The C++ API lets you load a flatbuffer model and invoke an interpreter directly.

1#include <iostream>
2#include <memory>
3#include "tensorflow/lite/interpreter.h"
4#include "tensorflow/lite/kernels/register.h"
5#include "tensorflow/lite/model.h"
6
7int main() {
8    auto model = tflite::FlatBufferModel::BuildFromFile("/data/local/tmp/model.tflite");
9    if (!model) {
10        std::cerr << "Failed to load model\n";
11        return 1;
12    }
13
14    tflite::ops::builtin::BuiltinOpResolver resolver;
15    std::unique_ptr<tflite::Interpreter> interpreter;
16    tflite::InterpreterBuilder(*model, resolver)(&interpreter);
17
18    if (!interpreter || interpreter->AllocateTensors() != kTfLiteOk) {
19        std::cerr << "Failed to prepare interpreter\n";
20        return 1;
21    }
22
23    float* input = interpreter->typed_input_tensor<float>(0);
24    input[0] = 1.0f;
25
26    if (interpreter->Invoke() != kTfLiteOk) {
27        std::cerr << "Inference failed\n";
28        return 1;
29    }
30
31    const float* output = interpreter->typed_output_tensor<float>(0);
32    std::cout << "Output: " << output[0] << "\n";
33    return 0;
34}

This is the kind of native code you run from the NDK side.

CMake Integration

Your native project needs to link the TensorFlow Lite library.

1cmake_minimum_required(VERSION 3.22)
2project(native_inference)
3
4add_library(native_inference SHARED native_inference.cpp)
5
6find_library(log-lib log)
7
8add_library(tensorflowlite SHARED IMPORTED)
9set_target_properties(tensorflowlite PROPERTIES
10    IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/src/main/jniLibs/${ANDROID_ABI}/libtensorflowlite.so
11)
12
13target_link_libraries(native_inference tensorflowlite ${log-lib})

The exact library path depends on how you package TensorFlow Lite into the project, but this shows the overall shape.

JNI Is Still Common Even If Inference Is Native

"Not using the Java API" usually means inference runs in native code, not that the entire Android app avoids Java or Kotlin. Many apps still use JNI as the boundary between the Android framework layer and the C++ inference layer.

That hybrid setup is normal because the Android app lifecycle, permissions, UI, and asset handling are still often easier at the managed layer.

Model and Input Handling Matter More Than the Language Boundary

Running inference from C++ does not remove the need to handle:

• model input shape and type
• preprocessing consistency
• output tensor interpretation
• delegate compatibility such as CPU, GPU, or NNAPI paths

The native API changes where the code lives, not the core ML correctness requirements.

When Native Inference Is Worth It

Using the NDK side directly can make sense when:

• your app already has a large native codebase
• you want to keep preprocessing and inference in C++
• JNI crossings would otherwise be frequent and awkward
• you are sharing inference logic with other native platforms

If your app is mostly Kotlin or Java, the managed TensorFlow Lite API may still be simpler overall.

Common Pitfalls

The biggest pitfall is trying to embed the full TensorFlow runtime on Android when the actual use case is ordinary on-device inference. TensorFlow Lite is usually the intended solution.

Another issue is assuming native inference means zero Java or Kotlin integration. In practice, many apps still use JNI around the native inference core.

Developers also forget ABI packaging and end up shipping libraries for the wrong architecture or missing some required native artifacts.

Finally, do not debug only the build system. Many Android ML problems are actually input-shape, preprocessing, or model-conversion issues rather than NDK issues.

Summary

• For Android NDK inference, use TensorFlow Lite native APIs rather than the full TensorFlow runtime in most cases.
• Load the .tflite model in C++ and run inference through a native interpreter.
• Link the TensorFlow Lite library through CMake and package the correct ABI-specific binaries.
• JNI is still a common and reasonable bridge to the Android app layer.
• The hard parts remain model packaging, preprocessing, and output handling, not just the language boundary.