Introduction

Zero-copy transfer between cv::Mat and TensorFlow is mostly a memory ownership problem, not a syntax trick. The short answer is that you can wrap external memory, but only if the OpenCV buffer layout matches the tensor you want and you keep that memory alive for as long as TensorFlow uses it.

Why a Simple Constructor Is Not Enough

A normal TensorFlow tensor constructor allocates its own buffer. So code like "make a tensor and then point it at mat.data" is not the default behavior.

To avoid copying, you need one of these patterns:

• wrap external memory with an API that accepts a custom deallocator
• create a TensorFlow buffer object that references the cv::Mat memory
• guarantee the original cv::Mat storage remains valid throughout tensor use

If you skip the lifetime rule, zero-copy becomes use-after-free very quickly.

Start by Checking Memory Layout

A zero-copy mapping is only plausible when the cv::Mat is contiguous and its dtype matches the tensor dtype.

Example checks:

1#include <opencv2/opencv.hpp>
2#include <iostream>
3
4int main() {
5    cv::Mat image(224, 224, CV_8UC3);
6
7    std::cout << "continuous: " << image.isContinuous() << "\n";
8    std::cout << "rows: " << image.rows << "\n";
9    std::cout << "cols: " << image.cols << "\n";
10    std::cout << "channels: " << image.channels() << "\n";
11}

If the matrix is not contiguous, TensorFlow cannot safely interpret it as one dense tensor buffer without either copying or reshaping first.

Channel order matters too. OpenCV images are often BGR, while TensorFlow models commonly expect RGB.

A Practical Zero-Copy Pattern with the TensorFlow C API

The C API is a convenient way to wrap externally owned memory because TF_NewTensor accepts a raw pointer and a deallocator callback.

1#include <tensorflow/c/tf_tensor.h>
2#include <opencv2/opencv.hpp>
3#include <memory>
4
5void NoOpDeallocator(void* data, size_t len, void* arg) {
6    auto* holder = static_cast<std::shared_ptr<cv::Mat>*>(arg);
7    delete holder;
8}
9
10int main() {
11    auto mat = std::make_shared<cv::Mat>(224, 224, CV_8UC3);
12
13    int64_t dims[3] = {mat->rows, mat->cols, mat->channels()};
14    auto* holder = new std::shared_ptr<cv::Mat>(mat);
15
16    TF_Tensor* tensor = TF_NewTensor(
17        TF_UINT8,
18        dims,
19        3,
20        mat->data,
21        mat->total() * mat->elemSize(),
22        &NoOpDeallocator,
23        holder
24    );
25
26    TF_DeleteTensor(tensor);
27}

The important trick is that the deallocator owns a shared_ptr to the cv::Mat. That keeps the underlying image buffer alive until TensorFlow releases the tensor.

What This Actually Guarantees

This pattern avoids copying the image bytes into a second buffer. It does not guarantee:

• automatic color conversion
• automatic batch dimension insertion
• compatibility with every TensorFlow C++ helper type

If the model expects shape NHWC, you may still need to interpret the image as 1 x H x W x C conceptually, even if the underlying bytes are unchanged.

For example, the same image buffer may need dimensions:

int64_t dims[4] = {1, mat->rows, mat->cols, mat->channels()};

if the downstream model expects a batch dimension.

Why True Zero-Copy Is Not Always Worth It

Zero-copy sounds ideal, but sometimes a small explicit copy is the safer engineering decision:

• color conversion may already require a new buffer
• resizing often creates a new image anyway
• many pipelines normalize to float32, which requires transformation
• debugging ownership bugs in C++ is expensive

If your preprocessing already converts uint8 BGR into normalized float32 RGB, you are not truly zero-copy anymore, and forcing the issue may only make the code more fragile.

That is why many production inference pipelines copy once into the exact tensor layout they need and optimize elsewhere.

Common Pitfalls

• Wrapping mat.data without keeping the cv::Mat buffer alive long enough.
• Ignoring isContinuous() and assuming every cv::Mat has dense storage.
• Forgetting channel order differences between OpenCV and model expectations.
• Calling the result "zero-copy" even though preprocessing still allocates a converted buffer.
• Confusing external-memory tensors in the C API with ordinary C++ tensor constructors that allocate their own storage.

Summary

• Zero-copy import from cv::Mat to TensorFlow is possible only when the memory layout is compatible and the buffer lifetime is managed correctly.
• The TensorFlow C API makes this practical with TF_NewTensor and a custom deallocator.
• The wrapped OpenCV buffer must stay alive until TensorFlow finishes using it.
• Shape, dtype, batch dimension, and channel order still need to match the model.
• In many real pipelines, one explicit copy is simpler and safer than chasing perfect zero-copy semantics.