Input data for TVM C++ API


#1

Hi, there. I am trying to implement the cpp example in the docs/how_to/deploy.md, by running the compiled modules using TVM c++ API, and facing the error regarding the input data.

In the example, it loads the binary input data by

DLTensor* x;
int in_ndim = 4;
int64_t in_shape[4] = {1, 3, 224, 224};
TVMArrayAlloc(in_shape, in_ndim, dtype_code, dtype_bits, dtype_lanes, device_type, device_id, &x);
// load image data saved in binary
std::ifstream data_fin("cat.bin", std::ios::binary);
data_fin.read(static_cast<char*>(x->data), 3 * 224 * 224 * 4);

In my case, I won’t use binary data, but the Mat data from opencv instead (I don’t find the “cat.bin” from the project either). Here is the way I do:

cv::Mat img = cv::imread("test.jpg");
cv::Mat resized_img;
cv::resize(img, resized_img, cv::Size(224,224));
memcpy(static_cast<unsigned char*>(x->data), resized_img.data, 3*224*224);

Since the resized_img.data is in uchar* format, x->data was converted to uchar* as well.

But it gives ‘Segmentation fault (core dumped)’ error while running the memcpy command.

Here are my two questions:

  1. For the cat.bin data, why the data length is 3*224*224*4? More specific, what does the fourth dimension ‘4’ represent here? (generally, 3 (channels) * 224 (width) * 224 (height) * 1 (sizeof(uchar/char)), right?)
  2. What’s the correct way to set the x->data if using the image data from Mat format?

Thanks!


#2

Regarding (1) It looks like the * 4 is because dtype_code is kDLFloat, so each element of the tensor has size 4, and the use of uchar* is just so that sizes can be calculated in units of bytes.

As for setting the x->data element correctly, take a look at the DLTensor struct in DLPack:


The key thing here is that the shape and strides arrays match the data you have. You will need to find some way of unpacking the Mat format into memory (a cursory google suggests a few c++ libs are readily available), and then either construct a DLTensor to correctly map to your in-memory data, or copy your in-memory data into a compact buffer and build a DLTensor for that.

Side note - the header docs for DLTensor say that if strides is NULL, this indicates the tensor is compact. I think this means that the last dimension is innermost, but I can’t actually see this documented anywhere in DLPack. It might be good to figure out what ordering ‘compact’ means and get it added to the docs.

Hope that helps :slight_smile:


#3

Really thanks for your reply! I will try it and update the post later.