How to make the the result of coreml model is correct using mali?


#1

My device is RK3399.I fllow this tutorial:http://nnvm.tvmlang.org/tutorials/from_coreml.html#sphx-glr-tutorials-from-coreml-py.
When I let target = “llvm”, ctx = tvm.cpu(0),the result is ok:(‘Top-1 id’, 287, ‘class name’, ‘lynx, catamount’).
When I let target = tvm.target.mali(), ctx=tvm.cl(0),the result is wrong:(‘Top-1 id’, 0, ‘class name’, ‘‘tench, Tinca tinca’’).
Does anyone have this problem?
Thank you very much!


#2

Can you inspect the output and check the disparity between versions?

e.g., np.allclose(output_cpu.asnumpy(), output_maliclasnumpy())?


#3

My output is:
output_shape = (1000,)
tvm_output = m.get_output(0, tvm.nd.empty(output_shape, dtype)).asnumpy()
Is not ok?
How can I check the disparity between versions?


#4

.asnumpy() will give you numpy array versions of the output so you are free to check however you wish; numpy.allclose is one method.


#5

I check it. When I let target = tvm.target.mali(), ctx=tvm.cl(0), all elements in the tvm_output array are equal to zero.
When I let target = “llvm”, ctx=tvm.cpu(0), all elements in the tvm_output array are not equal to zero.


#6

Ok, that’s interesting. Can you try using the time evaluator to check the run time of the cl version?


#7

The run time of the cl version is 0.117s.
The run time of the cpu version is 1.008s.


#8

This is my coreml.py

import nnvm
import tvm
import coremltools as cm
import numpy as np
from PIL import Image

model_file = ‘mobilenet.mlmodel’
mlmodel = cm.models.MLModel(model_file)

we can load the graph as NNVM compatible model

sym, params = nnvm.frontend.from_coreml(mlmodel)

######################################################################

Load a test image

from PIL import Image
img = Image.open(‘cat.png’).resize((224, 224))
#x = np.transpose(img, (2, 0, 1))[np.newaxis, :]
image = np.asarray(img)
image = image.transpose((2, 0, 1))
x = image[np.newaxis, :]
######################################################################

Compile the model on NNVM

---------------------------

We should be familiar with the process right now.

import nnvm.compiler
#target = ‘llvm’
target_host = “llvm -target=aarch64-linux-gnu -mattr=+neon”
shape_dict = {‘image’: x.shape}
graph, lib, params = nnvm.compiler.build(sym, target=tvm.target.mali(), shape=shape_dict, params=params,target_host=target_host);
print (“build ok”)
######################################################################

Execute on TVM

-------------------

The process is no different from other example

from tvm.contrib import graph_runtime
#ctx = tvm.cpu(0)
ctx = tvm.cl(0)
dtype = ‘float32’
m = graph_runtime.create(graph, lib, ctx)

set inputs

m.set_input(‘image’, tvm.nd.array(x.astype(dtype)))
m.set_input(**params)

execute

m.run()

test

num_test=1
num_warmup=1
print(“warm up…”)
warm_up_timer = m.module.time_evaluator(“run”, ctx, num_warmup)
warm = warm_up_timer()
print warm.mean
print(“test…”)
ftimer = m.module.time_evaluator(“run”, ctx, num_test)
prof_res = ftimer()

print(“cost per image: %.4fs” % prof_res.mean)

print(“backend: TVM-mali\tmodel: %s\tdtype: %s\tcost:%.4fs” % (mlmodel, dtype, prof_res.mean))

get outputs

output_shape = (1000,)
tvm_output = m.get_output(0, tvm.nd.empty(output_shape, dtype)).asnumpy()
print tvm_output
top1 = np.argmax(tvm_output)

synset_name = ‘synset.txt’
with open(synset_name) as f:
synset = eval(f.read())
print(‘Top-1 id’, top1, ‘class name’, synset[top1])


#9

Can you try specifying the target slightly differently; e.g., instead of tvm.target.mali() try opencl -device=mali -model=Mali-T860MP4@800Mhz


#10

I try it:target=“opencl -device=mali -model=Mali-T860MP4@800Mhz”
But all elements in the tvm_output array are also equal to zero.


#11

Looking at the code posted, are you doing module.run()?


#12

Yes,i’m donging module.run().


#13

When I run the program I will uncomment it “m.run()”


#14

Does your mali device run this expamle ok?