How to get multi output from module.get_output()?


#1

Currently, I find that all examples / tutorials are only one output. However, we have some models maybe have multi output layers, for example, ssd mobilenets.

For this situation, how we get multi output?

Additionally, I find that sometimes our output shape is (1000,), sometimes is (1, 1000),both are ok?


#2
    # get nth output with out_shape as output shape
    out = module.get_output(n, out=tvm.nd.empty(out_shape, ctx=ctx))

#3

Thanks. It works. But how about output_shape is different? for example, ssd_mobilenet, our two output_shape are different. We should call two times of module.get_output(0, out=tvm.nd_empty(out_shape_0, ctx=ctx)) and module.get_output(1, out=tvm.nd.empty(out_shape_1), ctx=ctx) or out_shape = (out_shape_0, out_shape_1), then module.get_output(1, out=tvm.nd.empty(out_shape), ctx=ctx)???


#4

tvm.nd.empty(out_shape, ctx=ctx) allocates an array, so you can allocate several arrays for each output. It is recommended not to allocate arrays repetitively, but allocate it once and pass it to out parameter


#5

So, you means I can:
module.get_output(0, out=tvm.nd_empty(out_shape_0, ctx=ctx))
module.get_output(1, out=tvm.nd.empty(out_shape_1), ctx=ctx)


#6

I think @tqchen means

# allocate once
out_0 = tvm.nd.empty(out_shape_0, ctx=ctx)
out_1 = tvm.nd.empty(out_shape_1, ctx=ctx)

# infer for many times
for i in range(...):
    # set input and module.run
    ...
    # pass as arguments
    module.get_output(0, out_0)
    module.get_output(1, out_1)