What exactly do OrtArena, Ort::MemoryInfo::CreateCpu and Ort::Value::CreateTensor do?

[mmaras1999]

Hi,
I was wondering how exactly does Ort::Value::CreateTensor work. I have a model that I'm running with CUDA provider, I want to avoid copying data as much as possible.

If I understand correctly, the Ort::Value is some wrapper for data. I'm currently creating it like this:

`auto memory_info = Ort::MemoryInfo::CreateCpu(
OrtAllocatorType::OrtArenaAllocator,
OrtMemType::OrtMemTypeDefault
);

    Ort::Value input_tensor = Ort::Value::CreateTensor<float>(
        memory_info, 
        input_tensor_data.data(), 
        BATCH_SIZE * INPUT_SIZE,
        input_shape.data(),
        input_shape.size()
    );

    Ort::Value output_tensor = Ort::Value::CreateTensor<float>(
        memory_info, 
        output_tensor_data.data(), 
        BATCH_SIZE * OUTPUT_SIZE,
        output_shape.data(),
        output_shape.size()
    );`

So I was wondering, what exactly does it do with the data? I'm assuming that it's not copying the buffers, but I don't really get the MemoryInfo argument importance.

From what I've understood, the MemoryInfo suggests where the data buffer resides, in my case on CPU. But what exactly do the OrtArenaAllocator do? Why does it need to allocate any data whatsoever? Does it actually copy the data from the buffer somewhere?

[gramalingam]

Hi, I think it would be better to ask this question in the onnxruntime repo.

[mmaras1999]

Yeah, I had two repos opened and wrote the question in the wrong one facepalm. Thanks for pointing out.