TFLite Multipose model input error for android

[Raaed-Ilham]

I have been trying to work with the movenet multipose lightning model. The model gives the following error

java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input:0) with 3 bytes from a Java Buffer with 122880 bytes.

The model, downloaded from the official tfhub/kaggle page ,
requests inputs of (1,1,1,3) of Datatype.UINT8

However in the documentation on the model page it asks for inputs in the form of 1HW*3 where h and w are multiples of 32
ideally 160,256 for example.
I have opened this issue on recommendation from @sushreebarsa after viewing another user with a similar issue given here:

#53127

Currently if 1,1 are given as H and W it results in a bunch of random points on the left hand side of the phone screen which do not seem to move with the user. Any changes to the input size of the image and model only change the latter java buffer size, the tensor image size always remains at 3 bytes in the error messages.
Please let me know if its possible to resolve this issue or if theres something wrong with the model itself. If it is an error with the model could converting the the original model to tflite version with different input parameters be possible somehow.

Relevant code given below

     
   imageProcessor = new ImageProcessor.Builder().add(new ResizeOp(160,256, ResizeOp.ResizeMethod.BILINEAR)).build();
         private AutoModel1 model;

   try {
            model = AutoModel1.newInstance(this);
        }
        catch (IOException e) {

        }
        
      
                public void onSurfaceTextureUpdated(@NonNull SurfaceTexture p0) {
                bitmap = textureView.getBitmap();
                TensorImage tensorImage = new TensorImage(DataType.UINT8);
                tensorImage.load(bitmap);
                tensorImage = imageProcessor.process(tensorImage);


                int[] shape = new int[] {1,160,256,3};


               TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(shape,DataType.UINT8);
               // TensorBuffer inputFeature0 = TensorBuffer.createDynamic(DataType.UINT8);

                Log.d("size of input buffer", String.valueOf(inputFeature0.getFlatSize()));


                inputFeature0.loadBuffer(tensorImage.getBuffer());

                Log.d("Buffer created", "true");

                AutoModel1.Outputs outputs = model.process(inputFeature0);

                Log.d("Model processed", "outputs created");


                TensorBuffer temp =  outputs.getOutputFeature0AsTensorBuffer();

                float[] outputFeature0 = outputs.getOutputFeature0AsTensorBuffer().getFloatArray();



                Bitmap mutable = bitmap.copy(Bitmap.Config.ARGB_8888, true);
                Canvas canvas = new Canvas(mutable);
                int h = bitmap.getHeight();
                int w = bitmap.getWidth();
                int x = 0;


                Log.d("output__", String.valueOf(outputFeature0.length));
                while (x <= 51) {
                    float score_val =  outputFeature0[x+2];

                    Log.d("score values", String.valueOf(Math.abs(score_val)));
                    //if (outputFeature0[x+2] > 0.3f) {
                    canvas.drawCircle(Math.abs(outputFeature0[x+1] * w), Math.abs(outputFeature0[x] * h), 10f, paint);

                     //}
                    x += 3;
                }

                imageView.setImageBitmap(mutable);
            }

[sawantkumar]

Hi @Raaed-Ilham ,

I will replicate the issue and i will get back to you .

[sawantkumar]

Hi @Raaed-Ilham ,

Did you check out this . This example
uses the movenet thunder model similar to yours.