glReadPixels does not support offset parameter crucial to read pixels asynchronously

[LobbyDivinus]

Overview

When you normally use glReadPixels to grab one or multiple pixel colors from the screen the GPU and CPU have to sync up in order to fulfill the request. This is expensive as it basically stalls the GPU for some precious milliseconds, even if we want to read just a single pixel.

To solve this, in OpenGL ES 3.0 PixelBufferObjects (PBOs) were introduced. These allow to read pixel data asynchronously. basically:

1. Making the request to read the data
2. Coming back later to actually read the data, e.g. in the next frame

Some Overview about using PBOs

Especially in a mobile environment it makes sense to use PBOs if possible.

The issue

In an ideal world, the mapping of OpenGL API into the Java world is complete and supports all specified functionality.

However, that seems to not be the case. More specifically, the function

void glReadPixels( GLint x,
   GLint y,
   GLsizei width,
   GLsizei height,
   GLenum format,
   GLenum type,
   void * data);

should - according to the docs - allow to specify an offset (that is, an integer) for the data parameter. This offset is then used as an offset inte the currently bound PBO, if any is bound.

However, in GL30 we only have the following corresponding method:

public void glReadPixels (int x, int y, int width, int height, int format, int type, Buffer pixels);

Which crashes when we try to put anything else than a Buffer as value for pixels.

Implementation

I noticed the commit 605e636
that does add the requested functionality but for glTexImage2D and glTexSubImage2D. So I guess this could be used as a guide on how to do it.

Desktop

On Desktop the lwjgl function in GL11:

    public static void nglReadPixels(int x, int y, int width, int height, int format, int type, long pixels) {

could be used.

Android

On Android the GLES30 function

    public static native void glReadPixels(int x, int y, int width, int height, int format, int type, int offset);

could be used. However, note that this is only available with Android 7.0 aka API 24 aka Nougat. So basically, Android API had the same issue as the current libGDX API for a long time. I guess the only solution to support the function on older versions of Android would be to implement it via jni.

iOS

I haven't looked into it, but I assume that code has to be added in jni/iosgl to make this work.

Version of libGDX and/or relevant dependencies

1.12.1

Please select the affected platforms

All of them, since the GL30 interface does not support it.

[Tom-Ski]

Have you checked out PixelBufferObjectTest?

[LobbyDivinus]

I suppose you mean this one: PixelBufferObjectTest.java

That is a nice example for how to use PBOs to upload textures asynchronouly and indicates that the offset parameter for glTexImage2D and glTexSubImage2D was added for that specific reason. However, I don't think it helps with reading pixel data.