Unable to draw texture in gtk4-rs application

[faern]

I have been able to render OpenGL nicely in GLAreas in a gtk4-rs application in general. But using textures simply does not work for me. I reported this to gtk4-rs, but I figured the issue might be with glium, so I report it here also.

I get the image texture examples from glium to run just fine! But when I port that exact same glium code over to render in a GTK4 application, the textures simply don't show up. It's as if the texture(...) function in the fragment shader just returns transparent. The GL context has DebugCallbackBehavior::DebugMessageOnError and no errors or warnings are printed.

I have created a single-file example of my problem (plus corresponding Cargo.toml). This example is initially based off of the glium image example merged with the gtk4-rs OpenGL example.

Cargo.toml

[package]
name = "gtk_glium_texture2"
edition = "2021"

[dependencies]
gtk4 = "0.8.0"
glium = "0.34.0"
gl_loader = "0.1.2"
image = "0.25"

main.rs

use core::ffi::c_void;
use glium::backend::{Backend, Context, Facade};
use glium::debug::DebugCallbackBehavior;
use glium::index::{NoIndices, PrimitiveType};
use glium::texture::{RawImage2d, Texture2d};
use glium::{
    implement_vertex, program, uniform, Frame, IncompatibleOpenGl, Program, Surface,
    SwapBuffersError, VertexBuffer,
};
use gtk4::prelude::*;
use gtk4::{Application, ApplicationWindow, GLArea};
use std::io::Cursor;
use std::rc::Rc;
use std::time::Duration;

fn main() {
    let application = Application::builder()
        .application_id("com.example.Gtk4GliumTexture")
        .build();

    application.connect_activate(|app| {
        let window = ApplicationWindow::builder()
            .application(app)
            .title("Texture")
            .default_width(600)
            .default_height(400)
            .build();

        let glarea = GLArea::builder().vexpand(true).build();

        window.set_child(Some(&glarea));
        window.show();

        let facade = GtkFacade::from_glarea(&glarea).unwrap();

        let opengl_texture = load_texture(&facade);
        let vertex_buffer = create_rectangle_buffer(&facade);
        let program = create_program(&facade);

        glarea.connect_render(move |_glarea, _glcontext| {
            let context = facade.get_context();

            let uniforms = uniform! {
                matrix: [
                    [1.0, 0.0, 0.0, 0.0],
                    [0.0, 1.0, 0.0, 0.0],
                    [0.0, 0.0, 1.0, 0.0],
                    [0.0, 0.0, 0.0, 1.0f32]
                ],
                tex: &opengl_texture
            };

            let mut frame = Frame::new(context.clone(), context.get_framebuffer_dimensions());
            frame.clear_color(0.0, 0.3, 0.0, 1.0);
            frame
                .draw(
                    &vertex_buffer,
                    NoIndices(PrimitiveType::TriangleStrip),
                    &program,
                    &uniforms,
                    &Default::default(),
                )
                .unwrap();
            frame.finish().unwrap();
            gtk4::glib::signal::Propagation::Proceed
        });

        // This makes the GLArea redraw 60 times per second
        let frame_time = Duration::new(0, 1_000_000_000 / 60);
        gtk4::glib::source::timeout_add_local(frame_time, move || {
            glarea.queue_draw();
            gtk4::glib::ControlFlow::Continue
        });
    });

    application.run();
}

#[derive(Copy, Clone)]
pub struct TexVertex {
    position: [f32; 2],
    tex_coords: [f32; 2],
}
implement_vertex!(TexVertex, position, tex_coords);

fn create_rectangle_buffer<F: Facade>(context: &F) -> VertexBuffer<TexVertex> {
    glium::VertexBuffer::new(
        context,
        &[
            TexVertex {
                position: [-0.9, 0.9],
                tex_coords: [0.0, 1.0],
            },
            TexVertex {
                position: [0.9, 0.9],
                tex_coords: [1.0, 1.0],
            },
            TexVertex {
                position: [-0.9, -0.9],
                tex_coords: [0.0, 0.0],
            },
            TexVertex {
                position: [0.9, -0.9],
                tex_coords: [1.0, 0.0],
            },
        ],
    )
    .unwrap()
}

fn load_texture<F: Facade>(context: &F) -> Texture2d {
    let image = image::load(
        Cursor::new(&include_bytes!("opengl.png")[..]),
        image::ImageFormat::Png,
    )
    .unwrap()
    .to_rgba8();

    let image_dimensions = image.dimensions();
    let image = RawImage2d::from_raw_rgba_reversed(&image.into_raw(), image_dimensions);

    Texture2d::new(context, image).unwrap()
}

fn create_program<F: Facade>(context: &F) -> Program {
    program!(context,
        140 => {
            vertex: "
                #version 140

                uniform mat4 matrix;
                in vec2 position;
                in vec2 tex_coords;

                out vec2 v_tex_coords;

                void main() {
                    gl_Position = matrix * vec4(position, 0.0, 1.0);
                    v_tex_coords = tex_coords;
                }
            ",
            fragment: "
                #version 140

                uniform sampler2D tex;
                in vec2 v_tex_coords;

                out vec4 f_color;

                void main() {
                    f_color = texture(tex, v_tex_coords);

                    // Just setting the color draws a rectangle.
                    // So everything in the setup seems to work, except sampling the texture
                    // f_color = vec4(0.5, 0.1, 0.2, 1.0);
                }
            "
        },
    )
    .unwrap()
}

// === glium to gtk4 helper glue code below ===

struct GLAreaBackend {
    glarea: GLArea,
}

unsafe impl Backend for GLAreaBackend {
    fn swap_buffers(&self) -> Result<(), SwapBuffersError> {
        // GTK swaps the buffers after each "render" signal itself
        Ok(())
    }

    unsafe fn get_proc_address(&self, symbol: &str) -> *const c_void {
        gl_loader::get_proc_address(symbol) as *const _
    }

    fn get_framebuffer_dimensions(&self) -> (u32, u32) {
        let allocation = self.glarea.allocation();

        // On high-resolution screens, the number of pixels in the frame buffer
        // is higher than the allocation. This is indicated by the scale
        // factor.
        let scale = self.glarea.scale_factor();

        (
            (allocation.width() * scale) as u32,
            (allocation.height() * scale) as u32,
        )
    }

    fn resize(&self, _: (u32, u32)) {}

    fn is_current(&self) -> bool {
        // GTK makes OpenGL current itself on each "render" signal
        true
    }

    unsafe fn make_current(&self) {
        self.glarea.make_current();
    }
}

impl GLAreaBackend {
    fn new(glarea: GLArea) -> Self {
        Self { glarea }
    }
}

pub struct GtkFacade {
    context: Rc<Context>,
}

impl GtkFacade {
    pub fn from_glarea(glarea: &GLArea) -> Result<Self, IncompatibleOpenGl> {
        gl_loader::init_gl();

        let context = unsafe {
            Context::new(
                GLAreaBackend::new(glarea.clone()),
                true,
                DebugCallbackBehavior::DebugMessageOnError,
            )
        }?;

        Ok(Self { context: context })
    }
}

impl Facade for GtkFacade {
    fn get_context(&self) -> &Rc<Context> {
        &self.context
    }
}

And you of course also need an image to load. Grab any PNG and put in the src/ directory.

I also reported it to the small but nice helper library gtk4-glium: remcokranenburg/gtk4-glium#2

This might be related to #2017 since the symptoms are similar. But I don't know.

[Tomiyou]

I also encountered this exact same issue. I did notice something however, if I load the texture each time in the GTKs render loop (draw() function) without caching it, the issue disappears and the texture is successfully displayed:

pub fn draw(&mut self) {
    let mut frame = Frame::new(
        self.context.clone(),
        self.context.get_framebuffer_dimensions(),
    );

    //////////////////////////////////////////////////
    // Not cached - THIS WORKS
    let image = RawImage2d::from_raw_rgba_reversed(image, dimensions);
    let texture = CompressedTexture2d::new(context, image).unwrap();

    // Cached - THIS DOESN'T WORK
    if self.texture.is_none() {
        let image = RawImage2d::from_raw_rgba_reversed(image, dimensions);
        let texture = CompressedTexture2d::new(context, image).unwrap();
        self.texture = Some(texture);
    }
    let texture = self.opengl_texture.as_ref().unwrap();
    //////////////////////////////////////////////////

    let uniforms = uniform! {
        tex: texture,
    };
    frame.clear_color(0., 0., 0., 1.);
    frame.draw(...).unwrap();
    frame.finish().unwrap();
}

Does not matter if I load the texture in init function or during the render itself, as long as you cache it (in other words, try and reuse texture between re-renders) the image will NOT be displayed.

Interestingly, when trying to cache the texture (so when it does not work), OpenGL debug messages with print this

Source: Api             Severity: Notification          Type: Performance               Id: 1                                                                                                   
Disabling CCS because a renderbuffer is also bound for sampling.                                                                                                                                
                                                                                                                                                                                                
Source: Api             Severity: Notification          Type: Performance               Id: 1                                                                                                   
Disabling CCS because a renderbuffer is also bound for sampling.                                                                                                                                
                                                                                                                                                                                                
Source: Api             Severity: Notification          Type: Performance               Id: 1                                                                                                   
Disabling CCS because a renderbuffer is also bound for sampling.                                                                                                                                

This is 100% related to the issue, but I don't have enough OpenGL knowledge to debug it.

[faern]

Yes, glium and gtk4 seem to compete for the global OpenGL state and overwrite each others' textures. I reported this issue to the gtk4-glium crate and the discussion/debugging got a bit further in that thread: remcokranenburg/gtk4-glium#2