Renderbuffer

[iceiix]

glTexImage2DMultisample is officially unavailable in glow (porting in #262) because it is not available on WebGL, see: #262 (comment) - as a replacement should either:

• (merged in Disable multisampled textures, glTexImage2DMultisample #447) disable multisampling (glTexImage2D), or
• replace multisample texture with a multisample renderbuffer (glRenderbufferStorageMultisample)

See also this previous issue (one of the earliest fixed) with NUM_SAMPLES: #5 (comment)

or a third option (temporary to unblock #262):

• (workaround implemented) 3. bypass glow for the glTexImage2DMultisample call, calling the OpenGL backend directly (in gl46.rs)

[iceiix]

Updated the wrappers to use glow added in #262, but there's still a lot of work to be done to switch over to use multisample renderbuffers.

Once it is done, can remove the use of the custom fork of glow iceiix/glow#1 and try to run on WebGL (#171).

References:

• https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexImage2D.xhtml
• https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glTexImage2DMultisample.xhtml
• https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glRenderbufferStorageMultisample.xhtml
• https://learnopengl.com/Advanced-OpenGL/Anti-Aliasing
• https://open.gl/framebuffers
• https://stackoverflow.com/questions/2213030/whats-the-concept-of-and-differences-between-framebuffer-and-renderbuffer-in-op - framebuffer: containers with pointers, renderbuffer: render to in native data format (faster than textures)
• https://www.khronos.org/opengl/wiki/Framebuffer_Object
• https://stackoverflow.com/questions/30427944/how-to-render-color-and-depth-in-multisample-texture
• https://www.khronos.org/opengl/wiki/Texture_Storage
• https://stackoverflow.com/questions/24072447/cannot-render-to-texture-with-multisampling
• https://stackoverflow.com/questions/47934444/webgl-framebuffer-multisampling - outlines the required flow
• Glow #262 (comment) - API calls to replace

[iceiix]

With #262 #444, this problem now manifests itself with this error running under wasm, WebGL: ERROR: 0:7: 'sampler2DMS' : Illegal use of reserved word:

sampler2DMS will also have to be removed for WebGL. https://learnopengl.com/Advanced-OpenGL/Anti-Aliasing explains:

It is possible to directly pass a multisampled texture image to a fragment shader instead of first resolving it. GLSL gives us the option to sample the texture image per subsample so we can create our own custom anti-aliasing algorithms.

...this passing of the multisampled texture image to the fragment shader is an implementation of a custom anti-aliasing algorithm, texelFetch(tcolor, C, i) fetching the ith subsample:

// src/render/shaders/trans_frag.glsl
uniform sampler2DMS tcolor;

uniform int samples;
...
    for (int i = 1; i < samples; i++) {
        col += texelFetch(tcolor, C, i);
    }
    col /= float(samples);

do we really need a custom AA algorithm? It was in Steven since the earliest commits.

ryanisaacg/quicksilver#476 has some more background: "WASM doesn't support MSAA except when rendering to a framebuffer", including http://www.realtimerendering.com/blog/webgl-2-new-features/ "New features: Multisampled Renderbuffers".

Previously, if we wanted antialiasing we would either have to render it to the default backbuffer or perform our own post-process AA (such as FXAA or SMAA) on content rendered to a texture.

Now, with Multisampled Renderbuffers, we can use the general rendering pipeline in WebGL to provide multisampled antialiasing (MSAA):

pre-z pass –> rendering pass to FBO –> postprocessing pass –> render to window

renderbufferStorageMultisample is the relevant function here.

Pay attention to the fact that the multisample renderbuffers cannot be directly bound to textures, but they can be resolved to single-sample textures using the blitFramebuffer call. This is a new feature in WebGL 2 as well

FXAA: Fast Approximate Anti-Aliasing and SMAA: Enhanced Subpixel Morphological Antialiasing are both post-processing techniques, operating on the rendered pixels. Not strictly necessary, FXAA or SMAA can be added on later. But the immediate problem of removing the use of glTexImage2DMultisample for web support (#446) remains...

[iceiix]

Back to working on disabling multisampling. Got past the incomplete framebuffer error, but now hits glGetError = 1282 (Invalid operation). glow's gl46 https://docs.rs/gl46/0.1.3/gl46/ says it has these useful features for debugging:

debug_trace_calls: if cfg!(debug_assertions), any call to a GL function will trace! what was called and with what args.
debug_automatic_glGetError: If cfg!(debug_assertions), this will automatically call glGetError after every call to any other GL function. If an error code occurs it's shown via error! along with the name of the function that had the error.

but glow = { git = "https://github.com/iceiix/glow", rev = "b354346dee69ff0ca7ccef67f7580dfbb697423b", features = ["debug_trace_calls", "debug_automatic_glGetError"] } fails with no such features, not piped through gl46 to glow? First enabled debug-assertions (since we set opt-level = 1, and have to in order to avoid #443, which disables debug-assertions by default), then glow = { path = "../glow" } and modified glow/Cargo.toml:

diff --git a/Cargo.toml b/Cargo.toml
index 699e63f..48b4c35 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -28,8 +28,10 @@ name = "glow"
 path = "src/lib.rs"
 
 [features]
-default = ["web-sys"]
+default = ["web-sys", "debug_trace_calls", "debug_automatic_glGetError"]
 web-sys = ["web_sys", "js-sys", "wasm-bindgen"]
+debug_trace_calls = []
+debug_automatic_glGetError = []
 
 [target.'cfg(not(any(target_arch = "wasm32")))'.dependencies]
 glutin = { version = "0.24", optional = true }

This works in a pinch to log all the calls and check each for errors. Lots of output, but the first few errors are:

calling gl.BindTexture(0xDE1, 2);
calling gl.GetError();
calling gl.TexSubImage2D(0xDE1, 0, 0, 0, 512, 512, 0x1903, 0x1401, 0x7feb78040000);
calling gl.GetError();
calling gl.Uniform1i(12, 1);
calling gl.GetError();
calling gl.BindVertexArray(2);
calling gl.GetError();
calling gl.DrawArrays(0x0, 0, 102400);
calling gl.GetError();
calling gl.UseProgram(7);
calling gl.GetError();
calling gl.UniformMatrix4fv(1, 1, 0, 0x7feb869910a0);
calling gl.GetError();
calling gl.UniformMatrix4fv(5, 1, 0, 0x7feb869910e0);
calling gl.GetError();
calling gl.Uniform1i(11, 0);
calling gl.GetError();
calling gl.Uniform1f(10, 0.8);
calling gl.GetError();
calling gl.Uniform1f(9, 0.7051707);
calling gl.GetError();
calling gl.BindFramebuffer(0x8CA8, 2);
calling gl.GetError();
calling gl.BindFramebuffer(0x8CA9, 1);
calling gl.GetError();
calling gl.BlitFramebuffer(0, 0, 1708, 960, 0, 0, 1708, 960, 256, 0x2600);
calling gl.GetError();
Invalid Operation to glBlitFramebuffer.

glBlitFramebuffer, then glBindTexture:

calling gl.ActiveTexture(0x84C2);
calling gl.GetError();
calling gl.BindTexture(0x9100, 6);
calling gl.GetError();
Invalid Operation to glBindTexture.

and glBlitFramebuffer, glBindTexture, glBlitFramebuffer, glBindTexture, repeatedly. Only call to blit is in src/render/mod.rs tick() to copy the depth buffer:

        // Copy the depth buffer
        trans.main.bind_read();
        trans.trans.bind_draw();
        gl::blit_framebuffer(
            0,
            0,
            physical_width as i32,
            physical_height as i32,
            0,
            0,
            physical_width as i32,
            physical_height as i32,
            gl::ClearFlags::Depth,
            gl::NEAREST,
        );

Is fb_depth created correctly? Was using gl::DEPTH_COMPONENT format/internalformat and gl::FLOAT type (image_2d_sample didn't require type). Fixed by matching parameters of image_2d_ex same as trans_depth. Next failure was glBindTexture, set breakpoint in src/gl46.rs:4265, step up, found it was a missing gl::TEXTURE_2D_MULTISAMPLE in draw(). Correcting to TEXTURE_2D, works!

[iceiix]

Multisampling disabled, after:

before (with MSAA, master branch):

any significant difference? The fire is different between these two frames but it is animated, so... either way doesn't seem to be too significant of a change with MSAA disabled.

[iceiix]

Testing wasm now, gets further but WebGL doesn't like the clouds shaders:

"invalid shader type", I bet this is because of src/render/shaders/clouds_geo.glsl, used by src/render/clouds.rs for a gl::GEOMETRY_SHADER. https://learnopengl.com/Advanced-OpenGL/Geometry-Shader. https://news.ycombinator.com/item?id=13893412 confirms WebGL 2 doesn't have: geometry shaders, compute shaders, tessellation shaders. Will have to do without...

As for this specific pull request, the choice now is to either merge the MSAA removal now, or work on renderbuffer MSAA first...

[iceiix]

First trying to use renderbuffers without multisampling to get the hang of it (then should be straightforward to increase the samples, glRenderbufferStorage is the same as glRenderbufferStorageMultisample with samples = 0). The status quo:

main (Framebuffer):

• fb_color (Texture)
• fb_depth (Texture)

trans (Framebuffer):

• accum (Texture)
• revealage (Texture)
• trans_depth (Texture)

these objects are encapsulated in TransInfo. Changing fb_color and fb_depth to Renderbuffers... compiles but nothing renders, there is a problem: renderbuffers, unlike textures, cannot be accessed from shaders. trans_frag.glsl accesses taccum, trevealage, and tcolor, as sampler2D. Only textures can be accessed from shaders. That is, this doesn't work:

main (Framebuffer):

• fb_color (Renderbuffer)
• fb_depth (Renderbuffer)

A way around this it to create another framebuffer, backed by textures, then blit from the backed-by-renderbuffer framebuffer to the backed-by-textures renderbuffer. Not familiar enough with the existing rendering pipeline to know how to implement this, exactly.. also further complicated by the need to render non-transparent and transparent separately.