WIP: A software renderer

[dbr]

Something I wanted for writing test cases for some imgui widgets, but might be useful for a few reasons:

1. An alternative backend as discussed in Removing imgui-gfx-renderer? #444 - not being dependent on any particular hardware or OS makes it quite a nice reference (assuming it works properly)

2. Writing more comprehensive tests for imgui-rs code, since it can run on any CI platform (doesn't need actual graphics hardware, or platform-specific things like Xvfb)

3. I have a half-formed idea that it would be possible to write mdbook preprocessor which would allow you to write docs including snippets like:

   # let ui = setup_renderer().display_size([100.0, 200.0])
   ui.button(im_str!("Example button!")
   # ui.render_png()

   ..and so building the mdbook would magically and automatically generate all the screenshots (which would be much less tedious than manually cropping screenshots, and also serve as some form of high-level test case)

The renderer seems to work quite well in the limited testing I've given it - but there are a few oddities/issues:

1. The first few frames never render properly - I'm pretty sure this is due to how imgui calculates layouts etc
2. Had to duplicate the Transform::invert code from tiny_skia as it's currently private. Need to open an issue with that project to see if it's something that can be made pub (if not, the implementation can be rewritten in a much shorter way)
3. The TestHelper struct could also be improved drastically, and should probably have a better name. It's intended to make the "just render this widget and give me a PNG" case as simple
4. It would be helpful to have some kind of imgui-software-support which lets you easily emulate mouse/keyboard input etc

Thoughts?

[sanbox-irl]

Hey dbr -- I really like this PR and this idea. It would be stupendous if our examples ran on this, and therefore would allow us to do actual GUI tests. Like I'd love to mock up mouse inputs to make sure that we're handling over frame checks and what not.

That being said, I'm not sure about tiny-skia. Have you seen https://github.com/zesterer/euc? It's pure rust, which is nice, but it also looks a bit simpler in API to use.

What do you think?

[sanbox-irl]

also dbr, if you have a second, would you mind getting in contact with me? No pressure if you don't want to, but would love to chat in a faster way -- my discord is sanbox_irl#7777 and my email is jjspira on gmail

[thomcc]

Neat! I have a few concerns though... ideally, we'd either:

1. Avoid exposing tiny_skia as part of the public API.
2. Call it something more specific like imgui-tiny_skia-renderer.

Personally, I'd greatly prefer the first of these, as I don't think skia-style path-based drawing is actually what we want here, and so we'd want to move away from it eventually (either to some other crate or whatever. For this kind of thing I suspect we could do a rasterizer that supports what imgui needs in under 1kloc even, tbh)

Anyway, I only care that we work this stuff out before publishing it on crates.io. If we just wanted to use it for internal testing, we can figure it out later.

[dbr]

That being said, I'm not sure about tiny-skia. Have you seen https://github.com/zesterer/euc? It's pure rust, which is nice, but it also looks a bit simpler in API to use.

tiny-skia is pure Rust also, it doesn't have much to do with the skia C++ library aside from using it as a reference

I mostly selected it as we were already using resvg in the project I prototyped this in, and it was a fairly small dependency and fast (if I remember right, it ran quicker than the C++ implementation linked in readme, which is a hand written rasterizer with various optimizations like merging text tris into single boxes)

That said, replacing it with something more directly aimed as rendering polygons would likely simplify the code a bit, and either way hiding the implementation from public API seems perfectly reasonable. The C++ implementation was about 700 lines so is certainly plausible to just write the rasterizer from scratch, it is almost entirely just requires basic filled/textured polygons

I wonder if there's something spiritually similar to mint but for pixmaps - as the PixmapRef was probably the only real benefit of exposing it (I've written to many "convert this image to [B, G, R, ...] to [R, R, R, R, ... G, G, G G, ... B, B, B, B, ...] etc methods to want to expose a simple slice of pixel values 😅 )

Anyway, I only care that we work this stuff out before publishing it on crates.io. If we just wanted to use it for internal testing, we can figure it out later.

Agreed entirely - my initial plan for this is to try and get a mdbook showing generated screenshots (/GIF's) of various IMGUI widgets, as part docs/part integration test. I think I have a good idea of how to go about this now, shall give this a go hopefully some time this week!

[sanbox-irl]

Ah sorry I misunderstood the skia! I don't mind at all what implementation we use for our own example purposes. I think eventually releasing this would be excellent, but just getting it for tests would be really nice. Really, really nice.

[thomcc]

tiny-skia is pure Rust also, it doesn't have much to do with the skia C++ library aside from using it as a reference

My concern is more that path rendering (like the skia api) and triangle rendering (like what the draw data expects) are... different enough that it seems likely to be a bit of whackamole for weirdness. To the point where I worry it would not be an effective tool for debugging certain kinds of issues.

You linked https://github.com/emilk/imgui_software_renderer/blob/master/src/imgui_sw.cpp in the readme, and IMO its closer to what we want, although that code is more complex than I think we need.

Anyway I actually have background in software rendering (I've written production ones, and often use triangle rasterization for game programming stuff¹) so I'll take a quick stab at this to put my money where my mouth is. (Either way I think the important thing is to not tie down the approach much)

Footnotes

1. Really; it's great for the kind of "oh no this part of my game is grid-based, but now needs to work with a shape which is... too weird for the grid 🙀" problems that comes up all the time. (I actually don't know how else people solve them, TBH) ↩

[dbr]

I wonder if there's something spiritually similar to mint but for pixmaps

Closest thing I could find is https://lib.rs/crates/rgb

I think I have a good idea of how to go about this now, shall give this a go hopefully some time this week!

As an update on this: the plan seems to work - basically a "documentation generation" crate which uses mdbook as a library in build.rs to extract the examples, then creates a library with a bunch of functions that can be called (by same crate) as an mdbook pre-processor binary to output images (my first attempt a while ago tried to do everything as the mdbook-preprocessor, easy to extract the examples, but too fiddly to build them)

I now have "both halves" of this functioning in a basic way, so now just need to shove them together

[sanbox-irl]

love it!

[dbr]

Oh forgot to update here - the WIP code is here (barely presentable, but wouldn't take much to tidy it to a "releasable" state)

https://github.com/dbr/imgui-book-prototype

You run mdbook build in the docs folder, and it will:

1. Invoke the preprocessor cargo run --release --bin imgui-book-processor --
2. The build.rs for the preprocessor extracts all fenced codeblocks with start like ```imgui-example, wraps them up in functions which execute them with the software renderer and produce an image
3. The processor is then run, it then does the same codeblock extraction, invokes the appropriate function created above, and inserts the image

The end result is something like this:

# The Second

Simplest example, display code and render output

```imgui-example
use imgui::im_str;
ui.text(im_str!("Basic example"))
```

Hide code, but display resulting image:
```imgui-example,hide_code
use imgui::im_str;
ui.text(im_str!("Example with no code visible!"))
```

..becomes something like this:

(the terrible aliasing is Firefox resizing the image down for the screenshot, the 1:1 scale screenshots look perfectly reasonable)