Replies: 4 comments 8 replies
-
|
Feedback from the direct users of gfx-hal.
"csnewman":
TL;DR: in both cases sticking to Vulkan API is not critical. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for starting this discussion @kvark! I think this post has a great summary of the existing structure and interesting ideas for the future. Some initial thoughts:
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for responding, as well as rehashing the reasoning we used to align to Vulkan!
We still want to be able to run the real games and Vulkan CTS on top of gfx-portability, even if a lot of logic from the current gfx-hal/backends is moved in there. So this aspect is unchanged.
That is what I was thinking as well! My rough idea was to cut gfx-hal into the basis needed by wgpu repository, and the rest needed by gfx-portability. We'd move the relevant bits out into As for moving wgpu-rs into The main question, I guess, that is left unclear (to me, at least), is how would we structure this low-level GPU interaction layer in a way that is both optimal for |
Beta Was this translation helpful? Give feedback.
-
Right, that's currently the case. This has also changed over time though, and it's not clear that we wouldn't gain more external users again if we continued down the current path. For example, from the Bevy discussion, it seems like there's some interest in a lower level abstraction than wgpu, and I think gfx-hal could fill that role. Of course it's difficult to try to guess how many potential future external users we'll have though, so it might just be the case that we have to plan based on the current state only and acknowledge that it might limit us in the future. For example, if wpgu+gfx-hal were merged, it probably wouldn't be easy to reverse this if people are interested in using a lower level abstraction. I think there's an interesting path in which projects could start with wgpu and later drop down to gfx-hal if they decide they need more control.
Yeah I think the wgpu-rs vs. wgpu-core repository split is definitely worth solving. Maybe we could do something like we've been doing with
What would you think about the backend-specific extension idea I mentioned above? |
Beta Was this translation helpful? Give feedback.
-
|
The change over time is strictly downwards, and I think we've given it enough time now. gfx-hal was born around the end of 2017, so it's more than 3 years old now. Of course it was rough, needed polish, etc, but still there's been a lot of time to think. Our user research failed. People who can understand Vulkan strongly preferred to just use it straight (e.g. via Ash) and pretend that it's available everywhere. People who can't understand Vulkan were unable to pick up gfx-hal. If there is new interest in a low level abstraction (and this is getting less likely now with Rafx growing), we can build something from the pieces and blocks of logic used for wgpu and gfx-portability. The important part of this story is - we'll unlikely need the slow paths included in this abstraction anyway, so the slow paths need to migrate to gfx-portability.
This is the direction we were been heading, I think. We have this Naga-specific shader module creation, we have a limited swapchain model. We'd get some flags on the secondary command buffers limiting their use. We'd get some flags on the command pools limiting their use. And so forth... It could be seen as just another way to reach to the same end state. The problem with this approach is that gfx-hal would have to bear a complexity of a very complex API, and all the Vulkan-specific workarounds would still be needed. Couple that with the fact that gfx-rs isn't tested at all by itself, and you get a disaster: a huge unsafe library, which only tested downstream by wgpu and portability. |
Beta Was this translation helpful? Give feedback.
-
Yeah that seems reasonable to me. I think the fact that rafx-api and piet-gpu-hal started recently indicate that there's interest in a low-level HAL, but they didn't want to use gfx-hal for their use cases for some (completely valid) reasons. For example, maybe they accepted some additional limitations to achieve offline shader compilation, or simply wanted more control over certain implementation details, etc. I'm happy to see other low level graphics abstractions in the Rust community, but personally I think it's also slightly unfortunate that we couldn't combine efforts on a single HAL (in some form -- not necessarily in the shape of how gfx-hal exists today) to make it work for all of our use cases somehow. The transfer example mentioned by @raphlinus is a good one. If there's a faster approach path to do unaligned buffer <-> image transfers on DX12 when certain conditions are met, it would be great if piet-gpu-hal, gfx-portability, rafx-api, and any other HALs could all benefit from one implementation of that path (including any fallback paths, like row-by-row copying if necessary). There are lots of other examples of translation paths/emulation similar to that (including driver workarounds) that all HALs will probably need to handle at some level, but currently they'll all be developed and tested independently. For example, driver workarounds that exist in gfx-hal can't benefit rafx-api or piet-gpu-hal and this ultimately impacts end-users. Maybe this is too idealistic but it would be really nice if the Rust ecosystem could collaborate more closely on a graphics HAL. It takes a lot of effort to make a high quality HAL, but there are a limited number of people in the intersection of open source + Rust + interested in two or more graphics APIs. Efforts like gpu-alloc help a bit, but I'm thinking more along the lines of what gfx-hal currently does. If there's not much interest in doing that, then it seems ok to start breaking apart gfx-hal and turn it into an implementation detail of wgpu and gfx-portability. |
Beta Was this translation helpful? Give feedback.
-
|
I see what you are saying. Perhaps there is a future where all of the parties can agree on some basic building blocks for graphics abstraction. The current tendency, however, is the opposite: between macroquad, Rg3d, Rafx, and piet-gpu-hal, authors are progressively choosing the path of fewest dependencies. This ends up with better compile times, higher developer velocity (but has its own costs, of course), and the users buy into it. Perhaps our potential collaborators aren't going to join until they see WebGPU succeeding in its goals. |
Beta Was this translation helpful? Give feedback.
-
|
My thoughts, not sure how useful they'll be. First, as someone who has tried (and mostly failed) to create layered stacks of Rust ecosystem components, I heartily endorse pulling back from making gfx-hal a publicly supported API, and move it into being an implementation detail of wgpu and/or a Vulkan portability implementation. I definitely considered gfx-hal for my work and will explain why I chose not. First, for context, I should say what I'm doing. In the short term, I want to do research on GPU rendering. For that, leaky abstractions can really get in the way. I want to be able to experiment with "advanced" features like subgroups, memory model, etc., even if (for the second part) I end up not using or not prioritizing them. I also need to spend a lot of time debugging performance (and correctness) issues, so I need timer queries, and am at the mercy of bad/incomplete tooling. I've been doing this work with a thin homespun layer over ash, and have been much happier. The second, medium to long term goal, which has some tension with the first, is to deliver piet-gpu as a renderer for UI. That goal has different requirements, portability first among them, which make a wgpu-like layer more appealing, but there are challenges. Mostly, I'm interested in having the shader compilation pipeline as much as possible at build time, so I don't need to have the costs (app startup time, binary size, etc) of runtime shader compilation. In any case, for both of these goals, I'm really interested in a tiny fragment of the capabilities provided by a GPU. For the most part, I only care about running compute shaders. (I will need to run the rasterization pipeline for some things, as it turns out, but again in a pretty simple way compared to 3D engines). The narrow scope makes it a lot more feasible to build my own layer, which I am doing. So the reasons not to use gfx-hal are similar to those not to use wgpu, mostly shader compilation. If and when naga gets the ability to output dxil directly, that might change the calculus. The need to bundle dxc just to access subgroups is not appealing, especially as I can solve this problem by running dxc at build time (and having dxil in the repo so "cargo run" just works). The fact that gfx-hal is not officially documented or stable has been a friction point; for all its flaws, the fact that Vulkan is standard and documented has been a major help in my work. I'm also finding that emulating Vulkan semantics is causing impedance mismatch. One example is unaligned buffer ↔︎ image transfers. These are likely to be important in piet-gpu, and I'm concerned that the polyfill (row-by-row) copy in hal is not ideal. I'd rather polyfill it with a compute shader, but I also see the challenges - if writing to an image, it needs to have storage usage as opposed to copy_dst. All this, plus managing the resources for the compute shader pipeline, feel like they're better handled at a higher level. (I should also say: figuring out the absolute best way to do this is a nontrivial project on its own, as it seems like it requires gathering empirical data on the tradeoff between row-by-row copy and having different image layout transitions, across a wide range of GPUs in the fleet. There's no good place where this kind of knowledge lives right now, something I hope will change). Also as part of shipping piet-gpu, a major goal would be interop so people can build apps that mix 2d and 3d content - I'm hearing considerable demand for that. There are a bunch of different ways to approach that, depending on how tightly coupled the integration should be. Obviously one way to do it is in the compositor, where the 2d and 3d subsystems barely need to interact, but there's also benefit in, for example, being able to render 2d textures that can be used in the 3d scene. I feel like a decisive win would be to standardize on wgpu (see linebender/druid#891 for more discussion of these ideas, from a while back but I think still basically valid). For this interop goal, I don't think targeting gfx-hal as an interop layer is that compelling, largely because you want things like allocation to be common, plus a good story for tracking resource lifetime across these layers. If the world fragments and there are different 3d APIs people target, then I think some serious engineering effort will be needed toward interop. (and/or abandoning tighter integration and emphasizing use of the compositor in Druid) Bottom line: when wgpu is more done, it will be a very compelling target for piet-gpu, and I very much hope to support it. Targeting a lower level abstraction wasn't as appealing. It's likely we'll continue targeting our own piet-gpu-hal layer because it is lightweight and gives lower friction access to advanced compute features, but if wgpu is sufficiently lean and featureful (tension, I know), it could change the calculus and at that point I'd be happy to make piet-gpu-hal go away. |
Beta Was this translation helpful? Give feedback.
-
|
Is there any translation layer from modern APIs (Vk, DX11/12, Metal) to WebGPU (for the web) in the works? I'm interested in contributing to such a project. I know gfx-portability focused on Vulkan to gfx-hal backends such as Vulkan, DX11, Metal. Would it be a good place to start if one were interested in creating such a translation layer? It seems like not the greatest time to start such a project, considering this thread however this seems the closest and I'd rather not start a completely new project. My goal is to port existing Vulkan based apps to the web by translating calls to WebGPU calls. |
Beta Was this translation helpful? Give feedback.
-
|
gfx-rs had the WebGPU backend started, the mock is there in the code. But it got stalled on the problem with async functions in WebGPU, such as requesting device, adapter, or mapping. If this can be solved somehow, then gfx-backend-webgpu can be continued and evolve within the gfx-portability project, mapping from Vulkan to WebGPU. |
Beta Was this translation helpful? Give feedback.
-
|
Do you have any plans to implement GNM/GNMs backend? |
Beta Was this translation helpful? Give feedback.
-
|
We do not have the NDAs required to have access to it, plus the fact that rust doesn't natively support Playstation at all. Unfortunately, Sony and open source do not mix well. |
Beta Was this translation helpful? Give feedback.
-
Context for this:
Situation
gfx-hal is an implementation of Vulkan-like API in Rust. It's very low level (explicit synchronization and memory management), fully unsafe, hard to use, and fully unsafe. From the inception, we considered the following major clients:
WebRender
There was an experimental fork of WR using gfx-hal. See performance data in szeged/webrender#187. It makes total sense for WR to use such a low unsafe abstraction, because WR's use-case is very limited, and it needs maximum performance with portability.
Unfortunately, Mozilla hasn't extended the contract with the Szeged team, and the fork is now abandoned. In more than a year (passed since then), WR moved ahead with even more platform specific features (mainly, direct composition), and reviving the fork will be difficult. If this work is going to be kicking off, it's likely going to be a complete re-implementation of gfx-hal WR backend, using the old code as a reference.
Amethyst
Amethyst's graphics stack is based on Rendy, which uses gfx-hal. It was a very promising direction, but Rendy ended up being somewhat over-engineered, and eventually @zakarumych abandoned it. Like with dinosaurs, the surviving bits evolved significantly and became gpu-alloc and gpu-descriptor, used by wgpu-core today.
The future of Amethyst is unclear. I guess, some brainpower and energy got drained by Bevy (and maybe Rg3D?). It's also evident that Rendy isn't going to be developed further. So if Amethyst still has energy to move forward, it would need to adopt a different graphics stack, which is a heavy task.
Vulkan Portability
gfx-portability got us a lot of hard testing (on Vulkan CTS as well as real applications). It allowed us to produce artifacts, like Dota2-based benchmarks on https://gfx-rs.github.io/, which highlight the effectiveness of the stack. It helped to identify hundreds of issues in Metal and D3D12, as well as shader translation.
It's still a good thing to help Vulkan Portability moving forward, gives us coverage and potential real-world users. There is even a bit of interest in the D3D12 backend, but it's unclear if it's going to materialize into any help/motivation.
If gfx-hal changes its path, gfx-portability would still be able to develop, albeit with a thicker layer translating Vulkan to gfx-hal. Right now this layer is very thin, close to zero overhead.
Interestingly, Rust ecosystem doesn't use gfx-portability much, and instead sticks with MoltenVK, despite it being a huge unsafe C++ dependency. Sometimes it feels to me that people just close their eyes to the fact that Vulkan is not available everywhere.
WebGPU
wgpu has a lot of responsibilities: providing a higher level API that is fully validated. It's the lowest level Rust safe graphics API out there, and it's still a work in progress. wgpu outsources all of the portability needs to gfx-hal, effectively targeting Vulkan. That allows wgpu to focus on what matters for WebGPU (including native). It also leaves an emergency plan on the table: if anything goes wrong with gfx-hal, it can easily switch to raw Vulkan (e.g. Ash), given how gfx-hal is similar.
The downside for wgpu is - slower velocity to delivering fixes, adding new functionality, such as OpenXR support. Also, potential overhead on platforms where we can use the native APIs more efficiently. For example, on D3D12 there are command pools, but you can't be recording more than one command buffer at a time with the same pool. But since you can do it in Vulkan, gfx-hal allows that as well, and then it has to create a non-trivial scheme of associating these command pools with recorded buffers. And this scheme (as a part of gfx-backend-dx12) runs on top of the command pool management of
wgpu, which may be sub-optimal (more in development cost than runtime cost, but still).There is a lot of gfx-hal surface that wgpu doesn't need: sub-passes is one big example, but also - various workaround for resource clearing and transfers. Mapping secondary command buffers to everything is a royal pain in itself, and it's not helping wgpu to implement Render Bundle concept efficiently.
What inevitably happens is that wgpu carves out the features it needs from gfx-hal. I.e. it forces the latter to adopt API features (or restrictions) specifically for wgpu use, and these path are always expected to be happy paths. For render bundles, gfx-hal would need some extra info for secondary command buffers, so that they can be implemented on top of D3D12 bundles or Metal indirect command buffers. Overall, this shapes gfx-hal towards just being an extension of wgpu logic, but also with more functionality to make gfx-portability work.
Roads Ahead
The situation today is that wgpu is the only driving force behind gfx-rs, and it's getting slowed down by gfx-rs in terms of development friction. If gfx-rs throws out everything that wgpu doesn't need, thus leaving only "happy paths" implemented by the backends, we'll need a way for gfx-portability to fill these gaps manually. For example, gfx-portability would need to handle the row strides not aligned to 256 on D3D12 when copying data. There is no need for this complexity to live any higher than at this level. gfx-portability would have all those slow paths in it to make Vulkan happy. And it would need to deal with secondary command buffers, which is a major pain. This is an "ideal" scenario, and in it gfx-rs is very small and fast, but also - open to users so that they can implement their own workarounds.
Previously, @msiglreith (who originally prototyped gfx-hal back in the day) suggested an interesting idea of gfx-bal in #2249. It describes a model where gfx-rs just offers a bunch of higher level functional blocks, not necessarily portable. The other libraries would then be able to build higher level APIs from these Lego-like blocks. APIs like the current gfx-hal, gfx-portability, or wgpu. In practice, I'm having a hard time imagining what those blocks going to look like. It seems more likely that gfx-bal would just dissolve into backend-specific things like D3D12 root signatures and Metal argument buffers, wrapping d3d12-rs and metal-rs. And even then, it's unclear how much those functional blocks are going to be re-usable (or opinionated).
Slimming down gfx-rs responsibilities would help to move forward, but would not necessarily solve the current issues. As such, wgpu would no longer be able to move to pure Vulkan as a backup plan. Another problem that it would still have the development friction if gfx-rs is in a separate repository.
I'm starting this discussion to provide the necessary context and call of ideas!
Beta Was this translation helpful? Give feedback.
All reactions