iOS Support

[cart]

It should be possible to run Bevy Apps on iOS

[MichaelHills]

Hi, what’s involved in setting up Bevy on iOS? My understanding is that it uses winit which should already support iOS?

[MichaelHills]

I followed this guide to just get a Rust lib working from an ios project. Then I tried to substitute a bevy example app in. During cargo lipo --release it fails to build bevy-glsl-to-spirv.

It looks like there's no prebuilt binary in that repo so it tries to build glslang and then cmake fails

  -- Check for working C compiler: /usr/bin/cc -- broken
...
Building C object CMakeFiles/cmTC_97014.dir/testCCompiler.c.o
      /usr/bin/cc   -fPIC -m64 -m64 -mios-simulator-version-min=7.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk -fembed-bitcode  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -o CMakeFiles/cmTC_97014.dir/testCCompiler.c.o   -c /Users/mikeh/rust-ios-example/rust/target/x86_64-apple-ios/release/build/bevy-glsl-to-spirv-51cc402e67486116/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]

Apparently that last line is a key sign something is off. I take it the build for glslang doesn't work properly to target aarch64-apple-ios or x86_64-apple-ios.

[MichaelHills]

I tried a couple of things, but I'm new to rust and new to cross-compiling to iOS, though slowly figuring things out. The approach of spawning the glslValidator process probably isn't going to fly on iOS. MoltenVK apparently can build the iOS static lib libglslang.a so I considered trying to interface with that. The bevy-glsl-to-spirv repo says TODO to switch to shaderc-rs so I figured I'd have a crack at that but had trouble cross-compiling for iOS.

I ended up stubbing out the code in crates/bevy_render/src/shader/shader.rs just to see how far I could get. I also had to disable the optional bevy audio as it didn't compile either (forgot why).

I managed to finally get something to build, on the simulator gfx-rs / gfx-backend-metal failed with Target OS is incompatible: library was not compiled for the simulator so I ran it on my phone instead to which I got

EventLoop cannot be run after a call to UIApplicationMain on iOS.

Stopping for tonight. Here's my diff so far to get where i got to...

diff --git a/Cargo.toml b/Cargo.toml
index 90b2368d..3254fd6c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -13,7 +13,8 @@ readme = "README.md"
 exclude = ["assets/**/*", "tools/**/*", ".github/**/*", "crates/**/*"]
 
 [features]
-default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+#default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+default = ["bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr"]
 profiler = ["bevy_ecs/profiler", "bevy_diagnostic/profiler"]
 wgpu_trace = ["bevy_wgpu/trace"]
 
@@ -22,10 +23,10 @@ png = ["bevy_render/png"]
 hdr = ["bevy_render/hdr"]
 
 # Audio format support (MP3 is enabled by default)
-mp3 = ["bevy_audio/mp3"]
-flac = ["bevy_audio/flac"]
-wav = ["bevy_audio/wav"]
-vorbis = ["bevy_audio/vorbis"]
+#mp3 = ["bevy_audio/mp3"]
+#flac = ["bevy_audio/flac"]
+#wav = ["bevy_audio/wav"]
+#vorbis = ["bevy_audio/vorbis"]
 
 serialize = ["bevy_input/serialize"]
 
@@ -56,7 +57,7 @@ bevy_ui = { path = "crates/bevy_ui", version = "0.1" }
 bevy_window = { path = "crates/bevy_window", version = "0.1" }
 
 # bevy (optional)
-bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
+#bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
 bevy_gltf = { path = "crates/bevy_gltf", optional = true, version = "0.1" }
 bevy_wgpu = { path = "crates/bevy_wgpu", optional = true, version = "0.1" }
 bevy_winit = { path = "crates/bevy_winit", optional = true, version = "0.1" }
diff --git a/crates/bevy_render/Cargo.toml b/crates/bevy_render/Cargo.toml
index ea29b856..53fe2af4 100644
--- a/crates/bevy_render/Cargo.toml
+++ b/crates/bevy_render/Cargo.toml
@@ -24,7 +24,7 @@ bevy_window = { path = "../bevy_window", version = "0.1" }
 
 # rendering
 spirv-reflect = "0.2.3"
-bevy-glsl-to-spirv = "0.1.7"
+#bevy-glsl-to-spirv = { path = "/Users/mikeh/repos/glsl-to-spirv" } #"0.1.7"
 image = { version = "0.23", default-features = false }
 
 # misc
diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..dd448e74 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,6 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
+//use bevy_glsl_to_spirv::compile;
 use std::{io::Read, marker::Copy};
 
 /// The stage of a shader
@@ -11,25 +11,26 @@ pub enum ShaderStage {
     Compute,
 }
 
-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
-}
+//impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
+//    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
+//        match self {
+//            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
+//            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
+//            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
+//        }
+//    }
+//}
 
 fn glsl_to_spirv(
     glsl_source: &str,
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
+    //let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
     let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    //output.read_to_end(&mut spv_bytes).unwrap();
+    //bytes_to_words(&spv_bytes)
+    spv_bytes
 }
 
 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

[MichaelHills]

Looks like cross-compiling iOS stuff with cmake-rs is currently broken rust-lang/cmake-rs#96 so that's probably the source of some of my woes

[MichaelHills]

Success! I managed to get the 3d_scene demo loading in portrait mode (albeit a bit warped) on my phone.

• I pinned cmake-rs to 0.1.43 because that's the version which had the working iOS support
• Dropped bevy-glsl-to-spirv in favour of using shaderc-rs directly from crates/bevy_render/src/shader/shader.rs
• Created a main.swift to call out to my bevy_main() function directly which was the 3d_sprite main function renamed like this

#[no_mangle]
pub extern fn bevy_main() {

In xcode I had to add the static lib / header and search paths as per this site and I also had to link against lc++. Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Trying to figure out how to get it into landscape mode now.

[cart]

Whoa that's huge! Thanks for the hard work here. People are going to love this!

[MichaelHills]

No worries. :) When I saw the post on Bevy on HN I thought it looked pretty awesome. I previously had started making a PC game in UE4 but life got busy. Decided I wanted to prototype it out as a mobile game instead and learn some Rust/ECS while I'm at it.

How should I go about getting stuff landed? I think the switch to shaderc-rs in general can be done now (tested a couple of examples still work). Did you want that to be done inside of bevy-glsl-to-spirv? I've currently just implemented it directly in crates/bevy_render/src/shader/shader.rs while hacking this all up.

In order to get iOS integration to "just work" the following issues need to be solved properly:

• Land using shaderc-rs in bevy
• Get that iOS support commit in cmake-rs fixed up and re-landed there
• In shaderc-rs I had to edit CMakeLists.txt in the glsl and shaderc submodules (see below), so maybe... get those landed there?
• Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

shaderc

diff --git a/glslc/CMakeLists.txt b/glslc/CMakeLists.txt
index acf6fb0..4ec0067 100644
--- a/glslc/CMakeLists.txt
+++ b/glslc/CMakeLists.txt
@@ -40,6 +40,7 @@ shaderc_add_asciidoc(glslc_doc_README README)

 if(SHADERC_ENABLE_INSTALL)
   install(TARGETS glslc_exe
+    BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
     RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
 endif(SHADERC_ENABLE_INSTALL)

glslang

diff --git a/StandAlone/CMakeLists.txt b/StandAlone/CMakeLists.txt
index 2cf2899c..f84fdd2b 100644
--- a/StandAlone/CMakeLists.txt
+++ b/StandAlone/CMakeLists.txt
@@ -50,11 +50,13 @@ endif(WIN32)

 if(ENABLE_GLSLANG_INSTALL)
     install(TARGETS glslangValidator EXPORT glslangValidatorTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
     install(EXPORT glslangValidatorTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)

     if(ENABLE_SPVREMAPPER)
         install(TARGETS spirv-remap EXPORT spirv-remapTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
         install(EXPORT spirv-remapTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)
     endif()

Also here's the change to shader.rs

diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..2e710cd0 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,7 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
-use std::{io::Read, marker::Copy};
+use std::{marker::Copy};
 
 /// The stage of a shader
 #[derive(Hash, Eq, PartialEq, Copy, Clone, Debug)]
@@ -11,14 +10,14 @@ pub enum ShaderStage {
     Compute,
 }
 
-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
+impl Into<shaderc::ShaderKind> for ShaderStage {
+   fn into(self) -> shaderc::ShaderKind {
+       match self {
+           ShaderStage::Vertex => shaderc::ShaderKind::Vertex,
+           ShaderStage::Fragment => shaderc::ShaderKind::Fragment,
+           ShaderStage::Compute => shaderc::ShaderKind::Compute,
+       }
+   }
 }
 
 fn glsl_to_spirv(
@@ -26,10 +25,19 @@ fn glsl_to_spirv(
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
-    let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    let mut compiler = shaderc::Compiler::new().unwrap();
+    let mut options = shaderc::CompileOptions::new().unwrap();
+    if let Some(shader_defs) = shader_defs {
+        for def in shader_defs.iter() {
+            options.add_macro_definition(def, None);
+        }
+    }
+
+    let binary_result = compiler.compile_into_spirv(
+        glsl_source, stage.into(),
+        "shader.glsl", "main", Some(&options)).unwrap();
+
+    binary_result.as_binary().to_vec()
 }
 
 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

[MichaelHills]

Stumbled across this issue in CMake https://gitlab.kitware.com/cmake/cmake/-/issues/20956 which suggests that the whole BUNDLE DESTINATION thing might be due to some CMake default that doesn't make sense when building libs, and I don't need to try and get BUNDLE DESINATION added to shaderc and glsl.

[naithar]

Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Godot works in same way, though it implements it's own main which gets used by iOS application.

Obj-C can be used too, but this would require only main.mm file like this:

@import UIKit;

extern "C" void bevy_main(void);

int main(int argc, char * argv[]) {
    bevy_main();
}

And additional values in Other C Flags in Build Settings: -fmodules -fcxx-modules.

I've tested with 2d -> sprite.rs example, but got empty window.
I guess asset loading would require some work for iOS, but correct (?) texture handle was returned Handle<Texture>(82f42690-2c58-4843-a88a-aa066583f97d), so I'm not sure.

• Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

This PR (RustAudio/coreaudio-sys#33) seems to be addressing this issue, but I wasn't able to make it work even with this changes. But maybe it's just me as I don't really have much experience with rust.

[MichaelHills]

@naithar oh cool glad to see someone else getting it working too. Yeah I didn’t get the sprite example working, I also assumed it was related to asset loading but I haven’t looked into that yet.

Interesting I’ll try pulling that audio PR later to see if it works.

[MichaelHills]

By the way were you able to get the simulator working? I’m still getting the issue with Target OS is incompatible: library was not compiled for the simulator but supposedly there should be simulator metal support gfx-rs/gfx#2873 so not sure yet what the problem with that is.

[naithar]

No, I've got some error about unsupported architecture from dependency. I'll try to look more into it after I figure out asset loading :)

Godot uses 13.0 sdk version for simulator, since this the only simulator version supporting Metal. But while it does build, MoltenVK fails and application doesn't launch in the end.
So cargo lipo might be using lower sdk version that actually required for bevy.

[MichaelHills]

Looks like the ios audio support is still in-progress as per your link, I tried and failed to get that branch to work as well. I'll leave audio disabled for now and might have a try at getting the simulator going.

[naithar]

So compiled binary of bevy faces the same problem - not displaying sprite at specified path (icon.png).
Providing a full path fixed this for macOS, but haven't for iOS.

[MichaelHills]

I played around with the paths on iOS on my phone, I added icon.png to my xcode project and examined the various paths...

Bundle.main.bundlePath -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app
std::env::current_exe() -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/rust-ios
Bundle.main.path(forResource: "icon", ofType: "png") -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png

I tried std::fs::canonicalize(std::env::current_exe().unwrap().join("..").join("icon.png")).unwrap(); to produce /private/var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png and pass that to the asset loader. Logs in the loader showed it was successfully loading the file. I guess you already checked the texture handle... but still nothing shows up on screen.

I tried adding some color to the texture to see if it'd help, it didn't. I also tried just the color by itself and no texture, but that doesn't seem to work even on mac.

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                texture: texture_handle.into(),
            }),
            ..Default::default()
        });

If we can get a red square rendering, that'd be a good first step. I edited the fragment shader to show only red, worked on mac, didn't work on iOS.

[naithar]

It seems like iOS doesn't really handle/load assets in any way.
Also compiled binaries doesn't really like relative paths too, as I've reported in #344

I'm looking for a way to debug rust on iOS, so problem could be traced, but I've got no progress yet.

[MichaelHills]

In crates/bevy_render/src/render_graph/nodes/pass_node.rs I added some logs, basically can_draw_indexed() returns false because one of the "bind groups" is None. I don't know what a bind group is though.

                                    println!("maybe draw indexed can_draw={} index_buf={} bgs={} vbs={}", draw_state.can_draw(), draw_state.index_buffer.is_some(), draw_state.bind_groups.iter().all(|b| b.is_some()), draw_state.vertex_buffers.iter().all(|v| v.is_some()));
                                    for bg in draw_state.bind_groups.iter() {
                                        println!("bg is some {}", bg.is_some());

                                    }
                                    if draw_state.can_draw_indexed() {

logs are

1 visible entities
maybe draw indexed can_draw=false index_buf=true bgs=false vbs=true
bg is some true
bg is some false
bg is some true

[MichaelHills]

Oh nevermind, it was because I had left this in the frag shader

# ifdef COLORMATERIAL_TEXTURE
layout(set = 1, binding = 1) uniform texture2D ColorMaterial_texture;
layout(set = 1, binding = 2) uniform sampler ColorMaterial_texture_sampler;
# endif

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

[naithar]

I've put some logs into fn load_asset(&self, load_request: &LoadRequest) -> Result<TAsset, AssetLoadError> and that's the result.

loading untyped at /private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png
loading: Handle<Texture>(7841c797-5cf9-401d-b1ce-67245a08cd42)
loading asset: LoadRequest { path: "/private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png", handle_id: HandleId(7841c797-5cf9-401d-b1ce-67245a08cd42), handler_index: 1, version: 0 }
file exists: true
bytes read: [... very big array ...]

So iOS actually seems to be loading assets correctly in the end if full path is given.

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

Maybe iOS handles drawing differently or shaders are not compiled so it fails to draw anything?

Edit:
Seems like shaders are also compiled correctly, so I guess something might be wrong with shader itself or something is not supported on iOS.

[MichaelHills]

I finally got my red square working. WIthout the image/texture to determine the sprite size, you have to set the size manually:

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                // texture: texture_handle.into(),
            }),
            sprite: Sprite { size: Vec2::new(100.0, 100.0) },
            ..Default::default()
        });

so I think the shaders are fine... but still something going wrong in image or texture loading, not sure what yet.

[MichaelHills]

in crates/bevy_render/src/texture/image_texture_loader.rs

        println!("do dyn img");
        let dyn_img = image::load_from_memory_with_format(bytes.as_slice(), img_format)?;
        println!("got dyn img");

It seems to never finish load_from_memory_with_format (at least I don't see the log afterwards), so this explains why the texture loading is not working.

Now the whole "it has a handle" thing is actually because you get a handle back but the png is loaded into a texture on a background thread. Eventually the handle will resolve to a valid texture, but only once it has loaded.

Discovered this initially by adding logs to crates/bevy_sprite/src/sprite.rs

pub fn sprite_system(
    materials: Res<Assets<ColorMaterial>>,
    textures: Res<Assets<Texture>>,
    mut query: Query<(&mut Sprite, &Handle<ColorMaterial>)>,
) {
    println!("sprite system");
    for (mut sprite, handle) in &mut query.iter() {
        let material = materials.get(&handle).unwrap();
        println!("got material with color {} {} {}", material.color.r, material.color.g, material.color.b);
        if let Some(texture_handle) = material.texture {
            println!("got texture handle");
            if let Some(texture) = textures.get(&texture_handle) {
                println!("got texture size {} {}", texture.size.x(), texture.size.y());
                sprite.size = texture.size;
            }
        }
    }
}

and noticed that on iOS it never printed the texture size but on mac it did.

Edit: finally got the error message out from the image loading... Format error decoding Png: CgBI chunk appeared before IHDR chunk

I think somehow the PNG is getting corrupted when added to the iOS project? It's smaller...

ios bytes=12244
mac bytes=15713

[MichaelHills]

Well, I found our problem! You need to disable XCode "Remove text metadata from PNG". I disabled everything though just to be sure... I guess the image loading library can't handle whatever XCode does to it.

My phone is now showing a red bevy bird. 👍

[cart]

Very exciting! Great work here!

[naithar]

Adding std::env::set_current_dir(std::env::current_exe().unwrap().parent().unwrap()); works as workaround for iOS and macOS .app not loading assets at relative path.

[simlay]

Including in cmake-rs unfortunately got reverted :( rust-lang/cmake-rs#95 I tried to spark some discussion but didn't go anywhere.

I'd like to eventually work on this again. It's valuable for a bunch of iOS projects.

I might have a go, but I'm not sure where's the best place to start.

Yeah! @MichaelHills, that would be awesome! There are few rust iOS folks and I'd like there to be more! You can find me on discord in the Bevy server (is that what they're called?) along with most of the other rust discords as simlay#3120.

If I were to try, do I need the 3 of your branches? rust-bindgen, coreaudio-rs, coreaudio-sys?

Right now, I think so. I'm all for a better work flow.

• my rust-bindgen branch objc-category-inheritance might not be needed anymore as most of the core stuff is either on crates.io or on the master branch. I wish Added Objc category inheritance rust-lang/rust-bindgen#1784 was easier/faster to merge, but it's best that a core rust-lang project be slow and stable rather than quick and breaking.

• my coreaudio-sys branch add-ios-support which is mostly just an update to the build.rs

• my coreaudio-rs branch ios-support. This branch might not actually be all that helpful. Some of the updates were for more modern rust (swapping try! for the ?, adding dyn on traits because). In hindsight, I should have not worried about the warnings and kept it to the smallest of changes.

Why did you need to make changes to rust-bindgen for iOS CoreAudio?

CoreAudio for iOS has a bunch of objective-c in the headers and so bindgen (which uses clang to get the AST from the headers), and so just to get it to compile it required adding -x objective-c as a clang parameter to bindgen, this resulted in an issue with the objective-c generics not generating working rust.

After that, I slowly added to make all the bindings more ergonomic:

• higher level "types" corresponding to the objective-c classes
• inheritance tree generation
• the bindgen names corresponding to objective-c classes
• up/down casting objective-c classes
• I hope the last of these is to fix the objective-c category inheritance.

It's possible I might have gone a bit far down the rabbit hole but these are still cool and useful features if you ask me.

I've spent so much time trying to get the objective-c Nullability stuff working but I've not had a ton of luck. :/.

I'm not going to claim to actually know enough CoreAudio to tell you that these features really help you all that much. For all I know, you could blacklist all of the objective-c and you'd be left with some bindings that are quicker to generate.

I don't know your knowledge level for iOS development stuff but I think making an example iOS project for coreaudio-sys or coreaudio-rs using xcodegen would be useful. I did something like this for uikit-sys. There's also some useful things in a blogpost I wrote about using uikit-sys as well.

Eventually, using cargo-dinghy in CI would also be a nice touch.

Anyway, I hope this is useful and gives some perspective.

[MichaelHills]

Success! I managed to play a wav directly with Rodeo. The API has changed a bit in Rodeo/CPAL latest versions compared to where bevy is at, so I don't know if something is broken with that integration now on the newer versions. (Short story is that the cpal stream seems to not be retained and gets dropped from memory and so the mixer weak reference is null.)

Here's what I did

• Rebased/squashed @simlay 's coreaudio-sys change on master and backed out the bindgen for the objective-c headers (wasn't needed)
• Manually fixed coreaudio-rs compile error, I think that code path is unused though
• No changes to rodio
• Hacked up cpal coreaudio by using the webaudio host as a "simple" starting point for a single output device, then hacked in a build_output_stream_raw implementation based off https://stackoverflow.com/a/14478420 and the original cpal macos stuff. I completely butchered this file, that's possibly why I had issues with bevy/rodio integration.

This might take a while to clean up the hacks. i.e. no mic input, and no device enumeration atm. There is a bunch of shared code with the macos coreaudio. The existing enumeration is macOS-specific. It might be hard to untangle this stuff.

I guess I'm initially thinking splitting up the cpal code from mod.rs / enumeration.rs to something like:

• mod.rs
• macos.rs
• macos_enumeration.rs
• ios.rs
• shared.rs

Then use target_os switch to pull in the correct implementation.

[MichaelHills]

I applied the audio_output.rs patch from here #310 and now I can finally play audio through Bevy and iOS. :)

[MichaelHills]

An update on where things are at:

• With @simlay 's An initial xcode setup for using xcode #539 we have an iOS example building out of the box! No local hacks required. No touch and no sound though.
• @naithar already landed simulator metal support upstream and it's been released, so simulator works great
• I landed a fix in winit so that starting the app in landscape mode the window size works fine (needs new winit release though)
• I'm working with @simlay with updates to coreaudio-sys Fixes iOS compiling. RustAudio/coreaudio-sys#33 and coreaudio-rs Added initial changes for iOS support. RustAudio/coreaudio-rs#69 as part of the audio support. Meanwhile I've been reading up on coreaudio docs to understand best to how implement the cpal host properly and productionize my hack.
• Implement basic touch support #334 seems to have stalled. I posted my touch support diffs earlier in this thread if you scroll up, that's what I'm currently using. If someone wants to look at that PR and look at my diffs and take over touch support that'd be helpful. 👍
• I filed a crash issue over at winit Crash in iOS when swiping up for the control center rust-windowing/winit#1705 but I can't seem to reproduce it on the latest bevy ios example even though I could reproduce it earlier. Once I get to rebasing my game to latest bevy I'll test it again.

Once touch and audio are working, I think we can call it as Bevy having iOS support? There might be bugs here and there and we can fix them as we go.

[cart]

#334 seems to have stalled. I posted my touch support diffs earlier in this thread if you scroll up, that's what I'm currently using. If someone wants to look at that PR and look at my diffs and take over touch support that'd be helpful.

Lol I can't find the diffs you're talking about because apparently my brain doesn't work 😄 I think I'll address my own comments in that pr, maybe apply your diff, then merge it. Alternatively i could address my comments and you could create a follow up pr (that way you get proper attribution).

Once touch and audio are working, I think we can call it as Bevy having iOS support? There might be bugs here and there and we can fix them as we go.

Yup that sounds good to me! I'm so happy that this came together so quickly ❤️

[MichaelHills]

Lol I can't find the diffs you're talking about because apparently my brain doesn't work 😄 I think I'll address my own comments in that pr, maybe apply your diff, then merge it. Alternatively i could address my comments and you could create a follow up pr (that way you get proper attribution).

I had implemented my own touch support in parallel to that PR because I didn't know about that PR. My comment is buried further up this page #87 (comment) (had to click load hidden items).

In my version I had the following and avoided using Input since it didn't map well. However, in my game code I never used the just_*. I just used active_touches to power my on-screen virtual joysticks and local system state to keep track of which touch id was which joystick. Perhaps the just_* stuff would be useful for UI buttons and things like that.

#[derive(Default)]
pub struct TouchInputState {
    pub active_touches: HashMap<u64, ActiveTouch>,
    pub just_pressed: HashSet<u64>,
    pub just_released: HashSet<u64>,
    pub just_cancelled: HashSet<u64>,
}

I also didn't implement TouchMotion event. I wasn't super clear on what's the correct API. Doing both TouchInputState and TouchMotion event lines up the closest with existing mouse implementation. Naming-wise TouchInputState would be better as TouchInput to line up with Input.

Probably best to just clean up and merge that other one and then I can follow up with anything useful from my diff. e.g. having dx() and dy() helpers was useful for calculating distance moved from when the touch was initiated. Might want to call it delta and return a Vec2 instead or something, not sure.

[MichaelHills]

Just an FYI, my personal situation has changed and I'll be strapped for time to continue developing my game / Bevy iOS support for the time being. I intend to finish off the audio support but will hand over a WIP cpal branch if I need to. Happy to keep helping out with testing though!

Once all the other pieces land, I still plan to improve the iOS example project with the following:

• add assets file/directory reference to show how to use assets (purely because I did it the wrong way at first which led me down the bad path of Xcode compressing/destroying the PNGs)
• add some png texture
• add audio background track (and maybe a press a button to play a sound)
• some kind of touch support, maybe just adding camera pitch/yaw look around

[CleanCut]

@MichaelHills Thank you for all your hard work. I love being around great contributors like you who push projects forward in meaningful ways. I hope you are able to return to your game in the future!

[cart]

@MichaelHills I agree with @CleanCut. You've done a bunch of great work already. Don't feel pressured to put in more work than you want to!

[dvtkrlbs]

The new version of rodio is now released. I am working on PR. (this helps with audio not working on macOS and ios)

[naithar]

I've made some changes to original @MichaelHills touch support diff here: In my fork
I plan to make a PR if #334 wouldn't be moving anywhere if that's okay :)
I'll also run some more test just to be sure that everything is working as intended.

[MichaelHills]

Update on audio:

• Fixes iOS compiling. RustAudio/coreaudio-sys#33
• Fix iOS compilation error and add macOS + iOS feedback example applications RustAudio/coreaudio-rs#72
• Refactor coreaudio host and add iOS support + ios-feedback example RustAudio/cpal#485

[dvtkrlbs]

Another problem is it takes a long time for a new version of rodio to release on crates.io. So even if these issues gets fixed you wont be able use them for a long time (rodio makes new releases for every 4 months or so) so you need to maintain your own bevy-audio (since crates.io crates cant use git dependencies).

[MichaelHills]

Another problem is it takes a long time for a new version of rodio to release on crates.io. So even if these issues gets fixed you wont be able use them for a long time (rodio makes new releases for every 4 months or so) so you need to maintain your own bevy-audio (since crates.io crates cant use git dependencies).

I haven't needed to modify Rodio. Does that mean we're ok? Or at worst can just use [patch.crates-io] to change the cpal version if there's a significant version bump.

[MichaelHills]

I've made some changes to original @MichaelHills touch support diff here: In my fork
I plan to make a PR if #334 wouldn't be moving anywhere if that's okay :)
I'll also run some more test just to be sure that everything is working as intended.

@naithar Looks like no movement on #334, want to put up yours? Or give some diffs so that #334 can be landed with original attribution, and then put up another PR for enhancements?

[naithar]

@MichaelHills yeah, seems like it. I'll rebase and make a PR soon.

[naithar]

Well, I found our problem! You need to disable XCode "Remove text metadata from PNG". I disabled everything though just to be sure... I guess the image loading library can't handle whatever XCode does to it.

this shouldn't be required anymore after #693. Listing assets folder in Copy bundle resources is enough for assets to work. Tested with iOS device and macOS .app file.

Also there is no need for a workaround for set_current_dir as PR seems to be solving this too.

[francesca64]

Is there any interest in using cargo-mobile to simplify generating Xcode projects / building / running on device / etc?

[cart]

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

[alice-i-cecile]

This now works!

[Rust-Ninja-Sabi]

Is there any tutorial about Bevy and ios today?