Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destroying MediaPlayer Gameobject stuck the main thread for over 20 seconds / Dolby not supported in ExoPlayer #1302

Closed
zyfeng413 opened this issue Sep 22, 2022 · 31 comments
Assignees
Labels
Android Android platform Fixed A fix will be in the next release question Further information is requested
Milestone

Comments

@zyfeng413
Copy link

zyfeng413 commented Sep 22, 2022

When the network video is opened without network or the network is interrupted in the middle of playing the network video,
the main thread will be stuck for a long time when quitting the playing, The m_Video.Call("CloseVideo") interface was found to be stuck.How can I solve this problem? Thank you!

environment:
Unity 2020.3.30
Platform:
Android

@zyfeng413 zyfeng413 added the question Further information is requested label Sep 22, 2022
@zyfeng413 zyfeng413 changed the title Destroying this Gameobject stuck the main thread for over 20 seconds Destroying MediaPlayer Gameobject stuck the main thread for over 20 seconds Sep 22, 2022
@Chris-RH Chris-RH added the Android Android platform label Sep 22, 2022
@Ste-RH
Copy link
Collaborator

Ste-RH commented Sep 23, 2022

What version of AVPro Video are you using?

What API Path are you using?

Which device(s) have you seen this on?

Can you send over a full logcat please?

@zyfeng413 zyfeng413 reopened this Sep 23, 2022
@Ste-RH
Copy link
Collaborator

Ste-RH commented Sep 23, 2022

@zyfeng413 On Android, the MediaPlayer API opens/closes videos on a thread, and so should not hit the main thread much/at-all. This is unfortunately something we cannot do in the ExoPlayer API path currently. So, if you are not already using the MediPlayer API path, and can, it might be worth trying that.

@zyfeng413
Copy link
Author

zyfeng413 commented Sep 24, 2022

@Ste-RH AVPro Video - Ultra Edition v2.6.2
API Path is MediaPlayer, the devices is Car machine SOC, I'll try a few more devices
recurrence:
Call mediaPlayer.openMedia without a network, then destroy MediaPlayer.gameObject

@Ste-RH
Copy link
Collaborator

Ste-RH commented Sep 26, 2022

I would also try using ExoPlayer to see if it is different

@zyfeng413
Copy link
Author

zyfeng413 commented Sep 26, 2022

I see this log from main.txt
W m.xiaopeng.nap: Long monitor contention with owner Binder:3037_1 (3066) at java.net.InetAddress[] libcore.io.Linux.android_getaddrinfo(java.lang.String, android.system.StructAddrinfo, int)(Linux.java:-2) waiters=0 in void android.media.MediaHTTPConnection.disconnect() for 26.129s
I hope it helps with the analysis.I also tried two other mobile devices (HarmonyOS 2.0.0 and Android 8.1.0), but did not reproduce the problem. Currently available only on Car Machine SOC (Android 10).

@Ste-RH
Copy link
Collaborator

Ste-RH commented Sep 27, 2022

Can you provide a link to the device? I am struggling to find it.

When does the pause happen? During the GO destroy? Could you try to wind it back a little more sensibly? (wait for the open to fail, then call Dispose() on the media player before the GO destroy)

It does feel like a driver or OS level issue with that device though - as you, us, other developers, are not seeing this issue on other devices. The IO logging you point at is not called directly by us in AVPro Video c# or plugin code - that is not to say we are not calling API level functions that do not call them. The MediaPlayer API path implementation in AVPro Video is threaded and so should not hit the main thread at all.

@zyfeng413
Copy link
Author

I'm sorry that I can't provide the device link at present, because our company's car is still in the unmarketed stage.
I can provide more logs to help with analysis.

"UnityMain" prio=5 tid=24 Blocked
| group="main" sCount=1 dsCount=0 flags=1 obj=0x155c0ed8 self=0x7ae5d31400
| sysTid=20649 nice=0 cgrp=default sched=0/0 handle=0x7a85a74d50
| state=S schedstat=( 58827471983 565784310 38244 ) utm=5401 stm=481 core=7 HZ=100
| stack=0x7a8596a000-0x7a8596c000 stackSize= 1071KB pos = 0x1488aeb0 thread_local_limit = 0x148c0000 thread_local_objects = 90
| held mutexes=
at android.media.MediaHTTPConnection.disconnect(MediaHTTPConnection.java:171)

  • waiting to lock <0x05d80245> (a android.media.MediaHTTPConnection) held by thread 162
    at android.media.IMediaHTTPConnection$Stub.onTransact(IMediaHTTPConnection.java:132)
    at android.os.Binder.execTransactInternal(Binder.java:1021)
    at android.os.Binder.execTransact(Binder.java:994)
    at android.media.MediaPlayer._reset(Native method)
    at android.media.MediaPlayer.reset(MediaPlayer.java:2145)
    at com.RenderHeads.AVProVideo.AVProVideoMediaPlayer.CloseVideoOnPlayer(AVProVideoMediaPlayer.java:82)
    at com.RenderHeads.AVProVideo.AVProVideoPlayer.CloseVideo(AVProVideoPlayer.java:408)
    at com.unity3d.player.UnityPlayer.nativeRender(Native method)
    at com.unity3d.player.UnityPlayer.access$600(unavailable:-1)
    at com.unity3d.player.UnityPlayer$g$1.handleMessage(unavailable:-1)
    at android.os.Handler.dispatchMessage(Handler.java:103)
    at android.os.Looper.loop(Looper.java:263)
    at com.unity3d.player.UnityPlayer$g.run(unavailable:-1)

"Binder:20618_8" prio=5 tid=162 Native
| group="main" sCount=1 dsCount=0 flags=1 obj=0x132c1310 self=0x78e99d2800
| sysTid=20987 nice=0 cgrp=default sched=0/0 handle=0x7914382d50
| state=S schedstat=( 577469522 307989946 1534 ) utm=55 stm=2 core=3 HZ=100
| stack=0x7914284000-0x7914286000 stackSize= 1023KB pos = 0x0 thread_local_limit = 0x0 thread_local_objects = 0
| held mutexes=
kernel: (couldn't read /proc/self/task/20987/stack)
native: #00 pc 00000000000cd5e4 /apex/com.android.runtime/lib64/bionic/libc.so (read+4)
native: #1 pc 00000000000d8e40 /apex/com.android.runtime/lib64/bionic/libc.so (__sread+40)
native: #2 pc 00000000000d8d3c /apex/com.android.runtime/lib64/bionic/libc.so (__srefill+260)
native: #3 pc 00000000000dce74 /apex/com.android.runtime/lib64/bionic/libc.so (fread_unlocked+204)
native: #4 pc 00000000000dcd64 /apex/com.android.runtime/lib64/bionic/libc.so (fread+72)
native: #5 pc 0000000000098ec0 /apex/com.android.runtime/lib64/bionic/libc.so (android_getaddrinfofornetcontext+2396)
native: #6 pc 000000000009853c /apex/com.android.runtime/lib64/bionic/libc.so (android_getaddrinfofornet+56)
native: #7 pc 000000000002c6f8 /apex/com.android.runtime/lib64/libjavacore.so (Linux_android_getaddrinfo(_JNIEnv*, _jobject*, _jstring*, _jobject*, int)+336)
at libcore.io.Linux.android_getaddrinfo(Native method)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:200)
at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:74)
at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
at java.net.InetAddress.getAllByName(InetAddress.java:1152)
at com.android.okhttp.Dns$1.lookup(Dns.java:41)
at com.android.okhttp.internal.http.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:178)
at com.android.okhttp.internal.http.RouteSelector.nextProxy(RouteSelector.java:144)
at com.android.okhttp.internal.http.RouteSelector.next(RouteSelector.java:86)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:176)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:542)
at android.media.MediaHTTPConnection.seekTo(MediaHTTPConnection.java:284)

  • locked <0x05d80245> (a android.media.MediaHTTPConnection)
    at android.media.MediaHTTPConnection.getMIMEType(MediaHTTPConnection.java:459)
  • locked <0x05d80245> (a android.media.MediaHTTPConnection)
    at android.media.IMediaHTTPConnection$Stub.onTransact(IMediaHTTPConnection.java:159)
    at android.os.Binder.execTransactInternal(Binder.java:1021)
    at android.os.Binder.execTransact(Binder.java:994)

@Ste-RH
Copy link
Collaborator

Ste-RH commented Sep 30, 2022

It looks like something has called 'seek' (not evident as to what this is as the callstack does not show enough information, and that seek is locking the main thread? Because the 'close' calls the MediaPlayer 'reset()' on a Runnable/Thread model.

I guess we are a bit stuck as the device you are having issue with is not available to us in any form! Even with the device in hand we could not promise to be able to address the issue. Though, it does appear to be limited to that device specifically? Is there anything 'special' about the version of Android on the device? Or the hardware used in the device?

@Chris-RH
Copy link
Collaborator

Chris-RH commented Oct 7, 2022

@zyfeng413

Is there any further information that you could provide? Is there anything 'special' about the version of Android on the device? Or the hardware used in the device?

@zyfeng413
Copy link
Author

zyfeng413 commented Oct 9, 2022

@Ste-RH @Chris-RH What's special about this device is that even if I turn off wifi and 4G, there's a built-in weak network that the system doesn't consider offline

@zyfeng413
Copy link
Author

@Ste-RH @Chris-RH Whether we can set the timeout period through the interface or other means?

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 10, 2022

Any timeout is beyond the scope of AVPro Video as (both) the network calls are happening deep inside the Android SDK

@zyfeng413
Copy link
Author

zyfeng413 commented Oct 12, 2022

This bothered me so much that I tried switching the API Path to ExoPlayer, which won't stuck the main thread, but unfortunately doesn't support playing our Dolby videos. But I can play it using the official exoplayer demo.

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 12, 2022

We will take a look to see if we can add support for the format you are using

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 12, 2022

The ExoPlayer demo app reports back that 'audio tracks are present but not supported by this device'.

Track info logged out is:

EventLogger: tracks [eventTime=0.36, mediaPos=0.00, window=0, period=0
EventLogger: group
EventLogger: [
EventLogger: [X] Track:0, id=1, mimeType=video/avc, codecs=avc1.640032, res=2400x1200, fps=25.0, supported=YES
EventLogger: ]
EventLogger: group
EventLogger: [
EventLogger: [ ] Track:0, id=2, mimeType=audio/eac3-joc, channels=6, sample_rate=48000, language=und, supported=NO_UNSUPPORTED_TYPE
EventLogger: ]
EventLogger: ]

I am using a Xiaomi Mi 10T

There is no reason I can see that, if the device supports eac3 then both ExoPlayer demo and AVPro Video would not play it back.

I will try some more devices I have in hand to see if any of them support eac3

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 12, 2022

The only other device I have to hand the might play it is an Oculus Quest 2, and it does not - either in the ExoPlayer demo app or AVPro Video. Again, it says the audio track is unsupported.

Maybe you could send over a full logcat of yo running the ExoPlayer demo app? You can email it over to unitysupport@renderheads.com if you wish to keep it private.

@zyfeng413
Copy link
Author

zyfeng413 commented Oct 14, 2022

I have consulted our colleagues in the system department, our system is Android10, and there are two modifications to the ExoPlayer demo to support audio. Here are the full logcat and code changes that play normally.
log.txt
img_v2_1978f788-079c-4e27-981e-013810fe4cag
img_v2_03aa9c30-dc36-4b00-b810-218fa690379g

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 14, 2022

Whilst we could, we have a policy to not change the core ExoPlayer code - so we cannot implement your custom changes. You will need to take them to ExoPlayer themselves to get support added.

That aside, I was curious and so tried a version of ExoPlayer with your changes in. They do not work for me. Maybe because your code is looking for 12 channels, and the video you sent states it is 6.

Track:0, id=2, mimeType=audio/eac3-joc, channels=6, sample_rate=48000, language=und, supported=NO_UNSUPPORTED_TYPE

@zyfeng413
Copy link
Author

This mimeType information is not entirely correct because it is Dolby ATMOS, the mime information is only referable, and the actual output channel in our system is 12

@zyfeng413
Copy link
Author

zyfeng413 commented Oct 18, 2022

We did a couple of comparison tests:

  1. ExoPlayer Demo in Car machine SOC: Regular video support but Dolby video Not Support
  2. ExoPlayer Demo with custom changes in Car machine SOC: Regular video and Dolby video all support
  3. AVPro v2.6.4 in Car machine SOC: Regular video and Dolby video all not support(no display and no sound)
  4. AVPro v1.9.4 in Car machine SOC: Regular video support but Dolby video Not Support(no sound)
  5. AVPro v2.6.4 in mobile phone: Regular video support but Dolby video Not Support(no sound)
  6. AVPro v1.9.4 in mobile phone: Regular video support but Dolby video Not Support(no sound)

Is there a way to fix this?

@Chris-RH
Copy link
Collaborator

As @Ste-RH mentioned, we could look into supporting Dolby video.

@zyfeng413
Copy link
Author

Thank you so much for your support. Look forward to good news on further support.

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 18, 2022

As previously stated. We have a policy of not adding to the core ExoPlayer code that we distribute with AVPro Video.

It appears you need your custom code added to the ExoPlayer codebase in order for it to work.

If your changes are generic, then you would be best advised to raise it on the ExoPlayer issues board, or create a pull request and add them into the codebase for review yourself.

If your changes are custom, then I would think ExoPlayer will not entertain its inclusion, just as we will not at RenderHeads. There is a solutions though. You could build your own set of ExoPlayer libraries up with your changes and use those. The only change we make to the core codebase is changing the package name to 'com.google.android.exoplr2avp'. To find out what version of ExoPlayer your version of AVProVideo is using, you can filter your logcat to 'ExoPlayerLib'.

@Ste-RH
Copy link
Collaborator

Ste-RH commented Oct 25, 2022

Bit of housekeeping on this issue. Link to ExoPlayer issue: google/ExoPlayer#10701

@Ste-RH Ste-RH changed the title Destroying MediaPlayer Gameobject stuck the main thread for over 20 seconds Destroying MediaPlayer Gameobject stuck the main thread for over 20 seconds / Dolby not supported in ExoPlayer Oct 26, 2022
@stale stale bot added the stale Stale label Nov 12, 2022
@Chris-RH Chris-RH added the pinned Prevent from being removed by stale-bot label Nov 14, 2022
@stale stale bot removed the stale Stale label Nov 14, 2022
@RenderHeads RenderHeads deleted a comment from stale bot Nov 16, 2022
@Ste-RH
Copy link
Collaborator

Ste-RH commented Nov 16, 2022

The ExoPlayer dev team have made some changes to the library. This will naturally follow into AVPro Video when they next release a version of ExoPlayer and we integrate it into AVPro Video. Once @zyfeng413 confirms the changes fix the issue we will send over a new set of ExoPlayer libs for testing within AVPro Video to verify.

@Ste-RH
Copy link
Collaborator

Ste-RH commented Dec 9, 2022

Looks like there has been some code added to ExoPlayer v2.18.2 for this. We will update AVPro Video to use the latest release for ExoPlayer in our next release.

@Ste-RH Ste-RH added this to the 0 - Next Release milestone Dec 9, 2022
@Ste-RH Ste-RH added Fixed A fix will be in the next release and removed pinned Prevent from being removed by stale-bot labels Dec 9, 2022
@Ste-RH Ste-RH self-assigned this Dec 9, 2022
@stale
Copy link

stale bot commented Dec 26, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale Stale label Dec 26, 2022
@Chris-RH Chris-RH removed the stale Stale label Dec 28, 2022
@stale
Copy link

stale bot commented Jan 7, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale Stale label Jan 7, 2023
@Chris-RH Chris-RH removed the stale Stale label Jan 9, 2023
@Ste-RH
Copy link
Collaborator

Ste-RH commented Jan 10, 2023

AVPro v2.7.0 was updated to use ExoPlayer v2.18.2. Do let us know if this addresses your issue @zyfeng413

@Chris-RH
Copy link
Collaborator

Any update @zyfeng413 ?

@Chris-RH
Copy link
Collaborator

I'm going to close this for now, as neither thread has had any contact from the OP for a while now. Please reopen if you still require assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Android Android platform Fixed A fix will be in the next release question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants