New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pion WebRTC RTP stream behavior - Media latching #2585
Comments
Hey @gbfarah You need to do a ICE Restart by default when a candidate has gone to selected it sits on the singular candidate. I like your suggestion and I think it should totally be possible. Maybe we make this an interface and you could have any custom logic you like? This would also satisfy the asks for people who want no ICE at all. Like this @lisay-yan What do you think of an implementation like this? I don't think we can offer a No ICE mode (because it isn't spec compliant). However we could allow you to implement your own selection logic where you could do whatever you want! |
Hi Sean, I think the simplest approach is to add a new boolean setting to Peerconnection (defaulted to false) called enableMediaLatching (true/false).... If the setting is true the peerconnection tracks the IP port of the incoming track and dynamically switches to outbound track accordingly. Let me know your thoughts |
I put up a proposal here I would love your thoughts @gbfarah! I want to support your use case, but I don't think we can make it just a boolean. I am worried about security/interoperability issues. I also think users may want the same feature (but with slight differences in behavior) I think the API suggested will cover everything. I can write all the code if you are in support of it. |
Hi Sean,
Thanks for the followup... From a security point of view I'm of the view
there is in fact no issue with boolean ... Keep in mind DTLS negotiates a
secret private key with consideration to the fingerprint as carried in the
SDP (ie man in middle attack not possible)
Currently pion appears to be accepting RTP from any endpoint (without
enforcement of IP addresses as detected by ice). Hence why in a IP address
change situation we are actually getting one way audio (inbound stream
accepted. Outbound stream still going to old address). All that we are
doing is making an allowance that the outbound RTP is symmetrically passed
back to the a same IP and port from where legitimate packets (ie ones that
successfully decrypted and the peer connection is passing up the track)
came from...
In terms of your suggested solution , can you please walk me through how it
addresses a mid stream IP address change. If is able to address above
mentioned issue that would be great
I really appreciate your help and feedback on this.
Thanks
…On Mon, Sep 25, 2023 at 4:36 AM Sean DuBois ***@***.***> wrote:
I put up a proposal here <pion/ice#623> I would
love your thoughts @gbfarah <https://github.com/gbfarah>!
I want to support your use case, but I don't think we can make it just a
boolean. I am worried about security/interoperability issues. I also think
users may want the same feature (but with slight differences in behavior)
I think the API suggested will cover everything. I can write all the code
if you are in support of it.
—
Reply to this email directly, view it on GitHub
<#2585 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC4WBL3NYLMYQJHOB242BKLX4B4RDANCNFSM6AAAAAA5A2LKAU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@Sean-Der Another interesting observation. Currently pion server appears to be doing media latching in the situation where the client end SDP contains no public candidate (ie all candidates are private addresses whilst pion is in cloud) .. (this can be replicated by simply disabled stun/ice server on the client).. In other words pion/webrtc must have some code that is performing the latching mechanism today where its trust inbound stream and responds to it on the basis of where it has come from. .All we really need to have this mechanism by boolean such that latching is constantly tracked on incoming packets (not just the ones at the start of a stream) Let me know your thoughts |
@Sean-Der Any feedback on this as this has become a little urgent. We are detecting some providers such at T-Mobile changes addresses every couple of minutes (sometimes multiple times a minute). Is there a quick solution whereby the latching functionality that is mentioned in the post directly above can be run constantly.... I don't believe there is a security issue as the two ends are using symmetric key encryption by this point |
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Sorry this took so long @gbfarah! I have opened a PR for this. You will get notified of incoming STUN traffic, and you switch candidates as you wish. Would love to hear your thoughts and hopefully it gets what you need |
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
Allow the user to perform custom processing for inbound STUN Binding requests. This allows users to do some of the following * Log incoming Binding Requests for debugging * Implement draft-thatcher-ice-renomination * Implement custom CandidatePair switching logic Resolves pion/webrtc#2539 Resolves pion/webrtc#2585 Resolves #623
@Sean-Der Thanks for the followup on this. In terms of feedback. I'm not sure if you saw my last comment on this thread. We were really hoping for a solution where you can prime a channel that it is running in latching mode and it dynamically latches onto streams.... Pion is already doing what we require when STUN is disabled in that it dynamically identifies and responds to an inbound stream.. What we really would like is that it continues to monitor the inbound stream and responds again if there is a network change. Correct me if I'm wrong the proposed solution requires a signalling of new candidates (something that we would prefer not to do) as we want to keep signalling as loosely coupled as possible Currently pion server appears to be doing media latching in the situation where the client end SDP contains no public candidate (ie all candidates are private addresses whilst pion is in cloud) .. (this can be replicated by simply disabled stun/ice server on the client).. In other words pion/webrtc must have some code that is performing the latching mechanism today where it trust inbound stream and responds to it on the basis of where it has come from. .All we really need to have this mechanism by boolean such that latching is constantly tracked on incoming packets (not just the ones at the start of a stream) |
I believe this does what you need! If you set the The initial media latching is happening because of We can't latch on media traffic because that makes a replay attack possible. A attacker could spoof the IP+Port and then you would start sending traffic at a unsuspecting target. ICE requires bi-directional communication. When someone sends you a Binding Request you must make a Binding Response. Just to make sure you get an ACK from the remote. |
Many thanks Sean... Does this mean that the client is the forced to perform
an ice regathering upon detecting a disconnection ? From a client
perspective it is an operation of no value given STUN is disabled ?
By latching onto media stream the client needs to do nothing but continue
streaming to the server (running pion)
In relation to replay attack, keep in mind DTLS (and underlying RTP) both
incorporate measures to prevent replay as part of their protocol.
Many thanks for your efforts and follow-up
…On Thu, 2 May 2024, 11:38 pm Sean DuBois, ***@***.***> wrote:
I believe this does what you need! If you set the BindingRequestHandler
and return true it will latch to the latest incoming ICE Traffic. I
tested this today I moved my phone between Wifi/Cell and I had no
interruption in service.
The initial media latching is happening because of Peer Reflexive
Candidates. When a ICE Agent sees inbound traffic that is authenticated
it will create a new candidate for it.
We can't latch on media traffic because that makes a replay attack
possible. A attacker could spoof the IP+Port and then you would start
sending traffic at a unsuspecting target. ICE requires bi-directional
communication. When someone sends you a Binding Request you must make a
Binding Response. Just to make sure you get an ACK from the remote.
—
Reply to this email directly, view it on GitHub
<#2585 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC4WBL44LNPSP5KEUGVJPXDZAI6V7AVCNFSM6AAAAAA5A2LKAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJQGUZDKNRUGM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Nope! The ICE Agent is constantly sending STUN Binding requests (to keep the connection alive). When the client roams it doesn't need to do anything. Pion will just see ICE traffic coming from a new IP/Port (and you can switch to it if you wish)
If it helps I am happy to get on a call and we can discuss details how it works/get a demo together https://siobud.com/meeting anything I can do to help I am here! |
Hi @Sean-Der Currently I have this code which kills peerConnection if ice connection fails (which I presume would need to change ?? or incorporate the new feature)
Many thanks in advance |
I am using Pion in a server setting where clients (which may be mobile) connect to it. The clients may have frequent IP address changes during a WebRTC call..
I am noticing the following behaviour. When an clients IP address changes, the incoming stream from client to Pion server is still being serviced by webrtc peerconnection (which is great) . The outbound stream from server to client still appears to be continuing to the original IP address.
Is there a way of manipulating the peerconnection to send RTP to the IP address port on which it receives an RTP stream. In telecom speak this is referred to as Comedia / Media latching being
"The mechanism to get the Public IP/PORT for return media from the first received media packets instead of using STUN/TURN to modify the IP/PORT inside the SDP is called “Media Latching”."
From a security standpoint I don't think media latching is an issue as the cryptographic side of both endpoints is untouched and they have communicated SDPs on external secure signalling channel.
The fact that one stream (the ingress stream from client) works means we are effectively half way there.
Any feedback is appreciated
The text was updated successfully, but these errors were encountered: