Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new ClientHellos #122

Merged
merged 2 commits into from Oct 11, 2022
Merged

Conversation

hwh33
Copy link
Contributor

@hwh33 hwh33 commented Oct 8, 2022

Also add faked support for token binding, ALPS, and delegated credentials.

This add new ClientHelloSpec based on:

  • Firefox 105
  • Chrome 106
  • Edge 106
  • Safari 16.0
  • 360 Secure Browser 11.0
  • QQ Browser 11.1

Unfortunately, the specs based on Edge 106 and 360 11.0 seem to incompatible with this library. I haven't had time yet to investigate why. I added the specs anyway in case they are useful to some people. Perhaps they will function in limited use cases. I did not update HelloEdge_Auto and Hello360_Auto to avoid breaking code dependent on those references.

@hwh33
Copy link
Contributor Author

hwh33 commented Oct 8, 2022

There's been some discussion about recent GFW blocking in net4people/bbs#129. In that thread, there was discussion about TLS fingerprinting and this library was specifically called out as a means to combat this. I can't say for sure that these latest ClientHellos will combat any fingerprinting the GFW is doing, but they should at least help clients look more like normal browser traffic.

@hwh33
Copy link
Contributor Author

hwh33 commented Oct 8, 2022

The diff is a little mangled for u_parrots.go. I didn't modify existing hellos, I only added new ones.

Actually the diff was correct. I'd goofed things on my end. Fixed now!

@hwh33 hwh33 force-pushed the harry/new-hellos branch 2 times, most recently from b702e96 to c82dafa Compare October 8, 2022 06:48
@gaukas
Copy link
Member

gaukas commented Oct 8, 2022

Happy to get this merged.

But does Edge 106 (also other browsers) use a different fingerprint than one of these existing Chrome fingerprint? I'm under the impression that it is based on Chromium.

@hwh33
Copy link
Contributor Author

hwh33 commented Oct 8, 2022

But does Edge 106 (also other browsers) use a different fingerprint than one of these existing Chrome fingerprint? I'm under the impression that it is based on Chromium.

Yeah, that's a good question. I haven't checked equality of these new hellos against any of the Chrome hellos beyond 83. I did this work on a fork; I didn't realize some newer hellos had been integrated upstream. You are correct that Edge is based on Chromium (and 360 and QQ IIRC).

I don't have time to check equality right now, but I can try to do so in the next couple of days.

@gaukas
Copy link
Member

gaukas commented Oct 8, 2022

I will run a check later based on your fork and let you know the result, and we can go from there.

@gaukas
Copy link
Member

gaukas commented Oct 8, 2022

I think HelloChrome_106 duplicates HelloChrome_102. For compatibility consideration, I would recommend not including HelloChrome_106 for now.

Otherwise, all good. All ClientHello fingerprints are unique.

u_common.go Outdated Show resolved Hide resolved
u_common.go Outdated Show resolved Hide resolved
Also add faked support for token binding, ALPS, and delegated credentials
@hwh33
Copy link
Contributor Author

hwh33 commented Oct 9, 2022

Thanks for running those checks @gaukas! I updated according to your suggestions.

@gaukas gaukas self-requested a review October 9, 2022 04:41
@gaukas
Copy link
Member

gaukas commented Oct 9, 2022

Thanks @hwh33!
And just one more thing before I get this merged... If I understand correctly, I think ApplicationSettingsExtension duplicates with the newly added FakeALPSExtension? Since we are not really implementing any functional support for ApplicationSettingsExtension, I think we'd rather keep the FakeALPSExtension and get rid of the former one. What do you think?

@hwh33
Copy link
Contributor Author

hwh33 commented Oct 11, 2022

Oh, I didn't notice ApplicationSettingsExtension. Hmm I agree that FakeALPSExtension is a better name since, as you say, there is no functional support for ALPS. However, removing ApplicationSettingsExtension would be a breaking change for users of this library.

I would recommend keeping the existing type and adding a note to indicate that it is a faked extension. I've done this here, in a separate commit: 432a2f0. What do you think?

I can squash commits before merging or keep both for posterity. Let me know which you would prefer.

common.go Show resolved Hide resolved
@gaukas
Copy link
Member

gaukas commented Oct 11, 2022

I'm think we may instead keep an alias.

type ApplicationSettingsExtension = FakeALPSExtension

or the other way.

I wonder, are there any difference between FakeALPSExtension and ApplicationSettingsExtension in your last commit, except for their naming?

@gaukas
Copy link
Member

gaukas commented Oct 11, 2022

But I guess that's another issue. How about this: I will get this merged, and we fix that issue with a new PR.
How does this sound to you?

Copy link
Member

@gaukas gaukas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@gaukas gaukas merged commit 425e019 into refraction-networking:master Oct 11, 2022
@blakebyrnes
Copy link
Contributor

Sorry to weigh in late on this, but the Chrome signature for 98+ is a tiny bit wrong. They changed in Chrome 98 to only include tls versions 1.3 and 1.2. They removed 1.1 and 1.0. Here's a link to the signature:
ulixee@e8f6488

We have a suite that automates testing tls signatures across browsers and operating systems here, if it's of use:
test suite: https://github.com/unblocked-web/double-agent/tree/41489d6f260cc3861574a0f606e99d04b3bfe799
data: https://github.com/unblocked-web/browser-profile-data/tree/main/profiles

Also potentially of note: we did an implementation of encrypted extensions in our branch. I don't think I can do a PR to your codebase though without a good bit of thought put into how additional handshake messages should be integrated into this project. Right now, you have to scatter them throughout the tls codebase. We pretty much just ported in from the BoringSSL test runner, but I didn't port in server side or unit tests, so that would be needed on top of this.
ulixee@8d0f1c4

FYI: Google sites will reject uTLS without this implementation.

@gaukas
Copy link
Member

gaukas commented Oct 12, 2022

They changed in Chrome 98 to only include tls versions 1.3 and 1.2.

Nice catch! I think we could make a change if that is the case... But anyways, in the latest Chrome 102's fingerprint we are only including TLS 1.3 and TLS 1.2 in supported versions.

encrypted extensions in our branch

Interesting. Does crypto/tls implement such? If so we still have a pending upstream synchronization. If not... we always welcome PRs!

Google sites will reject uTLS without this implementation.

Interesting observation. I thought Google was rejecting uTLS simply because we didn't implement ALPS extension.

@blakebyrnes
Copy link
Contributor

Nice catch! I think we could make a change if that is the case... But anyways, in the latest Chrome 102's fingerprint we are only including TLS 1.3 and TLS 1.2 in supported versions.

Oh, I totally missed 102 since it's out of order with the rest of the Chromes! My bad.

Interesting. Does crypto/tls implement such? If so we still have a pending upstream synchronization. If not... we always welcome PRs!

crypto/tls implements handshake messages.. which is where some of this code has to live. My point is that uTLS hasn't created a structure like the u_conn and u_common whereby connection state, and new handshake messages can be added. That means this codebase is going to get very messy for things like Compressed Certificates and Encrypted Extensions as long as there's no pattern. I'm honestly not sure the right pattern here.. I have a little bit of a hard time following the current flow.

Interesting observation. I thought Google was rejecting uTLS simply because we didn't implement ALPS extension.

Sort of the same thing. ALPS "implementation" is to respond to other handshake messages initiated by the server. It's a movement of "TLS extensions" into an encrypted follow-up handshake message, instead of broadcasting in the clear in the clientHello.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants