Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade trojan-go client's TLS fingerprints to some of the most popular ones #464

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

gfw-report
Copy link

trojan-go开发者你们好,

在这个pull request中,我们更新了trojan-go客户端的TLS指纹,使其与一些当下最流行的TLS指纹一致。我们希望这次更新可以缓解2022年10月3日以来的针对基于TLS翻墙软件的大规模封锁。具体来讲:

  • 将trojan-go客户端使用的Go标准库cryto/tls完全替换为uTLS
  • 将依赖的uTLS升级到最新版本v1.1.2

根据配置的不同,这个版本的trojan-go客户端会发送以下一种流行的Clienthello指纹。这些指纹已经不同于之前版本用Go的标准库发送的TLS指纹):

fingerprint 是否设置了sni TLS指纹 ID
Chrome (default) e47eae8f8c4887b6
Chrome (default) 90ac8a1dfa3b207c
iOS 133e933dd1dfea90
iOS cff7f10f631feddd
Firefox 7161e10829541aab
Firefox 56fa08d84940a06b

我们无意另起炉灶维护一个分支版本的trojan-go。我们之所以发布这个release是为了用户能够立即下载使用编译后的客户端。一旦我们的pull request请求被采纳,我们将归档我们的仓库。


Dear trojan-go developers,

In this pull request, we upgrade trojan-go client's TLS fingerprint to some of the most popular ones. We hope such change will mitigate the large-scale blocking of TLS-based censorship circumvention protocols since October 3, 2022. In particular,

  • Replace Go's standard cryto/tls with uTLS in trojan-go client.
  • Upagrade uTLS to the latest version v1.1.2.

Depending on one's configuration, the Clienthello sent by trojan-go client will look like one of the following values now (rather than the Go's TLS fingerprint in previous versions):

fingerprint sni value specified? TLS Fingerprint ID
Chrome (default) Yes e47eae8f8c4887b6
Chrome (default) No 90ac8a1dfa3b207c
iOS Yes 133e933dd1dfea90
iOS No cff7f10f631feddd
Firefox Yes 7161e10829541aab
Firefox No 56fa08d84940a06b

We do not intend to maintain a fork of trojan-go as a separate project. We made this release so that users can have compiled binaries to use immediately. We will archive our repo as soon as our pull request is merged to upstream.

We hope this change will mitigate [the large-scale blocking of TLS-based censorship circumvention protocols in China](net4people/bbs#129) since October 3, 2022.

Depending on one's configuration, the Clienthello sent by trojan-go client will look like one of the following values now (rather than the [Go's TLS fingerprint](https://tlsfingerprint.io/id/ad63dbc630ad9475) in previous versions):

| `fingerprint`      | `sni` value specified? | TLS Fingerprint ID |
| ----------- | ----------- |----------- |
| Chrome (default)       | Yes | [e47eae8f8c4887b6](https://tlsfingerprint.io/id/e47eae8f8c4887b6)|
| Chrome (default)       | No | [90ac8a1dfa3b207c](https://tlsfingerprint.io/id/90ac8a1dfa3b207c)|
| iOS       | Yes | [133e933dd1dfea90](https://tlsfingerprint.io/id/133e933dd1dfea90)|
| iOS       | No | [cff7f10f631feddd](https://tlsfingerprint.io/id/cff7f10f631feddd)|
| Firefox   | Yes | [7161e10829541aab](https://tlsfingerprint.io/id/7161e10829541aab)|
| Firefox   | No | [56fa08d84940a06b](https://tlsfingerprint.io/id/56fa08d84940a06b)|
@c2xusnpq6
Copy link

不懂就問~ 為什麼沒有中國能用的edge瀏覽器的指紋?

@gaukas
Copy link

gaukas commented Oct 6, 2022

中國能用的edge瀏覽器的指紋

Update: MS Edge does introduce new fingerprints. We will include such in the next tagged version.

Microsoft Edge shares the TLS Fingerprint with Chromium. Thus, we did not explicitly include a TLS ClientHello for MS Edge.

You are welcome to verify this using https://tlsfingerprint.io/ with your own pcap for MS Edge and Chromium.

@CXwudi
Copy link

CXwudi commented Oct 6, 2022

There is also an issue related to fingerprint and CDN #271, which I found it exists in this fork as well.

Would love to see it get fixed in this fork as well

@gaukas
Copy link

gaukas commented Oct 6, 2022

I think we had an issue earlier causing uTLS failing connections to cloudflare when parroting recent Chrome fingerprints. It was due to the implementation for certificate compression is missing while Chrome's ClientHello actively advertises it.

Issue mentioned above was fixed in refraction-networking/utls#95 and released in uTLS v1.1.1

@CXwudi
Copy link

CXwudi commented Oct 6, 2022

I think we had an issue earlier causing uTLS failing connections to cloudflare when parroting recent Chrome fingerprints. It was due to the implementation for certificate compression is missing while Chrome's ClientHello actively advertises it.

Issue mentioned above was fixed in refraction-networking/utls#95 and released in uTLS v1.1.1

But unfortunately, with this fork, the connection issue still exists

[INFO]  2022/10/06 14:28:57 adapter listening on tcp/udp: 0.0.0.0:1080
[INFO]  2022/10/06 14:28:57 No 'fingerprint' value specified in your configuration. Your trojan's TLS fingerprint will look like Chrome by default.
[WARN]  2022/10/06 14:28:57 tls sni is unspecified
[INFO]  2022/10/06 14:28:57 cert is unspecified, using default ca list
[WARN]  2022/10/06 14:28:57 empty websocket hostname
[INFO]  2022/10/06 14:29:09 socks connection from 192.168.1.18:49981 metadata outlook.office365.com:443
[INFO]  2022/10/06 14:29:09 socks connection from 192.168.1.18:49982 metadata outlook.office365.com:443
[INFO]  2022/10/06 14:29:09 socks connection from 192.168.1.18:49983 metadata mtalk.google.com:5228
[INFO]  2022/10/06 14:29:09 socks connection from 192.168.1.18:49984 metadata outlook.office365.com:443
[ERROR] 2022/10/06 14:29:09 github.com/p4gefau1t/trojan-go/proxy.(*Proxy).relayConnLoop.func1.1:proxy.go:66 proxy failed to dial connection | simplesocks failed to dial using underlying tunnel | no available mux client found | mux failed to dial | websocket failed to handshake with server | malformed HTTP response "\x00\x00\x12\x04\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x80\x00\x04\x00\x01\x00\x00\x00\x05\x00\xff\xff\xff\x00\x00\x04\b\x00\x00\x00\x00\x00\x7f\xff\x00\x00\x00\x00\b\a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"

@gaukas
Copy link

gaukas commented Oct 6, 2022

I think we had an issue earlier causing uTLS failing connections to cloudflare when parroting recent Chrome fingerprints. It was due to the implementation for certificate compression is missing while Chrome's ClientHello actively advertises it.

Issue mentioned above was fixed in refraction-networking/utls#95 and released in uTLS v1.1.1

But unfortunately, with this fork, the connection issue still exists

So it might not be a cloudflare issue, but rather a H2 issue within Trojan-Go implementation, possibly.

If possible, could you please check whether or not uTLS could perform TLS handshakes successfully with your server?

@CXwudi
Copy link

CXwudi commented Oct 6, 2022

I think we had an issue earlier causing uTLS failing connections to cloudflare when parroting recent Chrome fingerprints. It was due to the implementation for certificate compression is missing while Chrome's ClientHello actively advertises it.

Issue mentioned above was fixed in refraction-networking/utls#95 and released in uTLS v1.1.1

But unfortunately, with this fork, the connection issue still exists

So it might not be a cloudflare issue, but rather a H2 issue within Trojan-Go implementation, possibly.

If possible, could you please check whether or not uTLS could perform TLS handshakes successfully with your server?

Unfortunately, the same issue and error log, but I am using nginx to handle the SSL handshake and trojan-go is behind my nginx.

I suspect that this changes need to be restored to fix the issue

@gaukas
Copy link

gaukas commented Oct 6, 2022

need to be restored

Reverting such changes may effectively break TLS Fingerprint parroting, since you may no longer advertise h2.

A better way is to patch the functionality to support websocket under h2. I know the current version of uTLS may have issues with H2 under certain circumstances.

@gaukas
Copy link

gaukas commented Oct 6, 2022

patch the functionality to support websocket under h2

gorilla/websocket#417 seems related.

And as a temporary solution, I am thinking if it is viable to disable H2 on server ONLY. Also we need to force cloudflare's CDN nodes to disable H2 on your hostname: https://community.cloudflare.com/t/http-2-greyed-out/99786/6

By doing so, ClientHello may still advertise H2 but the negotiated ALPN will be http 1.1 due to server restrictions.

@CXwudi
Copy link

CXwudi commented Oct 6, 2022

You can't force Cloudflare to disable h2 from web UI, the button is greyed out (not sure if you can from other methods)
image
But you can for AWS CloudFront
image

But with h2 disable from AWS CloudFront, I get

[ERROR] 2022/10/06 15:55:19 github.com/p4gefau1t/trojan-go/proxy.(*Proxy).relayConnLoop.func1.1:proxy.go:66 proxy failed to dial connection | simplesocks failed to dial using underlying tunnel | no available mux client found | mux failed to dial | websocket cannot dial with underlying client | tls failed to handshake with remote server | EOF

Looks like some other issues, maybe more investigation can be done here, but original trojan-go is fine with AWS CloudFront so probably still some other issue in this fork.

@gaukas
Copy link

gaukas commented Oct 6, 2022

some other issues

Interesting. Seems the error happens during the TLS handshake. It would be nice if you would supply a pcap with sshkeylog for further analysis.

Btw, I can confirm that uTLS works great with h2 (not ws-over-h2).

`fingerprint` option now supports `Edge`, `Safari`, `360Browser`, and
`QQBrowser`, in addition to the existing `Chrome`, `Firefox`, and
`iOS` options.

Thanks to uTLS developers @hwh33 and @gaukas.
@passerby-b
Copy link

用这个版本依旧被封端口,反倒是用原版的没封

@doutuo
Copy link

doutuo commented Oct 12, 2022

原版的是 trojan-gfw 这个版的吗?

@gaukas
Copy link

gaukas commented Oct 12, 2022

Hi, we just released uTLS v1.1.3 with added ClientHello from:

  • Firefox 105
  • Chrome 106 (Duplicates 102)
  • Edge 106
  • Safari 16.0
  • 360 Secure Browser 11.0
  • QQ Browser 11.1

@gfw-report
Copy link
Author

We just updated to utls v1.1.3 to support more popular browser fingerprints: d135694

In particular, fingerprint option now supports Edge, Safari, 360Browser, and QQBrowser, in addition to the existing Chrome, Firefox, and iOS options.

Thank you @gaukas and @hwh33 for your time and efforts! That's really good work!

@Gowee
Copy link

Gowee commented Oct 13, 2022

Most of these browsers have ALPN set in their ClientHello, revealing the nominal inner protocol to an eavesdropper (mainly in TLS1.2). This results in prominent characteristics compared to cases where the eavesdropper cannot assume what is actually inside the TLS tunnel.

Even though the recent massive blocking seems to be based on more simple characteristics, I think introducing TLS fingerprints from those non-browser applications is also useful to mitigate potential issues in the future--even if they are not of the top amount of traffic on the Internet, the collateral damage to blocking them would be unacceptable.

@gaukas
Copy link

gaukas commented Oct 13, 2022

those non-browser applications

Then I am guessing you don't want to parrot a website on your proxy server either.

the collateral damage to blocking them would be unacceptable

Would you like to give an example/estimation or any insights, e.g., what are the top non-browser TLS applications and how popular are they?

@Gowee
Copy link

Gowee commented Oct 13, 2022

Then I am guessing you don't want to parrot a website on your proxy server either.

Would you like to give an example/estimation or any insights, e.g., what are the top non-browser TLS applications and how popular are they?

Even for browser-like traffic, protocol mismatch is problematic. Say, for HTTP/2, there would be plenty of control frames of typical sizes. If the eavesdropper determines H/2 is indicated by SeverHello for a TLS connection, then it is safe to expect the connection should contain repeated occurrences of TLS frames of those typical sizes and vice versa.

I didn't make any investigations. How about agent programs like Telegraf, database clients, and programs running on IoT devices (update: and other popular programming languages / TLS libraries such as Python / Ruby / Kotlin / ...)? Their traffic is incomparable to browsers on the Internet. But these kinds won't be exhaustive. The more choices we have, the lower possibility that the tunnel traffic is targeted.


Update:

@CXwudi
Copy link

CXwudi commented Nov 3, 2022

用了几天uTLS,端口被封了,提供几点反馈: 背景:最开始用的vless一键脚本,443端口在10月3号被封。后来不断更换协议、端口,vmess/trojan都换过,ip健在,但端口基本上很快就会被封。 期间服务器上还搭有一个Ark: Survival Envolved个人服务器用来和小伙伴一起玩耍,这个服务所用的几个端口也偶尔会被封,换端口即可。 大概10月28号更新xray到1.61,换用trojan+xtls, pc客户端更新V2rayN到5.37,客户端配置sni和chrome fingerprint,该配置一直到11月2号晚间一直正常。2号晚间在手机上也更新v2rayNG,同样配置sni和chrome fingerprint,看了一会儿youtube(一个30分钟左右的视频,为减少流量,画质为360p)。3号早上端口被封。

十分怀疑是和V2rayNG有关。之前还有过一次经历,记不太清时间了,也是换端口后如果只用pc,端口可以坚持若干天,但是后来用手机V2rayNG连接服务,到第二天端口就不行了。

其实我也有些怀疑, 墙已经拥有根据客户端特征封服务端, 或者只封客户端IP或端口. 也许有些人报的服务端被封, 可能是客户端这边被封了, 但大家很难想到封的是客户端而不是服务端

@ttc0419
Copy link

ttc0419 commented Dec 15, 2022

@p4gefau1t @Loyalsoldier Mind to take a look at this MR? 请问这个合并请求可以合并么?

@cary-sas
Copy link

cary-sas commented Dec 28, 2022

trojan-go开发者你们好,

在这个pull request中,我们更新了trojan-go客户端的TLS指纹,使其与一些当下最流行的TLS指纹一致。我们希望这次更新可以缓解2022年10月3日以来的针对基于TLS翻墙软件的大规模封锁。具体来讲:

  • 将trojan-go客户端使用的Go标准库cryto/tls完全替换为uTLS
  • 将依赖的uTLS升级到最新版本v1.1.2

根据配置的不同,这个版本的trojan-go客户端会发送以下一种流行的Clienthello指纹。这些指纹已经不同于之前版本用Go的标准库发送的TLS指纹):

fingerprint 是否设置了sni? TLS指纹 ID
Chrome (default) 是 e47eae8f8c4887b6
Chrome (default) 否 90ac8a1dfa3b207c
iOS 是 133e933dd1dfea90
iOS 否 cff7f10f631feddd
Firefox 是 7161e10829541aab
Firefox 否 56fa08d84940a06b
我们无意另起炉灶维护一个分支版本的trojan-go。我们之所以发布这个release是为了用户能够立即下载使用编译后的客户端。一旦我们的pull request请求被采纳,我们将归档我们的仓库。

Dear trojan-go developers,

In this pull request, we upgrade trojan-go client's TLS fingerprint to some of the most popular ones. We hope such change will mitigate the large-scale blocking of TLS-based censorship circumvention protocols since October 3, 2022. In particular,

  • Replace Go's standard cryto/tls with uTLS in trojan-go client.
  • Upagrade uTLS to the latest version v1.1.2.

Depending on one's configuration, the Clienthello sent by trojan-go client will look like one of the following values now (rather than the Go's TLS fingerprint in previous versions):

fingerprint sni value specified? TLS Fingerprint ID
Chrome (default) Yes e47eae8f8c4887b6
Chrome (default) No 90ac8a1dfa3b207c
iOS Yes 133e933dd1dfea90
iOS No cff7f10f631feddd
Firefox Yes 7161e10829541aab
Firefox No 56fa08d84940a06b
We do not intend to maintain a fork of trojan-go as a separate project. We made this release so that users can have compiled binaries to use immediately. We will archive our repo as soon as our pull request is merged to upstream.

你好,我用了你改过的项目,https://github.com/gfw-report/trojan-go ,比较困惑的是为何你的版本作为客户端 不支持 WS 套CDN 了呢 ?

@CXwudi
Copy link

CXwudi commented Dec 28, 2022

trojan-go开发者你们好,
在这个pull request中,我们更新了trojan-go客户端的TLS指纹,使其与一些当下最流行的TLS指纹一致。我们希望这次更新可以缓解2022年10月3日以来的针对基于TLS翻墙软件的大规模封锁。具体来讲:

  • 将trojan-go客户端使用的Go标准库cryto/tls完全替换为uTLS
  • 将依赖的uTLS升级到最新版本v1.1.2

根据配置的不同,这个版本的trojan-go客户端会发送以下一种流行的Clienthello指纹。这些指纹已经不同于之前版本用Go的标准库发送的TLS指纹):
fingerprint 是否设置了sni? TLS指纹 ID
Chrome (default) 是 e47eae8f8c4887b6
Chrome (default) 否 90ac8a1dfa3b207c
iOS 是 133e933dd1dfea90
iOS 否 cff7f10f631feddd
Firefox 是 7161e10829541aab
Firefox 否 56fa08d84940a06b
我们无意另起炉灶维护一个分支版本的trojan-go。我们之所以发布这个release是为了用户能够立即下载使用编译后的客户端。一旦我们的pull request请求被采纳,我们将归档我们的仓库。
Dear trojan-go developers,
In this pull request, we upgrade trojan-go client's TLS fingerprint to some of the most popular ones. We hope such change will mitigate the large-scale blocking of TLS-based censorship circumvention protocols since October 3, 2022. In particular,

  • Replace Go's standard cryto/tls with uTLS in trojan-go client.
  • Upagrade uTLS to the latest version v1.1.2.

Depending on one's configuration, the Clienthello sent by trojan-go client will look like one of the following values now (rather than the Go's TLS fingerprint in previous versions):
fingerprint sni value specified? TLS Fingerprint ID
Chrome (default) Yes e47eae8f8c4887b6
Chrome (default) No 90ac8a1dfa3b207c
iOS Yes 133e933dd1dfea90
iOS No cff7f10f631feddd
Firefox Yes 7161e10829541aab
Firefox No 56fa08d84940a06b
We do not intend to maintain a fork of trojan-go as a separate project. We made this release so that users can have compiled binaries to use immediately. We will archive our repo as soon as our pull request is merged to upstream.

你好,我用了你改过的项目,gfw-report/trojan-go ,比较困惑的是为何你的版本作为客户端 不支持 WS 套CDN 了呢 ?

#464 (comment)

@gaukas
Copy link

gaukas commented Dec 28, 2022

不支持 WS 套CDN 了呢 ?

Simply put:

Go websocket library doesn't work with HTTP2 transport. While all modern web browsers would advertise HTTP2 transport in their ClientHello and most of CDN providers would therefore select HTTP2 transport in ServerHello, effectively causing websocket library to fail.

@Potterli20
Copy link

Potterli20 commented Dec 29, 2022

不支持 WS 套CDN 了呢 ?

Simply put:

Go websocket library doesn't work with HTTP2 transport. While all modern web browsers would advertise HTTP2 transport in their ClientHello and most of CDN providers would therefore select HTTP2 transport in ServerHello, effectively causing websocket library to fail.

At present, you still have problems with tls, which cannot be used without ws, even with 1.1
refraction-networking/utls#150

@Potterli20
Copy link

Potterli20 commented Dec 29, 2022

不支持 WS 套CDN 了呢 ?

Simply put:

Go websocket library doesn't work with HTTP2 transport. While all modern web browsers would advertise HTTP2 transport in their ClientHello and most of CDN providers would therefore select HTTP2 transport in ServerHello, effectively causing websocket library to fail.

This is an old problem, which had a lot of problems with h2, and also with h2 in cf cdn.
gorilla/websocket#417

Inside, I found the third party utls for h2 in cf cdn, but none of them have updated

@gaukas
Copy link

gaukas commented Dec 29, 2022

you still have problems with tls, which cannot be used without ws, even with 1.1

Care to elaborate on it? I had an impression that at some point websocket had worked with ALPN set to HTTP/1.1 in uTLS's client hello. I could be wrong though.

third party utls for h2 in cf cdn

Any PR back into utls would be welcome. I can get them prioritized since H2 with WebSocket seems to be playing a huge role in circumvention.

@Potterli20
Copy link

you still have problems with tls, which cannot be used without ws, even with 1.1

Care to elaborate on it? I had an impression that at some point websocket had worked with ALPN set to HTTP/1.1 in uTLS's client hello. I could be wrong though.

third party utls for h2 in cf cdn

Any PR back into utls would be welcome. I can get them prioritized since H2 with WebSocket seems to be playing a huge role in circumvention.

I'll leave it to you to fix, and also suggest writing a yml update gomod and cyrpol/tls😂

@Potterli20
Copy link

Potterli20 commented Dec 29, 2022

Care to elaborate on it? I had an impression that at some point websocket had worked with ALPN set to HTTP/1.1 in uTLS's client hello. I could be wrong though.

This is really no, I changed a lot of clients with utls is the problem

@gaukas
Copy link

gaukas commented Dec 29, 2022

yml update gomod and cyrpol/tls

A workflow updating transient dependencies sounds legit, and I would thoroughly look into your PR.

Perhaps not for crypto/tls. It is almost guaranteed for us to have merge conflicts every time syncing with the upstream(crypto/tls) due to how utls was developed in its early stages.

@gaukas
Copy link

gaukas commented Dec 29, 2022

This is really no, I changed a lot of clients with utls is the problem

Interesting. Is it failing only when using utls, or also when using crypto/tls? If you recall?

@Potterli20
Copy link

This is really no, I changed a lot of clients with utls is the problem

Interesting. Is it failing only when using utls, or also when using crypto/tls? If you recall?

Sorry, I've fixed it on trojan-go-fork, but please update go.mod regularly, I've turned off pr and issues

@cary-sas
Copy link

cary-sas commented Jan 12, 2023

This is really no, I changed a lot of clients with utls is the problem

Interesting. Is it failing only when using utls, or also when using crypto/tls? If you recall?

Sorry, I've fixed it on trojan-go-fork, but please update go.mod regularly, I've turned off pr and issues

Unfortunately, I tried your latest update, the issue still exist, in addition, it even doesn't work without CDN. My binary version is arm v5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants