New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

优化反爬 #376

Open

b0123456789 opened this issue Oct 23, 2023 · 2 comments

b0123456789 commented Oct 23, 2023

用gout 爬取百度的数据不行
下面这个库就行

https://github.com/chzyer/go-fetcher

Author

b0123456789 commented Oct 23, 2023

test.zip 测试代码

Owner

guonaihong commented Oct 23, 2023 •

edited

我知道了，和User-Agent http header有关系。
临时做法，可以通过SetHeader修改这个字段的默认值。

SetHeader(gout.H{"User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) "})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment