Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add async support #86

Closed
wants to merge 34 commits into from
Closed

Add async support #86

wants to merge 34 commits into from

Conversation

Mikubill
Copy link
Contributor

@Mikubill Mikubill commented Sep 10, 2019

Made the following changes:

  1. Added asynchronous function and examples.
    (Most functions have the same functionality as the original function.)
  2. Modified .tarvis.yml to support asynchronous function testing.
  3. Modified the spaces and parentheses of some files.

The current test on TravisCI shows no serious problems but sometimes throws a timeout exception.

超时大概是由于请求太密集造成的?
刚准备pull的时候发现Proxy好像忘了弄QwQ
有一些修正是PyCharm自动完成的,如果出现问题可以revert掉

TravisCI: https://travis-ci.org/Mikubill/pixivpy/builds/583175481

@upbit
Copy link
Owner

upbit commented Sep 11, 2019

Thanks for the contribution!

关于async请求,有以下几点建议:

  1. 是否可以不拆分api,而是通过patch底层requests库实现异步?比如 import requests_async as requests(这样api修改可以统一,也可以选择是否使用async io)
  2. 如果1这里实现不了,可以考虑fork成新的项目(比如pixivpy-aysnc),传到pypi作为需要async的选择,我帮你外链进来;

Mikubill and others added 18 commits September 11, 2019 14:05
* Test Asyncio by orig

* Test Asyncio by orig

* fix travis.xml error

* fix async auth error

* fix async parse error

* fix async parse error

* fix async parse error

* dianose async parse error

* dianose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error

* diagnose async parse error
@Mikubill
Copy link
Contributor Author

Mikubill commented Sep 11, 2019

  1. aio.py 重写了 BaseAPI 中关于网络请求的方法(request, auth, download),以提供async支持 。
  2. 将 BaseAPI 中Auth方法的网络请求分离。
  3. illust_related 有时会引发pixiv返回错误,修正 seed_illust_ids 的 param。
  4. 将 self.parse_result(r) 移动到 requests_call 内部进行,使得async时函数能返回couroutine
  • 不拆分api
  • demo测试
  • Download测试
  • Proxy测试

@Mikubill
Copy link
Contributor Author

httpx暂不支持proxy.
ref: encode/httpx#36

@Mikubill
Copy link
Contributor Author

demo测试看起来没有太大的问题。
https://travis-ci.org/Mikubill/pixivpy/builds/583592877

不过有个疑问:pypi支持不同版本依赖控制吗?httpx貌似只能在python 3.6以上使用(pypi苦手

@upbit
Copy link
Owner

upbit commented Sep 11, 2019

不支持。如果有py3的限制,那可能还是以独立库的方式来维护比较好。
几个建议可以参考下:

  1. https://github.com/Mikubill/pixivpy改名成https://github.com/Mikubill/pixivpy_async之类的,注册pypi账号并上传py3编译的文件;
  2. 给我重新提个Merge Request,只包含README.md的修改(在Projects base on pixivpy附近加上你的项目链接和介绍)

另外,建议rebase你的34次修改 git rebase -i HEAD~34,整理成比较合理的commit后提交,这能使git主干更干净(需要force push,建议先开分支再合入你的master);

@Mikubill
Copy link
Contributor Author

了解~

@Mikubill
Copy link
Contributor Author

Closed

@Mikubill Mikubill closed this Sep 11, 2019
@Mikubill Mikubill mentioned this pull request Sep 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants