Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Library not working due to poor request #159

Open
JengRung opened this issue Jul 22, 2022 · 3 comments
Open

Library not working due to poor request #159

JengRung opened this issue Jul 22, 2022 · 3 comments
Assignees
Labels
Server Issues caused by the backend or server-related downtime.

Comments

@JengRung
Copy link

Describe the Bug

Getting the 'too many 503 error responses' error when calling the package

Steps To Reproduce

from hentai import Hentai

doujin = Hentai(177013)
print(Hentai.exists(doujin.id))

System Information

| OS Name | Python Version | App Version |
|:---------- --:|:-----------------:|:-----------:|
| Windows | 3.10.2 | 3.2.10 |

Error message & warnings

Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\requests\adapters.py", line 440, in send
    resp = conn.urlopen(
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 876, in urlopen
    return self.urlopen(
  [Previous line repeated 2 more times]
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 866, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "C:\Python310\lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\xxx\Desktop\hentai scrping\hentai\test.py", line 3, in <module>
    doujin = Hentai(177013)
  File "C:\Python310\lib\site-packages\hentai\hentai.py", line 531, in __init__
    self.__response = self.handler.get(self.api)
  File "C:\Python310\lib\site-packages\hentai\hentai.py", line 479, in get
    response = self.session.get(url, timeout=self.timeout, proxies=self.proxies or getproxies(), **kwargs)
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python310\lib\site-packages\requests\adapters.py", line 510, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='nhentai.net', port=443): Max retries exceeded with url: /api/gallery/177013 (Caused by ResponseError('too many 503 error responses'))

Additional context

I know that we all have a hard time scraping nhentai due to Cloudflare. Just wondering if there is a good solution now? Also is there any instruction on how to set up OAuth2 with this package?

@JengRung JengRung added the bug Something isn't working label Jul 22, 2022
@hentai-chan
Copy link
Owner

This is a duplicate of issue #155. A developer at nhentai.net needs to implement OAuth2 first (which is something I have no control over of) so that clients (such as this hentai library) can pick up on that change and follow suit. I haven't been able to contact the site admin myself. I think they added a rule which automatically enables cloudflare protection as soon as a certain threshold is crossed which is why sometimes the API works again. Maybe if more developers try to contact them via twitter (https://twitter.com/nhentaiOfficial) a solution can be found that works for everyone.

@hentai-chan hentai-chan added Server Issues caused by the backend or server-related downtime. and removed bug Something isn't working labels Jul 26, 2022
@hentai-chan
Copy link
Owner

Update: There's a potential fix in #163 on its way, let's see how this plays out.

@hentai-chan hentai-chan added this to Backlog in Global Task Tracker via automation Sep 14, 2022
@hentai-chan hentai-chan moved this from Backlog to In Progress in Global Task Tracker Sep 14, 2022
@hentai-chan hentai-chan linked a pull request Sep 17, 2022 that will close this issue
@hentai-chan
Copy link
Owner

Relevant comment about this issue:

#163 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Server Issues caused by the backend or server-related downtime.
Projects
Development

Successfully merging a pull request may close this issue.

2 participants