Skip to content
This repository has been archived by the owner on Mar 7, 2021. It is now read-only.

Issues: simplecrawler/simplecrawler

55 Open 254 Closed
55 Open 254 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Adding to the queue on "complete" callback doesn't work
#496 opened Nov 24, 2020 by Ivanca
crawler.supportedMimeTypes not moving after first page
#495 opened Nov 18, 2020 by ankurarora
Proxy for each request in the queue?
#493 opened Aug 24, 2020 by alex-w0
2
Generic error missing
#492 opened Aug 18, 2020 by Ivanca
1
Fail to decode application/x-gzip
#488 opened May 2, 2020 by FelixRe0
An in-range update of mocha is breaking the build 🚨 greenkeeper
#487 opened Apr 26, 2020 by greenkeeper bot
2
docs: does this spider support "waiting for all the JS to finish" because actually crawling the resulting page?
#481 opened Dec 29, 2019 by Pomax
2
How to await "fetchcomplete"?
#480 opened Dec 27, 2019 by Pradyumna-medarametla
1
SQLite FetchQueue Implementation for Simplecrawler
#478 opened Nov 8, 2019 by LeMoussel
Crawler stuck on url 'Exceeded maximum number of redirects'
#476 opened Sep 4, 2019 by stijn-lcp
1
Request path contains unescaped characters
#474 opened Jul 24, 2019 by guidodizi
feature: validate outgoing request headers before sending them
#467 opened Apr 8, 2019 by upugo-dev
Can't compile app because of simplecrawler
#452 opened Feb 27, 2019 by bruceandroid
1
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
#449 opened Jan 16, 2019 by motiejuss
2
Crawler.stop(true) not working as expected
#446 opened Dec 19, 2018 by braj1999
2
Crawling aint starting.
#438 opened Sep 21, 2018 by misoe92
1
Provision to Validate against Self signed certificate
#436 opened Aug 30, 2018 by braj1999
1
Feature request to support NTLM and Kerberos authentication
#434 opened Jul 3, 2018 by braj1999
1
Using Simplecrwaler for Tor pages
#431 opened May 29, 2018 by zabihimayvan
1
fetchStart is unable to modify request
#429 opened May 14, 2018 by gombosg
1
3
HSTS/307 detected as 301
#428 opened May 10, 2018 by rvizcaino80
6
Checking cookie domain can be problematic
#425 opened Apr 30, 2018 by ollieh-m
Using crawler.addDownloadCondition causes the scraper to complete without downloading anything.
#418 opened Feb 14, 2018 by g-a-v-i-n
4
I am using your socksv5 example I see the below issue
#417 opened Feb 7, 2018 by skhanam
Crawler hangs up on empty response body of robtos.txt
#416 opened Jan 7, 2018 by abbasharoon
3
ProTip! What’s not been updated in a month: updated:<2024-04-22.