Crawl sitemaps and filter the url-list based on query selectors or search terms
To install sitecrawlr from npm, run:
$ npm install -g sitecrawlr
node ./bin/sitecrawlr --help
mocha test
- Move the list-function to a separate command
- Make the search command accept input from std-in
- Make the search engine accept regexes
- Add ignore-case possibillity to search command
- Find out a more serious approach to test the stream filters
- Add some examples to the README
- Make the url-search run in paralell to speed things up
Copyright (c) 2015 Mikael Engström