Update Gousto.co.uk scraper. Closes #376 #511
Conversation
| class GoustoJson(AbstractScraper): | ||
| """ | ||
| Ad-hoc solution to https://github.com/hhursev/recipe-scrapers/issues/376 | ||
| Let's see if it stands the test of time and reevaluate. |
There was a problem hiding this comment.
Possibly time to discuss some of that re-evaluation?
It'd be nice to imagine a future where recipe webpages include relevant metadata on-page for user-agents to collect easily.
What's less clear to me is whether we can assume that that's happening already as a trend (allowing us to de-prioritize multi-request scrapers) or whether sites might become more evasive over time (meaning that we could help navigate towards the goal by permitting multi-request scrapers).
...with developer, dependency and ecosystem implications in each case.
There was a problem hiding this comment.
At this point in time I'm leaning towards Offline Use Only note in README.md and on scrape_me invocation.
Also, updating scrape_html to auto-pick the scraper based on the <link rel="canonical" .. > in the html.
In short, de-prioritizing multi-request scrapers.
Our package would aim to be good once it gets it's hands on a valid html from the sites listed. Not playing it clever bypassing browser/javascript checks or requirements.
There was a problem hiding this comment.
Ok, yep - reducing the interface to scrape_html only would seem to have a bunch of benefits.
It means we could require/enforce decoded str input, and we'd have fewer dependencies and modules related to networking.
The main drawback I have in mind is that rel="canonical" isn't found on all pages - so perhaps an optional (warn-if-different-to-canonical-URL-in-HTML) parameter to provide that when it isn't available?
No description provided.