Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a scraper status page #3950

Open
grossir opened this issue Apr 5, 2024 · 0 comments
Open

Create a scraper status page #3950

grossir opened this issue Apr 5, 2024 · 0 comments

Comments

@grossir
Copy link
Contributor

grossir commented Apr 5, 2024

Let's track here our ideas and requirements for a "scraper status page". Sometimes scrapers fail (for many reasons) and such a page would be useful to notice more errors that won't surface on Sentry

Background

In #3949 we deleted cl/scrapers/management/commands/cl_report_scrape_status.py which depended on the ErrorLog model and created a useful HTML report, that was sent via email and also to a slack channel

However, it seems nobody was really monitoring that, and both the content and the access to the report could be improved

Ideas

...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant