Skip to content

drzax/simple-scrape-template

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 

Repository files navigation

Simple GitHub scraper

This is a template repository for a simple GitHub scraper pioneered by Simon Wilison.

The 'simple' part

At this point the template only supports very simple fetching and committing of a JSON data file from somewhere on the internet.

Replace https://www.example.com/data.json in the fetch.yaml file with the URL of the data you want to scrape.

Commit and push the repo to GitHub and you're ready to go.

By default the scraper will run once per week, but you can change the cron schedule in the fetch.yaml file.

Data is stored in data.json.

You may need to update the permissions on the new repository to allow workflows to make commits to the repository.

In settings -> actions -> general look for the setting that says 'Read and write permissions' under the heading 'Workflow permissions'

About

No description or website provided.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published