Load GitHub datasets from Hub

[albertvillanova]

We have recurrently had connection errors when requesting GitHub because sometimes the site is not available.

This PR requests the Hub instead, once all GitHub datasets are mirrored on the Hub.

Fix #2048

Related to:

• ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/2.0.0/datasets/glue/glue.py #4051
• ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.15.1/datasets/wmt16/wmt16.py #3210
• ConnectionError: Couldn't reach https://raw.githubusercontent.com #2787
• ConnectionError: Couldn't reach common_voice.py #2075
• Cannot load wikitext #2036

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[lhoestq]

Currently the github datasets versioning is synced with the datasets lib versioning: when you load a github dataset using datasets==x.y.z, then the version of the dataset will be the one at the git tag x.y.z. This is for reproducibility reasons.

We could stop having this behavior and always use the latest version of the dataset, but when we do a breaking change it will break github datasets for previous versions of the library. It could be nice to think about tools that will allow backward compatibility if we ever need to to a breaking change in some datasets. Maybe a way to specify which revision of the dataset to use based on the datasets major version.

If we keep this behavior, then maybe add a note in setup.py to push to PyPI only after the Update Hub repositories CI job is done. It can take a few minutes to add the version tag to all the dataset repositories on the Hub. If we push to PyPI before the tags are pushed, then some users might get some 404 if at the same time they installed datasets and run load_dataset.

[albertvillanova]

@lhoestq I was going to increase the max_retries as done for metrics:

• Increase max retries for GitHub metrics #4063

But then I realized that loading from the Hub would work as well. That is why I opened this PR.

Definitely, we should decide which behavior we want:

• We have been working in the direction of eliminating the distinctions between canonical/community datasets
• If we continue to go in that direction, then passing (or not passing) revision should have the same behavior for canonical/community
• If we want to continue to tight the library version with the canonical datasets version, that is definitely a difference between canonical and community datasets

Not sure what could be better in the long term...

[albertvillanova]

We could stop having this behavior and always use the latest version of the dataset, but when we do a breaking change it will break github datasets for previous versions of the library.

Not sure of understanding this. Previous versions of the datasets library will continue to download GitHub datasets from GitHub, syncing library/dataset versions... Where is the problem?

[lhoestq]

Yes you're right, previous versions of datasets will still continue to download from github, but not future versions.
If we release datasets 2.1 by removing this behavior and if one day we release datasets 3.0 with a breaking change in the dataset scripts, then all version >=2.1 will break.

[lhoestq]

Ideally we should drop the differences between github datasets and community datasets, and maybe provide a way to fallback on an older version of a dataset repository if the user's datasets version is too old and incompatible with it.

[lhoestq]

I just noticed I literally opened the same PR lol

I'm still convinced that we should do a better version compatibility check but we can see that later IMO

[albertvillanova]

Normally in open source projects, when there is a duplicate PR, the latter is tagged as "duplicate" and closed. 😜

Let me make things clear in my mind: so you say that the blocking point that was preventing this PR from merging, now is no longer a blocking point and could be addresses in a subsequent PR?

[lhoestq]

Let me close the duplicate one, sorry

Let me make things clear my mind: so you say that the blocking point that was preventing this PR from merging now is no longer a blocking point and could be addresses in a subsequent PR?

Yes 🙈

[lhoestq]

Cool ! LGTM :)

Finally we'll remove the differences between Hub datasets and GitHub datasets ^^

(Note that after this PR, all the changes made to a dataset will affect all the datasets version from now on)

[albertvillanova]

Note that after this PR, all the changes made to a dataset will affect all the datasets version from now on

Yes, we have aligned this behavior with Hub datasets, as this is already the case for Hub datasets.