Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fastai upstream and downstream capacities for fastai>=2.4 and fastcore>=1.3.27 versions #678

Merged
merged 87 commits into from Apr 26, 2022
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
f237986
added a fastai mixin for upstream and downstream tasks with fastai l…
omarespejel Oct 17, 2021
16a10a2
Added check for right fastai version
omarespejel Oct 17, 2021
a0398d9
Added aditional documentation
omarespejel Oct 17, 2021
0e65896
Add support for fastai>=2.4 versions and fastcore>=1.3.27
omarespejel Feb 10, 2022
aea8b8e
Merge branch 'main' into add-fastai-mixin
omarespejel Feb 10, 2022
940c68d
Update docstrings, save_fastai_learner, push_to_hub_fastai, and chang…
omarespejel Feb 11, 2022
4b246f4
Merge branch 'add-fastai-mixin' of https://github.com/omarespejel/hug…
omarespejel Feb 11, 2022
807e7bb
trim triling whitespaces
omarespejel Feb 11, 2022
44493cf
Import fastai_utils.py in huggingface_hub/__init__py
omarespejel Feb 13, 2022
a7253ed
Add check_fastai_fastcore_versions function in fastai_utils.py
omarespejel Feb 13, 2022
5904986
Eliminate imports of libraries not used in fastai_utils.py
omarespejel Feb 13, 2022
18b4498
Add isort and black format to fastai_utils.py
omarespejel Feb 13, 2022
37b6b51
Change pickle_protocol argument from kwargs to explicit
omarespejel Feb 15, 2022
1c9cff5
Change kwargs arguments in the function from_pretrained_fastai to exp…
omarespejel Feb 15, 2022
2a40a8d
Simplify push_to_hub_fastai function, particularly the repo_id argument
omarespejel Feb 15, 2022
1a4012c
Eliminate search for a pickle document in from_pretrained_fastai func…
omarespejel Feb 16, 2022
a9ba5d2
Simplify push_to_hub_fastai and correct bug in from_pretrained_fastai
omarespejel Feb 16, 2022
68f2006
Add pickle.DEFAULT_PROTOCOL for get adequate protocol when exporting …
omarespejel Feb 17, 2022
97742cb
Allow to load only models in the Hub when using from_pretrained_fastai
omarespejel Feb 17, 2022
7ba1fdd
Eliminate cache_dir from from_pretrained_fastai for simplification
omarespejel Feb 17, 2022
bf7a033
Correct nit picks in push_to_hub_fastai
omarespejel Feb 17, 2022
d3da12a
Update with nits
omarespejel Feb 17, 2022
3a66991
Apply isort
omarespejel Feb 17, 2022
679187c
Replace config.json for pyproject.toml to check for fastai and fastco…
omarespejel Feb 21, 2022
24d3ddb
Isort imports
omarespejel Feb 21, 2022
b0eeb35
Make pyproject.toml automatically filled with fastai, fastcore, and p…
omarespejel Feb 22, 2022
2fb9400
add check_fastai_fastcore_pyproject_versions function to know the fas…
omarespejel Feb 24, 2022
c443bb8
Change library tomlkit for toml
omarespejel Feb 24, 2022
42b01c3
Add extras[fastai] with the toml library
omarespejel Feb 25, 2022
fb886e9
Change the way the token is asked in def push_to_hub_fastai(
omarespejel Feb 25, 2022
effac9f
Eliminate logger from imports
omarespejel Feb 25, 2022
db1cbe9
Fix nits
omarespejel Feb 25, 2022
c9ccff0
Remove typing.Union from imports
omarespejel Feb 25, 2022
19fad26
add fastai integration tests
omarespejel Feb 28, 2022
f72ac27
Import fastai in setup.py
omarespejel Mar 1, 2022
fd63e5a
Import toml inside check_fastai_fastcore_pyproject_versions function
omarespejel Mar 1, 2022
1435d2c
Merge branch 'main' of https://github.com/huggingface/huggingface_hub…
omarespejel Mar 1, 2022
2ee4c56
Merge branch 'huggingface:main' into add-fastai-mixin
omarespejel Mar 1, 2022
bd1c675
Merge branch 'add-fastai-mixin' of https://github.com/omarespejel/hug…
omarespejel Mar 1, 2022
2c38e7b
Nits in fastai_utils.py
omarespejel Mar 1, 2022
4013c79
Add build_fastai to python-tests.yml
omarespejel Mar 1, 2022
5e840b9
Add fastcore import to setup.py
omarespejel Mar 1, 2022
33061cb
Add require_fastai_fastcore() to skip tests
omarespejel Mar 2, 2022
6680065
Nits and documentation of raised errors improved
omarespejel Mar 4, 2022
8ce5894
Merge main branch changes
Mar 22, 2022
988d6cd
add strategy for fastai in python-tests.yml
Mar 22, 2022
d3585a3
Eliminate organization from push_to_hub_fastai
Mar 23, 2022
5699b43
Add python 3.7-3.10 to python-tests.yml
omarespejel Mar 31, 2022
8041b60
Resolve conflicts
omarespejel Mar 31, 2022
8587464
Change python version in tests to 3.9
omarespejel Mar 31, 2022
421aab3
Fix tests
omarespejel Apr 7, 2022
b2e7fa7
Fix conflict in config.py due to order of tf packages
omarespejel Apr 7, 2022
264537b
Merge branch 'main' into add-fastai-mixin
omarespejel Apr 7, 2022
a8e32a3
Fix delete_repo function in test_fastai_integration
omarespejel Apr 7, 2022
b061df8
Merge branch 'add-fastai-mixin' of https://github.com/omarespejel/hug…
omarespejel Apr 7, 2022
f297f7c
Isort test_fastai_integration
omarespejel Apr 7, 2022
de7368b
Replace the repo_id name for model_id
omarespejel Apr 11, 2022
3cfe4d5
Make fastai and fastcore versions flexible in setup.py
omarespejel Apr 11, 2022
050904a
Confirm fastai supports python 3.10 in python-tests-yml
omarespejel Apr 12, 2022
f3b2000
Fix docs in fastai_utils.py
omarespejel Apr 12, 2022
2d12300
Change the name of DummyModel for dummy_model
omarespejel Apr 12, 2022
edeee68
Handle pickling errors when exporting a fastai.Learner
omarespejel Apr 12, 2022
028c80a
Change name of internal functions in fastai_utils.py
omarespejel Apr 12, 2022
1bc2c41
black style to fastai_utils.py
omarespejel Apr 12, 2022
26c8d15
Eliminate unnecessary comments from fastai_utils.py
omarespejel Apr 12, 2022
a7de8c4
Add capacity to load a local fastai.Learner to from_pretrained_keras
omarespejel Apr 12, 2022
022c571
black format fastai_utils.py
omarespejel Apr 12, 2022
5dafb25
Come back to Python 3.9 instead of 3.10
omarespejel Apr 13, 2022
9a06a67
Change name name of save_fastai_learner to _save_pretrained_fastai in…
omarespejel Apr 13, 2022
004d318
Change the name to _save_pretrained_fastai in __init__.py
omarespejel Apr 13, 2022
947d58b
Fix nits in test_fastai_integration.py
omarespejel Apr 13, 2022
d6ee8bf
Add fastai integration to docs
omarespejel Apr 13, 2022
18d0d27
Fix nits
omarespejel Apr 21, 2022
7e63f46
Fix wording
omarespejel Apr 21, 2022
2512ead
Allow _save_pretrained_fastai to directly export the model in save_di…
omarespejel Apr 21, 2022
8633c0e
black fastai_utils.py
omarespejel Apr 21, 2022
363cb81
Make the requirement of having a pyproject.toml optional
omarespejel Apr 21, 2022
ac0ef8a
Add warnings if the pyproject.toml does not contain a "build-system" …
omarespejel Apr 25, 2022
2b28f4b
Change try-excepts for ifs in the warnings checking the fastai and fa…
omarespejel Apr 25, 2022
88f4cd0
Fix nits in documentation
omarespejel Apr 25, 2022
d0268a9
Move errors in _check_fastai_fastcore_pyproject_versions to condition…
omarespejel Apr 25, 2022
5ffd543
Misc improvements
osanseviero Apr 26, 2022
f36093e
Move the versions checks for fastai and fastcore to "else"'s
omarespejel Apr 26, 2022
cd4648a
Move the versions checks for fastai and fastcore to "else"'s
omarespejel Apr 26, 2022
1795ce8
Merge branch 'add-fastai-mixin' of https://github.com/omarespejel/hug…
omarespejel Apr 26, 2022
f9dd4b0
Merge branch 'main' of https://github.com/huggingface/huggingface_hub…
omarespejel Apr 26, 2022
465e0fa
Black reformat
omarespejel Apr 26, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/python-tests.yml
Expand Up @@ -90,6 +90,27 @@ jobs:
pip install .[testing,tensorflow]

- run: pytest -Werror::FutureWarning -sv ./tests/test_keras*

build_fastai:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7", "3.9"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does fastai not support 3.10?

Suggested change
python-version: ["3.7", "3.9"]
python-version: ["3.7", "3.10"]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we run this on a matrix of versions, that means 3 runners will run, eh? Is this even all that necessary? I see most the other ones seem to run just on 3.8.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think running in different py versions is important here, specially as the pickling is different. Torch and TF integrations test 3.7 and 3.10, so I would do same here
https://github.com/huggingface/huggingface_hub/blob/main/.github/workflows/python-tests.yml#L55


omarespejel marked this conversation as resolved.
Show resolved Hide resolved
steps:
- uses: actions/checkout@v2

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install --upgrade pip
pip install .[testing,fastai]

- run: pytest -Werror::FutureWarning -sv ./tests/test_fastai*

tests_lfs:
runs-on: ubuntu-latest
Expand Down
6 changes: 6 additions & 0 deletions setup.py
Expand Up @@ -27,6 +27,12 @@ def get_version() -> str:
"torch",
]

extras["fastai"] = [
"toml",
omarespejel marked this conversation as resolved.
Show resolved Hide resolved
"fastai==2.5.3",
"fastcore==1.3.29",
omarespejel marked this conversation as resolved.
Show resolved Hide resolved
]

extras["tensorflow"] = ["tensorflow", "pydot", "graphviz"]

extras["testing"] = [
Expand Down
5 changes: 5 additions & 0 deletions src/huggingface_hub/__init__.py
Expand Up @@ -30,6 +30,11 @@
TF2_WEIGHTS_NAME,
TF_WEIGHTS_NAME,
)
from .fastai_utils import (
from_pretrained_fastai,
push_to_hub_fastai,
save_fastai_learner,
)
omarespejel marked this conversation as resolved.
Show resolved Hide resolved
from .file_download import cached_download, hf_hub_download, hf_hub_url
from .hf_api import (
DatasetSearchArguments,
Expand Down