New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Jukebox model (replaces #16875) #17826
Conversation
Replaces (#16875) |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
As it was previously requested, you can now instantiate |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of last comments but we are almost there!
This model was contributed by [Arthur Zucker](https://huggingface.co/ArthurZ). | ||
The original code can be found [here](https://github.com/openai/jukebox). | ||
|
||
## JukeboxConfig |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would regroup all configs, then the tokenier, then the models (like for [CLIP]https://huggingface.co/docs/transformers/model_doc/clip)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay!
if isinstance(module, JukeboxPrior) or isinstance(module, JukeboxVQVAE): | ||
module.apply(module._init_weights) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clean!
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
This is a draft pull request.
What does this PR do?
This PR will progressively add the Jukebox model to the hub.
It is linked to #16870.
Currently planned steps (WIP)
transformeres-cli add-new-model-like
src/transformers/tokenization_jukebox.py
src/transformers/test_tokenization_jukebox.py
src/transformers/configuration_jukebox.py
src/transformers/modeling_jukebox.py
src/transformers/configuration_jukebox.py
docs/source/model_doc/jukebox.rst
src/transformers/tokenization_jukebox_fast.py
(will most probably use WordLevel tokenizer). Also requires to implement a converter functionclass JukeboxConverter(Converter):